Skip to contentSkip to conversation

mmapstress01: refactor to ltp framework

PATCH Superseded Edward Liaw via ltp 2 participants 9 messages Oct 6, 2022 – Oct 13, 2022

v4 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v4 cover mmapstress01: refactor to ltp framework

v3->v4:
* squash previous changes
* set defaults of nprocs=20 and max_runtime=12
* clean up comments
* return TBROK instead of TFAIL for unexpected failures

v2->v3:
* apply make check lint suggestions
* refactor cleanup
* use SAFE_FORK
* update license

v1->v2:
* clean up formatting
* remove accidental change to header comment
* use SAFE_MMAP

v0->v1:
* use tst_test framework
* use FILE_OFFSET_BITS=64 instead of LARGE_FILE
* use safe macros

Edward Liaw (4):
mmapstress01: refactor to tst_test framework
mmapstress01: default nprocs 20 and cleanup
mmapstress01: return TBROK for unexpected failures and TERRNO if
syscall fails
mmapstress01: use max_runtime

 testcases/kernel/mem/mmapstress/Makefile      |   2 +
 .../kernel/mem/mmapstress/mmapstress01.c      | 864 ++++++------------
 2 files changed, 276 insertions(+), 590 deletions(-)
Signature · 5 lines
2.38.0.rc1.362.ged0d419d3c-goog


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

PATCH v4 1/4 mmapstress01: refactor to tst_test framework

Use ltp framework and apply make check corrections. Reorder functions and variables. Use safe macros.
Drop leavefile option.
Build tests with FILE_OFFSET_BITS=64 instead of doing LARGE_FILE checks to switch between 32 and 64 bit types and syscalls.

---
 testcases/kernel/mem/mmapstress/Makefile      |   2 +
 .../kernel/mem/mmapstress/mmapstress01.c      | 845 ++++++------------
 2 files changed, 289 insertions(+), 558 deletions(-)
testcases/kernel/mem/mmapstress/Makefile +2 -0
index 744f099d8..b30bd34b8 100644
@@ -5,3 +5,5 @@ top_srcdir              ?= ../../../..
 
 include $(top_srcdir)/include/mk/testcases.mk
 include $(top_srcdir)/include/mk/generic_leaf_target.mk
+
+mmapstress01: CFLAGS += -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
testcases/kernel/mem/mmapstress/mmapstress01.c +287 -558

967 lines not on this page · raw message

Signature · 5 lines
2.38.0.rc1.362.ged0d419d3c-goog


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw

PATCH v4 2/4 mmapstress01: default nprocs 20 and cleanup

In reply to Edward Liaw via ltp

Drop usage information from top level comment and remove unnecessary function definitions.

---
 .../kernel/mem/mmapstress/mmapstress01.c      | 46 +++----------------
 1 file changed, 6 insertions(+), 40 deletions(-)
testcases/kernel/mem/mmapstress/mmapstress01.c +6 -40
index f68193706..67a2b747d 100644
@@ -18,33 +18,6 @@
  *  or times out (if a timeout value is specified).  When either of
  *  these things happens, it cleans up its kids, then checks the
  *  file to make sure it has the correct data.
- *
- *  usage:
- *	tmnoextend -p nprocs [-t minutes -f filesize -S sparseoffset
- *			      -r -o -m -l -d]
- *  where:
- *	-p nprocs	- specifies the number of mapping children
- *			  to create.  (nprocs + 1 children actually
- *			  get created, since one is the writer child)
- *	-t minutes	- specifies minutes to run.  If not specified,
- *			  default is to run forever until a SIGINT
- *			  is received.
- *	-f filesize	- initial filesize (defaults to FILESIZE)
- *	-S sparseoffset - when non-zero, causes a sparse area to
- *			  be left before the data, meaning that the
- *			  actual initial file size is sparseoffset +
- *			  filesize.  Useful for testing large files.
- *			  (default is 0).
- *	-r		- randomize number of pages map children check.
- *			  (random % MAXLOOPS).  If not specified, each
- *			  child checks MAXLOOPS pages.
- *	-o		- randomize offset of file to map. (default is 0)
- *	-m		- do random msync/fsyncs as well
- *	-l		- if set, the output file is not removed on
- *			  program exit.
- *	-d		- enable debug output
- *
- *  Compile with -DLARGE_FILE to enable file sizes > 2 GB.
  */
 
 #define _GNU_SOURCE 1
@@ -63,7 +36,6 @@
 #include "tst_test.h"
 
 #define MAXLOOPS	500	/* max pages for map children to write */
-#define	FILESIZE	4096	/* initial filesize set up by parent */
 #define TEST_FILE	"mmapstress01.out"
 
 #ifdef roundup
@@ -72,9 +44,6 @@
 #define roundup(x, y)	((((x)+((y)-1))/(y))*(y))
 
 static unsigned int initrand(void);
-static void child_mapper(char *file, unsigned int procno, unsigned int nprocs);
-static void fileokay(char *file, unsigned char *expbuf);
-static void sighandler(int sig);
 
 static char *debug;
 static char *do_sync;
@@ -87,9 +56,9 @@ static char *randloops;
 
 static int fd;
 static int finished;
-static int nprocs;
+static int nprocs = 20;
 static float alarmtime;
-static long long filesize = FILESIZE;
+static long long filesize = 4096;
 static long long sparseoffset;
 static size_t pagesize;
 static unsigned int pattern;
@@ -99,10 +68,11 @@ static struct tst_option options[] = {
 	{"f:", &opt_filesize, "Initial filesize (default 4096)"},
 	{"m", &do_sync, "Do random msync/fsyncs as well"},
 	{"o", &do_offset, "Randomize the offset of file to map"},
-	{"p:", &opt_nprocs, "Number of mapping children to create (required)"},
+	{"p:", &opt_nprocs,
+	 "Number of mapping children to create (default 20)"},
 	{"r", &randloops,
-	 "Randomize number of pages map children check (random % MAXLOOPS), "
-	 "otherwise each child checks MAXLOOPS pages"},
+	 "Randomize number of pages map children check (random % 500), "
+	 "otherwise each child checks 500 pages"},
 	{"S:", &opt_sparseoffset,
 	 "When non-zero, causes the sparse area to be left before the data, "
 	 "so that the actual initial filesize is sparseoffset + filesize "
@@ -115,10 +85,6 @@ static void setup(void)
 {
 	int pagesize = sysconf(_SC_PAGE_SIZE);
 
-	if (!opt_nprocs)
-		tst_brk(TBROK,
-			"missing number of mapping children, specify with -p nprocs");
-
 #if _FILE_OFFSET_BITS == 64
 	if (tst_parse_filesize(opt_filesize, &filesize, 0, LONG_MAX))
 #else
Signature · 5 lines
2.38.0.rc1.362.ged0d419d3c-goog


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#3 lore raw

PATCH v4 3/4 mmapstress01: return TBROK for unexpected failures and TERRNO if syscall fails

In reply to Edward Liaw via ltp
---
 testcases/kernel/mem/mmapstress/mmapstress01.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
testcases/kernel/mem/mmapstress/mmapstress01.c +7 -6
index 67a2b747d..9a18587cf 100644
@@ -155,7 +155,7 @@ static void child_mapper(char *file, unsigned int procno, unsigned int nprocs)
 	fd = SAFE_OPEN(file, O_RDWR);
 
 	if (statbuf.st_size - sparseoffset > UINT_MAX)
-		tst_brk(TFAIL, "size_t overflow when setting up map");
+		tst_brk(TBROK, "size_t overflow when setting up map");
 	mapsize = (size_t) (statbuf.st_size - sparseoffset);
 	mappages = roundup(mapsize, pagesize) / pagesize;
 	offset = sparseoffset;
@@ -212,7 +212,7 @@ static void child_mapper(char *file, unsigned int procno, unsigned int nprocs)
 		paddr = maddr + (randpage * pagesize);	/* page address */
 		if (msync(paddr, (mappages - randpage) * pagesize,
 			  MS_SYNC) == -1)
-			tst_brk(TFAIL, "msync failed");
+			tst_brk(TBROK | TERRNO, "msync failed");
 	}
 	SAFE_MUNMAP(maddr, mapsize);
 	exit(0);
@@ -237,7 +237,7 @@ static void fileokay(char *file, unsigned char *expbuf)
 	SAFE_LSEEK(fd, sparseoffset, SEEK_SET);
 
 	if (statbuf.st_size - sparseoffset > UINT_MAX)
-		tst_brk(TFAIL, "size_t overflow when setting up map");
+		tst_brk(TBROK, "size_t overflow when setting up map");
 	mapsize = (size_t) (statbuf.st_size - sparseoffset);
 
 	mappages = roundup(mapsize, pagesize) / pagesize;
@@ -383,13 +383,13 @@ static void run(void)
 			 */
 			if (!WIFEXITED(wait_stat)
 			    || WEXITSTATUS(wait_stat) != 0)
-				tst_brk(TFAIL, "child exit with err <x%x>",
+				tst_brk(TBROK, "child exit with err <x%x>",
 					wait_stat);
 			for (i = 0; i < nprocs; i++)
 				if (pid == pidarray[i])
 					break;
 			if (i == nprocs)
-				tst_brk(TFAIL, "unknown child pid %d, <x%x>",
+				tst_brk(TBROK, "unknown child pid %d, <x%x>",
 					pid, wait_stat);
 
 			pid = SAFE_FORK();
@@ -406,7 +406,8 @@ static void run(void)
 			 *  error...
 			 */
 			if (errno != EINTR || !finished)
-				tst_brk(TFAIL, "unexpected wait error");
+				tst_brk(TBROK | TERRNO,
+					"unexpected wait error");
 		}
 		SAFE_SIGPROCMASK(SIG_UNBLOCK, &set_mask, NULL);
 	}
Signature · 5 lines
2.38.0.rc1.362.ged0d419d3c-goog


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#4 lore raw

PATCH v4 4/4 mmapstress01: use max_runtime

In reply to Edward Liaw via ltp
---
 .../kernel/mem/mmapstress/mmapstress01.c      | 24 +++++--------------
 1 file changed, 6 insertions(+), 18 deletions(-)
testcases/kernel/mem/mmapstress/mmapstress01.c +6 -18
index 9a18587cf..83d3f387d 100644
@@ -48,7 +48,6 @@ static unsigned int initrand(void);
 static char *debug;
 static char *do_sync;
 static char *do_offset;
-static char *opt_alarmtime;
 static char *opt_filesize;
 static char *opt_nprocs;
 static char *opt_sparseoffset;
@@ -57,7 +56,6 @@ static char *randloops;
 static int fd;
 static int finished;
 static int nprocs = 20;
-static float alarmtime;
 static long long filesize = 4096;
 static long long sparseoffset;
 static size_t pagesize;
@@ -77,7 +75,6 @@ static struct tst_option options[] = {
 	 "When non-zero, causes the sparse area to be left before the data, "
 	 "so that the actual initial filesize is sparseoffset + filesize "
 	 "(default 0)"},
-	{"t:", &opt_alarmtime, "Number of minutes to run (default 0)"},
 	{},
 };
 
@@ -105,18 +102,10 @@ static void setup(void)
 		tst_brk(TBROK, "invalid number of mapping children '%s'",
 			opt_nprocs);
 
-	if (tst_parse_float(opt_alarmtime, &alarmtime, 0, FLT_MAX / 60))
-		tst_brk(TBROK, "invalid minutes to run '%s'", opt_alarmtime);
-
 	if (debug) {
 		tst_res(TINFO, "creating file <%s> with %lld bytes, pattern %d",
 			TEST_FILE, filesize, pattern);
-		if (alarmtime)
-			tst_res(TINFO, "running for %f minutes", alarmtime);
-		else
-			tst_res(TINFO, "running with no time limit");
 	}
-	alarmtime *= 60;
 }
 
 static void cleanup(void)
@@ -312,8 +301,7 @@ static void run(void)
 	pattern = seed & 0xff;
 
 	/*
-	 *  Plan for death by signal.  User may have specified
-	 *  a time limit, in which set an alarm and catch SIGALRM.
+	 *  Plan for death by signal or alarm.
 	 *  Also catch and cleanup with SIGINT.
 	 */
 	sa.sa_handler = sighandler;
@@ -323,10 +311,9 @@ static void run(void)
 	SAFE_SIGACTION(SIGQUIT, &sa, 0);
 	SAFE_SIGACTION(SIGTERM, &sa, 0);
 
-	if (alarmtime) {
-		SAFE_SIGACTION(SIGALRM, &sa, 0);
-		(void)alarm(alarmtime);
-	}
+	SAFE_SIGACTION(SIGALRM, &sa, 0);
+	alarm(tst_remaining_runtime());
+
 	fd = SAFE_OPEN(TEST_FILE, O_CREAT | O_TRUNC | O_RDWR, 0664);
 
 	buf = SAFE_MALLOC(pagesize);
@@ -415,7 +402,7 @@ static void run(void)
 	SAFE_SIGEMPTYSET(&set_mask);
 	SAFE_SIGADDSET(&set_mask, SIGALRM);
 	SAFE_SIGPROCMASK(SIG_BLOCK, &set_mask, NULL);
-	(void)alarm(0);
+	alarm(0);
 
 	/*
 	 *  Finished!  Check the file for sanity.
@@ -429,6 +416,7 @@ static struct tst_test test = {
 	.setup = setup,
 	.options = options,
 	.cleanup = cleanup,
+	.max_runtime = 12,
 	.needs_tmpdir = 1,
 	.forks_child = 1,
 };
Signature · 5 lines
2.38.0.rc1.362.ged0d419d3c-goog


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#5 lore raw
In reply to Edward Liaw via ltp

Hi!
Looks mostly done now, but there are still a few things to fix, some of them purely cosmetic:

* The finished global has to be volatile otherwise it may get miscompiled and it has to be reset at the start for the run() otherwise the -i option is broken

* The sighandler setup can be moved to the test setup()

* Now we do not use the pidarray to kill the child processes so it became mostly useless, I would consider getting rid of that too

* I'm failing to see why we need to block SIGALRM just before alarm(0) as far as I can tell it's completely useless. Also blocking signals while we refork looks questionable.

* I would consider getting rid of obvious comments, quite a few of them are commenting the obvious

The ifdefs in setup() does not look nice, maybe we should do something as:

#if _FILE_OFFSET_BITS == 64

# define FSIZE_MIN LONG_MIN
# define FSIZE_MAX LONG_MAX
#else
# define FSIZE_MIN INT_MIN
# define FSIZE_MAX INT_MAX
#endif

And use the FSIZE_MIN and FSIZE_MAX to avoid ifdefs in the test setup()

* The top level comment should be converted to docparse comment so that it's picked up by the documentation parser, that means that the comment should start with:

/*\
* [Description]
*

And should be formatted in asciidoc

Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#6 lore raw

Hi Cyril,
I sent a v5 with changes based on your comments.

Quoted 4 lines

* I'm failing to see why we need to block SIGALRM just before alarm(0) as far as I can tell it's completely useless. Also blocking signals while we refork looks questionable.

Is the reason it's blocking signals while reforking is due to the second choice in https://man7.org/linux/man-pages/man7/signal-safety.7.html? I'm not familiar with signal handling.

```
Block signal delivery in the main program when calling
functions that are unsafe or operating on global data that is
also accessed by the signal handler.
```

Thanks,
Edward

Signature · 2 lines
-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#7 lore raw

Hi!

Quoted 5 lines
Quoted 4 lines

* I'm failing to see why we need to block SIGALRM just before alarm(0) as far as I can tell it's completely useless. Also blocking signals while we refork looks questionable.

Quoted 8 lines

Is the reason it's blocking signals while reforking is due to the second choice in https://man7.org/linux/man-pages/man7/signal-safety.7.html? I'm not familiar with signal handling.

```
Block signal delivery in the main program when calling
functions that are unsafe or operating on global data that is
also accessed by the signal handler.
```

I do not think that this is a valid reason. What that paragraph is trying to tell is that you either have to avoid calling anything that modifies global state and may cause trouble or block signals in sections that do the same but in the normal program context. Either way that effectively makes sure that such code is never run concurently.

For example calling malloc()/free() modifies locks and data structures in libc, it's not safe to be called from signal handler and can easily cause deadlocks and corruptions. Technically you can avoid that either by not calling malloc()/free() from a signal handler, or by disabling signal handlers before you call malloc()/free() or anything that may possibly call malloc()/free() in the rest of the program.

In this case the signal handler is as simple as it gets, it only sets a global flag that is checked in the while () loop that reforks. And the only important thing we do there is the fork(). However fork() cannot be interrupted by a signal and return EINTR so it cannot even fail. It would probably made some sense if we wanted to setup different signal handlers in the child and avoid getting signals before we do so, but that's not the case here either.

So either this is a leftover code that was there to protect something that has been removed long in the past, or copy&pasted code from a different test that acutally made use of that. At least I do not see any other reason to have it there.

Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#8 lore raw
Quoted 22 lines

I do not think that this is a valid reason. What that paragraph is trying to tell is that you either have to avoid calling anything that modifies global state and may cause trouble or block signals in sections that do the same but in the normal program context. Either way that effectively makes sure that such code is never run concurently.

For example calling malloc()/free() modifies locks and data structures in libc, it's not safe to be called from signal handler and can easily cause deadlocks and corruptions. Technically you can avoid that either by not calling malloc()/free() from a signal handler, or by disabling signal handlers before you call malloc()/free() or anything that may possibly call malloc()/free() in the rest of the program.

In this case the signal handler is as simple as it gets, it only sets a global flag that is checked in the while () loop that reforks. And the only important thing we do there is the fork(). However fork() cannot be interrupted by a signal and return EINTR so it cannot even fail. It would probably made some sense if we wanted to setup different signal handlers in the child and avoid getting signals before we do so, but that's not the case here either.

Quoted 8 lines

So either this is a leftover code that was there to protect something that has been removed long in the past, or copy&pasted code from a different test that acutally made use of that. At least I do not see any other reason to have it there.

--
Cyril Hrubis
[email protected]

Thanks for the explanation! That makes sense.

Thanks,
Edward

Signature · 2 lines
-- 
Mailing list info: https://lists.linux.it/listinfo/ltp