mtest06/mmap1: rewrite to newlib

PATCH Superseded Jan Stancek 4 participants 8 messages Nov 26, 2018 – Feb 5, 2019

lore

Share & tools

lore

Jan Stancek

author

2018-11-26 08:21 UTC #0 lore raw

PATCH v2 1/1 mtest06/mmap1: rewrite to newlib

Fixes #165

This patch is aiming to rewrite mtest06/mmap1 from scratch, to address sporadic failures in #165.

Main difference is that it uses only one signal handler and no synchronization between threads, no sched_yield calls.

Instead each mmap/munmap increases a map/unmap counter. Upon hitting SIGSEGV or when comparing read value, these counter values are used to determine state of mapped area as observed by first thread. This isn't 100% accurrate as first thread might be faster than the check, but it allows second thread to race against map/unmap for its entire duration.

Signed-off-by Jan Stancek <[email protected]>

---
Changes in v2:
- add suggestion by Li to write/check for 'b'
- drop parameter from runtest file

 runtest/mm                           |   2 +-
 testcases/kernel/mem/mtest06/mmap1.c | 676 +++++++++++++----------------------
 2 files changed, 250 insertions(+), 428 deletions(-)

rewrite testcases/kernel/mem/mtest06/mmap1.c (93%)

runtest/mm +1 -1

index 76d2335622df..a09f39c1e02c 100644
@@ -14,7 +14,7 @@ mtest01w mtest01 -p80 -w
 
 #test for race conditions
 mtest05   mmstress
-mtest06   mmap1 -x 0.05
+mtest06   mmap1
 mtest06_2 mmap2 -x 0.002 -a -p
 mtest06_3 mmap3 -x 0.002 -p
 # Remains diabled till the infinite loop problem is solved

testcases/kernel/mem/mtest06/mmap1.c +249 ~~-427~~

676 lines not on this page · raw message

Signature · 1 line

1.8.3.1

Li Wang

2018-11-27 06:37 UTC #1 lore raw

PATCH v2 1/1 mtest06/mmap1: rewrite to newlib

testcases/kernel/mem/mtest06/mmap1.c line 77

Quoted 13 lines · Jan Stancek

testcases/kernel/mem/mtest06/mmap1.c +13 -0

+static void sig_handler(int signal, siginfo_t *info,
+       LTP_ATTRIBUTE_UNUSED void *ut)
+{
+       int ar_m, ar_u;
+
+       switch (signal) {
+       case SIGSEGV:
+               /* if we hit SIGSEGV between map/unmap, something is wrong */
+               ar_u = tst_atomic_load(&unmapcnt);
+               ar_m = tst_atomic_load(&mapcnt);
+               if (was_area_mapped(br_map, br_unmap, ar_m, ar_u)) {
+                       tst_res(TFAIL, "got sigsegv while mapped");
+                       _exit(TFAIL);

why not use tst_brk() here?

testcases/kernel/mem/mtest06/mmap1.c line 86

Quoted 9 lines

testcases/kernel/mem/mtest06/mmap1.c +9 -0

+               }
+
+               mapped_sigsegv_count++;
+               longjmp(jmpbuf, 1);
+               break;
+       default:
+               tst_res(TFAIL, "Unexpected signal - %d, addr: %p, exiting\n",
+                       signal, info->si_addr);
+               _exit(TBROK);

tst_brk() ?

testcases/kernel/mem/mtest06/mmap1.c line 93

Quoted 7 lines

testcases/kernel/mem/mtest06/mmap1.c +7 -0

+       }
+}
+
+void *map_write_unmap(void *ptr)
+{
+       long *args = ptr;
+       void *tmp;

remove tmp pointer?

Beside these tiny issues, patch v2 looks good to me.

Reviewed-by Li Wang <[email protected]>

--
Regards,
Li Wang

Jan Stancek

author

2018-11-27 08:54 UTC #2 lore raw

PATCH v2 1/1 mtest06/mmap1: rewrite to newlib

----- Original Message -----

Quoted 18 lines · Jan Stancek

Quoted 10 lines

testcases/kernel/mem/mtest06/mmap1.c +8 -0

+static void sig_handler(int signal, siginfo_t *info,
+       LTP_ATTRIBUTE_UNUSED void *ut)
+{
+       int ar_m, ar_u;
+
+       switch (signal) {
+       case SIGSEGV:
+               /* if we hit SIGSEGV between map/unmap, something is wrong

testcases/kernel/mem/mtest06/mmap1.c line 77

Quoted 5 lines

testcases/kernel/mem/mtest06/mmap1.c +5 -0

+               ar_u = tst_atomic_load(&unmapcnt);
+               ar_m = tst_atomic_load(&mapcnt);
+               if (was_area_mapped(br_map, br_unmap, ar_m, ar_u)) {
+                       tst_res(TFAIL, "got sigsegv while mapped");
+                       _exit(TFAIL);

why not use tst_brk() here?

Because it calls exit(), which is not advised to do in signal handler. (doc/test-writing-guidelines.txt, line 841)

Quoted 14 lines

testcases/kernel/mem/mtest06/mmap1.c line 86

Quoted 11 lines

testcases/kernel/mem/mtest06/mmap1.c +7 -0

+               }
+
+               mapped_sigsegv_count++;
+               longjmp(jmpbuf, 1);
+               break;
+       default:
+               tst_res(TFAIL, "Unexpected signal - %d, addr: %p,

exiting\n",

testcases/kernel/mem/mtest06/mmap1.c +2 -0

+                       signal, info->si_addr);
+               _exit(TBROK);

tst_brk() ?

Quoted 9 lines

testcases/kernel/mem/mtest06/mmap1.c line 93
Quoted 7 lines
testcases/kernel/mem/mtest06/mmap1.c +7 -0
+       }
+}
+
+void *map_write_unmap(void *ptr)
+{
+       long *args = ptr;
+       void *tmp;
remove tmp pointer?

It introduces warning in safe_munmap(), because map_address is volatile. That is why I kept 'tmp'.

Quoted 4 lines

Beside these tiny issues, patch v2 looks good to me.
Reviewed-by Li Wang <[email protected]>

Thanks,
Jan

Quoted 5 lines

--
Regards,
Li Wang

Cyril Hrubis

2018-11-29 13:44 UTC #3 lore raw

PATCH v2 1/1 mtest06/mmap1: rewrite to newlib

In reply to Jan Stancek

Hi!

Quoted 6 lines

Instead each mmap/munmap increases a map/unmap counter. Upon hitting SIGSEGV or when comparing read value, these counter values are used to determine state of mapped area as observed by first thread. This isn't 100% accurrate as first thread might be faster than the check, but it allows second thread to race against map/unmap for its entire duration.

Looks good to me, using atomic counters and comparing values before and after we access the memory is very clever as well. You can add my Reviewed-by.

Very minor comments below.

Quoted 53 lines

testcases/kernel/mem/mtest06/mmap1.c +53 -0

+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2018 Jan Stancek. All rights reserved.
+ */
+/*
+ * Test: Spawn 2 threads. First thread maps, writes and unmaps
+ * an area. Second thread tries to read from it. Second thread
+ * races against first thread. There is no synchronization
+ * between threads, but each mmap/munmap increases a counter
+ * that is checked to determine when has read occurred. If a read
+ * hit SIGSEGV in between mmap/munmap it is a failure. If a read
+ * between mmap/munmap worked, then its value must match expected
+ * value.
+ */
+#include <errno.h>
+#include <float.h>
+#include <pthread.h>
+#include <sched.h>
+#include <setjmp.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include "tst_test.h"
+#include "tst_safe_pthread.h"
+
+#define DISTANT_MMAP_SIZE (64*1024*1024)
+#define TEST_FILENAME "ashfile"
+
+/* seconds remaining before reaching timeout */
+#define STOP_THRESHOLD 10
+
+#define PROGRESS_SEC 3
+
+static int file_size = 1024;
+static int num_iter = 5000;
+static float exec_time = 0.05; /* default is 3 min */
+
+static void *distant_area;
+static char *str_exec_time;
+static jmp_buf jmpbuf;
+static volatile unsigned char *map_address;
+static unsigned long page_sz;
+
+static unsigned long mapped_sigsegv_count;
+static unsigned long map_count;
+static unsigned long threads_spawned;
+static unsigned long data_matched;
+static unsigned long repeated_reads;
+
+/* sequence id for each map/unmap performed */
+static int mapcnt, unmapcnt;
+/* stored sequence id before making read attempt */
+static int br_map, br_unmap;
+

testcases/kernel/mem/mtest06/mmap1.c line 63

Quoted 10 lines

testcases/kernel/mem/mtest06/mmap1.c +10 -0

+static struct tst_option options[] = {
+	{"x:", &str_exec_time, "Exec time (hours)"},
+	{NULL, NULL, NULL}
+};
+
+/* compare "before read" counters  with "after read" counters */
+static inline int was_area_mapped(int br_m, int br_u, int ar_m, int ar_u)
+{
+	return (br_m == ar_m && br_u == ar_u && br_m > br_u);
+}

Since the br_map and br_unmap are global I would consider passing only the values after to this function.

Quoted 146 lines

testcases/kernel/mem/mtest06/mmap1.c +146 -0

+static void sig_handler(int signal, siginfo_t *info,
+	LTP_ATTRIBUTE_UNUSED void *ut)
+{
+	int ar_m, ar_u;
+
+	switch (signal) {
+	case SIGSEGV:
+		/* if we hit SIGSEGV between map/unmap, something is wrong */
+		ar_u = tst_atomic_load(&unmapcnt);
+		ar_m = tst_atomic_load(&mapcnt);
+		if (was_area_mapped(br_map, br_unmap, ar_m, ar_u)) {
+			tst_res(TFAIL, "got sigsegv while mapped");
+			_exit(TFAIL);
+		}
+
+		mapped_sigsegv_count++;
+		longjmp(jmpbuf, 1);
+		break;
+	default:
+		tst_res(TFAIL, "Unexpected signal - %d, addr: %p, exiting\n",
+			signal, info->si_addr);
+		_exit(TBROK);
+	}
+}
+
+void *map_write_unmap(void *ptr)
+{
+	long *args = ptr;
+	void *tmp;
+	int i, j;
+
+	for (i = 0; i < num_iter; i++) {
+		map_address = SAFE_MMAP(distant_area,
+			(size_t) file_size, PROT_WRITE | PROT_READ,
+			MAP_SHARED, (int)args[0], 0);
+		tst_atomic_inc(&mapcnt);
+
+		for (j = 0; j < file_size; j++)
+			map_address[j] = 'b';
+
+		tmp = (void *)map_address;
+		tst_atomic_inc(&unmapcnt);
+		SAFE_MUNMAP(tmp, file_size);
+
+		map_count++;
+	}
+
+	return NULL;
+}
+
+void *read_mem(LTP_ATTRIBUTE_UNUSED void *ptr)
+{
+	int i, j, ar_map, ar_unmap;
+	unsigned char c;
+
+	for (i = 0; i < num_iter; i++) {
+		if (setjmp(jmpbuf) == 1)
+			continue;
+
+		for (j = 0; j < file_size; j++) {
+read_again:
+			br_map = tst_atomic_load(&mapcnt);
+			br_unmap = tst_atomic_load(&unmapcnt);
+
+			c = map_address[j];
+
+			ar_unmap = tst_atomic_load(&unmapcnt);
+			ar_map = tst_atomic_load(&mapcnt);
+
+			/*
+			 * Read above is racing against munmap and mmap
+			 * in other thread. While the address might be valid
+			 * the mapping could be in various stages of being
+			 * 'ready'. We only check the value, if we can be sure
+			 * read hapenned in between single mmap and munmap as
+			 * observed by first thread.
+			 */
+			if (was_area_mapped(br_map, br_unmap, ar_map,
+				ar_unmap)) {
+				switch (c) {
+				case 'a':
+					repeated_reads++;
+					goto read_again;
+				case 'b':
+					data_matched++;
+					break;
+				default:
+					tst_res(TFAIL, "value[%d] is %c", j, c);
+					break;
+				}
+			}
+		}
+	}
+
+	return NULL;
+}
+
+int mkfile(int size)
+{
+	int fd, i;
+
+	fd = SAFE_OPEN(TEST_FILENAME, O_RDWR | O_CREAT, 0600);
+	SAFE_UNLINK(TEST_FILENAME);
+
+	for (i = 0; i < size; i++)
+		SAFE_WRITE(1, fd, "a", 1);
+	SAFE_WRITE(1, fd, "\0", 1);
+
+	if (fsync(fd) == -1)
+		tst_brk(TBROK | TERRNO, "fsync()");
+
+	return fd;
+}
+
+static void setup(void)
+{
+	struct sigaction sigptr;
+
+	page_sz = getpagesize();
+
+	/*
+	 * Used as hint for mmap thread, so it doesn't interfere
+	 * with other potential (temporary) mappings from libc
+	 */
+	distant_area = SAFE_MMAP(0, DISTANT_MMAP_SIZE, PROT_WRITE | PROT_READ,
+			MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+	SAFE_MUNMAP(distant_area, (size_t)DISTANT_MMAP_SIZE);
+	distant_area += DISTANT_MMAP_SIZE / 2;
+
+	if (tst_parse_float(str_exec_time, &exec_time, 0, FLT_MAX)) {
+		tst_brk(TBROK, "Invalid number for exec_time '%s'",
+			str_exec_time);
+	}
+
+	sigptr.sa_sigaction = sig_handler;
+	sigemptyset(&sigptr.sa_mask);
+	sigptr.sa_flags = SA_SIGINFO | SA_NODEFER;
+	SAFE_SIGACTION(SIGSEGV, &sigptr, NULL);
+
+	tst_set_timeout((int)(exec_time * 3600));
+}
+
+static void run(void)
+{
+	pthread_t thid[2];
+	long chld_args[1];

testcases/kernel/mem/mtest06/mmap1.c line 220

Quoted 10 lines

testcases/kernel/mem/mtest06/mmap1.c +10 -0

+	int remaining = tst_timeout_remaining();
+	int elapsed = 0;
+
+	while (tst_timeout_remaining() > STOP_THRESHOLD) {
+		int fd = mkfile(file_size);
+
+		tst_atomic_store(0, &mapcnt);
+		tst_atomic_store(0, &unmapcnt);
+
+		chld_args[0] = fd;

I would have just casted the fd here to long or intptr_t (to make sure compiler padds it with zeroes) then to (void*) and passed the value directly, i.e. (void*)(inptr_t)fd and then back in the thread we do fd = (intptr_t)ptr.

Quoted 35 lines

testcases/kernel/mem/mtest06/mmap1.c +29 -0

+		SAFE_PTHREAD_CREATE(&thid[0], NULL, map_write_unmap, chld_args);
+		SAFE_PTHREAD_CREATE(&thid[1], NULL, read_mem, chld_args);
+		threads_spawned += 2;
+
+		SAFE_PTHREAD_JOIN(thid[0], NULL);
+		SAFE_PTHREAD_JOIN(thid[1], NULL);
+
+		close(fd);
+
+		if (remaining - tst_timeout_remaining() > PROGRESS_SEC) {
+			remaining = tst_timeout_remaining();
+			elapsed += PROGRESS_SEC;
+			tst_res(TINFO, "[%d] mapped: %lu, sigsegv hit: %lu, "
+				"threads spawned: %lu",	elapsed, map_count,
+				mapped_sigsegv_count, threads_spawned);
+			tst_res(TINFO, "[%d] repeated_reads: %ld, "
+				"data_matched: %lu", elapsed, repeated_reads,
+				data_matched);
+		}
+	}
+	tst_res(TPASS, "System survived.");
+}
+
+static struct tst_test test = {
+	.test_all = run,
+	.setup = setup,
+	.options = options,
+	.needs_tmpdir = 1,
+};

Signature · 5 lines

1.8.3.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Signature · 2 lines

Cyril Hrubis
[email protected]

Jan Stancek

author

2018-11-30 09:46 UTC #4 lore raw

PATCH v2 1/1 mtest06/mmap1: rewrite to newlib

----- Original Message -----

Quoted 13 lines

Hi!
Quoted 6 lines
Instead each mmap/munmap increases a map/unmap counter. Upon hitting SIGSEGV or when comparing read value, these counter values are used to determine state of mapped area as observed by first thread. This isn't 100% accurrate as first thread might be faster than the check, but it allows second thread to race against map/unmap for its entire duration.
Looks good to me, using atomic counters and comparing values before and after we access the memory is very clever as well. You can add my Reviewed-by.
Very minor comments below.

Quoted 8 lines

testcases/kernel/mem/mtest06/mmap1.c line 63
Quoted 5 lines
testcases/kernel/mem/mtest06/mmap1.c +5 -0
+/* compare "before read" counters  with "after read" counters */
+static inline int was_area_mapped(int br_m, int br_u, int ar_m, int ar_u)
+{
+	return (br_m == ar_m && br_u == ar_u && br_m > br_u);
+}
Since the br_map and br_unmap are global I would consider passing only the values after to this function.

Hi,

I find it more clear if it doesn't depend on globals. Since it's inlined there should be no penalty, so I kept it as is.

Quoted 9 lines

testcases/kernel/mem/mtest06/mmap1.c line 122

Quoted 9 lines

testcases/kernel/mem/mtest06/mmap1.c +9 -0

+
+void *read_mem(LTP_ATTRIBUTE_UNUSED void *ptr)
+{
+	int i, j, ar_map, ar_unmap;
+	unsigned char c;
+
+	for (i = 0; i < num_iter; i++) {
+		if (setjmp(jmpbuf) == 1)
+			continue;

I made 'i' volatile, since longjmp could clobber it.

Quoted 11 lines

testcases/kernel/mem/mtest06/mmap1.c +11 -0

+static void run(void)
+{
+	pthread_t thid[2];
+	long chld_args[1];
+	int remaining = tst_timeout_remaining();
+	int elapsed = 0;
+
+	while (tst_timeout_remaining() > STOP_THRESHOLD) {
+		int fd = mkfile(file_size);
+
+		tst_atomic_store(0, &mapcnt);

Quoted 8 lines

testcases/kernel/mem/mtest06/mmap1.c line 220
Quoted 3 lines
testcases/kernel/mem/mtest06/mmap1.c +3 -0
+		tst_atomic_store(0, &unmapcnt);
+
+		chld_args[0] = fd;
I would have just casted the fd here to long or intptr_t (to make sure compiler padds it with zeroes) then to (void*) and passed the value directly, i.e. (void*)(inptr_t)fd and then back in the thread we do fd = (intptr_t)ptr.

I changed this to pass int ptr directly, which requires no casts AFAICT.

... and pushed.

Thanks,
Jan

Naresh Kamboju

2019-02-04 08:34 UTC #5 lore raw

PATCH v2 1/1 mtest06/mmap1: rewrite to newlib

FYI,

mtest06/mmap1.c fails intermittently on arm64 devices due to timeout reason. The current timeout set as 15 minutes
export LTP_TIMEOUT_MUL=3

Test case output:
-----------------------
tst_test.c:1085: INFO: Timeout per run is 0h 15m 00s tst_test.c:1085: INFO: Timeout per run is 0h 09m 00s
mmap1.c:234: INFO: [3] mapped: 15000, sigsegv hit: 11375, threads spawned: 6 mmap1.c:237: INFO: [3] repeated_reads: 67, data_matched: 3440667 mmap1.c:234: INFO: [6] mapped: 30000, sigsegv hit: 21256, threads spawned: 12 mmap1.c:237: INFO: [6] repeated_reads: 2530, data_matched: 8455962 mmap1.c:234: INFO: [9] mapped: 40000, sigsegv hit: 28215, threads spawned: 16 mmap1.c:237: INFO: [9] repeated_reads: 4413, data_matched: 11314629 mmap1.c:234: INFO: [12] mapped: 55000, sigsegv hit: 38454, threads spawned: 22 mmap1.c:237: INFO: [12] repeated_reads: 9285, data_matched: 15892633 mmap1.c:234: INFO: [15] mapped: 60000, sigsegv hit: 42462, threads spawned: 24 mmap1.c:237: INFO: [15] repeated_reads: 11993, data_matched: 16723333 mmap1.c:234: INFO: [18] mapped: 65000, sigsegv hit: 46272, threads spawned: 26 mmap1.c:237: INFO: [18] repeated_reads: 11993, data_matched: 18043857 mmap1.c:234: INFO: [21] mapped: 80000, sigsegv hit: 58736, threads spawned: 32 mmap1.c:237: INFO: [21] repeated_reads: 14467, data_matched: 20310080 mmap1.c:234: INFO: [24] mapped: 85000, sigsegv hit: 62747, threads spawned: 34 mmap1.c:237: INFO: [24] repeated_reads: 14467, data_matched: 21065412 mmap1.c:234: INFO: [27] mapped: 95000, sigsegv hit: 67177, threads spawned: 38 mmap1.c:237: INFO: [27] repeated_reads: 14467, data_matched: 26278412 mmap1.c:234: INFO: [30] mapped: 110000, sigsegv hit: 75318, threads spawned: 44 mmap1.c:237: INFO: [30] repeated_reads: 18749, data_matched: 32880471 mmap1.c:234: INFO: [33] mapped: 130000, sigsegv hit: 86435, threads spawned: 52 mmap1.c:237: INFO: [33] repeated_reads: 20987, data_matched: 41628336 mmap1.c:234: INFO: [36] mapped: 150000, sigsegv hit: 95359, threads spawned: 60 mmap1.c:237: INFO: [36] repeated_reads: 20987, data_matched: 52762236 mmap1.c:234: INFO: [39] mapped: 175000, sigsegv hit: 107473, threads spawned: 70 mmap1.c:237: INFO: [39] repeated_reads: 20987, data_matched: 65170250 mmap1.c:234: INFO: [42] mapped: 195000, sigsegv hit: 115860, threads spawned: 78 mmap1.c:237: INFO: [42] repeated_reads: 24446, data_matched: 76553410 mmap1.c:234: INFO: [45] mapped: 220000, sigsegv hit: 130868, threads spawned: 88 mmap1.c:237: INFO: [45] repeated_reads: 28657, data_matched: 86358966 mmap1.c:234: INFO: [48] mapped: 245000, sigsegv hit: 145435, threads spawned: 98 mmap1.c:237: INFO: [48] repeated_reads: 34613, data_matched: 96820804 mmap1.c:234: INFO: [51] mapped: 260000, sigsegv hit: 154934, threads
spawned: 104
mmap1.c:237: INFO: [51] repeated_reads: 38042, data_matched: 102281865 mmap1.c:234: INFO: [54] mapped: 280000, sigsegv hit: 168403, threads
spawned: 112
mmap1.c:237: INFO: [54] repeated_reads: 44799, data_matched: 108884699 mmap1.c:234: INFO: [57] mapped: 305000, sigsegv hit: 188132, threads
spawned: 122
mmap1.c:237: INFO: [57] repeated_reads: 48995, data_matched: 113829351 mmap1.c:234: INFO: [60] mapped: 315000, sigsegv hit: 196457, threads
spawned: 126
mmap1.c:237: INFO: [60] repeated_reads: 51657, data_matched: 115444681 mmap1.c:234: INFO: [63] mapped: 330000, sigsegv hit: 208077, threads
spawned: 132
mmap1.c:237: INFO: [63] repeated_reads: 51657, data_matched: 118910807 mmap1.c:234: INFO: [66] mapped: 335000, sigsegv hit: 211847, threads
spawned: 134
mmap1.c:237: INFO: [66] repeated_reads: 51657, data_matched: 119936305 mmap1.c:234: INFO: [69] mapped: 355000, sigsegv hit: 225809, threads
spawned: 142
mmap1.c:237: INFO: [69] repeated_reads: 58357, data_matched: 125691672 mmap1.c:234: INFO: [72] mapped: 365000, sigsegv hit: 233748, threads
spawned: 146
mmap1.c:237: INFO: [72] repeated_reads: 63594, data_matched: 127682094 mmap1.c:234: INFO: [75] mapped: 375000, sigsegv hit: 240935, threads
spawned: 150
mmap1.c:237: INFO: [75] repeated_reads: 65813, data_matched: 130436913 mmap1.c:234: INFO: [78] mapped: 395000, sigsegv hit: 256994, threads
spawned: 158
mmap1.c:237: INFO: [78] repeated_reads: 68240, data_matched: 134313071 mmap1.c:234: INFO: [81] mapped: 400000, sigsegv hit: 260739, threads
spawned: 160
mmap1.c:237: INFO: [81] repeated_reads: 68240, data_matched: 135483078 mmap1.c:234: INFO: [84] mapped: 405000, sigsegv hit: 263510, threads
spawned: 162
mmap1.c:237: INFO: [84] repeated_reads: 68240, data_matched: 137744513 mmap1.c:234: INFO: [87] mapped: 415000, sigsegv hit: 271049, threads
spawned: 166
mmap1.c:237: INFO: [87] repeated_reads: 70071, data_matched: 140167936 mmap1.c:234: INFO: [90] mapped: 430000, sigsegv hit: 282504, threads
spawned: 172
mmap1.c:237: INFO: [90] repeated_reads: 74335, data_matched: 143263860 mmap1.c:234: INFO: [93] mapped: 435000, sigsegv hit: 286623, threads
spawned: 174
mmap1.c:237: INFO: [93] repeated_reads: 76581, data_matched: 144114430 mmap1.c:234: INFO: [96] mapped: 445000, sigsegv hit: 292401, threads
spawned: 178
mmap1.c:237: INFO: [96] repeated_reads: 78725, data_matched: 148160778 mmap1.c:234: INFO: [99] mapped: 450000, sigsegv hit: 296192, threads
spawned: 180
mmap1.c:237: INFO: [99] repeated_reads: 78725, data_matched: 149328825 mmap1.c:234: INFO: [102] mapped: 460000, sigsegv hit: 304280, threads
spawned: 184
mmap1.c:237: INFO: [102] repeated_reads: 78725, data_matched: 151183014 mmap1.c:234: INFO: [105] mapped: 465000, sigsegv hit: 308364, threads
spawned: 186
mmap1.c:237: INFO: [105] repeated_reads: 78725, data_matched: 152036389 Test timeouted, sending SIGKILL!
tst_test.c:1125: INFO: If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1
tst_test.c:1126: BROK: Test killed! (timeout?)

Test results comparison,
https://qa-reports.linaro.org/lkft/linux-mainline-oe/tests/ltp-mm-tests/mtest06

Any chance to minimize the test run time ?

Best regards
Naresh Kamboju

Quoted 72 lines · Jan Stancek

----- Original Message -----
Quoted 13 lines
Hi!
Quoted 6 lines
Instead each mmap/munmap increases a map/unmap counter. Upon hitting SIGSEGV or when comparing read value, these counter values are used to determine state of mapped area as observed by first thread. This isn't 100% accurrate as first thread might be faster than the check, but it allows second thread to race against map/unmap for its entire duration.
Looks good to me, using atomic counters and comparing values before and after we access the memory is very clever as well. You can add my Reviewed-by.
Very minor comments below.
Quoted 8 lines
testcases/kernel/mem/mtest06/mmap1.c line 63
Quoted 5 lines
testcases/kernel/mem/mtest06/mmap1.c +5 -0
+/* compare "before read" counters  with "after read" counters */
+static inline int was_area_mapped(int br_m, int br_u, int ar_m, int ar_u)
+{
+   return (br_m == ar_m && br_u == ar_u && br_m > br_u);
+}
Since the br_map and br_unmap are global I would consider passing only the values after to this function.
Hi,
I find it more clear if it doesn't depend on globals. Since it's inlined there should be no penalty, so I kept it as is.
Quoted 9 lines
testcases/kernel/mem/mtest06/mmap1.c line 122
Quoted 9 lines
testcases/kernel/mem/mtest06/mmap1.c +9 -0
+
+void *read_mem(LTP_ATTRIBUTE_UNUSED void *ptr)
+{
+   int i, j, ar_map, ar_unmap;
+   unsigned char c;
+
+   for (i = 0; i < num_iter; i++) {
+           if (setjmp(jmpbuf) == 1)
+                   continue;
I made 'i' volatile, since longjmp could clobber it.
Quoted 11 lines
Quoted 11 lines
testcases/kernel/mem/mtest06/mmap1.c +11 -0
+static void run(void)
+{
+   pthread_t thid[2];
+   long chld_args[1];
+   int remaining = tst_timeout_remaining();
+   int elapsed = 0;
+
+   while (tst_timeout_remaining() > STOP_THRESHOLD) {
+           int fd = mkfile(file_size);
+
+           tst_atomic_store(0, &mapcnt);
Quoted 8 lines
testcases/kernel/mem/mtest06/mmap1.c line 220
Quoted 3 lines
testcases/kernel/mem/mtest06/mmap1.c +3 -0
+           tst_atomic_store(0, &unmapcnt);
+
+           chld_args[0] = fd;
I would have just casted the fd here to long or intptr_t (to make sure compiler padds it with zeroes) then to (void*) and passed the value directly, i.e. (void*)(inptr_t)fd and then back in the thread we do fd = (intptr_t)ptr.
I changed this to pass int ptr directly, which requires no casts AFAICT.
... and pushed.
Thanks,
Jan
--
Mailing list info: https://lists.linux.it/listinfo/ltp

Jan Stancek

author

2019-02-04 09:05 UTC #6 lore raw

PATCH v2 1/1 mtest06/mmap1: rewrite to newlib

----- Original Message -----

Quoted 117 lines

FYI,
mtest06/mmap1.c fails intermittently on arm64 devices due to timeout reason. The current timeout set as 15 minutes
export LTP_TIMEOUT_MUL=3
Test case output:
-----------------------
tst_test.c:1085: INFO: Timeout per run is 0h 15m 00s tst_test.c:1085: INFO: Timeout per run is 0h 09m 00s
mmap1.c:234: INFO: [3] mapped: 15000, sigsegv hit: 11375, threads spawned: 6 mmap1.c:237: INFO: [3] repeated_reads: 67, data_matched: 3440667 mmap1.c:234: INFO: [6] mapped: 30000, sigsegv hit: 21256, threads spawned: 12 mmap1.c:237: INFO: [6] repeated_reads: 2530, data_matched: 8455962 mmap1.c:234: INFO: [9] mapped: 40000, sigsegv hit: 28215, threads spawned: 16 mmap1.c:237: INFO: [9] repeated_reads: 4413, data_matched: 11314629 mmap1.c:234: INFO: [12] mapped: 55000, sigsegv hit: 38454, threads spawned:
22
mmap1.c:237: INFO: [12] repeated_reads: 9285, data_matched: 15892633 mmap1.c:234: INFO: [15] mapped: 60000, sigsegv hit: 42462, threads spawned:
24
mmap1.c:237: INFO: [15] repeated_reads: 11993, data_matched: 16723333 mmap1.c:234: INFO: [18] mapped: 65000, sigsegv hit: 46272, threads spawned:
26
mmap1.c:237: INFO: [18] repeated_reads: 11993, data_matched: 18043857 mmap1.c:234: INFO: [21] mapped: 80000, sigsegv hit: 58736, threads spawned:
32
mmap1.c:237: INFO: [21] repeated_reads: 14467, data_matched: 20310080 mmap1.c:234: INFO: [24] mapped: 85000, sigsegv hit: 62747, threads spawned:
34
mmap1.c:237: INFO: [24] repeated_reads: 14467, data_matched: 21065412 mmap1.c:234: INFO: [27] mapped: 95000, sigsegv hit: 67177, threads spawned:
38
mmap1.c:237: INFO: [27] repeated_reads: 14467, data_matched: 26278412 mmap1.c:234: INFO: [30] mapped: 110000, sigsegv hit: 75318, threads spawned:
44
mmap1.c:237: INFO: [30] repeated_reads: 18749, data_matched: 32880471 mmap1.c:234: INFO: [33] mapped: 130000, sigsegv hit: 86435, threads spawned:
52
mmap1.c:237: INFO: [33] repeated_reads: 20987, data_matched: 41628336 mmap1.c:234: INFO: [36] mapped: 150000, sigsegv hit: 95359, threads spawned:
60
mmap1.c:237: INFO: [36] repeated_reads: 20987, data_matched: 52762236 mmap1.c:234: INFO: [39] mapped: 175000, sigsegv hit: 107473, threads spawned:
70
mmap1.c:237: INFO: [39] repeated_reads: 20987, data_matched: 65170250 mmap1.c:234: INFO: [42] mapped: 195000, sigsegv hit: 115860, threads spawned:
78
mmap1.c:237: INFO: [42] repeated_reads: 24446, data_matched: 76553410 mmap1.c:234: INFO: [45] mapped: 220000, sigsegv hit: 130868, threads spawned:
88
mmap1.c:237: INFO: [45] repeated_reads: 28657, data_matched: 86358966 mmap1.c:234: INFO: [48] mapped: 245000, sigsegv hit: 145435, threads spawned:
98
mmap1.c:237: INFO: [48] repeated_reads: 34613, data_matched: 96820804 mmap1.c:234: INFO: [51] mapped: 260000, sigsegv hit: 154934, threads
spawned: 104
mmap1.c:237: INFO: [51] repeated_reads: 38042, data_matched: 102281865 mmap1.c:234: INFO: [54] mapped: 280000, sigsegv hit: 168403, threads
spawned: 112
mmap1.c:237: INFO: [54] repeated_reads: 44799, data_matched: 108884699 mmap1.c:234: INFO: [57] mapped: 305000, sigsegv hit: 188132, threads
spawned: 122
mmap1.c:237: INFO: [57] repeated_reads: 48995, data_matched: 113829351 mmap1.c:234: INFO: [60] mapped: 315000, sigsegv hit: 196457, threads
spawned: 126
mmap1.c:237: INFO: [60] repeated_reads: 51657, data_matched: 115444681 mmap1.c:234: INFO: [63] mapped: 330000, sigsegv hit: 208077, threads
spawned: 132
mmap1.c:237: INFO: [63] repeated_reads: 51657, data_matched: 118910807 mmap1.c:234: INFO: [66] mapped: 335000, sigsegv hit: 211847, threads
spawned: 134
mmap1.c:237: INFO: [66] repeated_reads: 51657, data_matched: 119936305 mmap1.c:234: INFO: [69] mapped: 355000, sigsegv hit: 225809, threads
spawned: 142
mmap1.c:237: INFO: [69] repeated_reads: 58357, data_matched: 125691672 mmap1.c:234: INFO: [72] mapped: 365000, sigsegv hit: 233748, threads
spawned: 146
mmap1.c:237: INFO: [72] repeated_reads: 63594, data_matched: 127682094 mmap1.c:234: INFO: [75] mapped: 375000, sigsegv hit: 240935, threads
spawned: 150
mmap1.c:237: INFO: [75] repeated_reads: 65813, data_matched: 130436913 mmap1.c:234: INFO: [78] mapped: 395000, sigsegv hit: 256994, threads
spawned: 158
mmap1.c:237: INFO: [78] repeated_reads: 68240, data_matched: 134313071 mmap1.c:234: INFO: [81] mapped: 400000, sigsegv hit: 260739, threads
spawned: 160
mmap1.c:237: INFO: [81] repeated_reads: 68240, data_matched: 135483078 mmap1.c:234: INFO: [84] mapped: 405000, sigsegv hit: 263510, threads
spawned: 162
mmap1.c:237: INFO: [84] repeated_reads: 68240, data_matched: 137744513 mmap1.c:234: INFO: [87] mapped: 415000, sigsegv hit: 271049, threads
spawned: 166
mmap1.c:237: INFO: [87] repeated_reads: 70071, data_matched: 140167936 mmap1.c:234: INFO: [90] mapped: 430000, sigsegv hit: 282504, threads
spawned: 172
mmap1.c:237: INFO: [90] repeated_reads: 74335, data_matched: 143263860 mmap1.c:234: INFO: [93] mapped: 435000, sigsegv hit: 286623, threads
spawned: 174
mmap1.c:237: INFO: [93] repeated_reads: 76581, data_matched: 144114430 mmap1.c:234: INFO: [96] mapped: 445000, sigsegv hit: 292401, threads
spawned: 178
mmap1.c:237: INFO: [96] repeated_reads: 78725, data_matched: 148160778 mmap1.c:234: INFO: [99] mapped: 450000, sigsegv hit: 296192, threads
spawned: 180
mmap1.c:237: INFO: [99] repeated_reads: 78725, data_matched: 149328825 mmap1.c:234: INFO: [102] mapped: 460000, sigsegv hit: 304280, threads
spawned: 184
mmap1.c:237: INFO: [102] repeated_reads: 78725, data_matched: 151183014 mmap1.c:234: INFO: [105] mapped: 465000, sigsegv hit: 308364, threads
spawned: 186
mmap1.c:237: INFO: [105] repeated_reads: 78725, data_matched: 152036389 Test timeouted, sending SIGKILL!
tst_test.c:1125: INFO: If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1
tst_test.c:1126: BROK: Test killed! (timeout?)

Quoted 4 lines

Test results comparison,
https://qa-reports.linaro.org/lkft/linux-mainline-oe/tests/ltp-mm-tests/mtest06
Any chance to minimize the test run time ?

Default is 3 minutes, but it appears to be getting stuck before that. In [1], the output stopped after ~12-15 seconds, followed by timeout.

So either timeout arrived early (which seems unlikely) or test was stuck on something.

[1] https://qa-reports.linaro.org/lkft/linux-mainline-oe/build/v5.0-rc5/testrun/594677/log

Jan Stancek

author

2019-02-05 06:46 UTC #7 lore raw

PATCH v2 1/1 mtest06/mmap1: rewrite to newlib

----- Original Message -----

Quoted 135 lines

----- Original Message -----
Quoted 131 lines
FYI,
mtest06/mmap1.c fails intermittently on arm64 devices due to timeout reason.
The current timeout set as 15 minutes
export LTP_TIMEOUT_MUL=3
Test case output:
-----------------------
tst_test.c:1085: INFO: Timeout per run is 0h 15m 00s tst_test.c:1085: INFO: Timeout per run is 0h 09m 00s mmap1.c:234: INFO: [3] mapped: 15000, sigsegv hit: 11375, threads spawned:
6
mmap1.c:237: INFO: [3] repeated_reads: 67, data_matched: 3440667 mmap1.c:234: INFO: [6] mapped: 30000, sigsegv hit: 21256, threads spawned:
12
mmap1.c:237: INFO: [6] repeated_reads: 2530, data_matched: 8455962 mmap1.c:234: INFO: [9] mapped: 40000, sigsegv hit: 28215, threads spawned:
16
mmap1.c:237: INFO: [9] repeated_reads: 4413, data_matched: 11314629 mmap1.c:234: INFO: [12] mapped: 55000, sigsegv hit: 38454, threads spawned:
22
mmap1.c:237: INFO: [12] repeated_reads: 9285, data_matched: 15892633 mmap1.c:234: INFO: [15] mapped: 60000, sigsegv hit: 42462, threads spawned:
24
mmap1.c:237: INFO: [15] repeated_reads: 11993, data_matched: 16723333 mmap1.c:234: INFO: [18] mapped: 65000, sigsegv hit: 46272, threads spawned:
26
mmap1.c:237: INFO: [18] repeated_reads: 11993, data_matched: 18043857 mmap1.c:234: INFO: [21] mapped: 80000, sigsegv hit: 58736, threads spawned:
32
mmap1.c:237: INFO: [21] repeated_reads: 14467, data_matched: 20310080 mmap1.c:234: INFO: [24] mapped: 85000, sigsegv hit: 62747, threads spawned:
34
mmap1.c:237: INFO: [24] repeated_reads: 14467, data_matched: 21065412 mmap1.c:234: INFO: [27] mapped: 95000, sigsegv hit: 67177, threads spawned:
38
mmap1.c:237: INFO: [27] repeated_reads: 14467, data_matched: 26278412 mmap1.c:234: INFO: [30] mapped: 110000, sigsegv hit: 75318, threads spawned:
44
mmap1.c:237: INFO: [30] repeated_reads: 18749, data_matched: 32880471 mmap1.c:234: INFO: [33] mapped: 130000, sigsegv hit: 86435, threads spawned:
52
mmap1.c:237: INFO: [33] repeated_reads: 20987, data_matched: 41628336 mmap1.c:234: INFO: [36] mapped: 150000, sigsegv hit: 95359, threads spawned:
60
mmap1.c:237: INFO: [36] repeated_reads: 20987, data_matched: 52762236 mmap1.c:234: INFO: [39] mapped: 175000, sigsegv hit: 107473, threads spawned:
70
mmap1.c:237: INFO: [39] repeated_reads: 20987, data_matched: 65170250 mmap1.c:234: INFO: [42] mapped: 195000, sigsegv hit: 115860, threads spawned:
78
mmap1.c:237: INFO: [42] repeated_reads: 24446, data_matched: 76553410 mmap1.c:234: INFO: [45] mapped: 220000, sigsegv hit: 130868, threads spawned:
88
mmap1.c:237: INFO: [45] repeated_reads: 28657, data_matched: 86358966 mmap1.c:234: INFO: [48] mapped: 245000, sigsegv hit: 145435, threads spawned:
98
mmap1.c:237: INFO: [48] repeated_reads: 34613, data_matched: 96820804 mmap1.c:234: INFO: [51] mapped: 260000, sigsegv hit: 154934, threads
spawned: 104
mmap1.c:237: INFO: [51] repeated_reads: 38042, data_matched: 102281865 mmap1.c:234: INFO: [54] mapped: 280000, sigsegv hit: 168403, threads
spawned: 112
mmap1.c:237: INFO: [54] repeated_reads: 44799, data_matched: 108884699 mmap1.c:234: INFO: [57] mapped: 305000, sigsegv hit: 188132, threads
spawned: 122
mmap1.c:237: INFO: [57] repeated_reads: 48995, data_matched: 113829351 mmap1.c:234: INFO: [60] mapped: 315000, sigsegv hit: 196457, threads
spawned: 126
mmap1.c:237: INFO: [60] repeated_reads: 51657, data_matched: 115444681 mmap1.c:234: INFO: [63] mapped: 330000, sigsegv hit: 208077, threads
spawned: 132
mmap1.c:237: INFO: [63] repeated_reads: 51657, data_matched: 118910807 mmap1.c:234: INFO: [66] mapped: 335000, sigsegv hit: 211847, threads
spawned: 134
mmap1.c:237: INFO: [66] repeated_reads: 51657, data_matched: 119936305 mmap1.c:234: INFO: [69] mapped: 355000, sigsegv hit: 225809, threads
spawned: 142
mmap1.c:237: INFO: [69] repeated_reads: 58357, data_matched: 125691672 mmap1.c:234: INFO: [72] mapped: 365000, sigsegv hit: 233748, threads
spawned: 146
mmap1.c:237: INFO: [72] repeated_reads: 63594, data_matched: 127682094 mmap1.c:234: INFO: [75] mapped: 375000, sigsegv hit: 240935, threads
spawned: 150
mmap1.c:237: INFO: [75] repeated_reads: 65813, data_matched: 130436913 mmap1.c:234: INFO: [78] mapped: 395000, sigsegv hit: 256994, threads
spawned: 158
mmap1.c:237: INFO: [78] repeated_reads: 68240, data_matched: 134313071 mmap1.c:234: INFO: [81] mapped: 400000, sigsegv hit: 260739, threads
spawned: 160
mmap1.c:237: INFO: [81] repeated_reads: 68240, data_matched: 135483078 mmap1.c:234: INFO: [84] mapped: 405000, sigsegv hit: 263510, threads
spawned: 162
mmap1.c:237: INFO: [84] repeated_reads: 68240, data_matched: 137744513 mmap1.c:234: INFO: [87] mapped: 415000, sigsegv hit: 271049, threads
spawned: 166
mmap1.c:237: INFO: [87] repeated_reads: 70071, data_matched: 140167936 mmap1.c:234: INFO: [90] mapped: 430000, sigsegv hit: 282504, threads
spawned: 172
mmap1.c:237: INFO: [90] repeated_reads: 74335, data_matched: 143263860 mmap1.c:234: INFO: [93] mapped: 435000, sigsegv hit: 286623, threads
spawned: 174
mmap1.c:237: INFO: [93] repeated_reads: 76581, data_matched: 144114430 mmap1.c:234: INFO: [96] mapped: 445000, sigsegv hit: 292401, threads
spawned: 178
mmap1.c:237: INFO: [96] repeated_reads: 78725, data_matched: 148160778 mmap1.c:234: INFO: [99] mapped: 450000, sigsegv hit: 296192, threads
spawned: 180
mmap1.c:237: INFO: [99] repeated_reads: 78725, data_matched: 149328825 mmap1.c:234: INFO: [102] mapped: 460000, sigsegv hit: 304280, threads
spawned: 184
mmap1.c:237: INFO: [102] repeated_reads: 78725, data_matched: 151183014 mmap1.c:234: INFO: [105] mapped: 465000, sigsegv hit: 308364, threads
spawned: 186
mmap1.c:237: INFO: [105] repeated_reads: 78725, data_matched: 152036389 Test timeouted, sending SIGKILL!
tst_test.c:1125: INFO: If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1
tst_test.c:1126: BROK: Test killed! (timeout?)
Test results comparison,
https://qa-reports.linaro.org/lkft/linux-mainline-oe/tests/ltp-mm-tests/mtest06

Quoted 7 lines

Quoted 1 line
Any chance to minimize the test run time ?
Default is 3 minutes, but it appears to be getting stuck before that. In [1], the output stopped after ~12-15 seconds, followed by timeout.
So either timeout arrived early (which seems unlikely) or test was stuck on something.

I've been running it in loop over night, but no luck reproducing on my arm64 with 4.18. Is it possible for you to try reproducing the hang and attaching gdb?

Quoted 8 lines

[1]
https://qa-reports.linaro.org/lkft/linux-mainline-oe/build/v5.0-rc5/testrun/594677/log
--
Mailing list info: https://lists.linux.it/listinfo/ltp