Skip to contentSkip to conversation

tst_test: Add test multiplex function

RFC PATCH Superseded Cyril Hrubis 4 participants 10 messages Mar 6, 2019 – Mar 7, 2019 Library code

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/2 tst_test: Add test multiplex function

The test multiplex function is intended for running the test function in a loop for a different settings. The indended purpose is to run tests for both libc wrapper and raw syscall as well as for different syscall variants.

The commit itself adds a test_multiplex() function into the tst_test structure, if set this function is called in a loop before each test iteration and is responsible for changing the test variant into next one. The iterations continue until the function returns zero.

CC Mark Salyzyn <[email protected]>
CC Steve Muckle <[email protected]>
CC Jan Stancek <[email protected]>
---
 include/tst_test.h | 14 ++++++++++++++
 lib/tst_test.c     | 11 +++++++++--
 2 files changed, 23 insertions(+), 2 deletions(-)
include/tst_test.h +14 -0
index da41f776d..b747b24d5 100644
@@ -146,6 +146,20 @@ struct tst_test {
 	 */
 	int all_filesystems:1;
 
+	/*
+	 * If set this function is used to mutiplex between different test
+	 * variants.
+	 *
+	 * Multiplexers are the way how to run the same test for different
+	 * settings. The intended use is to test different syscall
+	 * wrappers/variants but the API is generic and does not limit the
+	 * usage in any way.
+	 *
+	 * Each time the function is called it switches to next test variant,
+	 * returns zero if all variants has been iterated over.
+	 */
+	int (*test_multiplex)(void);
+
 	/* Minimal device size in megabytes */
 	unsigned int dev_min_size;
 
lib/tst_test.c +9 -2
index 7dd890b8d..6b033c879 100644
@@ -1009,8 +1009,15 @@ static void testrun(void)
 		if (!cont)
 			break;
 
-		run_tests();
-		heartbeat();
+		if (tst_test->test_multiplex) {
+			while (tst_test->test_multiplex()) {
+				run_tests();
+				heartbeat();
+			}
+		} else {
+			run_tests();
+			heartbeat();
+		}
 	}
 
 	do_test_cleanup();
Signature · 1 line
2.19.2
#1 lore raw

RFC PATCH 2/2 syscalls/select04: Test four syscall variants

This commit makes use of the newly added test_multiplex() function to switch between different syscall variants/wrappers at runtime.

CC Mark Salyzyn <[email protected]>
CC Steve Muckle <[email protected]>
CC Jan Stancek <[email protected]>
---
 testcases/kernel/syscalls/select/select04.c   |  5 +-
 testcases/kernel/syscalls/select/select_mpx.h | 79 +++++++++++++++++++
 2 files changed, 83 insertions(+), 1 deletion(-)
 create mode 100644 testcases/kernel/syscalls/select/select_mpx.h
testcases/kernel/syscalls/select/select04.c +4 -1
index 86bdffcdf..300992233 100644
@@ -27,6 +27,8 @@
 
 #include "tst_timer_test.h"
 
+#include "select_mpx.h"
+
 static int fds[2];
 
 static int sample_fn(int clk_id, long long usec)
@@ -39,7 +41,7 @@ static int sample_fn(int clk_id, long long usec)
 	FD_SET(fds[0], &sfds);
 
 	tst_timer_start(clk_id);
-	TEST(select(1, &sfds, NULL, NULL, &timeout));
+	TEST(do_select(1, &sfds, NULL, NULL, &timeout));
 	tst_timer_stop();
 	tst_timer_sample();
 
@@ -69,5 +71,6 @@ static struct tst_test test = {
 	.scall = "select()",
 	.sample = sample_fn,
 	.setup = setup,
+	.test_multiplex = select_mpx,
 	.cleanup = cleanup,
 };
testcases/kernel/syscalls/select/select_mpx.h new file +79 -0

79 lines not on this page · raw message

Signature · 1 line
2.19.2
#2 lore raw

RFC PATCH 2/2 syscalls/select04: Test four syscall variants

_THANKS_ for the set of changes!

testcases/kernel/syscalls/select/select_mpx.h line 39

Quoted 11 lines · Cyril Hrubis
testcases/kernel/syscalls/select/select_mpx.h new file +11 -0
+	case 2: {
+		struct compat_sel_arg_struct arg = {
+			._n = (long)nfds,
+			._inp = (long)readfds,
+			._outp = (long)writefds,
+			._exp = (long)exceptfds,
+			._tvp = (long)timeout,
+		};
+
+		return tst_syscall(__NR_select, &arg);
+	}

This fragment is correct for 32-bit configurations and 64-bit compat, but for 64-bit binaries it is:

#if defined(__LP64__)
return tst_syscall(__NR_select, nfds, readfds, writefds, exceptfds, timeout); #else
. . .
#endif

-- Mark

#3 lore raw

RFC PATCH 1/2 tst_test: Add test multiplex function

In reply to Cyril Hrubis

----- Original Message -----

Quoted 9 lines

The test multiplex function is intended for running the test function in a loop for a different settings. The indended purpose is to run tests for both libc wrapper and raw syscall as well as for different syscall variants.

The commit itself adds a test_multiplex() function into the tst_test structure, if set this function is called in a loop before each test iteration and is responsible for changing the test variant into next one. The iterations continue until the function returns zero.

Hi,

on first look this looks like a workaround, because we have locked ourselves out of .test function for timer tests.

Is there some way we could allow .test again for times tests? Something user could override, provided he then runs the library function that iterates over all usec values.

(sorry for pseudo-code)
// test
void my_test(int n)
{
select_func = n;
test_timer_test();
}

// library
void test_timer_test()
{
for (i = 0; i < ARRAY_SIZE(tcases); i++)

    timer_test_fn(i);
}

Regards,
Jan

Quoted 61 lines
CC Mark Salyzyn <[email protected]>
CC Steve Muckle <[email protected]>
CC Jan Stancek <[email protected]>
---
 include/tst_test.h | 14 ++++++++++++++
 lib/tst_test.c     | 11 +++++++++--
 2 files changed, 23 insertions(+), 2 deletions(-)
include/tst_test.h +14 -0
index da41f776d..b747b24d5 100644
@@ -146,6 +146,20 @@ struct tst_test {
 	 */
 	int all_filesystems:1;
 
+	/*
+	 * If set this function is used to mutiplex between different test
+	 * variants.
+	 *
+	 * Multiplexers are the way how to run the same test for different
+	 * settings. The intended use is to test different syscall
+	 * wrappers/variants but the API is generic and does not limit the
+	 * usage in any way.
+	 *
+	 * Each time the function is called it switches to next test variant,
+	 * returns zero if all variants has been iterated over.
+	 */
+	int (*test_multiplex)(void);
+
 	/* Minimal device size in megabytes */
 	unsigned int dev_min_size;
 
lib/tst_test.c +9 -3
index 7dd890b8d..6b033c879 100644
@@ -1009,8 +1009,15 @@ static void testrun(void)
 		if (!cont)
 			break;
 
-		run_tests();
-		heartbeat();
+		if (tst_test->test_multiplex) {
+			while (tst_test->test_multiplex()) {
+				run_tests();
+				heartbeat();
+			}
+		} else {
+			run_tests();
+			heartbeat();
+		}
 	}
 
 	do_test_cleanup();
--

2.19.2

#4 lore raw

RFC PATCH 1/2 tst_test: Add test multiplex function

Hi!

Quoted 2 lines

on first look this looks like a workaround, because we have locked ourselves out of .test function for timer tests.

I do not follow you here, can you elaborate?

This patch has nothing to do with timer tests, it just allows whatever the test does to be done several times with a hook to change some settings prior to each iteration.

Signature · 2 lines
Cyril Hrubis
[email protected]
#5 lore raw

RFC PATCH 1/2 tst_test: Add test multiplex function

----- Original Message -----

Quoted 5 lines

Hi!

Quoted 2 lines

on first look this looks like a workaround, because we have locked ourselves out of .test function for timer tests.

I do not follow you here, can you elaborate?

If this wasn't timer test, I'd ask why don't we use existing .test and .tcnt, your test() func can be called with a parameter, so you could change the code to choose correct syscall/glibc func based on value of that parameter.

For normal tests, this looks almost and .test/.tcnt functionality, except test count can be also dynamic.

static int tcase = -1;

static void test(void)
{
switch (tcase) {
}
}

static int select_mpx(void)

{
    tcase++;
    if (tcase == 5)
      return 0;
    return 1;
}

static struct tst_test test = {
.test_multiplex = select_mpx,
.test_all = test,
}

Quoted 9 lines

This patch has nothing to do with timer tests, it just allows whatever the test does to be done several times with a hook to change some settings prior to each iteration.

--
Cyril Hrubis
[email protected]

#6 lore raw

RFC PATCH 1/2 tst_test: Add test multiplex function

Hi!

Quoted 11 lines
Quoted 4 lines
Quoted 2 lines

on first look this looks like a workaround, because we have locked ourselves out of .test function for timer tests.

I do not follow you here, can you elaborate?

If this wasn't timer test, I'd ask why don't we use existing .test and .tcnt, your test() func can be called with a parameter, so you could change the code to choose correct syscall/glibc func based on value of that parameter.

For normal tests, this looks almost and .test/.tcnt functionality, except test count can be also dynamic.

Well so does the .all_filesystems flag, however I see these concepts to be orthogonal to what the actual test does, which is the reason I want them hooked up in the library rather than to be part of the testcase code.

Signature · 2 lines
Cyril Hrubis
[email protected]
#7 lore raw

RFC PATCH 1/2 tst_test: Add test multiplex function

----- Original Message -----

Quoted 12 lines

Hi!

Quoted 11 lines
Quoted 4 lines
Quoted 2 lines

on first look this looks like a workaround, because we have locked ourselves out of .test function for timer tests.

I do not follow you here, can you elaborate?

If this wasn't timer test, I'd ask why don't we use existing .test and .tcnt,
your test() func can be called with a parameter, so you could change the code to choose correct syscall/glibc func based on value of that parameter.

Quoted 7 lines
Quoted 2 lines

For normal tests, this looks almost and .test/.tcnt functionality, except test count can be also dynamic.

Well so does the .all_filesystems flag, however I see these concepts to be orthogonal to what the actual test does, which is the reason I want them hooked up in the library rather than to be part of the testcase code.

OK, so it's like another level on top of current test functions.

I probably would like 'variant' somewhere in name more, but overall I'm not against the patch. Are you planning on adding some docs too?

Regards,
Jan

#8 lore raw

RFC PATCH 1/2 tst_test: Add test multiplex function

Hi!

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

If this wasn't timer test, I'd ask why don't we use existing .test and .tcnt,
your test() func can be called with a parameter, so you could change the code to choose correct syscall/glibc func based on value of that parameter.

For normal tests, this looks almost and .test/.tcnt functionality, except test count can be also dynamic.

Quoted 6 lines
Quoted 4 lines

Well so does the .all_filesystems flag, however I see these concepts to be orthogonal to what the actual test does, which is the reason I want them hooked up in the library rather than to be part of the testcase code.

OK, so it's like another level on top of current test functions.

Yes, the change to the library adds generic layer for that.

Quoted 2 lines

I probably would like 'variant' somewhere in name more, but overall I'm not against the patch. Are you planning on adding some docs too?

I thought that calling it multiplex was descriptive enough name.

There is some summary in:

https://github.com/linux-test-project/ltp/issues/506

I guess that I can expand the text a bit and put it in the doc/ directory in the source tree along with the change to the test library.

Signature · 2 lines
Cyril Hrubis
[email protected]
#9 lore raw

RFC PATCH 1/2 tst_test: Add test multiplex function

In reply to Cyril Hrubis
Quoted 10 lines · Cyril Hrubis

The test multiplex function is intended for running the test function in a loop for a different settings. The indended purpose is to run tests for both libc wrapper and raw syscall as well as for different syscall variants.

The commit itself adds a test_multiplex() function into the tst_test structure, if set this function is called in a loop before each test iteration and is responsible for changing the test variant into next one. The iterations continue until the function returns zero.

Quoted 4 lines
CC Mark Salyzyn <[email protected]>
CC Steve Muckle <[email protected]>
CC Jan Stancek <[email protected]>

Approach seems good to me.

Quoted 53 lines
---
  include/tst_test.h | 14 ++++++++++++++
  lib/tst_test.c     | 11 +++++++++--
  2 files changed, 23 insertions(+), 2 deletions(-)
include/tst_test.h +14 -0
index da41f776d..b747b24d5 100644
@@ -146,6 +146,20 @@ struct tst_test {
  	 */
  	int all_filesystems:1;
 
+	/*
+	 * If set this function is used to mutiplex between different test
+	 * variants.
+	 *
+	 * Multiplexers are the way how to run the same test for different
+	 * settings. The intended use is to test different syscall
+	 * wrappers/variants but the API is generic and does not limit the
+	 * usage in any way.
+	 *
+	 * Each time the function is called it switches to next test variant,
+	 * returns zero if all variants has been iterated over.
+	 */
+	int (*test_multiplex)(void);
+
  	/* Minimal device size in megabytes */
  	unsigned int dev_min_size;
 
lib/tst_test.c +9 -2
index 7dd890b8d..6b033c879 100644
@@ -1009,8 +1009,15 @@ static void testrun(void)
  		if (!cont)
  			break;
 
-		run_tests();
-		heartbeat();
+		if (tst_test->test_multiplex) {
+			while (tst_test->test_multiplex()) {
+				run_tests();
+				heartbeat();
+			}
+		} else {
+			run_tests();
+			heartbeat();
+		}
  	}
 
  	do_test_cleanup();