Introduce runtime and conver tests
v3 cover Introduce runtime and conver tests
This patchset introduces a concept of runtime, which is described well enough in the first patch of the series and converts all tests that were previously using timeout API to runtime API.
The main change to the v1 of the patchset is that the runtime is now per iteration, e.g. if we run the test for all_filesystem the runtime applies to a single filesystem only. And at the same time the overall test timeout is a sum of static timeout and runtime. The static part of the timeout is meant to cover test teardown and any inaccuracy in the test runtime accounting.
The patches that convert tests to runtime API fall into these classes:
- fzsync tests
- faster timeout tests
- I/O tests
- timer precision tests
- oom tests
- misc tests that run for some time (controlled by command line parameter)
fzsync tests
============
These tests previously defined their runtime based on scale factor that used slice of the default timeout (300s) for the test runtime.
Now the runtime is explicitly encoded in the test, which I find much more user friendly because it's clear how long will the test run.
faster timeout tests
====================
There were several tests that set low timeout becuase the test may lock up. In these cases the timeout was simply removed because the default value (30s) should be small enough.
I/O tests
=========
The aiodio tests were modified so that their runtime is capped to 30 minutes by default. That means that even on very slow machine they finish in a reasonable time.
timer precision tests
=====================
These test simply set the test timeout to be 110% of the sum of the sleep_time * samples. That is to accomodate for the context switches and upkeeping between the calls in the timer library.
oom tests
=========
Here we just set the .max_iteration_runtime to TST_RUNTIME_UNLIMITED, which has the very same effect as .timeout = -1 previously had.
misc test
=========
In this case the test code was simplified quite a lot since the runtime is now passed in a unified way and parsed in the test library.
v3:
- renamed max_iteration_runtime to just max_runtime and tst_set_runtime() to tst_set_max_runtime()
- fixed newlib_tests and CI runtest.sh
- merged the two patches for af_alg02
- added and fixed documentation
- fixed a few bugs in the tst_test.c (LTP_RUNTIME_MUL etc.)
Cyril Hrubis (29):
Introduce a concept of max runtime mtest06/mmap1: Convert to runtime mtest06/mmap3: Convert to runtime mtest01/mtest01: Convert to runtime cve/cve-2015-3290: Convert to runtime crypto/af_alg02: Convert to runtime. crypto/pcrypt_aead01: Convert to runtime syscalls/clock_gettime01: Remove useless timeout syscalls/fanotify22: Remove useless timeout syscalls/gettimeofday02: Convert to runtime. syscalls/inotify06: Convert to runtime. syscalls/inotify01: Remove now useless timeout syscalls/perf_event_open03: Convert to runtime syscalls/readv01: Remove now useless timeout syscalls/tgkill03: Remove now unused timeout syscalls/setsockopt09: Remove now useless timeout syscalls/userfaultfd01: Remove now useless timeout syscalls/move_pages12: Convert to runtime syscalls/rt_sigqueueinfo01: Remove now useless timeout mem/mallocstress: Convert to runtime mem/{oom,min_free_kbytes}: Convert to runtime
fuzzy_sync: Convert to runtime ltp-aiodio/dio_sparse,aiodio_sparse: Convert to runtime. ltp-aiodio/read_checkzero: Remove ltp-aiodio/dio_{truncate,append}: Convert to runtime ltp-aiodio/dio_read: Convert to runtime
timer_test: Convert to runtime ltp-aiodio/aiodio_append: Convert to runtime
tst_test: Remove timeout stubs
doc/c-test-api.txt | 42 +++-- doc/user-guide.txt | 30 +++- include/tst_fuzzy_sync.h | 19 +-- include/tst_test.h | 31 +++- lib/newlib_tests/.gitignore | 5 +- lib/newlib_tests/runtest.sh | 2 +- lib/newlib_tests/test10.c | 22 --- lib/newlib_tests/test12.c | 21 --- lib/newlib_tests/test13.c | 1 - lib/newlib_tests/test18.c | 22 --- lib/newlib_tests/test_children_cleanup.c | 1 - lib/newlib_tests/test_runtime01.c | 30 ++++ lib/newlib_tests/test_runtime02.c | 28 ++++ lib/newlib_tests/tst_fuzzy_sync01.c | 1 + lib/newlib_tests/tst_fuzzy_sync02.c | 1 + lib/newlib_tests/tst_fuzzy_sync03.c | 1 + lib/tst_test.c | 143 +++++++++++++----- lib/tst_timer_test.c | 12 +- runtest/ltp-aiodio.part4 | 2 - testcases/cve/cve-2015-3290.c | 8 +- testcases/kernel/crypto/af_alg02.c | 4 +- testcases/kernel/crypto/af_alg07.c | 1 + testcases/kernel/crypto/pcrypt_aead01.c | 3 +- testcases/kernel/io/ltp-aiodio/.gitignore | 1 - .../kernel/io/ltp-aiodio/aiodio_append.c | 25 ++- .../kernel/io/ltp-aiodio/aiodio_sparse.c | 26 +++- testcases/kernel/io/ltp-aiodio/common.h | 12 +- testcases/kernel/io/ltp-aiodio/dio_append.c | 4 + testcases/kernel/io/ltp-aiodio/dio_read.c | 9 ++ testcases/kernel/io/ltp-aiodio/dio_sparse.c | 6 +- testcases/kernel/io/ltp-aiodio/dio_truncate.c | 6 + .../kernel/io/ltp-aiodio/read_checkzero.c | 84 ---------- testcases/kernel/mem/mtest01/mtest01.c | 14 +- testcases/kernel/mem/mtest06/mmap1.c | 24 +-- testcases/kernel/mem/mtest06/mmap3.c | 12 +- testcases/kernel/mem/mtest07/mallocstress.c | 11 +- testcases/kernel/mem/oom/oom01.c | 2 +- testcases/kernel/mem/oom/oom02.c | 2 +- testcases/kernel/mem/oom/oom03.c | 2 +- testcases/kernel/mem/oom/oom04.c | 2 +- testcases/kernel/mem/oom/oom05.c | 2 +- testcases/kernel/mem/thp/thp04.c | 1 + .../kernel/mem/tunable/min_free_kbytes.c | 2 +- testcases/kernel/pty/pty03.c | 2 +- testcases/kernel/pty/pty05.c | 1 + testcases/kernel/pty/pty06.c | 1 + testcases/kernel/pty/pty07.c | 1 + testcases/kernel/sound/snd_seq01.c | 2 +- testcases/kernel/sound/snd_timer01.c | 1 + testcases/kernel/syscalls/bind/bind06.c | 2 +- .../syscalls/clock_gettime/clock_gettime01.c | 1 - .../kernel/syscalls/fanotify/fanotify22.c | 1 - .../syscalls/gettimeofday/gettimeofday02.c | 15 +- testcases/kernel/syscalls/inotify/inotify06.c | 7 +- testcases/kernel/syscalls/inotify/inotify09.c | 1 + testcases/kernel/syscalls/inotify/inotify11.c | 1 - .../kernel/syscalls/ipc/shmctl/shmctl05.c | 2 +- .../kernel/syscalls/move_pages/move_pages12.c | 4 +- .../perf_event_open/perf_event_open03.c | 15 +- testcases/kernel/syscalls/readv/readv01.c | 1 - .../rt_sigqueueinfo/rt_sigqueueinfo01.c | 1 - testcases/kernel/syscalls/sendmsg/sendmsg03.c | 1 + .../kernel/syscalls/setsockopt/setsockopt06.c | 2 +- .../kernel/syscalls/setsockopt/setsockopt07.c | 1 + .../kernel/syscalls/setsockopt/setsockopt09.c | 1 - testcases/kernel/syscalls/tgkill/tgkill03.c | 1 - .../syscalls/timerfd/timerfd_settime02.c | 1 + .../syscalls/userfaultfd/userfaultfd01.c | 1 - testcases/kernel/syscalls/writev/writev03.c | 2 +- 69 files changed, 419 insertions(+), 327 deletions(-) delete mode 100644 lib/newlib_tests/test10.c delete mode 100644 lib/newlib_tests/test12.c delete mode 100644 lib/newlib_tests/test18.c create mode 100644 lib/newlib_tests/test_runtime01.c create mode 100644 lib/newlib_tests/test_runtime02.c delete mode 100644 testcases/kernel/io/ltp-aiodio/read_checkzero.c
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 1/29 Introduce a concept of max runtime
This commit introduces a concept of max test runtime. In other words test runtime is capped at a certain value in order to make testruns more deterministic. Test is free to to finish before the runtime is used up, for example when maximal number of iterations was reached, but test must stop once the runtime has been used up.
Testcases that run for more than a second or two must check for remaining runtime by regular calls to tst_remaining_runtime() and should exit when zero is returned.
The test max runtime must be set either by the .max_iteration_runtime in the tst_test structure or in the test setup by a call to tst_set_runtime().
The test timeout is then computed as a sum of DEFAULT_TIMEOUT (currently set to 30 seconds) and the test runtime. The DEFAULT_TIMEOUT is nothing more than a safety margin for teardown of the test.
This commit also maps the -I parameter to the test max runtime if available and introduces LTP_RUNTIME_MUL enviroment variable so that we have an easy controll over the runtime cap.
Lastly but not least the function related to the timeout are turned into no-op by this commit and removed after all test are converted to the runtime API.
--- doc/c-test-api.txt | 42 +++++-- doc/user-guide.txt | 30 ++++- include/tst_test.h | 27 ++++ lib/newlib_tests/.gitignore | 5 +- lib/newlib_tests/runtest.sh | 2 +- lib/newlib_tests/test10.c | 22 ---- lib/newlib_tests/test12.c | 21 ---- lib/newlib_tests/test13.c | 1 - lib/newlib_tests/test18.c | 22 ---- lib/newlib_tests/test_children_cleanup.c | 1 - lib/newlib_tests/test_runtime01.c | 30 +++++ lib/newlib_tests/test_runtime02.c | 28 +++++ lib/tst_test.c | 150 +++++++++++++++++------ 13 files changed, 264 insertions(+), 117 deletions(-) delete mode 100644 lib/newlib_tests/test10.c delete mode 100644 lib/newlib_tests/test12.c delete mode 100644 lib/newlib_tests/test18.c create mode 100644 lib/newlib_tests/test_runtime01.c create mode 100644 lib/newlib_tests/test_runtime02.c
doc/c-test-api.txt +32 -10
@@ -93,14 +93,35 @@ in range of [0, '.tcnt' - 1]. IMPORTANT: Only one of '.test' and '.test_all' can be set at a time. -Each test has a default timeout set to 300s. The default timeout can be -overridden by setting '.timeout' in the test structure or by calling -'tst_set_timeout()' in the test 'setup()'. There are a few testcases whose run -time may vary arbitrarily, for these timeout can be disabled by setting it to --1. +Each test has a limit on how long it can run and the limit composes of two +parts max_runtime and timeout. The max_runtime is a limit for how long can the +'.test_all' or a set of '.test' functions take and the timeout is static part +that should cover the duration of test setup and cleanup plus some safety. -Test can find out how much time (in seconds) is remaining to timeout, -by calling 'tst_timeout_remaining()'. +Any test that runs for more than a second or two has to make sure to: + +- set the runtime either by setting the '.max_runtime' in tst_test or by + calling 'tst_set_runtime()' in the test setup + +- monitor remaning runtime by regular calls to 'tst_remaining_runtime()' and + exit when runtime has been used up + +Test is free to exit before max_runtime has been used up for example when +minimal number of iteration was finished. + +The limit is applied to a single call of the '.test_all' function that means +that for example when '.test_variants' or '.all_filesystems' is set the whole +test will be limited by 'variants * (max_runtime + timeout)' seconds and the +test runtime will be likely close to 'variants * max_runtime' seconds. + +[source,c] +------------------------------------------------------------------------------- +/* + * Returns number of seconds or zero in case that runtime has been used up. + */ + +int tst_remaining_runtime(void); +------------------------------------------------------------------------------- LAPI headers ++++++++++++
@@ -377,12 +398,13 @@ WARNING: This function is not thread safe. [source,c] ------------------------------------------------------------------------------- -void tst_set_timeout(unsigned int timeout); +void tst_set_max_runtime(int max_runtime); ------------------------------------------------------------------------------- -Allows for setting timeout per test iteration dynamically in the test setup(), +Allows for setting max_runtime per test iteration dynamically in the test setup(), the timeout is specified in seconds. There are a few testcases whose runtime -can vary arbitrarily, these can disable timeouts by setting it to -1. +can vary arbitrarily, these can disable timeouts by setting it to +TST_UNLIMITED_RUNTIME. [source,c] -------------------------------------------------------------------------------
doc/user-guide.txt +29 -1
@@ -22,9 +22,13 @@ For running LTP network tests see `testcases/network/README.md`. | 'LTP_SINGLE_FS_TYPE' | Testing only - specifies filesystem instead all supported (for tests with '.all_filesystems'). | 'LTP_DEV_FS_TYPE' | Filesystem used for testing (default: 'ext2'). -| 'LTP_TIMEOUT_MUL' | Multiply timeout, must be number >= 1 (> 1 is useful for +| 'LTP_TIMEOUT_MUL' | Multiplies timeout, must be number >= 0.1 (> 1 is useful for slow machines to avoid unexpected timeout). Variable is also used in shell tests, but ceiled to int. +| 'LTP_RUNTIME_MUL' | Multiplies maximal test iteration runtime. Tests + that run for more than a second or two are capped on + runtime. You can scale the default runtime both up + and down with this multiplier. | 'LTP_VIRT_OVERRIDE' | Overrides virtual machine detection in the test library. Setting it to empty string tell the library that system is not a virtual machine. Other possible
@@ -36,3 +40,27 @@ For running LTP network tests see `testcases/network/README.md`. and others, which imply it, shell: 'TST_NEEDS_TMPDIR=1'). | 'TST_NO_CLEANUP' | Disable running test cleanup (defined in 'TST_CLEANUP'). |============================================================================== + + +2. Test execution time and timeout +---------------------------------- + +The limit on how long a test can run does compose of two parts max_runtime and +timeout. The limit does apply to a single test variant, that means for example +that tests that run for all available filesystems will apply this limit for a +single filesytem only. + +The max_runtime is a cap on how long the run() function can take, for most +testcases this part is set to zero. For tests that do run for more than a +second or two the max_runtime has to be defined and the run() function has to +check actively how much runtime is left. + +Test runtime can be scaled up and down with 'LTP_RUNTIME_MUL' environment +variable or set on a commandline by the '-I' parameter. Hoewever be vary that +setting the runtime too low will cause long running tests to exit prematurely +possibly before the have a chance actually test anyting. + +The timeout part is a limit for the test setup and cleanup and also safety +margin for the runtime accounting. It's currently set to 30 seconds but may +change later. If your target machine is too slow it can be scaled up with the +'LTP_TIMEOUT_MUL' environment variable.
include/tst_test.h +27 -0
@@ -134,6 +134,8 @@ extern unsigned int tst_variant;#define TST_NO_HUGEPAGES ((unsigned long)-1) +#define TST_UNLIMITED_RUNTIME (-1) +struct tst_test {/* number of tests available in test() function */ unsigned int tcnt;
@@ -236,6 +238,18 @@ struct tst_test {/* override default timeout per test run, disabled == -1 */ int timeout; +/* +* Maximal test runtime in seconds. +* +* Any test that runs for more than a second or two should set this and +* also use tst_remaining_runtime() to exit when runtime was used up. +* Tests may finish sooner, for example if requested number of +* iterations was reached before the runtime runs out. +* +* If test runtime cannot be know in advance it should be set to +* TST_UNLIMITED_RUNTIME. +*/ +int max_runtime;void (*setup)(void );void (*cleanup)(void );
@@ -323,6 +337,19 @@ unsigned int tst_timeout_remaining(void);unsigned int tst_multiply_timeout (unsigned int timeout);void tst_set_timeout (int timeout); +/* +* Returns remaining test runtime. Test that runs for more than a few seconds +* should check if they should exit by calling this function regularly. +* +* The function returns remaining runtime in seconds. If runtime was used up +* zero is returned. +*/ +unsigned int tst_remaining_runtime (void ); + +/* +* Sets maximal test runtime in seconds. +*/ +void tst_set_max_runtime (int max_runtime);/* * Returns path to the test temporary directory in a newly allocated buffer.
lib/newlib_tests/.gitignore +2 -3
@@ -7,9 +7,7 @@ test06 test07 test08 test09 -test10 test11 -test12 test13 test14 test15
@@ -22,7 +20,6 @@ tst_safe_fileops tst_res_hexd tst_strstatus tst_print_result -test18 test19 test20 test22
@@ -56,3 +53,5 @@ tst_needs_cmds05 tst_needs_cmds06 tst_needs_cmds07 tst_needs_cmds08 +test_runtime01 +test_runtime02
lib/newlib_tests/runtest.sh +1 -1
@@ -1,7 +1,7 @@ #!/bin/sh # Copyright (c) 2021 Petr Vorel <[email protected]> -LTP_C_API_TESTS="${LTP_C_API_TESTS:-test05 test07 test09 test12 test15 test18 +LTP_C_API_TESTS="${LTP_C_API_TESTS:-test05 test07 test09 test15 test_runtime01 tst_needs_cmds01 tst_needs_cmds02 tst_needs_cmds03 tst_needs_cmds06 tst_needs_cmds07 tst_bool_expr test_exec test_timer tst_res_hexd tst_strstatus tst_fuzzy_sync03 test_zero_hugepage.sh test_kconfig.sh
lib/newlib_tests/test10.c deleted +0 -22
@@ -1,22 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-or-later -/* -* Copyright (c) 2016 Linux Test Project -*/ - -/* -* Test for watchdog timeout. -*/ - -#include "tst_test.h" - - -static void do_test (void ) -{ -sleep (2 ); -tst_res (TPASS ,"Not reached" ); -} - -static struct tst_test test = { - .test_all = do_test, - .timeout =1 , -};
lib/newlib_tests/test12.c deleted +0 -21
@@ -1,21 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-or-later -/* -* Copyright (c) 2016 Linux Test Project -*/ - -/* -* Test for timeout override. -*/ - -#include "tst_test.h" - -static void do_test (void ) -{ -sleep (1 ); -tst_res (TPASS ,"Passed!" ); -} - -static struct tst_test test = { - .timeout =2 , - .test_all = do_test, -};
lib/newlib_tests/test13.c +0 -1
@@ -20,7 +20,6 @@ static void do_test(void) }static struct tst_test test = { - .timeout =1 , .forks_child =1 , .test_all = do_test, };
lib/newlib_tests/test18.c deleted +0 -22
@@ -1,22 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-or-later -/* -* Copyright (c) 2018, Linux Test Project -*/ - -#include <stdlib.h> -#include <unistd.h> -#include "tst_test.h" - -static void run (void ) -{ -do { -sleep (1 ); - }while (tst_timeout_remaining () >=4 ); - -tst_res (TPASS ,"Timeout remaining: %d" ,tst_timeout_remaining ()); -} - -static struct tst_test test = { - .test_all = run, - .timeout =5 -};
lib/newlib_tests/test_children_cleanup.c +0 -1
@@ -39,5 +39,4 @@ static void run(void)static struct tst_test test = { .test_all = run, .forks_child =1 , - .timeout =10 , };
lib/newlib_tests/test_runtime01.c new file +30 -0
@@ -0,0 +1,30 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (c) 2018, Linux Test Project +* +* Runs for 4 seconds for each test iteration. +*/ +#include <stdlib.h> +#include <unistd.h> +#include "tst_test.h" + +static void run (void ) +{ +int runtime; + +tst_res (TINFO ,"Running variant %i" , tst_variant); + +do { + runtime =tst_remaining_runtime (); +tst_res (TINFO ,"Remaining runtime %d" , runtime); +sleep (1 ); + }while (runtime); + +tst_res (TPASS ,"Finished loop!" ); +} + +static struct tst_test test = { + .test_all = run, + .max_runtime =4 , + .test_variants =2 , +};
lib/newlib_tests/test_runtime02.c new file +28 -0
@@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (c) 2021, Linux Test Project +*/ +/* +* This test is set up so that the timeout is not long enough to guarantee +* enough runtime for two iterations, i.e. the timeout without offset and after +* scaling is too small and the tests ends up with TBROK. +* +* The default timeout in the test library is set to 30 seconds. The test +* runtime is set to 5 so the test should timeout after 35 seconds. +*/ + +#include <stdlib.h> +#include <unistd.h> +#include "tst_test.h" + +static void run (void ) +{ +tst_res (TINFO ,"Sleeping for 40 seconds" ); +sleep (40 ); +tst_res (TFAIL ,"Still alive" ); +} + +static struct tst_test test = { + .test_all = run, + .max_runtime =5 , +};
lib/tst_test.c +115 -35
244 lines not on this page · raw message
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 01/29] Introduce a concept of max runtime
Quoted 6 lines · Cyril Hrubis
This commit introduces a concept of max test runtime. In other words test runtime is capped at a certain value in order to make testruns more deterministic. Test is free to to finish before the runtime is used up, for example when maximal number of iterations was reached, but test must stop once the runtime has been used up.
Quoted 6 lines
Testcases that run for more than a second or two must check for remaining runtime by regular calls to
tst_remaining_runtime()and should exit when zero is returned.The test max runtime must be set either by the .max_iteration_runtime in
.max_runtime
Quoted 3 lines
the
tst_teststructure or in the test setup by a call totst_set_runtime().
tst_set_max_runtime()
Quoted 56 lines
The test timeout is then computed as a sum of
DEFAULT_TIMEOUT(currently set to 30 seconds) and the test runtime. TheDEFAULT_TIMEOUTis nothing more than a safety margin for teardown of the test.This commit also maps the -I parameter to the test max runtime if available and introduces
LTP_RUNTIME_MULenviroment variable so that we have an easy controll over the runtime cap.Lastly but not least the function related to the timeout are turned into no-op by this commit and removed after all test are converted to the runtime API.
Signed-off-by Cyril Hrubis <[email protected]>--- doc/c-test-api.txt | 42 +++++-- doc/user-guide.txt | 30 ++++- include/tst_test.h | 27 ++++ lib/newlib_tests/.gitignore | 5 +- lib/newlib_tests/runtest.sh | 2 +- lib/newlib_tests/test10.c | 22 ---- lib/newlib_tests/test12.c | 21 ---- lib/newlib_tests/test13.c | 1 - lib/newlib_tests/test18.c | 22 ---- lib/newlib_tests/test_children_cleanup.c | 1 - lib/newlib_tests/test_runtime01.c | 30 +++++ lib/newlib_tests/test_runtime02.c | 28 +++++ lib/tst_test.c | 150 +++++++++++++++++------ 13 files changed, 264 insertions(+), 117 deletions(-) delete mode 100644 lib/newlib_tests/test10.c delete mode 100644 lib/newlib_tests/test12.c delete mode 100644 lib/newlib_tests/test18.c create mode 100644 lib/newlib_tests/test_runtime01.c create mode 100644 lib/newlib_tests/test_runtime02.cdoc/c-test-api.txt +0
-3@@ -93,14 +93,35 @@ in range of [0, '.tcnt' - 1]. IMPORTANT: Only one of '.test' and '.test_all' can be set at a time. -Each test has a default timeout set to 300s. The default timeout can be -overridden by setting '.timeout' in the test structure or by calling -'tst_set_timeout()' in the test 'setup()'. There are a few testcaseswhose run
-time may vary arbitrarily, for these timeout can be disabled by setting it to
--1.doc/c-test-api.txt +2
-0+Each test has a limit on how long it can run and the limit composes of two +parts max_runtime and timeout. The max_runtime is a limit for how longcan the
+'.test_all' or a set of '.test' functions take and the timeout is static part
doc/c-test-api.txt in range of [0, '.tcnt' - 1]. · line 104
Quoted 8 lines
doc/c-test-api.txt +5
-2+that should cover the duration of test setup and cleanup plus some safety. -Test can find out how much time (in seconds) is remaining to timeout, -by calling 'tst_timeout_remaining()'. +Any test that runs for more than a second or two has to make sure to: + +- set the runtime either by setting the '.max_runtime' in tst_test or by + calling 'tst_set_runtime()' in the test setup
^ calling 'tst_set_max_runtime()'
Apart from the patch-22 tiny issue, the rest part looks good.
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [Automated-testing] [PATCH v3 01/29] Introduce a concept of max runtime
In reply to Cyril HrubisHello,
Quoted 22 lines · Cyril Hrubis
This commit introduces a concept of max test runtime. In other words test runtime is capped at a certain value in order to make testruns more deterministic. Test is free to to finish before the runtime is used up, for example when maximal number of iterations was reached, but test must stop once the runtime has been used up.
Testcases that run for more than a second or two must check for remaining runtime by regular calls to
tst_remaining_runtime()and should exit when zero is returned.The test max runtime must be set either by the .max_iteration_runtime in the
tst_teststructure or in the test setup by a call totst_set_runtime().The test timeout is then computed as a sum of
DEFAULT_TIMEOUT(currently set to 30 seconds) and the test runtime. TheDEFAULT_TIMEOUTis nothing more than a safety margin for teardown of the test.This commit also maps the -I parameter to the test max runtime if available and introduces
LTP_RUNTIME_MULenviroment variable so that we have an easy controll over the runtime cap.
Quoted 5 lines
Lastly but not least the function related to the timeout are turned into no-op by this commit and removed after all test are converted to the runtime API.
Signed-off-by Cyril Hrubis <[email protected]>
Signature · 5 lines
Thank you, Richard. -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 2/29 mtest06/mmap1: Convert to runtime
In reply to Cyril Hrubis--- testcases/kernel/mem/mtest06/mmap1.c | 24 ++++++------------------ 1 file changed, 6 insertions(+), 18 deletions(-)
testcases/kernel/mem/mtest06/mmap1.c +6 -18
@@ -42,10 +42,8 @@static int file_size =1024 ;static int num_iter =5000 ; -static float exec_time =0.05 ;/* default is 3 min */ static void *distant_area; -static char *str_exec_time;static jmp_buf jmpbuf;static volatile unsigned char *map_address;static unsigned long page_sz;
@@ -206,17 +204,10 @@ static void setup(void)SAFE_MUNMAP (distant_area, distant_mmap_size); distant_area += distant_mmap_size /2 ; -if (tst_parse_float (str_exec_time, &exec_time,0 ,FLT_MAX )) { -tst_brk (TBROK ,"Invalid number for exec_time '%s'" , - str_exec_time); - } - sigptr.sa_sigaction = sig_handler;sigemptyset (&sigptr.sa_mask); sigptr.sa_flags =SA_SIGINFO |SA_NODEFER ;SAFE_SIGACTION (SIGSEGV , &sigptr,NULL ); - -tst_set_timeout ((int )(exec_time *3600 )); }static void run (void )
@@ -224,8 +215,8 @@ static void run(void)pthread_t thid[2 ];int start, last_update; - start = last_update =tst_timeout_remaining (); -while (tst_timeout_remaining () >STOP_THRESHOLD ) { + start = last_update =tst_remaining_runtime (); +while (tst_remaining_runtime ()) {int fd =mkfile (file_size);tst_atomic_store (0 , &mapcnt);
@@ -240,11 +231,11 @@ static void run(void)close (fd); -if (last_update -tst_timeout_remaining () >=PROGRESS_SEC ) { - last_update =tst_timeout_remaining (); +if (last_update -tst_remaining_runtime () >=PROGRESS_SEC ) { + last_update =tst_remaining_runtime ();tst_res (TINFO ,"[%03d] mapped: %lu, sigsegv hit: %lu, " "threads spawned: %lu" , - start -tst_timeout_remaining (), + start - last_update, map_count, mapped_sigsegv_count, threads_spawned); tst_res(TINFO, " repeated_reads: %ld, "
@@ -258,9 +249,6 @@ static void run(void)static struct tst_test test = { .test_all = run, .setup = setup, - .options = (struct tst_option []) { - {"x:" , &str_exec_time,"Exec time (hours)" }, - {} - }, + .max_runtime =180 , .needs_tmpdir =1 , };
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 3/29 mtest06/mmap3: Convert to runtime
In reply to Cyril Hrubis--- testcases/kernel/mem/mtest06/mmap3.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)
testcases/kernel/mem/mtest06/mmap3.c +2 -10
@@ -23,11 +23,9 @@static char *str_loops;static char *str_threads;static char *map_private; -static char *str_exec_time;static int loops =1000 ;static int threads =40 ; -static float exec_time =24 ;static volatile int sig_caught;static int threads_running;
@@ -109,7 +107,7 @@ static void test_mmap(void)long i;pthread_t thids[threads]; -alarm (exec_time *3600 ); +alarm (tst_remaining_runtime ());while (!sig_caught) {for (i =0 ; i < threads; i++) {
@@ -138,11 +136,6 @@ static void setup(void)if (tst_parse_int (str_threads, &threads,1 ,INT_MAX ))tst_brk (TBROK ,"Invalid number of threads '%s'" , str_threads); -if (tst_parse_float (str_exec_time, &exec_time,0.0005 ,9000 )) -tst_brk (TBROK ,"Invalid execution time '%s'" , str_exec_time); - -tst_set_timeout (exec_time *3600 +300 ); -SAFE_SIGNAL (SIGALRM , sig_handler);SAFE_SIGNAL (SIGBUS , sig_handler);SAFE_SIGNAL (SIGSEGV , sig_handler);
@@ -155,7 +148,6 @@ static void setup(void)tst_res (TINFO ,"Number of loops %i" , loops);tst_res (TINFO ,"Number of threads %i" , threads);tst_res (TINFO ,"MAP_PRIVATE = %i" , map_private ?1 :0 ); -tst_res (TINFO ,"Execution time %fH" , exec_time); }static void cleanup (void )
@@ -184,11 +176,11 @@ static struct tst_test test = { {"l:", &str_loops, "Number of map-write-unmap loops"}, {"n:", &str_threads, "Number of worker threads"}, {"p", &map_private, "Turns on MAP_PRIVATE (default MAP_SHARED)"}, - {"x:", &str_exec_time, "float Execution time in hours (default 24H)"}, {} }, .needs_tmpdir = 1, .setup = setup, .cleanup = cleanup, .test_all = test_mmap, + .max_runtime = 86400, };
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 4/29 mtest01/mtest01: Convert to runtime
In reply to Cyril Hrubis--- testcases/kernel/mem/mtest01/mtest01.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)
testcases/kernel/mem/mtest01/mtest01.c +8 -6
@@ -41,8 +41,6 @@#define ALLOC_THRESHOLD (6*FIVE_HUNDRED_MB)#endif -#define STOP_THRESHOLD 15/* seconds remaining before reaching timeout */ -static pid_t *pid_list;static sig_atomic_t children_done;static int max_pids;
@@ -137,6 +135,7 @@ static void child_loop_alloc(unsigned long long alloc_bytes) {unsigned long bytecount =0 ;char *mem; +int runtime_rem;tst_res (TINFO ,"... child %d starting" ,getpid ());
@@ -153,12 +152,15 @@ static void child_loop_alloc(unsigned long long alloc_bytes)if (bytecount >= alloc_bytes)break ; } + + runtime_rem =tst_remaining_runtime (); +if (dowrite)tst_res (TINFO ,"... [t=%d] %lu bytes allocated and used in child %d" , -tst_timeout_remaining (), bytecount,getpid ()); + runtime_rem, bytecount,getpid ());else tst_res (TINFO ,"... [t=%d] %lu bytes allocated only in child %d" , -tst_timeout_remaining (), bytecount,getpid ()); + runtime_rem, bytecount,getpid ());kill (getppid (),SIGRTMIN );raise (SIGSTOP );
@@ -195,10 +197,9 @@ static void mem_test(void)/* wait in the loop for all children finish allocating */ while (children_done < pid_cntr) { -if (tst_timeout_remaining () <STOP_THRESHOLD ) { +if (!tst_remaining_runtime ()) {tst_res (TWARN ,"the remaininig time is not enough for testing" ); -break ; }
@@ -234,6 +235,7 @@ static struct tst_test test = { {"v", &verbose, "Verbose"}, {} }, + .max_runtime = 300, .setup = setup, .cleanup = cleanup, .test_all = mem_test,
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 5/29 cve/cve-2015-3290: Convert to runtime
In reply to Cyril Hrubis--- testcases/cve/cve-2015-3290.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
testcases/cve/cve-2015-3290.c +2 -6
@@ -395,10 +395,6 @@ static void *child_thread(void *arg LTP_ATTRIBUTE_UNUSED)return (void *)niter; } -#define TIMEOUT (180) -#define TIME_TO_GIVEUP (TIMEOUT - 5) -#define TIMER_TYPE CLOCK_MONOTONIC -static void do_child (void ) {int i, ncpus;
@@ -415,7 +411,7 @@ static void do_child(void)for (i =0 ; i < ncpus; i++)SAFE_PTHREAD_CREATE (&threads[i],NULL , child_thread,NULL ); -sleep (TIME_TO_GIVEUP ); +sleep (tst_remaining_runtime ()); running =0 ;for (i =0 ; i < ncpus; i++) {
@@ -468,7 +464,7 @@ static struct tst_test test = { .needs_root = 1, .needs_checkpoints = 1, .setup = setup, - .timeout = TIMEOUT, + .max_runtime = 180, .test_all = run, .tags = (const struct tst_tag []) { {"linux-git" ,"9b6e6a8334d5" },
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 6/29 crypto/af_alg02: Convert to runtime.
In reply to Cyril Hrubis--- testcases/kernel/crypto/af_alg02.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
testcases/kernel/crypto/af_alg02.c +2 -2
@@ -65,7 +65,7 @@ static void run(void)TST_CHECKPOINT_WAIT (0 );while (!completed) { -if (tst_timeout_remaining () <=10 ) { +if (!tst_remaining_runtime ()) {pthread_cancel (thr);tst_brk (TBROK ,"Timed out while reading from request socket." );
@@ -77,7 +77,7 @@ static void run(void)static struct tst_test test = { .test_all = run, - .timeout =20 , + .max_runtime =20 , .needs_checkpoints =1 , .tags = (const struct tst_tag []) { {"linux-git" ,"ecaaab564978" },
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 7/29 crypto/pcrypt_aead01: Convert to runtime
In reply to Cyril Hrubis--- testcases/kernel/crypto/pcrypt_aead01.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
testcases/kernel/crypto/pcrypt_aead01.c +2 -1
@@ -55,7 +55,7 @@ void run(void)if (TST_RET )tst_brk (TBROK |TRERRNO ,"del_alg" ); -if (tst_timeout_remaining () <10 ) { +if (!tst_remaining_runtime ()) {tst_res (TINFO ,"Time limit reached, stopping at " "%d iterations" , i);break ;
@@ -75,6 +75,7 @@ static struct tst_test test = { .test_all = run, .cleanup = cleanup, .needs_root = 1, + .max_runtime = 300, .tags = (const struct tst_tag []) { {"linux-git" ,"d76c68109f37" }, {"CVE" ,"2017-5754" },
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 8/29 syscalls/clock_gettime01: Remove useless timeout
In reply to Cyril Hrubis--- testcases/kernel/syscalls/clock_gettime/clock_gettime01.c | 1 - 1 file changed, 1 deletion(-)
testcases/kernel/syscalls/clock_gettime/clock_gettime01.c +0 -1
@@ -126,5 +126,4 @@ static struct tst_test test = { .test_variants = ARRAY_SIZE(variants), .setup = setup, .needs_root = 1, - .timeout = 10, };
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 9/29 syscalls/fanotify22: Remove useless timeout
In reply to Cyril HrubisI guess that the timeout was there to exit faster in a case that the test gets stuck. However that is no longer needed as the default timeout was shortented significantly.
--- testcases/kernel/syscalls/fanotify/fanotify22.c | 1 - 1 file changed, 1 deletion(-)
testcases/kernel/syscalls/fanotify/fanotify22.c +0 -1
@@ -305,7 +305,6 @@ static struct tst_test test = { {"linux-git", "124e7c61deb2"}, {} }, - .timeout = 10, .needs_cmds = (const char *[]) {"debugfs" , NULL
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 10/29 syscalls/gettimeofday02: Convert to runtime.
In reply to Cyril Hrubis--- .../kernel/syscalls/gettimeofday/gettimeofday02.c | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-)
testcases/kernel/syscalls/gettimeofday/gettimeofday02.c +2 -13
@@ -25,8 +25,6 @@#include "lapi/syscalls.h" static volatile sig_atomic_t done; -static char *str_rtime; -static int rtime =10 ;static void breakout (int sig) {
@@ -37,6 +35,7 @@ static void verify_gettimeofday(void) {struct __kernel_old_timeval tv1, tv2;unsigned long long cnt =0 ; +int rtime =tst_remaining_runtime (); done =0 ;
@@ -68,21 +67,11 @@ static void verify_gettimeofday(void)static void setup (void ) { -if (str_rtime) { - rtime =atoi (str_rtime); -if (rtime <=0 ) -tst_brk (TBROK ,"Invalid runtime '%s'" , str_rtime); -tst_set_timeout (rtime +60 ); - } -SAFE_SIGNAL (SIGALRM , breakout); }static struct tst_test test = { .setup = setup, - .options = (struct tst_option []) { - {"T:" , &str_rtime,"Test iteration runtime in seconds" }, - {}, - }, + .max_runtime =10 , .test_all = verify_gettimeofday, };
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 11/29 syscalls/inotify06: Convert to runtime.
In reply to Cyril Hrubis--- testcases/kernel/syscalls/inotify/inotify06.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
testcases/kernel/syscalls/inotify/inotify06.c +6 -1
@@ -87,6 +87,11 @@ static void verify_inotify(void)myinotify_add_watch (inotify_fd, names[i],IN_MODIFY ); }SAFE_CLOSE (inotify_fd); + +if (!tst_remaining_runtime ()) { +tst_res (TINFO ,"Test out of runtime, exitting" ); +break ; + } }/* We survived for given time - test succeeded */ tst_res (TPASS ,"kernel survived inotify beating" );
@@ -108,7 +113,7 @@ static void cleanup(void) }static struct tst_test test = { - .timeout =600 , + .max_runtime =600 , .needs_root =1 , .needs_tmpdir =1 , .forks_child =1 ,
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 12/29 syscalls/inotify01: Remove now useless timeout
In reply to Cyril Hrubis--- testcases/kernel/syscalls/inotify/inotify11.c | 1 - 1 file changed, 1 deletion(-)
testcases/kernel/syscalls/inotify/inotify11.c +0 -1
@@ -117,7 +117,6 @@ static void cleanup(void) }static struct tst_test test = { - .timeout =10 , .needs_tmpdir =1 , .forks_child =1 , .cleanup = cleanup,
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 13/29 syscalls/perf_event_open03: Convert to runtime
In reply to Cyril Hrubis--- .../syscalls/perf_event_open/perf_event_open03.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
testcases/kernel/syscalls/perf_event_open/perf_event_open03.c +8 -7
@@ -25,7 +25,7 @@const int iterations =12000000 ;static int fd =-1 ; -static int timeout; +static int runtime;static void setup (void ) {
@@ -43,12 +43,12 @@ static void setup(void)SAFE_FILE_SCANF (INTEL_PT_PATH ,"%d" , &ev.type); fd =perf_event_open (&ev,getpid (),-1 ,-1 ,0 ); - timeout =tst_timeout_remaining (); + runtime =tst_remaining_runtime (); }/* * Check how fast we can do the iterations after 5 seconds of runtime. -* If the rate is too small to complete for current timeout then +* If the rate is too small to complete for current runtime then * stop the test. */ static void check_progress (int i)
@@ -67,11 +67,11 @@ static void check_progress(int i)if (elapsed_ms >5000 ) { iter_per_ms = (float ) i / elapsed_ms;tst_res (TINFO ,"rate: %f iters/ms" , iter_per_ms); -tst_res (TINFO ,"needed rate for current test timeout: %f iters/ms" , - (float ) iterations / (timeout *1000 )); +tst_res (TINFO ,"needed rate for current test runtime: %f iters/ms" , + (float ) iterations / (runtime *1000 )); -if (iter_per_ms *1000 * (timeout -1 ) < iterations) -tst_brk (TCONF ,"System too slow to complete test in specified timeout" ); +if (iter_per_ms *1000 * (runtime -1 ) < iterations) +tst_brk (TCONF ,"System too slow to complete test in specified runtime" ); } }
@@ -108,6 +108,7 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .needs_root = 1, + .max_runtime = 300, .tags = (const struct tst_tag []) { {"linux-git" ,"7bdb157cdebb" }, {"CVE" ,"2020-25704" },
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 14/29 syscalls/readv01: Remove now useless timeout
In reply to Cyril Hrubis--- testcases/kernel/syscalls/readv/readv01.c | 1 - 1 file changed, 1 deletion(-)
testcases/kernel/syscalls/readv/readv01.c +0 -1
@@ -110,7 +110,6 @@ static struct tst_test test = { .test = test_readv, .tcnt = ARRAY_SIZE(testcase_list), .needs_tmpdir = 1, - .timeout = 15, .tags = (const struct tst_tag []) { {"linux-git" ,"19f18459330f" }, {}
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 15/29 syscalls/tgkill03: Remove now unused timeout
In reply to Cyril Hrubis--- testcases/kernel/syscalls/tgkill/tgkill03.c | 1 - 1 file changed, 1 deletion(-)
testcases/kernel/syscalls/tgkill/tgkill03.c +0 -1
@@ -117,5 +117,4 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .test = run, - .timeout = 20, };
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 16/29 syscalls/setsockopt09: Remove now useless timeout
In reply to Cyril Hrubis--- testcases/kernel/syscalls/setsockopt/setsockopt09.c | 1 - 1 file changed, 1 deletion(-)
testcases/kernel/syscalls/setsockopt/setsockopt09.c +0 -1
@@ -117,7 +117,6 @@ static struct tst_test test = { .test_all = run, .setup = setup, .cleanup = cleanup, - .timeout = 5, .taint_check = TST_TAINT_W | TST_TAINT_D, .needs_kconfigs = (const char *[]) {"CONFIG_USER_NS=y" ,
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 17/29 syscalls/userfaultfd01: Remove now useless timeout
In reply to Cyril Hrubis--- testcases/kernel/syscalls/userfaultfd/userfaultfd01.c | 1 - 1 file changed, 1 deletion(-)
testcases/kernel/syscalls/userfaultfd/userfaultfd01.c +0 -1
@@ -121,7 +121,6 @@ static void run(void)static struct tst_test test = { .test_all = run, .min_kver ="4.3" , - .timeout =20 };#else
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 18/29 syscalls/move_pages12: Convert to runtime
In reply to Cyril Hrubis--- testcases/kernel/syscalls/move_pages/move_pages12.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
testcases/kernel/syscalls/move_pages/move_pages12.c +2 -2
@@ -153,7 +153,6 @@ static void do_test(unsigned int n)void *ptr;pid_t cpid =-1 ;int status; -unsigned int twenty_percent = (tst_timeout_remaining () /5 ); addr =SAFE_MMAP (NULL , tcases[n].tpages * hpsz,PROT_READ |PROT_WRITE ,MAP_PRIVATE |MAP_ANONYMOUS |MAP_HUGETLB ,-1 ,0 );
@@ -198,7 +197,7 @@ static void do_test(unsigned int n)SAFE_MUNMAP (addr, tcases[n].tpages * hpsz); -if (tst_timeout_remaining () < twenty_percent) +if (!tst_remaining_runtime ())break ; }
@@ -341,6 +340,7 @@ static struct tst_test test = { .cleanup = cleanup, .test = do_test, .tcnt = ARRAY_SIZE(tcases), + .max_runtime = 240, .tags = (const struct tst_tag []) { {"linux-git" ,"e66f17ff7177" }, {"linux-git" ,"c9d398fa2378" },
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 19/29 syscalls/rt_sigqueueinfo01: Remove now useless timeout
In reply to Cyril Hrubis--- testcases/kernel/syscalls/rt_sigqueueinfo/rt_sigqueueinfo01.c | 1 - 1 file changed, 1 deletion(-)
testcases/kernel/syscalls/rt_sigqueueinfo/rt_sigqueueinfo01.c +0 -1
@@ -108,7 +108,6 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .needs_checkpoints = 1, - .timeout = 20, };#else
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 20/29 mem/mallocstress: Convert to runtime
In reply to Cyril Hrubis--- testcases/kernel/mem/mtest07/mallocstress.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
testcases/kernel/mem/mtest07/mallocstress.c +8 -3
@@ -62,7 +62,7 @@ static void my_yield(void)* 0: success * 1: failure */ -int allocate_free (int scheme) +int allocate_free (int scheme,int threadnum) {int loop;const int MAXPTRS =50 ;/* only 42 or so get used on 32 bit machine */
@@ -127,6 +127,11 @@ int allocate_free(int scheme) }my_yield (); + +if (!tst_remaining_runtime ()) { +tst_res (TINFO ,"Thread [%d]: Test runtime is over, exitting" , threadnum); +break ; + } }/* Success! */
@@ -141,7 +146,7 @@ void *alloc_mem(void *threadnum)TST_CHECKPOINT_WAIT (0 );/* thread N will use growth scheme N mod 4 */ - err =allocate_free (((uintptr_t )threadnum) %4 ); + err =allocate_free (((uintptr_t )threadnum) %4 , (uintptr_t )threadnum); tst_res(TINFO, "Thread [%d]: allocate_free() returned %d, %s. Thread exiting.\n", (int)(uintptr_t)threadnum, err,
@@ -189,7 +194,7 @@ static void cleanup(void) }static struct tst_test test = { - .timeout =600 , + .max_runtime =600 , .needs_checkpoints =1 , .setup = setup, .cleanup = cleanup,
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 21/29 mem/{oom, min_free_kbytes}: Convert to runtime
In reply to Cyril HrubisThis just sets the runtime to TST_UNLIMITED_RUNTIME which disables the timeout.
--- testcases/kernel/mem/oom/oom01.c | 2 +- testcases/kernel/mem/oom/oom02.c | 2 +- testcases/kernel/mem/oom/oom03.c | 2 +- testcases/kernel/mem/oom/oom04.c | 2 +- testcases/kernel/mem/oom/oom05.c | 2 +- testcases/kernel/mem/tunable/min_free_kbytes.c | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-)
testcases/kernel/mem/oom/oom01.c +1 -1
@@ -63,7 +63,7 @@ static void cleanup(void)static struct tst_test test = { .needs_root =1 , .forks_child =1 , - .timeout =-1 , + .max_runtime =TST_UNLIMITED_RUNTIME , .setup = setup, .cleanup = cleanup, .test_all = verify_oom,
testcases/kernel/mem/oom/oom02.c +1 -1
@@ -70,7 +70,7 @@ static void cleanup(void)static struct tst_test test = { .needs_root =1 , .forks_child =1 , - .timeout =-1 , + .max_runtime =TST_UNLIMITED_RUNTIME , .setup = setup, .cleanup = cleanup, .test_all = verify_oom,
testcases/kernel/mem/oom/oom03.c +1 -1
@@ -95,7 +95,7 @@ static void cleanup(void)static struct tst_test test = { .needs_root =1 , .forks_child =1 , - .timeout =-1 , + .max_runtime =TST_UNLIMITED_RUNTIME , .setup = setup, .cleanup = cleanup, .test_all = verify_oom,
testcases/kernel/mem/oom/oom04.c +1 -1
@@ -91,7 +91,7 @@ static void cleanup(void)static struct tst_test test = { .needs_root =1 , .forks_child =1 , - .timeout =-1 , + .max_runtime =TST_UNLIMITED_RUNTIME , .setup = setup, .cleanup = cleanup, .test_all = verify_oom,
testcases/kernel/mem/oom/oom05.c +1 -1
@@ -113,7 +113,7 @@ void cleanup(void)static struct tst_test test = { .needs_root =1 , .forks_child =1 , - .timeout =-1 , + .max_runtime =TST_UNLIMITED_RUNTIME , .setup = setup, .cleanup = cleanup, .test_all = verify_oom,
testcases/kernel/mem/tunable/min_free_kbytes.c +1 -1
@@ -231,7 +231,7 @@ static void cleanup(void)static struct tst_test test = { .needs_root =1 , .forks_child =1 , - .timeout =-1 , + .max_runtime =TST_UNLIMITED_RUNTIME , .setup = setup, .cleanup = cleanup, .test_all = min_free_kbytes_test,
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 22/29 fuzzy_sync: Convert to runtime
In reply to Cyril HrubisThe runtime for fuzzy sync tests are computed as timeout * exec_time_p. The default timeout was 300 and the default exec_time_p was 0.5.
--- include/tst_fuzzy_sync.h | 19 ++++--------------- lib/newlib_tests/tst_fuzzy_sync01.c | 1 + lib/newlib_tests/tst_fuzzy_sync02.c | 1 + lib/newlib_tests/tst_fuzzy_sync03.c | 1 + testcases/kernel/crypto/af_alg07.c | 1 + testcases/kernel/mem/thp/thp04.c | 1 + testcases/kernel/pty/pty03.c | 2 +- testcases/kernel/pty/pty05.c | 1 + testcases/kernel/pty/pty06.c | 1 + testcases/kernel/pty/pty07.c | 1 + testcases/kernel/sound/snd_seq01.c | 2 +- testcases/kernel/sound/snd_timer01.c | 1 + testcases/kernel/syscalls/bind/bind06.c | 2 +- testcases/kernel/syscalls/inotify/inotify09.c | 1 + .../kernel/syscalls/ipc/shmctl/shmctl05.c | 2 +- testcases/kernel/syscalls/sendmsg/sendmsg03.c | 1 + .../kernel/syscalls/setsockopt/setsockopt06.c | 2 +- .../kernel/syscalls/setsockopt/setsockopt07.c | 1 + .../syscalls/timerfd/timerfd_settime02.c | 1 + testcases/kernel/syscalls/writev/writev03.c | 2 +- 20 files changed, 23 insertions(+), 21 deletions(-)
include/tst_fuzzy_sync.h +4 -15
@@ -160,15 +160,6 @@ struct tst_fzsync_pair {int b_cntr;/** Internal; Used by tst_fzsync_pair_exit() and fzsync_pair_wait() */ int exit; -/** -* The maximum desired execution time as a proportion of the timeout -* -* A value x so that 0 < x < 1 which decides how long the test should -* be run for (assuming the loop limit is not exceeded first). -* -* Defaults to 0.5 (~150 seconds with default timeout). -*/ -float exec_time_p;/** Internal; The test time remaining on tst_fzsync_pair_reset() */ float exec_time_start;/**
@@ -214,7 +205,6 @@ static inline void tst_fzsync_pair_init(struct tst_fzsync_pair *pair)CHK (avg_alpha,0 ,1 ,0.25 );CHK (min_samples,20 ,INT_MAX ,1024 );CHK (max_dev_ratio,0 ,1 ,0.1 ); -CHK (exec_time_p,0 ,1 ,0.5 );CHK (exec_loops,20 ,INT_MAX ,3000000 );if (tst_ncpus_available () <=1 )
@@ -291,7 +281,7 @@ static inline void tst_fzsync_pair_reset(struct tst_fzsync_pair *pair,if (run_b)SAFE_PTHREAD_CREATE (&pair->thread_b,0 , run_b,0 ); - pair->exec_time_start = (float )tst_timeout_remaining (); + pair->exec_time_start = (float )tst_remaining_runtime (); }/**
@@ -644,10 +634,9 @@ static inline void tst_fzsync_wait_b(struct tst_fzsync_pair *pair)*/ static inline int tst_fzsync_run_a (struct tst_fzsync_pair *pair) { -float rem_p =1 -tst_timeout_remaining () / pair->exec_time_start; +float rem_p =1 -tst_remaining_runtime () / pair->exec_time_start; -if ((pair->exec_time_p *SAMPLING_SLICE < rem_p) - && (pair->sampling >0 )) { +if ((SAMPLING_SLICE < rem_p) && (pair->sampling >0 )) {tst_res (TINFO ,"Stopped sampling at %d (out of %d) samples, " "sampling time reached 50%% of the total time limit" , pair->exec_loop, pair->min_samples);
@@ -655,7 +644,7 @@ static inline int tst_fzsync_run_a(struct tst_fzsync_pair *pair)tst_fzsync_pair_info (pair); } -if (pair->exec_time_p < rem_p) { +if (rem_p >=1 ) {tst_res (TINFO ,"Exceeded execution time, requesting exit" );tst_atomic_store (1 , &pair->exit);
lib/newlib_tests/tst_fuzzy_sync01.c +1 -0
@@ -227,4 +227,5 @@ static struct tst_test test = { .test = run, .setup = setup, .cleanup = cleanup, + .max_runtime = 150, };
lib/newlib_tests/tst_fuzzy_sync02.c +1 -0
@@ -169,4 +169,5 @@ static struct tst_test test = { .test = run, .setup = setup, .cleanup = cleanup, + .max_runtime = 150, };
lib/newlib_tests/tst_fuzzy_sync03.c +1 -0
@@ -99,4 +99,5 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .test_all = run, + .max_runtime = 150, };
testcases/kernel/crypto/af_alg07.c +1 -0
@@ -125,6 +125,7 @@ static struct tst_test test = { .cleanup = cleanup, .min_kver = "4.10.0", .min_cpus = 2, + .max_runtime = 150, .taint_check = TST_TAINT_W | TST_TAINT_D, .tags = (const struct tst_tag []) { {"linux-git" ,"ff7b11aa481f" },
testcases/kernel/mem/thp/thp04.c +1 -0
@@ -163,6 +163,7 @@ static struct tst_test test = { .test_all = run, .setup = setup, .cleanup = cleanup, + .max_runtime = 150, .tags = (const struct tst_tag []) { {"linux-git" ,"a8f97366452e" }, {"linux-git" ,"8310d48b125d" },
testcases/kernel/pty/pty03.c +1 -1
@@ -135,7 +135,6 @@ static void do_test(unsigned int n)static void setup (void ) { fzp.min_samples =20 ; - fzp.exec_time_p =0.1 ;tst_fzsync_pair_init (&fzp); }
@@ -151,6 +150,7 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .needs_root = 1, + .max_runtime = 30, .tags = (const struct tst_tag []) { {"linux-git" ,"0ace17d568241" }, {"CVE" ,"2020-14416" },
testcases/kernel/pty/pty05.c +1 -0
@@ -97,6 +97,7 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .taint_check = TST_TAINT_W | TST_TAINT_D, + .max_runtime = 150, .tags = (const struct tst_tag []) { {"linux-git" ,"82f2341c94d27" }, {"CVE" ,"2017-2636" },
testcases/kernel/pty/pty06.c +1 -0
@@ -97,6 +97,7 @@ static struct tst_test test = { .cleanup = cleanup, .needs_root = 1, .taint_check = TST_TAINT_W | TST_TAINT_D, + .max_runtime = 150, .tags = (const struct tst_tag []) { {"linux-git" ,"ca4463bf8438" }, {}
testcases/kernel/pty/pty07.c +1 -0
@@ -110,6 +110,7 @@ static struct tst_test test = { .cleanup = cleanup, .needs_root = 1, .taint_check = TST_TAINT_W | TST_TAINT_D, + .max_runtime = 150, .tags = (const struct tst_tag []) { {"linux-git" ,"6cd1ed50efd8" }, {}
testcases/kernel/sound/snd_seq01.c +1 -1
@@ -123,7 +123,7 @@ static struct tst_test test = { .tcnt = ARRAY_SIZE(testfunc_list), .setup = setup, .cleanup = cleanup, - .timeout = 120, + .max_runtime = 60, .taint_check = TST_TAINT_W | TST_TAINT_D, .tags = (const struct tst_tag []) { {"linux-git" ,"d15d662e89fc" },
testcases/kernel/sound/snd_timer01.c +1 -0
@@ -139,6 +139,7 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .taint_check = TST_TAINT_W | TST_TAINT_D, + .max_runtime = 150, .tags = (const struct tst_tag []) { {"linux-git" ,"d11662f4f798" }, {"linux-git" ,"ba3021b2c79b" },
testcases/kernel/syscalls/bind/bind06.c +1 -1
@@ -102,7 +102,7 @@ static struct tst_test test = { .test_all = run, .setup = setup, .cleanup = cleanup, - .timeout = 600, + .max_runtime = 300, .taint_check = TST_TAINT_W | TST_TAINT_D, .needs_kconfigs = (const char *[]) {"CONFIG_USER_NS=y" ,
testcases/kernel/syscalls/inotify/inotify09.c +1 -0
@@ -94,6 +94,7 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .test_all = verify_inotify, + .max_runtime = 150, .tags = (const struct tst_tag []) { {"linux-git" ,"d90a10e2444b" }, {}
testcases/kernel/syscalls/ipc/shmctl/shmctl05.c +1 -1
@@ -106,7 +106,7 @@ static void cleanup(void) }static struct tst_test test = { - .timeout =20 , + .max_runtime =10 , .setup = setup, .test_all = do_test, .cleanup = cleanup,
testcases/kernel/syscalls/sendmsg/sendmsg03.c +1 -0
@@ -105,6 +105,7 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .taint_check = TST_TAINT_W | TST_TAINT_D, + .max_runtime = 150, .tags = (const struct tst_tag []) { {"linux-git" ,"8f659a03a0ba" }, {"CVE" ,"2017-17712" },
testcases/kernel/syscalls/setsockopt/setsockopt06.c +1 -1
@@ -46,7 +46,6 @@ static void setup(void)SAFE_FILE_PRINTF ("/proc/self/gid_map" ,"0 %d 1" , real_gid); fzsync_pair.exec_loops =100000 ; - fzsync_pair.exec_time_p =0.9 ;tst_fzsync_pair_init (&fzsync_pair); }
@@ -123,6 +122,7 @@ static struct tst_test test = { .test_all = run, .setup = setup, .cleanup = cleanup, + .max_runtime = 270, .taint_check = TST_TAINT_W | TST_TAINT_D, .needs_kconfigs = (const char *[]) {"CONFIG_USER_NS=y" ,
testcases/kernel/syscalls/setsockopt/setsockopt07.c +1 -0
@@ -136,6 +136,7 @@ static struct tst_test test = { .test_all = run, .setup = setup, .cleanup = cleanup, + .max_runtime = 150, .needs_kconfigs = (const char *[]) {"CONFIG_USER_NS=y" ,"CONFIG_NET_NS=y" ,
testcases/kernel/syscalls/timerfd/timerfd_settime02.c +1 -0
@@ -112,6 +112,7 @@ static struct tst_test test = { .cleanup = cleanup, .min_kver = "2.6.25", .taint_check = TST_TAINT_W | TST_TAINT_D, + .max_runtime = 150, .tags = (const struct tst_tag []) { {"linux-git" ,"1e38da300e1e" }, {"CVE" ,"2017-10661" },
testcases/kernel/syscalls/writev/writev03.c +1 -1
@@ -49,7 +49,6 @@ static void setup(void) mapfd =SAFE_OPEN (MAPFILE ,O_CREAT |O_RDWR |O_TRUNC ,0644 );SAFE_WRITE (1 , mapfd, buf,BUF_SIZE ); - fzsync_pair.exec_time_p =0.25 ;tst_fzsync_pair_init (&fzsync_pair); }
@@ -146,6 +145,7 @@ static struct tst_test test = { .min_cpus = 2, .setup = setup, .cleanup = cleanup, + .max_runtime = 75, .tags = (const struct tst_tag []) { {"linux-git" ,"d4690f1e1cda" }, {}
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Quoted 60 lines · Cyril Hrubis
include/tst_fuzzy_sync.h +0
-0@@ -160,15 +160,6 @@ struct tst_fzsync_pair {int b_cntr;/** Internal; Used by tst_fzsync_pair_exit() and
fzsync_pair_wait()*/int exit;include/tst_fuzzy_sync.h +0
-2-/** -* The maximum desired execution time as a proportion of the timeout
include/tst_fuzzy_sync.h +0
-2-* -* A value x so that 0 < x < 1 which decides how long the test should
include/tst_fuzzy_sync.h +0
-6-* be run for (assuming the loop limit is not exceeded first). -* -* Defaults to 0.5 (~150 seconds with default timeout). -*/ -float exec_time_p;/** Internal; The test time remaining on tst_fzsync_pair_reset() */ float exec_time_start;/** @@ -214,7 +205,6 @@ static inline void tst_fzsync_pair_init(struct tst_fzsync_pair *pair) CHK(avg_alpha, 0, 1, 0.25);CHK (min_samples,20 ,INT_MAX ,1024 );CHK (max_dev_ratio,0 ,1 ,0.1 ); -CHK (exec_time_p,0 ,1 ,0.5 );CHK (exec_loops,20 ,INT_MAX ,3000000 );if (tst_ncpus_available() <= 1)include/tst_fuzzy_sync.h +1
-1@@ -291,7 +281,7 @@ static inline void tst_fzsync_pair_reset(struct tst_fzsync_pair *pair, if (run_b) SAFE_PTHREAD_CREATE(&pair->thread_b, 0, run_b, 0); - pair->exec_time_start = (float )tst_timeout_remaining (); + pair->exec_time_start = (float )tst_remaining_runtime (); }/**
include/tst_fuzzy_sync.h +2
-3@@ -644,10 +634,9 @@ static inline void tst_fzsync_wait_b(structtst_fzsync_pair *pair) */ static inline int tst_fzsync_run_a (struct tst_fzsync_pair *pair) { -float rem_p =1 -tst_timeout_remaining () / pair->exec_time_start; +float rem_p =1 -tst_remaining_runtime () / pair->exec_time_start; -if ((pair->exec_time_p *SAMPLING_SLICE < rem_p) - && (pair->sampling >0 )) { +if ((SAMPLING_SLICE < rem_p) && (pair->sampling >0 )) { tst_res(TINFO, "Stopped sampling at %d (out of %d)samples, "
"sampling time reached 50%% of the total time
limit",pair->exec_loop, pair->min_samples);
include/tst_fuzzy_sync.h static inline int tst_fzsync_run_a(struct · line 649
Quoted 7 lines
include/tst_fuzzy_sync.h +1
-1@@ -655,7 +644,7 @@ static inline int tst_fzsync_run_a(struct tst_fzsync_pair *pair)tst_fzsync_pair_info (pair ); } -if (pair->exec_time_p < rem_p) { +if (rem_p >=1 ) {
I hit a new problem while testing new pty03, that seems here will fall into an infinite loop and test timed out finally. The printf shows rem_p will be overflow I haven't figured out why.
But with comparing with 0.9, it always gets passed on to the same system.
include/tst_fuzzy_sync.h +0 -0
@@ -644,7 +644,7 @@ static inline int tst_fzsync_run_a(struct
tst_fzsync_pair *pair)
tst_fzsync_pair_info(pair);
}- if (rem_p >= 1) {
+ if (rem_p >= 0.9) {
tst_res(TINFO,
"Exceeded execution time, requesting exit");
tst_atomic_store(1, &pair->exit);----------------------
# ./pty03
...
../../../include/tst_fuzzy_sync.h:295: TINFO: start_a - start_b: { avg =
-90ns, avg_dev = 8ns, dev_ratio = 0.09 }
../../../include/tst_fuzzy_sync.h:295: TINFO: end_a - start_a : { avg =
2606ns, avg_dev = 127ns, dev_ratio = 0.05 }
../../../include/tst_fuzzy_sync.h:295: TINFO: end_b - start_b : { avg =
8785ns, avg_dev = 600ns, dev_ratio = 0.07 }
../../../include/tst_fuzzy_sync.h:295: TINFO: end_a - end_b : { avg =
-6269ns, avg_dev = 566ns, dev_ratio = 0.09 }
../../../include/tst_fuzzy_sync.h:295: TINFO: spins : { avg =
3269 , avg_dev = 200 , dev_ratio = 0.06 }
Test timeouted, sending SIGKILL!
tst_test.c:1573: TINFO: If you are running on slow machine, try exporting
LTP_TIMEOUT_MUL > 1
tst_test.c:1575: TBROK: Test killed! (timeout?)Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Hi!
Quoted 71 lines
Quoted 59 lines
include/tst_fuzzy_sync.h +0
-0@@ -160,15 +160,6 @@ struct tst_fzsync_pair {int b_cntr;/** Internal; Used by tst_fzsync_pair_exit() and
fzsync_pair_wait()*/int exit;include/tst_fuzzy_sync.h +0
-2-/** -* The maximum desired execution time as a proportion of the timeout
include/tst_fuzzy_sync.h +0
-2-* -* A value x so that 0 < x < 1 which decides how long the test should
include/tst_fuzzy_sync.h +0
-6-* be run for (assuming the loop limit is not exceeded first). -* -* Defaults to 0.5 (~150 seconds with default timeout). -*/ -float exec_time_p;/** Internal; The test time remaining on tst_fzsync_pair_reset() */ float exec_time_start;/** @@ -214,7 +205,6 @@ static inline void tst_fzsync_pair_init(struct tst_fzsync_pair *pair) CHK(avg_alpha, 0, 1, 0.25);CHK (min_samples,20 ,INT_MAX ,1024 );CHK (max_dev_ratio,0 ,1 ,0.1 ); -CHK (exec_time_p,0 ,1 ,0.5 );CHK (exec_loops,20 ,INT_MAX ,3000000 );if (tst_ncpus_available() <= 1)include/tst_fuzzy_sync.h +1
-1@@ -291,7 +281,7 @@ static inline void tst_fzsync_pair_reset(struct tst_fzsync_pair *pair, if (run_b) SAFE_PTHREAD_CREATE(&pair->thread_b, 0, run_b, 0); - pair->exec_time_start = (float )tst_timeout_remaining (); + pair->exec_time_start = (float )tst_remaining_runtime (); }/**
include/tst_fuzzy_sync.h +2
-3@@ -644,10 +634,9 @@ static inline void tst_fzsync_wait_b(structtst_fzsync_pair *pair) */ static inline int tst_fzsync_run_a (struct tst_fzsync_pair *pair) { -float rem_p =1 -tst_timeout_remaining () / pair->exec_time_start; +float rem_p =1 -tst_remaining_runtime () / pair->exec_time_start; -if ((pair->exec_time_p *SAMPLING_SLICE < rem_p) - && (pair->sampling >0 )) { +if ((SAMPLING_SLICE < rem_p) && (pair->sampling >0 )) { tst_res(TINFO, "Stopped sampling at %d (out of %d)samples, "
"sampling time reached 50%% of the total time
limit",pair->exec_loop, pair->min_samples);include/tst_fuzzy_sync.h static inline int tst_fzsync_run_a(struct · line 649
Quoted 7 lines
include/tst_fuzzy_sync.h +1
-1@@ -655,7 +644,7 @@ static inline int tst_fzsync_run_a(struct tst_fzsync_pair *pair)tst_fzsync_pair_info (pair ); } -if (pair->exec_time_p < rem_p) { +if (rem_p >=1 ) {I hit a new problem while testing new pty03, that seems here will fall into an infinite loop and test timed out finally. The printf shows
rem_pwill be overflow I haven't figured out why.
Quoted 1 line
But with comparing with 0.9, it always gets passed on to the same system.
That is strange, since we do:
rem_p = 1 - tst_remaining_runtime()/pair->time_exec_start;
And the tst_remaining_runtime() should return 0 once the time is up so the end result should be that rem_p will end up 1 sooner or later.
Anyways we can as well use the value from tst_remainig_runtime() directly instead as:
include/tst_fuzzy_sync.h +3 -2
@@ -634,7 +634,8 @@ static inline void tst_fzsync_wait_b(struct tst_fzsync_pair *pair)*/ static inline int tst_fzsync_run_a (struct tst_fzsync_pair *pair) { -float rem_p =1 -tst_remaining_runtime () / pair->exec_time_start; +int remaining_runtime =tst_remaining_runtime (); +float rem_p =1 - remaining_runtime / pair->exec_time_start;if ((SAMPLING_SLICE < rem_p) && (pair->sampling >0 )) { tst_res(TINFO, "Stopped sampling at %d (out of %d) samples, "
@@ -644,7 +645,7 @@ static inline int tst_fzsync_run_a(struct tst_fzsync_pair *pair)tst_fzsync_pair_info (pair); } -if (rem_p >=1 ) { +if (!remaining_runtime) {tst_res (TINFO ,"Exceeded execution time, requesting exit" );tst_atomic_store (1 , &pair->exit);
Does that fix your problem?
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Quoted 8 lines · Cyril Hrubis
Quoted 8 lines
Review
Quoted 3 lines
(quoted diff) +1
-1- if (pair->exec_time_p < rem_p) { + if (rem_p >= 1) {I hit a new problem while testing new pty03, that seems here will fall into an infinite loop and test timed out finally. The printf shows
rem_pwill be overflow I haven't figured out why.
Quoted 6 lines
Quoted 1 line
But with comparing with 0.9, it always gets passed on to the same system.
That is strange, since we do:
rem_p = 1 - tst_remaining_runtime()/pair->time_exec_start;
I guess the root cause is that 'pair->time_exec_start' has a possibility to reach zero. in pty03 it has ".tcnt = 9" which made the tst_fzsync_pair_reset()
to be re-run many times, but in that function 'pair->time_exec_start' will be set only based on the original .max_runtime, with time elapsed the remaining
time tends to be zero.
debug log
--------------../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is 0.000000 ../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is 0.000000 ../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is 1.000000 ../../../include/tst_fuzzy_sync.h:654: TINFO: Exceeded execution time, requesting exit
pty03.c:132: TPASS: Did not crash with PPS TTY discipline pty03.c:106: TINFO: Creating PTY with CAIF line discipline ../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is -nan pty03.c:91: TCONF: You don't appear to have the CAIF TTY line discipline:
EINVAL (22)
pty03.c:106: TINFO: Creating PTY with GSM line discipline ../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is -nan ../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is -nan ../../../include/tst_fuzzy_sync.h:651: TINFO: rem_p is -nan
Quoted 5 lines
And the
tst_remaining_runtime()should return 0 once the time is up so the end result should be thatrem_pwill end up 1 sooner or later.
Yes, the pair->time_exec_start will be reset to 0 as well then becomes the denominator. That makes rem_p get an overflowed negative number, then the test falls into an infinite loop from this moment.
Quoted 33 lines
Anyways we can as well use the value from
tst_remainig_runtime()directly instead as:include/tst_fuzzy_sync.h +2
-1@@ -634,7 +634,8 @@ static inline void tst_fzsync_wait_b(structtst_fzsync_pair *pair) */ static inline int tst_fzsync_run_a (struct tst_fzsync_pair *pair) { -float rem_p =1 -tst_remaining_runtime () / pair->exec_time_start; +int remaining_runtime =tst_remaining_runtime (); +float rem_p =1 - remaining_runtime / pair->exec_time_start;if ((SAMPLING_SLICE < rem_p) && (pair->sampling >0 )) {tst_res(TINFO, "Stopped sampling at %d (out of %d)
samples, "include/tst_fuzzy_sync.h +1
-1@@ -644,7 +645,7 @@ static inline int tst_fzsync_run_a(struct tst_fzsync_pair *pair)tst_fzsync_pair_info (pair ); } -if (rem_p >=1 ) { +if (!remaining_runtime) {tst_res (TINFO ,"Exceeded execution time, requesting exit" );tst_atomic_store(1, &pair->exit);
Quoted 2 lines
Does that fix your problem?
This amendment is more readable but does not solve the original problem.
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Hi!
Quoted 11 lines
Quoted 10 lines
Quoted 5 lines
I hit a new problem while testing new pty03, that seems here will fall into an infinite loop and test timed out finally. The printf shows
rem_pwill be overflow I haven't figured out why.But with comparing with 0.9, it always gets passed on to the same system.
That is strange, since we do:
rem_p = 1 - tst_remaining_runtime()/pair->time_exec_start;
Quoted 6 lines
I guess the root cause is that '
pair->time_exec_start' has a possibility to reach zero. in pty03 it has ".tcnt = 9" which made thetst_fzsync_pair_reset()
to be re-run many times, but in that function 'pair->time_exec_start' will be set only based on the original .max_runtime, with time elapsed the remaining time tends to be zero.
I guess that that the interaction of tcnt and runtime is not optimal here. You are right that as long as we call tst_fzsync_pair_reset() on each invocation of the run() function we may eventually get to state where the runtime is exhausted, especially on slower hardware we end up with division by zero and overflow.
The cleanest solution would be to rewrite the test to use .test_variants = 9 and setting the .max_runtime to a smaller value. That way we would have precisely defined runtime for each iteration. What do you think?
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Quoted 28 lines · Cyril Hrubis
Hi!
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I hit a new problem while testing new pty03, that seems here will fall into an infinite loop and test timed out finally. The
printf
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
shows
rem_pwill be overflow I haven't figured out why.But with comparing with 0.9, it always gets passed on to the same
system.
Quoted 10 lines
Quoted 5 lines
That is strange, since we do:
rem_p = 1 - tst_remaining_runtime()/pair->time_exec_start;I guess the root cause is that '
pair->time_exec_start' has a possibility to reach zero. in pty03 it has ".tcnt = 9" which made thetst_fzsync_pair_reset()
to be re-run many times, but in that function 'pair->time_exec_start'will
Quoted 2 lines
be set only based on the original .max_runtime, with time elapsed the remaining time tends to be zero.
I guess that that the interaction of tcnt and runtime is not optimal here. You are right that as long as we call
tst_fzsync_pair_reset()on each invocation of therun()function we may eventually get to state where the runtime is exhausted, especially on slower hardware we end up with division by zero and overflow.
Quoted 5 lines
The cleanest solution would be to rewrite the test to use .test_variants = 9
and setting the .max_runtime to a smaller value. That way we would have precisely defined runtime for each iteration. What do you think?
Yes, that should work, but people have to painstakingly remember that to avoid this interaction of tcnt and runtime in test writing.
Another simple choice I can think of is to use tst_set_max_runtime() to reset both 'timeout' and 'runtime' at the beginning of each invocation of the run().
And maybe we should invoke it in the tst_fzsync_pair_reset directly? I'm not very sure about fixing this in fzsync-lib or out.
testcases/kernel/pty/pty03.c +1 -0
@@ -105,6 +105,7 @@ static void do_test(unsigned int n)tst_res (TINFO ,"Creating PTY with %s line discipline" , ldisc->name); +tst_set_max_runtime (test.runtime);tst_fzsync_pair_reset (&fzp, hangup);while (tst_fzsync_run_a (&fzp)) { ptmx =SAFE_OPEN ("/dev/ptmx" ,O_RDONLY );
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
In reply to Cyril HrubisHello,
Quoted 25 lines · Cyril Hrubis
Hi!
Quoted 11 lines
Quoted 10 lines
Quoted 5 lines
I hit a new problem while testing new pty03, that seems here will fall into an infinite loop and test timed out finally. The printf shows
rem_pwill be overflow I haven't figured out why.But with comparing with 0.9, it always gets passed on to the same system.
That is strange, since we do:
rem_p = 1 - tst_remaining_runtime()/pair->time_exec_start;Quoted 6 lines
I guess the root cause is that '
pair->time_exec_start' has a possibility to reach zero. in pty03 it has ".tcnt = 9" which made thetst_fzsync_pair_reset()
to be re-run many times, but in that function 'pair->time_exec_start' will be set only based on the original .max_runtime, with time elapsed the remaining time tends to be zero.I guess that that the interaction of tcnt and runtime is not optimal here. You are right that as long as we call
tst_fzsync_pair_reset()on each invocation of therun()function we may eventually get to state where the runtime is exhausted, especially on slower hardware we end up with division by zero and overflow.
Quoted 3 lines
The cleanest solution would be to rewrite the test to use .test_variants = 9 and setting the .max_runtime to a smaller value. That way we would have precisely defined runtime for each iteration. What do you think?
Or each test case (defined by tcnt) could be given an equal share of the runtime?
Signature · 5 lines
Thank you, Richard. -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Hi!
Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
I guess the root cause is that '
pair->time_exec_start' has a possibility to reach zero. in pty03 it has ".tcnt = 9" which made thetst_fzsync_pair_reset()
to be re-run many times, but in that function 'pair->time_exec_start' will be set only based on the original .max_runtime, with time elapsed the remaining time tends to be zero.I guess that that the interaction of tcnt and runtime is not optimal here. You are right that as long as we call
tst_fzsync_pair_reset()on each invocation of therun()function we may eventually get to state where the runtime is exhausted, especially on slower hardware we end up with division by zero and overflow.
Quoted 6 lines
Quoted 3 lines
The cleanest solution would be to rewrite the test to use .test_variants = 9 and setting the .max_runtime to a smaller value. That way we would have precisely defined runtime for each iteration. What do you think?
Or each test case (defined by tcnt) could be given an equal share of the runtime?
That would be solution as well. If everyone agrees on this I will go with this version, it should be as easy as:
lib/tst_test.c +2 -0
@@ -1349,6 +1349,8 @@ static void run_tests(void)if (results_equal (&saved_results, results))tst_brk (TBROK ,"Test %i haven't reported results!" , i); + +kill (getppid (),SIGUSR1 ); } }
With this we do reset the test timeout after each test iteration.
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [Automated-testing] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Hi all,
Quoted 20 lines
Hi!
Quoted 19 lines
Quoted 13 lines
Quoted 6 lines
I guess the root cause is that '
pair->time_exec_start' has a possibility to reach zero. in pty03 it has ".tcnt = 9" which made thetst_fzsync_pair_reset()
to be re-run many times, but in that function 'pair->time_exec_start' will be set only based on the original .max_runtime, with time elapsed the remaining time tends to be zero.I guess that that the interaction of tcnt and runtime is not optimal here. You are right that as long as we call
tst_fzsync_pair_reset()on each invocation of therun()function we may eventually get to state where the runtime is exhausted, especially on slower hardware we end up with division by zero and overflow.Quoted 3 lines
The cleanest solution would be to rewrite the test to use .test_variants = 9 and setting the .max_runtime to a smaller value. That way we would have precisely defined runtime for each iteration. What do you think?
Or each test case (defined by tcnt) could be given an equal share of the runtime?
LGTM
Quoted 17 lines
That would be solution as well. If everyone agrees on this I will go with this version, it should be as easy as:
lib/tst_test.c +2
-0@@ -1349,6 +1349,8 @@ static void run_tests(void)if (results_equal (&saved_results, results))tst_brk (TBROK ,"Test %i haven't reported results!" , i); + +kill (getppid (),SIGUSR1 ); } }
Quoted 1 line
With this we do reset the test timeout after each test iteration.
+1.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
In reply to Cyril HrubisQuoted 12 lines · Cyril Hrubis
Quoted 3 lines
Or each test case (defined by tcnt) could be given an equal share of the runtime?
That would be solution as well. If everyone agrees on this I will go with this version, it should be as easy as:
lib/tst_test.c +0
-0
lib/tst_test.c static void run_tests(void) · line 1355
Quoted 7 lines
lib/tst_test.c +2
-0@@ -1349,6 +1349,8 @@ static void run_tests(void)if (results_equal (&saved_results, results))tst_brk (TBROK ,"Test %i haven't reported results!", i); + +kill (getppid (),SIGUSR1 );
I have to say NO here.
This is only reset timeout for each test, the problem we're encountered is the max_runtime exhausting, we have to reset 'results->max_runtime' but not .timeout.
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Quoted 25 lines · Li Wang
Quoted 12 lines · Cyril Hrubis
Quoted 3 lines
Or each test case (defined by tcnt) could be given an equal share of the runtime?
That would be solution as well. If everyone agrees on this I will go with this version, it should be as easy as:
lib/tst_test.c +0
-0lib/tst_test.c static void run_tests(void) · line 1355
Quoted 7 lines
lib/tst_test.c +2
-0@@ -1349,6 +1349,8 @@ static void run_tests(void)if (results_equal (&saved_results, results))tst_brk (TBROK ,"Test %i haven't reported results!", i); + +kill (getppid (),SIGUSR1 );I have to say NO here.
Quoted 4 lines
This is only reset timeout for each test, the problem we're encountered is the
max_runtimeexhausting, we have to reset 'results->max_runtime' but not .timeout.
Okay, my perspective is shortsighted as well.
This solution is correct in the direction but overlooks the significant global value 'tst_start_time'. If that value reflush within tcnt loop we don't need to reset max_runtime again, actually the real work in my previous patch is to invoke heartbeat() which touches tst_start_time.
So I would suggest using heartbeat() instead of only sending SIGUSR1 to lib_pid. Or, do simply revision like:
lib/tst_test.c +0 -0
@@ -1349,6 +1349,10 @@ static void run_tests(void)if (results_equal (&saved_results, results)) tst_brk(TBROK, "Test %i haven't reported results!",
i);
+
+ if (tst_clock_gettime(CLOCK_MONOTONIC, &tst_start_time))
+ tst_res(TWARN | TERRNO, "tst_clock_gettime()
failed");
+ kill(getppid(), SIGUSR1);
}}
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Hi!
Quoted 9 lines
Okay, my perspective is shortsighted as well.
This solution is correct in the direction but overlooks the significant global value '
tst_start_time'. If that value reflush within tcnt loop we don't need to resetmax_runtimeagain, actually the real work in my previous patch is to invokeheartbeat()which touchestst_start_time.So I would suggest using
heartbeat()instead of only sending SIGUSR1 tolib_pid. Or, do simply revision like:
Ah, right, I guess that we should just call heartbeat() before each iteration of the test then.
So basically:
lib/tst_test.c +22 -19
@@ -1317,6 +1317,24 @@ static void do_cleanup(void)cleanup_ipc (); } +static void heartbeat (void ) +{ +if (tst_clock_gettime (CLOCK_MONOTONIC , &tst_start_time)) +tst_res (TWARN |TERRNO ,"tst_clock_gettime() failed" ); + +if (getppid () ==1 ) { +tst_res (TFAIL ,"Main test process might have exit!" ); +/* +* We need kill the task group immediately since the +* main process has exit. +*/ +kill (0 ,SIGKILL ); +exit (TBROK ); + } + +kill (getppid (),SIGUSR1 ); +} +static void run_tests (void ) {unsigned int i;
@@ -1324,6 +1342,7 @@ static void run_tests(void)if (!tst_test->test) { saved_results = *results; +heartbeat (); tst_test->test_all ();if (getpid () != main_pid) {
@@ -1339,6 +1358,7 @@ static void run_tests(void)for (i =0 ; i < tst_test->tcnt; i++) { saved_results = *results; +heartbeat (); tst_test->test (i);if (getpid () != main_pid) {
@@ -1349,6 +1369,8 @@ static void run_tests(void)if (results_equal (&saved_results, results))tst_brk (TBROK ,"Test %i haven't reported results!" , i); + +kill (getppid (),SIGUSR1 ); } }
@@ -1379,24 +1401,6 @@ static void add_paths(void)free (new_path); } -static void heartbeat (void ) -{ -if (tst_clock_gettime (CLOCK_MONOTONIC , &tst_start_time)) -tst_res (TWARN |TERRNO ,"tst_clock_gettime() failed" ); - -if (getppid () ==1 ) { -tst_res (TFAIL ,"Main test process might have exit!" ); -/* -* We need kill the task group immediately since the -* main process has exit. -*/ -kill (0 ,SIGKILL ); -exit (TBROK ); - } - -kill (getppid (),SIGUSR1 ); -} -static void testrun (void ) {unsigned int i =0 ;
@@ -1425,7 +1429,6 @@ static void testrun(void)break ;run_tests (); -heartbeat (); }do_test_cleanup ();
I guess that this should go in a separate patch on the top of the "Introduce concept of max runtime".
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Quoted 62 lines · Cyril Hrubis
Hi!
Quoted 9 lines
Okay, my perspective is shortsighted as well.
This solution is correct in the direction but overlooks the significant global value '
tst_start_time'. If that value reflush within tcnt loop we don't need to resetmax_runtimeagain, actually the real work in my previous patch is to invokeheartbeat()which touchestst_start_time.So I would suggest using
heartbeat()instead of only sending SIGUSR1 tolib_pid. Or, do simply revision like:Ah, right, I guess that we should just call
heartbeat()before each iteration of the test then.So basically:
lib/tst_test.c +20
-0@@ -1317,6 +1317,24 @@ static void do_cleanup(void)cleanup_ipc (); } +static void heartbeat (void ) +{ +if (tst_clock_gettime (CLOCK_MONOTONIC , &tst_start_time)) +tst_res (TWARN |TERRNO ,"tst_clock_gettime() failed" ); + +if (getppid () ==1 ) { +tst_res (TFAIL ,"Main test process might have exit!" ); +/* +* We need kill the task group immediately since the +* main process has exit. +*/ +kill (0 ,SIGKILL ); +exit (TBROK ); + } + +kill (getppid (),SIGUSR1 ); +} +static void run_tests (void ) {unsigned int i;@@ -1324,6 +1342,7 @@ static void run_tests(void)if (!tst_test->test) { saved_results = *results; +heartbeat (); tst_test->test_all ();if (getpid () != main_pid) {@@ -1339,6 +1358,7 @@ static void run_tests(void)for (i =0 ; i < tst_test->tcnt; i++) { saved_results = *results; +heartbeat (); tst_test->test (i);if (getpid () != main_pid) {
lib/tst_test.c static void run_tests(void) · line 1375
Quoted 7 lines
lib/tst_test.c +2
-0@@ -1349,6 +1369,8 @@ static void run_tests(void)if (results_equal (&saved_results, results))tst_brk (TBROK ,"Test %i haven't reported results!", i); + +kill (getppid (),SIGUSR1 );
As we already invoke heartbeat() at the beginning of each tcnt loop, why are we still sending SIGUSR1 here?
Otherwise this version looks good to me.
Quoted 38 lines
lib/tst_test.c +0
-0}}
lib/tst_test.c +0
-19@@ -1379,24 +1401,6 @@ static void add_paths(void)free (new_path); } -static void heartbeat (void ) -{ -if (tst_clock_gettime (CLOCK_MONOTONIC , &tst_start_time)) -tst_res (TWARN |TERRNO ,"tst_clock_gettime() failed" ); - -if (getppid () ==1 ) { -tst_res (TFAIL ,"Main test process might have exit!" ); -/* -* We need kill the task group immediately since the -* main process has exit. -*/ -kill (0 ,SIGKILL ); -exit (TBROK ); - } - -kill (getppid (),SIGUSR1 ); -} -static void testrun (void ) {unsigned int i =0 ;@@ -1425,7 +1429,6 @@ static void testrun(void)break ;run_tests (); -heartbeat (); }do_test_cleanup ();
Quoted 3 lines
I guess that this should go in a separate patch on the top of the "Introduce concept of max runtime".
Agree.
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Hi!
Quoted 6 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ + kill(getppid(), SIGUSR1);As we already invoke
heartbeat()at the beginning of each tcnt loop, why are we still sending SIGUSR1 here?
My bad, I did forget to remove it.
Quoted 5 lines
Quoted 3 lines
I guess that this should go in a separate patch on the top of the "Introduce concept of max runtime".
Agree.
Should I send a V4 with this patch included?
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Quoted 3 lines · Cyril Hrubis
Should I send a V4 with this patch included?
No need. You can push with this new patch adding, then I will kick start release testing for the whole LTP from our side.
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Hi!
Quoted 3 lines
No need. You can push with this new patch adding, then I will kick start release testing for the whole LTP from our side.
Great, please do so.
Andrea is going to send small fix to the aiodio tests that should be merged as well, otherwise these may fail rarely.
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Richard Palethorpe
Reviewed-byRe: [LTP] [PATCH v3 22/29] fuzzy_sync: Convert to runtime
Hello,
Quoted 9 lines · Cyril Hrubis
Hi!
Quoted 3 lines
No need. You can push with this new patch adding, then I will kick start release testing for the whole LTP from our side.
Great, please do so.
Andrea is going to send small fix to the aiodio tests that should be merged as well, otherwise these may fail rarely.
With these changes please add:
Signature · 5 lines
Thank you, Richard. -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 23/29 ltp-aiodio/dio_sparse, aiodio_sparse: Convert to runtime.
In reply to Cyril HrubisChange the code so that the test runtime is capped at the maximal runtime allowed for the test.
For this to work we also have to:
- check the run_child in the io_read() in the inner loop otherwise the test will timeout while waiting for the children to finish
- run the AIO writer in a separate process and kill it when we are out of runtime in the aiodio_sparse
--- .../kernel/io/ltp-aiodio/aiodio_sparse.c | 26 ++++++++++++++++--- testcases/kernel/io/ltp-aiodio/common.h | 6 ++++- testcases/kernel/io/ltp-aiodio/dio_sparse.c | 6 ++++- 3 files changed, 32 insertions(+), 6 deletions(-)
testcases/kernel/io/ltp-aiodio/aiodio_sparse.c +22 -4
@@ -189,7 +189,7 @@ static void run(void) {char *filename ="file.bin" ;int status; -int i; +int i, pid; *run_child =1 ;
@@ -200,9 +200,27 @@ static void run(void) } } -tst_res (TINFO ,"Parent create a sparse file" ); + pid =SAFE_FORK (); +if (!pid) { +aiodio_sparse (filename, alignment, writesize, filesize, numaio); +return ; + } + +tst_res (TINFO ,"Child %i creates a sparse file" , pid); + +for (;;) { +if (SAFE_WAITPID (pid,NULL ,WNOHANG )) +break ; -aiodio_sparse (filename, alignment, writesize, filesize, numaio); +sleep (1 ); + +if (!tst_remaining_runtime ()) { +tst_res (TINFO ,"Test out of runtime, exitting" ); +kill (pid,SIGKILL ); +SAFE_WAITPID (pid,NULL ,0 ); +break ; + } + }if (SAFE_WAITPID (-1 , &status,WNOHANG ))tst_res (TFAIL ,"Non zero bytes read" );
@@ -229,7 +247,7 @@ static struct tst_test test = {"tmpfs" ,NULL }, - .timeout = 1800, + .max_runtime = 1800, };#else TST_TEST_TCONF ("test requires libaio and its development packages" );
testcases/kernel/io/ltp-aiodio/common.h +5 -1
@@ -63,7 +63,7 @@ static inline void io_read(const char *filename, int filesize, volatile int *runtst_res (TINFO ,"child %i reading file" ,getpid ()); -while (*run_child) { +for (;;) {off_t offset =0 ;char *bufoff;
@@ -80,9 +80,13 @@ static inline void io_read(const char *filename, int filesize, volatile int *run } offset += r; } + +if (!*run_child) + goto exit; } } +exit:SAFE_CLOSE (fd); }
testcases/kernel/io/ltp-aiodio/dio_sparse.c +5 -1
@@ -51,6 +51,10 @@ static void dio_sparse(int fd, int align, long long fs, int ws, long long off)SAFE_LSEEK (fd, off,SEEK_SET );for (i = off; i < fs;) { +if (!tst_remaining_runtime ()) { +tst_res (TINFO ,"Test runtime is over, exitting" ); +return ; + } w =SAFE_WRITE (0 , fd, bufptr, ws); i += w; }
@@ -135,5 +139,5 @@ static struct tst_test test = { "tmpfs",NULL }, - .timeout = 1800, + .max_runtime = 1800, };
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 24/29 ltp-aiodio/read_checkzero: Remove
In reply to Cyril HrubisRemove leftovers that are not used anymore.
--- runtest/ltp-aiodio.part4 | 2 - testcases/kernel/io/ltp-aiodio/.gitignore | 1 - .../kernel/io/ltp-aiodio/read_checkzero.c | 84 ------------------- 3 files changed, 87 deletions(-) delete mode 100644 testcases/kernel/io/ltp-aiodio/read_checkzero.c
runtest/ltp-aiodio.part4 +0 -2
@@ -59,8 +59,6 @@ ADI009 dio_append DIT000 dio_truncate DIT001 dio_truncate DIT002 dio_truncate -#Running read_checkzero -#gread_checkzero #Running dio_read DOR000 dio_read -n 1 -i 100 -r 512k -w 512k -s 32M DOR001 dio_read -n 10 -i 30 -r 512k -w 512k -s 32M
testcases/kernel/io/ltp-aiodio/.gitignore +0 -1
@@ -7,4 +7,3 @@ /dio_truncate /dio_read /dirty -/read_checkzero
testcases/kernel/io/ltp-aiodio/read_checkzero.c deleted +0 -84
84 lines not on this page · raw message
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 25/29 ltp-aiodio/dio_{truncate, append}: Convert to runtime
In reply to Cyril HrubisChange the code so that the test runtime is capped at the maximal runtime allowed for the test, which is set to 30 minutes by default.
The io_append() in common.h has to check for runtime to make this work.
--- testcases/kernel/io/ltp-aiodio/common.h | 6 +++++- testcases/kernel/io/ltp-aiodio/dio_append.c | 4 ++++ testcases/kernel/io/ltp-aiodio/dio_truncate.c | 6 ++++++ 3 files changed, 15 insertions(+), 1 deletion(-)
testcases/kernel/io/ltp-aiodio/common.h +5 -1
@@ -44,9 +44,13 @@ static inline void io_append(const char *path, char pattern, int flags, size_t b fd =SAFE_OPEN (path, flags,0666 ); -for (i =0 ; i < bcount; i++) +for (i =0 ; i < bcount; i++) {SAFE_WRITE (1 , fd, bufptr, bs); +if (!tst_remaining_runtime ()) +break ; + } +free (bufptr);SAFE_CLOSE (fd); }
testcases/kernel/io/ltp-aiodio/dio_append.c +4 -0
@@ -71,6 +71,9 @@ static void run(void)io_append (filename,0 ,O_DIRECT |O_WRONLY |O_CREAT , writesize, appends); +if (!tst_remaining_runtime ()) +tst_res (TINFO ,"Test out of runtime, exitting" ); +if (SAFE_WAITPID (-1 , &status,WNOHANG ))tst_res (TFAIL ,"Non zero bytes read" );else
@@ -87,6 +90,7 @@ static struct tst_test test = { .cleanup = cleanup, .needs_tmpdir = 1, .forks_child = 1, + .max_runtime = 1800, .options = (struct tst_option []) { {"n:" , &str_numchildren,"Number of processes (default 16)" }, {"w:" , &str_writesize,"Write size for each append (default 64K)" },
testcases/kernel/io/ltp-aiodio/dio_truncate.c +6 -0
@@ -142,6 +142,11 @@ static void run(void) fail =1 ;break ; } + +if (!tst_remaining_runtime ()) { +tst_res (TINFO ,"Test out of runtime, exitting" ); +break ; + } }if (fail)
@@ -158,6 +163,7 @@ static struct tst_test test = { .cleanup = cleanup, .needs_tmpdir = 1, .forks_child = 1, + .max_runtime = 1800, .options = (struct tst_option []) { {"n:" , &str_numchildren,"Number of threads (default 16)" }, {"s:" , &str_filesize,"Size of file (default 64K)" },
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 26/29 ltp-aiodio/dio_read: Convert to runtime
In reply to Cyril HrubisAnd cap the runtime on 30 minutes by default.
--- testcases/kernel/io/ltp-aiodio/dio_read.c | 9 +++++++++ 1 file changed, 9 insertions(+)
testcases/kernel/io/ltp-aiodio/dio_read.c +9 -0
@@ -51,6 +51,9 @@ static void do_buffered_writes(int fd, char *bufptr, long long fsize, long longtst_brk (TBROK ,"pwrite: wrote %lld bytes out of %lld" , w, wsize);SAFE_FSYNC (fd); + +if (!tst_remaining_runtime ()) +return ; } }
@@ -75,6 +78,11 @@ static int do_direct_reads(char *filename, char *bufptr, long long fsize, long l goto exit; } +if (!tst_remaining_runtime ()) { +tst_res (TINFO ,"Test out of runtime, exitting" ); + goto exit; + } + w =pread (fd, bufptr, rsize, offset);if (w <0 )tst_brk (TBROK ,"pread: %s" ,tst_strerrno (-w));
@@ -170,6 +178,7 @@ static struct tst_test test = { .cleanup = cleanup, .needs_tmpdir = 1, .forks_child = 1, + .max_runtime = 1800, .options = (struct tst_option []) { {"n:" , &str_numchildren,"Number of threads (default 8)" }, {"w:" , &str_writesize,"Size of writing blocks (default 32M)" },
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 26/29] ltp-aiodio/dio_read: Convert to runtime
Hi Cyril,
...
Quoted 12 lines
+++
b/testcases/kernel/io/ltp-aiodio/dio_read.ctestcases/kernel/io/ltp-aiodio/dio_read.c +3
-0@@ -51,6 +51,9 @@ static void do_buffered_writes(int fd, char *bufptr, long long fsize, long longtst_brk (TBROK ,"pwrite: wrote %lld bytes out of %lld" , w, wsize);SAFE_FSYNC (fd); + +if (!tst_remaining_runtime ()) +return ; } }
testcases/kernel/io/ltp-aiodio/dio_read.c static int do_direct_reads(char *filename, char *bufptr, long long fsize, long l · line 82
Quoted 5 lines
testcases/kernel/io/ltp-aiodio/dio_read.c +2
-0@@ -75,6 +78,11 @@ static int do_direct_reads(char *filename, char *bufptr, long long fsize, long l goto exit; } +if (!tst_remaining_runtime ()) { +tst_res (TINFO ,"Test out of runtime, exitting" );
nit: s/exitting/exiting/
Kind regards,
Petr
Quoted 1 line
testcases/kernel/io/ltp-aiodio/dio_read.c +1
-0+ goto exit;
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 27/29 timer_test: Convert to runtime
In reply to Cyril Hrubis--- lib/tst_timer_test.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
lib/tst_timer_test.c +9 -3
@@ -420,6 +420,9 @@ static struct tst_option options[] = {static void parse_timer_opts (void ) { +size_t i; +long long runtime_us =0 ; +if (str_sleep_time) {if (tst_parse_int (str_sleep_time, &sleep_time,0 ,INT_MAX )) { tst_brk(TBROK,
@@ -441,14 +444,17 @@ static void parse_timer_opts(void)if (!sample_cnt) sample_cnt =500 ; -long long timeout = sleep_time * sample_cnt /1000000 ; - -tst_set_timeout (timeout + timeout/10 ); + runtime_us = sleep_time * sample_cnt; test->test_all = single_timer_test; test->test =NULL ; test->tcnt =0 ; + }else { +for (i =0 ; i <ARRAY_SIZE (tcases); i++) + runtime_us += tcases[i].usec * tcases[i].samples; } + +tst_set_max_runtime ((runtime_us + runtime_us/10 )/1000000 ); }struct tst_test *tst_timer_test_setup (struct tst_test *timer_test)
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 28/29 ltp-aiodio/aiodio_append: Convert to runtime
In reply to Cyril HrubisCap the test runtime on 30 minutes by default.
For this to work we have to run the process that issues the aio requests in a child process and kill it when runtime is over.
--- .../kernel/io/ltp-aiodio/aiodio_append.c | 25 ++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)
testcases/kernel/io/ltp-aiodio/aiodio_append.c +22 -3
@@ -141,7 +141,7 @@ static void run(void) {char *filename ="aiodio_append" ;int status; -int i; +int i, pid; *run_child =1 ;
@@ -152,9 +152,27 @@ static void run(void) } } -tst_res (TINFO ,"Parent append to file" ); + pid =SAFE_FORK (); +if (!pid) { +aiodio_append (filename, appends, alignment, writesize, numaio); +return ; + } + +tst_res (TINFO ,"Child %i appends to a file" , pid); + +for (;;) { +if (SAFE_WAITPID (pid,NULL ,WNOHANG )) +break ; -aiodio_append (filename, appends, alignment, writesize, numaio); +sleep (1 ); + +if (!tst_remaining_runtime ()) { +tst_res (TINFO ,"Test out of runtime, exitting" ); +kill (pid,SIGKILL ); +SAFE_WAITPID (pid,NULL ,0 ); +break ; + } + }if (SAFE_WAITPID (-1 , &status,WNOHANG ))tst_res (TFAIL ,"Non zero bytes read" );
@@ -172,6 +190,7 @@ static struct tst_test test = { .cleanup = cleanup, .needs_tmpdir = 1, .forks_child = 1, + .max_runtime = 1800, .options = (struct tst_option []) { {"n:" , &str_numchildren,"Number of threads (default 16)" }, {"s:" , &str_writesize,"Size of the file to write (default 64K)" },
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 29/29 tst_test: Remove timeout stubs
In reply to Cyril Hrubis--- include/tst_test.h | 4 ---- lib/tst_test.c | 11 ----------- 2 files changed, 15 deletions(-)
include/tst_test.h +0 -4
@@ -236,8 +236,6 @@ struct tst_test {unsigned int mnt_flags;void *mnt_data; -/* override default timeout per test run, disabled == -1 */ -int timeout;/* * Maximal test runtime in seconds. *
@@ -333,9 +331,7 @@ const char *tst_strsig(int sig);*/ const char *tst_strstatus (int status); -unsigned int tst_timeout_remaining (void );unsigned int tst_multiply_timeout (unsigned int timeout); -void tst_set_timeout (int timeout);/* * Returns remaining test runtime. Test that runs for more than a few seconds
lib/tst_test.c +0 -11
@@ -1471,12 +1471,6 @@ static void sigint_handler(int sig LTP_ATTRIBUTE_UNUSED) } } -unsigned int tst_timeout_remaining (void ) -{ -tst_brk (TBROK ,"Stub called!" ); -return 0 ; -} -unsigned int tst_remaining_runtime (void ) {static struct timespec now;
@@ -1509,11 +1503,6 @@ unsigned int tst_multiply_timeout(unsigned int timeout)return timeout * timeout_mul; } -void tst_set_timeout (int timeout) -{ -tst_brk (TBROK ,"Stub called!" ); -} -static void set_timeout (void ) {unsigned int timeout =DEFAULT_TIMEOUT ;
Signature · 5 lines
2.35.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
cover two follow up fix for Introduce-of-max-runtime
In reply to Cyril HrubisI run a quick test for verifying those patches with the latest mainline kernel. Apart from these two tiny issues, the result looks good.
kernel: 5.18.0-rc7
arch: x86_64, ppc64le, s390x, aarch64
git-repo: https://github.com/wangli5665/ltp
Ps. I'm going to submit more tests against different RHEL products later.
Li Wang (2):
testcases: make use of runtime
mmap3: reset runtime to 10s
runtest/mm | 2 +- testcases/cve/cve-2014-0196.c | 1 + testcases/cve/cve-2016-7117.c | 1 + testcases/cve/cve-2017-2671.c | 1 + testcases/kernel/mem/mtest06/mmap3.c | 2 +- testcases/network/can/cve/can_bcm01.c | 1 + testcases/network/packet/fanout01.c | 1 + testcases/network/sockets/vsock01.c | 1 + 8 files changed, 8 insertions(+), 2 deletions(-)
Signature · 5 lines
2.31.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 1/2 testcases: make use of runtime
If test using the fzsync lib framework, it has to be set with .max_runtime.
--- testcases/cve/cve-2014-0196.c | 1 + testcases/cve/cve-2016-7117.c | 1 + testcases/cve/cve-2017-2671.c | 1 + testcases/network/can/cve/can_bcm01.c | 1 + testcases/network/packet/fanout01.c | 1 + testcases/network/sockets/vsock01.c | 1 + 6 files changed, 6 insertions(+)
testcases/cve/cve-2014-0196.c +1 -0
@@ -141,6 +141,7 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .test_all = run, + .max_runtime = 60, .tags = (const struct tst_tag []) { {"linux-git" ,"4291086b1f08" }, {"CVE" ,"2014-0196" },
testcases/cve/cve-2016-7117.c +1 -0
@@ -150,6 +150,7 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .min_kver = "2.6.33", + .max_runtime = 60, .tags = (const struct tst_tag []) { {"linux-git" ,"a2e2725541fa" }, {"CVE" ,"2016-7117" },
testcases/cve/cve-2017-2671.c +1 -0
@@ -109,6 +109,7 @@ static struct tst_test test = { .test_all = run, .cleanup = cleanup, .needs_root = 1, + .max_runtime = 20, .tags = (const struct tst_tag []) { {"linux-git" ,"43a6684519ab" }, {"CVE" ,"2017-2671" },
testcases/network/can/cve/can_bcm01.c +1 -0
@@ -142,6 +142,7 @@ static struct tst_test test = { .taint_check = TST_TAINT_W | TST_TAINT_D, .needs_root = 1, .skip_in_compat = 1, + .max_runtime = 30, .needs_drivers = (const char *const []) {"vcan" ,"can-bcm" ,
testcases/network/packet/fanout01.c +1 -0
@@ -106,6 +106,7 @@ static struct tst_test test = { .test_all = run, .cleanup = cleanup, .needs_root = 1, + .max_runtime = 180, .tags = (const struct tst_tag []) { {"CVE" ,"2017-15649" }, {"linux-git" ,"4971613c1639" },
testcases/network/sockets/vsock01.c +1 -0
@@ -111,6 +111,7 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .taint_check = TST_TAINT_W | TST_TAINT_D, + .max_runtime = 60, .needs_kconfigs = (const char *[]) {"CONFIG_VSOCKETS_LOOPBACK" ,NULL
Signature · 5 lines
2.31.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/2] testcases: make use of runtime
Hi!
This should be ammended to the "fuzzy_sync: Convert to runtime" patch.
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/2] testcases: make use of runtime
Quoted 2 lines
This should be ammended to the "
fuzzy_sync: Convert to runtime" patch.
Yes, that would be better. Feel free to merge into that one.
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 2/2 mmap3: reset runtime to 10s
In reply to Li WangAs we have removed the "-x" support from this test, there is no way to reduce running time by multiply small float number.
Let's set 10 seconds for the max_runtime (approximately equal to -x 0.002).
--- runtest/mm | 2 +- testcases/kernel/mem/mtest06/mmap3.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
runtest/mm +1 -1
@@ -16,7 +16,7 @@ mtest01w mtest01 -p80 -w mtest05 mmstress mtest06 mmap1 mtest06_2 mmap2 -x 0.002 -a -p -mtest06_3 mmap3 -x 0.002 -p +mtest06_3 mmap3 -p # Remains diabled till the infinite loop problem is solved #mtest-6_4 shmat1 -x 0.00005
testcases/kernel/mem/mtest06/mmap3.c +1 -1
@@ -182,5 +182,5 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .test_all = test_mmap, - .max_runtime = 86400, + .max_runtime = 10, };
Signature · 5 lines
2.31.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 2/2] mmap3: reset runtime to 10s
Hi!
Quoted 2 lines
As we have removed the "-x" support from this test, there is no way to reduce running time by multiply small float number.
Actually the -x has been replaced by -I so we may as well change the runtest file to pass -I instead.
On the other hand I guess that the original runtime was way too long for a default value.
Quoted 1 line
Let's set 10 seconds for the
max_runtime(approximately equal to -x 0.002).
If we are touching the code we may as well pick slightly longer interval, what about a minute so that we have chance to do some testing even on slower targets?
Quoted 36 lines
Signed-off-by Li Wang <[email protected]>--- runtest/mm | 2 +- testcases/kernel/mem/mtest06/mmap3.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)runtest/mm +1
-1@@ -16,7 +16,7 @@ mtest01w mtest01 -p80 -w mtest05 mmstress mtest06 mmap1 mtest06_2 mmap2 -x 0.002 -a -p -mtest06_3 mmap3 -x 0.002 -p +mtest06_3 mmap3 -p # Remains diabled till the infinite loop problem is solved #mtest-6_4 shmat1 -x 0.00005testcases/kernel/mem/mtest06/mmap3.c +1
-1@@ -182,5 +182,5 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .test_all = test_mmap, - .max_runtime = 86400, + .max_runtime = 10, };Signature · 5 lines
2.31.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 2/2] mmap3: reset runtime to 10s
Quoted 7 lines · Cyril Hrubis
Hi!
Quoted 2 lines
As we have removed the "-x" support from this test, there is no way to reduce running time by multiply small float number.
Actually the -x has been replaced by -I so we may as well change the runtest file to pass -I instead.
Good to know, thanks!
Quoted 11 lines
On the other hand I guess that the original runtime was way too long for a default value.
Quoted 1 line
Let's set 10 seconds for the
max_runtime(approximately equal to -x0.002).
If we are touching the code we may as well pick slightly longer interval, what about a minute so that we have chance to do some testing even on slower targets?
Make sense!
Feel free to add your amend when patchset merging, I'd not send V2 to bring additional review work.
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime
In reply to Li WangHi!
Quoted 2 lines
I run a quick test for verifying those patches with the latest mainline kernel. Apart from these two tiny issues, the result looks good.
Can we merge the patchset now?
There are three things that should be fixed over the latest version, that are:
- the heartbeat() call after each test for .tcnt
- the missing runtime for fuzzy sync
- default to something more sane in mmap3 and fix the runtest file (could be ammended to the convert mmap3 patch as well)
Should I send v4 after all?
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime
Cyril Hrubis <[email protected]> wrote:
Hi!
Quoted 6 lines
Quoted 1 line
I run a quick test for verifying those patches with the latest mainline
kernel.
Quoted 1 line
Apart from these two tiny issues, the result looks good.
Can we merge the patchset now?
I think yes, at least my test shows it is safe.
Quoted 11 lines
There are three things that should be fixed over the latest version, that are:
- the
heartbeat()call after each test for .tcnt
- the missing runtime for fuzzy sync
- default to something more sane in mmap3 and fix the runtest file (could be ammended to the convert mmap3 patch as well)Should I send v4 after all?
No need, unless someone explicitly explains they will do a review/test for V4.
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 0/2] two follow up fix for Introduce-of-max-runtime
Hi Li, Cyril,
Quoted 4 lines
Quoted 1 line
Should I send v4 after all?
No need, unless someone explicitly explains they will do a review/test for V4.
I only briefly looked at the patchset, but LGTM.
+1 for merging.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Cyril, all,
For whole patchset including that extra commit of unsent v4:
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi,
very nit: I suppose it's "exiting", not exitting.
It's in several commits.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp