tst_test: Add min_runtime to control lower bound of scaled runtime
PATCH v2 1/1 tst_test: Add min_runtime to control lower bound of scaled runtime
When LTP_RUNTIME_MUL is set to a value < 1.0 (commonly in CI or fast test modes), the effective runtime of tests may be scaled down too aggressively. For some tests especially those relying on probabilistic triggers or sampling (e.g., fuzzy sync, CVE stress loops, long fault injection), this can result in too few iterations or missed conditions, leading to unreliable results.
This patch introduces a new field: .min_runtime;
If set, this specifies the minimum allowed runtime (in seconds) after applying the LTP_RUNTIME_MUL scaling factor. The final runtime is calculated as:
MAX(runtime * LTP_RUNTIME_MUL, min_runtime)
If min_runtime is not set, a default minimum of 1 second is enforced.
This approach replaces the need for special flags such as TST_NO_FRAC_RUNTIME_MUL and provides a more systematic, flexible solution.
The following tests are updated to set .min_runtime:
- preadv203
- cve-2016-7117
- tst_fuzzy_sync01
- tst_fuzzy_sync02
---
Notes:
v1 --> v2:
* replace the .flags by a new min_runtime filed
* set 1 second as the default minimal runtime regardless LTP_RUNTIME_MUL value
include/tst_test.h | 9 +++++++++ lib/newlib_tests/tst_fuzzy_sync01.c | 2 +- lib/newlib_tests/tst_fuzzy_sync02.c | 2 +- lib/tst_test.c | 11 +++++++++-- testcases/cve/cve-2016-7117.c | 2 +- testcases/kernel/syscalls/preadv2/preadv203.c | 2 +- 6 files changed, 22 insertions(+), 6 deletions(-)
include/tst_test.h +9 -0
@@ -460,6 +460,14 @@ struct tst_fs {* (e.g., TIMEOUT_MUL), ensuring consistent test duration across * different environments (e.g., debug vs. stock kernels). * +* @min_runtime: Optional lower bound (in seconds) applied after runtime is +* scaled by LTP_RUNTIME_MUL. If the scaled runtime is smaller +* than this value, it will be clamped up to min_runtime. +* This is useful for tests that require a minimum execution +* time to gather sufficient samples or trigger events (e.g., +* probabilistic or fuzzy synchronization tests). +* If not set, a default minimum of 1 second is enforced. +* * @setup: Setup callback is called once at the start of the test in order to * prepare the test environment. *
@@ -584,6 +592,7 @@ struct tst_fs {int timeout;int runtime; +int min_runtime;void (*setup)(void );void (*cleanup)(void );
lib/newlib_tests/tst_fuzzy_sync01.c +1 -1
@@ -246,5 +246,5 @@ static struct tst_test test = { .test = run, .setup = setup, .cleanup = cleanup, - .runtime = 150, + .min_runtime = 150, };
lib/newlib_tests/tst_fuzzy_sync02.c +1 -1
@@ -223,5 +223,5 @@ static struct tst_test test = { .test = run, .setup = setup, .cleanup = cleanup, - .runtime = 150, + .min_runtime = 150, };
lib/tst_test.c +9 -2
@@ -627,10 +627,14 @@ static void parse_mul(float *mul, const char *env_name, float min, float max)static int multiply_runtime (unsigned int runtime) {static float runtime_mul =-1 ; +int min_runtime =1 ; + +if (tst_test->min_runtime) + min_runtime =MAX (1 , tst_test->min_runtime);parse_mul (&runtime_mul,"LTP_RUNTIME_MUL" ,0.0099 ,100 ); -return runtime * runtime_mul; +return MAX (runtime * runtime_mul, min_runtime); } static struct option {
@@ -662,7 +666,7 @@ static void print_help(void)fprintf (stderr,"LTP_SINGLE_FS_TYPE Specifies filesystem instead all supported (for .all_filesystems)\n" );fprintf (stderr,"LTP_FORCE_SINGLE_FS_TYPE Testing only. The same as LTP_SINGLE_FS_TYPE but ignores test skiplist.\n" );fprintf (stderr,"LTP_TIMEOUT_MUL Timeout multiplier (must be a number >=1)\n" ); -fprintf (stderr,"LTP_RUNTIME_MUL Runtime multiplier (must be a number >=1)\n" ); +fprintf (stderr,"LTP_RUNTIME_MUL Runtime multiplier (must be a number >0)\n" );fprintf (stderr,"LTP_VIRT_OVERRIDE Overrides virtual machine detection (values: \"\"|kvm|microsoft|xen|zvm)\n" );fprintf (stderr,"TMPDIR Base directory for template directory (for .needs_tmpdir, default: %s)\n" ,TEMPDIR );fprintf (stderr,"\n" );
@@ -1996,6 +2000,9 @@ void tst_run_tcases(int argc, char *argv[], struct tst_test *self)uname (&uval);tst_res (TINFO ,"Tested kernel: %s %s %s" , uval.release, uval.version, uval.machine); +if (tst_test->min_runtime) + tst_test->runtime =tst_test->min_runtime; +if (tst_test->runtime) context->runtime =multiply_runtime (tst_test->runtime);
testcases/cve/cve-2016-7117.c +1 -1
@@ -151,7 +151,7 @@ static struct tst_test test = { .test_all = run, .setup = setup, .cleanup = cleanup, - .runtime = 60, + .min_runtime = 60, .tags = (const struct tst_tag []) { {"linux-git" ,"34b88a68f26a" }, {"CVE" ,"2016-7117" },
testcases/kernel/syscalls/preadv2/preadv203.c +1 -1
@@ -278,6 +278,6 @@ static struct tst_test test = { .mntpoint = MNTPOINT, .mount_device = 1, .all_filesystems = 1, - .runtime = 60, + .min_runtime = 60, .needs_root = 1, };
Signature · 5 lines
2.49.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Hi!
Quoted 75 lines · Li Wang
When
LTP_RUNTIME_MULis set to a value < 1.0 (commonly in CI or fast test modes), the effective runtime of tests may be scaled down too aggressively. For some tests especially those relying on probabilistic triggers or sampling (e.g., fuzzy sync, CVE stress loops, long fault injection), this can result in too few iterations or missed conditions, leading to unreliable results.This patch introduces a new field: .min_runtime;
If set, this specifies the minimum allowed runtime (in seconds) after applying the
LTP_RUNTIME_MULscaling factor. The final runtime is calculated as:
MAX(runtime * LTP_RUNTIME_MUL, min_runtime)If
min_runtimeis not set, a default minimum of 1 second is enforced.This approach replaces the need for special flags such as
TST_NO_FRAC_RUNTIME_MULand provides a more systematic, flexible solution.The following tests are updated to set .min_runtime:
- preadv203
- cve-2016-7117
-tst_fuzzy_sync01
-tst_fuzzy_sync02Suggested-by Martin Doucha <[email protected]>Signed-off-by Li Wang <[email protected]>Cc Ian Wienand <[email protected]>---
Notes:
v1 --> v2:
* replace the .flags by a newmin_runtimefiled
* set 1 second as the default minimal runtime regardlessLTP_RUNTIME_MULvalueinclude/tst_test.h | 9 +++++++++ lib/newlib_tests/tst_fuzzy_sync01.c | 2 +- lib/newlib_tests/tst_fuzzy_sync02.c | 2 +- lib/tst_test.c | 11 +++++++++-- testcases/cve/cve-2016-7117.c | 2 +- testcases/kernel/syscalls/preadv2/preadv203.c | 2 +- 6 files changed, 22 insertions(+), 6 deletions(-)include/tst_test.h +9
-0@@ -460,6 +460,14 @@ struct tst_fs {* (e.g., TIMEOUT_MUL), ensuring consistent test duration across * different environments (e.g., debug vs. stock kernels). * +* @min_runtime: Optional lower bound (in seconds) applied after runtime is +* scaled by LTP_RUNTIME_MUL. If the scaled runtime is smaller +* than this value, it will be clamped up to min_runtime. +* This is useful for tests that require a minimum execution +* time to gather sufficient samples or trigger events (e.g., +* probabilistic or fuzzy synchronization tests). +* If not set, a default minimum of 1 second is enforced. +* * @setup: Setup callback is called once at the start of the test in order to * prepare the test environment. * @@ -584,6 +592,7 @@ struct tst_fs {int timeout;int runtime; +int min_runtime;void (*setup)(void );void (*cleanup)(void );lib/newlib_tests/tst_fuzzy_sync01.c +0
-0
lib/newlib_tests/tst_fuzzy_sync01.c static struct tst_test test = { · line 249
Quoted 5 lines
lib/newlib_tests/tst_fuzzy_sync01.c +1
-1@@ -246,5 +246,5 @@ static struct tst_test test = { .test = run, .setup = setup, .cleanup = cleanup, - .runtime = 150, + .min_runtime = 150,
Lib tests don't need .min_runtime unless you specifically want to test that LTP_RUNTIME_MUL gets handled correctly.
Quoted 42 lines
lib/newlib_tests/tst_fuzzy_sync01.c +0
-0};lib/newlib_tests/tst_fuzzy_sync02.c +1
-1@@ -223,5 +223,5 @@ static struct tst_test test = { .test = run, .setup = setup, .cleanup = cleanup, - .runtime = 150, + .min_runtime = 150, };lib/tst_test.c +6
-2@@ -627,10 +627,14 @@ static void parse_mul(float *mul, const char *env_name, float min, float max)static int multiply_runtime (unsigned int runtime) {static float runtime_mul =-1 ; +int min_runtime =1 ; + +if (tst_test->min_runtime) + min_runtime =MAX (1 , tst_test->min_runtime);parse_mul (&runtime_mul,"LTP_RUNTIME_MUL" ,0.0099 ,100 ); -return runtime * runtime_mul; +return MAX (runtime * runtime_mul, min_runtime); } static struct option {@@ -662,7 +666,7 @@ static void print_help(void)fprintf (stderr,"LTP_SINGLE_FS_TYPE Specifies filesystem instead all supported (for .all_filesystems)\n" );fprintf (stderr,"LTP_FORCE_SINGLE_FS_TYPE Testing only. The same as LTP_SINGLE_FS_TYPE but ignores test skiplist.\n" );fprintf (stderr,"LTP_TIMEOUT_MUL Timeout multiplier (must be a number >=1)\n" ); -fprintf (stderr,"LTP_RUNTIME_MUL Runtime multiplier (must be a number >=1)\n" ); +fprintf (stderr,"LTP_RUNTIME_MUL Runtime multiplier (must be a number >0)\n" );fprintf (stderr,"LTP_VIRT_OVERRIDE Overrides virtual machine detection (values: \"\"|kvm|microsoft|xen|zvm)\n" );fprintf (stderr,"TMPDIR Base directory for template directory (for .needs_tmpdir, default: %s)\n" ,TEMPDIR );fprintf (stderr,"\n" );
lib/tst_test.c void tst_run_tcases(int argc, char *argv[], struct tst_test *self) · line 2004
Quoted 5 lines
lib/tst_test.c +2
-0@@ -1996,6 +2000,9 @@ void tst_run_tcases(int argc, char *argv[], struct tst_test *self)uname (&uval);tst_res (TINFO ,"Tested kernel: %s %s %s" , uval.release, uval.version, uval.machine); +if (tst_test->min_runtime) + tst_test->runtime =tst_test->min_runtime;
If .runtime is set, .min_runtime should not override it. Setting .runtime < .min_runtime should trigger TBROK during lib setup.
Quoted 9 lines
lib/tst_test.c +1
-0+if (tst_test->runtime) context->runtime =multiply_runtime (tst_test->runtime);testcases/cve/cve-2016-7117.c +0
-0
testcases/cve/cve-2016-7117.c static struct tst_test test = { · line 154
Quoted 5 lines
testcases/cve/cve-2016-7117.c +1
-1@@ -151,7 +151,7 @@ static struct tst_test test = { .test_all = run, .setup = setup, .cleanup = cleanup, - .runtime = 60, + .min_runtime = 60,
The .runtime value in fuzzy_sync tests is calibrated to reliably reproduce the bug. You can usually set much smaller .min_runtime based on how long it takes to run 1024 iterations of the fuzzy loop, with some safety margin (x2?).
Quoted 15 lines
testcases/cve/cve-2016-7117.c +0
-0.tags = (const struct tst_tag[]) { {"linux-git" ,"34b88a68f26a" }, {"CVE" ,"2016-7117" },testcases/kernel/syscalls/preadv2/preadv203.c +1
-1@@ -278,6 +278,6 @@ static struct tst_test test = { .mntpoint = MNTPOINT, .mount_device = 1, .all_filesystems = 1, - .runtime = 60, + .min_runtime = 60, .needs_root = 1, };
Signature · 10 lines
Martin Doucha [email protected] SW Quality Engineer SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -- Mailing list info: https://lists.linux.it/listinfo/ltp
Quoted 83 lines · Martin Doucha
Hi!
Quoted 2 lines · Li Wang
When
LTP_RUNTIME_MULis set to a value < 1.0 (commonly in CI or fast test modes), the effective runtime of tests may be scaled down tooaggressively.
Quoted 1 line
For some tests especially those relying on probabilistic triggers or
sampling
Quoted 1 line
(e.g., fuzzy sync, CVE stress loops, long fault injection), this can
result in
Quoted 5 lines
too few iterations or missed conditions, leading to unreliable results.
This patch introduces a new field: .min_runtime;
If set, this specifies the minimum allowed runtime (in seconds) after
applying
Quoted 8 lines
the
LTP_RUNTIME_MULscaling factor. The final runtime is calculated as:
MAX(runtime * LTP_RUNTIME_MUL, min_runtime)If
min_runtimeis not set, a default minimum of 1 second is enforced.This approach replaces the need for special flags such as
TST_NO_FRAC_RUNTIME_MULand provides a more systematic, flexiblesolution.
Quoted 31 lines
The following tests are updated to set .min_runtime:
- preadv203
- cve-2016-7117
-tst_fuzzy_sync01
-tst_fuzzy_sync02Suggested-by Martin Doucha <[email protected]>Signed-off-by Li Wang <[email protected]>Cc Ian Wienand <[email protected]>---
Notes:
v1 --> v2:
* replace the .flags by a newmin_runtimefiled
* set 1 second as the default minimal runtime regardlessLTP_RUNTIME_MULvalueinclude/tst_test.h | 9 +++++++++ lib/newlib_tests/tst_fuzzy_sync01.c | 2 +- lib/newlib_tests/tst_fuzzy_sync02.c | 2 +- lib/tst_test.c | 11 +++++++++-- testcases/cve/cve-2016-7117.c | 2 +- testcases/kernel/syscalls/preadv2/preadv203.c | 2 +- 6 files changed, 22 insertions(+), 6 deletions(-)include/tst_test.h +0
-0include/tst_test.h struct tst_fs { · line 460
Quoted 1 line
include/tst_test.h +0
-0@@ -460,6 +460,14 @@ struct tst_fs {* (e.g., TIMEOUT_MUL), ensuring consistent test duration across
include/tst_test.h struct tst_fs {
Quoted 3 lines
include/tst_test.h +1
-0* different environments (e.g., debug vs. stock kernels). * +* @min_runtime: Optional lower bound (in seconds) applied after runtime is
include/tst_test.h struct tst_fs {
Quoted 8 lines
include/tst_test.h +7
-0+* scaled by LTP_RUNTIME_MUL. If the scaled runtime is smaller +* than this value, it will be clamped up to min_runtime. +* This is useful for tests that require a minimum execution +* time to gather sufficient samples or trigger events (e.g., +* probabilistic or fuzzy synchronization tests). +* If not set, a default minimum of 1 second is enforced. +* * @setup: Setup callback is called once at the start of the test in order to
Review
Quoted 11 lines
include/tst_test.h +1
-0* prepare the test environment. * @@ -584,6 +592,7 @@ struct tst_fs {int timeout;int runtime; +int min_runtime;void (*setup)(void );void (*cleanup)(void );(quoted diff) +0
-0
b/lib/newlib_tests/tst_fuzzy_sync01.c
Quoted 13 lines
lib/newlib_tests/tst_fuzzy_sync01.c static struct tst_test test = { · line 249
Quoted 9 lines
index
b1390f460..16f30de57100644lib/newlib_tests/tst_fuzzy_sync01.c +1
-1@@ -246,5 +246,5 @@ static struct tst_test test = { .test = run, .setup = setup, .cleanup = cleanup, - .runtime = 150, + .min_runtime = 150,Lib tests don't need .min_runtime unless you specifically want to test that
LTP_RUNTIME_MULgets handled correctly.
I set .min_runtime for all fuzzy_sync tests that contain "pair.min_samples = XXX",
because the dedicated sampling times need more time and it is hard to tell the real minimal time, so I just reuse the old .runtime value.
And yes we can ignore this in lib tests, but I want to make them as examples.
Quoted 55 lines
Review
Quoted 3 lines
lib/newlib_tests/tst_fuzzy_sync01.c +0
-0};(quoted diff) +0
-0
b/lib/newlib_tests/tst_fuzzy_sync02.clib/tst_test.c static void parse_mul(float *mul, const char
Quoted 15 lines
index
bc079f6ff..3b0fb8a9b100644lib/newlib_tests/tst_fuzzy_sync02.c +1
-1@@ -223,5 +223,5 @@ static struct tst_test test = { .test = run, .setup = setup, .cleanup = cleanup, - .runtime = 150, + .min_runtime = 150, };lib/tst_test.c +0
-0@@ -627,10 +627,14 @@ static void parse_mul(float *mul, const char
*env_name, float min, float max)Quoted 16 lines
lib/tst_test.c +5
-1static int multiply_runtime (unsigned int runtime) {static float runtime_mul =-1 ; +int min_runtime =1 ; + +if (tst_test->min_runtime) + min_runtime =MAX (1 , tst_test->min_runtime);parse_mul (&runtime_mul,"LTP_RUNTIME_MUL" ,0.0099 ,100 ); -return runtime * runtime_mul; +return MAX (runtime * runtime_mul, min_runtime); } static struct option {lib/tst_test.c static void print_help(void) · line 666
Quoted 1 line
lib/tst_test.c +0
-0@@ -662,7 +666,7 @@ static void print_help(void) fprintf(stderr, "LTP_SINGLE_FS_TYPE Specifies filesysteminstead all supported (for .all_filesystems)\n");
lib/tst_test.c static void print_help(void) · line 667
Quoted 1 line
lib/tst_test.c +0
-0fprintf(stderr, "LTP_FORCE_SINGLE_FS_TYPE Testing only. The sameas
LTP_SINGLE_FS_TYPEbut ignores test skiplist.\n");lib/tst_test.c static void print_help(void) · line 668
Quoted 1 line
lib/tst_test.c +0
-0fprintf(stderr, "LTP_TIMEOUT_MUL Timeout multiplier (mustbe a number >=1)\n");
lib/tst_test.c static void print_help(void) · line 665
Quoted 1 line
lib/tst_test.c +0
-1- fprintf(stderr, "LTP_RUNTIME_MUL Runtime multiplier (mustbe a number >=1)\n");
lib/tst_test.c static void print_help(void) · line 669
Quoted 1 line
lib/tst_test.c +1
-0+ fprintf(stderr, "LTP_RUNTIME_MUL Runtime multiplier (mustbe a number >0)\n");
lib/tst_test.c static void print_help(void) · line 670
Quoted 1 line
lib/tst_test.c +0
-0fprintf(stderr, "LTP_VIRT_OVERRIDE Overrides virtualmachine detection (values: \"\"|kvm|microsoft|xen|zvm)\n");
lib/tst_test.c static void print_help(void) · line 671
Quoted 1 line
lib/tst_test.c +0
-0fprintf(stderr, "TMPDIR Base directory fortemplate directory (for .needs_tmpdir, default: %s)\n", TEMPDIR);
lib/tst_test.c void tst_run_tcases(int argc, char *argv[], struct
Quoted 2 lines
lib/tst_test.c +0
-0fprintf (stderr,"\n" );@@ -1996,6 +2000,9 @@ void tst_run_tcases(int argc, char *argv[], struct
tst_test*self)lib/tst_test.c void tst_run_tcases(int argc, char *argv[], struct · line 2001
Quoted 2 lines
lib/tst_test.c +0
-0uname (&uval); tst_res(TINFO, "Tested kernel: %s %s %s", uval.release,
Quoted 9 lines
uval.version, uval.machine);
lib/tst_test.c void tst_run_tcases(int argc, char *argv[], struct tst_test *self) · line 2004
Quoted 3 lines
lib/tst_test.c +2
-0+if (tst_test->min_runtime) + tst_test->runtime =tst_test->min_runtime;If .runtime is set, .min_runtime should not override it. Setting .runtime < .min_runtime should trigger TBROK during lib setup.
Not necessary, in most cases, we don't need to set both simultaneously. Even if .runtime is less than .min_runtime, MAX() will automatically choose .min_runtime. (but I strongly do not recommend set both)
Or, maybe here we can override if .runtime not being set to avoid lib complains no .runtime set failure.
- if (tst_test->min_runtime)
+ if (tst_test->min_runtime && !tst_test->runtime)
Quoted 7 lines
Review
Quoted 6 lines
lib/tst_test.c +1
-0+if (tst_test->runtime) context->runtime =multiply_runtime (tst_test->runtime);(quoted diff) +0
-0
b/testcases/cve/cve-2016-7117.c
Quoted 15 lines
testcases/cve/cve-2016-7117.c static struct tst_test test = { · line 154
Quoted 9 lines
index
dbec2b28b..8c488f447100644testcases/cve/cve-2016-7117.c +1
-1@@ -151,7 +151,7 @@ static struct tst_test test = { .test_all = run, .setup = setup, .cleanup = cleanup, - .runtime = 60, + .min_runtime = 60,The .runtime value in
fuzzy_synctests is calibrated to reliably reproduce the bug. You can usually set much smaller .min_runtime based on how long it takes to run 1024 iterations of the fuzzy loop, with some safety margin (x2?).
In theory, yes, but the difficulty is thatthe calibration time depends on the various test platforms and system load conditions.
Here I only reset the .runtime to .min_runtime is for tests contain 'pair.min_smaples', which typically has a large value (e.g. 10000).
For those with the default 1024 iterations fuzzy_sync tests I did nothing for them, we may need more experiments to get a proper value.
Quoted 29 lines
Review
Quoted 5 lines
testcases/cve/cve-2016-7117.c +0
-0.tags = (const struct tst_tag[]) { {"linux-git" ,"34b88a68f26a" }, {"CVE" ,"2016-7117" },(quoted diff) +0
-0
b/testcases/kernel/syscalls/preadv2/preadv203.ctestcases/kernel/syscalls/preadv2/preadv203.c static struct tst_test test = { · line 283
Quoted 11 lines
index
72a35d3ab..472543e5c100644testcases/kernel/syscalls/preadv2/preadv203.c +1
-1@@ -278,6 +278,6 @@ static struct tst_test test = { .mntpoint = MNTPOINT, .mount_device = 1, .all_filesystems = 1, - .runtime = 60, + .min_runtime = 60, .needs_root = 1, };--
Martin Doucha [email protected]
SW Quality Engineer
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic
Signature · 5 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp
Quoted 48 lines · Li Wang via ltp
When
LTP_RUNTIME_MULis set to a value < 1.0 (commonly in CI or fast test modes), the effective runtime of tests may be scaled down too aggressively. For some tests especially those relying on probabilistic triggers or sampling (e.g., fuzzy sync, CVE stress loops, long fault injection), this can result in too few iterations or missed conditions, leading to unreliable results.This patch introduces a new field: .min_runtime;
If set, this specifies the minimum allowed runtime (in seconds) after applying the
LTP_RUNTIME_MULscaling factor. The final runtime is calculated as:
MAX(runtime * LTP_RUNTIME_MUL, min_runtime)If
min_runtimeis not set, a default minimum of 1 second is enforced.This approach replaces the need for special flags such as
TST_NO_FRAC_RUNTIME_MULand provides a more systematic, flexible solution.The following tests are updated to set .min_runtime:
- preadv203
- cve-2016-7117
-tst_fuzzy_sync01
-tst_fuzzy_sync02Suggested-by Martin Doucha <[email protected]>Signed-off-by Li Wang <[email protected]>Cc Ian Wienand <[email protected]>---
Notes:
v1 --> v2:
* replace the .flags by a newmin_runtimefiled
* set 1 second as the default minimal runtime regardlessLTP_RUNTIME_MULvalueinclude/tst_test.h | 9 +++++++++ lib/newlib_tests/tst_fuzzy_sync01.c | 2 +- lib/newlib_tests/tst_fuzzy_sync02.c | 2 +- lib/tst_test.c | 11 +++++++++-- testcases/cve/cve-2016-7117.c | 2 +- testcases/kernel/syscalls/preadv2/preadv203.c | 2 +- 6 files changed, 22 insertions(+), 6 deletions(-)include/tst_test.h +0
-0
include/tst_test.h struct tst_fs { · line 469
Quoted 10 lines
include/tst_test.h +7
-0@@ -460,6 +460,14 @@ struct tst_fs {* (e.g., TIMEOUT_MUL), ensuring consistent test duration across * different environments (e.g., debug vs. stock kernels). * +* @min_runtime: Optional lower bound (in seconds) applied after runtime is +* scaled by LTP_RUNTIME_MUL. If the scaled runtime is smaller +* than this value, it will be clamped up to min_runtime. +* This is useful for tests that require a minimum execution +* time to gather sufficient samples or trigger events (e.g., +* probabilistic or fuzzy synchronization tests). +* If not set, a default minimum of 1 second is enforced.
Base your logic once .min_runtime is set then .runtime value effectively becomes irrelevant. So i guess we need mention it in above description?
Quoted 104 lines
include/tst_test.h +2
-0+* * @setup: Setup callback is called once at the start of the test in order to * prepare the test environment. * @@ -584,6 +592,7 @@ struct tst_fs {int timeout;int runtime; +int min_runtime;void (*setup)(void );void (*cleanup)(void );lib/newlib_tests/tst_fuzzy_sync01.c +1
-1@@ -246,5 +246,5 @@ static struct tst_test test = { .test = run, .setup = setup, .cleanup = cleanup, - .runtime = 150, + .min_runtime = 150, };lib/newlib_tests/tst_fuzzy_sync02.c +1
-1@@ -223,5 +223,5 @@ static struct tst_test test = { .test = run, .setup = setup, .cleanup = cleanup, - .runtime = 150, + .min_runtime = 150, };lib/tst_test.c +9
-2@@ -627,10 +627,14 @@ static void parse_mul(float *mul, const char *env_name, float min, float max)static int multiply_runtime (unsigned int runtime) {static float runtime_mul =-1 ; +int min_runtime =1 ; + +if (tst_test->min_runtime) + min_runtime =MAX (1 , tst_test->min_runtime);parse_mul (&runtime_mul,"LTP_RUNTIME_MUL" ,0.0099 ,100 ); -return runtime * runtime_mul; +return MAX (runtime * runtime_mul, min_runtime); } static struct option {@@ -662,7 +666,7 @@ static void print_help(void)fprintf (stderr,"LTP_SINGLE_FS_TYPE Specifies filesystem instead all supported (for .all_filesystems)\n" );fprintf (stderr,"LTP_FORCE_SINGLE_FS_TYPE Testing only. The same as LTP_SINGLE_FS_TYPE but ignores test skiplist.\n" );fprintf (stderr,"LTP_TIMEOUT_MUL Timeout multiplier (must be a number >=1)\n" ); -fprintf (stderr,"LTP_RUNTIME_MUL Runtime multiplier (must be a number >=1)\n" ); +fprintf (stderr,"LTP_RUNTIME_MUL Runtime multiplier (must be a number >0)\n" );fprintf (stderr,"LTP_VIRT_OVERRIDE Overrides virtual machine detection (values: \"\"|kvm|microsoft|xen|zvm)\n" );fprintf (stderr,"TMPDIR Base directory for template directory (for .needs_tmpdir, default: %s)\n" ,TEMPDIR );fprintf (stderr,"\n" );@@ -1996,6 +2000,9 @@ void tst_run_tcases(int argc, char *argv[], struct tst_test *self)uname (&uval);tst_res (TINFO ,"Tested kernel: %s %s %s" , uval.release, uval.version, uval.machine); +if (tst_test->min_runtime) + tst_test->runtime =tst_test->min_runtime; +if (tst_test->runtime) context->runtime =multiply_runtime (tst_test->runtime);testcases/cve/cve-2016-7117.c +1
-1@@ -151,7 +151,7 @@ static struct tst_test test = { .test_all = run, .setup = setup, .cleanup = cleanup, - .runtime = 60, + .min_runtime = 60, .tags = (const struct tst_tag []) { {"linux-git" ,"34b88a68f26a" }, {"CVE" ,"2016-7117" },testcases/kernel/syscalls/preadv2/preadv203.c +1
-1@@ -278,6 +278,6 @@ static struct tst_test test = { .mntpoint = MNTPOINT, .mount_device = 1, .all_filesystems = 1, - .runtime = 60, + .min_runtime = 60, .needs_root = 1, };Signature · 5 lines
2.49.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Quoted 8 lines · Wei Gao
Quoted 1 line
+ * @min_runtime: Optional lower bound (in seconds) applied after
runtime is
Quoted 6 lines
+ * scaled by LTP_RUNTIME_MUL. If the scaled runtime is smaller + * than this value, it will be clamped up to min_runtime. + * This is useful for tests that require a minimum execution + * time to gather sufficient samples or trigger events (e.g., + * probabilistic or fuzzy synchronization tests). + * If not set, a default minimum of 1 second is enforced.
Quoted 4 lines
Base your logic once .min_runtime is set then .runtime value effectively becomes irrelevant.
So i guess we need mention it in above description?
Yes, combined with Martin's comment, we should at least ensure that .runtime is not overwritten when it exists.
+ if (tst_test->min_runtime && !tst_test->runtime)
+ tst_test->runtime = tst_test->min_runtime;
Also, I make the .min_runtime uses the value of .runtime (then remove .runtime) for those fuzzy_sync tests, that have a large 'pair.min_smaples' value.
But the default 1024 min_samples I still have no idea how long the .min_runtime needs. Maybe we can estimate and set .min_runtime on a slow system manually.
For others, which do not have strict requirements on execution time, we only need to set .runtime.
Signature · 5 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp
Quoted 19 lines · Li Wang
Quoted 8 lines · Wei Gao
Quoted 1 line
+ * @min_runtime: Optional lower bound (in seconds) applied after
runtime is
Quoted 6 lines
+ * scaled by LTP_RUNTIME_MUL. If the scaled runtime is smaller + * than this value, it will be clamped up to min_runtime. + * This is useful for tests that require a minimum execution + * time to gather sufficient samples or trigger events (e.g., + * probabilistic or fuzzy synchronization tests). + * If not set, a default minimum of 1 second is enforced.Quoted 4 lines
Base your logic once .min_runtime is set then .runtime value effectively becomes irrelevant.
So i guess we need mention it in above description?Yes, combined with Martin's comment, we should at least ensure that .runtime is not overwritten when it exists.
Review
Quoted 2 lines
(quoted diff) +2
-0+ if (tst_test->min_runtime && !tst_test->runtime) + tst_test->runtime = tst_test->min_runtime;
Thanks for clarification. I think above logic is better.
Quoted 6 lines
Also, I make the .min_runtime uses the value of .runtime (then remove .runtime) for those
fuzzy_synctests, that have a large 'pair.min_smaples' value.
Quoted 6 lines
But the default 1024
min_samplesI still have no idea how long the .min_runtime needs. Maybe we can estimate and set .min_runtime on a slow system manually.For others, which do not have strict requirements on execution time, we only need to set .runtime.
No experience on *good* value for .min_runtime. But i guess reusing value of .runtime is the safest option.
Quoted 4 lines
Signature · 3 lines
-- Regards, Li Wang
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi All,
Quoted 4 lines · Li Wang
But the default 1024
min_samplesI still have no idea how long the .min_runtime needs. Maybe we can estimate and set .min_runtime on a slow system manually.
I choose to run the fuzzy_sync tests on a Cortex-A55 CPU with only one core assigned, using the taskset command to simulate an extreme single-core execution scenario.
Tested on:
Cortex-A55, 1.2GHz
Linux 5.14, aarch64For example:
time taskset -c 0 ./cve-2016-7117 (hacked the test only do sampling)
This setup allows me to evaluate how long the sampling phase takes under constrained conditions.
Based on the observed sampling duration, I apply the following policy for setting .min_runtime:
1. If the sampling time is very short (less than 1 second), I simply set .min_runtime = 2.
2. If the sampling time is longer than 2 seconds but still less than .runtime, I set .min_runtime to twice the sampling time, while keeping .runtime unchanged.
3. If the sampling time exceeds .runtime, I also double the sampling time for .min_runtime, but leave .runtime as is.
Signature · 5 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp
Quoted 30 lines · Li Wang
Hi All,
Quoted 4 lines · Li Wang
But the default 1024
min_samplesI still have no idea how long the .min_runtime needs. Maybe we can estimate and set .min_runtime on a slow system manually.I choose to run the
fuzzy_synctests on a Cortex-A55 CPU with only one core assigned, using the taskset command to simulate an extreme single-core execution scenario.Tested on:
Cortex-A55, 1.2GHz Linux 5.14, aarch64For example:
time taskset -c 0 ./cve-2016-7117 (hacked the test only do sampling)This setup allows me to evaluate how long the sampling phase takes under constrained conditions.
Based on the observed sampling duration, I apply the following policy for setting .min_runtime:
1. If the sampling time is very short (less than 1 second), I simply set .min_runtime = 2.
Quoted 7 lines
2. If the sampling time is longer than 2 seconds but still less than .runtime, I set .min_runtime to twice the sampling time, while keeping .runtime unchanged.
3. If the sampling time exceeds .runtime, I also double the sampling time for .min_runtime, but leave .runtime as is.
3. If the sampling time exceeds .runtime, I simply set .min_runtime to the .runtime value and remove .runtime.
Signature · 5 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp