lib: ensure multiply_runtime() returns at least 1 second
PATCH 1/1 lib: ensure multiply_runtime() returns at least 1 second
This patch introduces support for fractional LTP_RUNTIME_MUL values in the range (0, 1], allowing fine-grained control over test runtimes. For example, setting LTP_RUNTIME_MUL=0.1 can reduce a 300s test to just 30s, which helps fit CI execution within 40 minutes.
To avoid skipping tests with low .runtime (e.g. nice05 with .runtime=3), this patch also modifies multiply_runtime() to always return at least 1. This ensures that even after floating-point multiplication, a test is never silently skipped due to runtime being rounded to zero.
Follow-up: a6a369c5ee ("lib: redefine the overall timeout logic of test")
--- lib/tst_test.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/tst_test.c +5 -2
@@ -626,11 +626,14 @@ static void parse_mul(float *mul, const char *env_name, float min, float max)static int multiply_runtime (unsigned int runtime) { +int adj;static float runtime_mul =-1 ;parse_mul (&runtime_mul,"LTP_RUNTIME_MUL" ,0.0099 ,100 ); -return runtime * runtime_mul; + adj = runtime * runtime_mul; + +return adj >0 ? adj :1 ; } static struct option {
@@ -662,7 +665,7 @@ static void print_help(void)fprintf (stderr,"LTP_SINGLE_FS_TYPE Specifies filesystem instead all supported (for .all_filesystems)\n" );fprintf (stderr,"LTP_FORCE_SINGLE_FS_TYPE Testing only. The same as LTP_SINGLE_FS_TYPE but ignores test skiplist.\n" );fprintf (stderr,"LTP_TIMEOUT_MUL Timeout multiplier (must be a number >=1)\n" ); -fprintf (stderr,"LTP_RUNTIME_MUL Runtime multiplier (must be a number >=1)\n" ); +fprintf (stderr,"LTP_RUNTIME_MUL Runtime multiplier (must be a number > 0)\n" );fprintf (stderr,"LTP_VIRT_OVERRIDE Overrides virtual machine detection (values: \"\"|kvm|microsoft|xen|zvm)\n" );fprintf (stderr,"TMPDIR Base directory for template directory (for .needs_tmpdir, default: %s)\n" ,TEMPDIR );fprintf (stderr,"\n" );
Signature · 5 lines
2.49.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
RFC PATCH 1/1 tst_test: Add flags to control runtime scaling in timing sensitive tests
Some tests, such as those using fuzzy synchronization or probabilistic triggers (e.g. preadv203), require sufficient runtime to collect meaningful results. These tests often rely on thresholds like fzsync_pair.min_samples, which must be met during the runtime window.
When LTP_RUNTIME_MUL is set to a value < 1.0 (commonly in CI environments), tests may run for too short a time, failing to gather enough samples and silently producing false negatives or unstable results.
This patch introduces the following test flags in enum tst_flag to provide fine-grained control over runtime and timeout scaling:
- TST_NO_RUNTIME_MUL:
Ignore LTP_RUNTIME_MUL and use the original .runtime value.
- TST_NO_TIMEOUT_MUL:
Ignore LTP_TIMEOUT_MUL and use the original .timeout value.
- TST_NO_FRACTIONAL_RUNTIME_MUL:
If LTP_RUNTIME_MUL is less than 1.0, round it up to 1.0 to preserve intended test duration.
This helps ensure that timing sensitive tests are not impacted by externally applied runtime reductions in CI or developer environments.
Example usage:
.flags = TST_NO_RUNTIME_MUL | TST_NO_TIMEOUT_MUL,
Follow-up: https://lists.linux.it/pipermail/ltp/2025-June/043960.html
---
Notes:
This patch also introduces two additional flags:
- TST_NO_RUNTIME_MUL
- TST_NO_TIMEOUT_MUL
These two flags are not currently used by any tests, but are included as examples of how the flags mechanism can be extended in the future to support more control scenarios.
Or, maybe we can drop them. I want to get suggestions here.
include/tst_test.h | 24 +++++++++++++++++++ lib/newlib_tests/tst_fuzzy_sync01.c | 1 + lib/newlib_tests/tst_fuzzy_sync02.c | 1 + lib/tst_test.c | 10 ++++++++ testcases/cve/cve-2016-7117.c | 1 + testcases/kernel/syscalls/preadv2/preadv203.c | 1 + 6 files changed, 38 insertions(+)
include/tst_test.h +24 -0
@@ -291,6 +291,26 @@ struct tst_fs {const char *min_kver; }; +/** +* enum tst_flag - Optional flags to control test behavior. +* These flags can be set in the .flags field of struct tst_test. +* +* @TST_NO_RUNTIME_MUL: +* Ignore the value of LTP_RUNTIME_MUL and use the unscaled runtime. +* +* @TST_NO_TIMEOUT_MUL: +* Ignore the value of LTP_TIMEOUT_MUL and use the unscaled timeout. +* +* @TST_NO_FRACTIONAL_RUNTIME_MUL: +* If LTP_RUNTIME_MUL is less than 1.0, ignore it (treat as 1.0). +* This prevents runtimes from being scaled down too low. +*/ +enum tst_flag { +TST_NO_RUNTIME_MUL =1 <<0 , +TST_NO_TIMEOUT_MUL =1 <<1 , +TST_NO_FRACTIONAL_RUNTIME_MUL =1 <<2 , +}; +/** * struct tst_test - A test description. *
@@ -516,6 +536,8 @@ struct tst_fs {* * @tags: A {} terminated array of test tags. See struct tst_tag for details. * +* @flags: Bitmask of enum tst_flag values to control runtime behavior. +* * @needs_cmds: A NULL terminated array of commands required for the test to run. * * @needs_cgroup_ver: If set the test will run only if the specified cgroup
@@ -608,6 +630,8 @@ struct tst_fs {const struct tst_tag *tags; +unsigned int flags; +const char *const *needs_cmds;const enum tst_cg_ver needs_cgroup_ver;
lib/newlib_tests/tst_fuzzy_sync01.c +1 -0
@@ -247,4 +247,5 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .runtime = 150, + .flags = TST_NO_FRACTIONAL_RUNTIME_MUL, };
lib/newlib_tests/tst_fuzzy_sync02.c +1 -0
@@ -224,4 +224,5 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .runtime = 150, + .flags = TST_NO_FRACTIONAL_RUNTIME_MUL, };
lib/tst_test.c +10 -0
@@ -631,6 +631,13 @@ static int multiply_runtime(unsigned int runtime)parse_mul (&runtime_mul,"LTP_RUNTIME_MUL" ,0.0099 ,100 ); +if ((tst_test->flags &TST_NO_RUNTIME_MUL )) + runtime_mul =1 ; + +if ((tst_test->flags &TST_NO_FRACTIONAL_RUNTIME_MUL ) && + (runtime_mul <1.0 )) + runtime_mul =1 ; + adj = runtime * runtime_mul;return adj >0 ? adj :1 ;
@@ -1801,6 +1808,9 @@ unsigned int tst_multiply_timeout(unsigned int timeout)if (tst_has_slow_kconfig ()) timeout *=4 ; +if ((tst_test->flags &TST_NO_TIMEOUT_MUL )) + timeout_mul =1 ; +return timeout * timeout_mul; }
testcases/cve/cve-2016-7117.c +1 -0
@@ -152,6 +152,7 @@ static struct tst_test test = { .setup = setup, .cleanup = cleanup, .runtime = 60, + .flags = TST_NO_FRACTIONAL_RUNTIME_MUL, .tags = (const struct tst_tag []) { {"linux-git" ,"34b88a68f26a" }, {"CVE" ,"2016-7117" },
testcases/kernel/syscalls/preadv2/preadv203.c +1 -0
@@ -284,5 +284,6 @@ static struct tst_test test = { .mount_device = 1, .all_filesystems = 1, .runtime = 60, + .flags = TST_NO_FRACTIONAL_RUNTIME_MUL, .needs_root = 1, };
Signature · 5 lines
2.49.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [RFC PATCH] tst_test: Add flags to control runtime scaling in timing sensitive tests
Quoted 15 lines · Li Wang via ltp
Some tests, such as those using fuzzy synchronization or probabilistic triggers (e.g. preadv203), require sufficient runtime to collect meaningful results. These tests often rely on thresholds like
fzsync_pair.min_samples, which must be met during the runtime window.When
LTP_RUNTIME_MULis set to a value < 1.0 (commonly in CI environments), tests may run for too short a time, failing to gather enough samples and silently producing false negatives or unstable results.This patch introduces the following test flags in
enum tst_flagto provide fine-grained control over runtime and timeout scaling:-
TST_NO_RUNTIME_MUL:
IgnoreLTP_RUNTIME_MULand use the original .runtime value.
Quoted 6 lines
-
TST_NO_TIMEOUT_MUL:
IgnoreLTP_TIMEOUT_MULand use the original .timeout value.-
TST_NO_FRACTIONAL_RUNTIME_MUL:
IfLTP_RUNTIME_MULis less than 1.0, round it up to 1.0 to preserve intended test duration.
I think it'd make more sense to define a minimal runtime in the LTP library (e.g. 1 second) and allow tests to override the value with tst_test.min_runtime. Then you can apply fractional LTP_RUNTIME_MUL safely to any test.
Signature · 10 lines
Martin Doucha [email protected] SW Quality Engineer SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [RFC PATCH] tst_test: Add flags to control runtime scaling in timing sensitive tests
Quoted 26 lines · Martin Doucha
Quoted 1 line · Li Wang via ltp
Some tests, such as those using fuzzy synchronization or probabilistic
triggers
Quoted 1 line
(e.g. preadv203), require sufficient runtime to collect meaningful
results.
Quoted 1 line
These tests often rely on thresholds like
fzsync_pair.min_samples,which must
Quoted 3 lines
be met during the runtime window.
When
LTP_RUNTIME_MULis set to a value < 1.0 (commonly in CIenvironments),
Quoted 4 lines
tests may run for too short a time, failing to gather enough samples and silently producing false negatives or unstable results.
This patch introduces the following test flags in
enum tst_flagtoprovide
Quoted 10 lines
fine-grained control over runtime and timeout scaling:
-
TST_NO_RUNTIME_MUL:
IgnoreLTP_RUNTIME_MULand use the original .runtime value.-
TST_NO_TIMEOUT_MUL:
IgnoreLTP_TIMEOUT_MULand use the original .timeout value.-
TST_NO_FRACTIONAL_RUNTIME_MUL:
IfLTP_RUNTIME_MULis less than 1.0, round it up to 1.0 to
Quoted 8 lines
preserve
Quoted 1 line
intended test duration.
I think it'd make more sense to define a minimal runtime in the LTP library (e.g. 1 second) and allow tests to override the value with tst_test.min_runtime. Then you can apply fractional
LTP_RUNTIME_MULsafely to any test.
That has been achieved in
https://lists.linux.it/pipermail/ltp/2025-June/043960.html.
which avoids .runtime being rounded to zero with fractional LTP_RUNTIME_MUL.
This patch introduces TST_NO_FRACTIONAL_RUNTIME_MUL, mainly to solve the problem that some special tests (e.g. fuzzy-lib, preadv203) should not reduce the runtime during execution. Setting a minimum runtime (e.g. 1 second)
does not meet the requirements.
Signature · 5 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [RFC PATCH] tst_test: Add flags to control runtime scaling in timing sensitive tests
Quoted 44 lines · Li Wang
Quoted 26 lines · Martin Doucha
Quoted 1 line · Li Wang via ltp
Some tests, such as those using fuzzy synchronization or probabilistic
triggers
Quoted 1 line
(e.g. preadv203), require sufficient runtime to collect meaningful
results.
Quoted 1 line
These tests often rely on thresholds like
fzsync_pair.min_samples,which must
Quoted 3 lines
be met during the runtime window.
When
LTP_RUNTIME_MULis set to a value < 1.0 (commonly in CIenvironments),
Quoted 4 lines
tests may run for too short a time, failing to gather enough samples and silently producing false negatives or unstable results.
This patch introduces the following test flags in
enum tst_flagtoprovide
Quoted 10 lines
fine-grained control over runtime and timeout scaling:
-
TST_NO_RUNTIME_MUL:
IgnoreLTP_RUNTIME_MULand use the original .runtime value.-
TST_NO_TIMEOUT_MUL:
IgnoreLTP_TIMEOUT_MULand use the original .timeout value.-
TST_NO_FRACTIONAL_RUNTIME_MUL:
IfLTP_RUNTIME_MULis less than 1.0, round it up to 1.0 toQuoted 8 lines
preserve
Quoted 1 line
intended test duration.
I think it'd make more sense to define a minimal runtime in the LTP library (e.g. 1 second) and allow tests to override the value with tst_test.min_runtime. Then you can apply fractional
LTP_RUNTIME_MULsafely to any test.That has been achieved in
https://lists.linux.it/pipermail/ltp/2025-June/043960.html.
which avoids .runtime being rounded to zero with fractionalLTP_RUNTIME_MUL.
Quoted 7 lines
This patch introduces
TST_NO_FRACTIONAL_RUNTIME_MUL, mainly to solve the problem that some special tests (e.g. fuzzy-lib, preadv203) should
not reduce the runtime during execution. Setting a minimum runtime (e.g. 1 second)
does not meet the requirements.
Oh, it suddenly dawned on me what you were really suggesting.
Use the default minimum runtime of 1 second, and override it with an arbitrarily large value if necessary, and ensure that the result of (runtime * running_mul) is never less than the new .min_runtime value.
This is an elegant approach, thanks!
Signature · 5 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp
Please ignore this patch, refactored into another one:
https://lists.linux.it/pipermail/ltp/2025-June/043983.html
Signature · 5 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp