Skip to contentSkip to conversation

lib: ensure multiply_runtime() returns at least 1 second

PATCH In review Li Wang via ltp 2 participants 6 messages Jun 20, 2025 – Jun 23, 2025 Library code

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 lib: ensure multiply_runtime() returns at least 1 second

This patch introduces support for fractional LTP_RUNTIME_MUL values in the range (0, 1], allowing fine-grained control over test runtimes. For example, setting LTP_RUNTIME_MUL=0.1 can reduce a 300s test to just 30s, which helps fit CI execution within 40 minutes.

To avoid skipping tests with low .runtime (e.g. nice05 with .runtime=3), this patch also modifies multiply_runtime() to always return at least 1. This ensures that even after floating-point multiplication, a test is never silently skipped due to runtime being rounded to zero.

Follow-up: a6a369c5ee ("lib: redefine the overall timeout logic of test")

Cc Ian Wienand <[email protected]>
---
 lib/tst_test.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/tst_test.c +5 -2
index 495e022f7..467299e37 100644
@@ -626,11 +626,14 @@ static void parse_mul(float *mul, const char *env_name, float min, float max)
 
 static int multiply_runtime(unsigned int runtime)
 {
+	int adj;
 	static float runtime_mul = -1;
 
 	parse_mul(&runtime_mul, "LTP_RUNTIME_MUL", 0.0099, 100);
 
-	return runtime * runtime_mul;
+	adj = runtime * runtime_mul;
+
+	return adj > 0 ? adj : 1;
 }
 
 static struct option {
@@ -662,7 +665,7 @@ static void print_help(void)
 	fprintf(stderr, "LTP_SINGLE_FS_TYPE       Specifies filesystem instead all supported (for .all_filesystems)\n");
 	fprintf(stderr, "LTP_FORCE_SINGLE_FS_TYPE Testing only. The same as LTP_SINGLE_FS_TYPE but ignores test skiplist.\n");
 	fprintf(stderr, "LTP_TIMEOUT_MUL          Timeout multiplier (must be a number >=1)\n");
-	fprintf(stderr, "LTP_RUNTIME_MUL          Runtime multiplier (must be a number >=1)\n");
+	fprintf(stderr, "LTP_RUNTIME_MUL          Runtime multiplier (must be a number > 0)\n");
 	fprintf(stderr, "LTP_VIRT_OVERRIDE        Overrides virtual machine detection (values: \"\"|kvm|microsoft|xen|zvm)\n");
 	fprintf(stderr, "TMPDIR                   Base directory for template directory (for .needs_tmpdir, default: %s)\n", TEMPDIR);
 	fprintf(stderr, "\n");
Signature · 5 lines
2.49.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

RFC PATCH 1/1 tst_test: Add flags to control runtime scaling in timing sensitive tests

Some tests, such as those using fuzzy synchronization or probabilistic triggers (e.g. preadv203), require sufficient runtime to collect meaningful results. These tests often rely on thresholds like fzsync_pair.min_samples, which must be met during the runtime window.

When LTP_RUNTIME_MUL is set to a value < 1.0 (commonly in CI environments), tests may run for too short a time, failing to gather enough samples and silently producing false negatives or unstable results.

This patch introduces the following test flags in enum tst_flag to provide fine-grained control over runtime and timeout scaling:

- TST_NO_RUNTIME_MUL:
Ignore LTP_RUNTIME_MUL and use the original .runtime value.

- TST_NO_TIMEOUT_MUL:
Ignore LTP_TIMEOUT_MUL and use the original .timeout value.

- TST_NO_FRACTIONAL_RUNTIME_MUL:
If LTP_RUNTIME_MUL is less than 1.0, round it up to 1.0 to preserve intended test duration.

This helps ensure that timing sensitive tests are not impacted by externally applied runtime reductions in CI or developer environments.

Example usage:

.flags = TST_NO_RUNTIME_MUL | TST_NO_TIMEOUT_MUL,

Follow-up: https://lists.linux.it/pipermail/ltp/2025-June/043960.html

---

Notes:
This patch also introduces two additional flags:

- TST_NO_RUNTIME_MUL
- TST_NO_TIMEOUT_MUL

These two flags are not currently used by any tests, but are included as examples of how the flags mechanism can be extended in the future to support more control scenarios.

Or, maybe we can drop them. I want to get suggestions here.

 include/tst_test.h                            | 24 +++++++++++++++++++
 lib/newlib_tests/tst_fuzzy_sync01.c           |  1 +
 lib/newlib_tests/tst_fuzzy_sync02.c           |  1 +
 lib/tst_test.c                                | 10 ++++++++
 testcases/cve/cve-2016-7117.c                 |  1 +
 testcases/kernel/syscalls/preadv2/preadv203.c |  1 +
 6 files changed, 38 insertions(+)
include/tst_test.h +24 -0
index 6fd8cbae3..72470d40b 100644
@@ -291,6 +291,26 @@ struct tst_fs {
 	const char *min_kver;
 };
 
+/**
+ * enum tst_flag - Optional flags to control test behavior.
+ * These flags can be set in the .flags field of struct tst_test.
+ *
+ * @TST_NO_RUNTIME_MUL:
+ *   Ignore the value of LTP_RUNTIME_MUL and use the unscaled runtime.
+ *
+ * @TST_NO_TIMEOUT_MUL:
+ *   Ignore the value of LTP_TIMEOUT_MUL and use the unscaled timeout.
+ *
+ * @TST_NO_FRACTIONAL_RUNTIME_MUL:
+ *   If LTP_RUNTIME_MUL is less than 1.0, ignore it (treat as 1.0).
+ *   This prevents runtimes from being scaled down too low.
+ */
+enum tst_flag {
+	TST_NO_RUNTIME_MUL = 1 << 0,
+	TST_NO_TIMEOUT_MUL = 1 << 1,
+	TST_NO_FRACTIONAL_RUNTIME_MUL = 1 << 2,
+};
+
 /**
  * struct tst_test - A test description.
  *
@@ -516,6 +536,8 @@ struct tst_fs {
  *
  * @tags: A {} terminated array of test tags. See struct tst_tag for details.
  *
+ * @flags: Bitmask of enum tst_flag values to control runtime behavior.
+ *
  * @needs_cmds: A NULL terminated array of commands required for the test to run.
  *
  * @needs_cgroup_ver: If set the test will run only if the specified cgroup
@@ -608,6 +630,8 @@ struct tst_fs {
 
 	const struct tst_tag *tags;
 
+	unsigned int flags;
+
 	const char *const *needs_cmds;
 
 	const enum tst_cg_ver needs_cgroup_ver;
lib/newlib_tests/tst_fuzzy_sync01.c +1 -0
index b1390f460..a527ba5eb 100644
@@ -247,4 +247,5 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.runtime = 150,
+	.flags = TST_NO_FRACTIONAL_RUNTIME_MUL,
 };
lib/newlib_tests/tst_fuzzy_sync02.c +1 -0
index bc079f6ff..689a0b090 100644
@@ -224,4 +224,5 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.runtime = 150,
+	.flags = TST_NO_FRACTIONAL_RUNTIME_MUL,
 };
lib/tst_test.c +10 -0
index 467299e37..b2c2d8d64 100644
@@ -631,6 +631,13 @@ static int multiply_runtime(unsigned int runtime)
 
 	parse_mul(&runtime_mul, "LTP_RUNTIME_MUL", 0.0099, 100);
 
+	if ((tst_test->flags & TST_NO_RUNTIME_MUL))
+		runtime_mul = 1;
+
+	if ((tst_test->flags & TST_NO_FRACTIONAL_RUNTIME_MUL) &&
+			(runtime_mul < 1.0))
+		runtime_mul = 1;
+
 	adj = runtime * runtime_mul;
 
 	return adj > 0 ? adj : 1;
@@ -1801,6 +1808,9 @@ unsigned int tst_multiply_timeout(unsigned int timeout)
 	if (tst_has_slow_kconfig())
 		timeout *= 4;
 
+	if ((tst_test->flags & TST_NO_TIMEOUT_MUL))
+		timeout_mul = 1;
+
 	return timeout * timeout_mul;
 }
 
testcases/cve/cve-2016-7117.c +1 -0
index dbec2b28b..4000b9bc1 100644
@@ -152,6 +152,7 @@ static struct tst_test test = {
 	.setup = setup,
 	.cleanup = cleanup,
 	.runtime = 60,
+	.flags = TST_NO_FRACTIONAL_RUNTIME_MUL,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "34b88a68f26a"},
 		{"CVE", "2016-7117"},
testcases/kernel/syscalls/preadv2/preadv203.c +1 -0
index 128e7ae34..c5f7edda1 100644
@@ -284,5 +284,6 @@ static struct tst_test test = {
 	.mount_device = 1,
 	.all_filesystems = 1,
 	.runtime = 60,
+	.flags = TST_NO_FRACTIONAL_RUNTIME_MUL,
 	.needs_root = 1,
 };
Signature · 5 lines
2.49.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw

Re: [LTP] [RFC PATCH] tst_test: Add flags to control runtime scaling in timing sensitive tests

Quoted 15 lines · Li Wang via ltp

Some tests, such as those using fuzzy synchronization or probabilistic triggers (e.g. preadv203), require sufficient runtime to collect meaningful results. These tests often rely on thresholds like fzsync_pair.min_samples, which must be met during the runtime window.

When LTP_RUNTIME_MUL is set to a value < 1.0 (commonly in CI environments), tests may run for too short a time, failing to gather enough samples and silently producing false negatives or unstable results.

This patch introduces the following test flags in enum tst_flag to provide fine-grained control over runtime and timeout scaling:

- TST_NO_RUNTIME_MUL:
Ignore LTP_RUNTIME_MUL and use the original .runtime value.

Quoted 6 lines

- TST_NO_TIMEOUT_MUL:
Ignore LTP_TIMEOUT_MUL and use the original .timeout value.

- TST_NO_FRACTIONAL_RUNTIME_MUL:
If LTP_RUNTIME_MUL is less than 1.0, round it up to 1.0 to preserve intended test duration.

I think it'd make more sense to define a minimal runtime in the LTP library (e.g. 1 second) and allow tests to override the value with tst_test.min_runtime. Then you can apply fractional LTP_RUNTIME_MUL safely to any test.

Signature · 10 lines
Martin Doucha   [email protected]
SW Quality Engineer
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#3 lore raw

Re: [LTP] [RFC PATCH] tst_test: Add flags to control runtime scaling in timing sensitive tests

Quoted 26 lines · Martin Doucha
Quoted 1 line · Li Wang via ltp

Some tests, such as those using fuzzy synchronization or probabilistic

triggers

Quoted 1 line

(e.g. preadv203), require sufficient runtime to collect meaningful

results.

Quoted 1 line

These tests often rely on thresholds like fzsync_pair.min_samples,

which must

Quoted 3 lines

be met during the runtime window.

When LTP_RUNTIME_MUL is set to a value < 1.0 (commonly in CI

environments),

Quoted 4 lines

tests may run for too short a time, failing to gather enough samples and silently producing false negatives or unstable results.

This patch introduces the following test flags in enum tst_flag to

provide

Quoted 10 lines

fine-grained control over runtime and timeout scaling:

- TST_NO_RUNTIME_MUL:
Ignore LTP_RUNTIME_MUL and use the original .runtime value.

- TST_NO_TIMEOUT_MUL:
Ignore LTP_TIMEOUT_MUL and use the original .timeout value.

- TST_NO_FRACTIONAL_RUNTIME_MUL:
If LTP_RUNTIME_MUL is less than 1.0, round it up to 1.0 to

Quoted 8 lines

preserve

Quoted 1 line

intended test duration.

I think it'd make more sense to define a minimal runtime in the LTP library (e.g. 1 second) and allow tests to override the value with tst_test.min_runtime. Then you can apply fractional LTP_RUNTIME_MUL safely to any test.

That has been achieved in
https://lists.linux.it/pipermail/ltp/2025-June/043960.html.
which avoids .runtime being rounded to zero with fractional LTP_RUNTIME_MUL.

This patch introduces TST_NO_FRACTIONAL_RUNTIME_MUL, mainly to solve the problem that some special tests (e.g. fuzzy-lib, preadv203) should not reduce the runtime during execution. Setting a minimum runtime (e.g. 1 second)
does not meet the requirements.

Signature · 5 lines
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#4 lore raw

Re: [LTP] [RFC PATCH] tst_test: Add flags to control runtime scaling in timing sensitive tests

Quoted 44 lines · Li Wang
Quoted 26 lines · Martin Doucha
Quoted 1 line · Li Wang via ltp

Some tests, such as those using fuzzy synchronization or probabilistic

triggers

Quoted 1 line

(e.g. preadv203), require sufficient runtime to collect meaningful

results.

Quoted 1 line

These tests often rely on thresholds like fzsync_pair.min_samples,

which must

Quoted 3 lines

be met during the runtime window.

When LTP_RUNTIME_MUL is set to a value < 1.0 (commonly in CI

environments),

Quoted 4 lines

tests may run for too short a time, failing to gather enough samples and silently producing false negatives or unstable results.

This patch introduces the following test flags in enum tst_flag to

provide

Quoted 10 lines

fine-grained control over runtime and timeout scaling:

- TST_NO_RUNTIME_MUL:
Ignore LTP_RUNTIME_MUL and use the original .runtime value.

- TST_NO_TIMEOUT_MUL:
Ignore LTP_TIMEOUT_MUL and use the original .timeout value.

- TST_NO_FRACTIONAL_RUNTIME_MUL:
If LTP_RUNTIME_MUL is less than 1.0, round it up to 1.0 to

Quoted 8 lines

preserve

Quoted 1 line

intended test duration.

I think it'd make more sense to define a minimal runtime in the LTP library (e.g. 1 second) and allow tests to override the value with tst_test.min_runtime. Then you can apply fractional LTP_RUNTIME_MUL safely to any test.

That has been achieved in
https://lists.linux.it/pipermail/ltp/2025-June/043960.html.
which avoids .runtime being rounded to zero with fractional LTP_RUNTIME_MUL.

Quoted 7 lines

This patch introduces TST_NO_FRACTIONAL_RUNTIME_MUL, mainly to solve the problem that some special tests (e.g. fuzzy-lib, preadv203) should
not reduce the runtime during execution. Setting a minimum runtime (e.g. 1 second)
does not meet the requirements.

Oh, it suddenly dawned on me what you were really suggesting.

Use the default minimum runtime of 1 second, and override it with an arbitrarily large value if necessary, and ensure that the result of (runtime * running_mul) is never less than the new .min_runtime value.

This is an elegant approach, thanks!

Signature · 5 lines
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp