tst_fuzzy_sync: refine code and comment
PATCH 1/3 tst_fuzzy_sync: refine code and comment
I just polished some things preventing me from understanding the test logic. Hopefully, it makes things easier to read.
--- lib/newlib_tests/tst_fuzzy_sync01.c | 22 ++++++------ lib/newlib_tests/tst_fuzzy_sync02.c | 53 ++++++++++++++++++++++++----- 2 files changed, 55 insertions(+), 20 deletions(-)
lib/newlib_tests/tst_fuzzy_sync01.c +11 -11
@@ -10,7 +10,7 @@* * We make the simplifying assumptions that: * - Each thread contains a single contiguous critical section. -* - The threads only interact through a single variable. +* - The threads only interact through a single variable(H: Hit). * - The various timings are constant except for variations introduced * by the environment. *
@@ -50,9 +50,9 @@* range is required. * * When entering their critical sections, both threads increment the -* 'c' counter variable atomically. They both also increment it when -* leaving their critical sections. We record the value of 'c' when A -* increments it. From the recorded values of 'c' we can deduce if the +* 'H' counter variable atomically. They both also increment it when +* leaving their critical sections. We record the value of 'H' when A +* increments it. From the recorded values of 'H' we can deduce if the * critical sections overlap and their ordering. * * Start (cs) | End (ct) | Ordering
@@ -90,7 +90,7 @@ struct race {const struct window b; }; -static int c; +static int H ;static struct tst_fzsync_pair pair;static const struct race races[] = {
@@ -162,15 +162,15 @@ static void *worker(void *v)const struct window b = races[i].b;while (tst_fzsync_run_b (&pair)) { -if (tst_atomic_load (&c)) +if (tst_atomic_load (&H ))tst_brk (TBROK ,"Counter should now be zero" );tst_fzsync_start_race_b (&pair);delay (b.critical_s); -tst_atomic_add_return (1 , &c); +tst_atomic_add_return (1 , &H );delay (b.critical_t); -tst_atomic_add_return (1 , &c); +tst_atomic_add_return (1 , &H );delay (b.return_t);tst_fzsync_end_race_b (&pair);
@@ -192,9 +192,9 @@ static void run(unsigned int i)tst_fzsync_start_race_a (&pair);delay (a.critical_s); - cs =tst_atomic_add_return (1 , &c); + cs =tst_atomic_add_return (1 , &H );delay (a.critical_t); - ct =tst_atomic_add_return (1 , &c); + ct =tst_atomic_add_return (1 , &H );delay (a.return_t);tst_fzsync_end_race_a (&pair);
@@ -206,7 +206,7 @@ static void run(unsigned int i)else critical++; - r =tst_atomic_add_return (-4 , &c); + r =tst_atomic_add_return (-4 , &H );if (r)tst_brk (TBROK ,"cs = %d, ct = %d, r = %d" , cs, ct, r);
lib/newlib_tests/tst_fuzzy_sync02.c +44 -9
99 lines not on this page · raw message
Signature · 5 lines
2.47.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 2/3 tst_fuzzy_sync: ingnore test result if loop termination occurs
This patch updates the behavior of tst_fuzzy_sync tests to address scenarios where test results may be unreliable due to resource constraints.
Specifically, if pair->exit is set (indicating exceeded loop termination), the test result is ignored, and a TCONF status is returned. This ensures invalid results are not falsely reported as failures or successes.
tst_fuzzy_sync.h:654: TINFO: Exceeded execution loops, requesting exit /lib/newlib_tests/tst_fuzzy_sync02.c:164: TFAIL: 2| =:15
--- lib/newlib_tests/tst_fuzzy_sync01.c | 21 +++++++++++++++++++++ lib/newlib_tests/tst_fuzzy_sync02.c | 21 +++++++++++++++++++++ 2 files changed, 42 insertions(+)
lib/newlib_tests/tst_fuzzy_sync01.c +21 -0
@@ -212,10 +212,31 @@ static void run(unsigned int i)if (critical >100 ) {tst_fzsync_pair_cleanup (&pair); +tst_atomic_store (0 , &pair.exit);break ; } } +/* +* If `pair->exit` is true, the test may fail to meet expected +* results due to resource constraints in shared CI environments +* (e.g., GitHub Actions). Limited control over CPU allocation +* can cause delays or interruptions in CPU time slices due to +* contention with other jobs. +* +* Binding the test to a single CPU core (e.g., via `taskset -c 0`) +* can worsen this by increasing contention, leading to performance +* degradation and premature loop termination. +* +* To ensure valid and reliable results in scenarios (e.g., HW, VM, CI), +* it is best to ingore test result when loop termination occurs, +* avoiding unnecessary false postive. +*/ +if (pair.exit) { +tst_res (TCONF ,"Test may not be able to generate a valid result" ); +return ; + } + tst_res(critical > 50 ? TPASS : TFAIL, "acs:%-2d act:%-2d art:%-2d | =:%-4d -:%-4d +:%-4d", a.critical_s, a.critical_t, a.return_t,
lib/newlib_tests/tst_fuzzy_sync02.c +21 -0
@@ -192,10 +192,31 @@ static void run(unsigned int i)if (critical >100 ) {tst_fzsync_pair_cleanup (&pair); +tst_atomic_store (0 , &pair.exit);break ; } } +/* +* If `pair->exit` is true, the test may fail to meet expected +* results due to resource constraints in shared CI environments +* (e.g., GitHub Actions). Limited control over CPU allocation +* can cause delays or interruptions in CPU time slices due to +* contention with other jobs. +* +* Binding the test to a single CPU core (e.g., via `taskset -c 0`) +* can worsen this by increasing contention, leading to performance +* degradation and premature loop termination. +* +* To ensure valid and reliable results in scenarios (e.g., HW, VM, CI), +* it is best to ingore test result when loop termination occurs, +* avoiding unnecessary false postive. +*/ +if (pair.exit) { +tst_res (TCONF ,"Test may not be able to generate a valid result" ); +return ; + } +tst_res (critical >50 ?TPASS :TFAIL ,"%d| =:%-4d" , i, critical); }
Signature · 5 lines
2.47.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 2/3] tst_fuzzy_sync: ingnore test result if loop termination occurs
Hi Li,
Quoted 9 lines
This patch updates the behavior of
tst_fuzzy_synctests to address scenarios where test results may be unreliable due to resource constraints.Specifically, if
pair->exitis set (indicating exceeded loop termination), the test result is ignored, and a TCONF status is returned. This ensures invalid results are not falsely reported as failures or successes.tst_fuzzy_sync.h:654: TINFO: Exceeded execution loops, requesting exit
/lib/newlib_tests/tst_fuzzy_sync02.c:164: TFAIL: 2| =:15
LGTM, good catch.
Below are noted typos, could you please fix them before merge?
Quoted 22 lines
Signed-off-by Li Wang <[email protected]>--- lib/newlib_tests/tst_fuzzy_sync01.c | 21 +++++++++++++++++++++ lib/newlib_tests/tst_fuzzy_sync02.c | 21 +++++++++++++++++++++ 2 files changed, 42 insertions(+)lib/newlib_tests/tst_fuzzy_sync01.c +4
-0@@ -212,10 +212,31 @@ static void run(unsigned int i)if (critical >100 ) {tst_fzsync_pair_cleanup (&pair); +tst_atomic_store (0 , &pair.exit);break ; } } +/* +* If `pair->exit` is true, the test may fail to meet expected +* results due to resource constraints in shared CI environments
lib/newlib_tests/tst_fuzzy_sync01.c static void run(unsigned int i) · line 232
Quoted 10 lines
lib/newlib_tests/tst_fuzzy_sync01.c +10
-0+* (e.g., GitHub Actions). Limited control over CPU allocation +* can cause delays or interruptions in CPU time slices due to +* contention with other jobs. +* +* Binding the test to a single CPU core (e.g., via `taskset -c 0`) +* can worsen this by increasing contention, leading to performance +* degradation and premature loop termination. +* +* To ensure valid and reliable results in scenarios (e.g., HW, VM, CI), +* it is best to ingore test result when loop termination occurs,
s/ingore/ignore/
lib/newlib_tests/tst_fuzzy_sync01.c static void run(unsigned int i) · line 233
Quoted 1 line
lib/newlib_tests/tst_fuzzy_sync01.c +1
-0+* avoiding unnecessary false postive.
s/postive/positive/
Quoted 25 lines
lib/newlib_tests/tst_fuzzy_sync01.c +6
-0+*/ +if (pair.exit) { +tst_res (TCONF ,"Test may not be able to generate a valid result" ); +return ; + } + tst_res(critical > 50 ? TPASS : TFAIL, "acs:%-2d act:%-2d art:%-2d | =:%-4d -:%-4d +:%-4d", a.critical_s, a.critical_t, a.return_t,lib/newlib_tests/tst_fuzzy_sync02.c +4
-0@@ -192,10 +192,31 @@ static void run(unsigned int i)if (critical >100 ) {tst_fzsync_pair_cleanup (&pair); +tst_atomic_store (0 , &pair.exit);break ; } } +/* +* If `pair->exit` is true, the test may fail to meet expected +* results due to resource constraints in shared CI environments
lib/newlib_tests/tst_fuzzy_sync02.c static void run(unsigned int i) · line 212
Quoted 10 lines
lib/newlib_tests/tst_fuzzy_sync02.c +10
-0+* (e.g., GitHub Actions). Limited control over CPU allocation +* can cause delays or interruptions in CPU time slices due to +* contention with other jobs. +* +* Binding the test to a single CPU core (e.g., via `taskset -c 0`) +* can worsen this by increasing contention, leading to performance +* degradation and premature loop termination. +* +* To ensure valid and reliable results in scenarios (e.g., HW, VM, CI), +* it is best to ingore test result when loop termination occurs,
s/ingore/ignore/
lib/newlib_tests/tst_fuzzy_sync02.c static void run(unsigned int i) · line 213
Quoted 1 line
lib/newlib_tests/tst_fuzzy_sync02.c +1
-0+* avoiding unnecessary false postive.
s/postive/positive/
Also there is original typo from Richie:
s/Emperically/Empirically/
Kind regards,
Petr
Quoted 5 lines
lib/newlib_tests/tst_fuzzy_sync02.c +5
-0+*/ +if (pair.exit) { +tst_res (TCONF ,"Test may not be able to generate a valid result" ); +return ; + }
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 3/3 newlib_test: enable all tst_fuzzy_sync tests
In reply to Li Wang---
Notes:
This patch based on Peter's work:
Message-ID: <[email protected]>
CI: https://github.com/wangli5665/ltp/actions/runs/12452324779/lib/newlib_tests/runtest.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
lib/newlib_tests/runtest.sh +3 -1
@@ -2,7 +2,7 @@ # Copyright (c) 2021-2024 Petr Vorel <[email protected]> # TODO "unknown failure, exit code": test_assert test08 tst_cgroup01 tst_cgroup02 tst_res_flags variant -# TODO TFAIL: tst_fuzzy_sync01 tst_fuzzy_sync02 test_macros0[1-6] test23 test26 +# TODO TFAIL: test_macros0[1-6] test23 test26 # TODO TBROK: test_exec_child test_kconfig01 test_kconfig02 tst_needs_cmds04 tst_needs_cmds05 test_runtime02 test01 test02 test03 test04 test06 test11 test13 test22 test25 tst_safe_fileops # TODO TWARN: test_guarded_buf test14 tst_capability01 tst_print_result LTP_C_API_TESTS="${LTP_C_API_TESTS:-
@@ -20,6 +20,8 @@ tst_bool_expr tst_capability02 tst_device tst_expiration_timer +tst_fuzzy_sync01 +tst_fuzzy_sync02 tst_fuzzy_sync03 tst_needs_cmds0[1-36-8] tst_res_hexd
Signature · 5 lines
2.47.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 3/3] newlib_test: enable all tst_fuzzy_sync tests
Hi Li,
Quoted 14 lines
Notes:
This patch based on Peter's work: Message-ID: <[email protected]> CI: https://github.com/wangli5665/ltp/actions/runs/12452324779/lib/newlib_tests/runtest.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)lib/newlib_tests/runtest.sh +0
-0
lib/newlib_tests/runtest.sh line 5
Quoted 4 lines
lib/newlib_tests/runtest.sh +0
-1@@ -2,7 +2,7 @@ # Copyright (c) 2021-2024 Petr Vorel <[email protected]> # TODO "unknown failure, exit code": test_assert test08 tst_cgroup01 tst_cgroup02 tst_res_flags variant -# TODO TFAIL: tst_fuzzy_sync01 tst_fuzzy_sync02 test_macros0[1-6] test23 test26
Ah, I should have mark it as tst_fuzzy_sync0[12]
Anyway, feel free to merge before I merge my patchset, I have no problem to rebase mine.
Kind regards,
Petr
Quoted 13 lines
lib/newlib_tests/runtest.sh +3
-0+# TODO TFAIL: test_macros0[1-6] test23 test26 # TODO TBROK: test_exec_child test_kconfig01 test_kconfig02 tst_needs_cmds04 tst_needs_cmds05 test_runtime02 test01 test02 test03 test04 test06 test11 test13 test22 test25 tst_safe_fileops # TODO TWARN: test_guarded_buf test14 tst_capability01 tst_print_result LTP_C_API_TESTS="${LTP_C_API_TESTS:-@@ -20,6 +20,8 @@ tst_bool_expr tst_capability02 tst_device tst_expiration_timer +tst_fuzzy_sync01 +tst_fuzzy_sync02 tst_fuzzy_sync03 tst_needs_cmds0[1-36-8] tst_res_hexd
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Li,
Quoted 2 lines
I just polished some things preventing me from understanding the test logic. Hopefully, it makes things easier to read.
LGTM, thanks!
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Li,
would you mind to change before merge trailing "\*/" to "*/" (removing "/") in both files and extra blank line with "*" in tst_fuzzy_sync02.c?
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Quoted 6 lines · Petr Vorel
Hi Li,
would you mind to change before merge trailing "\*/" to "*/" (removing "/") in both files and extra blank line with "*" in
tst_fuzzy_sync02.c?
Thanks for the review, I merge the patch set (with your suggestions).
Signature · 5 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp