Skip to contentSkip to conversation

ltp-pan: Report failure if testcases failed to run

RFC PATCH In review Sagi Nakash 3 participants 5 messages Nov 30, 2021 – Jan 21, 2022

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 ltp-pan: Report failure if testcases failed to run

In case a test fails to run in run_child() for some reason (signaled via SIGTERM/SIGTOP, execve() failure, etc.), runltp still reports success:
"INFO: ltp-pan reported all tests PASS".

Failed tests are reported via check_pids(), by checking their exit status, yet it doesn't catch tests which weren't able to run at all.

exit_stat holds the exit status code: 0 in case of success in all tests, otherwise, the number of failed tests.

Fix this by increasing exit_stat also when a test fails to run.

---
 pan/ltp-pan.c | 2 ++
 1 file changed, 2 insertions(+)
pan/ltp-pan.c +2 -0
index 0bdb51477..eff9a867d 100644
@@ -551,6 +551,8 @@ int main(int argc, char **argv)
 				++num_active;
 			if ((cpid != -1 || sequential) && starts > 0)
 				--starts;
+			if (cpid == -1 && track_exit_stats)
+				exit_stat++;
 
 			if (sequential)
 				if (++c >= coll->cnt)
Signature · 5 lines
2.25.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

Ping

Quoted 34 lines · On Sat, Nov 27, 2021, 14:16 Sagi Nakash <[email protected]> wrote:

In case a test fails to run in run_child() for some reason (signaled via SIGTERM/SIGTOP, execve() failure, etc.), runltp still reports success:
"INFO: ltp-pan reported all tests PASS".

Failed tests are reported via check_pids(), by checking their exit status, yet it doesn't catch tests which weren't able to run at all.

exit_stat holds the exit status code: 0 in case of success in all tests, otherwise, the number of failed tests.

Fix this by increasing exit_stat also when a test fails to run.

---
 pan/ltp-pan.c | 2 ++
 1 file changed, 2 insertions(+)
pan/ltp-pan.c +2 -1
index 0bdb51477..eff9a867d 100644
@@ -551,6 +551,8 @@ int main(int argc, char **argv)
                                ++num_active;
                        if ((cpid != -1 || sequential) && starts > 0)
                                --starts;
+                       if (cpid == -1 && track_exit_stats)
+                               exit_stat++;
 
                        if (sequential)
                                if (++c >= coll->cnt)
--

2.25.1

Signature · 2 lines
-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw

Ping

Quoted 38 lines · On Fri, Dec 3, 2021, 18:41 Sagi Nakash <[email protected]> wrote:

Ping

Quoted 34 lines · On Sat, Nov 27, 2021, 14:16 Sagi Nakash <[email protected]> wrote:

In case a test fails to run in run_child() for some reason (signaled via SIGTERM/SIGTOP, execve() failure, etc.), runltp still reports success:
"INFO: ltp-pan reported all tests PASS".

Failed tests are reported via check_pids(), by checking their exit status, yet it doesn't catch tests which weren't able to run at all.

exit_stat holds the exit status code: 0 in case of success in all tests, otherwise, the number of failed tests.

Fix this by increasing exit_stat also when a test fails to run.

---
 pan/ltp-pan.c | 2 ++
 1 file changed, 2 insertions(+)
pan/ltp-pan.c +2 -1
index 0bdb51477..eff9a867d 100644
@@ -551,6 +551,8 @@ int main(int argc, char **argv)
                                ++num_active;
                        if ((cpid != -1 || sequential) && starts > 0)
                                --starts;
+                       if (cpid == -1 && track_exit_stats)
+                               exit_stat++;
 
                        if (sequential)
                                if (++c >= coll->cnt)
--

2.25.1

Signature · 2 lines
-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
Reviewed-by on v1 1/1
#3 lore raw
In reply to Sagi Nakash

Hi,

@all: Do we want to include this one in the release?

Kind regards,
Petr

Quoted 30 lines

In case a test fails to run in run_child() for some reason (signaled via SIGTERM/SIGTOP, execve() failure, etc.), runltp still reports success:
"INFO: ltp-pan reported all tests PASS".

Failed tests are reported via check_pids(), by checking their exit status, yet it doesn't catch tests which weren't able to run at all.

exit_stat holds the exit status code: 0 in case of success in all tests, otherwise, the number of failed tests.

Fix this by increasing exit_stat also when a test fails to run.

---
 pan/ltp-pan.c | 2 ++
 1 file changed, 2 insertions(+)
pan/ltp-pan.c +2 -0
index 0bdb51477..eff9a867d 100644
@@ -551,6 +551,8 @@ int main(int argc, char **argv)
 				++num_active;
 			if ((cpid != -1 || sequential) && starts > 0)
 				--starts;
+			if (cpid == -1 && track_exit_stats)
+				exit_stat++;
 
 			if (sequential)
 				if (++c >= coll->cnt)
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#4 lore raw

Hi!

Quoted 3 lines

@all: Do we want to include this one in the release?

I do not think that this is complete enough. If we wanted to fix this we whould have to change the code to write a record to the failcmdfile as well.

Also increasing the variable on each failure seems wrong as well, it may as well overflow on large enough runtest file...

Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp