Re: [LTP] [PATCH] ltp-pan.c: fix error of Total Tests

Cyril Hrubis

author

2012-05-16 14:56 UTC #1 lore raw

Hi!

Quoted 13 lines

When execvp() fails, the testcases is counted in Total Tests, but not added in logfile. In fact the testcases result should be added in logfile.
static pid_t

(quoted diff) +2 -1

-run_child(struct coll_entry *colle, struct tag_pgrp *active, int quiet_mode)
+run_child(struct coll_entry *colle, struct tag_pgrp *active, int quiet_mode,
+	  int *failcnt, int fmt_print, FILE * logfile)
 {
     ssize_t errlen;
     int cpid;

@@ -978,6 +982,26 @@ run_child(struct coll_entry *colle, struct tag_pgrp *active, int quiet_mode)
 	    termtype = "unknown";
 	}
 	time(&end_time);

Review run_child(struct coll_entry *colle, struct tag_pgrp *active, int quiet_mode) · line 994

Quoted 10 lines

(quoted diff) +10 -0

+	if (logfile != NULL) {
+		if (!fmt_print)
+			fprintf(logfile,
+				"tag=%s stime=%d dur=%d exit=%s "
+				"stat=%d core=%s cu=%d cs=%d\n",
+				colle->name, (int)(active->mystime),
+				(int) (end_time - active->mystime), termtype,
+				termid, (status & 0200) ? "yes" : "no",
+				0, 0);
+		else {

I suggest to add curly brackets around the multiline fprintf too.

Review run_child(struct coll_entry *colle, struct tag_pgrp *active, int quiet_mode) · line 1007

Quoted 13 lines

(quoted diff) +10 -0

+			if (termid != 0)
+				++*failcnt;
+
+			fprintf(logfile, "%-30.30s %-10.10s %-5d\n",
+				colle->name, ((termid != 0) ? "FAIL" : "PASS"),
+				termid);
+		}
+		fflush(logfile);
+	}
+
 	if (!quiet_mode)
 	{
 		//write_test_start(active, errbuf);

Otherwise I don't have any objection to adding this into the ltp-pan. Although I don't understand the messy pan code that much (and I don't have any intentions to study it more closely).

Do have anybody else an opinion?

Signature · 13 lines

Cyril Hrubis
[email protected]

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Garrett Cooper

2012-05-17 02:31 UTC #4 lore raw

Quoted 9 lines · Wanlong Gao

Quoted 2 lines · Peng Haitao
When execvp() fails, the testcases is counted in Total Tests, but not added in logfile. In fact the testcases result should be added in logfile.
I prefer adding this for matching the Total Tests amount. So,
Acked-by Wanlong Gao <[email protected]>

It's nice that this is finally being fixed. This is one issue that
has been outstanding for several years that needed to be resolved (I noticed it during/after I developed the execltp python script). Thanks!
-Garrett

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Wanlong Gao

2012-05-17 02:34 UTC #5 lore raw

Quoted 6 lines · Garrett Cooper

Quoted 5 lines · Wanlong Gao
Quoted 2 lines · Peng Haitao
When execvp() fails, the testcases is counted in Total Tests, but not added in logfile. In fact the testcases result should be added in logfile.

Quoted 8 lines

Quoted 4 lines
I prefer adding this for matching the Total Tests amount. So,
Acked-by Wanlong Gao <[email protected]>
It's nice that this is finally being fixed. This is one issue that
has been outstanding for several years that needed to be resolved (I noticed it during/after I developed the execltp python script).

So, Garrett, you can ACK this?

Thanks,
Wanlong Gao

Quoted 3 lines

Thanks!
-Garrett

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Garrett Cooper

2012-05-17 02:39 UTC #6 lore raw

Quoted 11 lines · Wanlong Gao

Quoted 9 lines · Garrett Cooper
Quoted 8 lines · Wanlong Gao
Quoted 2 lines · Peng Haitao
When execvp() fails, the testcases is counted in Total Tests, but not added in logfile. In fact the testcases result should be added in logfile.
I prefer adding this for matching the Total Tests amount. So,

Quoted 8 lines

Quoted 5 lines
Quoted 1 line
Acked-by Wanlong Gao <[email protected]>
It's nice that this is finally being fixed. This is one issue that has been outstanding for several years that needed to be resolved (I noticed it during/after I developed the execltp python script).
So, Garrett, you can ACK this?

I will ACK the sentiment, but I prefer to test it in order to ensure that it actually works...

You can easily trigger this problem with the following spoofed runtest scenario:

aok echo "hello"
donotexist donotexist
exit_quickly exit 1
true true

The first will fail success with "hello" output, the second should fail with ENOENT, the third will fail without output, and the fourth should success with output.

If this functions as I described above, I'll be more than happy to ACK the patch.

Thanks!
-Garrett

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Garrett Cooper

2012-05-17 02:40 UTC #7 lore raw

Quoted 24 lines · Garrett Cooper

Quoted 11 lines · Wanlong Gao
Quoted 9 lines · Garrett Cooper
Quoted 8 lines · Wanlong Gao
Quoted 2 lines · Peng Haitao
When execvp() fails, the testcases is counted in Total Tests, but not added in logfile. In fact the testcases result should be added in logfile.
I prefer adding this for matching the Total Tests amount. So,
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Acked-by Wanlong Gao <[email protected]>
It's nice that this is finally being fixed. This is one issue that has been outstanding for several years that needed to be resolved (I noticed it during/after I developed the execltp python script).
So, Garrett, you can ACK this?
I will ACK the sentiment, but I prefer to test it in order to ensure that it actually works...

Quoted 8 lines

You can easily trigger this problem with the following spoofed runtest scenario:
aok echo "hello"
donotexist donotexist
exit_quickly exit 1
true true
The first will fail success with "hello" output, the second should

s/fail success/pass/

Quoted 8 lines

fail with ENOENT, the third will fail without output, and the fourth should success with output.
If this functions as I described above, I'll be more than happy to ACK the patch.
Thanks!
-Garrett

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Peng Haitao

2012-05-17 02:47 UTC #8 lore raw

Garrett Cooper said the following on 2012-5-17 10:40:

Quoted 7 lines

Quoted 7 lines
You can easily trigger this problem with the following spoofed runtest scenario:
aok echo "hello"
donotexist donotexist
exit_quickly exit 1
true true

The test log is as follows:

Running tests.......
<<<test_start>>>
tag=aok stime=1337251363
cmdline="echo "hello""
contacts=""
analysis=exit
<<<test_output>>>
hello
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=donotexist stime=1337251363
cmdline="donotexist"
contacts=""
analysis=exit
<<<test_output>>>
<<<execution_status>>>
initiation_status="pan(17520): execvp of 'donotexist' (tag donotexist) failed. errno:2 No such file or directory"
duration=0 termination_type=exited termination_id=2 corefile=no cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=exit_quickly stime=1337251363
cmdline="exit 1"
contacts=""
analysis=exit
<<<test_output>>>
<<<execution_status>>>
initiation_status="pan(17520): execvp of 'exit' (tag exit_quickly) failed. errn o:2 No such file or directory"
duration=0 termination_type=exited termination_id=2 corefile=no cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=true stime=1337251363
cmdline="true"
contacts=""
analysis=exit
<<<test_output>>>
incrementing stop
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no cutime=0 cstime=0
<<<test_end>>>
INFO: ltp-pan reported all tests PASS
LTP Version: 20120401-38-g3d3e94e

       ###############################################################

            Done executing testcases.
            LTP Version:  20120401-38-g3d3e94e
       ###############################################################

# cat results/LTP_RUN_ON-2012_May_17-18h_42m_42s.log
Test Start Time: Thu May 17 18:42:43 2012
-----------------------------------------
Testcase                       Result     Exit Value
--------                       ------     ----------
aok                            PASS       0
donotexist                     FAIL       2
exit_quickly                   FAIL       2
true                           PASS       0

-----------------------------------------------
Total Tests: 4
Total Failures: 2
Kernel Version: 2.6.32-220.el6.x86_64
Machine Architecture: x86_64
Hostname: RHEL6U2-x86-64

Thanks.

Signature · 2 lines

Best Regards,
Peng Haitao

Quoted 14 lines

Quoted 1 line
The first will fail success with "hello" output, the second should
s/fail success/pass/
Quoted 8 lines
fail with ENOENT, the third will fail without output, and the fourth should success with output.
If this functions as I described above, I'll be more than happy to ACK the patch.
Thanks!
-Garrett

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Garrett Cooper

2012-05-17 04:39 UTC #9 lore raw

Quoted 36 lines · Peng Haitao

Garrett Cooper said the following on 2012-5-17 10:40:
Quoted 7 lines
Quoted 7 lines
You can easily trigger this problem with the following spoofed runtest scenario:
aok echo "hello"
donotexist donotexist
exit_quickly exit 1
true true
The test log is as follows:
Running tests.......
<<<test_start>>>
tag=aok stime=1337251363
cmdline="echo "hello""
contacts=""
analysis=exit
<<<test_output>>>
hello
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=0 corefile=no
cutime=0 cstime=0
<<<test_end>>>
<<<test_start>>>
tag=donotexist stime=1337251363
cmdline="donotexist"
contacts=""
analysis=exit
<<<test_output>>>
<<<execution_status>>>
initiation_status="pan(17520): execvp of 'donotexist' (tag donotexist) failed. errno:2 No such file or directory" duration=0 termination_type=exited termination_id=2 corefile=no
cutime=0 cstime=0
<<<test_end>>>

Quoted 8 lines

<<<test_start>>>
tag=exit_quickly stime=1337251363
cmdline="exit 1"
contacts=""
analysis=exit
<<<test_output>>>
<<<execution_status>>>
initiation_status="pan(17520): execvp of 'exit' (tag exit_quickly) failed. errn o:2 No such file or directory"

I suppose it would have worked, had this actually been a command and not a shell built-in. Could you please replace 'exit 1' with 'false'?

...

Thanks!
-Garrett

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Peng Haitao

2012-05-17 05:05 UTC #10 lore raw

Garrett Cooper said the following on 2012-5-17 12:39:

Quoted 9 lines

Quoted 8 lines
<<<test_start>>>
tag=exit_quickly stime=1337251363
cmdline="exit 1"
contacts=""
analysis=exit
<<<test_output>>>
<<<execution_status>>>
initiation_status="pan(17520): execvp of 'exit' (tag exit_quickly) failed. errn o:2 No such file or directory"

Quoted 3 lines

I suppose it would have worked, had this actually been a command and not a shell built-in. Could you please replace 'exit 1' with 'false'?

The log is as follows:

<<<test_start>>>
tag=exit_quickly stime=1337259686
cmdline="false"
contacts=""
analysis=exit
<<<test_output>>>
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=1 corefile=no cutime=0 cstime=0
<<<test_end>>>

# cat /opt/ltp/results/LTP_RUN_ON-2012_May_17-21h_01m_26s.log
Test Start Time: Thu May 17 21:01:26 2012
-----------------------------------------
Testcase                       Result     Exit Value
--------                       ------     ----------
aok                            PASS       0
donotexist                     FAIL       2
exit_quickly                   FAIL       1
true                           PASS       0

-----------------------------------------------
Total Tests: 4
Total Failures: 2
Kernel Version: 2.6.32-220.el6.x86_64
Machine Architecture: x86_64
Hostname: RHEL6U2-x86-64

Thanks.

Signature · 14 lines

Best Regards,
Peng Haitao


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Garrett Cooper

Acked-by

2012-05-17 05:52 UTC #11 lore raw

Quoted 17 lines · Peng Haitao

Garrett Cooper said the following on 2012-5-17 12:39:
Quoted 9 lines
Quoted 8 lines
<<<test_start>>>
tag=exit_quickly stime=1337251363
cmdline="exit 1"
contacts=""
analysis=exit
<<<test_output>>>
<<<execution_status>>>
initiation_status="pan(17520): execvp of 'exit' (tag exit_quickly) failed. errn o:2 No such file or directory"
Quoted 3 lines
I suppose it would have worked, had this actually been a command and not a shell built-in. Could you please replace 'exit 1' with 'false'?
The log is as follows:

Quoted 11 lines

<<<test_start>>>
tag=exit_quickly stime=1337259686
cmdline="false"
contacts=""
analysis=exit
<<<test_output>>>
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=1 corefile=no cutime=0 cstime=0
<<<test_end>>>

LGTM!

Acked-by Garrett Cooper <[email protected]>

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Cyril Hrubis

author

2012-05-17 11:08 UTC #12 lore raw

Hi!
Commited, thanks everyone involved ;).

Signature · 13 lines

Cyril Hrubis
[email protected]

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list