Skip to contentSkip to conversation

openposix: Remove timer_*/speculative tests for invalid timerid

PATCH In review Avinesh Kumar via ltp 4 participants 9 messages Aug 11 – Sep 29

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 openposix: Remove timer_*/speculative tests for invalid timerid

From: Avinesh Kumar <[email protected]>

timer_delete/speculative/5-{1,2}, timer_getoverrun/speculative/6-{1,2,3}, timer_gettime/speculative/6-{1,2,3}, and timer_settime/speculative/12-{1,2,3} all test behavior for an invalid timerid that POSIX explicitly leaves undefined (Issue 6/TC2 made EINVAL optional; some of these functions' ERRORS sections define no errors at all). Remove them instead of special-casing a SIGSEGV some implementations produce for this input[2].

[0] https://pubs.opengroup.org/onlinepubs/9799919799/functions/timer_getoverrun.html
[1] https://pubs.opengroup.org/onlinepubs/9799919799/functions/timer_delete.html
[2] https://lore.kernel.org/ltp/[email protected]/

---
 .../interfaces/timer_delete/coverage.txt      |  5 +-
 .../interfaces/timer_delete/speculative/5-1.c | 40 --------
 .../interfaces/timer_delete/speculative/5-2.c | 94 -------------------
 .../timer_delete/speculative/LDLIBS           |  1 -
 .../timer_getoverrun/speculative/6-1.c        | 39 --------
 .../timer_getoverrun/speculative/6-2.c        | 58 ------------
 .../timer_getoverrun/speculative/6-3.c        | 50 ----------
 .../timer_getoverrun/speculative/LDLIBS       |  1 -
 .../interfaces/timer_gettime/coverage.txt     |  6 +-
 .../timer_gettime/speculative/6-1.c           | 38 --------
 .../timer_gettime/speculative/6-2.c           | 48 ----------
 .../timer_gettime/speculative/6-3.c           | 52 ----------
 .../timer_gettime/speculative/LDLIBS          |  1 -
 .../interfaces/timer_settime/coverage.txt     | 10 +-
 .../timer_settime/speculative/12-1.c          | 42 ---------
 .../timer_settime/speculative/12-2.c          | 57 -----------
 .../timer_settime/speculative/12-3.c          | 59 ------------
 .../timer_settime/speculative/LDLIBS          |  1 -
 18 files changed, 15 insertions(+), 587 deletions(-)
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_delete/speculative/5-1.c
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_delete/speculative/5-2.c
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_delete/speculative/LDLIBS
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/speculative/6-1.c
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/speculative/6-2.c
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/speculative/6-3.c
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/speculative/LDLIBS
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_gettime/speculative/6-1.c
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_gettime/speculative/6-2.c
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_gettime/speculative/6-3.c
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_gettime/speculative/LDLIBS
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_settime/speculative/12-1.c
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_settime/speculative/12-2.c
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_settime/speculative/12-3.c
 delete mode 100644 testcases/open_posix_testsuite/conformance/interfaces/timer_settime/speculative/LDLIBS
testcases/open_posix_testsuite/conformance/interfaces/timer_delete/coverage.txt +4 -1
index 51b8c5a711d9..11276c261286 100644
@@ -5,4 +5,7 @@ Assertion	Covered?
 2		YES
 3		YES
 4		YES
-5		YES
+5		WON'T*
+
+* EINVAL is optional here (POSIX Issue 6/TC2; no errors are defined
+at all as of Issue 8), so not testable.
testcases/open_posix_testsuite/conformance/interfaces/timer_delete/speculative/5-1.c deleted +0 -40
deleted file mode 100644
index 912cf5800e6f..000000000000
@@ -1,40 +0,0 @@
-/*
- * Copyright (c) 2002, Intel Corporation. All rights reserved.
- * Created by:  julie.n.fleischer REMOVE-THIS AT intel DOT com
- * This file is licensed under the GPL license.  For the full content
- * of this license, see the COPYING file at the top level of this
- * source tree.
-
- * Test to see if timer_delete() returns -1 and sets errno==EINVAL if
- * timerid is not a valid timer ID or not.
- * Since this is a "may" requirement, either option is a PASS.
- */
-
-#include <time.h>
-#include <stdio.h>
-#include <errno.h>
-#include "posixtest.h"
-
-#define BOGUSTIMERID 99999
-
-int test_main(int argc PTS_ATTRIBUTE_UNUSED, char **argv PTS_ATTRIBUTE_UNUSED)
-{
-	timer_t tid;
-	int tval = BOGUSTIMERID;
-	tid = (timer_t) & tval;
-
-	if (timer_delete(tid) == -1) {
-		if (errno == EINVAL) {
-			printf
-			    ("timer_delete() returned -1 and set errno=EINVAL\n");
-			return PTS_PASS;
-		} else {
-			printf
-			    ("timer_delete() returned -1, but didn't set errno!=EINVAL\n");
-			return PTS_FAIL;
-		}
-	}
-
-	printf("timer_delete() did not return -1\n");
-	return PTS_PASS;
-}
testcases/open_posix_testsuite/conformance/interfaces/timer_delete/speculative/5-2.c deleted +0 -94

94 lines not on this page · raw message

testcases/open_posix_testsuite/conformance/interfaces/timer_delete/speculative/LDLIBS deleted +0 -1
deleted file mode 100644
index 08f7cd60c2ad..000000000000
@@ -1 +0,0 @@
--lrt
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/speculative/6-1.c deleted +0 -39
deleted file mode 100644
index 6e18560e5084..000000000000
@@ -1,39 +0,0 @@
-/*
- * Copyright (c) 2002, Intel Corporation. All rights reserved.
- * Created by:  julie.n.fleischer REMOVE-THIS AT intel DOT com
- * This file is licensed under the GPL license.  For the full content
- * of this license, see the COPYING file at the top level of this
- * source tree.
- *
- * Test to see if timer_getoverrun() sets errno=EINVAL if no timers have been
- * created yet or not.  Since this is a "may" assertion, either way is
- * a pass.
- */
-
-#include <time.h>
-#include <stdio.h>
-#include <errno.h>
-#include "posixtest.h"
-
-#define BOGUSTID 9999
-
-int test_main(int argc PTS_ATTRIBUTE_UNUSED, char **argv PTS_ATTRIBUTE_UNUSED)
-{
-	timer_t tid;
-	int tval = BOGUSTID;
-	tid = (timer_t) & tval;
-
-	if (timer_getoverrun(tid) == -1) {
-		if (EINVAL == errno) {
-			printf("fcn returned -1 and errno=EINVAL\n");
-			return PTS_PASS;
-		} else {
-			printf("fcn returned -1, but errno!=EINVAL\n");
-			printf("Test FAILED\n");
-			return PTS_FAIL;
-		}
-	}
-
-	printf("fcn did not return -1\n");
-	return PTS_PASS;
-}
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/speculative/6-2.c deleted +0 -58

58 lines not on this page · raw message

testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/speculative/6-3.c deleted +0 -50

50 lines not on this page · raw message

testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/speculative/LDLIBS deleted +0 -1
deleted file mode 100644
index 08f7cd60c2ad..000000000000
@@ -1 +0,0 @@
--lrt
testcases/open_posix_testsuite/conformance/interfaces/timer_gettime/coverage.txt +5 -1
index aa9a47e4b5c2..dcfee8dfa2fe 100644
@@ -6,4 +6,8 @@ Assertion	Coverage
 3		YES
 4		YES
 5		YES
-6		YES
+6		WON'T*
+
+* POSIX explicitly states the behavior is undefined for an invalid
+timerid here (Issue 6/TC2 made EINVAL optional, not required), so
+not testable.
testcases/open_posix_testsuite/conformance/interfaces/timer_gettime/speculative/6-1.c deleted +0 -38
deleted file mode 100644
index d09c2f70901d..000000000000
@@ -1,38 +0,0 @@
-/*
- * Copyright (c) 2002, Intel Corporation. All rights reserved.
- * Created by:  julie.n.fleischer REMOVE-THIS AT intel DOT com
- * This file is licensed under the GPL license.  For the full content
- * of this license, see the COPYING file at the top level of this
- * source tree.
- *
- * Test to see if timer_gettime() sets errno = EINVAL if no timers have been
- * created yet and it is called with a bogus timer ID.  Since this is a "may"
- * assertion, either way is pass.
- */
-
-#include <time.h>
-#include <stdio.h>
-#include <errno.h>
-#include "posixtest.h"
-
-#define BOGUSTID 9999
-
-int test_main(int argc PTS_ATTRIBUTE_UNUSED, char **argv PTS_ATTRIBUTE_UNUSED)
-{
-	timer_t tid;
-	struct itimerspec its;
-	int tval = BOGUSTID;
-	tid = (timer_t) & tval;
-	if (timer_gettime(tid, &its) == -1) {
-		if (EINVAL == errno) {
-			printf("fcn returned -1 and errno==EINVAL\n");
-			return PTS_PASS;
-		} else {
-			printf("fcn returned -1 but errno!=EINVAL\n");
-			printf("Test FAILED\n");
-			return PTS_FAIL;
-		}
-	}
-	printf("fcn did not return -1\n");
-	return PTS_PASS;
-}
testcases/open_posix_testsuite/conformance/interfaces/timer_gettime/speculative/6-2.c deleted +0 -48

48 lines not on this page · raw message

testcases/open_posix_testsuite/conformance/interfaces/timer_gettime/speculative/6-3.c deleted +0 -52

52 lines not on this page · raw message

testcases/open_posix_testsuite/conformance/interfaces/timer_gettime/speculative/LDLIBS deleted +0 -1
deleted file mode 100644
index 08f7cd60c2ad..000000000000
@@ -1 +0,0 @@
--lrt
testcases/open_posix_testsuite/conformance/interfaces/timer_settime/coverage.txt +6 -4
index 8e6034a21a6c..a4cd9d379fd7 100644
@@ -12,14 +12,16 @@ Assertion	Covered?
 9		YES
 10		YES
 11		YES
-12		YES
+12		WON'T**
 13		YES
 
 *Decided against testing this assertion as it would be quite difficult
 to get an accurate test.
 
+** POSIX explicitly states the behavior is undefined for an invalid
+timerid here (Issue 6/TC2 made EINVAL optional, not required), so
+not testable.
+
 Possible other items to test:
 assertion 6 - What happens if it_interval < it_value? - Would be easy to
-use code from 6-1 and replace numbers to test this.
-
-May also want to run canned boundary tests on tid values.
+use code from 6-1 and replace numbers to test this.
\ No newline at end of file
testcases/open_posix_testsuite/conformance/interfaces/timer_settime/speculative/12-1.c deleted +0 -42

42 lines not on this page · raw message

testcases/open_posix_testsuite/conformance/interfaces/timer_settime/speculative/12-2.c deleted +0 -57

57 lines not on this page · raw message

testcases/open_posix_testsuite/conformance/interfaces/timer_settime/speculative/12-3.c deleted +0 -59

59 lines not on this page · raw message

testcases/open_posix_testsuite/conformance/interfaces/timer_settime/speculative/LDLIBS deleted +0 -1
deleted file mode 100644
index 08f7cd60c2ad..000000000000
@@ -1 +0,0 @@
--lrt
Signature · 5 lines
2.55.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

Hi Avinesh,

testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/speculative/6-1.c

Quoted 4 lines · Avinesh Kumar

openposix: Remove timer_*/speculative tests for invalid timerid

testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/speculative/6-1.c deleted +0 -0
deleted file mode 100644

Could timer_getoverrun/coverage.txt also add assertion 6 as WON'T with the rationale used by the other coverage updates? Its table still stops at assertion 5 although assertions.xml defines assertion 6, so deleting all speculative/6-* tests leaves the assertion's disposition undocumented.

Verdict - Needs revision

---
Note:

The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.

Regards,
LTP AI Reviewer

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#3 lore raw
In reply to Avinesh Kumar via ltp

Hi!
The only concern I have here is that there are UNIX-like OSes that actually implement all the optional behavior and in that case these are valid tests. AFAIK OpenBSD implements are these "may" clauses that Linux implementation ignores completely.

Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#4 lore raw

Hi Cyril,

Quoted 4 lines

The only concern I have here is that there are UNIX-like OSes that actually implement all the optional behavior and in that case these are valid tests. AFAIK OpenBSD implements are these "may" clauses that Linux implementation ignores completely.

This is the problem with POSIX systems.. I would suggest to keep them and simply ignore them if running on Linux. Do we have a mechanism for that or we should do it at runner level?

--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#5 lore raw

Hi!

Quoted 3 lines

This is the problem with POSIX systems.. I would suggest to keep them and simply ignore them if running on Linux. Do we have a mechanism for that or we should do it at runner level?

I guess that we can simply #ifndef __linux__ these.

Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#6 lore raw

Hi Cyril, Andrea,

Thanks for the review and suggesting alternative approach.

Quoted 7 lines

Hi!

Quoted 3 lines

This is the problem with POSIX systems.. I would suggest to keep them and simply ignore them if running on Linux. Do we have a mechanism for that or we should do it at runner level?

I guess that we can simply #ifndef __linux__ these.

If we agree on this, I can send revised patch as suggested for all the tests under conformance/interfaces/timer_[delete|getoverrun|gettime| settime]/speculative/

testcases/open_posix_testsuite/conformance/interfaces/timer_delete/speculative/5-1.c +0 -0
@@ -19,6 +19,11 @@
 
 int test_main(int argc PTS_ATTRIBUTE_UNUSED, char **argv 

PTS_ATTRIBUTE_UNUSED)
{
+#ifdef __linux__
+ printf("Linux does not implement this optional behavior\n");
+ return PTS_UNSUPPORTED;
+#else

        timer_t tid;
        int tval = BOGUSTIMERID;
        tid = (timer_t) & tval;

@@ -37,4 +42,5 @@ int test_main(int argc PTS_ATTRIBUTE_UNUSED, char **argv PTS_ATTRIBUTE_UNUSED)

        printf("timer_delete() did not return -1\n");
        return PTS_PASS;

+#endif

--
Regards,
Avinesh

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#7 lore raw

Hi!

Quoted 10 lines
Quoted 6 lines
Quoted 3 lines

This is the problem with POSIX systems.. I would suggest to keep them and simply ignore them if running on Linux. Do we have a mechanism for that or we should do it at runner level?

I guess that we can simply #ifndef __linux__ these.

If we agree on this, I can send revised patch as suggested for all the tests under conformance/interfaces/timer_[delete|getoverrun|gettime| settime]/speculative/

Looks good to me.

Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#8 lore raw
In reply to Cyril Hrubis
Quoted 5 lines · Cyril Hrubis

Hi!
The only concern I have here is that there are UNIX-like OSes that actually implement all the optional behavior and in that case these are valid tests. AFAIK OpenBSD implements are these "may" clauses that Linux implementation ignores completely.

Hi, the behavior was optional until 2013 edition of POSIX[1]. Starting from 2016 edition[2], the behavior is undefined. The tests are no longer valid. We should drop them.

[1] https://pubs.opengroup.org/onlinepubs/9699919799.2013edition/functions/timer_delete.html
[2] https://pubs.opengroup.org/onlinepubs/9699919799.2016edition/functions/timer_delete.html

Signature · 10 lines
Martin Doucha   [email protected]
SW Quality Engineer
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp