Skip to contentSkip to conversation

clocks/invaliddates.c: relax acceptable delta

PATCH In review Jan Stancek 4 participants 6 messages Jun 18, 2020 – Jun 19, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 clocks/invaliddates.c: relax acceptable delta

Test allows just 5ms delta for PASS, and test randomly fails in environments with shared resources and increased steal time.

Relax the condition and also print deltas if test fails. Remove DEBUG ifdefs and main parameters to avoid unused variable warning.

---
 .../functional/timers/clocks/invaliddates.c   | 25 +++++++------------
 1 file changed, 9 insertions(+), 16 deletions(-)
testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c +9 -16
index face334fd250..d4116b1e9bc0 100644
@@ -18,8 +18,7 @@
 
 #define NUMTESTS 5
 
-#define ACCEPTABLESECDELTA 0
-#define ACCEPTABLENSECDELTA 5000000
+#define ACCEPTABLESECDELTA 2
 
 static int testtimes[NUMTESTS][2] = {
 	{INT32_MAX, 999999999},	// large number
@@ -29,7 +28,7 @@ static int testtimes[NUMTESTS][2] = {
 	{1049623200, 999999999},	// daylight savings 2003
 };
 
-int main(int argc, char *argv[])
+int main(void)
 {
 	struct timespec tpset, tpget, tsreset;
 	int secdelta, nsecdelta;
@@ -44,9 +43,6 @@ int main(int argc, char *argv[])
 	for (i = 0; i < NUMTESTS; i++) {
 		tpset.tv_sec = testtimes[i][0];
 		tpset.tv_nsec = testtimes[i][1];
-#ifdef DEBUG
-		printf("Test: %ds %dns\n", testtimes[i][0], testtimes[i][1]);
-#endif
 		if (clock_settime(CLOCK_REALTIME, &tpset) == 0) {
 			if (clock_gettime(CLOCK_REALTIME, &tpget) == -1) {
 				printf("Error in clock_gettime()\n");
@@ -58,16 +54,13 @@ int main(int argc, char *argv[])
 				nsecdelta = nsecdelta + 1000000000;
 				secdelta = secdelta - 1;
 			}
-#ifdef DEBUG
-			printf("Delta:  %ds %dns\n", secdelta, nsecdelta);
-#endif
-			if ((secdelta > ACCEPTABLESECDELTA) || (secdelta < 0)) {
-				printf("clock does not appear to be set\n");
-				failure = 1;
-			}
-			if ((nsecdelta > ACCEPTABLENSECDELTA) ||
-			    (nsecdelta < 0)) {
-				printf("clock does not appear to be set\n");
+
+			if ((secdelta > ACCEPTABLESECDELTA)
+				|| (secdelta < 0)) {
+				printf("FAIL: test(%d,%d), secdelta: %d,"
+					" nsecdelta: %d\n",
+					testtimes[i][0], testtimes[i][1],
+					secdelta, nsecdelta);
 				failure = 1;
 			}
 		} else {
Signature · 1 line
2.18.1
#1 lore raw

PATCH 1/1 clocks/invaliddates.c: relax acceptable delta

Hi Jan,

Quoted 2 lines

Test allows just 5ms delta for PASS, and test randomly fails in environments with shared resources and increased steal time.

Good.

Quoted 3 lines

Relax the condition and also print deltas if test fails. Remove DEBUG ifdefs and main parameters to avoid unused variable warning.

Also thanks for cleanup.

Kind regards,
Petr

#2 lore raw

PATCH 1/1 clocks/invaliddates.c: relax acceptable delta

In reply to Jan Stancek
Quoted 21 lines · Jan Stancek

Test allows just 5ms delta for PASS, and test randomly fails in environments with shared resources and increased steal time.

Relax the condition and also print deltas if test fails. Remove DEBUG ifdefs and main parameters to avoid unused variable warning.

---
 .../functional/timers/clocks/invaliddates.c   | 25 +++++++------------
 1 file changed, 9 insertions(+), 16 deletions(-)

diff --git a/testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c b/testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c index face334fd250..d4116b1e9bc0 100644
---
a/testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c
+++
b/testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c

testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c line 22

Quoted 7 lines
testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c +1 -2
@@ -18,8 +18,7 @@
 
 #define NUMTESTS 5
 
-#define ACCEPTABLESECDELTA 0
-#define ACCEPTABLENSECDELTA 5000000
+#define ACCEPTABLESECDELTA 2
 

It's hard to say what size of the second-delta is a proper value, but to increase it can obviously decrease the failure probability. If there is no better way I'd go with this patch.

Quoted 54 lines
testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c +1 -4
 static int testtimes[NUMTESTS][2] = {
        {INT32_MAX, 999999999}, // large number
@@ -29,7 +28,7 @@ static int testtimes[NUMTESTS][2] = {
        {1049623200, 999999999},        // daylight savings 2003
 };
 
-int main(int argc, char *argv[])
+int main(void)
 {
        struct timespec tpset, tpget, tsreset;
        int secdelta, nsecdelta;
@@ -44,9 +43,6 @@ int main(int argc, char *argv[])
        for (i = 0; i < NUMTESTS; i++) {
                tpset.tv_sec = testtimes[i][0];
                tpset.tv_nsec = testtimes[i][1];
-#ifdef DEBUG
-               printf("Test: %ds %dns\n", testtimes[i][0],
 testtimes[i][1]);
-#endif
                if (clock_settime(CLOCK_REALTIME, &tpset) == 0) {
                        if (clock_gettime(CLOCK_REALTIME, &tpget) == -1) {
                                printf("Error in clock_gettime()\n");
testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c +7 -10
@@ -58,16 +54,13 @@ int main(int argc, char *argv[])
                                nsecdelta = nsecdelta + 1000000000;
                                secdelta = secdelta - 1;
                        }
-#ifdef DEBUG
-                       printf("Delta:  %ds %dns\n", secdelta, nsecdelta);
-#endif
-                       if ((secdelta > ACCEPTABLESECDELTA) || (secdelta <
 0)) {
-                               printf("clock does not appear to be
 set\n");
-                               failure = 1;
-                       }
-                       if ((nsecdelta > ACCEPTABLENSECDELTA) ||
-                           (nsecdelta < 0)) {
-                               printf("clock does not appear to be
 set\n");
+
+                       if ((secdelta > ACCEPTABLESECDELTA)
+                               || (secdelta < 0)) {
+                               printf("FAIL: test(%d,%d), secdelta: %d,"
+                                       " nsecdelta: %d\n",
+                                       testtimes[i][0], testtimes[i][1],
+                                       secdelta, nsecdelta);
                                failure = 1;
                        }
                } else {

--
2.18.1

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20200619/1ffe0c9f/attachment-0001.htm>
#3 lore raw

PATCH 1/1 clocks/invaliddates.c: relax acceptable delta

Quoted 26 lines · Li Wang
Quoted 24 lines · Jan Stancek

Test allows just 5ms delta for PASS, and test randomly fails in environments with shared resources and increased steal time.

Relax the condition and also print deltas if test fails. Remove DEBUG ifdefs and main parameters to avoid unused variable warning.

---
 .../functional/timers/clocks/invaliddates.c   | 25 +++++++------------
 1 file changed, 9 insertions(+), 16 deletions(-)

diff --git a/testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c b/testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c index face334fd250..d4116b1e9bc0 100644
---
a/testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c
+++
b/testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c

(quoted diff) +0 -0
@@ -18,8 +18,7 @@
 
 #define NUMTESTS 5...
 
Quoted 9 lines

testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c line 22

Quoted 4 lines
testcases/open_posix_testsuite/functional/timers/clocks/invaliddates.c +1 -2
-#define ACCEPTABLESECDELTA 0
-#define ACCEPTABLENSECDELTA 5000000
+#define ACCEPTABLESECDELTA 2
 

It's hard to say what size of the second-delta is a proper value, but to increase it can obviously decrease the failure probability. If there is no better way I'd go with this patch.

But in another word, this test is just focused on invalid dates[1], so the deviation(caused by sharing resource) in seconds(even 10 seconds) bound is acceptable I think.

If someone insists to make thing more precise, we can make use of function tst_is_virt() to apply this method only for virtual machines. But anyway that's not very important:).

[1].

static int testtimes[NUMTESTS][2] = {
    {INT32_MAX, 999999999}, // large number
    {946713600, 999999999}, // Y2K - Jan 1, 2000
    {951811200, 999999999}, // Y2K - Feb 29, 2000
    {1078041600, 999999999}, // Y2K - Feb 29, 2004
    {1049623200, 999999999}, // daylight savings 2003
};
Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20200619/32028224/attachment.htm>
#4 lore raw
In reply to Jan Stancek

Hi Jan
Acked-by.

ps: I like leaving a comment in delta, just my personal habit.

Best Regards
Yang Xu