Skip to contentSkip to conversation

open_posix/timer_getoverrun/2-3: Fix test for systems with low timer precision

PATCH New Joerg Vehlow 3 participants 9 messages Oct 11, 2019 – Oct 18, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 open_posix/timer_getoverrun/2-3: Fix test for systems with low timer precision

From: Joerg Vehlow <[email protected]>

On systems with low timer precision the test always fails, because the allowed maximum number of overruns is calculated from the expected overruns + 10%. If the expected overruns is less than 200, there is no tollerance. This happens, if the precision of the timer is less than or equal to 4ms. E.g. qemu-arm64 without high resolution timer the accuracy is only 4ms.

---
 .../conformance/interfaces/timer_getoverrun/2-3.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +10 -5
index 96b7d01e6..3df3a9f01 100644
@@ -55,6 +55,7 @@ int main(void)
 	int overruns;
 	int valuensec, intervalnsec, expectedoverruns;
 	int fudge;
+	int duration;
 
 	if (sigemptyset(&set) != 0) {
 		perror("sigemptyset() did not return success\n");
@@ -94,11 +95,15 @@ int main(void)
 
 	valuensec = tsres.tv_nsec;
 	intervalnsec = 2 * valuensec;
-	//expectedoverruns = (1000000000 - valuensec) / intervalnsec;
-	expectedoverruns = 1000000000 / intervalnsec - 1;
+	expectedoverruns = 0;
+	duration = 0;
+	while (expectedoverruns < 1000) {
+		duration++;
+		expectedoverruns = duration * (1000000000 / intervalnsec - 1);
+	}
 
-	printf("value = %d sec, interval = %d nsec, "
-	       "expected overruns = %d\n", 1, intervalnsec, expectedoverruns);
+	printf("duration = %d sec, interval = %d nsec, "
+	       "expected overruns = %d\n", duration, intervalnsec, expectedoverruns);
 
 	its.it_interval.tv_sec = 0;
 	its.it_interval.tv_nsec = intervalnsec;
@@ -113,7 +118,7 @@ int main(void)
 	}
 	//tssleep.tv_nsec = valuensec + (expectedoverruns*intervalnsec);
 	tssleep.tv_nsec = 0;
-	tssleep.tv_sec = 2;
+	tssleep.tv_sec = duration + 1;
 	if (nanosleep(&tssleep, NULL) != 0) {
 		perror("nanosleep() did not return success\n");
 		return PTS_UNRESOLVED;
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/1 open_posix/timer_getoverrun/2-3: Fix test for systems with low timer precision

----- Original Message -----

Quoted 2 lines

From: Joerg Vehlow <[email protected]>

Hi,

Quoted 3 lines

On systems with low timer precision the test always fails, because the allowed
maximum number of overruns is calculated from the expected overruns + 10%.

Did you mean 1% here?
fudge = expectedoverruns / 100;

Quoted 3 lines

If the expected overruns is less than 200, there is no tollerance. This happens, if the precision of the timer is less than or equal to 4ms. E.g. qemu-arm64 without high resolution timer the accuracy is only 4ms.

Would tweaking tolerance work too? E.g. use float, round up.

Quoted 23 lines
---
 .../conformance/interfaces/timer_getoverrun/2-3.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c b/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c index 96b7d01e6..3df3a9f01 100644
---
a/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c
+++
b/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c

testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +1 -0
@@ -55,6 +55,7 @@ int main(void)
 	int overruns;
 	int valuensec, intervalnsec, expectedoverruns;
 	int fudge;
+	int duration;
 
 	if (sigemptyset(&set) != 0) {
 		perror("sigemptyset() did not return success\n");

testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c int main(void) · line 102

Quoted 10 lines
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +5 -2
@@ -94,11 +95,15 @@ int main(void)
 
 	valuensec = tsres.tv_nsec;
 	intervalnsec = 2 * valuensec;
-	//expectedoverruns = (1000000000 - valuensec) / intervalnsec;
-	expectedoverruns = 1000000000 / intervalnsec - 1;
+	expectedoverruns = 0;
+	duration = 0;
+	while (expectedoverruns < 1000) {
+		duration++;
+		expectedoverruns = duration * (1000000000 / intervalnsec - 1);

I was assuming -1 in original code is to cope with final timer expiration of tssleep.tv_sec, which might not be counted as "overrun". What does the -1 do in your formula? Why is it inside brackets?

When I try to force different interval values, it fails for me (on x86):
3ms

# ./timer_getoverrun_2-3.run-test 
duration = 7 sec, interval = 6000000 nsec, expected overruns = 1155
1166 overruns occurred
FAIL:  1166 overruns sent; expected 1155

5ms

# ./timer_getoverrun_2-3.run-test 
duration = 11 sec, interval = 10000000 nsec, expected overruns = 1089
1100 overruns occurred
FAIL:  1100 overruns sent; expected 1089

Regards,
Jan

#2 lore raw

PATCH 1/1 open_posix/timer_getoverrun/2-3: Fix test for systems with low timer precision

Hi,

Quoted 5 lines
Quoted 3 lines

On systems with low timer precision the test always fails, because the allowed
maximum number of overruns is calculated from the expected overruns + 10%.

Did you mean 1% here?
fudge = expectedoverruns / 100;

Of course, you are right

Quoted 5 lines
Quoted 4 lines

If the expected overruns is less than 200, there is no tollerance. This happens, if the precision of the timer is less than or equal to 4ms. E.g. qemu-arm64 without high resolution timer the accuracy is only 4ms.

Would tweaking tolerance work too? E.g. use float, round up.

Actually rounding up would not work. I tried it with a timer accuracy of 4ms and I needed
at least a fudge of two. Rounding up would only result in one. That it why I decided to
go for extending the duration

Quoted 4 lines

I was assuming -1 in original code is to cope with final timer expiration of tssleep.tv_sec, which might not be counted as "overrun". What does the -1 do in your formula? Why is it inside brackets?

You may be right. I am not completely sure what it is used for, but I guess you are right.

Quoted 8 lines

When I try to force different interval values, it fails for me (on x86):
3ms

# ./timer_getoverrun_2-3.run-test
duration = 7 sec, interval = 6000000 nsec, expected overruns = 1155
1166 overruns occurred
FAIL:  1166 overruns sent; expected 1155
Quoted 5 lines

5ms

# ./timer_getoverrun_2-3.run-test
duration = 11 sec, interval = 10000000 nsec, expected overruns = 1089
1100 overruns occurred
FAIL:  1100 overruns sent; expected 1089

How is it possible to force the timer resolution? I tested on qemu with aarch64 and x86_64 with and without highres=off and ir worked.

If the -1 is outside the brackets, it would fix both of your test cases. (1155 + 7 - 1)? = 1161; +1% = 1172 -> expected overruns [1161, 1172) (1089 + 11 - 1) = 1099; +1% = 1109 ->expected overruns [1099, 1109)

J?rg

#3 lore raw

PATCH 1/1 open_posix/timer_getoverrun/2-3: Fix test for systems with low timer precision

Quoted 19 lines · Joerg Vehlow

Actually rounding up would not work. I tried it with a timer accuracy of 4ms and I needed
at least a fudge of two. Rounding up would only result in one. That it why I decided to
go for extending the duration

Quoted 4 lines

I was assuming -1 in original code is to cope with final timer expiration of tssleep.tv_sec, which might not be counted as "overrun". What does the -1 do in your formula? Why is it inside brackets?

You may be right. I am not completely sure what it is used for, but I guess you are right.

Quoted 8 lines

When I try to force different interval values, it fails for me (on x86):
3ms

# ./timer_getoverrun_2-3.run-test
duration = 7 sec, interval = 6000000 nsec, expected overruns = 1155
1166 overruns occurred
FAIL:  1166 overruns sent; expected 1155
Quoted 7 lines
Quoted 5 lines

5ms

# ./timer_getoverrun_2-3.run-test
duration = 11 sec, interval = 10000000 nsec, expected overruns = 1089
1100 overruns occurred
FAIL:  1100 overruns sent; expected 1089

How is it possible to force the timer resolution? I tested on qemu with aarch64 and x86_64 with and without highres=off and ir worked.

I didn't change timer resolution, I only used larger interval values as multiple of timer resolution:
intervalnsec = largeX * timer_resolution

I'd prefer we tweak the tolerance rather than make test run longer. I'm thinking just allow ~50ms of extra overruns, and don't be so strict about absolute number of overruns. (KVM guests and s390 lpars tend to suffer from higher steal time).

testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +8 -3
index 96b7d01e6ffe..66f8b583a5a6 100644
@@ -94,11 +94,17 @@ int main(void)
  
         valuensec = tsres.tv_nsec;
         intervalnsec = 2 * valuensec;
-       //expectedoverruns = (1000000000 - valuensec) / intervalnsec;
         expectedoverruns = 1000000000 / intervalnsec - 1;
  
+       /*
+        * waking up from sleep isn't instant, we can overshoot.
+        * Allow up to ~50ms worth of extra overruns.
+        */
+       fudge = 50000000 / intervalnsec + 1;
+
         printf("value = %d sec, interval = %d nsec, "
-              "expected overruns = %d\n", 1, intervalnsec, expectedoverruns);
+              "expected overruns = %d, fudge = %d\n", 1,
+              intervalnsec, expectedoverruns, fudge);
  
         its.it_interval.tv_sec = 0;
         its.it_interval.tv_nsec = intervalnsec;
@@ -146,7 +152,6 @@ int main(void)
          * extra expiries after the nanosleep completes so do
          * a range check.
          */
-       fudge = expectedoverruns / 100;
         if (overruns >= expectedoverruns && overruns < expectedoverruns + fudge) {
                 printf("Test PASSED\n");
                 return PTS_PASS;
#4 lore raw

PATCH 1/1 open_posix/timer_getoverrun/2-3: Fix test for systems with low timer precision

Quoted 5 lines

I didn't change timer resolution, I only used larger interval values as multiple of timer resolution:
? intervalnsec = largeX * timer_resolution

ah ok

Quoted 36 lines

I'd prefer we tweak the tolerance rather than make test run longer. I'm thinking just allow ~50ms of extra overruns, and don't be so strict about absolute number of overruns. (KVM guests and s390 lpars tend to suffer from higher steal time).

(quoted diff) +0 -0

a/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c b/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c

index 96b7d01e6ffe..66f8b583a5a6 100644
---
a/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c
+++
b/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c

(quoted diff) +6 -1
@@ -94,11 +94,17 @@ int main(void)
 
 ??????? valuensec = tsres.tv_nsec;
 ??????? intervalnsec = 2 * valuensec;
-?????? //expectedoverruns = (1000000000 - valuensec) / intervalnsec;
 ??????? expectedoverruns = 1000000000 / intervalnsec - 1;
 
+?????? /*
+??????? * waking up from sleep isn't instant, we can overshoot.
+??????? * Allow up to ~50ms worth of extra overruns.
+??????? */
+?????? fudge = 50000000 / intervalnsec + 1;
+

??????? printf("value = %d sec, interval = %d nsec, " -????????????? "expected overruns = %d\n", 1, intervalnsec, expectedoverruns);
+????????????? "expected overruns = %d, fudge = %d\n", 1, +????????????? intervalnsec, expectedoverruns, fudge);

??????? its.it_interval.tv_sec = 0;
??????? its.it_interval.tv_nsec = intervalnsec;

(quoted diff) +0 -0
@@ -146,7 +152,6 @@ int main(void)
Quoted 9 lines

???????? * extra expiries after the nanosleep completes so do ???????? * a range check.
???????? */
-?????? fudge = expectedoverruns / 100;
??????? if (overruns >= expectedoverruns && overruns < expectedoverruns + fudge) {
??????????????? printf("Test PASSED\n"); ??????????????? return PTS_PASS;

I wonder if there was a way to do this more accurate, like busy waiting while signals
are blocked. But I would also be fine with your solution.

#5 lore raw

PATCH 1/1 timer_getoverrun/2-3: increase tolerance for overshoot

Joerg reports that test fails on systems with low timer precision (qemu-arm64, 4ms timer precision) due to way we calculate maximum number of overruns. If the expected overruns is less than 200, there is no tollerance.

Allow up to ~50ms of extra overruns.

---
 .../conformance/interfaces/timer_getoverrun/2-3.c             | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +8 -3
index 96b7d01e6ffe..66f8b583a5a6 100644
@@ -94,11 +94,17 @@ int main(void)
 
 	valuensec = tsres.tv_nsec;
 	intervalnsec = 2 * valuensec;
-	//expectedoverruns = (1000000000 - valuensec) / intervalnsec;
 	expectedoverruns = 1000000000 / intervalnsec - 1;
 
+	/*
+	 * waking up from sleep isn't instant, we can overshoot.
+	 * Allow up to ~50ms worth of extra overruns.
+	 */
+	fudge = 50000000 / intervalnsec + 1;
+
 	printf("value = %d sec, interval = %d nsec, "
-	       "expected overruns = %d\n", 1, intervalnsec, expectedoverruns);
+	       "expected overruns = %d, fudge = %d\n", 1,
+	       intervalnsec, expectedoverruns, fudge);
 
 	its.it_interval.tv_sec = 0;
 	its.it_interval.tv_nsec = intervalnsec;
@@ -146,7 +152,6 @@ int main(void)
 	 * extra expiries after the nanosleep completes so do
 	 * a range check.
 	 */
-	fudge = expectedoverruns / 100;
 	if (overruns >= expectedoverruns && overruns < expectedoverruns + fudge) {
 		printf("Test PASSED\n");
 		return PTS_PASS;
Signature · 1 line
1.8.3.1
#6 lore raw

PATCH 1/1 timer_getoverrun/2-3: increase tolerance for overshoot

Hi!

Quoted 19 lines

Joerg reports that test fails on systems with low timer precision (qemu-arm64, 4ms timer precision) due to way we calculate maximum number of overruns. If the expected overruns is less than 200, there is no tollerance.

Allow up to ~50ms of extra overruns.

---
 .../conformance/interfaces/timer_getoverrun/2-3.c             | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +0 -0
index 96b7d01e6ffe..66f8b583a5a6 100644
@@ -94,11 +94,17 @@ int main(void)
 

testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c int main(void) · line 103

Quoted 10 lines
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +5 -1
 	valuensec = tsres.tv_nsec;
 	intervalnsec = 2 * valuensec;
-	//expectedoverruns = (1000000000 - valuensec) / intervalnsec;
 	expectedoverruns = 1000000000 / intervalnsec - 1;
 
+	/*
+	 * waking up from sleep isn't instant, we can overshoot.
+	 * Allow up to ~50ms worth of extra overruns.
+	 */
+	fudge = 50000000 / intervalnsec + 1;

I wonder if we can measure the sleep with monotonic clock, then we can be more precise, since we would know how much we overshoot. But it's probably not worth of complicating the code.

Either way I'm fine with this version as well, acked.

Quoted 21 lines
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +2 -2
 	printf("value = %d sec, interval = %d nsec, "
-	       "expected overruns = %d\n", 1, intervalnsec, expectedoverruns);
+	       "expected overruns = %d, fudge = %d\n", 1,
+	       intervalnsec, expectedoverruns, fudge);
 
 	its.it_interval.tv_sec = 0;
 	its.it_interval.tv_nsec = intervalnsec;
@@ -146,7 +152,6 @@ int main(void)
 	 * extra expiries after the nanosleep completes so do
 	 * a range check.
 	 */
-	fudge = expectedoverruns / 100;
 	if (overruns >= expectedoverruns && overruns < expectedoverruns + fudge) {
 		printf("Test PASSED\n");
 		return PTS_PASS;
Signature · 5 lines
1.8.3.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 2 lines
Cyril Hrubis
[email protected]
#7 lore raw

PATCH 1/1 timer_getoverrun/2-3: increase tolerance for overshoot

----- Original Message -----

Quoted 6 lines

Hi!
I wonder if we can measure the sleep with monotonic clock, then we can be more precise, since we would know how much we overshoot. But it's probably not worth of complicating the code.

Either way I'm fine with this version as well, acked.

I pushed current patch. If we find it's still causing issues I'll look into the idea with monotonic clock.

Thanks,
Jan

#8 lore raw

PATCH 1/1 timer_getoverrun/2-3: increase tolerance for overshoot

Hi,

Am 18.10.2019 um 09:58 schrieb Jan Stancek:

Quoted 10 lines

----- Original Message -----

Quoted 6 lines

Hi!
I wonder if we can measure the sleep with monotonic clock, then we can be more precise, since we would know how much we overshoot. But it's probably not worth of complicating the code.

Either way I'm fine with this version as well, acked.

I pushed current patch. If we find it's still causing issues I'll look into the idea with monotonic clock.

Thanks for fixing this. I'd like to mention, that CLOCK_MONOTONIC is an optionalfeature in posix. I do not know if there are systems missing it, that might run these test.

J?rg