open_posix/timer_getoverrun/2-3: Fix test for systems with low timer precision
PATCH 1/1 open_posix/timer_getoverrun/2-3: Fix test for systems with low timer precision
From: Joerg Vehlow <[email protected]>
On systems with low timer precision the test always fails, because the allowed maximum number of overruns is calculated from the expected overruns + 10%. If the expected overruns is less than 200, there is no tollerance. This happens, if the precision of the timer is less than or equal to 4ms. E.g. qemu-arm64 without high resolution timer the accuracy is only 4ms.
--- .../conformance/interfaces/timer_getoverrun/2-3.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +10 -5
@@ -55,6 +55,7 @@ int main(void)int overruns;int valuensec, intervalnsec, expectedoverruns;int fudge; +int duration;if (sigemptyset (&set) !=0 ) {perror ("sigemptyset() did not return success\n" );
@@ -94,11 +95,15 @@ int main(void) valuensec = tsres.tv_nsec; intervalnsec =2 * valuensec; -//expectedoverruns = (1000000000 - valuensec) / intervalnsec; - expectedoverruns =1000000000 / intervalnsec -1 ; + expectedoverruns =0 ; + duration =0 ; +while (expectedoverruns <1000 ) { + duration++; + expectedoverruns = duration * (1000000000 / intervalnsec -1 ); + } -printf ("value = %d sec, interval = %d nsec, " -"expected overruns = %d\n" ,1 , intervalnsec, expectedoverruns); +printf ("duration = %d sec, interval = %d nsec, " +"expected overruns = %d\n" , duration, intervalnsec, expectedoverruns); its.it_interval.tv_sec =0 ; its.it_interval.tv_nsec = intervalnsec;
@@ -113,7 +118,7 @@ int main(void) }//tssleep.tv_nsec = valuensec + (expectedoverruns*intervalnsec); tssleep.tv_nsec =0 ; - tssleep.tv_sec =2 ; + tssleep.tv_sec = duration +1 ;if (nanosleep (&tssleep,NULL ) !=0 ) {perror ("nanosleep() did not return success\n" );return PTS_UNRESOLVED ;
Signature · 1 line
2.20.1
PATCH 1/1 open_posix/timer_getoverrun/2-3: Fix test for systems with low timer precision
----- Original Message -----
Quoted 2 lines
From: Joerg Vehlow <[email protected]>
Hi,
Quoted 3 lines
On systems with low timer precision the test always fails, because the allowed
maximum number of overruns is calculated from the expected overruns + 10%.
Did you mean 1% here?
fudge = expectedoverruns / 100;
Quoted 3 lines
If the expected overruns is less than 200, there is no tollerance. This happens, if the precision of the timer is less than or equal to 4ms. E.g. qemu-arm64 without high resolution timer the accuracy is only 4ms.
Would tweaking tolerance work too? E.g. use float, round up.
Quoted 23 lines
Signed-off-by Joerg Vehlow <[email protected]>--- .../conformance/interfaces/timer_getoverrun/2-3.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)diff --git a/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c b/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c index
96b7d01e6..3df3a9f01100644
---
a/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c
+++
b/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.ctestcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +1
-0@@ -55,6 +55,7 @@ int main(void)int overruns;int valuensec, intervalnsec, expectedoverruns;int fudge; +int duration;if (sigemptyset (&set) !=0 ) {perror ("sigemptyset() did not return success\n" );
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c int main(void) · line 102
Quoted 10 lines
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +5
-2@@ -94,11 +95,15 @@ int main(void) valuensec = tsres.tv_nsec; intervalnsec =2 * valuensec; -//expectedoverruns = (1000000000 - valuensec) / intervalnsec; - expectedoverruns =1000000000 / intervalnsec -1 ; + expectedoverruns =0 ; + duration =0 ; +while (expectedoverruns <1000 ) { + duration++; + expectedoverruns = duration * (1000000000 / intervalnsec -1 );
I was assuming -1 in original code is to cope with final timer expiration of tssleep.tv_sec, which might not be counted as "overrun". What does the -1 do in your formula? Why is it inside brackets?
When I try to force different interval values, it fails for me (on x86):
3ms
# ./timer_getoverrun_2-3.run-test duration = 7 sec, interval = 6000000 nsec, expected overruns = 1155 1166 overruns occurred FAIL: 1166 overruns sent; expected 1155
5ms
# ./timer_getoverrun_2-3.run-test duration = 11 sec, interval = 10000000 nsec, expected overruns = 1089 1100 overruns occurred FAIL: 1100 overruns sent; expected 1089
Regards,
Jan
PATCH 1/1 open_posix/timer_getoverrun/2-3: Fix test for systems with low timer precision
Hi,
Quoted 5 lines
Quoted 3 lines
On systems with low timer precision the test always fails, because the allowed
maximum number of overruns is calculated from the expected overruns + 10%.Did you mean 1% here?
fudge = expectedoverruns / 100;
Of course, you are right
Quoted 5 lines
Quoted 4 lines
If the expected overruns is less than 200, there is no tollerance. This happens, if the precision of the timer is less than or equal to 4ms. E.g. qemu-arm64 without high resolution timer the accuracy is only 4ms.
Would tweaking tolerance work too? E.g. use float, round up.
Actually rounding up would not work. I tried it with a timer accuracy of 4ms and I needed
at least a fudge of two. Rounding up would only result in one. That it why I decided to
go for extending the duration
Quoted 4 lines
I was assuming -1 in original code is to cope with final timer expiration of tssleep.tv_sec, which might not be counted as "overrun". What does the -1 do in your formula? Why is it inside brackets?
You may be right. I am not completely sure what it is used for, but I guess you are right.
Quoted 8 lines
When I try to force different interval values, it fails for me (on x86):
3ms# ./timer_getoverrun_2-3.run-test duration = 7 sec, interval = 6000000 nsec, expected overruns = 1155 1166 overruns occurred FAIL: 1166 overruns sent; expected 1155
Quoted 5 lines
5ms
# ./timer_getoverrun_2-3.run-test duration = 11 sec, interval = 10000000 nsec, expected overruns = 1089 1100 overruns occurred FAIL: 1100 overruns sent; expected 1089
How is it possible to force the timer resolution? I tested on qemu with aarch64 and x86_64 with and without highres=off and ir worked.
If the -1 is outside the brackets, it would fix both of your test cases. (1155 + 7 - 1)? = 1161; +1% = 1172 -> expected overruns [1161, 1172) (1089 + 11 - 1) = 1099; +1% = 1109 ->expected overruns [1099, 1109)
J?rg
PATCH 1/1 open_posix/timer_getoverrun/2-3: Fix test for systems with low timer precision
Quoted 19 lines · Joerg Vehlow
Actually rounding up would not work. I tried it with a timer accuracy of 4ms and I needed
at least a fudge of two. Rounding up would only result in one. That it why I decided to
go for extending the durationQuoted 4 lines
I was assuming -1 in original code is to cope with final timer expiration of tssleep.tv_sec, which might not be counted as "overrun". What does the -1 do in your formula? Why is it inside brackets?
You may be right. I am not completely sure what it is used for, but I guess you are right.
Quoted 8 lines
When I try to force different interval values, it fails for me (on x86):
3ms# ./timer_getoverrun_2-3.run-test duration = 7 sec, interval = 6000000 nsec, expected overruns = 1155 1166 overruns occurred FAIL: 1166 overruns sent; expected 1155
Quoted 7 lines
Quoted 5 lines
5ms
# ./timer_getoverrun_2-3.run-test duration = 11 sec, interval = 10000000 nsec, expected overruns = 1089 1100 overruns occurred FAIL: 1100 overruns sent; expected 1089How is it possible to force the timer resolution? I tested on qemu with aarch64 and
x86_64with and without highres=off and ir worked.
I didn't change timer resolution, I only used larger interval values as multiple of timer resolution:
intervalnsec = largeX * timer_resolution
I'd prefer we tweak the tolerance rather than make test run longer. I'm thinking just allow ~50ms of extra overruns, and don't be so strict about absolute number of overruns. (KVM guests and s390 lpars tend to suffer from higher steal time).
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +8 -3
@@ -94,11 +94,17 @@ int main(void) valuensec = tsres.tv_nsec; intervalnsec =2 * valuensec; -//expectedoverruns = (1000000000 - valuensec) / intervalnsec; expectedoverruns =1000000000 / intervalnsec -1 ; +/* +* waking up from sleep isn't instant, we can overshoot. +* Allow up to ~50ms worth of extra overruns. +*/ + fudge =50000000 / intervalnsec +1 ; +printf ("value = %d sec, interval = %d nsec, " -"expected overruns = %d\n" ,1 , intervalnsec, expectedoverruns); +"expected overruns = %d, fudge = %d\n" ,1 , + intervalnsec, expectedoverruns, fudge); its.it_interval.tv_sec =0 ; its.it_interval.tv_nsec = intervalnsec;
@@ -146,7 +152,6 @@ int main(void)* extra expiries after the nanosleep completes so do * a range check. */ - fudge = expectedoverruns /100 ;if (overruns >= expectedoverruns && overruns < expectedoverruns + fudge) {printf ("Test PASSED\n" );return PTS_PASS ;
PATCH 1/1 open_posix/timer_getoverrun/2-3: Fix test for systems with low timer precision
Quoted 5 lines
I didn't change timer resolution, I only used larger interval values as multiple of timer resolution:
? intervalnsec = largeX *timer_resolution
ah ok
Quoted 36 lines
I'd prefer we tweak the tolerance rather than make test run longer. I'm thinking just allow ~50ms of extra overruns, and don't be so strict about absolute number of overruns. (KVM guests and s390 lpars tend to suffer from higher steal time).
(quoted diff) +0
-0a/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c b/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c
index
96b7d01e6ffe..66f8b583a5a6100644
---
a/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c
+++
b/testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c(quoted diff) +6
-1@@ -94,11 +94,17 @@ int main(void) ??????? valuensec = tsres.tv_nsec; ??????? intervalnsec = 2 * valuensec; -?????? //expectedoverruns = (1000000000 - valuensec) / intervalnsec; ??????? expectedoverruns = 1000000000 / intervalnsec - 1; +?????? /* +??????? * waking up from sleep isn't instant, we can overshoot. +??????? * Allow up to ~50ms worth of extra overruns. +??????? */ +?????? fudge = 50000000 / intervalnsec + 1; +??????? printf("value = %d sec, interval = %d nsec, " -????????????? "expected overruns = %d\n", 1, intervalnsec, expectedoverruns);
+????????????? "expected overruns = %d, fudge = %d\n", 1, +????????????? intervalnsec, expectedoverruns, fudge);??????? its.it_interval.tv_sec = 0;
??????? its.it_interval.tv_nsec = intervalnsec;(quoted diff) +0
-0@@ -146,7 +152,6 @@ int main(void)
Quoted 9 lines
???????? * extra expiries after the nanosleep completes so do ???????? * a range check.
???????? */
-?????? fudge = expectedoverruns / 100;
??????? if (overruns >= expectedoverruns && overruns < expectedoverruns + fudge) {
???????????????printf("Test PASSED\n"); ??????????????? returnPTS_PASS;
I wonder if there was a way to do this more accurate, like busy waiting while signals
are blocked. But I would also be fine with your solution.
PATCH 1/1 timer_getoverrun/2-3: increase tolerance for overshoot
Joerg reports that test fails on systems with low timer precision (qemu-arm64, 4ms timer precision) due to way we calculate maximum number of overruns. If the expected overruns is less than 200, there is no tollerance.
Allow up to ~50ms of extra overruns.
--- .../conformance/interfaces/timer_getoverrun/2-3.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +8 -3
@@ -94,11 +94,17 @@ int main(void) valuensec = tsres.tv_nsec; intervalnsec =2 * valuensec; -//expectedoverruns = (1000000000 - valuensec) / intervalnsec; expectedoverruns =1000000000 / intervalnsec -1 ; +/* +* waking up from sleep isn't instant, we can overshoot. +* Allow up to ~50ms worth of extra overruns. +*/ + fudge =50000000 / intervalnsec +1 ; +printf ("value = %d sec, interval = %d nsec, " -"expected overruns = %d\n" ,1 , intervalnsec, expectedoverruns); +"expected overruns = %d, fudge = %d\n" ,1 , + intervalnsec, expectedoverruns, fudge); its.it_interval.tv_sec =0 ; its.it_interval.tv_nsec = intervalnsec;
@@ -146,7 +152,6 @@ int main(void)* extra expiries after the nanosleep completes so do * a range check. */ - fudge = expectedoverruns /100 ;if (overruns >= expectedoverruns && overruns < expectedoverruns + fudge) {printf ("Test PASSED\n" );return PTS_PASS ;
Signature · 1 line
1.8.3.1
PATCH 1/1 timer_getoverrun/2-3: increase tolerance for overshoot
Hi!
Quoted 19 lines
Joerg reports that test fails on systems with low timer precision (qemu-arm64, 4ms timer precision) due to way we calculate maximum number of overruns. If the expected overruns is less than 200, there is no tollerance.
Allow up to ~50ms of extra overruns.
Reported-by Joerg Vehlow <[email protected]>Acked-by Joerg Vehlow <[email protected]>Signed-off-by Jan Stancek <[email protected]>--- .../conformance/interfaces/timer_getoverrun/2-3.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +0
-0@@ -94,11 +94,17 @@ int main(void)
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c int main(void) · line 103
Quoted 10 lines
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +5
-1valuensec = tsres.tv_nsec; intervalnsec =2 * valuensec; -//expectedoverruns = (1000000000 - valuensec) / intervalnsec; expectedoverruns =1000000000 / intervalnsec -1 ; +/* +* waking up from sleep isn't instant, we can overshoot. +* Allow up to ~50ms worth of extra overruns. +*/ + fudge =50000000 / intervalnsec +1 ;
I wonder if we can measure the sleep with monotonic clock, then we can be more precise, since we would know how much we overshoot. But it's probably not worth of complicating the code.
Either way I'm fine with this version as well, acked.
Quoted 21 lines
testcases/open_posix_testsuite/conformance/interfaces/timer_getoverrun/2-3.c +2
-2printf ("value = %d sec, interval = %d nsec, " -"expected overruns = %d\n" ,1 , intervalnsec, expectedoverruns); +"expected overruns = %d, fudge = %d\n" ,1 , + intervalnsec, expectedoverruns, fudge); its.it_interval.tv_sec =0 ; its.it_interval.tv_nsec = intervalnsec;@@ -146,7 +152,6 @@ int main(void)* extra expiries after the nanosleep completes so do * a range check. */ - fudge = expectedoverruns /100 ;if (overruns >= expectedoverruns && overruns < expectedoverruns + fudge) {printf ("Test PASSED\n" );return PTS_PASS ;Signature · 5 lines
1.8.3.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 1/1 timer_getoverrun/2-3: increase tolerance for overshoot
----- Original Message -----
Quoted 6 lines
Hi!
I wonder if we can measure the sleep with monotonic clock, then we can be more precise, since we would know how much we overshoot. But it's probably not worth of complicating the code.Either way I'm fine with this version as well, acked.
I pushed current patch. If we find it's still causing issues I'll look into the idea with monotonic clock.
Thanks,
Jan
PATCH 1/1 timer_getoverrun/2-3: increase tolerance for overshoot
Hi,
Am 18.10.2019 um 09:58 schrieb Jan Stancek:
Quoted 10 lines
----- Original Message -----
Quoted 6 lines
Hi!
I wonder if we can measure the sleep with monotonic clock, then we can be more precise, since we would know how much we overshoot. But it's probably not worth of complicating the code.Either way I'm fine with this version as well, acked.
I pushed current patch. If we find it's still causing issues I'll look into the idea with monotonic clock.
Thanks for fixing this. I'd like to mention, that CLOCK_MONOTONIC is an optionalfeature in posix. I do not know if there are systems missing it, that might run these test.
J?rg