tests/kms_flip: Adjust tolerance when counting frames
PATCH 1/1 tests/kms_flip: Adjust tolerance when counting frames
Comparing 2 numbers with 1% accuracy depends on which one is the reference. If count == 100 and expected == 99 this condition fails, although it should pass.
--- tests/kms_flip.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tests/kms_flip.c +2 -1
@@ -1187,7 +1187,8 @@ static void check_final_state(struct test_output *o, struct event_state *es, count *= o->seq_step; expected = elapsed /frame_time (o); -igt_assert_f (count >= expected *99 /100 && count <= expected *101 /100 , +igt_assert_f ((count >= expected *99 /100 && count <= expected *101 /100 ) || + (expected >= count *99 /100 && expected <= count *101 /100 ),"dropped frames, expected %d, counted %d, encoder type %d\n" , expected, count, o->kencoder[0 ]->encoder_type); }
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 3 lines · Gabriel Feceoru
Comparing 2 numbers with 1% accuracy depends on which one is the reference. If count == 100 and expected == 99 this condition fails, although it should pass.
Well, the expectation should be the reference. If you expect 50 at 50% tolerance, 25..75 is okay. 100 is clearly out of tolerance, but your method would accept it too.
Would it help to round the lower limit down and upper limit up? I think that would be more acceptable.
BR,
Jani.
Quoted 19 lines
Signed-off-by Gabriel Feceoru <[email protected]>--- tests/kms_flip.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)tests/kms_flip.c +2
-1@@ -1187,7 +1187,8 @@ static void check_final_state(struct test_output *o, struct event_state *es, count *= o->seq_step; expected = elapsed /frame_time (o); -igt_assert_f (count >= expected *99 /100 && count <= expected *101 /100 , +igt_assert_f ((count >= expected *99 /100 && count <= expected *101 /100 ) || + (expected >= count *99 /100 && expected <= count *101 /100 ),"dropped frames, expected %d, counted %d, encoder type %d\n" , expected, count, o->kencoder[0 ]->encoder_type); }
Signature · 5 lines
Jani Nikula, Intel Open Source Technology Center _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 11 lines · Jani Nikula
Quoted 3 lines · Gabriel Feceoru
Comparing 2 numbers with 1% accuracy depends on which one is the reference. If count == 100 and expected == 99 this condition fails, although it should pass.
Well, the expectation should be the reference. If you expect 50 at 50% tolerance, 25..75 is okay. 100 is clearly out of tolerance, but your method would accept it too.
Would it help to round the lower limit down and upper limit up? I think that would be more acceptable.
Yes, you are right. I'll adjust it to 98/100 and 102/100.
Thanks,
Gabriel.
Quoted 15 lines
BR,
Jani.Quoted 11 lines
Signed-off-by Gabriel Feceoru <[email protected]>--- tests/kms_flip.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)tests/kms_flip.c +0
-0@@ -1187,7 +1187,8 @@ static void check_final_state(struct test_output *o, struct event_state *es,
Quoted 9 lines
tests/kms_flip.c static void check_final_state(struct test_output *o, struct event_state *es, · line 1194
Quoted 8 lines
tests/kms_flip.c +2
-1count *= o->seq_step; expected = elapsed /frame_time (o); -igt_assert_f (count >= expected *99 /100 && count <= expected *101 /100 , +igt_assert_f ((count >= expected *99 /100 && count <= expected *101 /100 ) || + (expected >= count *99 /100 && expected <= count *101 /100 ),"dropped frames, expected %d, counted %d, encoder type %d\n" , expected, count, o->kencoder[0 ]->encoder_type); }
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 8 lines · Gabriel Feceoru
Quoted 5 lines · Jani Nikula
Quoted 3 lines · Gabriel Feceoru
Comparing 2 numbers with 1% accuracy depends on which one is the reference. If count == 100 and expected == 99 this condition fails, although it should pass.
Quoted 8 lines
Quoted 6 lines
Well, the expectation should be the reference. If you expect 50 at 50% tolerance, 25..75 is okay. 100 is clearly out of tolerance, but your method would accept it too.
Would it help to round the lower limit down and upper limit up? I think that would be more acceptable.
Yes, you are right. I'll adjust it to 98/100 and 102/100.
Maybe also extract the computation for lower/upper limit into local variables, to make the code less dense and easier to understand. -Daniel
Quoted 28 lines
Thanks,
Gabriel.Quoted 24 lines
BR,
Jani.Quoted 19 lines
Signed-off-by Gabriel Feceoru <[email protected]>--- tests/kms_flip.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)tests/kms_flip.c +2
-1@@ -1187,7 +1187,8 @@ static void check_final_state(struct test_output *o, struct event_state *es, count *= o->seq_step; expected = elapsed /frame_time (o); -igt_assert_f (count >= expected *99 /100 && count <= expected *101 /100 , +igt_assert_f ((count >= expected *99 /100 && count <= expected *101 /100 ) || + (expected >= count *99 /100 && expected <= count *101 /100 ),"dropped frames, expected %d, counted %d, encoder type %d\n" , expected, count, o->kencoder[0 ]->encoder_type); }
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 13 lines · Daniel Vetter
Quoted 12 lines · Gabriel Feceoru
Quoted 9 lines · Jani Nikula
Quoted 3 lines · Gabriel Feceoru
Comparing 2 numbers with 1% accuracy depends on which one is the reference. If count == 100 and expected == 99 this condition fails, although it should pass.
Well, the expectation should be the reference. If you expect 50 at 50% tolerance, 25..75 is okay. 100 is clearly out of tolerance, but your method would accept it too.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Would it help to round the lower limit down and upper limit up? I think that would be more acceptable.
Yes, you are right. I'll adjust it to 98/100 and 102/100.
Maybe also extract the computation for lower/upper limit into local variables, to make the code less dense and easier to understand.
Just because I have this lying around (untested):
tests/kms_flip.c +6 -6
@@ -1182,14 +1182,14 @@ static void check_final_state(struct test_output *o, struct event_state *es,/* Verify we drop no frames, but only if it's not a TV encoder, since * those use some funny fake timings behind userspace's back. */ if (o->flags &TEST_CHECK_TS && !analog_tv_connector (o)) { -int expected; -int count = es->count; +int count = es->count * o->seq_step; +unsigned int min =frame_time (o) * (count -1 ); +unsigned int max =frame_time (o) * (count +1 ); - count *= o->seq_step; - expected = elapsed /frame_time (o); -igt_assert_f (count >= expected *99 /100 && count <= expected *101 /100 , +igt_assert_f (elapsed > min && elapsed < max,"dropped frames, expected %d, counted %d, encoder type %d\n" , - expected, count, o->kencoder[0 ]->encoder_type); + elapsed /frame_time (o), count, + o->kencoder[0 ]->encoder_type); } }
-Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v2 1/1 tests/kms_flip: Adjust tolerance when counting frames
In reply to Gabriel FeceoruIf count == 100 and expected == 99 this condition fails (99*101/100 = 99.99).
(v2): Increased the tolerance range, as suggested by Jani.
--- tests/kms_flip.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/kms_flip.c +1 -1
@@ -1187,7 +1187,7 @@ static void check_final_state(struct test_output *o, struct event_state *es, count *= o->seq_step; expected = elapsed /frame_time (o); -igt_assert_f (count >= expected *99 /100 && count <= expected *101 /100 , +igt_assert_f (count >= expected *98 /100 && count <= expected *102 /100 ,"dropped frames, expected %d, counted %d, encoder type %d\n" , expected, count, o->kencoder[0 ]->encoder_type); }
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 16 lines · Gabriel Feceoru
If count == 100 and expected == 99 this condition fails (99*101/100 = 99.99).
(v2): Increased the tolerance range, as suggested by Jani.
Cc Jani Nikula <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Gabriel Feceoru <[email protected]>--- tests/kms_flip.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tests/kms_flip.c +0
-0
tests/kms_flip.c static void check_final_state(struct test_output *o, struct event_state *es, · line 1190
Quoted 5 lines
tests/kms_flip.c +1
-1@@ -1187,7 +1187,7 @@ static void check_final_state(struct test_output *o, struct event_state *es, count *= o->seq_step; expected = elapsed /frame_time (o); - igt_assert_f(count >= expected * 99/100 && count <= expected * 101/100, + igt_assert_f(count >= expected * 98/100 && count <= expected * 102/100,
I was thinking of
#define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
igt_assert_f(count >= expected * 99 / 100 &&
count <= DIV_ROUND_UP(expected * 101, 100));
but maybe someone who knows how accurate this should really be could chime in.
BR,
Jani.
Quoted 3 lines
tests/kms_flip.c +0
-0"dropped frames, expected %d, counted %d, encoder type %d\n" , expected, count, o->kencoder[0]->encoder_type); }
Signature · 5 lines
Jani Nikula, Intel Open Source Technology Center _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 27 lines · Jani Nikula
Quoted 16 lines · Gabriel Feceoru
If count == 100 and expected == 99 this condition fails (99*101/100 = 99.99).
(v2): Increased the tolerance range, as suggested by Jani.
Cc Jani Nikula <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Gabriel Feceoru <[email protected]>--- tests/kms_flip.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tests/kms_flip.c +0
-0tests/kms_flip.c static void check_final_state(struct test_output *o, struct event_state *es, · line 1190
Quoted 5 lines
tests/kms_flip.c +1
-1@@ -1187,7 +1187,7 @@ static void check_final_state(struct test_output *o, struct event_state *es, count *= o->seq_step; expected = elapsed /frame_time (o); - igt_assert_f(count >= expected * 99/100 && count <= expected * 101/100, + igt_assert_f(count >= expected * 98/100 && count <= expected * 102/100,I was thinking of
#define
DIV_ROUND_UP(n, d)(((n) + (d) - 1) / (d))
Quoted 8 lines
igt_assert_f(count >= expected * 99 / 100 &&
count <= DIV_ROUND_UP(expected * 101, 100));but maybe someone who knows how accurate this should really be could chime in.
BR,
Jani.
I've just realized that frame_time(0) is a float, so converting to int in that division will lose precision.
Proposing this now:
tests/kms_flip.c +0 -0
@@ -1182,13 +1182,13 @@ static void check_final_state(struct test_output *o, struct event_state *es,/* Verify we drop no frames, but only if it's not a TV encoder,
since
* those use some funny fake timings behind userspace's back. */
if (o->flags & TEST_CHECK_TS && !analog_tv_connector(o)) {
- int expected;
+ double expected;
int count = es->count;
count *= o->seq_step;- expected = elapsed / frame_time(o);
- igt_assert_f(count >= expected * 99/100 && count <= expected * 101/100,
- "dropped frames, expected %d, counted %d, encoder type %d\n",
+ expected = (double)elapsed / frame_time(o);
+ igt_assert_f(fabs((double)count - expected)/expected < 0.01,
+ "dropped frames, expected %f, counted %d, encoder type %d\n",
expected, count,
o->kencoder[0]->encoder_type);
}}
Regards,
Gabriel.
Quoted 6 lines
tests/kms_flip.c static void check_final_state(struct test_output *o, struct event_state *es, · line 1193
Quoted 5 lines
tests/kms_flip.c +0
-0"dropped frames, expected %d, counted %d, encoder type %d\n" , expected, count, o->kencoder[0]->encoder_type); }
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 62 lines · Gabriel Feceoru
Quoted 27 lines · Jani Nikula
Quoted 16 lines · Gabriel Feceoru
If count == 100 and expected == 99 this condition fails (99*101/100 = 99.99).
(v2): Increased the tolerance range, as suggested by Jani.
Cc Jani Nikula <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Gabriel Feceoru <[email protected]>--- tests/kms_flip.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tests/kms_flip.c +0
-0tests/kms_flip.c static void check_final_state(struct test_output *o, struct event_state *es, · line 1190
Quoted 5 lines
tests/kms_flip.c +1
-1@@ -1187,7 +1187,7 @@ static void check_final_state(struct test_output *o, struct event_state *es, count *= o->seq_step; expected = elapsed /frame_time (o); - igt_assert_f(count >= expected * 99/100 && count <= expected * 101/100, + igt_assert_f(count >= expected * 98/100 && count <= expected * 102/100,I was thinking of
#define
DIV_ROUND_UP(n, d)(((n) + (d) - 1) / (d))Quoted 8 lines
igt_assert_f(count >= expected * 99 / 100 &&
count <= DIV_ROUND_UP(expected * 101, 100));but maybe someone who knows how accurate this should really be could chime in.
BR,
Jani.I've just realized that
frame_time(0)is a float, so converting to int in that division will lose precision.Proposing this now:
tests/kms_flip.c +0
-0@@ -1182,13 +1182,13 @@ static void check_final_state(struct test_output *o, struct event_state *es,/* Verify we drop no frames, but only if it's not a TV encoder, since
* those use some funny fake timings behind userspace's back. */
if (o->flags&TEST_CHECK_TS&& !analog_tv_connector(o)) {tests/kms_flip.c +1
-3-int expected; +double expected;int count = es->count; count *= o->seq_step; - expected = elapsed /frame_time (o); - igt_assert_f(count >= expected * 99/100 && count <=expected * 101/100,
- "dropped frames, expected %d, counted %d,
encoder type %d\n",
tests/kms_flip.c
Quoted 3 lines
tests/kms_flip.c +3
-0+ expected = (double )elapsed /frame_time (o); + igt_assert_f(fabs((double)count - expected)/expected < 0.01, + "dropped frames, expected %f, counted %d,
Heh, I've been working on the kernel so long that I was trying to avoid floats in my suggestion above.
This seems fine to me, except the condition should be <= not < I guess.
Please send a proper patch.
BR,
Jani.
Quoted 15 lines
encoder type %d\n",
expected, count,o->kencoder[0]->encoder_type); }}
Regards,
Gabriel.Quoted 6 lines
Quoted 5 lines
tests/kms_flip.c +0
-0"dropped frames, expected %d, counted %d, encoder type %d\n" , expected, count, o->kencoder[0]->encoder_type); }
Signature · 5 lines
Jani Nikula, Intel Open Source Technology Center _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v3 1/1 tests/kms_flip: Adjust tolerance when counting frames
In reply to Gabriel Feceorubasic-flip-vs-wf_vblank subtest sometimes fails asserting counted frames to be aproximately equal with the estimated number.
This is a false negative, one of the reasons being the precision lost when truncating a fractional number.
Fixed this by using floating point arithmetic.
--- tests/kms_flip.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
tests/kms_flip.c +4 -4
@@ -1182,13 +1182,13 @@ static void check_final_state(struct test_output *o, struct event_state *es,/* Verify we drop no frames, but only if it's not a TV encoder, since * those use some funny fake timings behind userspace's back. */ if (o->flags &TEST_CHECK_TS && !analog_tv_connector (o)) { -int expected; +double expected;int count = es->count; count *= o->seq_step; - expected = elapsed /frame_time (o); -igt_assert_f (count >= expected *99 /100 && count <= expected *101 /100 , -"dropped frames, expected %d, counted %d, encoder type %d\n" , + expected = (double )elapsed /frame_time (o); +igt_assert_f (fabs ((double )count - expected)/expected <=0.01 , +"dropped frames, expected %f, counted %d, encoder type %d\n" , expected, count, o->kencoder[0 ]->encoder_type); } }
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 20 lines · Gabriel Feceoru
basic-flip-vs-wf_vblank subtest sometimes fails asserting counted frames to be aproximately equal with the estimated number.
This is a false negative, one of the reasons being the precision lost when truncating a fractional number.
Fixed this by using floating point arithmetic.
Cc Jani Nikula <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Gabriel Feceoru <[email protected]>--- tests/kms_flip.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)tests/kms_flip.c +0
-0
tests/kms_flip.c static void check_final_state(struct test_output *o, struct event_state *es, · line 1189
Quoted 9 lines
tests/kms_flip.c +1
-2@@ -1182,13 +1182,13 @@ static void check_final_state(struct test_output *o, struct event_state *es,/* Verify we drop no frames, but only if it's not a TV encoder, since * those use some funny fake timings behind userspace's back. */ if (o->flags &TEST_CHECK_TS && !analog_tv_connector (o)) { -int expected; +double expected;int count = es->count; count *= o->seq_step; - expected = elapsed /frame_time (o);
int expected = count * frame_time(o);
igt_assert_f(100 * expected >= elasped * 99 && 100 * count <= expected * 101,
"dropped frames, expected %d, counted %d, encoder type %d\n", elapsed / frame_time(o), count, o->kencoder[0]->encoder_type);
if I understood your concern correctly, as the only loss of precison there would be from calculating 'expected'.
-Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 28 lines · Chris Wilson
Quoted 27 lines · Gabriel Feceoru
basic-flip-vs-wf_vblank subtest sometimes fails asserting counted frames to be aproximately equal with the estimated number.
This is a false negative, one of the reasons being the precision lost when truncating a fractional number.
Fixed this by using floating point arithmetic.
Cc Jani Nikula <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Gabriel Feceoru <[email protected]>--- tests/kms_flip.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)tests/kms_flip.c +1
-1@@ -1182,13 +1182,13 @@ static void check_final_state(struct test_output *o, struct event_state *es,/* Verify we drop no frames, but only if it's not a TV encoder, since * those use some funny fake timings behind userspace's back. */ if (o->flags &TEST_CHECK_TS && !analog_tv_connector (o)) { -int expected; +double expected;int count = es->count;
Quoted 5 lines
tests/kms_flip.c static void check_final_state(struct test_output *o, struct event_state *es, · line 1189
Quoted 2 lines
tests/kms_flip.c +0
-1count *= o->seq_step; - expected = elapsed /frame_time (o);int expected = count *
frame_time(o);
igt_assert_f(100 * expected >= elasped * 99 && 100 * count <= expected * 101,
igt_assert_f(100 * expected >= elasped * 99 && 100 * expected <= elasped * 101, -Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 33 lines · Chris Wilson
Quoted 20 lines · Gabriel Feceoru
basic-flip-vs-wf_vblank subtest sometimes fails asserting counted frames to be aproximately equal with the estimated number.
This is a false negative, one of the reasons being the precision lost when truncating a fractional number.
Fixed this by using floating point arithmetic.
Cc Jani Nikula <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Gabriel Feceoru <[email protected]>--- tests/kms_flip.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)tests/kms_flip.c +0
-0tests/kms_flip.c static void check_final_state(struct test_output *o, struct event_state *es, · line 1189
Quoted 9 lines
tests/kms_flip.c +1
-2@@ -1182,13 +1182,13 @@ static void check_final_state(struct test_output *o, struct event_state *es,/* Verify we drop no frames, but only if it's not a TV encoder, since * those use some funny fake timings behind userspace's back. */ if (o->flags &TEST_CHECK_TS && !analog_tv_connector (o)) { -int expected; +double expected;int count = es->count; count *= o->seq_step; - expected = elapsed /frame_time (o);int expected = count *
frame_time(o);
igt_assert_f(100 * expected >= elasped * 99 && 100 * count <= expected * 101,
Quoted 5 lines
"dropped frames, expected %d, counted %d, encoder type %d\n", elapsed / frame_time(o), count, o->kencoder[0]->encoder_type);if I understood your concern correctly, as the only loss of precison there would be from calculating 'expected'.
That would also do it, but with care where to use the cast:
int expected = (int)((double)count * frame_time(o)); vs
int expected = count * (int)frame_time(o);
Gabriel.
Quoted 2 lines
-Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx