realtime: Fix the pass criterion of pi_perf test case.
PATCH 1/1 realtime: Fix the pass criterion of pi_perf test case.
Hi,
The pass criterion in pi_perf test case is wrong. It compares the minimum amount of time taken by the low priority thread with the maximum amount of time taken by the high priority thread to calculate the PI delay. Obviously, these min and max don't necessarily happen in the same iteration, resulting in a number of false failures. The correct way is to compare the time taken by low and high priority threads in each iteration and then find the maximum delay experienced by high priority thread across the iterations. This patch implements the change. Additionally, this patch removes lock_wait_dat array, which is not needed anymore as well as makes a couple of messages easier to understand.
This patch changes the messages displayed by this test case slightly. They look like the following now:
Low prio thread started
High prio thread started
Busy 0 started
Busy 1 started
Busy 2 started
Busy 3 started
Time taken for high prio thread to get the lock once released by low prio thread
Min delay = 10 us
Max delay = 36 us
Average delay = 17.06 us
Standard Deviation = 8.11 us
Quantiles:
99.0% < 36
Criteria: High prio lock wait time < (Low prio lock held time + 200 us)
Result: PASS
I have tested this by running 1000 iterations of pi_perf test on a couple of machines.
Thanks,
Sripathi.
---
realtime: Fix the pass criterion of pi_perf test case.
testcases/realtime/func/pi_perf/pi_perf.c +12 -14
@@ -1,6 +1,6 @@/****************************************************************************** * -* Copyright © International Business Machines Corp., 2007, 2008 +* Copyright © International Business Machines Corp., 2007, 2008, 2009 * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by
@@ -36,6 +36,8 @@* * HISTORY * 2007-Nov-20: Initial version by Sripathi Kodi <[email protected]> +* 2009-Jul-03: Pass criteria corrected by Sripathi Kodi +* <[email protected]> * *****************************************************************************/
@@ -71,9 +73,9 @@ static unsigned int high_work_time = DEFstatic unsigned int busy_work_time;static int num_busy =-1 ; -nsec_t low_unlock, high_get_lock; +nsec_t low_unlock, max_pi_delay; -stats_container_t lock_wait_dat, low_dat, cpu_delay_dat; +stats_container_t low_dat, cpu_delay_dat;stats_container_t cpu_delay_hist;stats_quantiles_t cpu_delay_quantiles;
@@ -165,10 +167,9 @@ void * low_prio_thread(void *arg)void *high_prio_thread (void *arg) { -nsec_t high_start, high_spent, high_end; +nsec_t high_start, high_end, high_get_lock;unsigned int i; -stats_container_init (&lock_wait_dat, iterations);stats_container_init (&cpu_delay_dat, iterations);stats_container_init (&cpu_delay_hist,HIST_BUCKETS );stats_quantiles_init (&cpu_delay_quantiles, (int )log10 (iterations));
@@ -184,14 +185,11 @@ void * high_prio_thread(void *arg) high_start =rt_gettime ();pthread_mutex_lock (&lock); high_end =rt_gettime (); - high_spent = high_end - high_start; high_get_lock = high_end - low_unlock;busy_work_ms (high_work_time);pthread_mutex_unlock (&lock); - lock_wait_dat.records[i].x = i; - lock_wait_dat.records[i].y = high_spent /NS_PER_US ; cpu_delay_dat.records[i].x = i; cpu_delay_dat.records[i].y = high_get_lock /NS_PER_US ;
@@ -206,14 +204,16 @@ void * high_prio_thread(void *arg) "Latency (us)", "Samples", &cpu_delay_hist, "steps");printf ("Time taken for high prio thread to get the lock once released by low prio thread\n" ); -printf ("Min wait time = %ld us\n" ,stats_min (&cpu_delay_dat)); -printf ("Max wait time = %ld us\n" ,stats_max (&cpu_delay_dat)); -printf ("Average wait time = %4.2f us\n" ,stats_avg (&cpu_delay_dat)); +printf ("Min delay = %ld us\n" ,stats_min (&cpu_delay_dat)); +printf ("Max delay = %ld us\n" ,stats_max (&cpu_delay_dat)); +printf ("Average delay = %4.2f us\n" ,stats_avg (&cpu_delay_dat));printf ("Standard Deviation = %4.2f us\n" ,stats_stddev (&cpu_delay_dat));printf ("Quantiles:\n" );stats_quantiles_calc (&cpu_delay_dat, &cpu_delay_quantiles);stats_quantiles_print (&cpu_delay_quantiles); + max_pi_delay =stats_max (&cpu_delay_dat); +return NULL ; }
@@ -261,13 +261,11 @@ int main(int argc, char *argv[]) }join_threads (); -printf ("Low prio lock held time (min) = %ld us\n" ,stats_min (&low_dat)); -printf ("High prio lock wait time (max) = %ld us\n" ,stats_max (&lock_wait_dat));printf ("Criteria: High prio lock wait time < " "(Low prio lock held time + %d us)\n" , (int )pass_criteria); ret =0 ; -if (stats_max (&lock_wait_dat) >stats_min (&low_dat) + (int )pass_criteria) +if (max_pi_delay > pass_criteria) ret =1 ;printf ("Result: %s\n" , ret ?"FAIL" :"PASS" );
------------------------------------------------------------------------------
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
Quoted 36 lines · Sripathi Kodi
Hi,
The pass criterion in
pi_perftest case is wrong. It compares the minimum amount of time taken by the low priority thread with the maximum amount of time taken by the high priority thread to calculate the PI delay. Obviously, these min and max don't necessarily happen in the same iteration, resulting in a number of false failures. The correct way is to compare the time taken by low and high priority threads in each iteration and then find the maximum delay experienced by high priority thread across the iterations. This patch implements the change. Additionally, this patch removeslock_wait_datarray, which is not needed anymore as well as makes a couple of messages easier to understand.This patch changes the messages displayed by this test case slightly. They look like the following now:
Low prio thread started
High prio thread started
Busy 0 started
Busy 1 started
Busy 2 started
Busy 3 started
Time taken for high prio thread to get the lock once released by low prio thread
Min delay = 10 us
Max delay = 36 us
Average delay = 17.06 us
Standard Deviation = 8.11 us
Quantiles:
99.0% < 36
Criteria: High prio lock wait time < (Low prio lock held time + 200 us)
Result: PASSI have tested this by running 1000 iterations of
pi_perftest on a couple of machines.
Quoted 8 lines
Thanks,
Sripathi.---
realtime: Fix the pass criterion of
pi_perftest case.Signed-off-by Sripathi Kodi <[email protected]>
Hi Sri,
Is this based on the latest snapshot. I get the following failures:
patching file testcases/realtime/func/pi_perf/pi_perf.c Hunk #1 FAILED at 1.
Can you please resend ?
Regards--
Subrata
Quoted 89 lines
Acked-by Will Schmidt <[email protected]>Acked-by Gowrishankar <[email protected]>diff -uprN
ltp_cvs_3jul_org/testcases/realtime/func/pi_perf/pi_perf.cltp_cvs_3jul/testcases/realtime/func/pi_perf/pi_perf.ctestcases/realtime/func/pi_perf/pi_perf.c +11
-12@@ -1,6 +1,6 @@/****************************************************************************** * -* Copyright © International Business Machines Corp., 2007, 2008 +* Copyright © International Business Machines Corp., 2007, 2008, 2009 * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -36,6 +36,8 @@* * HISTORY * 2007-Nov-20: Initial version by Sripathi Kodi <[email protected]> +* 2009-Jul-03: Pass criteria corrected by Sripathi Kodi +* <[email protected]> * *****************************************************************************/ @@ -71,9 +73,9 @@ static unsigned int high_work_time = DEFstatic unsigned int busy_work_time;static int num_busy =-1 ; -nsec_t low_unlock, high_get_lock; +nsec_t low_unlock, max_pi_delay; -stats_container_t lock_wait_dat, low_dat, cpu_delay_dat; +stats_container_t low_dat, cpu_delay_dat;stats_container_t cpu_delay_hist;stats_quantiles_t cpu_delay_quantiles;@@ -165,10 +167,9 @@ void * low_prio_thread(void *arg)void *high_prio_thread (void *arg) { -nsec_t high_start, high_spent, high_end; +nsec_t high_start, high_end, high_get_lock;unsigned int i; -stats_container_init (&lock_wait_dat, iterations);stats_container_init (&cpu_delay_dat, iterations);stats_container_init (&cpu_delay_hist,HIST_BUCKETS );stats_quantiles_init (&cpu_delay_quantiles, (int )log10 (iterations));@@ -184,14 +185,11 @@ void * high_prio_thread(void *arg) high_start =rt_gettime ();pthread_mutex_lock (&lock); high_end =rt_gettime (); - high_spent = high_end - high_start; high_get_lock = high_end - low_unlock;busy_work_ms (high_work_time);pthread_mutex_unlock (&lock); - lock_wait_dat.records[i].x = i; - lock_wait_dat.records[i].y = high_spent /NS_PER_US ; cpu_delay_dat.records[i].x = i; cpu_delay_dat.records[i].y = high_get_lock /NS_PER_US ;@@ -206,14 +204,16 @@ void * high_prio_thread(void *arg) "Latency (us)", "Samples", &cpu_delay_hist, "steps");printf ("Time taken for high prio thread to get the lock once released by low prio thread\n" ); -printf ("Min wait time = %ld us\n" ,stats_min (&cpu_delay_dat)); -printf ("Max wait time = %ld us\n" ,stats_max (&cpu_delay_dat)); -printf ("Average wait time = %4.2f us\n" ,stats_avg (&cpu_delay_dat)); +printf ("Min delay = %ld us\n" ,stats_min (&cpu_delay_dat)); +printf ("Max delay = %ld us\n" ,stats_max (&cpu_delay_dat)); +printf ("Average delay = %4.2f us\n" ,stats_avg (&cpu_delay_dat));printf ("Standard Deviation = %4.2f us\n" ,stats_stddev (&cpu_delay_dat));printf ("Quantiles:\n" );stats_quantiles_calc (&cpu_delay_dat, &cpu_delay_quantiles);stats_quantiles_print (&cpu_delay_quantiles); + max_pi_delay =stats_max (&cpu_delay_dat); +return NULL ; }@@ -261,13 +261,11 @@ int main(int argc, char *argv[]) }join_threads (); -printf ("Low prio lock held time (min) = %ld us\n" ,stats_min (&low_dat));
testcases/realtime/func/pi_perf/pi_perf.c int main(int argc, char *argv[]) · line 271
Quoted 10 lines
testcases/realtime/func/pi_perf/pi_perf.c +1
-2-printf ("High prio lock wait time (max) = %ld us\n" ,stats_max (&lock_wait_dat));printf ("Criteria: High prio lock wait time < " "(Low prio lock held time + %d us)\n" , (int )pass_criteria); ret =0 ; -if (stats_max (&lock_wait_dat) >stats_min (&low_dat) + (int )pass_criteria) +if (max_pi_delay > pass_criteria) ret =1 ;printf ("Result: %s\n" , ret ?"FAIL" :"PASS" );
------------------------------------------------------------------------------
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
Quoted 48 lines · On Monday 06 July 2009 21:20:49 Subrata Modak wrote:
Quoted 36 lines · Sripathi Kodi
Hi,
The pass criterion in
pi_perftest case is wrong. It compares the minimum amount of time taken by the low priority thread with the maximum amount of time taken by the high priority thread to calculate the PI delay. Obviously, these min and max don't necessarily happen in the same iteration, resulting in a number of false failures. The correct way is to compare the time taken by low and high priority threads in each iteration and then find the maximum delay experienced by high priority thread across the iterations. This patch implements the change. Additionally, this patch removeslock_wait_datarray, which is not needed anymore as well as makes a couple of messages easier to understand.This patch changes the messages displayed by this test case slightly. They look like the following now:
Low prio thread started
High prio thread started
Busy 0 started
Busy 1 started
Busy 2 started
Busy 3 started
Time taken for high prio thread to get the lock once released by low prio thread
Min delay = 10 us
Max delay = 36 us
Average delay = 17.06 us
Standard Deviation = 8.11 us
Quantiles:
99.0% < 36
Criteria: High prio lock wait time < (Low prio lock held time + 200 us)
Result: PASSI have tested this by running 1000 iterations of
pi_perftest on a couple of machines.Quoted 8 lines
Thanks,
Sripathi.---
realtime: Fix the pass criterion of
pi_perftest case.Signed-off-by Sripathi Kodi <[email protected]>Hi Sri,
Quoted 7 lines
Is this based on the latest snapshot. I get the following failures:
patching file
testcases/realtime/func/pi_perf/pi_perf.cHunk #1 FAILED at 1.Can you please resend ?
Sorry Subrata, the patch seems to be fine on my machine, but is going bad in my mail client. If you don't mind, I have attached the patch to this mail for now.
Thanks,
Sripathi.
--
realtime: Fix the pass criterion of pi_perf test case.
<snip>
testcases/realtime/func/pi_perf/pi_perf.c +12 -14
@@ -1,6 +1,6 @@/****************************************************************************** * -* Copyright © International Business Machines Corp., 2007, 2008 +* Copyright © International Business Machines Corp., 2007, 2008, 2009 * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by
@@ -36,6 +36,8 @@* * HISTORY * 2007-Nov-20: Initial version by Sripathi Kodi <[email protected]> +* 2009-Jul-03: Pass criteria corrected by Sripathi Kodi +* <[email protected]> * *****************************************************************************/
@@ -71,9 +73,9 @@ static unsigned int high_work_time = DEFstatic unsigned int busy_work_time;static int num_busy =-1 ; -nsec_t low_unlock, high_get_lock; +nsec_t low_unlock, max_pi_delay; -stats_container_t lock_wait_dat, low_dat, cpu_delay_dat; +stats_container_t low_dat, cpu_delay_dat;stats_container_t cpu_delay_hist;stats_quantiles_t cpu_delay_quantiles;
@@ -165,10 +167,9 @@ void * low_prio_thread(void *arg)void *high_prio_thread (void *arg) { -nsec_t high_start, high_spent, high_end; +nsec_t high_start, high_end, high_get_lock;unsigned int i; -stats_container_init (&lock_wait_dat, iterations);stats_container_init (&cpu_delay_dat, iterations);stats_container_init (&cpu_delay_hist,HIST_BUCKETS );stats_quantiles_init (&cpu_delay_quantiles, (int )log10 (iterations));
@@ -184,14 +185,11 @@ void * high_prio_thread(void *arg) high_start =rt_gettime ();pthread_mutex_lock (&lock); high_end =rt_gettime (); - high_spent = high_end - high_start; high_get_lock = high_end - low_unlock;busy_work_ms (high_work_time);pthread_mutex_unlock (&lock); - lock_wait_dat.records[i].x = i; - lock_wait_dat.records[i].y = high_spent /NS_PER_US ; cpu_delay_dat.records[i].x = i; cpu_delay_dat.records[i].y = high_get_lock /NS_PER_US ;
@@ -206,14 +204,16 @@ void * high_prio_thread(void *arg) "Latency (us)", "Samples", &cpu_delay_hist, "steps");printf ("Time taken for high prio thread to get the lock once released by low prio thread\n" ); -printf ("Min wait time = %ld us\n" ,stats_min (&cpu_delay_dat)); -printf ("Max wait time = %ld us\n" ,stats_max (&cpu_delay_dat)); -printf ("Average wait time = %4.2f us\n" ,stats_avg (&cpu_delay_dat)); +printf ("Min delay = %ld us\n" ,stats_min (&cpu_delay_dat)); +printf ("Max delay = %ld us\n" ,stats_max (&cpu_delay_dat)); +printf ("Average delay = %4.2f us\n" ,stats_avg (&cpu_delay_dat));printf ("Standard Deviation = %4.2f us\n" ,stats_stddev (&cpu_delay_dat));printf ("Quantiles:\n" );stats_quantiles_calc (&cpu_delay_dat, &cpu_delay_quantiles);stats_quantiles_print (&cpu_delay_quantiles); + max_pi_delay =stats_max (&cpu_delay_dat); +return NULL ; }
@@ -261,13 +261,11 @@ int main(int argc, char *argv[]) }join_threads (); -printf ("Low prio lock held time (min) = %ld us\n" ,stats_min (&low_dat)); -printf ("High prio lock wait time (max) = %ld us\n" ,stats_max (&lock_wait_dat));printf ("Criteria: High prio lock wait time < " "(Low prio lock held time + %d us)\n" , (int )pass_criteria); ret =0 ; -if (stats_max (&lock_wait_dat) >stats_min (&low_dat) + (int )pass_criteria) +if (max_pi_delay > pass_criteria) ret =1 ;printf ("Result: %s\n" , ret ?"FAIL" :"PASS" );
------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge
This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry App World(TM) will have the opportunity to enter the BlackBerry Developer Challenge. See full prize details at: http://p.sf.net/sfu/blackberry
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
Quoted 55 lines · Sripathi Kodi
Quoted 54 lines · On Monday 06 July 2009 21:20:49 Subrata Modak wrote:
Quoted 36 lines · Sripathi Kodi
Hi,
The pass criterion in
pi_perftest case is wrong. It compares the minimum amount of time taken by the low priority thread with the maximum amount of time taken by the high priority thread to calculate the PI delay. Obviously, these min and max don't necessarily happen in the same iteration, resulting in a number of false failures. The correct way is to compare the time taken by low and high priority threads in each iteration and then find the maximum delay experienced by high priority thread across the iterations. This patch implements the change. Additionally, this patch removeslock_wait_datarray, which is not needed anymore as well as makes a couple of messages easier to understand.This patch changes the messages displayed by this test case slightly. They look like the following now:
Low prio thread started
High prio thread started
Busy 0 started
Busy 1 started
Busy 2 started
Busy 3 started
Time taken for high prio thread to get the lock once released by low prio thread
Min delay = 10 us
Max delay = 36 us
Average delay = 17.06 us
Standard Deviation = 8.11 us
Quantiles:
99.0% < 36
Criteria: High prio lock wait time < (Low prio lock held time + 200 us)
Result: PASSI have tested this by running 1000 iterations of
pi_perftest on a couple of machines.Quoted 8 lines
Thanks,
Sripathi.---
realtime: Fix the pass criterion of
pi_perftest case.Signed-off-by Sripathi Kodi <[email protected]>Hi Sri,
Is this based on the latest snapshot. I get the following failures:
patching file
testcases/realtime/func/pi_perf/pi_perf.cHunk #1 FAILED at 1.
Quoted 8 lines
Quoted 1 line
Can you please resend ?
Sorry Subrata, the patch seems to be fine on my machine, but is going bad in my mail client. If you don't mind, I have attached the patch to this mail for now.
Thanks,
Sripathi.
Thanks.
Regards--
Subrata
Quoted 10 lines
--
realtime: Fix the pass criterion of
pi_perftest case.Signed-off-by Sripathi Kodi <[email protected]>Acked-by Will Schmidt <[email protected]>Acked-by Gowrishankar <[email protected]><snip>
------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge
This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry App World(TM) will have the opportunity to enter the BlackBerry Developer Challenge. See full prize details at: http://p.sf.net/sfu/blackberry
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list