1us latency?

DISCUSSION pavel 4 participants 9 messages Aug 3, 2015 – Aug 6, 2015

Share & tools

pavel

author

2015-08-03 11:21 UTC #0 lore raw

# ./cyclictest -S -p fifo
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.00 0.03 0.05 1/218 4240

T: 0 ( 4174) P: 9 I:1000 C: 430806 Min: 1 Act: 1 Avg: 1 Max: 10
T: 1 ( 4175) P: 9 I:1500 C: 287204 Min: 1 Act: 1 Avg: 1 Max: 6
T: 2 ( 4176) P: 9 I:2000 C: 215403 Min: 1 Act: 2 Avg: 1 Max: 11
T: 3 ( 4177) P: 9 I:2500 C: 172322 Min: 1 Act: 1 Avg: 1 Max: 9
T: 4 ( 4178) P: 9 I:3000 C: 143602 Min: 1 Act: 1 Avg: 1 Max: 10
T: 5 ( 4179) P: 9 I:3500 C: 123087 Min: 1 Act: 1 Avg: 1 Max: 11
T: 6 ( 4180) P: 9 I:4000 C: 107701 Min: 1 Act: 1 Avg: 1 Max: 10
T: 7 ( 4181) P: 9 I:4500 C: 108232 Min: 1 Act: 2 Avg: 1 Max: 11

It possible? 1us latency? o_O

kernel- 3.18.19-rt15
cyclictest V 0.92
Intel(R) Xeon(R) CPU E3-1246 v3 @ 3.50GHz
MSI MS-7823/B85M-G43 (MS-7823), BIOS V3.14B3 06/23/2014

Clark Williams

2015-08-03 16:44 UTC #1 lore raw

Quoted 15 lines · On Mon, 3 Aug 2015 14:13:27 +0300 pavel <[email protected]> wrote:

# ./cyclictest -S -p fifo
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.00 0.03 0.05 1/218 4240
T: 0 ( 4174) P: 9 I:1000 C: 430806 Min: 1 Act: 1 Avg: 1 Max: 10
T: 1 ( 4175) P: 9 I:1500 C: 287204 Min: 1 Act: 1 Avg: 1 Max: 6
T: 2 ( 4176) P: 9 I:2000 C: 215403 Min: 1 Act: 2 Avg: 1 Max: 11
T: 3 ( 4177) P: 9 I:2500 C: 172322 Min: 1 Act: 1 Avg: 1 Max: 9
T: 4 ( 4178) P: 9 I:3000 C: 143602 Min: 1 Act: 1 Avg: 1 Max: 10
T: 5 ( 4179) P: 9 I:3500 C: 123087 Min: 1 Act: 1 Avg: 1 Max: 11
T: 6 ( 4180) P: 9 I:4000 C: 107701 Min: 1 Act: 1 Avg: 1 Max: 10
T: 7 ( 4181) P: 9 I:4500 C: 108232 Min: 1 Act: 2 Avg: 1 Max: 11

Quoted 7 lines

It possible? 1us latency? o_O
kernel- 3.18.19-rt15
cyclictest V 0.92
Intel(R) Xeon(R) CPU E3-1246 v3 @ 3.50GHz
MSI MS-7823/B85M-G43 (MS-7823), BIOS V3.14B3 06/23/2014 --

No, your latency is 11us. Max latency is what we care about, not the average.

pavel

author

2015-08-03 18:06 UTC #2 lore raw

03.08.2015 19:44, Clark Williams пишет:

Quoted 20 lines

Quoted 17 lines · On Mon, 3 Aug 2015 14:13:27 +0300 pavel <[email protected]> wrote:
# ./cyclictest -S -p fifo
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.00 0.03 0.05 1/218 4240
T: 0 ( 4174) P: 9 I:1000 C: 430806 Min: 1 Act: 1 Avg: 1 Max: 10
T: 1 ( 4175) P: 9 I:1500 C: 287204 Min: 1 Act: 1 Avg: 1 Max: 6
T: 2 ( 4176) P: 9 I:2000 C: 215403 Min: 1 Act: 2 Avg: 1 Max: 11
T: 3 ( 4177) P: 9 I:2500 C: 172322 Min: 1 Act: 1 Avg: 1 Max: 9
T: 4 ( 4178) P: 9 I:3000 C: 143602 Min: 1 Act: 1 Avg: 1 Max: 10
T: 5 ( 4179) P: 9 I:3500 C: 123087 Min: 1 Act: 1 Avg: 1 Max: 11
T: 6 ( 4180) P: 9 I:4000 C: 107701 Min: 1 Act: 1 Avg: 1 Max: 10
T: 7 ( 4181) P: 9 I:4500 C: 108232 Min: 1 Act: 2 Avg: 1 Max: 11
It possible? 1us latency? o_O

Quoted 7 lines

Quoted 5 lines
kernel- 3.18.19-rt15
cyclictest V 0.92
Intel(R) Xeon(R) CPU E3-1246 v3 @ 3.50GHz
MSI MS-7823/B85M-G43 (MS-7823), BIOS V3.14B3 06/23/2014 --
No, your latency is 11us. Max latency is what we care about, not the average.

By the way, max values appear only at the start, then they are roughly equal to the average. --
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

pavel

author

2015-08-03 18:36 UTC #3 lore raw

1. Without patch

# ./cyclictest -S -p fifo -D60s
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.00 0.01 0.05 1/207 12378

T: 0 (12361) P: 9 I:1000 C: 59993 Min: 1 Act: 1 Avg: 1 Max: 8
T: 1 (12362) P: 9 I:1500 C: 39995 Min: 1 Act: 1 Avg: 1 Max: 9
T: 2 (12363) P: 9 I:2000 C: 29996 Min: 1 Act: 1 Avg: 1 Max: 12
T: 3 (12364) P: 9 I:2500 C: 23997 Min: 1 Act: 1 Avg: 1 Max: 9
T: 4 (12365) P: 9 I:3000 C: 19997 Min: 1 Act: 1 Avg: 1 Max: 6
T: 5 (12366) P: 9 I:3500 C: 17141 Min: 1 Act: 1 Avg: 1 Max: 11
T: 6 (12367) P: 9 I:4000 C: 14998 Min: 0 Act: 1 Avg: 1 Max: 10
T: 7 (12368) P: 9 I:4500 C: 13331 Min: 0 Act: 1 Avg: 1 Max: 6

2. With patch

 # ./cyclictest -S -p fifo -D60s
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.05 0.04 0.05 1/206 12469

T: 0 (12452) P: 9 I:1000 C: 59997 Min: 1 Act: 1 Avg: 1 Max: 3
T: 1 (12453) P: 9 I:1500 C: 39998 Min: 1 Act: 1 Avg: 1 Max: 2
T: 2 (12454) P: 9 I:2000 C: 29998 Min: 1 Act: 1 Avg: 1 Max: 4
T: 3 (12455) P: 9 I:2500 C: 23999 Min: 1 Act: 1 Avg: 1 Max: 4
T: 4 (12456) P: 9 I:3000 C: 19999 Min: 1 Act: 1 Avg: 1 Max: 4
T: 5 (12457) P: 9 I:3500 C: 17142 Min: 1 Act: 1 Avg: 1 Max: 3
T: 6 (12458) P: 9 I:4000 C: 14999 Min: 1 Act: 1 Avg: 1 Max: 2
T: 7 (12459) P: 9 I:4500 C: 13332 Min: 1 Act: 1 Avg: 1 Max: 3

Patch ---

src/cyclictest/cyclictest.c +3 -0

index 34053c5..84a70de 100644
@@ -1727,6 +1727,9 @@ static void print_stat(FILE *fp, struct thread_param *par, int index, int verbos
 {
        struct thread_stat *stat = par->stats;
 
+       if ( stat->cycles < 5000)
+                stat->max = 0;
+
        if (!verbose) {
                if (quiet != 1) {
                        char *fmt;

---

03.08.2015 20:59, pavel пишет:

Quoted 17 lines

Quoted 17 lines
Quoted 17 lines
# ./cyclictest -S -p fifo
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.00 0.03 0.05 1/218 4240
T: 0 ( 4174) P: 9 I:1000 C: 430806 Min: 1 Act: 1 Avg: 1 Max: 10
T: 1 ( 4175) P: 9 I:1500 C: 287204 Min: 1 Act: 1 Avg: 1 Max: 6
T: 2 ( 4176) P: 9 I:2000 C: 215403 Min: 1 Act: 2 Avg: 1 Max: 11
T: 3 ( 4177) P: 9 I:2500 C: 172322 Min: 1 Act: 1 Avg: 1 Max: 9
T: 4 ( 4178) P: 9 I:3000 C: 143602 Min: 1 Act: 1 Avg: 1 Max: 10
T: 5 ( 4179) P: 9 I:3500 C: 123087 Min: 1 Act: 1 Avg: 1 Max: 11
T: 6 ( 4180) P: 9 I:4000 C: 107701 Min: 1 Act: 1 Avg: 1 Max: 10
T: 7 ( 4181) P: 9 I:4500 C: 108232 Min: 1 Act: 2 Avg: 1 Max: 11

Quoted 5 lines

Quoted 3 lines
Quoted 1 line
It possible? 1us latency? o_O
No, your latency is 11us. Max latency is what we care about, not the average.
By the way, max values appear only at the start, then they are roughly equal to the average.

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Clark Williams

2015-08-03 18:53 UTC #4 lore raw

Quoted 54 lines · On Mon, 3 Aug 2015 21:36:26 +0300 pavel <[email protected]> wrote:

1. Without patch
# ./cyclictest -S -p fifo -D60s
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.00 0.01 0.05 1/207 12378
T: 0 (12361) P: 9 I:1000 C: 59993 Min: 1 Act: 1 Avg: 1 Max: 8
T: 1 (12362) P: 9 I:1500 C: 39995 Min: 1 Act: 1 Avg: 1 Max: 9
T: 2 (12363) P: 9 I:2000 C: 29996 Min: 1 Act: 1 Avg: 1 Max: 12
T: 3 (12364) P: 9 I:2500 C: 23997 Min: 1 Act: 1 Avg: 1 Max: 9
T: 4 (12365) P: 9 I:3000 C: 19997 Min: 1 Act: 1 Avg: 1 Max: 6
T: 5 (12366) P: 9 I:3500 C: 17141 Min: 1 Act: 1 Avg: 1 Max: 11
T: 6 (12367) P: 9 I:4000 C: 14998 Min: 0 Act: 1 Avg: 1 Max: 10
T: 7 (12368) P: 9 I:4500 C: 13331 Min: 0 Act: 1 Avg: 1 Max: 6
2. With patch
  # ./cyclictest -S -p fifo -D60s
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.05 0.04 0.05 1/206 12469
T: 0 (12452) P: 9 I:1000 C: 59997 Min: 1 Act: 1 Avg: 1 Max: 3
T: 1 (12453) P: 9 I:1500 C: 39998 Min: 1 Act: 1 Avg: 1 Max: 2
T: 2 (12454) P: 9 I:2000 C: 29998 Min: 1 Act: 1 Avg: 1 Max: 4
T: 3 (12455) P: 9 I:2500 C: 23999 Min: 1 Act: 1 Avg: 1 Max: 4
T: 4 (12456) P: 9 I:3000 C: 19999 Min: 1 Act: 1 Avg: 1 Max: 4
T: 5 (12457) P: 9 I:3500 C: 17142 Min: 1 Act: 1 Avg: 1 Max: 3
T: 6 (12458) P: 9 I:4000 C: 14999 Min: 1 Act: 1 Avg: 1 Max: 2
T: 7 (12459) P: 9 I:4500 C: 13332 Min: 1 Act: 1 Avg: 1 Max: 3
Patch ---
src/cyclictest/cyclictest.c +0 -0
index 34053c5..84a70de 100644
@@ -1727,6 +1727,9 @@ static void print_stat(FILE *fp, struct thread_param *par, 
int index, int verbos
  {
         struct thread_stat *stat = par->stats;
src/cyclictest/cyclictest.c +3 -1
+       if ( stat->cycles < 5000)
+                stat->max = 0;
+
         if (!verbose) {
                 if (quiet != 1) {
                         char *fmt;
 
---

Quoted 24 lines

03.08.2015 20:59, pavel пишет:
Quoted 22 lines
Quoted 17 lines
Quoted 17 lines
# ./cyclictest -S -p fifo
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.00 0.03 0.05 1/218 4240
T: 0 ( 4174) P: 9 I:1000 C: 430806 Min: 1 Act: 1 Avg: 1 Max: 10
T: 1 ( 4175) P: 9 I:1500 C: 287204 Min: 1 Act: 1 Avg: 1 Max: 6
T: 2 ( 4176) P: 9 I:2000 C: 215403 Min: 1 Act: 2 Avg: 1 Max: 11
T: 3 ( 4177) P: 9 I:2500 C: 172322 Min: 1 Act: 1 Avg: 1 Max: 9
T: 4 ( 4178) P: 9 I:3000 C: 143602 Min: 1 Act: 1 Avg: 1 Max: 10
T: 5 ( 4179) P: 9 I:3500 C: 123087 Min: 1 Act: 1 Avg: 1 Max: 11
T: 6 ( 4180) P: 9 I:4000 C: 107701 Min: 1 Act: 1 Avg: 1 Max: 10
T: 7 ( 4181) P: 9 I:4500 C: 108232 Min: 1 Act: 2 Avg: 1 Max: 11
Quoted 3 lines
Quoted 1 line
It possible? 1us latency? o_O
No, your latency is 11us. Max latency is what we care about, not the average.
By the way, max values appear only at the start, then they are roughly equal to the average.

Interesting. Betting that's page faults and cache filling.

I don't think we want to arbitrarily pick some number of cycles for a "settle time" (i.e. a grace period for the application to reach steady state). Possibly we should add an option for that? Specify some number of cycles or some amount of time that where the measurement threads run before actual measurements start?

  $ cyclictest --numa -p95 -m --settle=10ms

That would say "run the measurement threads for ten milliseconds before actually starting the measurement period". That would allow them to fault in and fill cache lines before starting real work.

Anyone else have an opinion?

Clark
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

pavel

author

2015-08-03 19:03 UTC #5 lore raw

03.08.2015 21:53, Clark Williams пишет:

Quoted 86 lines

Quoted 54 lines · On Mon, 3 Aug 2015 21:36:26 +0300 pavel <[email protected]> wrote:
1. Without patch
# ./cyclictest -S -p fifo -D60s
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.00 0.01 0.05 1/207 12378
T: 0 (12361) P: 9 I:1000 C: 59993 Min: 1 Act: 1 Avg: 1 Max: 8
T: 1 (12362) P: 9 I:1500 C: 39995 Min: 1 Act: 1 Avg: 1 Max: 9
T: 2 (12363) P: 9 I:2000 C: 29996 Min: 1 Act: 1 Avg: 1 Max: 12
T: 3 (12364) P: 9 I:2500 C: 23997 Min: 1 Act: 1 Avg: 1 Max: 9
T: 4 (12365) P: 9 I:3000 C: 19997 Min: 1 Act: 1 Avg: 1 Max: 6
T: 5 (12366) P: 9 I:3500 C: 17141 Min: 1 Act: 1 Avg: 1 Max: 11
T: 6 (12367) P: 9 I:4000 C: 14998 Min: 0 Act: 1 Avg: 1 Max: 10
T: 7 (12368) P: 9 I:4500 C: 13331 Min: 0 Act: 1 Avg: 1 Max: 6
2. With patch
   # ./cyclictest -S -p fifo -D60s
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.05 0.04 0.05 1/206 12469
T: 0 (12452) P: 9 I:1000 C: 59997 Min: 1 Act: 1 Avg: 1 Max: 3
T: 1 (12453) P: 9 I:1500 C: 39998 Min: 1 Act: 1 Avg: 1 Max: 2
T: 2 (12454) P: 9 I:2000 C: 29998 Min: 1 Act: 1 Avg: 1 Max: 4
T: 3 (12455) P: 9 I:2500 C: 23999 Min: 1 Act: 1 Avg: 1 Max: 4
T: 4 (12456) P: 9 I:3000 C: 19999 Min: 1 Act: 1 Avg: 1 Max: 4
T: 5 (12457) P: 9 I:3500 C: 17142 Min: 1 Act: 1 Avg: 1 Max: 3
T: 6 (12458) P: 9 I:4000 C: 14999 Min: 1 Act: 1 Avg: 1 Max: 2
T: 7 (12459) P: 9 I:4500 C: 13332 Min: 1 Act: 1 Avg: 1 Max: 3
Patch ---
src/cyclictest/cyclictest.c +0 -0
index 34053c5..84a70de 100644
@@ -1727,6 +1727,9 @@ static void print_stat(FILE *fp, struct thread_param *par,
int index, int verbos
   {
          struct thread_stat *stat = par->stats;
src/cyclictest/cyclictest.c +3 -1
+       if ( stat->cycles < 5000)
+                stat->max = 0;
+
          if (!verbose) {
                  if (quiet != 1) {
                          char *fmt;
 
---
Quoted 21 lines
03.08.2015 20:59, pavel пишет:
Quoted 20 lines
Quoted 15 lines
Quoted 15 lines
# ./cyclictest -S -p fifo
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.00 0.03 0.05 1/218 4240
T: 0 ( 4174) P: 9 I:1000 C: 430806 Min: 1 Act: 1 Avg: 1 Max: 10
T: 1 ( 4175) P: 9 I:1500 C: 287204 Min: 1 Act: 1 Avg: 1 Max: 6
T: 2 ( 4176) P: 9 I:2000 C: 215403 Min: 1 Act: 2 Avg: 1 Max: 11
T: 3 ( 4177) P: 9 I:2500 C: 172322 Min: 1 Act: 1 Avg: 1 Max: 9
T: 4 ( 4178) P: 9 I:3000 C: 143602 Min: 1 Act: 1 Avg: 1 Max: 10
T: 5 ( 4179) P: 9 I:3500 C: 123087 Min: 1 Act: 1 Avg: 1 Max: 11
T: 6 ( 4180) P: 9 I:4000 C: 107701 Min: 1 Act: 1 Avg: 1 Max: 10
T: 7 ( 4181) P: 9 I:4500 C: 108232 Min: 1 Act: 2 Avg: 1 Max: 11
Quoted 3 lines
Quoted 1 line
It possible? 1us latency? o_O
No, your latency is 11us. Max latency is what we care about, not the average.
By the way, max values appear only at the start, then they are roughly equal to the average.
Interesting. Betting that's page faults and cache filling.
I don't think we want to arbitrarily pick some number of cycles for a "settle time" (i.e. a grace period for the application to reach steady state). Possibly we should add an option for that? Specify some number of cycles or some amount of time that where the measurement threads run before actual measurements start?

Quoted 7 lines

   $ cyclictest --numa -p95 -m --settle=10ms
That would say "run the measurement threads for ten milliseconds before actually starting the measurement period". That would allow them to fault in and fill cache lines before starting real work.
Anyone else have an opinion?

Add option and yet another utility for detect steady state, when the difference is not more than 10% (like an oldschool resistors with silver line :))

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Frank Rowand

2015-08-04 20:27 UTC #6 lore raw

In reply to Clark Williams

Quoted 75 lines · Clark Williams

Quoted 72 lines · On Mon, 3 Aug 2015 21:36:26 +0300 pavel <[email protected]> wrote:
1. Without patch
# ./cyclictest -S -p fifo -D60s
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.00 0.01 0.05 1/207 12378
T: 0 (12361) P: 9 I:1000 C: 59993 Min: 1 Act: 1 Avg: 1 Max: 8
T: 1 (12362) P: 9 I:1500 C: 39995 Min: 1 Act: 1 Avg: 1 Max: 9
T: 2 (12363) P: 9 I:2000 C: 29996 Min: 1 Act: 1 Avg: 1 Max: 12
T: 3 (12364) P: 9 I:2500 C: 23997 Min: 1 Act: 1 Avg: 1 Max: 9
T: 4 (12365) P: 9 I:3000 C: 19997 Min: 1 Act: 1 Avg: 1 Max: 6
T: 5 (12366) P: 9 I:3500 C: 17141 Min: 1 Act: 1 Avg: 1 Max: 11
T: 6 (12367) P: 9 I:4000 C: 14998 Min: 0 Act: 1 Avg: 1 Max: 10
T: 7 (12368) P: 9 I:4500 C: 13331 Min: 0 Act: 1 Avg: 1 Max: 6
2. With patch
  # ./cyclictest -S -p fifo -D60s
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.05 0.04 0.05 1/206 12469
T: 0 (12452) P: 9 I:1000 C: 59997 Min: 1 Act: 1 Avg: 1 Max: 3
T: 1 (12453) P: 9 I:1500 C: 39998 Min: 1 Act: 1 Avg: 1 Max: 2
T: 2 (12454) P: 9 I:2000 C: 29998 Min: 1 Act: 1 Avg: 1 Max: 4
T: 3 (12455) P: 9 I:2500 C: 23999 Min: 1 Act: 1 Avg: 1 Max: 4
T: 4 (12456) P: 9 I:3000 C: 19999 Min: 1 Act: 1 Avg: 1 Max: 4
T: 5 (12457) P: 9 I:3500 C: 17142 Min: 1 Act: 1 Avg: 1 Max: 3
T: 6 (12458) P: 9 I:4000 C: 14999 Min: 1 Act: 1 Avg: 1 Max: 2
T: 7 (12459) P: 9 I:4500 C: 13332 Min: 1 Act: 1 Avg: 1 Max: 3
Patch ---
src/cyclictest/cyclictest.c +0 -0
index 34053c5..84a70de 100644
@@ -1727,6 +1727,9 @@ static void print_stat(FILE *fp, struct thread_param *par, 
int index, int verbos
  {
         struct thread_stat *stat = par->stats;
src/cyclictest/cyclictest.c +3 -1
+       if ( stat->cycles < 5000)
+                stat->max = 0;
+
         if (!verbose) {
                 if (quiet != 1) {
                         char *fmt;
 
---
03.08.2015 20:59, pavel пишет:
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
# ./cyclictest -S -p fifo
defaulting realtime priority to 9
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.00 0.03 0.05 1/218 4240
T: 0 ( 4174) P: 9 I:1000 C: 430806 Min: 1 Act: 1 Avg: 1 Max: 10
T: 1 ( 4175) P: 9 I:1500 C: 287204 Min: 1 Act: 1 Avg: 1 Max: 6
T: 2 ( 4176) P: 9 I:2000 C: 215403 Min: 1 Act: 2 Avg: 1 Max: 11
T: 3 ( 4177) P: 9 I:2500 C: 172322 Min: 1 Act: 1 Avg: 1 Max: 9
T: 4 ( 4178) P: 9 I:3000 C: 143602 Min: 1 Act: 1 Avg: 1 Max: 10
T: 5 ( 4179) P: 9 I:3500 C: 123087 Min: 1 Act: 1 Avg: 1 Max: 11
T: 6 ( 4180) P: 9 I:4000 C: 107701 Min: 1 Act: 1 Avg: 1 Max: 10
T: 7 ( 4181) P: 9 I:4500 C: 108232 Min: 1 Act: 2 Avg: 1 Max: 11

Quoted 8 lines

Quoted 6 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
It possible? 1us latency? o_O
No, your latency is 11us. Max latency is what we care about, not the average.
By the way, max values appear only at the start, then they are roughly equal to the average.
Interesting. Betting that's page faults and cache filling.

You might want to try running a background load on the system that pollutes the cache and TLB and see if you get larger values after the start up period.

Quoted 13 lines

I don't think we want to arbitrarily pick some number of cycles for a "settle time" (i.e. a grace period for the application to reach steady state). Possibly we should add an option for that? Specify some number of cycles or some amount of time that where the measurement threads run before actual measurements start?
  $ cyclictest --numa -p95 -m --settle=10ms
That would say "run the measurement threads for ten milliseconds before actually starting the measurement period". That would allow them to fault in and fill cache lines before starting real work.

Quoted 9 lines

Anyone else have an opinion?
Clark
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
.

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Sebastian Andrzej Siewior

2015-08-06 13:13 UTC #7 lore raw

In reply to Clark Williams

Quoted 10 lines · * Clark Williams | 2015-08-03 13:53:26 [-0500]:

On Mon, 3 Aug 2015 21:36:26 +0300
Interesting. Betting that's page faults and cache filling.
I don't think we want to arbitrarily pick some number of cycles for a "settle time" (i.e. a grace period for the application to reach steady state). Possibly we should add an option for that? Specify some number of cycles or some amount of time that where the measurement threads run before actual measurements start?

Quoted 7 lines

 $ cyclictest --numa -p95 -m --settle=10ms
That would say "run the measurement threads for ten milliseconds before actually starting the measurement period". That would allow them to fault in and fill cache lines before starting real work.
Anyone else have an opinion?

Wouldn't you have everything in-memory after once cycle of each thread?

Quoted 1 line

Clark

Sebastian

Clark Williams

2015-08-06 15:16 UTC #8 lore raw

Quoted 14 lines · On Thu, 6 Aug 2015 15:12:59 +0200 Sebastian Andrzej Siewior…

Quoted 12 lines · * Clark Williams | 2015-08-03 13:53:26 [-0500]:
On Mon, 3 Aug 2015 21:36:26 +0300
Interesting. Betting that's page faults and cache filling.
I don't think we want to arbitrarily pick some number of cycles for a "settle time" (i.e. a grace period for the application to reach steady state). Possibly we should add an option for that? Specify some number of cycles or some amount of time that where the measurement threads run before actual measurements start?
 $ cyclictest --numa -p95 -m --settle=10ms

Quoted 7 lines

Quoted 5 lines
That would say "run the measurement threads for ten milliseconds before actually starting the measurement period". That would allow them to fault in and fill cache lines before starting real work.
Anyone else have an opinion?
Wouldn't you have everything in-memory after once cycle of each thread?

I had to go through the timerthread() routine a couple of times to convince myself, but I think you're right.

So if we wanted to discount the paging-in overhead, we could have each thread do a "dummy" pass through the timer loop (i.e. do everything but just not record the results) and then start recording measurements. I may hack together an option to try that and see what sort of results we get.

Clark