Skip to contentSkip to conversation

perfcounters: record time running and time enabled for each counter

PATCH In review Paul Mackerras 4 participants 7 messages Mar 25, 2009 – Mar 26, 2009 PowerPC

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 perfcounters: record time running and time enabled for each counter

Impact: new functionality

Currently, if there are more counters enabled than can fit on the CPU, the kernel will multiplex the counters on to the hardware using round-robin scheduling. That isn't too bad for sampling counters, but for counting counters it means that the value read from a counter represents some unknown fraction of the true count of events that occurred while the counter was enabled.

This remedies the situation by keeping track of how long each counter is enabled for, and how long it is actually on the cpu and counting events. These times are recorded in nanoseconds using the task clock for per-task counters and the cpu clock for per-cpu counters.

These values can be supplied to userspace on a read from the counter. Userspace requests that they be supplied after the counter value by setting the PERF_FORMAT_TOTAL_TIME_ENABLED and/or PERF_FORMAT_TOTAL_TIME_RUNNING bits in the hw_event.read_format field when creating the counter. (There is no way to change the read format after the counter is created, though it would be possible to add some way to do that.)

Using this information it is possible for userspace to scale the count it reads from the counter to get an estimate of the true count:

true_count_estimate = count * total_time_enabled / total_time_running

This also lets userspace detect the situation where the counter never got to go on the cpu: total_time_running == 0.

This functionality has been requested by the PAPI developers, and will be generally needed for interpreting the count values from counting counters correctly.

In the implementation, this keeps 5 time values (in nanoseconds) for each counter: total_time_enabled and total_time_running are used when the counter is in state OFF or ERROR and for reporting back to userspace. When the counter is in state INACTIVE or ACTIVE, it is the tstamp_enabled, tstamp_running and tstamp_stopped values that are relevant, and total_time_enabled and total_time_running are determined from them. (tstamp_stopped is only used in INACTIVE state.) The reason for doing it like this is that it means that only counters being enabled or disabled at sched-in and sched-out time need to be updated. There are no new loops that iterate over all counters to update total_time_enabled or total_time_running.

This also keeps separate child_total_time_running and child_total_time_enabled fields that get added in when reporting the totals to userspace. They are separate fields so that they can be atomic. We don't want to use atomics for total_time_running, total_time_enabled etc., because then we would have to use atomic sequences to update them, which are slower than regular arithmetic and memory accesses.

It is possible to measure total_time_running by adding a task_clock counter to each group of counters, and total_time_enabled can be measured approximately with a top-level task_clock counter (though inaccuracies will creep in if you need to disable and enable groups since it is not possible in general to disable/enable the top-level task_clock counter simultaneously with another group). However, that adds extra overhead - I measured around 15% increase in the context switch latency reported by lat_ctx (from lmbench) when a task_clock counter was added to each of 2 groups, and around 25% increase when a task_clock counter was added to each of 4 groups. (In both cases a top-level task-clock counter was also added.)

In contrast, the code added in this commit gives better information with no overhead that I could measure (in fact in some cases I measured lower times with this code, but the differences were all less than one standard deviation).

---
Hopefully it's now clear from the comments and the commit message that all the times are in units of approximately 1/pi attocenturies. 8-)

 arch/powerpc/kernel/perf_counter.c |    2 +
 include/linux/perf_counter.h       |   53 ++++++++++++
 kernel/perf_counter.c              |  157 +++++++++++++++++++++++++++++++-----
 3 files changed, 191 insertions(+), 21 deletions(-)
arch/powerpc/kernel/perf_counter.c +2 -0
index d48596a..df007fe 100644
@@ -455,6 +455,8 @@ static void counter_sched_in(struct perf_counter *counter, int cpu)
 {
 	counter->state = PERF_COUNTER_STATE_ACTIVE;
 	counter->oncpu = cpu;
+	counter->tstamp_running += counter->ctx->time_now -
+		counter->tstamp_stopped;
 	if (is_software_counter(counter))
 		counter->hw_ops->enable(counter);
 }
include/linux/perf_counter.h +53 -0

77 lines not on this page · raw message

kernel/perf_counter.c +136 -21

330 lines not on this page · raw message

Signature · 1 line
1.5.6.3
Acked-by on v3 1/1
#1 lore raw
Quoted 66 lines · Paul Mackerras

Impact: new functionality

Currently, if there are more counters enabled than can fit on the CPU, the kernel will multiplex the counters on to the hardware using round-robin scheduling. That isn't too bad for sampling counters, but for counting counters it means that the value read from a counter represents some unknown fraction of the true count of events that occurred while the counter was enabled.

This remedies the situation by keeping track of how long each counter is enabled for, and how long it is actually on the cpu and counting events. These times are recorded in nanoseconds using the task clock for per-task counters and the cpu clock for per-cpu counters.

These values can be supplied to userspace on a read from the counter. Userspace requests that they be supplied after the counter value by setting the PERF_FORMAT_TOTAL_TIME_ENABLED and/or PERF_FORMAT_TOTAL_TIME_RUNNING bits in the hw_event.read_format field when creating the counter. (There is no way to change the read format after the counter is created, though it would be possible to add some way to do that.)

Using this information it is possible for userspace to scale the count it reads from the counter to get an estimate of the true count:

true_count_estimate = count * total_time_enabled / total_time_running

This also lets userspace detect the situation where the counter never got to go on the cpu: total_time_running == 0.

This functionality has been requested by the PAPI developers, and will be generally needed for interpreting the count values from counting counters correctly.

In the implementation, this keeps 5 time values (in nanoseconds) for each counter: total_time_enabled and total_time_running are used when the counter is in state OFF or ERROR and for reporting back to userspace. When the counter is in state INACTIVE or ACTIVE, it is the tstamp_enabled, tstamp_running and tstamp_stopped values that are relevant, and total_time_enabled and total_time_running are determined from them. (tstamp_stopped is only used in INACTIVE state.) The reason for doing it like this is that it means that only counters being enabled or disabled at sched-in and sched-out time need to be updated. There are no new loops that iterate over all counters to update total_time_enabled or total_time_running.

This also keeps separate child_total_time_running and child_total_time_enabled fields that get added in when reporting the totals to userspace. They are separate fields so that they can be atomic. We don't want to use atomics for total_time_running, total_time_enabled etc., because then we would have to use atomic sequences to update them, which are slower than regular arithmetic and memory accesses.

It is possible to measure total_time_running by adding a task_clock counter to each group of counters, and total_time_enabled can be measured approximately with a top-level task_clock counter (though inaccuracies will creep in if you need to disable and enable groups since it is not possible in general to disable/enable the top-level task_clock counter simultaneously with another group). However, that adds extra overhead - I measured around 15% increase in the context switch latency reported by lat_ctx (from lmbench) when a task_clock counter was added to each of 2 groups, and around 25% increase when a task_clock counter was added to each of 4 groups. (In both cases a top-level task-clock counter was also added.)

Quoted 6 lines

In contrast, the code added in this commit gives better information with no overhead that I could measure (in fact in some cases I measured lower times with this code, but the differences were all less than one standard deviation).

Looks good,

Paul, should we perhaps also put a format header in the sys_read() output?

#2 lore raw
Quoted 72 lines · Peter Zijlstra
Quoted 71 lines · Paul Mackerras

Impact: new functionality

Currently, if there are more counters enabled than can fit on the CPU, the kernel will multiplex the counters on to the hardware using round-robin scheduling. That isn't too bad for sampling counters, but for counting counters it means that the value read from a counter represents some unknown fraction of the true count of events that occurred while the counter was enabled.

This remedies the situation by keeping track of how long each counter is enabled for, and how long it is actually on the cpu and counting events. These times are recorded in nanoseconds using the task clock for per-task counters and the cpu clock for per-cpu counters.

These values can be supplied to userspace on a read from the counter. Userspace requests that they be supplied after the counter value by setting the PERF_FORMAT_TOTAL_TIME_ENABLED and/or PERF_FORMAT_TOTAL_TIME_RUNNING bits in the hw_event.read_format field when creating the counter. (There is no way to change the read format after the counter is created, though it would be possible to add some way to do that.)

Using this information it is possible for userspace to scale the count it reads from the counter to get an estimate of the true count:

true_count_estimate = count * total_time_enabled / total_time_running

This also lets userspace detect the situation where the counter never got to go on the cpu: total_time_running == 0.

This functionality has been requested by the PAPI developers, and will be generally needed for interpreting the count values from counting counters correctly.

In the implementation, this keeps 5 time values (in nanoseconds) for each counter: total_time_enabled and total_time_running are used when the counter is in state OFF or ERROR and for reporting back to userspace. When the counter is in state INACTIVE or ACTIVE, it is the tstamp_enabled, tstamp_running and tstamp_stopped values that are relevant, and total_time_enabled and total_time_running are determined from them. (tstamp_stopped is only used in INACTIVE state.) The reason for doing it like this is that it means that only counters being enabled or disabled at sched-in and sched-out time need to be updated. There are no new loops that iterate over all counters to update total_time_enabled or total_time_running.

This also keeps separate child_total_time_running and child_total_time_enabled fields that get added in when reporting the totals to userspace. They are separate fields so that they can be atomic. We don't want to use atomics for total_time_running, total_time_enabled etc., because then we would have to use atomic sequences to update them, which are slower than regular arithmetic and memory accesses.

It is possible to measure total_time_running by adding a task_clock counter to each group of counters, and total_time_enabled can be measured approximately with a top-level task_clock counter (though inaccuracies will creep in if you need to disable and enable groups since it is not possible in general to disable/enable the top-level task_clock counter simultaneously with another group). However, that adds extra overhead - I measured around 15% increase in the context switch latency reported by lat_ctx (from lmbench) when a task_clock counter was added to each of 2 groups, and around 25% increase when a task_clock counter was added to each of 4 groups. (In both cases a top-level task-clock counter was also added.)

In contrast, the code added in this commit gives better information with no overhead that I could measure (in fact in some cases I measured lower times with this code, but the differences were all less than one standard deviation).

Quoted 5 lines
Quoted 1 line

Looks good,

thanks - it even applied cleanly on top of your mmap changed :-)

Quoted 2 lines

Paul, should we perhaps also put a format header in the sys_read() output?

Yeah - we should standardize on a common perf_event_type binary record structure for all transports.

Ingo

#3 lore raw
In reply to Peter Zijlstra
Quoted 2 lines · Peter Zijlstra

Paul, should we perhaps also put a format header in the sys_read() output?

Hmmm..., possibly. It could be useful to have a PERF_FORMAT_GROUP, and then we'd want headers, I guess.

However, to simplify life for userland, we should make it possible to predict the layout for the simple cases (the cases where they have only asked for fixed-length items). For example, if the user only wants the counter value, they should be able to do the read() and know that the counter value is at some fixed offset in the buffer they just read, provided they got the number of bytes they expected. It would be unfortunate if we have to have code to step through a series of tagged records to find the counter value, and additionally have code to cope with not finding the tag we're expecting.

Paul.

#4 lore raw

[tip:perfcounters/core] perf_counter: record time running and time enabled for each counter

In reply to Paul Mackerras
Commit-ID:  03ced439c94d692363307eeb3abe0bbfa2d08a01
Gitweb:     http://git.kernel.org/tip/03ced439c94d692363307eeb3abe0bbfa2d08a01
Author:     Paul Mackerras <[email protected]>
AuthorDate: Wed, 25 Mar 2009 22:46:58 +1100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Wed, 25 Mar 2009 13:24:00 +0100

perf_counter: record time running and time enabled for each counter

Impact: new functionality

Currently, if there are more counters enabled than can fit on the CPU, the kernel will multiplex the counters on to the hardware using round-robin scheduling. That isn't too bad for sampling counters, but for counting counters it means that the value read from a counter represents some unknown fraction of the true count of events that occurred while the counter was enabled.

This remedies the situation by keeping track of how long each counter is enabled for, and how long it is actually on the cpu and counting events. These times are recorded in nanoseconds using the task clock for per-task counters and the cpu clock for per-cpu counters.

These values can be supplied to userspace on a read from the counter. Userspace requests that they be supplied after the counter value by setting the PERF_FORMAT_TOTAL_TIME_ENABLED and/or PERF_FORMAT_TOTAL_TIME_RUNNING bits in the hw_event.read_format field when creating the counter. (There is no way to change the read format after the counter is created, though it would be possible to add some way to do that.)

Using this information it is possible for userspace to scale the count it reads from the counter to get an estimate of the true count:

true_count_estimate = count * total_time_enabled / total_time_running

This also lets userspace detect the situation where the counter never got to go on the cpu: total_time_running == 0.

This functionality has been requested by the PAPI developers, and will be generally needed for interpreting the count values from counting counters correctly.

In the implementation, this keeps 5 time values (in nanoseconds) for each counter: total_time_enabled and total_time_running are used when the counter is in state OFF or ERROR and for reporting back to userspace. When the counter is in state INACTIVE or ACTIVE, it is the tstamp_enabled, tstamp_running and tstamp_stopped values that are relevant, and total_time_enabled and total_time_running are determined from them. (tstamp_stopped is only used in INACTIVE state.) The reason for doing it like this is that it means that only counters being enabled or disabled at sched-in and sched-out time need to be updated. There are no new loops that iterate over all counters to update total_time_enabled or total_time_running.

This also keeps separate child_total_time_running and child_total_time_enabled fields that get added in when reporting the totals to userspace. They are separate fields so that they can be atomic. We don't want to use atomics for total_time_running, total_time_enabled etc., because then we would have to use atomic sequences to update them, which are slower than regular arithmetic and memory accesses.

It is possible to measure total_time_running by adding a task_clock counter to each group of counters, and total_time_enabled can be measured approximately with a top-level task_clock counter (though inaccuracies will creep in if you need to disable and enable groups since it is not possible in general to disable/enable the top-level task_clock counter simultaneously with another group). However, that adds extra overhead - I measured around 15% increase in the context switch latency reported by lat_ctx (from lmbench) when a task_clock counter was added to each of 2 groups, and around 25% increase when a task_clock counter was added to each of 4 groups. (In both cases a top-level task-clock counter was also added.)

In contrast, the code added in this commit gives better information with no overhead that I could measure (in fact in some cases I measured lower times with this code, but the differences were all less than one standard deviation).

[ v2: address review comments by Andrew Morton. ]

LKML-Reference: <[email protected]>

---
 arch/powerpc/kernel/perf_counter.c |    2 +
 include/linux/perf_counter.h       |   53 ++++++++++++
 kernel/perf_counter.c              |  157 +++++++++++++++++++++++++++++++-----
 3 files changed, 191 insertions(+), 21 deletions(-)
arch/powerpc/kernel/perf_counter.c +2 -0
index d48596a..df007fe 100644
@@ -455,6 +455,8 @@ static void counter_sched_in(struct perf_counter *counter, int cpu)
 {
 	counter->state = PERF_COUNTER_STATE_ACTIVE;
 	counter->oncpu = cpu;
+	counter->tstamp_running += counter->ctx->time_now -
+		counter->tstamp_stopped;
 	if (is_software_counter(counter))
 		counter->hw_ops->enable(counter);
 }
include/linux/perf_counter.h +53 -0

77 lines not on this page · raw message

kernel/perf_counter.c +136 -21

330 lines not on this page · raw message

#5 lore raw

Re: [tip:perfcounters/core] perf_counter: record time running and time enabled for each counter

Quoted 8 lines · Paul Mackerras
Commit-ID:  03ced439c94d692363307eeb3abe0bbfa2d08a01
Gitweb:     http://git.kernel.org/tip/03ced439c94d692363307eeb3abe0bbfa2d08a01
Author:     Paul Mackerras <[email protected]>
AuthorDate: Wed, 25 Mar 2009 22:46:58 +1100
Committer:  Ingo Molnar <[email protected]>
CommitDate: Wed, 25 Mar 2009 13:24:00 +0100

perf_counter: record time running and time enabled for each counter

[snip]

Quoted 7 lines

These values can be supplied to userspace on a read from the counter. Userspace requests that they be supplied after the counter value by setting the PERF_FORMAT_TOTAL_TIME_ENABLED and/or PERF_FORMAT_TOTAL_TIME_RUNNING bits in the hw_event.read_format field when creating the counter. (There is no way to change the read format after the counter is created, though it would be possible to add some way to do that.)

This change looks good to me, except:

How do I know which value comes first after the counter value if I have both PERF_FORMAT_TOTAL_TIME_ENABLED and PERF_FORMAT_TOTAL_TIME_RUNNING set? Are they in the order of their enum values? The order should be described somewhere. Is there anything to keep a kernel code change from inadvertantly reversing the position of two values at some point in the future?

Regards,

- Corey

Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]

#6 lore raw

Re: [tip:perfcounters/core] perf_counter: record time running and time enabled for each counter

Quoted 5 lines · Corey Ashford

This change looks good to me, except:

How do I know which value comes first after the counter value if I have both PERF_FORMAT_TOTAL_TIME_ENABLED and PERF_FORMAT_TOTAL_TIME_RUNNING set? Are they in the order of their enum values?

Yes.

Quoted 2 lines

The order should be
described somewhere.

This comment that I added to perf_counter.h:

/*
* Bits that can be set in hw_event.read_format to request that
* reads on the counter should return the indicated quantities,
* in increasing order of bit value, after the counter value. */

describes the order ("in increasing order of bit value"). So you will get the count, total time enabled, total time running, in that order.

Quoted 3 lines

Is there anything to keep a kernel code change
from inadvertantly reversing the position of two values at some point in the future?

Well, that would be an ABI breakage. Once it's upstream, we won't break the ABI, and if someone does break it and no-one realizes at the time, we'll revert their change once we do realize. (With a possible exception for security holes that can't be fixed without breaking the ABI, of course, but I can't see how that would be the case here.)

Until it goes into Linus' tree, we can still change the ABI in incompatible ways. Ingo and Peter are talking about using a tagged format for the data obtained from read(), which would solve your problem in another way, though it would make the userspace code more complicated (it would have to check that the tags were what it was expecting, and cope with the situation where it doesn't get a value it needs).

Paul.