Skip to contentSkip to conversation

perfcounters: record time running and time enabled for each counter

PATCH Superseded Paul Mackerras 5 participants 11 messages Mar 21, 2009 – Mar 23, 2009

Subsystems (12)
Documentation x86 VFS PowerPC +8 more subsystems

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 perfcounters: record time running and time enabled for each counter

Impact: new functionality

Currently, if there are more counters enabled than can fit on the CPU, the kernel will multiplex the counters on to the hardware using round-robin scheduling. That isn't too bad for sampling counters, but for counting counters it means that the value read from a counter represents some unknown fraction of the true count of events that occurred while the counter was enabled.

This remedies the situation by keeping track of how long each counter is enabled for, and how long it is actually on the cpu and counting events. These times are recorded in nanoseconds using the task clock for per-task counters and the cpu clock for per-cpu counters.

These values can be supplied to userspace on a read from the counter. Userspace requests that they be supplied after the counter value by setting the PERF_FORMAT_TIME_ENABLED and/or PERF_FORMAT_TIME_RUNNING bits in the hw_event.read_format field when creating the counter. (There is no way to change the read format after the counter is created, though it would be possible to add some way to do that.)

Using this information it is possible for userspace to scale the count it reads from the counter to get an estimate of the true count:

true_count_estimate = count * time_enabled / time_running

This also lets userspace detect the situation where the counter never got to go on the cpu: time_running == 0.

This functionality has been requested by the PAPI developers, and will be generally needed for interpreting the count values from counting counters correctly.

In the implementation, this keeps 5 time values for each counter:
time_enabled and time_running are used when the counter is in state OFF or ERROR and for reporting back to userspace. When the counter is in state INACTIVE or ACTIVE, it is the start_enabled, start_running and last_stopped values that are relevant, and time_enabled and time_running are determined from them. (last_stopped is only used in INACTIVE state.) The reason for doing it like this is that it means that only counters being enabled or disabled at sched-in and sched-out time need to be updated. There are no new loops that iterate over all counters to update time_enabled or time_running.

This also keeps separate child_time_running and child_time_enabled fields that get added in when reporting the totals to userspace. They are separate fields so that they can be atomic. We don't want to use atomics for time_running, time_enabled etc., because then we would have to use atomic sequences to update them, which are slower than regular arithmetic and memory accesses.

It is possible to measure time_running by adding a task_clock counter to each group of counters, and time_enabled can be measured approximately with a top-level task_clock counter (though inaccuracies will creep in if you need to disable and enable groups since it is not possible in general to disable/enable the top-level task_clock counter simultaneously with another group). However, that adds extra overhead - I measured around 15% increase in the context switch latency reported by lat_ctx (from lmbench) when a task_clock counter was added to each of 2 groups, and around 25% increase when a task_clock counter was added to each of 4 groups. In contrast, the code added in this commit gives better information with no overhead that I could measure (in fact in some cases I measured lower times with this code, but the differences were all small).

---
This is in the rfc branch of my perfcounters.git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/paulus/perfcounters.git rfc

 arch/powerpc/kernel/perf_counter.c |    2 +
 include/linux/perf_counter.h       |   20 +++++
 kernel/perf_counter.c              |  158 +++++++++++++++++++++++++++++++-----
 3 files changed, 159 insertions(+), 21 deletions(-)
arch/powerpc/kernel/perf_counter.c +2 -0
index 6413d9c..0d59c8a 100644
@@ -454,6 +454,8 @@ static void counter_sched_in(struct perf_counter *counter, int cpu)
 {
 	counter->state = PERF_COUNTER_STATE_ACTIVE;
 	counter->oncpu = cpu;
+	counter->start_running += counter->ctx->time_now -
+		counter->last_stopped;
 	if (is_software_counter(counter))
 		counter->hw_ops->enable(counter);
 }
include/linux/perf_counter.h +20 -0
index 98f5990..b1224f9 100644
@@ -83,6 +83,16 @@ enum perf_counter_record_type {
 };
 
 /*
+ * Bits that can be set in hw_event.read_format to request that
+ * reads on the counter should return the indicated quantities,
+ * in increasing order of bit value, after the counter value.
+ */
+enum perf_counter_read_format {
+	PERF_FORMAT_TIME_ENABLED	=  1,
+	PERF_FORMAT_TIME_RUNNING	=  2,
+};
+
+/*
  * Hardware event to monitor via a performance monitoring counter:
  */
 struct perf_counter_hw_event {
@@ -234,6 +244,12 @@ struct perf_counter {
 	enum perf_counter_active_state	prev_state;
 	atomic64_t			count;
 
+	u64				time_enabled;
+	u64				time_running;
+	u64				start_enabled;
+	u64				start_running;
+	u64				last_stopped;
+
 	struct perf_counter_hw_event	hw_event;
 	struct hw_perf_counter		hw;
 
@@ -243,6 +259,8 @@ struct perf_counter {
 
 	struct perf_counter		*parent;
 	struct list_head		child_list;
+	atomic64_t			child_time_enabled;
+	atomic64_t			child_time_running;
 
 	/*
 	 * Protect attach/detach and child_list:
@@ -290,6 +308,8 @@ struct perf_counter_context {
 	int			nr_active;
 	int			is_active;
 	struct task_struct	*task;
+	u64			time_now;
+	u64			time_lost;
 #endif
 };
 
kernel/perf_counter.c +137 -21

331 lines not on this page · raw message

Signature · 1 line
1.5.6.3
maintainer
#1 lore raw
Quoted 1 line · On Sat, 21 Mar 2009 23:04:16 +1100 Paul Mackerras <[email protected]> wrote:

{innocent civilian mode}

Quoted 22 lines
include/linux/perf_counter.h +10 -0
index 98f5990..b1224f9 100644
@@ -83,6 +83,16 @@ enum perf_counter_record_type {
 };
 
 /*
+ * Bits that can be set in hw_event.read_format to request that
+ * reads on the counter should return the indicated quantities,
+ * in increasing order of bit value, after the counter value.
+ */
+enum perf_counter_read_format {
+	PERF_FORMAT_TIME_ENABLED	=  1,
+	PERF_FORMAT_TIME_RUNNING	=  2,
+};
+
+/*
  * Hardware event to monitor via a performance monitoring counter:
  */
 struct perf_counter_hw_event {

include/linux/perf_counter.h struct perf_counter { · line 248

Quoted 5 lines
include/linux/perf_counter.h +2 -0
@@ -234,6 +244,12 @@ struct perf_counter {
 	enum perf_counter_active_state	prev_state;
 	atomic64_t			count;
 
+	u64				time_enabled;
+	u64				time_running;

These look like times. I see no indication (here) as to the units.

include/linux/perf_counter.h struct perf_counter { · line 249

Quoted 1 line
include/linux/perf_counter.h +1 -0
+	u64				start_enabled;

This looks like a boolean, but it's u64.

include/linux/perf_counter.h struct perf_counter { · line 250

Quoted 1 line
include/linux/perf_counter.h +1 -0
+	u64				start_running;

hard to say.

include/linux/perf_counter.h struct perf_counter { · line 251

Quoted 1 line
include/linux/perf_counter.h +1 -0
+	u64				last_stopped;

probably a time, unknown units.

Perhaps one of the reasons why this code is confusing is the blurring between the "time" at which an event occured and the "time" between the occurrence of two events. A weakness in English, I guess. Using the term "interval" in the latter case will help a lot.

include/linux/perf_counter.h struct perf_counter { · line 263

Quoted 9 lines
include/linux/perf_counter.h +2 -0
 	struct perf_counter_hw_event	hw_event;
 	struct hw_perf_counter		hw;
 
@@ -243,6 +259,8 @@ struct perf_counter {
 
 	struct perf_counter		*parent;
 	struct list_head		child_list;
+	atomic64_t			child_time_enabled;
+	atomic64_t			child_time_running;

These read like booleans, but why are they atomic64_t's?

include/linux/perf_counter.h struct perf_counter_context { · line 314

Quoted 10 lines
include/linux/perf_counter.h +2 -0
 	/*
 	 * Protect attach/detach and child_list:
@@ -290,6 +308,8 @@ struct perf_counter_context {
 	int			nr_active;
 	int			is_active;
 	struct task_struct	*task;
+	u64			time_now;
+	u64			time_lost;
 #endif
 };

I don't have a copy of this header file handy, but from the snippet I see here, it doesn't look as though it is as clear and as understadable as we can possibly make it?

Painstaking documentation of the data structures is really really valuable.

Quoted 37 lines
kernel/perf_counter.c +22 -0
index f054b8c..cabc820 100644
@@ -109,6 +109,7 @@ counter_sched_out(struct perf_counter *counter,
 		return;
 
 	counter->state = PERF_COUNTER_STATE_INACTIVE;
+	counter->last_stopped = ctx->time_now;
 	counter->hw_ops->disable(counter);
 	counter->oncpu = -1;
 
@@ -245,6 +246,59 @@ retry:
 }
 
 /*
+ * Get the current time for this context.
+ * If this is a task context, we use the task's task clock,
+ * or for a per-cpu context, we use the cpu clock.
+ */
+static u64 get_context_time(struct perf_counter_context *ctx, int update)
+{
+	struct task_struct *curr = ctx->task;
+
+	if (!curr)
+		return cpu_clock(smp_processor_id());
+
+	return __task_delta_exec(curr, update) + curr->se.sum_exec_runtime;
+}
+
+/*
+ * Update the record of the current time in a context.
+ */
+static void update_context_time(struct perf_counter_context *ctx, int update)
+{
+	ctx->time_now = get_context_time(ctx, update) - ctx->time_lost;
+}

kernel/perf_counter.c retry: · line 279

Quoted 10 lines
kernel/perf_counter.c +10 -0
+
+/*
+ * Update the time_enabled and time_running fields for a counter.
+ */
+static void update_counter_times(struct perf_counter *counter)
+{
+	struct perf_counter_context *ctx = counter->ctx;
+	u64 run_end;
+
+	if (counter->state >= PERF_COUNTER_STATE_INACTIVE) {

This is a plain old state machine?

Placing significance in this manner on the ordinal value of particular states is unusual and unexpected. Also a bit fragile, as people would _expect_ to be able to insert new states in any old place.

Hopefully the comments at the definition site clear all this up ;)

Quoted 10 lines
kernel/perf_counter.c +10 -0
+		counter->time_enabled = ctx->time_now - counter->start_enabled;
+		if (counter->state == PERF_COUNTER_STATE_INACTIVE)
+			run_end = counter->last_stopped;
+		else
+			run_end = ctx->time_now;
+		counter->time_running = run_end - counter->start_running;
+	}
+}
+
+/*

kernel/perf_counter.c retry: · line 299

Quoted 10 lines
kernel/perf_counter.c +10 -0
+ * Update time_enabled and time_running for all counters in a group.
+ */
+static void update_group_times(struct perf_counter *leader)
+{
+	struct perf_counter *counter;
+
+	update_counter_times(leader);
+	list_for_each_entry(counter, &leader->sibling_list, list_entry)
+		update_counter_times(counter);
+}

The locking for the list walk is? It _looks_ like spin_lock_irq(ctx->lock), but I wasn't able to verify all callsites.

Quoted 15 lines
kernel/perf_counter.c +7 -1
 	/*
 	 * Return end-of-file for a read on a counter that is in
@@ -1202,10 +1296,27 @@ perf_read_hw(struct perf_counter *counter, char __user *buf, size_t count)
 		return 0;
 
 	mutex_lock(&counter->mutex);
-	cntval = perf_counter_read(counter);
+	values[0] = perf_counter_read(counter);
+	n = 1;
+	if (counter->hw_event.read_format & PERF_FORMAT_TIME_ENABLED)
+		values[n++] = counter->time_enabled +
+			atomic64_read(&counter->child_time_enabled);
+	if (counter->hw_event.read_format & PERF_FORMAT_TIME_RUNNING)
+		values[n++] = counter->time_running +

kernel/perf_counter.c perf_read_hw(struct perf_counter *counter, char __user *buf, size_t count) · line 1314

Quoted 10 lines
kernel/perf_counter.c +7 -1
+			atomic64_read(&counter->child_time_running);
 	mutex_unlock(&counter->mutex);
 
-	return put_user(cntval, (u64 __user *) buf) ? -EFAULT : sizeof(cntval);
+	if (count != n * sizeof(u64))
+		return -EINVAL;
+
+	if (!access_ok(VERIFY_WRITE, buf, count))
+		return -EFAULT;
+	

<panics>

Oh.

It would be a lot more reassuring to verify uptr', rather than buf' here.

The patch adds new trailing whitespace. checkpatch helps.

kernel/perf_counter.c perf_read_hw(struct perf_counter *counter, char __user *buf, size_t count) · line 1317

Quoted 3 lines
kernel/perf_counter.c +3 -0
+	for (i = 0; i < n; ++i)
+		if (__put_user(values[i], uptr + i))
+			return -EFAULT;

And here we iterate across n', whereas we verified count'.

Can this be cleaned up a bit? Bear in mind that any maintenance errors which result from this coding will cause security holes.

Quoted 2 lines
kernel/perf_counter.c +1 -0
+	return count;
 }
#2 lore raw
Quoted 18 lines · Andrew Morton

include/linux/perf_counter.h struct perf_counter { · line 248

Quoted 2 lines
include/linux/perf_counter.h +2 -0
+	u64				time_enabled;
+	u64				time_running;

These look like times. I see no indication (here) as to the units.

include/linux/perf_counter.h struct perf_counter { · line 249

Quoted 1 line
include/linux/perf_counter.h +1 -0
+	u64				start_enabled;

This looks like a boolean, but it's u64.

include/linux/perf_counter.h struct perf_counter { · line 250

Quoted 1 line
include/linux/perf_counter.h +1 -0
+	u64				start_running;

hard to say.

include/linux/perf_counter.h struct perf_counter { · line 251

Quoted 1 line
include/linux/perf_counter.h +1 -0
+	u64				last_stopped;

probably a time, unknown units.

Quoted 5 lines

Perhaps one of the reasons why this code is confusing is the blurring between the "time" at which an event occured and the "time" between the occurrence of two events. A weakness in English, I guess. Using the term "interval" in the latter case will help a lot.

What we use in the scheduler is "sum_time" or "runtime". A bit of a tongue twister but unambiguous.

Ingo

#3 lore raw
In reply to Andrew Morton
Quoted 8 lines · Andrew Morton

include/linux/perf_counter.h struct perf_counter { · line 263

Quoted 6 lines
include/linux/perf_counter.h +2 -0
@@ -243,6 +259,8 @@ struct perf_counter {
 
 	struct perf_counter		*parent;
 	struct list_head		child_list;
+	atomic64_t			child_time_enabled;
+	atomic64_t			child_time_running;

These read like booleans, but why are they atomic64_t's?

for inherited counters these values get collected back into a 'parent counter' in a lockless way - hence to not lose statistics on SMP they need to be atomic64_t.

Quoted 4 lines
Quoted 4 lines
include/linux/perf_counter.h +0 -0
@@ -290,6 +308,8 @@ struct perf_counter_context {
 	int			nr_active;
 	int			is_active;
 	struct task_struct	*task;
Quoted 8 lines

include/linux/perf_counter.h struct perf_counter_context { · line 314

Quoted 4 lines
include/linux/perf_counter.h +2 -0
+	u64			time_now;
+	u64			time_lost;
 #endif
 };

I don't have a copy of this header file handy, but from the snippet I see here, it doesn't look as though it is as clear and as understadable as we can possibly make it?

i'll post the latest perfcounters tree - it's time for that anyway.

Ingo

#4 lore raw

[tree] Performance Counters for Linux, v7

The latest perfcounters/core git tree can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git perfcounters/core

There have been lots of updates since the last release:

- enhanced PowerPC support
- support for software event profiling
- enhanced x86 support (support for AMD CPUs)
- lots of finetuning to the ABI details to make it PAPI capable

There's also been lots of updates to the userspace tools (kerneltop and perfstat):

http://redhat.com/~mingo/perfcounters/kerneltop.c
http://redhat.com/~mingo/perfcounters/perfstat.c

Ingo

------------------>
Ingo Molnar (52):
performance counters: documentation
performance counters: x86 support
x86, perfcounters: read out MSR_CORE_PERF_GLOBAL_STATUS with counters disabled
perfcounters: select ANON_INODES
perfcounters, x86: simplify disable/enable of counters perfcounters, x86: clean up debug code
perfcounters: consolidate global-disable codepaths
perf counters: restructure the API
perf counters: add support for group counters
perf counters: group counter, fixes
perf counters: hw driver API
perf counters: implement PERF_COUNT_CPU_CLOCK
perf counters: consolidate hw_perf save/restore APIs
perf counters: implement PERF_COUNT_TASK_CLOCK
perf counters: add prctl interface to disable/enable counters perf counters: clean up state transitions
perf counters: update docs
x86: implement atomic64_t on 32-bit
perfcounters: restructure x86 counter math
perfcounters: implement "counter inheritance"
perfcounters: fix task clock counter
perfcounters: add context switch counter
perfcounters: add task migrations counter
perfcounters: add nr-of-faults counter
perfcounters: fix non-intel-perfmon CPUs
perfcounters, x86: fix sw counters on non-PMC CPUs
perfcounters: fix lapic initialization
perfcounters: release CPU context when exiting task counters
perfcounters: flush on setuid exec
perfcounters: use hw_event.disable flag
perfcounters: remove warnings
perfcounters: tweak group scheduling
x86, perfcounters: rename intel_arch_perfmon.h => perf_counter.h x86, perfcounters: prepare for fixed-mode PMCs
perfcounters: add fixed-mode PMC enumeration
x86, perfcounters: refactor code for fixed-function PMCs
perfcounters: hw ops rename
perfcounters: fix task clock counter
perfcounters: pull inherited counters
perfcounters: fix init context lock
perfcounters: enable lowlevel pmc code to schedule counters
x86, perfcounters: print out the ->used bitmask
perfcounters: remove ->nr_inherited
perfcounters: generalize the counter scheduler
perfcounters: add PERF_COUNT_BUS_CYCLES
x86, perfcounters: add support for fixed-function pmcs
perfcounters: include asm/perf_counter.h only if CONFIG_PERF_COUNTERS=y
perfcounters: fix "perf counters kills oprofile" bug, v2
perfcounters: remove duplicate definition of LOCAL_PERF_VECTOR
perfcounters: fix acpi_idle_do_entry() workaround
perfcounters: fix reserved bits sizing
perfcounters: fix crash on perfmon v1 systems

Jaswinder Singh (1):
x86: perf_counter.c intel_perfmon_event_map and max_intel_perfmon_events should be static

Jaswinder Singh Rajput (7):
x86: perf_counter remove unwanted hw_perf_enable_all
x86: irqinit_32.c fix compilation warning
x86: prepare perf_counter to add more cpus
x86: AMD Support for perf_counter
x86: decent declarations in perf_counter.c
x86: use pr_info in perf_counter.c
x86: perf_counter cleanup

Mike Galbraith (7):
perfcounters: throttle on too high IRQ rates
perfcounters: ratelimit performance counter interrupts
perfcounters fix section mismatch warning in perf_counter.c::perf_counters_lapic_init()
perfcounters: fix refcounting bug
perfcounters: fix "perf counters kill oprofile" bug
perf_counters: account NMI interrupts
perfcounters: fix use after free in perf_release()

Paul Mackerras (27):
perf_counter: Fix return value from dummy hw_perf_counter_init
perf_counter: Fix the cpu_clock software counter
perf_counter: Add optional hw_perf_group_sched_in arch function
perf_counter: Add dummy perf_counter_print_debug function powerpc/perf_counter: Add perf_counter system call on powerpc
powerpc: Provide a way to defer perf counter work until interrupts are enabled powerpc/perf_counter: Add generic support for POWER-family PMU hardware powerpc/perf_counter: Add support for PPC970 family powerpc/perf_counter: Add support for POWER6
perf_counter: Always schedule all software counters in powerpc/perf_counter: Make sure PMU gets enabled properly
perf_counter: Add support for pinned and exclusive counter groups
perf_counter: Add counter enable/disable ioctls
perf_counters: make software counters work as per-cpu counters
perf_counters: allow users to count user, kernel and/or hypervisor events
perfcounters: fix refcounting bug, take 2
perfcounters: make context switch and migration software counters work again perfcounters/powerpc: Make exclude_kernel bit work on Apple G5 processors perfcounters/powerpc: Add support for POWER5 processors
perfcounters: fix a few minor cleanliness issues
perfcounters: provide expansion room in the ABI perfcounters/powerpc: fix oops with multiple counters in a group perfcounters/powerpc: add support for POWER5+ processors perfcounters/powerpc: add support for POWER4 processors
perfcounters: abstract wakeup flag setting in core to fix powerpc build
perf_counter: powerpc: clean up perc_counter_interrupt
perfcounters: fix type/event_id layout on big-endian systems

Peter Zijlstra (19):
perfcounters: IRQ and NMI support on AMD CPUs
perfcounters: IRQ and NMI support on AMD CPUs, fix
x86: perf_counter cleanup
perf_counter: x86: fix 32-bit irq_period assumption
perf_counter: use list_move_tail()
perf_counter: add comment to barrier
perf_counter: x86: use ULL postfix for 64bit constants
perf_counter: software counter event infrastructure
perf_counter: provide pagefault software events
perf_counter: provide major/minor page fault software events
perf_counter: hrtimer based sampling for software time events
perf_counter: add an event_list
perf_counter: fix hrtimer sampling
perf_counter: fix uninitialized usage of event_list
perf_counter: generic context switch event
perf_counter: fix up counter free paths
perf_counter: hook up the tracepoint events
perf_counter: revamp syscall input ABI
perf_counter: unify irq output code

Thomas Gleixner (4):
performance counters: core code
perf counters: protect them against CSTATE transitions perf counters: clean up 'raw' type API
perf counters: expand use of counter->event

Tim Blechmann (1):
perf_counter: include missing header

Yinghai Lu (2):
perf_counter: more barrier in blank weak function
x86: make irqinit_32.c more like irqinit_64.c, v2

 Documentation/perf-counters.txt           |  147 ++
 arch/powerpc/include/asm/hw_irq.h         |   39 +
 arch/powerpc/include/asm/paca.h           |    1 +
 arch/powerpc/include/asm/perf_counter.h   |   72 +
 arch/powerpc/include/asm/systbl.h         |    1 +
 arch/powerpc/include/asm/unistd.h         |    3 +-
 arch/powerpc/kernel/Makefile              |    2 +
 arch/powerpc/kernel/asm-offsets.c         |    1 +
 arch/powerpc/kernel/entry_64.S            |    9 +
 arch/powerpc/kernel/irq.c                 |    5 +
 arch/powerpc/kernel/perf_counter.c        |  822 ++++++++++
 arch/powerpc/kernel/power4-pmu.c          |  557 +++++++
 arch/powerpc/kernel/power5+-pmu.c         |  452 ++++++
 arch/powerpc/kernel/power5-pmu.c          |  475 ++++++
 arch/powerpc/kernel/power6-pmu.c          |  283 ++++
 arch/powerpc/kernel/ppc970-pmu.c          |  375 +++++
 arch/powerpc/mm/fault.c                   |    8 +-
 arch/powerpc/platforms/Kconfig.cputype    |    1 +
 arch/x86/Kconfig                          |    1 +
 arch/x86/ia32/ia32entry.S                 |    3 +-
 arch/x86/include/asm/atomic_32.h          |  218 +++
 arch/x86/include/asm/hardirq.h            |    1 +
 arch/x86/include/asm/hw_irq.h             |    2 +
 arch/x86/include/asm/intel_arch_perfmon.h |   31 -
 arch/x86/include/asm/perf_counter.h       |   98 ++
 arch/x86/include/asm/thread_info.h        |    4 +-
 arch/x86/include/asm/unistd_32.h          |    1 +
 arch/x86/include/asm/unistd_64.h          |    3 +-
 arch/x86/kernel/apic/apic.c               |    4 +
 arch/x86/kernel/cpu/Makefile              |   12 +-
 arch/x86/kernel/cpu/amd.c                 |    4 +
 arch/x86/kernel/cpu/common.c              |    2 +
 arch/x86/kernel/cpu/perf_counter.c        |  989 ++++++++++++
 arch/x86/kernel/cpu/perfctr-watchdog.c    |    4 +-
 arch/x86/kernel/entry_64.S                |    5 +
 arch/x86/kernel/irq.c                     |    5 +
 arch/x86/kernel/irqinit_32.c              |   59 +-
 arch/x86/kernel/irqinit_64.c              |   12 +-
 arch/x86/kernel/signal.c                  |    7 +-
 arch/x86/kernel/syscall_table_32.S        |    1 +
 arch/x86/kernel/traps.c                   |   15 +-
 arch/x86/mm/fault.c                       |   10 +-
 arch/x86/oprofile/nmi_int.c               |    7 +-
 arch/x86/oprofile/op_model_ppro.c         |   10 +-
 drivers/acpi/processor_idle.c             |    4 +
 drivers/char/sysrq.c                      |    2 +
 fs/exec.c                                 |    8 +
 include/linux/init_task.h                 |   13 +
 include/linux/kernel_stat.h               |    8 +
 include/linux/perf_counter.h              |  367 +++++
 include/linux/prctl.h                     |    3 +
 include/linux/sched.h                     |   13 +-
 include/linux/syscalls.h                  |    5 +
 init/Kconfig                              |   35 +
 kernel/Makefile                           |    1 +
 kernel/exit.c                             |   13 +-
 kernel/fork.c                             |    1 +
 kernel/perf_counter.c                     | 2438 +++++++++++++++++++++++++++++
 kernel/sched.c                            |   87 +-
 kernel/sys.c                              |    7 +
 kernel/sys_ni.c                           |    3 +
 61 files changed, 7676 insertions(+), 93 deletions(-)
Documentation/perf-counters.txt new file +147 -0
new file mode 100644
index 0000000..fddd321
@@ -0,0 +1,147 @@
+
+Performance Counters for Linux
+------------------------------
+
+Performance counters are special hardware registers available on most modern
+CPUs. These registers count the number of certain types of hw events: such
+as instructions executed, cachemisses suffered, or branches mis-predicted -
+without slowing down the kernel or applications. These registers can also
+trigger interrupts when a threshold number of events have passed - and can
+thus be used to profile the code that runs on that CPU.
+
+The Linux Performance Counter subsystem provides an abstraction of these
+hardware capabilities. It provides per task and per CPU counters, counter
+groups, and it provides event capabilities on top of those.
+
+Performance counters are accessed via special file descriptors.
+There's one file descriptor per virtual counter used.
+
+The special file descriptor is opened via the perf_counter_open()
+system call:
+
+   int sys_perf_counter_open(struct perf_counter_hw_event *hw_event_uptr,
+			     pid_t pid, int cpu, int group_fd);
+
+The syscall returns the new fd. The fd can be used via the normal
+VFS system calls: read() can be used to read the counter, fcntl()
+can be used to set the blocking mode, etc.
+
+Multiple counters can be kept open at a time, and the counters
+can be poll()ed.
+
+When creating a new counter fd, 'perf_counter_hw_event' is:
+
+/*
+ * Hardware event to monitor via a performance monitoring counter:
+ */
+struct perf_counter_hw_event {
+	s64			type;
+
+	u64			irq_period;
+	u32			record_type;
+
+	u32			disabled     :  1, /* off by default */
+				nmi	     :  1, /* NMI sampling   */
+				raw	     :  1, /* raw event type */
+				__reserved_1 : 29;
+
+	u64			__reserved_2;
+};
+
+/*
+ * Generalized performance counter event types, used by the hw_event.type
+ * parameter of the sys_perf_counter_open() syscall:
+ */
+enum hw_event_types {
+	/*
+	 * Common hardware events, generalized by the kernel:
+	 */
+	PERF_COUNT_CYCLES		=  0,
+	PERF_COUNT_INSTRUCTIONS		=  1,
+	PERF_COUNT_CACHE_REFERENCES	=  2,
+	PERF_COUNT_CACHE_MISSES		=  3,
+	PERF_COUNT_BRANCH_INSTRUCTIONS	=  4,
+	PERF_COUNT_BRANCH_MISSES	=  5,
+
+	/*
+	 * Special "software" counters provided by the kernel, even if
+	 * the hardware does not support performance counters. These
+	 * counters measure various physical and sw events of the
+	 * kernel (and allow the profiling of them as well):
+	 */
+	PERF_COUNT_CPU_CLOCK		= -1,
+	PERF_COUNT_TASK_CLOCK		= -2,
+	/*
+	 * Future software events:
+	 */
+	/* PERF_COUNT_PAGE_FAULTS	= -3,
+	   PERF_COUNT_CONTEXT_SWITCHES	= -4, */
+};
+
+These are standardized types of events that work uniformly on all CPUs
+that implements Performance Counters support under Linux. If a CPU is
+not able to count branch-misses, then the system call will return
+-EINVAL.
+
+More hw_event_types are supported as well, but they are CPU
+specific and are enumerated via /sys on a per CPU basis. Raw hw event
+types can be passed in under hw_event.type if hw_event.raw is 1.
+For example, to count "External bus cycles while bus lock signal asserted"
+events on Intel Core CPUs, pass in a 0x4064 event type value and set
+hw_event.raw to 1.
+
+'record_type' is the type of data that a read() will provide for the
+counter, and it can be one of:
+
+/*
+ * IRQ-notification data record type:
+ */
+enum perf_counter_record_type {
+	PERF_RECORD_SIMPLE		=  0,
+	PERF_RECORD_IRQ			=  1,
+	PERF_RECORD_GROUP		=  2,
+};
+
+a "simple" counter is one that counts hardware events and allows
+them to be read out into a u64 count value. (read() returns 8 on
+a successful read of a simple counter.)
+
+An "irq" counter is one that will also provide an IRQ context information:
+the IP of the interrupted context. In this case read() will return
+the 8-byte counter value, plus the Instruction Pointer address of the
+interrupted context.
+
+The parameter 'hw_event_period' is the number of events before waking up
+a read() that is blocked on a counter fd. Zero value means a non-blocking
+counter.
+
+The 'pid' parameter allows the counter to be specific to a task:
+
+ pid == 0: if the pid parameter is zero, the counter is attached to the
+ current task.
+
+ pid > 0: the counter is attached to a specific task (if the current task
+ has sufficient privilege to do so)
+
+ pid < 0: all tasks are counted (per cpu counters)
+
+The 'cpu' parameter allows a counter to be made specific to a full
+CPU:
+
+ cpu >= 0: the counter is restricted to a specific CPU
+ cpu == -1: the counter counts on all CPUs
+
+(Note: the combination of 'pid == -1' and 'cpu == -1' is not valid.)
+
+A 'pid > 0' and 'cpu == -1' counter is a per task counter that counts
+events of that task and 'follows' that task to whatever CPU the task
+gets schedule to. Per task counters can be created by any user, for
+their own tasks.
+
+A 'pid == -1' and 'cpu == x' counter is a per CPU counter that counts
+all events on CPU-x. Per CPU counters need CAP_SYS_ADMIN privilege.
+
+Group counters are created by passing in a group_fd of another counter.
+Groups are scheduled at once and can be used with PERF_RECORD_GROUP
+to record multi-dimensional timestamps.
+
arch/powerpc/include/asm/hw_irq.h +39 -0
index f75a5fc..94361c0 100644
@@ -131,5 +131,44 @@ static inline int irqs_disabled_flags(unsigned long flags)
  */
 struct hw_interrupt_type;
 
+#ifdef CONFIG_PERF_COUNTERS
+static inline unsigned long get_perf_counter_pending(void)
+{
+	unsigned long x;
+
+	asm volatile("lbz %0,%1(13)"
+		: "=r" (x)
+		: "i" (offsetof(struct paca_struct, perf_counter_pending)));
+	return x;
+}
+
+static inline void set_perf_counter_pending(void)
+{
+	asm volatile("stb %0,%1(13)" : :
+		"r" (1),
+		"i" (offsetof(struct paca_struct, perf_counter_pending)));
+}
+
+static inline void clear_perf_counter_pending(void)
+{
+	asm volatile("stb %0,%1(13)" : :
+		"r" (0),
+		"i" (offsetof(struct paca_struct, perf_counter_pending)));
+}
+
+extern void perf_counter_do_pending(void);
+
+#else
+
+static inline unsigned long get_perf_counter_pending(void)
+{
+	return 0;
+}
+
+static inline void set_perf_counter_pending(void) {}
+static inline void clear_perf_counter_pending(void) {}
+static inline void perf_counter_do_pending(void) {}
+#endif /* CONFIG_PERF_COUNTERS */
+
 #endif	/* __KERNEL__ */
 #endif	/* _ASM_POWERPC_HW_IRQ_H */
arch/powerpc/include/asm/paca.h +1 -0
index 082b3ae..6ef0557 100644
@@ -99,6 +99,7 @@ struct paca_struct {
 	u8 soft_enabled;		/* irq soft-enable flag */
 	u8 hard_enabled;		/* set if irqs are enabled in MSR */
 	u8 io_sync;			/* writel() needs spin_unlock sync */
+	u8 perf_counter_pending;	/* PM interrupt while soft-disabled */
 
 	/* Stuff for accurate time accounting */
 	u64 user_time;			/* accumulated usermode TB ticks */
arch/powerpc/include/asm/perf_counter.h new file +72 -0

72 lines not on this page · raw message

arch/powerpc/include/asm/systbl.h +1 -0
index 72353f6..d312eec 100644
@@ -322,3 +322,4 @@ SYSCALL_SPU(epoll_create1)
 SYSCALL_SPU(dup3)
 SYSCALL_SPU(pipe2)
 SYSCALL(inotify_init1)
+SYSCALL_SPU(perf_counter_open)
arch/powerpc/include/asm/unistd.h +2 -1
index e07d0c7..7cef5af 100644
@@ -341,10 +341,11 @@
 #define __NR_dup3		316
 #define __NR_pipe2		317
 #define __NR_inotify_init1	318
+#define __NR_perf_counter_open	319
 
 #ifdef __KERNEL__
 
-#define __NR_syscalls		319
+#define __NR_syscalls		320
 
 #define __NR__exit __NR_exit
 #define NR_syscalls	__NR_syscalls
arch/powerpc/kernel/Makefile +2 -0
index 8d1a419..8e5e2c7 100644
@@ -94,6 +94,8 @@ obj-$(CONFIG_AUDIT)		+= audit.o
 obj64-$(CONFIG_AUDIT)		+= compat_audit.o
 
 obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
+obj-$(CONFIG_PERF_COUNTERS)	+= perf_counter.o power4-pmu.o ppc970-pmu.o \
+				   power5-pmu.o power5+-pmu.o power6-pmu.o
 
 obj-$(CONFIG_8XX_MINIMAL_FPEMU) += softemu8xx.o
 
arch/powerpc/kernel/asm-offsets.c +1 -0
index 19ee491..3734973 100644
@@ -131,6 +131,7 @@ int main(void)
 	DEFINE(PACAKMSR, offsetof(struct paca_struct, kernel_msr));
 	DEFINE(PACASOFTIRQEN, offsetof(struct paca_struct, soft_enabled));
 	DEFINE(PACAHARDIRQEN, offsetof(struct paca_struct, hard_enabled));
+	DEFINE(PACAPERFPEND, offsetof(struct paca_struct, perf_counter_pending));
 	DEFINE(PACASLBCACHE, offsetof(struct paca_struct, slb_cache));
 	DEFINE(PACASLBCACHEPTR, offsetof(struct paca_struct, slb_cache_ptr));
 	DEFINE(PACACONTEXTID, offsetof(struct paca_struct, context.id));
arch/powerpc/kernel/entry_64.S +9 -0
index 383ed6e..f30b4e5 100644
@@ -526,6 +526,15 @@ ALT_FW_FTR_SECTION_END_IFCLR(FW_FEATURE_ISERIES)
 2:
 	TRACE_AND_RESTORE_IRQ(r5);
 
+#ifdef CONFIG_PERF_COUNTERS
+	/* check paca->perf_counter_pending if we're enabling ints */
+	lbz	r3,PACAPERFPEND(r13)
+	and.	r3,r3,r5
+	beq	27f
+	bl	.perf_counter_do_pending
+27:
+#endif /* CONFIG_PERF_COUNTERS */
+
 	/* extract EE bit and use it to restore paca->hard_enabled */
 	ld	r3,_MSR(r1)
 	rldicl	r4,r3,49,63		/* r0 = (r3 >> 15) & 1 */
arch/powerpc/kernel/irq.c +5 -0
index 1b55ffd..26204a4 100644
@@ -135,6 +135,11 @@ notrace void raw_local_irq_restore(unsigned long en)
 			iseries_handle_interrupts();
 	}
 
+	if (get_perf_counter_pending()) {
+		clear_perf_counter_pending();
+		perf_counter_do_pending();
+	}
+
 	/*
 	 * if (get_paca()->hard_enabled) return;
 	 * But again we need to take care that gcc gets hard_enabled directly
arch/powerpc/kernel/perf_counter.c new file +822 -0

822 lines not on this page · raw message

arch/powerpc/kernel/power4-pmu.c new file +557 -0

557 lines not on this page · raw message

arch/powerpc/kernel/power5+-pmu.c new file +452 -0

452 lines not on this page · raw message

arch/powerpc/kernel/power5-pmu.c new file +475 -0

475 lines not on this page · raw message

arch/powerpc/kernel/power6-pmu.c new file +283 -0

283 lines not on this page · raw message

arch/powerpc/kernel/ppc970-pmu.c new file +375 -0

84 lines not on this page · raw message

arch/powerpc/mm/fault.c +7 -1
index 91c7b86..de37a3a 100644

Lines of this file are not shown. View the raw message.

arch/powerpc/platforms/Kconfig.cputype +1 -0
index e868b5c..dc0f3c9 100644

Lines of this file are not shown. View the raw message.

arch/x86/Kconfig +1 -0
index a2d5f39..f5d7d29 100644

Lines of this file are not shown. View the raw message.

arch/x86/ia32/ia32entry.S +2 -1
index 097a6b6..e4baa06 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/atomic_32.h +218 -0

218 lines not on this page · raw message

arch/x86/include/asm/hardirq.h +1 -0
index 039db6a..2545442 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/hw_irq.h +2 -0
index b762ea4..ae80f64 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/intel_arch_perfmon.h deleted +0 -31
deleted file mode 100644
index fa0fd06..0000000

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/perf_counter.h new file +98 -0

98 lines not on this page · raw message

arch/x86/include/asm/thread_info.h +3 -1
index 431c246..83d2b73 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/unistd_32.h +1 -0
index f2bba78..7e47658 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/unistd_64.h +2 -1
index d2e415e..53025fe 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/apic/apic.c +4 -0
index 85eb8e1..b0e5e71 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/cpu/Makefile +7 -5
index 4e242f9..3efcb2b 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/cpu/amd.c +4 -0
index 7e4a459..fd69c51 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/cpu/common.c +2 -0
index c4f6678..a86769e 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/cpu/perf_counter.c new file +989 -0

989 lines not on this page · raw message

arch/x86/kernel/cpu/perfctr-watchdog.c +2 -2
index f6c70a1..d6f5b9f 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/entry_64.S +5 -0
index a331ec3..3f129d9 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/irq.c +5 -0
index b8ac3b6..33d0723 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/irqinit_32.c +36 -23

59 lines not on this page · raw message

arch/x86/kernel/irqinit_64.c +9 -3
index c7a49e0..5c9ed86 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/signal.c +6 -1
index dfcc74a..62f2164 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/syscall_table_32.S +1 -0
index 3bdb648..b7607c4 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/traps.c +6 -9
index 8f20204..e447aa8 100644

Lines of this file are not shown. View the raw message.

arch/x86/mm/fault.c +8 -2
index 3fcd79a..f70b901 100644

Lines of this file are not shown. View the raw message.

arch/x86/oprofile/nmi_int.c +4 -3
index 202864a..c638685 100644

Lines of this file are not shown. View the raw message.

arch/x86/oprofile/op_model_ppro.c +9 -1
index 10131fb..4da7230 100644

Lines of this file are not shown. View the raw message.

drivers/acpi/processor_idle.c +4 -0
index 7bc22a4..08def2f 100644

Lines of this file are not shown. View the raw message.

drivers/char/sysrq.c +2 -0
index 6f8579d..1777977 100644

Lines of this file are not shown. View the raw message.

fs/exec.c +8 -0
index 929b580..af1600c 100644

Lines of this file are not shown. View the raw message.

include/linux/init_task.h +13 -0
index af1de95..ca226a9 100644

Lines of this file are not shown. View the raw message.

include/linux/kernel_stat.h +8 -0
index 0c8b89f..b6d2887 100644

Lines of this file are not shown. View the raw message.

include/linux/perf_counter.h new file +367 -0

367 lines not on this page · raw message

include/linux/prctl.h +3 -0
index 48d887e..b00df4c 100644

Lines of this file are not shown. View the raw message.

include/linux/sched.h +12 -1
index 19187a2..1410859 100644

Lines of this file are not shown. View the raw message.

include/linux/syscalls.h +5 -0
index 0cff9bb..dfe2a44 100644

Lines of this file are not shown. View the raw message.

init/Kconfig +35 -0
index d8c95e1..215deb7 100644

Lines of this file are not shown. View the raw message.

kernel/Makefile +1 -0
index 3e43fd1..5714563 100644

Lines of this file are not shown. View the raw message.

kernel/exit.c +9 -4
index 167e1e3..f52c24e 100644

Lines of this file are not shown. View the raw message.

kernel/fork.c +1 -0
index 39b1062..656d798 100644

Lines of this file are not shown. View the raw message.

kernel/perf_counter.c new file +2438 -0

2,438 lines not on this page · raw message

kernel/sched.c +84 -3

87 lines not on this page · raw message

kernel/sys.c +7 -0
index 37f458e..7306f94 100644

Lines of this file are not shown. View the raw message.

kernel/sys_ni.c +3 -0
index 27dad29..68320f6 100644

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Part of the diff is not shown here. View the raw message.

#5 lore raw

Re: [tree] Performance Counters for Linux, v7

Quoted 6 lines · Ingo Molnar

There's also been lots of updates to the userspace tools (kerneltop and perfstat):

http://redhat.com/~mingo/perfcounters/kerneltop.c
http://redhat.com/~mingo/perfcounters/perfstat.c

If possible, can we also prepare git tree for these userspace tools by this way we can also look on logs and check incremental changes otherwise we leads to:

[jaswinder@hpdv5 20090321]$ cc -O2 -g -lrt -Wall -W -o perfstat perfstat.c
perfstat.c:57:40: error: include/linux/perf_counter.h: No such file or directory
perfstat.c:83: warning: ‘struct perf_counter_hw_event’ declared inside parameter list
perfstat.c:83: warning: its scope is only this definition or declaration, which is probably not what you want perfstat.c:127: error: ‘PERF_TYPE_SOFTWARE’ undeclared here (not in a function)
perfstat.c:127: error: ‘PERF_COUNT_TASK_CLOCK’ undeclared here (not in a function)
perfstat.c:128: error: ‘PERF_COUNT_CPU_MIGRATIONS’ undeclared here (not in a function) perfstat.c:129: error: ‘PERF_COUNT_CONTEXT_SWITCHES’ undeclared here (not in a function) perfstat.c:130: error: ‘PERF_COUNT_PAGE_FAULTS’ undeclared here (not in a function)
perfstat.c:131: error: ‘PERF_TYPE_HARDWARE’ undeclared here (not in a function)
perfstat.c:131: error: ‘PERF_COUNT_CPU_CYCLES’ undeclared here (not in a function)
perfstat.c:132: error: ‘PERF_COUNT_INSTRUCTIONS’ undeclared here (not in a function)
perfstat.c:133: error: ‘PERF_COUNT_CACHE_REFERENCES’ undeclared here (not in a function) perfstat.c:134: error: ‘PERF_COUNT_CACHE_MISSES’ undeclared here (not in a function)
perfstat.c: In function ‘type_valid’:
perfstat.c:164: error: ‘PERF_HW_EVENTS_MAX’ undeclared (first use in this function)
perfstat.c:164: error: (Each undeclared identifier is reported only once
perfstat.c:164: error: for each function it appears in.)
perfstat.c:166: error: ‘PERF_SW_EVENTS_MAX’ undeclared (first use in this function)
perfstat.c:167: error: ‘PERF_TYPE_TRACEPOINT’ undeclared (first use in this function)
perfstat.c: In function ‘event_name’:
perfstat.c:194: error: ‘PERF_TYPE_TRACEPOINT’ undeclared (first use in this function)
perfstat.c: In function ‘create_counter’:
perfstat.c:286: error: storage size of ‘hw_event’ isn’t known
perfstat.c:290: error: ‘PERF_RECORD_SIMPLE’ undeclared (first use in this function)
perfstat.c:286: warning: unused variable ‘hw_event’
perfstat.c: In function ‘main’:
perfstat.c:385: error: ‘PERF_COUNT_CPU_CLOCK’ undeclared (first use in this function) [jaswinder@hpdv5 20090321]$
[jaswinder@hpdv5 20090321]$ cc -O2 -g -lrt -Wall -W -o perfstat perfstat.c -I/home/jaswinder/jaswinder-git/linux-2.6-tip [jaswinder@hpdv5 20090321]$ ./perfstat -e 1 -e 3 -e 5 ls -lR /usr/include/
Usage: perfstat [<events...>] <cmd...>

PerfStat Options (up to 64 event types can be specified):

-e EID --event_id=EID # event type ID
0: CPU cycles
1: instructions 2: cache accesses 3: cache misses 4: branch instructions 5: branch prediction misses 6: bus cycles

-s # system-wide collection

-c <cmd..> --command=<cmd..> # command+arguments to be timed.

[jaswinder@hpdv5 20090321]$ ./perfstat -e 0,1,2 -c ls
Usage: perfstat [<events...>] <cmd...>

PerfStat Options (up to 64 event types can be specified):

-e EID --event_id=EID # event type ID
0: CPU cycles
1: instructions 2: cache accesses 3: cache misses 4: branch instructions 5: branch prediction misses 6: bus cycles

-s # system-wide collection

-c <cmd..> --command=<cmd..> # command+arguments to be timed.

[jaswinder@hpdv5 20090321]$

Thanks,

--
JSR

#6 lore raw

Re: [tree] Performance Counters for Linux, v7

Quoted 9 lines · Jaswinder Singh Rajput
Quoted 6 lines · Ingo Molnar

There's also been lots of updates to the userspace tools (kerneltop and perfstat):

http://redhat.com/~mingo/perfcounters/kerneltop.c
http://redhat.com/~mingo/perfcounters/perfstat.c

Quoted 6 lines

If possible, can we also prepare git tree for these userspace tools by this way we can also look on logs and check incremental changes otherwise we leads to:

[jaswinder@hpdv5 20090321]$ cc -O2 -g -lrt -Wall -W -o perfstat perfstat.c perfstat.c:57:40: error: include/linux/perf_counter.h: No such file or directory

to get the latest perfcounter tools, do this on latest -tip:

cd Documentation/perf_counter/
make

which will build the sample tools. kerneltop and perfstat has been unified into a single .c file by Wu Fengguang, with updates by Peter Zijstra.

Ingo

#7 lore raw
In reply to Andrew Morton
Quoted 4 lines · Andrew Morton

Perhaps one of the reasons why this code is confusing is the blurring between the "time" at which an event occured and the "time" between the occurrence of two events. A weakness in English, I guess. Using the term "interval" in the latter case will help a lot.

Except that we aren't measuring an "interval", we're measuring the combined length of a whole series of intervals. What's a good word for that?

Quoted 4 lines

include/linux/perf_counter.h struct perf_counter { · line 263

Quoted 2 lines
include/linux/perf_counter.h +2 -0
+	atomic64_t			child_time_enabled;
+	atomic64_t			child_time_running;

These read like booleans, but why are they atomic64_t's?

OK so this file could use more comments, but I did answer that question in the patch description.

Quoted 14 lines

kernel/perf_counter.c perf_read_hw(struct perf_counter *counter, char __user *buf, size_t count) · line 1314

Quoted 7 lines
kernel/perf_counter.c +6 -1
-	return put_user(cntval, (u64 __user *) buf) ? -EFAULT : sizeof(cntval);
+	if (count != n * sizeof(u64))
+		return -EINVAL;
+
+	if (!access_ok(VERIFY_WRITE, buf, count))
+		return -EFAULT;
+	

<panics>

Oh.

It would be a lot more reassuring to verify uptr', rather than buf' here.

Quoted 7 lines

The patch adds new trailing whitespace. checkpatch helps.

kernel/perf_counter.c perf_read_hw(struct perf_counter *counter, char __user *buf, size_t count) · line 1317

Quoted 3 lines
kernel/perf_counter.c +3 -0
+	for (i = 0; i < n; ++i)
+		if (__put_user(values[i], uptr + i))
+			return -EFAULT;

And here we iterate across n', whereas we verified count'.

And the fact that we just verified count == n * 8, four lines above, doesn't give you any comfort?

Paul.

maintainer
#8 lore raw
Quoted 10 lines · On Sun, 22 Mar 2009 10:13:35 +1100 Paul Mackerras <[email protected]> wrote:
Quoted 4 lines · Andrew Morton

Perhaps one of the reasons why this code is confusing is the blurring between the "time" at which an event occured and the "time" between the occurrence of two events. A weakness in English, I guess. Using the term "interval" in the latter case will help a lot.

Except that we aren't measuring an "interval", we're measuring the combined length of a whole series of intervals. What's a good word for that?

foo_total_time?

It doesn't matter so much if the thing has a comment at the definition site.

Quoted 5 lines
Quoted 4 lines

include/linux/perf_counter.h struct perf_counter { · line 263

Quoted 2 lines
include/linux/perf_counter.h +2 -0
+	atomic64_t			child_time_enabled;
+	atomic64_t			child_time_running;

These read like booleans, but why are they atomic64_t's?

Quoted 16 lines

OK so this file could use more comments, but I did answer that question in the patch description.

Quoted 13 lines

kernel/perf_counter.c perf_read_hw(struct perf_counter *counter, char __user *buf, size_t count) · line 1314

Quoted 7 lines
kernel/perf_counter.c +6 -1
-	return put_user(cntval, (u64 __user *) buf) ? -EFAULT : sizeof(cntval);
+	if (count != n * sizeof(u64))
+		return -EINVAL;
+
+	if (!access_ok(VERIFY_WRITE, buf, count))
+		return -EFAULT;
+	

<panics>

Oh.

It would be a lot more reassuring to verify uptr', rather than buf' here.

This?

Quoted 10 lines
Quoted 7 lines

The patch adds new trailing whitespace. checkpatch helps.

kernel/perf_counter.c perf_read_hw(struct perf_counter *counter, char __user *buf, size_t count) · line 1317

Quoted 3 lines
kernel/perf_counter.c +3 -0
+	for (i = 0; i < n; ++i)
+		if (__put_user(values[i], uptr + i))
+			return -EFAULT;

And here we iterate across n', whereas we verified count'.

And the fact that we just verified count == n * 8, four lines above, doesn't give you any comfort?

	access_ok(..., uptr, n * sizeof(*uptr))

might be most robust.

Or fix up the types (if needed) and copy the whole thing with copy_to_user()

Is it really so performance-sensitive that we can't use plain old put_user()?

#9 lore raw
Quoted 24 lines · Andrew Morton
Quoted 10 lines
Quoted 10 lines

kernel/perf_counter.c perf_read_hw(struct perf_counter *counter, char __user *buf, size_t count) · line 1314

Quoted 7 lines
kernel/perf_counter.c +6 -1
-	return put_user(cntval, (u64 __user *) buf) ? -EFAULT : sizeof(cntval);
+	if (count != n * sizeof(u64))
+		return -EINVAL;
+
+	if (!access_ok(VERIFY_WRITE, buf, count))
+		return -EFAULT;
+	

<panics>

Quoted 3 lines
Quoted 3 lines

Oh.

It would be a lot more reassuring to verify uptr', rather than buf' here.

This?

Quoted 8 lines
Quoted 7 lines

The patch adds new trailing whitespace. checkpatch helps.

kernel/perf_counter.c perf_read_hw(struct perf_counter *counter, char __user *buf, size_t count) · line 1317

Quoted 3 lines
kernel/perf_counter.c +3 -0
+	for (i = 0; i < n; ++i)
+		if (__put_user(values[i], uptr + i))
+			return -EFAULT;

And here we iterate across n', whereas we verified count'.

Quoted 6 lines
Quoted 2 lines

And the fact that we just verified count == n * 8, four lines above, doesn't give you any comfort?

	access_ok(..., uptr, n * sizeof(*uptr))

might be most robust.

I'm not sure quite what you're getting at. Are you saying

(a) you think there is an actual security problem in that code
(b) you can't tell if there is a problem in that code, or
(c) you think there isn't a problem but you had to spend too much brain power working that out, or
(d) you think there isn't a problem but there might be in future if someone changed the code?

Quoted 3 lines

Or fix up the types (if needed) and copy the whole thing with copy_to_user()

Is it really so performance-sensitive that we can't use plain old put_user()?

The sort of people that use performance-monitoring tools tend to be sensitive to performance issues in their tools, for some reason. :)

Is copy_to_user on 8 bytes practically as fast as put_user, these days?

Paul.

#10 lore raw
In reply to Paul Mackerras
Quoted 10 lines · Paul Mackerras
Quoted 4 lines · Andrew Morton

Perhaps one of the reasons why this code is confusing is the blurring between the "time" at which an event occured and the "time" between the occurrence of two events.  A weakness in English, I guess.  Using the term "interval" in the latter case will help a lot.

Except that we aren't measuring an "interval", we're measuring the combined length of a whole series of intervals.  What's a good word for that?

As Ingo pointed out, 'runtime' is a decent choice for that.