Performance Counters for Linux, v2

[ Performance counters are special hardware registers available on most
modern CPUs. These register count the number of certain types of hw
events: such as instructions executed, cachemisses suffered, or branches mis-predicted, without slowing down the kernel or applications. These registers can also trigger interrupts when a threshold number of events have passed - and can thus be used to profile the code that runs on that CPU. ]

This is version 2 of our Performance Counters subsystem implementation.

The biggest user-visible change in this release is a new user-space text-mode profiling utility that is based on this code: KernelTop.

KernelTop can be downloaded from:

http://redhat.com/~mingo/perfcounters/kerneltop.c

It's a standalone .c file that needs no extra libraries - it only needs a CONFIG_PERF_COUNTERS=y kernel to run on.

This utility is intended for kernel developers - it's basically a dynamic kernel profiler that gets hardware counter events dispatched to it continuously, which it feeds into a histogram and outputs it periodically.

Here is a screenshot of it:

------------------------------------------------------------------------------
KernelTop: 250880 irqs/sec [NMI, 10000 cycles], (all, cpu: 0)
------------------------------------------------------------------------------

             events         RIP          kernel function
             ______   ________________   _______________

              17319 - ffffffff8106f8fa : audit_syscall_exit
              16300 - ffffffff81042ce2 : sys_rt_sigprocmask
              11031 - ffffffff8106fdc8 : audit_syscall_entry
              10880 - ffffffff8100bd8d : rff_trace
               9780 - ffffffff810a232f : kfree	[ehci_hcd]
               9707 - ffffffff81298cb7 : _spin_lock_irq	[ehci_hcd]
               7903 - ffffffff8106db17 : unroll_tree_refs
               7266 - ffffffff81138d10 : copy_user_generic_string
               5751 - ffffffff8100be45 : sysret_check
               4803 - ffffffff8100bea8 : sysret_signal
               4696 - ffffffff8100bdb0 : system_call
               4425 - ffffffff8100bdc0 : system_call_after_swapgs
               2855 - ffffffff810ae183 : path_put	[ext3]
               2773 - ffffffff8100bedb : auditsys
               1589 - ffffffff810b6864 : dput	[sunrpc]
               1253 - ffffffff8100be40 : ret_from_sys_call
                690 - ffffffff8105034c : current_kernel_time	[ext3]
                673 - ffffffff81042bd4 : sys_sigprocmask
                611 - ffffffff8100bf25 : sysret_audit

It will correctly profile core kernel, module space and vsyscall areas as well. It allows the use of the most common hw counters: cycles, instructions, branches, cachemisses, cache-references and branch-misses.

KernelTop does not have to be started/stopped - it will continously profile the system and updates the histogram as the workload changes. The histogram is not cumulative: old workload effects will time out gradually. For example if the system goes idle, then the profiler output will go down to near zero within 10-20 seconds. So there's no need to stop or restart profiling - it all updates automatically, as the workflow changes its characteristics.

KernelTop can also profile raw event IDs. For example, on a Core2 CPU, to profile the "Number of instruction length decoder stalls" (raw event 0x0087) during a hackbench run, i did this:

  $ ./kerneltop -e -$(printf "%d\n" 0x00000087) -c 10000 -n 1

------------------------------------------------------------------------------
KernelTop: 331 irqs/sec [NMI, 10000 raw:0087], (all, 2 CPUs)
------------------------------------------------------------------------------

             events         RIP          kernel function
             ______   ________________   _______________

               1016 - ffffffff802a611e : kmem_cache_alloc_node
                898 - ffffffff804ca381 : sock_wfree
                 64 - ffffffff80567306 : schedule
                 50 - ffffffff804cdb39 : skb_release_head_state
                 45 - ffffffff8053ed54 : unix_write_space
                 33 - ffffffff802a6a4d : __kmalloc_node
                 18 - ffffffff802a642c : cache_alloc_refill
                 13 - ffffffff804cdd50 : __alloc_skb
                  7 - ffffffff8053ec0a : unix_shutdown

[ The printf is done to pass in a negative event number as a parameter. ]

We also made a good number of internal changes as well to the subsystem:

There's a new "counter group record" facility that is a straightforward extension of the existing "irq record" notification type. This record type can be set on a 'master' counter, and if the master counter triggers an IRQ or an NMI, all the 'secondary' counters are read out atomically and are put into the counter-group record. The result can then be read() out by userspace via a single system call. (Based on extensive feedback from Paul Mackerras and David Miller, thanks guys!)

The other big change is the support of virtual task counters via counter
scheduling: a task can specify more counters than there are on the CPU, the kernel will then schedule the counters periodically to spread out hw resources. So for example if a task starts 6 counters on a CPU that has only two hardware counters, it still gets this output:

 counter[0 cycles              ]:   5204680573 , delta:   1733680843 events
 counter[1 instructions        ]:   1364468045 , delta:    454818351 events
 counter[2 cache-refs          ]:        12732 , delta:         4399 events
 counter[3 cache-misses        ]:         1009 , delta:          336 events

counter[4 branch-instructions ]: 125993304 , delta: 42006998 events
counter[5 branch-misses ]: 1946 , delta: 649 events

See this sample code at:

http://redhat.com/~mingo/perfcounters/hello-loop.c

There's also now the ability to do NMI profiling: this works both for per CPU and per task counters. NMI counters are transparent and are enabled via the PERF_COUNTER_NMI bit in the "hardware event type" parameter of the sys_perf_counter_open() system call.

There's also more generic x86 support: all 4 generic PMCs of Nehalem / Core i7 are supported - i've run 4 instances of KernelTop and they used up four separate PMCs.

There's also perf counters debug output that can be triggered via sysrq, for diagnostic purposes.

Ingo, Thomas

------------------->

The latest performance counters experimental git tree can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git perfcounters/core

--------------->
Ingo Molnar (2):
performance counters: documentation
performance counters: x86 support

Thomas Gleixner (1):
performance counters: core code

 Documentation/perf-counters.txt                |  104 +++
 arch/x86/Kconfig                               |    1 +
 arch/x86/ia32/ia32entry.S                      |    3 +-
 arch/x86/include/asm/hardirq_32.h              |    1 +
 arch/x86/include/asm/hw_irq.h                  |    2 +
 arch/x86/include/asm/intel_arch_perfmon.h      |   34 +-
 arch/x86/include/asm/irq_vectors.h             |    5 +
 arch/x86/include/asm/mach-default/entry_arch.h |    5 +
 arch/x86/include/asm/pda.h                     |    1 +
 arch/x86/include/asm/thread_info.h             |    4 +-
 arch/x86/include/asm/unistd_32.h               |    1 +
 arch/x86/include/asm/unistd_64.h               |    3 +-
 arch/x86/kernel/apic.c                         |    2 +
 arch/x86/kernel/cpu/Makefile                   |   12 +-
 arch/x86/kernel/cpu/common.c                   |    2 +
 arch/x86/kernel/cpu/perf_counter.c             |  571 ++++++++++++++
 arch/x86/kernel/entry_64.S                     |    6 +
 arch/x86/kernel/irq.c                          |    5 +
 arch/x86/kernel/irqinit_32.c                   |    3 +
 arch/x86/kernel/irqinit_64.c                   |    5 +
 arch/x86/kernel/signal_32.c                    |    8 +-
 arch/x86/kernel/signal_64.c                    |    5 +
 arch/x86/kernel/syscall_table_32.S             |    1 +
 drivers/char/sysrq.c                           |    2 +
 include/linux/perf_counter.h                   |  171 +++++
 include/linux/sched.h                          |    9 +
 include/linux/syscalls.h                       |    6 +
 init/Kconfig                                   |   29 +
 kernel/Makefile                                |    1 +
 kernel/fork.c                                  |    1 +
 kernel/perf_counter.c                          |  945 ++++++++++++++++++++++++
 kernel/sched.c                                 |   24 +
 kernel/sys_ni.c                                |    3 +
 33 files changed, 1954 insertions(+), 21 deletions(-)
 create mode 100644 Documentation/perf-counters.txt
 create mode 100644 arch/x86/kernel/cpu/perf_counter.c
 create mode 100644 include/linux/perf_counter.h
 create mode 100644 kernel/perf_counter.c

new file mode 100644
index 0000000..19033a0
@@ -0,0 +1,104 @@
+
+Performance Counters for Linux
+------------------------------
+
+Performance counters are special hardware registers available on most modern
+CPUs. These registers count the number of certain types of hw events: such
+as instructions executed, cachemisses suffered, or branches mis-predicted -
+without slowing down the kernel or applications. These registers can also
+trigger interrupts when a threshold number of events have passed - and can
+thus be used to profile the code that runs on that CPU.
+
+The Linux Performance Counter subsystem provides an abstraction of these
+hardware capabilities. It provides per task and per CPU counters, and
+it provides event capabilities on top of those.
+
+Performance counters are accessed via special file descriptors.
+There's one file descriptor per virtual counter used.
+
+The special file descriptor is opened via the perf_counter_open()
+system call:
+
+ int
+ perf_counter_open(u32 hw_event_type,
+                   u32 hw_event_period,
+                   u32 record_type,
+                   pid_t pid,
+                   int cpu);
+
+The syscall returns the new fd. The fd can be used via the normal
+VFS system calls: read() can be used to read the counter, fcntl()
+can be used to set the blocking mode, etc.
+
+Multiple counters can be kept open at a time, and the counters
+can be poll()ed.
+
+When creating a new counter fd, 'hw_event_type' is one of:
+
+ enum hw_event_types {
+	PERF_COUNT_CYCLES,
+	PERF_COUNT_INSTRUCTIONS,
+	PERF_COUNT_CACHE_REFERENCES,
+	PERF_COUNT_CACHE_MISSES,
+	PERF_COUNT_BRANCH_INSTRUCTIONS,
+	PERF_COUNT_BRANCH_MISSES,
+ };
+
+These are standardized types of events that work uniformly on all CPUs
+that implements Performance Counters support under Linux. If a CPU is
+not able to count branch-misses, then the system call will return
+-EINVAL.
+
+[ Note: more hw_event_types are supported as well, but they are CPU
+  specific and are enumerated via /sys on a per CPU basis. Raw hw event
+  types can be passed in as negative numbers. For example, to count
+  "External bus cycles while bus lock signal asserted" events on Intel
+  Core CPUs, pass in a -0x4064 event type value. ]
+
+The parameter 'hw_event_period' is the number of events before waking up
+a read() that is blocked on a counter fd. Zero value means a non-blocking
+counter.
+
+'record_type' is the type of data that a read() will provide for the
+counter, and it can be one of:
+
+  enum perf_record_type {
+	PERF_RECORD_SIMPLE,
+	PERF_RECORD_IRQ,
+  };
+
+a "simple" counter is one that counts hardware events and allows
+them to be read out into a u64 count value. (read() returns 8 on
+a successful read of a simple counter.)
+
+An "irq" counter is one that will also provide an IRQ context information:
+the IP of the interrupted context. In this case read() will return
+the 8-byte counter value, plus the Instruction Pointer address of the
+interrupted context.
+
+The 'pid' parameter allows the counter to be specific to a task:
+
+ pid == 0: if the pid parameter is zero, the counter is attached to the
+ current task.
+
+ pid > 0: the counter is attached to a specific task (if the current task
+ has sufficient privilege to do so)
+
+ pid < 0: all tasks are counted (per cpu counters)
+
+The 'cpu' parameter allows a counter to be made specific to a full
+CPU:
+
+ cpu >= 0: the counter is restricted to a specific CPU
+ cpu == -1: the counter counts on all CPUs
+
+Note: the combination of 'pid == -1' and 'cpu == -1' is not valid.
+
+A 'pid > 0' and 'cpu == -1' counter is a per task counter that counts
+events of that task and 'follows' that task to whatever CPU the task
+gets schedule to. Per task counters can be created by any user, for
+their own tasks.
+
+A 'pid == -1' and 'cpu == x' counter is a per CPU counter that counts
+all events on CPU-x. Per CPU counters need CAP_SYS_ADMIN privilege.
+

index ac22bb7..5a2d74a 100644
@@ -651,6 +651,7 @@ config X86_UP_IOAPIC
 config X86_LOCAL_APIC
 	def_bool y
 	depends on X86_64 || (X86_32 && (X86_UP_APIC || (SMP && !X86_VOYAGER) || X86_GENERICARCH))
+	select HAVE_PERF_COUNTERS
 
 config X86_IO_APIC
 	def_bool y

index 256b00b..3c14ed0 100644
@@ -823,7 +823,8 @@ ia32_sys_call_table:
 	.quad compat_sys_signalfd4
 	.quad sys_eventfd2
 	.quad sys_epoll_create1
-	.quad sys_dup3			/* 330 */
+	.quad sys_dup3				/* 330 */
 	.quad sys_pipe2
 	.quad sys_inotify_init1
+	.quad sys_perf_counter_open
 ia32_syscall_end:

index 5ca135e..b3e475d 100644
@@ -9,6 +9,7 @@ typedef struct {
 	unsigned long idle_timestamp;
 	unsigned int __nmi_count;	/* arch dependent */
 	unsigned int apic_timer_irqs;	/* arch dependent */
+	unsigned int apic_perf_irqs;	/* arch dependent */
 	unsigned int irq0_irqs;
 	unsigned int irq_resched_count;
 	unsigned int irq_call_count;

index b97aecb..c22900e 100644
@@ -30,6 +30,8 @@
 /* Interrupt handlers registered during init_IRQ */
 extern void apic_timer_interrupt(void);
 extern void error_interrupt(void);
+extern void perf_counter_interrupt(void);
+
 extern void spurious_interrupt(void);
 extern void thermal_interrupt(void);
 extern void reschedule_interrupt(void);

index fa0fd06..71598a9 100644
@@ -1,22 +1,24 @@
 #ifndef _ASM_X86_INTEL_ARCH_PERFMON_H
 #define _ASM_X86_INTEL_ARCH_PERFMON_H
 
-#define MSR_ARCH_PERFMON_PERFCTR0		0xc1
-#define MSR_ARCH_PERFMON_PERFCTR1		0xc2
+#define MSR_ARCH_PERFMON_PERFCTR0			      0xc1
+#define MSR_ARCH_PERFMON_PERFCTR1			      0xc2
 
-#define MSR_ARCH_PERFMON_EVENTSEL0		0x186
-#define MSR_ARCH_PERFMON_EVENTSEL1		0x187
+#define MSR_ARCH_PERFMON_EVENTSEL0			     0x186
+#define MSR_ARCH_PERFMON_EVENTSEL1			     0x187
 
-#define ARCH_PERFMON_EVENTSEL0_ENABLE	(1 << 22)
-#define ARCH_PERFMON_EVENTSEL_INT	(1 << 20)
-#define ARCH_PERFMON_EVENTSEL_OS	(1 << 17)
-#define ARCH_PERFMON_EVENTSEL_USR	(1 << 16)
+#define ARCH_PERFMON_EVENTSEL0_ENABLE			  (1 << 22)
+#define ARCH_PERFMON_EVENTSEL_INT			  (1 << 20)
+#define ARCH_PERFMON_EVENTSEL_OS			  (1 << 17)
+#define ARCH_PERFMON_EVENTSEL_USR			  (1 << 16)
 
-#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_SEL	(0x3c)
-#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_UMASK	(0x00 << 8)
-#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_INDEX (0)
+#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_SEL		      0x3c
+#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_UMASK		(0x00 << 8)
+#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_INDEX 		 0
 #define ARCH_PERFMON_UNHALTED_CORE_CYCLES_PRESENT \
-	(1 << (ARCH_PERFMON_UNHALTED_CORE_CYCLES_INDEX))
+		(1 << (ARCH_PERFMON_UNHALTED_CORE_CYCLES_INDEX))
+
+#define ARCH_PERFMON_BRANCH_MISSES_RETIRED			 6
 
 union cpuid10_eax {
 	struct {

@@ -28,4 +30,12 @@ union cpuid10_eax {
 	unsigned int full;
 };
 
+#ifdef CONFIG_PERF_COUNTERS
+extern void init_hw_perf_counters(void);
+extern void perf_counters_lapic_init(int nmi);
+#else
+static inline void init_hw_perf_counters(void)		{ }
+static inline void perf_counters_lapic_init(int nmi)	{ }
+#endif
+
 #endif /* _ASM_X86_INTEL_ARCH_PERFMON_H */

index 0005adb..b8d277f 100644
@@ -87,6 +87,11 @@
 #define LOCAL_TIMER_VECTOR	0xef
 
 /*
+ * Performance monitoring interrupt vector:
+ */
+#define LOCAL_PERF_VECTOR	0xee
+
+/*
  * First APIC vector available to drivers: (vectors 0x30-0xee) we
  * start at 0x31(0x41) to spread out vectors evenly between priority
  * levels. (0x80 is the syscall vector)

index 6b1add8..ad31e5d 100644
@@ -25,10 +25,15 @@ BUILD_INTERRUPT(irq_move_cleanup_interrupt,IRQ_MOVE_CLEANUP_VECTOR)
  * a much simpler SMP time architecture:
  */
 #ifdef CONFIG_X86_LOCAL_APIC
+
 BUILD_INTERRUPT(apic_timer_interrupt,LOCAL_TIMER_VECTOR)
 BUILD_INTERRUPT(error_interrupt,ERROR_APIC_VECTOR)
 BUILD_INTERRUPT(spurious_interrupt,SPURIOUS_APIC_VECTOR)
 
+#ifdef CONFIG_PERF_COUNTERS
+BUILD_INTERRUPT(perf_counter_interrupt, LOCAL_PERF_VECTOR)
+#endif
+
 #ifdef CONFIG_X86_MCE_P4THERMAL
 BUILD_INTERRUPT(thermal_interrupt,THERMAL_APIC_VECTOR)
 #endif

index 2fbfff8..90a8d9d 100644
@@ -30,6 +30,7 @@ struct x8664_pda {
 	short isidle;
 	struct mm_struct *active_mm;
 	unsigned apic_timer_irqs;
+	unsigned apic_perf_irqs;
 	unsigned irq0_irqs;
 	unsigned irq_resched_count;
 	unsigned irq_call_count;

index e44d379..810bf26 100644
@@ -80,6 +80,7 @@ struct thread_info {
 #define TIF_SYSCALL_AUDIT	7	/* syscall auditing active */
 #define TIF_SECCOMP		8	/* secure computing */
 #define TIF_MCE_NOTIFY		10	/* notify userspace of an MCE */
+#define TIF_PERF_COUNTERS	11	/* notify perf counter work */
 #define TIF_NOTSC		16	/* TSC is not accessible in userland */
 #define TIF_IA32		17	/* 32bit process */
 #define TIF_FORK		18	/* ret_from_fork */

@@ -103,6 +104,7 @@ struct thread_info {
 #define _TIF_SYSCALL_AUDIT	(1 << TIF_SYSCALL_AUDIT)
 #define _TIF_SECCOMP		(1 << TIF_SECCOMP)
 #define _TIF_MCE_NOTIFY		(1 << TIF_MCE_NOTIFY)
+#define _TIF_PERF_COUNTERS	(1 << TIF_PERF_COUNTERS)
 #define _TIF_NOTSC		(1 << TIF_NOTSC)
 #define _TIF_IA32		(1 << TIF_IA32)
 #define _TIF_FORK		(1 << TIF_FORK)

@@ -135,7 +137,7 @@ struct thread_info {
 
 /* Only used for 64 bit */
 #define _TIF_DO_NOTIFY_MASK						\
-	(_TIF_SIGPENDING|_TIF_MCE_NOTIFY|_TIF_NOTIFY_RESUME)
+	(_TIF_SIGPENDING|_TIF_MCE_NOTIFY|_TIF_PERF_COUNTERS|_TIF_NOTIFY_RESUME)
 
 /* flags to check in __switch_to() */
 #define _TIF_WORK_CTXSW							\

index f2bba78..7e47658 100644
@@ -338,6 +338,7 @@
 #define __NR_dup3		330
 #define __NR_pipe2		331
 #define __NR_inotify_init1	332
+#define __NR_perf_counter_open	333
 
 #ifdef __KERNEL__
 

index d2e415e..53025fe 100644
@@ -653,7 +653,8 @@ __SYSCALL(__NR_dup3, sys_dup3)
 __SYSCALL(__NR_pipe2, sys_pipe2)
 #define __NR_inotify_init1			294
 __SYSCALL(__NR_inotify_init1, sys_inotify_init1)
-
+#define __NR_perf_counter_open		295
+__SYSCALL(__NR_perf_counter_open, sys_perf_counter_open)
 
 #ifndef __NO_STUBS
 #define __ARCH_WANT_OLD_READDIR

index 16f9487..8ab8c18 100644
@@ -31,6 +31,7 @@
 #include <linux/dmi.h>
 #include <linux/dmar.h>
 
+#include <asm/intel_arch_perfmon.h>
 #include <asm/atomic.h>
 #include <asm/smp.h>
 #include <asm/mtrr.h>

@@ -1147,6 +1148,7 @@ void __cpuinit setup_local_APIC(void)
 		apic_write(APIC_ESR, 0);
 	}
 #endif
+	perf_counters_lapic_init(0);
 
 	preempt_disable();
 

index 82ec607..89e5336 100644
@@ -1,5 +1,5 @@
 #
-# Makefile for x86-compatible CPU details and quirks
+# Makefile for x86-compatible CPU details, features and quirks
 #
 
 obj-y			:= intel_cacheinfo.o addon_cpuid_features.o

@@ -16,11 +16,13 @@ obj-$(CONFIG_CPU_SUP_CENTAUR_64)	+= centaur_64.o
 obj-$(CONFIG_CPU_SUP_TRANSMETA_32)	+= transmeta.o
 obj-$(CONFIG_CPU_SUP_UMC_32)		+= umc.o
 
-obj-$(CONFIG_X86_MCE)	+= mcheck/
-obj-$(CONFIG_MTRR)	+= mtrr/
-obj-$(CONFIG_CPU_FREQ)	+= cpufreq/
+obj-$(CONFIG_PERF_COUNTERS)		+= perf_counter.o
 
-obj-$(CONFIG_X86_LOCAL_APIC) += perfctr-watchdog.o
+obj-$(CONFIG_X86_MCE)			+= mcheck/
+obj-$(CONFIG_MTRR)			+= mtrr/
+obj-$(CONFIG_CPU_FREQ)			+= cpufreq/
+
+obj-$(CONFIG_X86_LOCAL_APIC)		+= perfctr-watchdog.o
 
 quiet_cmd_mkcapflags = MKCAP   $@
       cmd_mkcapflags = $(PERL) $(srctree)/$(src)/mkcapflags.pl $< $@

index b9c9ea0..4461011 100644
@@ -17,6 +17,7 @@
 #include <asm/mmu_context.h>
 #include <asm/mtrr.h>
 #include <asm/mce.h>
+#include <asm/intel_arch_perfmon.h>
 #include <asm/pat.h>
 #include <asm/asm.h>
 #include <asm/numa.h>

@@ -750,6 +751,7 @@ void __init identify_boot_cpu(void)
 #else
 	vgetcpu_set_mode();
 #endif
+	init_hw_perf_counters();
 }
 
 void __cpuinit identify_secondary_cpu(struct cpuinfo_x86 *c)

index b86f332..ad70f59 100644
@@ -869,6 +869,12 @@ END(error_interrupt)
 ENTRY(spurious_interrupt)
 	apicinterrupt SPURIOUS_APIC_VECTOR,smp_spurious_interrupt
 END(spurious_interrupt)
+
+#ifdef CONFIG_PERF_COUNTERS
+ENTRY(perf_counter_interrupt)
+	apicinterrupt LOCAL_PERF_VECTOR,smp_perf_counter_interrupt
+END(perf_counter_interrupt)
+#endif
 				
 /*
  * Exception entry points.

index d1d4dc5..d92bc71 100644
@@ -56,6 +56,10 @@ static int show_other_interrupts(struct seq_file *p)
 	for_each_online_cpu(j)
 		seq_printf(p, "%10u ", irq_stats(j)->apic_timer_irqs);
 	seq_printf(p, "  Local timer interrupts\n");
+	seq_printf(p, "CNT: ");
+	for_each_online_cpu(j)
+		seq_printf(p, "%10u ", irq_stats(j)->apic_perf_irqs);
+	seq_printf(p, "  Performance counter interrupts\n");
 #endif
 #ifdef CONFIG_SMP
 	seq_printf(p, "RES: ");

@@ -160,6 +164,7 @@ u64 arch_irq_stat_cpu(unsigned int cpu)
 
 #ifdef CONFIG_X86_LOCAL_APIC
 	sum += irq_stats(cpu)->apic_timer_irqs;
+	sum += irq_stats(cpu)->apic_perf_irqs;
 #endif
 #ifdef CONFIG_SMP
 	sum += irq_stats(cpu)->irq_resched_count;

index 845aa98..de2bb7c 100644
@@ -160,6 +160,9 @@ void __init native_init_IRQ(void)
 	/* IPI vectors for APIC spurious and error interrupts */
 	alloc_intr_gate(SPURIOUS_APIC_VECTOR, spurious_interrupt);
 	alloc_intr_gate(ERROR_APIC_VECTOR, error_interrupt);
+# ifdef CONFIG_PERF_COUNTERS
+	alloc_intr_gate(LOCAL_PERF_VECTOR, perf_counter_interrupt);
+# endif
 #endif
 
 #if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_X86_MCE_P4THERMAL)

index ff02353..eb04dd9 100644
@@ -204,6 +204,11 @@ static void __init apic_intr_init(void)
 	/* IPI vectors for APIC spurious and error interrupts */
 	alloc_intr_gate(SPURIOUS_APIC_VECTOR, spurious_interrupt);
 	alloc_intr_gate(ERROR_APIC_VECTOR, error_interrupt);
+
+	/* Performance monitoring interrupt: */
+#ifdef CONFIG_PERF_COUNTERS
+	alloc_intr_gate(LOCAL_PERF_VECTOR, perf_counter_interrupt);
+#endif
 }
 
 void __init native_init_IRQ(void)

index d6dd057..6d39c27 100644
@@ -6,7 +6,9 @@
  */
 #include <linux/list.h>
 
+#include <linux/perf_counter.h>
 #include <linux/personality.h>
+#include <linux/tracehook.h>
 #include <linux/binfmts.h>
 #include <linux/suspend.h>
 #include <linux/kernel.h>

@@ -17,7 +19,6 @@
 #include <linux/errno.h>
 #include <linux/sched.h>
 #include <linux/wait.h>
-#include <linux/tracehook.h>
 #include <linux/elf.h>
 #include <linux/smp.h>
 #include <linux/mm.h>

@@ -694,6 +695,11 @@ do_notify_resume(struct pt_regs *regs, void *unused, __u32 thread_info_flags)
 		tracehook_notify_resume(regs);
 	}
 
+	if (thread_info_flags & _TIF_PERF_COUNTERS) {
+		clear_thread_flag(TIF_PERF_COUNTERS);
+		perf_counter_notify(regs);
+	}
+
 #ifdef CONFIG_X86_32
 	clear_thread_flag(TIF_IRET);
 #endif /* CONFIG_X86_32 */

index a5c9627..066a13f 100644
@@ -7,6 +7,7 @@
  *  2000-2002   x86-64 support by Andi Kleen
  */
 
+#include <linux/perf_counter.h>
 #include <linux/sched.h>
 #include <linux/mm.h>
 #include <linux/smp.h>

@@ -493,6 +494,10 @@ do_notify_resume(struct pt_regs *regs, void *unused, __u32 thread_info_flags)
 		clear_thread_flag(TIF_NOTIFY_RESUME);
 		tracehook_notify_resume(regs);
 	}
+	if (thread_info_flags & _TIF_PERF_COUNTERS) {
+		clear_thread_flag(TIF_PERF_COUNTERS);
+		perf_counter_notify(regs);
+	}
 
 #ifdef CONFIG_X86_32
 	clear_thread_flag(TIF_IRET);

index d44395f..496726d 100644
@@ -332,3 +332,4 @@ ENTRY(sys_call_table)
 	.long sys_dup3			/* 330 */
 	.long sys_pipe2
 	.long sys_inotify_init1
+	.long sys_perf_counter_open

index ce0d9da..52146c2 100644
@@ -25,6 +25,7 @@
 #include <linux/kbd_kern.h>
 #include <linux/proc_fs.h>
 #include <linux/quotaops.h>
+#include <linux/perf_counter.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/suspend.h>

@@ -244,6 +245,7 @@ static void sysrq_handle_showregs(int key, struct tty_struct *tty)
 	struct pt_regs *regs = get_irq_regs();
 	if (regs)
 		show_regs(regs);
+	perf_counter_print_debug();
 }
 static struct sysrq_key_op sysrq_showregs_op = {
 	.handler	= sysrq_handle_showregs,

index 55e30d1..4c53027 100644
@@ -71,6 +71,7 @@ struct sched_param {
 #include <linux/fs_struct.h>
 #include <linux/compiler.h>
 #include <linux/completion.h>
+#include <linux/perf_counter.h>
 #include <linux/pid.h>
 #include <linux/percpu.h>
 #include <linux/topology.h>

@@ -1326,6 +1327,7 @@ struct task_struct {
 	struct list_head pi_state_list;
 	struct futex_pi_state *pi_state_cache;
 #endif
+	struct perf_counter_context perf_counter_ctx;
 #ifdef CONFIG_NUMA
 	struct mempolicy *mempolicy;
 	short il_next;

@@ -2285,6 +2287,13 @@ static inline void inc_syscw(struct task_struct *tsk)
 #define TASK_SIZE_OF(tsk)	TASK_SIZE
 #endif
 
+/*
+ * Call the function if the target task is executing on a CPU right now:
+ */
+extern void task_oncpu_function_call(struct task_struct *p,
+				     void (*func) (void *info), void *info);
+
+
 #ifdef CONFIG_MM_OWNER
 extern void mm_update_next_owner(struct mm_struct *mm);
 extern void mm_init_owner(struct mm_struct *mm, struct task_struct *p);

index 04fb47b..6cce728 100644
@@ -624,4 +624,10 @@ asmlinkage long sys_fallocate(int fd, int mode, loff_t offset, loff_t len);
 
 int kernel_execve(const char *filename, char *const argv[], char *const envp[]);
 
+asmlinkage int
+sys_perf_counter_open(u32 hw_event_type,
+		      u32 hw_event_period,
+		      u32 record_type,
+		      pid_t pid,
+		      int cpu);
 #endif

index f763762..78bede2 100644
@@ -732,6 +732,35 @@ config AIO
           by some high performance threaded applications. Disabling
           this option saves about 7k.
 
+config HAVE_PERF_COUNTERS
+	bool
+
+menu "Performance Counters"
+
+config PERF_COUNTERS
+	bool "Kernel Performance Counters"
+	depends on HAVE_PERF_COUNTERS
+	default y
+	help
+	  Enable kernel support for performance counter hardware.
+
+	  Performance counters are special hardware registers available
+	  on most modern CPUs. These registers count the number of certain
+	  types of hw events: such as instructions executed, cachemisses
+	  suffered, or branches mis-predicted - without slowing down the
+	  kernel or applications. These registers can also trigger interrupts
+	  when a threshold number of events have passed - and can thus be
+	  used to profile the code that runs on that CPU.
+
+	  The Linux Performance Counter subsystem provides an abstraction of
+	  these hardware capabilities, available via a system call. It
+	  provides per task and per CPU counters, and it provides event
+	  capabilities on top of those.
+
+	  Say Y if unsure.
+
+endmenu
+
 config VM_EVENT_COUNTERS
 	default y
 	bool "Enable VM event counters for /proc/vmstat" if EMBEDDED

index 19fad00..1f184a1 100644
@@ -89,6 +89,7 @@ obj-$(CONFIG_HAVE_GENERIC_DMA_COHERENT) += dma-coherent.o
 obj-$(CONFIG_FUNCTION_TRACER) += trace/
 obj-$(CONFIG_TRACING) += trace/
 obj-$(CONFIG_SMP) += sched_cpupri.o
+obj-$(CONFIG_PERF_COUNTERS) += perf_counter.o
 
 ifneq ($(CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER),y)
 # According to Alan Modra <[email protected]>, the -fno-omit-frame-pointer is

index 2a372a0..441fadf 100644
@@ -975,6 +975,7 @@ static struct task_struct *copy_process(unsigned long clone_flags,
 		goto fork_out;
 
 	rt_mutex_init_task(p);
+	perf_counter_init_task(p);
 
 #ifdef CONFIG_PROVE_LOCKING
 	DEBUG_LOCKS_WARN_ON(!p->hardirqs_enabled);

index b7480fb..254d56d 100644
@@ -2212,6 +2212,27 @@ static int sched_balance_self(int cpu, int flag)
 
 #endif /* CONFIG_SMP */
 
+/**
+ * task_oncpu_function_call - call a function on the cpu on which a task runs
+ * @p:		the task to evaluate
+ * @func:	the function to be called
+ * @info:	the function call argument
+ *
+ * Calls the function @func when the task is currently running. This might
+ * be on the current CPU, which just calls the function directly
+ */
+void task_oncpu_function_call(struct task_struct *p,
+			      void (*func) (void *info), void *info)
+{
+	int cpu;
+
+	preempt_disable();
+	cpu = task_cpu(p);
+	if (task_curr(p))
+		smp_call_function_single(cpu, func, info, 1);
+	preempt_enable();
+}
+
 /***
  * try_to_wake_up - wake up a thread
  * @p: the to-be-woken-up thread

@@ -2534,6 +2555,7 @@ prepare_task_switch(struct rq *rq, struct task_struct *prev,
 		    struct task_struct *next)
 {
 	fire_sched_out_preempt_notifiers(prev, next);
+	perf_counter_task_sched_out(prev, cpu_of(rq));
 	prepare_lock_switch(rq, next);
 	prepare_arch_switch(next);
 }

@@ -2574,6 +2596,7 @@ static void finish_task_switch(struct rq *rq, struct task_struct *prev)
 	 */
 	prev_state = prev->state;
 	finish_arch_switch(prev);
+	perf_counter_task_sched_in(current, cpu_of(rq));
 	finish_lock_switch(rq, prev);
 #ifdef CONFIG_SMP
 	if (current->sched_class->post_schedule)

@@ -4296,6 +4319,7 @@ void scheduler_tick(void)
 	rq->idle_at_tick = idle_cpu(cpu);
 	trigger_load_balance(rq, cpu);
 #endif
+	perf_counter_task_tick(curr, cpu);
 }
 
 #if defined(CONFIG_PREEMPT) && (defined(CONFIG_DEBUG_PREEMPT) || \

index e14a232..4be8bbc 100644
@@ -174,3 +174,6 @@ cond_syscall(compat_sys_timerfd_settime);
 cond_syscall(compat_sys_timerfd_gettime);
 cond_syscall(sys_eventfd);
 cond_syscall(sys_eventfd2);
+
+/* performance counters: */
+cond_syscall(sys_perf_counter_open);

I played with this a little, and while it works neat, I wanted a feature added where it shows a detailed profile for the top function. I've hacked this in quickly (the usability isn't all that great yet) and put the source code up at
http://www.tglx.de/~arjan/kerneltop-0.02.tar.gz

with this it looks like this:

$ sudo ./kerneltop --vmlinux=/home/arjan/linux-2.6.git/vmlinux

------------------------------------------------------------------------------
KernelTop: 274 irqs/sec [NMI, 1000000 cycles], (all, 2 CPUs)
------------------------------------------------------------------------------

             events         RIP          kernel function
             ______   ________________   _______________

                230 - 00000000c04189e9 : read_hpet
                 82 - 00000000c0409439 : mwait_idle_with_hints
                 77 - 00000000c051a7b7 : acpi_os_read_port
                 52 - 00000000c053cb3a : acpi_idle_enter_bm
                 38 - 00000000c0418d93 : hpet_next_event
                 19 - 00000000c051a802 : acpi_os_write_port
                 14 - 00000000c04f8704 : __copy_to_user_ll
                 13 - 00000000c0460c20 : get_page_from_freelist
                  7 - 00000000c041c96c : kunmap_atomic
                  5 - 00000000c06a30d2 : _spin_lock	[joydev]
                  4 - 00000000c04f79b7 : vsnprintf	[snd_seq]
                  4 - 00000000c06a3048 : _spin_lock_irqsave	[pcspkr]
                  3 - 00000000c0403b3c : irq_entries_start
                  3 - 00000000c0423fee : run_rebalance_domains
                  3 - 00000000c0425e2c : scheduler_tick
                  3 - 00000000c0430938 : get_next_timer_interrupt
                  3 - 00000000c043cdfa : __update_sched_clock
                  3 - 00000000c0448b14 : update_iter
                  2 - 00000000c04304bd : run_timer_softirq

Showing details for read_hpet

       0	c04189e9 <read_hpet>:
       2	c04189e9:	a1 b0 e0 89 c0       	mov    0xc089e0b0,%eax
       0	
       0	/*
       0	 * Clock source related code
       0	 */
       0	static cycle_t read_hpet(void)
       0	{
       1	c04189ee:	55                   	push   %ebp
       0	c04189ef:	89 e5                	mov    %esp,%ebp
       1	c04189f1:	05 f0 00 00 00       	add    $0xf0,%eax
       0	c04189f6:	8b 00                	mov    (%eax),%eax
       0		return (cycle_t)hpet_readl(HPET_COUNTER);
       0	}
     300	c04189f8:	31 d2                	xor    %edx,%edx
       0	c04189fa:	5d                   	pop    %ebp
       0	c04189fb:	c3                   	ret    
       0	

As is usual with profile outputs, the cost for the function always gets added to the instruction after the really guilty one. I'd move the count one back, but this is hard if the previous instruction was a (conditional) jump...

Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

ah, very nice idea!

ok, picked it up :-)

very nice and useful output! This for example shows that it's the readl() on the HPET_COUNTER IO address that is causing the overhead. That is to be expected - HPET is mapped uncached and the access goes out to the chipset.

yeah. Sometimes the delay can be multiple instructions - so it's best to leave the profiling picture as pristine as possible, and let the kernel developer chose the right counter type that displays particular problem areas in the most expressive way.

For example when i'm doing SMP scalability work, i generally look at cachemiss counts, for cacheline bouncing. The following kerneltop output shows last-level data-cache misses in the kernel during a tbench 64 run on a 16-way box, using latest mainline -git:

------------------------------------------------------------------------------
KernelTop: 3744 irqs/sec [NMI, 1000 cache-misses], (all, 16 CPUs)
------------------------------------------------------------------------------

             events         RIP          kernel function
             ______   ________________   _______________

               7757 - ffffffff804d723e : dst_release
               7649 - ffffffff804e3611 : eth_type_trans
               6402 - ffffffff8050e470 : tcp_established_options
               5975 - ffffffff804fa054 : ip_rcv_finish
               5530 - ffffffff80365fb0 : copy_user_generic_string!
               3979 - ffffffff804ccf0c : skb_push
               3474 - ffffffff804fe6cb : ip_queue_xmit
               1950 - ffffffff804cdcdd : skb_release_head_state
               1595 - ffffffff804cce4f : skb_copy_and_csum_dev
               1365 - ffffffff80501079 : __inet_lookup_established
                908 - ffffffff804fa5fc : ip_local_deliver_finish
                743 - ffffffff8036cbcc : unmap_single
                452 - ffffffff80569402 : _read_lock
                411 - ffffffff80283010 : get_page_from_freelist
                410 - ffffffff80505b16 : tcp_sendmsg
                406 - ffffffff8028631a : put_page
                386 - ffffffff80509067 : tcp_ack
                204 - ffffffff804d2d55 : netif_rx
                194 - ffffffff8050b94b : tcp_data_queue

Cachemiss event samples tend to line up quite close to the instruction that causes them.

Looking at pure cycles (same workload) gives a different view:

------------------------------------------------------------------------------
KernelTop: 27357 irqs/sec [NMI, 1000000 cycles], (all, 16 CPUs)
------------------------------------------------------------------------------

             events         RIP          kernel function
             ______   ________________   _______________

              16602 - ffffffff80365fb0 : copy_user_generic_string!
               7947 - ffffffff80505b16 : tcp_sendmsg
               7450 - ffffffff80509067 : tcp_ack
               7384 - ffffffff80332881 : avc_has_perm_noaudit
               6888 - ffffffff80504e7c : tcp_recvmsg
               6564 - ffffffff8056745e : schedule
               6170 - ffffffff8050ecd5 : tcp_transmit_skb
               4949 - ffffffff8020a75b : __switch_to
               4417 - ffffffff8050cc4f : tcp_rcv_established
               4283 - ffffffff804d723e : dst_release
               3842 - ffffffff804fed58 : ip_finish_output
               3760 - ffffffff804fe6cb : ip_queue_xmit
               3580 - ffffffff80501079 : __inet_lookup_established
               3540 - ffffffff80514ce5 : tcp_v4_rcv
               3475 - ffffffff8026c31f : audit_syscall_exit
               3411 - ffffffffff600130 : vread_hpet
               3267 - ffffffff802a73de : kfree
               3058 - ffffffff804d39ed : dev_queue_xmit
               3047 - ffffffff804eecf8 : nf_iterate

Cycles overhead tends to be harder to match up with instructions.

Ingo

Ingo,

I try to use patch V5 and the latest kerneltop to collect some cachemiss data.

It seems kerneltop just shows the first instruction ip address of functions. Does the latest kerneltop include the enhancement from Arjan? As you know, with oprofile, we can get detailed instrument ip address which causes the cache miss although the ip address should go back one instruction mostly.

Just like above detailed information.

Thanks,
Yanmin

As a matter of fact, the original kerneltop has parameter -s to support it. But kerneltop has a bug to show details of the symbol. sym_filter_entry should be initiated after qsort.

below is an example.
#./kerneltop --vmlinux=/root/linux-2.6.28_slqb1230flush/vmlinux -d 20 -e 3 -f 1000 -s flush_free_list

------------------------------------------------------------------------------
KernelTop: 20297 irqs/sec [NMI, 10000 cache-misses], (all, 8 CPUs)
------------------------------------------------------------------------------

events RIP kernel function
______ ______ ________________ _______________

           12816.00 - ffffffff803d5760 : copy_user_generic_string!
           11751.00 - ffffffff80647a2c : unix_stream_recvmsg
           10215.00 - ffffffff805eda5f : sock_alloc_send_skb
            9738.00 - ffffffff80284821 : flush_free_list
            6749.00 - ffffffff802854a1 : __kmalloc_track_caller
            3663.00 - ffffffff805f09fa : skb_dequeue
            3591.00 - ffffffff80284be2 : kmem_cache_alloc       [qla2xxx]
            3501.00 - ffffffff805f15f5 : __alloc_skb
            1296.00 - ffffffff803d8eb4 : list_del       [qla2xxx]
            1110.00 - ffffffff805f0ed2 : kfree_skb
Showing details for flush_free_list
       0        ffffffff8028488a:       78 00 00 
       0        ffffffff8028488d:       49 8d 04 00             lea    (%r8,%rax,1),%rax
       0        ffffffff80284891:       4c 8b 31                mov    (%rcx),%r14
    1143        ffffffff80284894:       48 c1 e8 0c             shr    $0xc,%rax
       0        ffffffff80284898:       48 6b c0 38             imul   $0x38,%rax,%rax
       0        ffffffff8028489c:       48 8d 1c 10             lea    (%rax,%rdx,1),%rbx
       0        ffffffff802848a0:       48 8b 03                mov    (%rbx),%rax
    3195        ffffffff802848a3:       25 00 40 00 00          and    $0x4000,%eax

The disassembly of lots of functions are big, so the new kernertop truncates them by filtering some instructions whose count are smaller than count_filter. It just shows the instructions whose count are more than count_filter and 3 instructions ahead of the reported instructions. For example, before printing
3195 ffffffff802848a3: 25 00 40 00 00 and $0x4000,%eax
the new kerneltop prints other 3 instructions:
0 ffffffff80284898: 48 6b c0 38 imul $0x38,%rax,%rax

0        ffffffff8028489c:       48 8d 1c 10             lea    (%rax,%rdx,1),%rbx
0        ffffffff802848a0:       48 8b 03                mov    (%rbx),%rax

So users can go back quickly to find the instruction who really causes the event (not the reported instruction by the performance counter).

Below is the patch against kernetop of Dec/23/2008 version.

yanmin

---

@@ -3,7 +3,7 @@
 
    Build with:
 
-     cc -O6 -Wall `pkg-config --cflags glib-2.0` -c -o kerneltop.o kerneltop.c
+     cc -O6 -Wall `pkg-config --cflags --libs glib-2.0` -o kerneltop kerneltop.c
 
    Sample output:
 

@@ -291,8 +291,6 @@ static void process_options(int argc, ch
 		else
 			event_count[counter] = 100000;
 	}
-	if (nr_counters == 1)
-		count_filter = 0;
 }
 
 static uint64_t			min_ip;

@@ -307,7 +305,7 @@ struct sym_entry {
 
 #define MAX_SYMS		100000
 
-static unsigned int sym_table_count;
+static int sym_table_count;
 
 struct sym_entry		*sym_filter_entry;
 

@@ -350,7 +348,7 @@ static struct sym_entry		tmp[MAX_SYMS];
 
 static void print_sym_table(void)
 {
-	unsigned int i, printed;
+	int i, printed;
 	int counter;
 
 	memcpy(tmp, sym_table, sizeof(sym_table[0])*sym_table_count);

@@ -494,7 +492,6 @@ static int read_symbol(FILE *in, struct 
 			printf("symbol filter start: %016lx\n", filter_start);
 			printf("                end: %016lx\n", filter_end);
 			filter_end = filter_start = 0;
-			sym_filter_entry = NULL;
 			sym_filter = NULL;
 			sleep(1);
 		}

@@ -502,7 +499,6 @@ static int read_symbol(FILE *in, struct 
 	if (filter_match == 0 && sym_filter && !strcmp(s->sym, sym_filter)) {
 		filter_match = 1;
 		filter_start = s->addr;
-		sym_filter_entry = s;
 	}
 
 	return 0;

@@ -538,6 +534,16 @@ static void parse_symbols(void)
 	last->sym = "<end>";
 
 	qsort(sym_table, sym_table_count, sizeof(sym_table[0]), compare_addr);
+
+	if (filter_end) {
+		int count;
+		for (count=0; count < sym_table_count; count ++) {
+			if (!strcmp(sym_table[count].sym, sym_filter)) {
+				sym_filter_entry = &sym_table[count];
+				break;
+			}
+		}
+	}
 }
 
 

@@ -617,11 +623,27 @@ static void lookup_sym_in_vmlinux(struct
 	}
 }
 
+void show_lines(GList *item_queue, int item_queue_count)
+{
+	int i;
+	struct source_line *line;
+
+	for (i = 0; i < item_queue_count; i++) {
+		line = item_queue->data;
+		printf("%8li\t%s\n", line->count, line->line);
+		item_queue = g_list_next(item_queue);
+	}
+}
+
+#define TRACE_COUNT     3
+
 static void show_details(struct sym_entry *sym)
 {
 	struct source_line *line;
 	GList *item;
 	int displayed = 0;
+	GList *item_queue;
+	int item_queue_count = 0;
 
 	if (!sym->source)
 		lookup_sym_in_vmlinux(sym);

@@ -633,16 +655,28 @@ static void show_details(struct sym_entr
 	item = sym->source;
 	while (item) {
 		line = item->data;
-		item = g_list_next(item);
 		if (displayed && strstr(line->line, ">:"))
 			break;
 
-		printf("%8li\t%s\n", line->count, line->line);
+		if (!item_queue_count)
+			item_queue = item;
+		item_queue_count ++;
+
+		if (line->count >= count_filter) {
+			show_lines(item_queue, item_queue_count);
+			item_queue_count = 0;
+			item_queue = NULL;
+		} else if (item_queue_count > TRACE_COUNT) {
+			item_queue = g_list_next(item_queue);
+			item_queue_count --;
+		}
+
+		line->count = 0;
 		displayed++;
 		if (displayed > 300)
 			break;
+		item = g_list_next(item);
 	}
-	exit(0);
 }
 
 /*

Still not good enough, I'm sorry.

* I have no guarantee that the secondary counters were all counting at the same time(s) as the master counter, so the numbers are virtually useless.

* I might legitimately want to be notified based on any of the "secondary" counters reaching particular values. The "master" vs. "secondary" distinction is an artificial one that is going to make certain reasonable use-cases impossible.

These things are both symptoms of the fact that you still have the abstraction at the wrong level. The basic abstraction really needs to be a counter-set, not an individual counter.

I think your patch can be extended to do counter-sets without complicating the interface too much. We could have:

struct event_spec {
	u32 hw_event_type;
	u32 hw_event_period;
	u64 hw_raw_ctrl;
};

int perf_counterset_open(u32 n_counters,
struct event_spec *counters, u32 record_type, pid_t pid,
int cpu);

and then you could have perf_counter_open as a simple wrapper around perf_counterset_open.

With an approach like this we can also provide an "exclusive" mode for the PMU (e.g. with a flag bit in record_type or n_counters), which means that the counter-set occupies the whole PMU. That will give a way for userspace to specify all the details of how the PMU is to be programmed, which in turn means that the kernel doesn't need to know all the arcane details of every event on every processor; it just needs to know the common events.

I notice the implementation also still assumes it can add any counter at any time subject only to a limit on the number of counters in use. That will have to be fixed before it is usable on powerpc (and apparently on some x86 processors too).

Paul.

If you want a _guarantee_ that multiple counters can count at once you can still do it: for example by using the separate, orthogonal reservation mechanism we had in -v1 already.

Also, you dont _have to_ overcommit counters.

Your whole statistical argument that group readout is a must-have for precision is fundamentally flawed as well: counters _themselves_, as used by most applications, by their nature, are a statistical sample to begin with. There's way too many hardware events to track each of them unintrusively - so this type of instrumentation is _all_ sampling based, and fundamentally so. (with a few narrow exceptions such as single-event interrupts for certain rare event types)

This means that the only correct technical/mathematical argument is to talk about "levels of noise" and how they compare and correlate - and i've seen no actual measurements or estimations pro or contra. Group readout of counters can reduce noise for sure, but it is wrong for you to try to turn this into some sort of all-or-nothing property. Other sources of noise tend to be of much higher of magnitude.

You need really stable workloads to see such low noise levels that group readout of counters starts to matter - and the thing is that often such 'stable' workloads are rather boringly artificial, because in real life there's no such thing as a stable workload.

Finally, the basic API to user-space is not the way to impose rigid "I own the whole PMU" notion that you are pushing. That notion can be achieved in different, system administration means - and a perf-counter reservation facility was included in the v1 patchset.

Note that you are doing something that is a kernel design no-no: you are trying to design a "guarantee" for hardware constraints by complicating it into the userpace ABI - and that is a fundamentally losing proposition.

It's a tail-wags-the-dog design situation that we are routinely resisting in the upstream kernel: you are putting hardware constraints ahead of usability, you are putting hardware constraints ahead of sane interface design - and such an approach is wrong and shortsighted on every level.

It's also shortsighted because it's a red herring: there's nothing that forbids the counter scheduler from listening to the hw constraints, for CPUs where there's a lot of counter constraints.

the secondary counters can cause records too - independently of the master counter. This is because the objects (and fds) are separate so there's no restriction at all on the secondary counters. This is a lot less natural to do if you have a "vector of counters" abstraction.

Being per object is a very fundamental property of Linux, and you have to understand and respect that down to your bone if you want to design new syscall ABIs for Linux.

The "perfmon v3 light" system calls, all five of them, are a classic loundry list of what _not_ to do in new Linux APIs: they are too specific, too complex and way too limited on every level.

Per object and per fd abstractions are a _very strong_ conceptual property of Linux. Look at what they bring in the performance counters case:

- All the VFS syscalls work naturally: sys_read(), sys_close(), sys_dup(), you name it.

- It makes all counters poll()able. Any subset of them, and at any time, independently of any context descriptor. Look at kerneltop.c: it has a USE_POLLING switch to switch to a poll() loop, and it just works the way you'd expect it to work.

- We can share fds between monitor threads and you can do a thread pool that works down new events - without forcing any counter scheduling in the monitored task.

- It makes the same task monitorable by multiple monitors, trivially so. There's no forced context notion that privatizes the PMU - with some 'virtual context' extra dimension slapped on top of it.

- Doing a proper per object abstraction simplifies event and error handling significantly: instead of having to work down a vector of counters and demultiplexing events and matching them up to individual counters, the demultiplexing is done by the _kernel_.

- It makes counter scheduling very dynamic. Instead of exposing user-space to a static "counter allocation" (with all the insane ABI and kernel internal complications this brings), perf-counters subsystem does not expose user-space to such scheduling details _at all_.

- Difference in complexity. The "v3 light" version of perfmon (which does not even schedule any PMU contexts), contains these core kernel files:

         19 files changed, 4424 insertions(+)

Our code has this core kernel impact:

         10 files changed, 1191 insertions(+)

And in some areas it's already more capable than "perfmon v3". The difference is very obvious.

All in one, using the 1:1 fd:counter design is a powerful, modern Linux abstraction to its core. It's much easier to think about for application developers as well, so we'll see a much sharper adoption rate.

Also, i noticed that your claims about our code tend to be rather abstract and are often dwelling on issues that IMO have no big practical relevance - so may i suggest the following approach instead to break the (mutual!) cycle of miscommunication: if you think an issue is important, could you please point out the problem in practical terms what you think would not be possible with our scheme? We tend to prioritize items by practical value.

Things like: "kerneltop would not be as accurate with: ..., to the level of adding 5% of extra noise.". Would that work for you?

This needless vectoring and the exposing of contexts would kill many good properties of the new subsystem, without any tangible benefits - see above.

This is really scheduling school 101: a hardware context allocation is the _last_ thing we want to expose to user-space in this particular case. This is a fundamental property of hardware resource scheduling. We _dont_ want to tie the hands of the kernel by putting resource scheduling into user-space!

Your arguments remind me a bit of the "user-space threads have to be scheduled in user-space!" N:M threading design discussions we had years ago. IBM folks were pushing NGPT very strongly back then and claimed that it's the right design for high-performance threading, etc. etc.

In reality, doing user-space scheduling for cheap-to-context-switch hardware resources was a fundamentally wrong proposition back then too, and it is still the wrong concept today as well.

You can already allocate "exclusive" counters in a guaranteed way via our code, here and today.

There's constrained PMCs on x86 too, as you mention. Instead of repeating the answer that i gave before (that this is easy and natural), how about this approach: if we added real, working support for constrained PMCs on x86, that will then address this point of yours rather forcefully, correct?

Ingo

From: Ingo Molnar <[email protected]>
Date: Mon, 8 Dec 2008 12:33:18 +0100

There are a lot of people who are going to fundamentally disagree with this, myself included.

A lot of things are being stated about what people do with this stuff, but I think there are people working longer in this area who quite possibly know a lot better. But they were blindsided by this new work instead of being consulted, which was pretty unnice.

Ingo, could you please describe how PEBS and IBS fit into your model?

Thanks.

-Andi

[email protected]

Is that this?

" - There's a /sys based reservation facility that allows the allocation
of a certain number of hw counters for guaranteed sysadmin access."

Sounds like I can't do that as an ordinary user, even on my own processes...

I don't want the whole PMU all the time, I just want it while my monitored process is running, and only on the CPU where it is running.

No - at least on the machines I'm familiar with, I can count every single cache miss and hit at every level of the memory hierarchy, every single TLB miss, every load and store instruction, etc. etc.

I want to be able to work out things like cache hit rates, just as one example. To do that I need two numbers that are directly comparable because they relate to the same set of instructions. If I have a count of L1 Dcache hits for one set of instructions and a count of L1 Dcache misses over some different stretch of instructions, the ratio of them doesn't mean anything.

Your argument about "it's all statistical" is bogus because even if the things we are measuring are statistical, that's still no excuse for being sloppy about how we make our estimates. And not being able to have synchronized counters is just sloppy. The users want it, the hardware provides it, so that makes it a must-have as far as I am concerned.

What can you back that assertion up with?

More unsupported assertions, that sound wrong to me...

Only for root, which isn't good enough.

What I was proposing was NOT a rigid notion - you don't have to own the whole PMU if you are happy to use the events that the kernel knows about. If you do want the whole PMU, you can have it while the process you're monitoring is running, and the kernel will context-switch it between you and other users, who can also have the whole PMU when their processes are running.

Perhaps you have misunderstood my proposal. A counter-set doesn't have to be the whole PMU, and you can have multiple counter-sets active at the same time as long as they fit. You can even have multiple "whole PMU" counter-sets and the kernel will multiplex them onto the real PMU.

Well, I'll ignore the patronizing tone (but please try to avoid it in future).

The PRIMARY reason for wanting counter-sets is because THAT IS WHAT THE USERS WANT. A "usable" and "sane" interface design that doesn't do what users want is useless.

Anyway, my proposal is just as "usable" as yours, since users still have perf_counter_open, exactly as in your proposal. Users with simpler requirements can do things exactly the same way as with your proposal.

Handling the counter constraints is indeed a matter of implementation, and as I noted previously, your current proposed implementation doesn't handle them.

It's the choice of a single counter as being your "object" that I object to. :)

Which is not necessarily a good thing. Fundamentally, if you are trying to measure something, and you get a number, you need to know what exactly got measured.

For example, suppose I am trying to count TLB misses during the execution of a program. If my TLB miss counter keeps getting bumped off because the kernel is scheduling my counter along with a dozen other counters, then I *at least* want to know about it, and preferably control it. Otherwise I'll be getting results that vary by an order of magnitude with no way to tell why.

For simple things, yes it is simpler. But it can't do the more complex things in any sort of clean or sane way.

... because the design of your code is wrong at an abstract level ...

OK, here's an example. I have an application whose execution has several different phases, and I want to measure the L1 Icache hit rate and the L1 Dcache hit rate as a function of time and make a graph. So I need counters for L1 Icache accesses, L1 Icache misses, L1 Dcache accesses, and L1 Dcache misses. I want to sample at 1ms intervals. The CPU I'm running on has two counters.

With your current proposal, I don't see any way to make sure that the counter scheduler counts L1 Dcache accesses and L1 Dcache misses at the same time, then schedules L1 Icache accesses and L1 Icache misses. I could end up with L1 Dcache accesses and L1 Icache accesses, then L1 Dcache misses and L1 Icache misses - and get a nonsensical situation like the misses being greater than the accesses.

No. Where did you get contexts from? I didn't write anything about contexts. Please read what I wrote.

Please drop the patronizing tone, again.

What user-space applications want to be able to do is this:

* Ensure that a set of counters are all counting at the same time.

* Know when counters get scheduled on and off the process so that the results can be interpreted properly. Either that or be able to control the scheduling.

* Sophisticated applications want to be able to do things with the PMU that the kernel doesn't necessarily understand.

You'd rather provide useless numbers to userspace? :)

Your arguments remind me of a filesystem that a colleague of mine once designed that only had files, but no directories (you could have "/" characters in the filenames, though). This whole discussion is a bit like you arguing that directories are an unnecessary complication that only messes up the interface and adds extra system calls.

But then I don't get context-switching between processes.

It still means we end up having to add something approaching 29,000 lines of code and 320kB to the kernel, just for the IBM 64-bit PowerPC processors. (I don't guarantee that code is optimal, but that is some indication of the complexity required.)

I am perfectly happy to add code for the kernel to know about the most commonly-used, simple events on those processors. But I surely don't want to have to teach the kernel about every last event and every last capability of those machines' PMUs.

For example, there is a facility on POWER6 where certain instructions can be selected (based on (instruction_word & mask) == value) and marked, and then there are events that allow you to measure how long marked instructions take in various stages of execution. How would I make such a feature available for applications to use, within your framework?

Paul.

yes, agreed, this is a valid special case of simple counter readout - we'll add support to couple counters like that.

Note that this issue does not impact use of multiple counters in profilers. (i.e. anything that is not a pure readout of the counter, along linear time, as your example above suggests).

Once we start sampling the context, grouping of counters becomes irrelevant (and a hindrance) and static frequency sampling becomes an inferior method of sampling.

( The highest quality statistical approach is the kind of multi-counter
sampling model you can see implemented in KernelTop for example, where the counters are independently sampled. Can go on in great detail about this if you are interested - this is the far more interesting usecase in practice. )

Ingo

This is an example of a sampling problem, but one where the thing being sampled is a derived statistic from two counter values.

I don't agree that this is really a "special case". There are lots of derived statistics that are interesting for performance analysis, starting with CPI (cycles per instruction), proportions of various instructions in the code, cache hit/miss rates for various different caches, etc., etc.

Well, that's the sampling vs. counting distinction that I made in my other email. We need to do both well.

As far as I can see, my "counter set" proposal does everything yours does (since a counter set can be just a single counter), and also cleanly accommodates what's needed for counting and for sampling derived statistics. No?

Paul.

[email protected] wrote on 12/07/2008 05:22:12 PM:
[snip]

events

events

events

events

events

events

Hello Ingo,

I posted some questions about this capability in your proposal on LKML, but I wasn't able to get the reply threaded in properly. Could you take a look at this post, please?

http://lkml.org/lkml/2008/12/5/299

Thanks for your consideration,

- Corey

Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]

Hi,

That is unfortunately not generic enough.You need a bit more flexibility than master/secondaries, I am afraid. What tools want is to be able to express:
- when event X overflows, record values of events J, K
- when event Y overflows, record values of events Z, J

I am not making this up. I know tools that do just that, i.e., that is collecting
two distinct profiles in a single run. This is how, for instance, you can collect
a flat profile and the call graph in one run, very much like gprof.

When a you get a notification and you read out the sample, you'd like to know if which order values are returned. Given you do not expose counters, I would assume the only possibility would be return them in file descriptor order.
But that assumes at the time you create the file descriptor for an event you have
all the other file descriptors you need...

Core/Atom have 5 counters, Nehalem has 7.
Why are you not using all of them already?

no, Nehalem has 4 generic purpose PMCs and 3 fixed-purpose PMCs (7 total), Core/Atom has 2 generic PMCs and 3 fixed-purpose PMCs (5 total). Saying that it has 7 is misleading. (and even the generic PMCs have constraints)

Ingo

From: Ingo Molnar <[email protected]>
Date: Tue, 9 Dec 2008 12:02:46 +0100

...

Even you just did.

which portion of my point stressing the general purpose attribute was unclear to you?

Saying it has 7 is misleading in the same way as if i told you now:
"look, i have four eyes!". (they are: left eye looking right, left eye looking left, right eye looking left and right eye looking right)

Nehalem has 4 general purpose PMCs, not 7. Yes, it has 7 counters but they are not all general purpose. The P4 has 18.

Ingo

From: Ingo Molnar <[email protected]>
Date: Tue, 9 Dec 2008 12:22:25 +0100

I'm just teasing you because you picked a trite point from stephane's email instead of the meat later on, which I would have found more interesting to hear you comment on.

I'm interested in Ingo's comments on that argument as well but I don't feel the need to act like were are all in a kindergarten.

You are two astonishing developers and I'm sure you can demonstrate that you can have a pure technical discussion on this topic as you did several times in the past.

Regards,

Paolo
http://paolo.ciarrocchi.googlepages.com/
http://mypage.vodafone.it/

hm, the new group code in perfcounters-v2 can already do this. Have you tried to use it and it didnt work? If so then that's a bug. Nothing in the design prevents that kind of group readout.

[ We could (and probably will) enhance the grouping relationship some
more, but group readouts are a fundamentally inferior mode of profiling. (see below for the explanation) ]

yeah, but it's still the fundamentally wrong thing to do.

Being able to extract high-quality performance information from the system is the cornerstone of our design, and chosing the right sampling model permeates the whole issue of single-counter versus group-readout.

I dont think finer design aspects of kernel support for performance counters can be argued without being on the same page about this, so please let me outline our view on these things, in (boringly) verbose detail - spiked with examples and code as well.

Firstly, sampling "at 1msec intervals" or any fixed period is a _very_ wrong mindset - and cross-sampling counters is a similarly wrong mindset.

When there are two (or more) hw metrics to profile, the ideally best (i.e. the statistically most stable and most relevant) sampling for the two statistical variables (say of l2_misses versus l2_accesses) is to sample them independently, via their own metric. Not via a static 1khz rate - or via picking one of the variables to generate samples.

[ Sidenote: as long as the hw supports such sort of independent sampling
- lets assume so for the sake of argument - not all CPUs are capable of that - most modern CPUs do though. ]

Static frequency [time] sampling has a number of disadvantages that drastically reduce its precision and reduce its utility, and 'group' sampling where one counter controls the events has similar problems:

- It under-samples rare events such as cachemisses.

An example: say we have a workload that executes 1 billion instructions a second, of which 5000 generate a cachemiss. Only one in 200,000 instructions generates a cachemiss. The chance for a static sampling IRQ to hit exactly an instruction that causes the cachemiss is 1:200 (0.5%) in every second. That is very low probability, and the profile would not be very helpful - even though it samples at a seemingly adequate frequency of 1000 events per second!

With per event counters and per event sampling that KernelTop uses, we get an event next to the instruction that causes a cachemiss with a 100% certainty, all the time. The profile and its per instruction aspects suddenly become a whole lot more accurate and whole lot more interesting.

- Static frequency and group sampling also runs the risk of systematic error/skew of sampling if any workload component has any correlation with the "1msec" global sampling period.

For example: say we profile a workload that runs a timer every 20 msecs. In such a case the profile could be skewed assymetrically against [or in favor of] that timer activity that it does every 10 milliseconds.

Good sampling wants the samples to be generated in proportion to the variable itself, not proportional to absolute time.

- Static sampling also over-samples when the workload activity goes down (when it goes more idle).

For example: we profile a fluctuating workload that is sometimes only 0.2% busy, i.e. running only for 2 milliseconds every second. Still we keep interrupting it at 1 khz - that can be a very brutal systematic skew if the sampling overhead is 2 microseconds, totalling to 2 msecs overhead every second - so 50% of what runs on the CPU will be sampling code - impacting/skewing the sampled code.

Good sampling wants to 'follow' the ebb and flow of the actual hw events that the CPU has.

The best way to sample two metrics such as "cache accesses" and "cache misses" (or say "cache misses" versus "TLB misses") is to sample the two variables _independently_, and to build independent histograms out of them.

The combination (or 'grouping') of the measured variables is thus done at the output stage _after_ data acquisition, to provide a weighted histogram (or a split-view double histogram).

For example, in a "l2 misses" versus "l2 accesses" case, the highest quality of sampling is to use two independent sampling IRQs with such sampling parameters:

- one notification every 200 L2 cache misses
- one notification every 10,000 L2 cache accesses

[ this is a ballpark figure - the sample rate is a function of the
averages of the workload and the characteristics of the CPU. ]

And at the output stage display a combination of:

l2_accesses[pc]
l2_misses[pc]
l2_misses[pc] / l2_accesseses[pc]

Note that if we had a third variable as well - say icache_misses[], we could combine the three metrics:

l2_misses[pc] / l2_accesses[pc] / icache_misses[pc]

( such a view expresses the miss/access ratio in a branch-weighted
fashion: it weighs down instructions that also show signs of icache pressure and goes for the functions with a high dcache rate but low icache pressure - i.e. commonly executed functions with a high data miss rate. )

Sampling at a static frequency is acceptable as well in some cases, and will lead to an output that is usable for some things. It's just not the best sampling model, and it's not usable at all for certain important things such as highly derived views, good instruction level profiles or rare hw events.

I've uploaded a new version of kerneltop.c that has such a multi-counter sampling model that follows this statistical model:

http://redhat.com/~mingo/perfcounters/kerneltop.c

Example of usage:

I've started a tbench 64 localhost workload on a 16way x86 box. I want to check the miss/refs ratio. I first did a sample one of the metrics, cache-references:

$ ./kerneltop -e 2 -c 100000 -C 2

------------------------------------------------------------------------------
KernelTop: 1311 irqs/sec [NMI, 10000 cache-refs], (all, cpu: 2)
------------------------------------------------------------------------------

             events         RIP          kernel function
             ______   ________________   _______________

            5717.00 - ffffffff803666c0 : copy_user_generic_string!
             355.00 - ffffffff80507646 : tcp_sendmsg
             315.00 - ffffffff8050abcb : tcp_ack
             222.00 - ffffffff804fbb20 : ip_rcv_finish
             215.00 - ffffffff8020a75b : __switch_to
             194.00 - ffffffff804d0b76 : skb_copy_datagram_iovec
             187.00 - ffffffff80502b5d : __inet_lookup_established
             183.00 - ffffffff8051083d : tcp_transmit_skb
             160.00 - ffffffff804e4fc9 : eth_type_trans
             156.00 - ffffffff8026ae31 : audit_syscall_exit

Then i checked the characteristics of the other metric [cache-misses]:

$ ./kerneltop -e 3 -c 200 -C 2

------------------------------------------------------------------------------
KernelTop: 1362 irqs/sec [NMI, 200 cache-misses], (all, cpu: 2)
------------------------------------------------------------------------------

             events         RIP          kernel function
             ______   ________________   _______________

            1419.00 - ffffffff803666c0 : copy_user_generic_string!
            1075.00 - ffffffff804e4fc9 : eth_type_trans
            1059.00 - ffffffff804d8baa : dst_release
             949.00 - ffffffff80510004 : tcp_established_options
             841.00 - ffffffff804fbb20 : ip_rcv_finish
             569.00 - ffffffff804ce808 : skb_push
             454.00 - ffffffff80502b5d : __inet_lookup_established
             453.00 - ffffffff805001a3 : ip_queue_xmit
             298.00 - ffffffff804cf5d8 : skb_release_head_state
             247.00 - ffffffff804ce74b : skb_copy_and_csum_dev

then, to get the "combination" view of the two counters, i appended the two command lines:

 $ ./kerneltop -e 3 -c 200 -e 2 -c 10000 -C 2

------------------------------------------------------------------------------
KernelTop: 2669 irqs/sec [NMI, cache-misses/cache-refs], (all, cpu: 2)
------------------------------------------------------------------------------

             weight         RIP          kernel function
             ______   ________________   _______________

              35.20 - ffffffff804ce74b : skb_copy_and_csum_dev
              33.00 - ffffffff804cb740 : sock_alloc_send_skb
              31.26 - ffffffff804ce808 : skb_push
              22.43 - ffffffff80510004 : tcp_established_options
              19.00 - ffffffff8027d250 : find_get_page
              15.76 - ffffffff804e4fc9 : eth_type_trans
              15.20 - ffffffff804d8baa : dst_release
              14.86 - ffffffff804cf5d8 : skb_release_head_state
              14.00 - ffffffff802217d5 : read_hpet
              12.00 - ffffffff804ffb7f : __ip_local_out
              11.97 - ffffffff804fc0c8 : ip_local_deliver_finish
               8.54 - ffffffff805001a3 : ip_queue_xmit

[ It's interesting to see that a seemingly common function,
copy_user_generic_string(), got eliminated from the top spots - because there are other functions whose relative cachemiss rate is far more serious. ]

The above "derived" profile output is relatively stable under kerneltop with the use of ~2600 sample irqs/sec and the 2 seconds default refresh. I'd encourage you to try to achieve the same quality of output with static 2600 hz sampling - it wont work with the kind of event rates i've worked with above, no matter whether you read out a single counter or a group of counters, atomically or not. (because we just dont get notification PCs at the relevant hw events - we get PCs with a time sample)

And that is just one 'rare' event type (cachemisses) - if we had two such sources (say l2 cachemisses and TLB misses) then such type of combined view would only be possible if we got independent events from both hardware events.

And note that once you accept that the highest quality approach is to sample the hw events independently, all the "group readout" approaches become a second-tier mechanism. KernelTop uses that model and works just fine without any group readout and it is making razor sharp profiles, down to the instruction level.

[ Note that there's special-cases where group-sampling can limp along
with acceptable results: if one of the two counters has so many events that sampling by time or sampling by the rare event type gives relevant context info. But the moment both event sources are rare, the group model breaks down completely and produces meaningless results. It's just a fundamentally wrong kind of abstraction to mix together unrelated statistical variables. And that's one of the fundamental design problems i see with perfmon-v3. ]

Ingo

Regardless of sampling method, don't you still want some way to enable/disable the various counters as close to simultaneously as possible?

Chris

If it's about counter control for the monitored task, then we sure could do something about that. (apps/libraries could thus select a subset of functions to profile/measure, runtime, etc.)

If it's about counter control for the profiler/debugger, i'm not sure how useful that is - do you have a good usecase for it?

Ingo

I'm sure that others could give more usecases, but I was thinking about cases like "I want to test _these_ multiple metrics simultaneously over _this_ specific section of code". In a case like this, it seems desirable to start/stop the various performance counters as close together as possible, especially if the section of code being tested is short.

Chris

If your hardware does not interrupt on overflow I don't think you have any choice in the matter. I know such hardware is less than ideal but it exists so it should be supported.

I think you could still set up the counters as Ingo describes and then sample the counters (as opposed to the program) at a suitable interval (chosen such that the counters won't overflow more than once between samples).

Chris

Hi,

That's not for you to say. This is decision for the tool writers.

There is absolutely nothing wrong with this. In fact, people do this kind of measurements all the time. Your horizon seems a bit too limited, maybe.

Certain PMU features do not count events, they capture information about where they occur, so they are more like buffers. Sometimes, they are hosted in registers. For instance, Itanium has long been able to capture where cache misses occur. The data is stored in a couple of PMU registers and only one cache miss at a time. There is a PMU event that counts how many misses are captured. So you program that event into a counter and when it overflows you want to read out the pair of data registers containing the last captured cache miss. Thus, when event X overflows, you capture values in registers Z, Y. There is nothing wrong with this. You do the same thing when you want to sample on a branch trace buffer, like X86 LBR. Again nothing wrong with this. In fact you can collect both at the same time and in independent manner.

Did I talk about static sampling period?

Who talked about periods expressed as events per second?

I did not talk about that. If you had looked at the perfmon API, you would have noticed that it does not know anything about sampling periods. It only sees register values. Tools are free to pick whatever value they like. And the value, by definition. is defined as the number of occurrences of the event, not the number of occurrences per seconds. You can say:
every 2000 cache miss, take a sample, just program that counter with -2000.

You have no guarantee on how close the RIP is compared to where the cache miss occurred. It can be several of instructions away (NMI or not by the way). There is nothing software can do about it, neither my inferior design nor your superior design.

And you think you cannot do independent sampling with perfmon3?

As for 'razor sharp', that is your interpretation. As far as I know a RIP is always
pointing to an instruction anyway. What you seem to be ignoring here is the fact that
the RIP is as good as the hardware can give you. And it just happens that on ALL processor architectures it is off compared to where the event actually occurred. It can be several cycles away actually: skid. Your superior design does not improve that precision whatsoever. It has to be handled at the hardware level. Why do you think AMD added IBS, why Intel added PEBS on X86 and why Intel added IP-EAR on Itanium2? Even PEBS is not solving that issue completely. As far I know the quality of your profiles are as good as Oprofile,
VTUNE, or perfmon.

Again an unfounded statement, perfmon3 does not mandate what is recorded on overflow. It does not mandate how many events you can sample on at the same time. It does not know about sampling periods, it only knows about data register values and reset values on overflow. For each counters, you can freely specify what you want recorded using a simple bitmask.

Are we on the same page, then?

------------------------------------------------------------------------------
SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada. The future of the web can't happen without you. Join us at MIX09 to help pave the way to the Next Web now. Learn more and register at
http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/

Thanks for taking the time to write all this down, and I will respond in detail once I have thought about it some more.

The thing that stands out to me immediately, though, is that you are concentrating entirely on _sampling_ as opposed to _counting_. Perhaps this is the main reason we have been disagreeing.

Now of course sampling is interesting, but counting is also interesting, whether over the whole execution of a program or over short intervals during the execution of a program (based either on time, or on the execution of certain functions).

It seems to me that a well-designed performance monitor infrastructure should support both counting and sampling. And for counting, getting high-quality data requires synchronized counters (ones that all start counting and stop counting at the same times).

Looking back at the discussion so far, I can see that your arguments make more sense if you are only wanting to do sampling. And I have been arguing for what I believe we need to do counting properly (I have focused more on counting because we already have infrastructure for sampling, namely oprofile).

So, can we agree to discuss both sampling and counting, and design an infrastructure that's good for both?

Paul.

When you say counting you should also include "event ring buffers with metadata", like PEBS on Intel x86.

-Andi

[email protected]

I'm not familiar with PEBS. Maybe it's something different again, neither sampling nor counting, but a third thing?

Paul.

Paul,

PEBS is an Intel only feature used for sampling. However, this time the hardware (and microcode) does the sampling for you. You point the CPU to a structure in memory, called DS, which then points to a region of memory you can designate, i.e., the sampling buffer. Buffer can be any size you want.

Then you program counter0 when an event and a sampling period. When the counter overflows, there is no interrupt, the microcode records the RIP and full machine state, and reloads the counter with the period specified in DS. The OS gets an interrupt ONLY when the buffer fills up. Overhead is thus minimized, but you have no control over the format of the samples. The precision (P) comes from the fact that the RIP is guaranteed to point the an instruction that is just after an instruction which generated the event you're sampling on. The catch is that no all events support PEBS, and only one counter works with PEBS on Core 2. Nehalem is better, more events support PEBS, all 4 generic counters do support PEBS. Furthermore,PEBS can now capture where cache misses occur, very much like what Itanium can do.

Needless to say all of this is supported by perfmon.

Hope this helps.

Yes it's a third thing. A CPU controlled ring event buffer. See Stephane's description.

There's also some crosses, e.g. AMD's IBS (which is essentially a counter + some additional registers to give more details about the interrupted instruction)

-Andi

[email protected]