Skip to contentSkip to conversation

Performance Counters for Linux, v6

announce Ingo Molnar 10 participants 38 messages Jan 21, 2009 – Mar 10, 2009

Subsystems (31)
lore
Share & tools
lore
#0 lore raw

We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:

http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patch

It's also available in tip/master at:

http://people.redhat.com/mingo/tip.git/README

There are many changes in the v6 release:

- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.

- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc instrumentation and PAPI support.

- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.

- The 'perfstat' utility (ex 'timec') has been updated:

http://people.redhat.com/mingo/perfcounters/perfstat.c

- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:

http://people.redhat.com/mingo/perfcounters/kerneltop.c

- Merged to latest mainline

- Various fixes and other updates

Ingo

------------------>
Andi Kleen (1):
x86, generic: mark complex bitops.h inlines as __always_inline

Brian Gerst (12):
x86-64: Move irq stats from PDA to per-cpu and consolidate with 32-bit.
x86-64: Move TLB state from PDA to per-cpu and consolidate with 32-bit.
x86-64: Convert irqstacks to per-cpu
x86-64: Convert exception stacks to per-cpu
x86-64: Move cpu number from PDA to per-cpu and consolidate with 32-bit.
x86-64: Move current task from PDA to per-cpu and consolidate with 32-bit.
x86-64: Move kernelstack from PDA to per-cpu.
x86-64: Move oldrsp from PDA to per-cpu.
x86-64: Move irqcount from PDA to per-cpu.
x86-64: Move nodenumber from PDA to per-cpu.
x86-64: Move isidle from PDA to per-cpu.
x86-64: Use absolute displacements for per-cpu accesses.

Christophe Saout (1):
xen: fix too early kmalloc call

Ingo Molnar (50):
performance counters: documentation
performance counters: x86 support
x86, perfcounters: read out MSR_CORE_PERF_GLOBAL_STATUS with counters disabled
perfcounters: select ANON_INODES
perfcounters, x86: simplify disable/enable of counters perfcounters, x86: clean up debug code
perfcounters: consolidate global-disable codepaths
perf counters: restructure the API
perf counters: add support for group counters
perf counters: group counter, fixes
perf counters: hw driver API
perf counters: implement PERF_COUNT_CPU_CLOCK
perf counters: consolidate hw_perf save/restore APIs
perf counters: implement PERF_COUNT_TASK_CLOCK
perf counters: add prctl interface to disable/enable counters perf counters: clean up state transitions
perf counters: update docs
x86: implement atomic64_t on 32-bit
perfcounters: restructure x86 counter math
perfcounters: implement "counter inheritance"
perfcounters: fix task clock counter
perfcounters: add context switch counter
perfcounters: add task migrations counter
perfcounters: add nr-of-faults counter
perfcounters: fix non-intel-perfmon CPUs
perfcounters, x86: fix sw counters on non-PMC CPUs
perfcounters: fix lapic initialization
perfcounters: release CPU context when exiting task counters
perfcounters: flush on setuid exec
perfcounters: use hw_event.disable flag
perfcounters: remove warnings
perfcounters: tweak group scheduling
x86, perfcounters: rename intel_arch_perfmon.h => perf_counter.h x86, perfcounters: prepare for fixed-mode PMCs
perfcounters: add fixed-mode PMC enumeration
x86, perfcounters: refactor code for fixed-function PMCs
perfcounters: hw ops rename
perfcounters: fix task clock counter
perfcounters: pull inherited counters
perfcounters: fix init context lock
perfcounters: enable lowlevel pmc code to schedule counters
x86, perfcounters: print out the ->used bitmask
perfcounters: remove ->nr_inherited
perfcounters: generalize the counter scheduler
perfcounters: add PERF_COUNT_BUS_CYCLES
x86, perfcounters: add support for fixed-function pmcs
perfcounters: include asm/perf_counter.h only if CONFIG_PERF_COUNTERS=y x86, cpufreq: remove leftover copymask_copy()
x86: fix broken flush_tlb_others_ipi(), fix
percpu: add optimized generic percpu accessors

Jan Beulich (2):
x86: fully honor "nolapic"
x86: avoid early crash in disable_local_APIC()

Jaswinder Singh (1):
x86: perf_counter.c intel_perfmon_event_map and max_intel_perfmon_events should be static

Jaswinder Singh Rajput (20):
x86: perf_counter remove unwanted hw_perf_enable_all
x86: smp.h remove obsolete function declaration
x86: smp.h move zap_low_mappings declartion to tlbflush.h
x86: smp.h move prefill_possible_map declartion to cpu.h
x86: smp.h move stack_processor_id declartion to cpu.h
x86: smp.h move safe_smp_processor_id declartion to cpu.h
x86: smp.h move cpu_physical_id declartion to cpu.h
x86: smp.h move boot_cpu_id declartion to cpu.h
x86: rename intel_mp_floating to mpf_intel
x86: rename all fields of mpf_intel mpf_X to X
x86: smp.h move cpu_callin_mask and cpu_callin_map declartion to cpumask.h
x86: smp.h move cpu_callout_mask and cpu_callout_map declartion to cpumask.h
x86: smp.h move cpu_initialized_mask and cpu_initialized declartion to cpumask.h
x86: smp.h move cpu_sibling_setup_mask and cpu_sibling_setup_map declartion to cpumask.h
x86: microcode_intel.c fix style problems
x86: msr.c fix style problems
x86: module_32.c fix style problems
x86: module_64.c fix style problems
x86: replacing mp_config_ioapic with mpc_ioapic
x86: replacing mp_config_intsrc with mpc_intsrc

Mike Travis (17):
cpumask: update irq_desc to use cpumask_var_t
cpumask: fix bug in use cpumask_var_t in irq_desc SGI UV cpumask: use static temp cpumask in flush_tlb
x86: cleanup remaining cpumask_t code in mce_amd_64.c
x86: reduce stack usage in init_intel_cacheinfo
cpumask: use cpumask_var_t in dcdbas.c
cpumask: reduce stack usage in find_lowest_rq
Xen: reduce memory required for cpu_evtchn_mask
irq: change references from NR_IRQS to nr_irqs
irq: use WARN() instead of WARN_ON().
irq: allocate irq_desc_ptrs array based on nr_irqs
irq: initialize nr_irqs based on nr_cpu_ids
kstat: modify kstat_irqs_legacy to be variable sized cpumask, irq: non-x86 build failures
irq: update all arches for new irq_desc
irq: update all arches for new irq_desc, fix
x86: cleanup early setup_percpu references

Paul Mackerras (13):
perf_counter: Fix return value from dummy hw_perf_counter_init
perf_counter: Fix the cpu_clock software counter
perf_counter: Add optional hw_perf_group_sched_in arch function
perf_counter: Add dummy perf_counter_print_debug function powerpc/perf_counter: Add perf_counter system call on powerpc
powerpc: Provide a way to defer perf counter work until interrupts are enabled powerpc/perf_counter: Add generic support for POWER-family PMU hardware powerpc/perf_counter: Add support for PPC970 family powerpc/perf_counter: Add support for POWER6
perf_counter: Always schedule all software counters in powerpc/perf_counter: Make sure PMU gets enabled properly
perf_counter: Add support for pinned and exclusive counter groups
perf_counter: Add counter enable/disable ioctls

Rusty Russell (5):
cpumask: Use topology_core_cpumask()/topology_thread_cpumask()
cpumask: convert misc driver functions
cpumask: convert drivers/net/sfc
cpumask: convert other misc kernel functions
x86: change flush_tlb_others to take a const struct cpumask

Suresh Siddha (2):
x86: fix broken flush_tlb_others_ipi()
x86, pat: fix reserve_memtype() for legacy 1MB range

Tejun Heo (15):
x86: fix pda_to_op()
x86: make early_per_cpu() a lvalue and use it
x86: make vmlinux_32.lds.S use PERCPU() macro
x86: make percpu symbols zerobased on SMP
x86: load pointer to pda into %gs while brining up a CPU
x86: use static _cpu_pda array
x86: fold pda into percpu area on SMP
x86: merge 64 and 32 SMP percpu handling
x86: make pda a percpu variable
x86: convert pda ops to wrappers around x86 percpu accessors
x86: misc clean up after the percpu update
x86: fix build bug introduced during merge
x86_64: initialize this_cpu_off to __per_cpu_load linker script: add missing VMLINUX_SYMBOL
linker script: add missing .data.percpu.page_aligned

Thomas Gleixner (4):
performance counters: core code
perf counters: protect them against CSTATE transitions perf counters: clean up 'raw' type API
perf counters: expand use of counter->event

Yinghai Lu (3):
perf_counter: more barrier in blank weak function
x86: arch_probe_nr_irqs
x86: make 32bit MAX_HARDIRQS_PER_CPU to be NR_VECTORS

 Documentation/cputopology.txt                  |    6 +-
 Documentation/perf-counters.txt                |  147 ++
 arch/alpha/kernel/irq.c                        |    2 +-
 arch/arm/kernel/irq.c                          |   18 +-
 arch/arm/kernel/vmlinux.lds.S                  |    1 +
 arch/arm/oprofile/op_model_mpcore.c            |    2 +-
 arch/blackfin/kernel/irqchip.c                 |    5 +
 arch/ia64/kernel/iosapic.c                     |    2 +-
 arch/ia64/kernel/irq.c                         |    4 +-
 arch/ia64/kernel/irq_ia64.c                    |   12 +-
 arch/ia64/kernel/msi_ia64.c                    |    4 +-
 arch/ia64/kernel/vmlinux.lds.S                 |    1 +
 arch/ia64/sn/kernel/msi_sn.c                   |    2 +-
 arch/mips/include/asm/irq.h                    |    2 +-
 arch/mips/kernel/irq-gic.c                     |    2 +-
 arch/mips/kernel/smtc.c                        |    6 +-
 arch/mips/mti-malta/malta-smtc.c               |    5 +-
 arch/mips/sgi-ip22/ip22-int.c                  |    2 +-
 arch/mips/sgi-ip22/ip22-time.c                 |    2 +-
 arch/mips/sibyte/bcm1480/smp.c                 |    3 +-
 arch/mips/sibyte/sb1250/smp.c                  |    3 +-
 arch/mn10300/kernel/mn10300-watchdog.c         |    3 +-
 arch/parisc/kernel/irq.c                       |    8 +-
 arch/powerpc/include/asm/hw_irq.h              |   31 +
 arch/powerpc/include/asm/paca.h                |    1 +
 arch/powerpc/include/asm/perf_counter.h        |   72 +
 arch/powerpc/include/asm/systbl.h              |    1 +
 arch/powerpc/include/asm/unistd.h              |    3 +-
 arch/powerpc/kernel/Makefile                   |    1 +
 arch/powerpc/kernel/asm-offsets.c              |    1 +
 arch/powerpc/kernel/entry_64.S                 |    9 +
 arch/powerpc/kernel/irq.c                      |   12 +-
 arch/powerpc/kernel/perf_counter.c             |  785 +++++++++
 arch/powerpc/kernel/power6-pmu.c               |  283 +++
 arch/powerpc/kernel/ppc970-pmu.c               |  375 ++++
 arch/powerpc/kernel/vmlinux.lds.S              |    1 +
 arch/powerpc/platforms/Kconfig.cputype         |    1 +
 arch/powerpc/platforms/pseries/xics.c          |    5 +-
 arch/powerpc/sysdev/mpic.c                     |    3 +-
 arch/sparc/kernel/irq_64.c                     |    5 +-
 arch/sparc/kernel/time_64.c                    |    2 +-
 arch/x86/Kconfig                               |    1 +
 arch/x86/ia32/ia32entry.S                      |   11 +-
 arch/x86/include/asm/apicnum.h                 |   12 +
 arch/x86/include/asm/atomic_32.h               |  218 +++
 arch/x86/include/asm/bitops.h                  |   14 +-
 arch/x86/include/asm/cpu.h                     |   21 +
 arch/x86/include/asm/cpumask.h                 |   28 +
 arch/x86/include/asm/current.h                 |   24 +-
 arch/x86/include/asm/hardirq_32.h              |    4 +
 arch/x86/include/asm/hardirq_64.h              |   25 +-
 arch/x86/include/asm/hw_irq.h                  |    2 +
 arch/x86/include/asm/intel_arch_perfmon.h      |   31 -
 arch/x86/include/asm/io_apic.h                 |   26 +-
 arch/x86/include/asm/irq_regs_32.h             |    4 +-
 arch/x86/include/asm/irq_vectors.h             |   18 +-
 arch/x86/include/asm/mach-default/entry_arch.h |    5 +
 arch/x86/include/asm/mmu_context_32.h          |   12 +-
 arch/x86/include/asm/mmu_context_64.h          |   16 +-
 arch/x86/include/asm/mpspec_def.h              |   23 +-
 arch/x86/include/asm/page_64.h                 |    4 +-
 arch/x86/include/asm/paravirt.h                |    8 +-
 arch/x86/include/asm/pda.h                     |  124 +--
 arch/x86/include/asm/percpu.h                  |  159 +-
 arch/x86/include/asm/perf_counter.h            |   95 +
 arch/x86/include/asm/processor.h               |    3 +
 arch/x86/include/asm/setup.h                   |    1 -
 arch/x86/include/asm/smp.h                     |   49 +-
 arch/x86/include/asm/system.h                  |    4 +-
 arch/x86/include/asm/thread_info.h             |   24 +-
 arch/x86/include/asm/tlbflush.h                |   17 +-
 arch/x86/include/asm/topology.h                |    8 +-
 arch/x86/include/asm/trampoline.h              |    1 +
 arch/x86/include/asm/unistd_32.h               |    1 +
 arch/x86/include/asm/unistd_64.h               |    3 +-
 arch/x86/include/asm/uv/uv_bau.h               |    3 +-
 arch/x86/kernel/acpi/boot.c                    |   96 +-
 arch/x86/kernel/acpi/sleep.c                   |    1 +
 arch/x86/kernel/apic.c                         |   26 +-
 arch/x86/kernel/asm-offsets_64.c               |    8 +-
 arch/x86/kernel/cpu/Makefile                   |   12 +-
 arch/x86/kernel/cpu/common.c                   |   82 +-
 arch/x86/kernel/cpu/cpufreq/acpi-cpufreq.c     |    2 -
 arch/x86/kernel/cpu/intel_cacheinfo.c          |   63 +-
 arch/x86/kernel/cpu/mcheck/mce_amd_64.c        |   21 +-
 arch/x86/kernel/cpu/perf_counter.c             |  695 ++++++++
 arch/x86/kernel/cpu/perfctr-watchdog.c         |    2 +-
 arch/x86/kernel/crash.c                        |    2 +-
 arch/x86/kernel/dumpstack_64.c                 |   35 +-
 arch/x86/kernel/entry_64.S                     |   46 +-
 arch/x86/kernel/head64.c                       |   23 +-
 arch/x86/kernel/head_64.S                      |   47 +-
 arch/x86/kernel/io_apic.c                      |  163 +-
 arch/x86/kernel/irq.c                          |   11 +-
 arch/x86/kernel/irq_32.c                       |    2 +-
 arch/x86/kernel/irq_64.c                       |    5 +-
 arch/x86/kernel/irqinit_32.c                   |    3 +
 arch/x86/kernel/irqinit_64.c                   |    5 +
 arch/x86/kernel/microcode_intel.c              |   10 +-
 arch/x86/kernel/module_32.c                    |    6 +-
 arch/x86/kernel/module_64.c                    |   32 +-
 arch/x86/kernel/mpparse.c                      |  142 +-
 arch/x86/kernel/msr.c                          |    2 +-
 arch/x86/kernel/nmi.c                          |   10 +-
 arch/x86/kernel/process_32.c                   |    5 +-
 arch/x86/kernel/process_64.c                   |   22 +-
 arch/x86/kernel/reboot.c                       |    1 +
 arch/x86/kernel/setup.c                        |    2 +-
 arch/x86/kernel/setup_percpu.c                 |  208 ++-
 arch/x86/kernel/signal.c                       |    7 +-
 arch/x86/kernel/smpboot.c                      |   69 +-
 arch/x86/kernel/smpcommon.c                    |   10 +-
 arch/x86/kernel/syscall_table_32.S             |    1 +
 arch/x86/kernel/tlb_32.c                       |   85 +-
 arch/x86/kernel/tlb_64.c                       |   76 +-
 arch/x86/kernel/tlb_uv.c                       |   16 +-
 arch/x86/kernel/vmlinux_32.lds.S               |    9 +-
 arch/x86/kernel/vmlinux_64.lds.S               |   22 +-
 arch/x86/kernel/x8664_ksyms_64.c               |    2 -
 arch/x86/mach-voyager/setup.c                  |    1 +
 arch/x86/mach-voyager/voyager_smp.c            |    6 +-
 arch/x86/mm/init_32.c                          |    1 -
 arch/x86/mm/pat.c                              |   37 +-
 arch/x86/oprofile/op_model_ppro.c              |    2 +-
 arch/x86/xen/enlighten.c                       |   47 +-
 arch/x86/xen/irq.c                             |    8 +-
 arch/x86/xen/mmu.c                             |    8 +-
 arch/x86/xen/multicalls.h                      |    2 +-
 arch/x86/xen/smp.c                             |   33 +-
 arch/x86/xen/xen-asm_64.S                      |   31 +-
 drivers/acpi/processor_idle.c                  |    8 +
 drivers/base/cpu.c                             |    2 +-
 drivers/base/topology.c                        |   33 +-
 drivers/char/sysrq.c                           |    2 +
 drivers/firmware/dcdbas.c                      |   12 +-
 drivers/misc/sgi-xp/xpc_main.c                 |    2 +-
 drivers/net/sfc/efx.c                          |   17 +-
 drivers/oprofile/buffer_sync.c                 |   22 +-
 drivers/oprofile/buffer_sync.h                 |    4 +
 drivers/oprofile/oprof.c                       |    9 +-
 drivers/pci/intr_remapping.c                   |    1 +
 drivers/xen/events.c                           |   26 +-
 drivers/xen/manage.c                           |    2 +-
 fs/exec.c                                      |    8 +
 include/asm-generic/bitops/__ffs.h             |    2 +-
 include/asm-generic/bitops/__fls.h             |    2 +-
 include/asm-generic/bitops/fls.h               |    2 +-
 include/asm-generic/bitops/fls64.h             |    4 +-
 include/asm-generic/percpu.h                   |   52 +
 include/asm-generic/sections.h                 |    2 +-
 include/asm-generic/vmlinux.lds.h              |   75 +-
 include/linux/init_task.h                      |   11 +
 include/linux/interrupt.h                      |    1 +
 include/linux/irq.h                            |   86 +-
 include/linux/irqnr.h                          |    1 +
 include/linux/kernel_stat.h                    |    8 +
 include/linux/perf_counter.h                   |  290 ++++
 include/linux/prctl.h                          |    3 +
 include/linux/sched.h                          |   12 +-
 include/linux/syscalls.h                       |    8 +
 include/linux/topology.h                       |    6 +
 init/Kconfig                                   |   30 +
 kernel/Makefile                                |    1 +
 kernel/exit.c                                  |   13 +-
 kernel/fork.c                                  |    1 +
 kernel/irq/chip.c                              |    5 +-
 kernel/irq/handle.c                            |   57 +-
 kernel/irq/internals.h                         |    7 +
 kernel/irq/manage.c                            |   12 +-
 kernel/irq/migration.c                         |   12 +-
 kernel/irq/numa_migrate.c                      |   19 +-
 kernel/irq/proc.c                              |    4 +-
 kernel/perf_counter.c                          | 2169 ++++++++++++++++++++++++
 kernel/sched.c                                 |   76 +-
 kernel/sched_rt.c                              |   36 +-
 kernel/softirq.c                               |    5 +
 kernel/sys.c                                   |    7 +
 kernel/sys_ni.c                                |    3 +
 lib/smp_processor_id.c                         |    2 +-
 179 files changed, 6925 insertions(+), 1342 deletions(-)
 create mode 100644 Documentation/perf-counters.txt
 create mode 100644 arch/powerpc/include/asm/perf_counter.h
 create mode 100644 arch/powerpc/kernel/perf_counter.c
 create mode 100644 arch/powerpc/kernel/power6-pmu.c
 create mode 100644 arch/powerpc/kernel/ppc970-pmu.c
 create mode 100644 arch/x86/include/asm/apicnum.h
 create mode 100644 arch/x86/include/asm/cpumask.h
 delete mode 100644 arch/x86/include/asm/intel_arch_perfmon.h
 create mode 100644 arch/x86/include/asm/perf_counter.h
 create mode 100644 arch/x86/kernel/cpu/perf_counter.c
 create mode 100644 include/linux/perf_counter.h
 create mode 100644 kernel/perf_counter.c
Documentation/cputopology.txt +3 -3
index 45932ec..b41f3e5 100644
@@ -18,11 +18,11 @@ For an architecture to support this feature, it must define some of
 these macros in include/asm-XXX/topology.h:
 #define topology_physical_package_id(cpu)
 #define topology_core_id(cpu)
-#define topology_thread_siblings(cpu)
-#define topology_core_siblings(cpu)
+#define topology_thread_cpumask(cpu)
+#define topology_core_cpumask(cpu)
 
 The type of **_id is int.
-The type of siblings is cpumask_t.
+The type of siblings is (const) struct cpumask *.
 
 To be consistent on all architectures, include/linux/topology.h
 provides default definitions for any of the above macros that are
Documentation/perf-counters.txt new file +147 -0

147 lines not on this page · raw message

arch/alpha/kernel/irq.c +1 -1
index 703731a..7bc7489 100644
@@ -55,7 +55,7 @@ int irq_select_affinity(unsigned int irq)
 		cpu = (cpu < (NR_CPUS-1) ? cpu + 1 : 0);
 	last_cpu = cpu;
 
-	irq_desc[irq].affinity = cpumask_of_cpu(cpu);
+	cpumask_copy(irq_desc[irq].affinity, cpumask_of(cpu));
 	irq_desc[irq].chip->set_affinity(irq, cpumask_of(cpu));
 	return 0;
 }
arch/arm/kernel/irq.c +12 -6
index 7141cee..4bb723e 100644
@@ -104,6 +104,11 @@ static struct irq_desc bad_irq_desc = {
 	.lock = SPIN_LOCK_UNLOCKED
 };
 
+#ifdef CONFIG_CPUMASK_OFFSTACK
+/* We are not allocating bad_irq_desc.affinity or .pending_mask */
+#error "ARM architecture does not support CONFIG_CPUMASK_OFFSTACK."
+#endif
+
 /*
  * do_IRQ handles all hardware IRQ's.  Decoded IRQs should not
  * come via this function.  Instead, they should provide their
@@ -161,7 +166,7 @@ void __init init_IRQ(void)
 		irq_desc[irq].status |= IRQ_NOREQUEST | IRQ_NOPROBE;
 
 #ifdef CONFIG_SMP
-	bad_irq_desc.affinity = CPU_MASK_ALL;
+	cpumask_setall(bad_irq_desc.affinity);
 	bad_irq_desc.cpu = smp_processor_id();
 #endif
 	init_arch_irq();
@@ -191,15 +196,16 @@ void migrate_irqs(void)
 		struct irq_desc *desc = irq_desc + i;
 
 		if (desc->cpu == cpu) {
-			unsigned int newcpu = any_online_cpu(desc->affinity);
-
-			if (newcpu == NR_CPUS) {
+			unsigned int newcpu = cpumask_any_and(desc->affinity,
+							      cpu_online_mask);
+			if (newcpu >= nr_cpu_ids) {
 				if (printk_ratelimit())
 					printk(KERN_INFO "IRQ%u no longer affine to CPU%u\n",
 					       i, cpu);
 
-				cpus_setall(desc->affinity);
-				newcpu = any_online_cpu(desc->affinity);
+				cpumask_setall(desc->affinity);
+				newcpu = cpumask_any_and(desc->affinity,
+							 cpu_online_mask);
 			}
 
 			route_irq(desc, i, newcpu);
arch/arm/kernel/vmlinux.lds.S +1 -0
index 0021607..85598f7 100644
@@ -65,6 +65,7 @@ SECTIONS
 #endif
 		. = ALIGN(4096);
 		__per_cpu_start = .;
+			*(.data.percpu.page_aligned)
 			*(.data.percpu)
 			*(.data.percpu.shared_aligned)
 		__per_cpu_end = .;
arch/arm/oprofile/op_model_mpcore.c +1 -1
index 6d6bd58..853d42b 100644
@@ -263,7 +263,7 @@ static void em_route_irq(int irq, unsigned int cpu)
 	const struct cpumask *mask = cpumask_of(cpu);
 
 	spin_lock_irq(&desc->lock);
-	desc->affinity = *mask;
+	cpumask_copy(desc->affinity, mask);
 	desc->chip->set_affinity(irq, mask);
 	spin_unlock_irq(&desc->lock);
 }
arch/blackfin/kernel/irqchip.c +5 -0
index ab8209c..5780d6d 100644
@@ -69,6 +69,11 @@ static struct irq_desc bad_irq_desc = {
 #endif
 };
 
+#ifdef CONFIG_CPUMASK_OFFSTACK
+/* We are not allocating a variable-sized bad_irq_desc.affinity */
+#error "Blackfin architecture does not support CONFIG_CPUMASK_OFFSTACK."
+#endif
+
 int show_interrupts(struct seq_file *p, void *v)
 {
 	int i = *(loff_t *) v, j;
arch/ia64/kernel/iosapic.c +1 -1
index 5cfd3d9..006ad36 100644
@@ -880,7 +880,7 @@ iosapic_unregister_intr (unsigned int gsi)
 	if (iosapic_intr_info[irq].count == 0) {
 #ifdef CONFIG_SMP
 		/* Clear affinity */
-		cpus_setall(idesc->affinity);
+		cpumask_setall(idesc->affinity);
 #endif
 		/* Clear the interrupt information */
 		iosapic_intr_info[irq].dest = 0;
arch/ia64/kernel/irq.c +2 -2
index a58f64c..226233a 100644
@@ -103,7 +103,7 @@ static char irq_redir [NR_IRQS]; // = { [0 ... NR_IRQS-1] = 1 };
 void set_irq_affinity_info (unsigned int irq, int hwid, int redir)
 {
 	if (irq < NR_IRQS) {
-		cpumask_copy(&irq_desc[irq].affinity,
+		cpumask_copy(irq_desc[irq].affinity,
 			     cpumask_of(cpu_logical_id(hwid)));
 		irq_redir[irq] = (char) (redir & 0xff);
 	}
@@ -148,7 +148,7 @@ static void migrate_irqs(void)
 		if (desc->status == IRQ_PER_CPU)
 			continue;
 
-		if (cpumask_any_and(&irq_desc[irq].affinity, cpu_online_mask)
+		if (cpumask_any_and(irq_desc[irq].affinity, cpu_online_mask)
 		    >= nr_cpu_ids) {
 			/*
 			 * Save it for phase 2 processing
arch/ia64/kernel/irq_ia64.c +8 -4
index 28d3d48..927ad02 100644
@@ -493,11 +493,13 @@ ia64_handle_irq (ia64_vector vector, struct pt_regs *regs)
 	saved_tpr = ia64_getreg(_IA64_REG_CR_TPR);
 	ia64_srlz_d();
 	while (vector != IA64_SPURIOUS_INT_VECTOR) {
+		struct irq_desc *desc = irq_to_desc(vector);
+
 		if (unlikely(IS_LOCAL_TLB_FLUSH(vector))) {
 			smp_local_flush_tlb();
-			kstat_this_cpu.irqs[vector]++;
+			kstat_incr_irqs_this_cpu(vector, desc);
 		} else if (unlikely(IS_RESCHEDULE(vector)))
-			kstat_this_cpu.irqs[vector]++;
+			kstat_incr_irqs_this_cpu(vector, desc);
 		else {
 			int irq = local_vector_to_irq(vector);
 
@@ -551,11 +553,13 @@ void ia64_process_pending_intr(void)
 	  * Perform normal interrupt style processing
 	  */
 	while (vector != IA64_SPURIOUS_INT_VECTOR) {
+		struct irq_desc *desc = irq_to_desc(vector);
+
 		if (unlikely(IS_LOCAL_TLB_FLUSH(vector))) {
 			smp_local_flush_tlb();
-			kstat_this_cpu.irqs[vector]++;
+			kstat_incr_irqs_this_cpu(vector, desc);
 		} else if (unlikely(IS_RESCHEDULE(vector)))
-			kstat_this_cpu.irqs[vector]++;
+			kstat_incr_irqs_this_cpu(vector, desc);
 		else {
 			struct pt_regs *old_regs = set_irq_regs(NULL);
 			int irq = local_vector_to_irq(vector);
arch/ia64/kernel/msi_ia64.c +2 -2
index 8903393..dcb6b7c 100644
@@ -75,7 +75,7 @@ static void ia64_set_msi_irq_affinity(unsigned int irq,
 	msg.data = data;
 
 	write_msi_msg(irq, &msg);
-	irq_desc[irq].affinity = cpumask_of_cpu(cpu);
+	cpumask_copy(irq_desc[irq].affinity, cpumask_of(cpu));
 }
 #endif /* CONFIG_SMP */
 
@@ -187,7 +187,7 @@ static void dmar_msi_set_affinity(unsigned int irq, const struct cpumask *mask)
 	msg.address_lo |= MSI_ADDR_DESTID_CPU(cpu_physical_id(cpu));
 
 	dmar_msi_write(irq, &msg);
-	irq_desc[irq].affinity = *mask;
+	cpumask_copy(irq_desc[irq].affinity, mask);
 }
 #endif /* CONFIG_SMP */
 
arch/ia64/kernel/vmlinux.lds.S +1 -0
index 10a7d47..f45e4e5 100644
@@ -219,6 +219,7 @@ SECTIONS
   .data.percpu PERCPU_ADDR : AT(__phys_per_cpu_start - LOAD_OFFSET)
 	{
 		__per_cpu_start = .;
+		*(.data.percpu.page_aligned)
 		*(.data.percpu)
 		*(.data.percpu.shared_aligned)
 		__per_cpu_end = .;
arch/ia64/sn/kernel/msi_sn.c +1 -1
index ca553b0..81e4289 100644
@@ -205,7 +205,7 @@ static void sn_set_msi_irq_affinity(unsigned int irq,
 	msg.address_lo = (u32)(bus_addr & 0x00000000ffffffff);
 
 	write_msi_msg(irq, &msg);
-	irq_desc[irq].affinity = *cpu_mask;
+	cpumask_copy(irq_desc[irq].affinity, cpu_mask);
 }
 #endif /* CONFIG_SMP */
 
arch/mips/include/asm/irq.h +1 -1
index abc62aa..3214ade 100644
@@ -66,7 +66,7 @@ extern void smtc_forward_irq(unsigned int irq);
  */
 #define IRQ_AFFINITY_HOOK(irq)						\
 do {									\
-    if (!cpu_isset(smp_processor_id(), irq_desc[irq].affinity)) {	\
+    if (!cpumask_test_cpu(smp_processor_id(), irq_desc[irq].affinity)) {\
 	smtc_forward_irq(irq);						\
 	irq_exit();							\
 	return;								\
arch/mips/kernel/irq-gic.c +1 -1
index 494a49a..87deb8f 100644
@@ -187,7 +187,7 @@ static void gic_set_affinity(unsigned int irq, const struct cpumask *cpumask)
 		set_bit(irq, pcpu_masks[first_cpu(tmp)].pcpu_mask);
 
 	}
-	irq_desc[irq].affinity = *cpumask;
+	cpumask_copy(irq_desc[irq].affinity, cpumask);
 	spin_unlock_irqrestore(&gic_lock, flags);
 
 }
arch/mips/kernel/smtc.c +4 -2
index b6cca01..5f5af7d 100644
@@ -686,7 +686,7 @@ void smtc_forward_irq(unsigned int irq)
 	 * and efficiency, we just pick the easiest one to find.
 	 */
 
-	target = first_cpu(irq_desc[irq].affinity);
+	target = cpumask_first(irq_desc[irq].affinity);
 
 	/*
 	 * We depend on the platform code to have correctly processed
@@ -921,11 +921,13 @@ void ipi_decode(struct smtc_ipi *pipi)
 	struct clock_event_device *cd;
 	void *arg_copy = pipi->arg;
 	int type_copy = pipi->type;
+	int irq = MIPS_CPU_IRQ_BASE + 1;
+
 	smtc_ipi_nq(&freeIPIq, pipi);
 	switch (type_copy) {
 	case SMTC_CLOCK_TICK:
 		irq_enter();
-		kstat_this_cpu.irqs[MIPS_CPU_IRQ_BASE + 1]++;
+		kstat_incr_irqs_this_cpu(irq, irq_to_desc(irq));
 		cd = &per_cpu(mips_clockevent_device, cpu);
 		cd->event_handler(cd);
 		irq_exit();
arch/mips/mti-malta/malta-smtc.c +3 -2
index aabd727..5ba3188 100644
@@ -116,7 +116,7 @@ struct plat_smp_ops msmtc_smp_ops = {
 
 void plat_set_irq_affinity(unsigned int irq, const struct cpumask *affinity)
 {
-	cpumask_t tmask = *affinity;
+	cpumask_t tmask;
 	int cpu = 0;
 	void smtc_set_irq_affinity(unsigned int irq, cpumask_t aff);
 
@@ -139,11 +139,12 @@ void plat_set_irq_affinity(unsigned int irq, const struct cpumask *affinity)
 	 * be made to forward to an offline "CPU".
 	 */
 
+	cpumask_copy(&tmask, affinity);
 	for_each_cpu(cpu, affinity) {
 		if ((cpu_data[cpu].vpe_id != 0) || !cpu_online(cpu))
 			cpu_clear(cpu, tmask);
 	}
-	irq_desc[irq].affinity = tmask;
+	cpumask_copy(irq_desc[irq].affinity, &tmask);
 
 	if (cpus_empty(tmask))
 		/*
arch/mips/sgi-ip22/ip22-int.c +1 -1
index f8b18af..0ecd5fe 100644
@@ -155,7 +155,7 @@ static void indy_buserror_irq(void)
 	int irq = SGI_BUSERR_IRQ;
 
 	irq_enter();
-	kstat_this_cpu.irqs[irq]++;
+	kstat_incr_irqs_this_cpu(irq, irq_to_desc(irq));
 	ip22_be_interrupt(irq);
 	irq_exit();
 }
arch/mips/sgi-ip22/ip22-time.c +1 -1
index 3dcb27e..c8f7d23 100644
@@ -122,7 +122,7 @@ void indy_8254timer_irq(void)
 	char c;
 
 	irq_enter();
-	kstat_this_cpu.irqs[irq]++;
+	kstat_incr_irqs_this_cpu(irq, irq_to_desc(irq));
 	printk(KERN_ALERT "Oops, got 8254 interrupt.\n");
 	ArcRead(0, &c, 1, &cnt);
 	ArcEnterInteractiveMode();
arch/mips/sibyte/bcm1480/smp.c +2 -1
index dddfda8..3146916 100644
@@ -178,9 +178,10 @@ struct plat_smp_ops bcm1480_smp_ops = {
 void bcm1480_mailbox_interrupt(void)
 {
 	int cpu = smp_processor_id();
+	int irq = K_BCM1480_INT_MBOX_0_0;
 	unsigned int action;
 
-	kstat_this_cpu.irqs[K_BCM1480_INT_MBOX_0_0]++;
+	kstat_incr_irqs_this_cpu(irq, irq_to_desc(irq));
 	/* Load the mailbox register to figure out what we're supposed to do */
 	action = (__raw_readq(mailbox_0_regs[cpu]) >> 48) & 0xffff;
 
arch/mips/sibyte/sb1250/smp.c +2 -1
index 5950a28..cad1400 100644
@@ -166,9 +166,10 @@ struct plat_smp_ops sb_smp_ops = {
 void sb1250_mailbox_interrupt(void)
 {
 	int cpu = smp_processor_id();
+	int irq = K_INT_MBOX_0;
 	unsigned int action;
 
-	kstat_this_cpu.irqs[K_INT_MBOX_0]++;
+	kstat_incr_irqs_this_cpu(irq, irq_to_desc(irq));
 	/* Load the mailbox register to figure out what we're supposed to do */
 	action = (____raw_readq(mailbox_regs[cpu]) >> 48) & 0xffff;
 
arch/mn10300/kernel/mn10300-watchdog.c +2 -1
index 10811e9..2e370d8 100644
@@ -130,6 +130,7 @@ void watchdog_interrupt(struct pt_regs *regs, enum exception_code excep)
 	 * the stack NMI-atomically, it's safe to use smp_processor_id().
 	 */
 	int sum, cpu = smp_processor_id();
+	int irq = NMIIRQ;
 	u8 wdt, tmp;
 
 	wdt = WDCTR & ~WDCTR_WDCNE;
@@ -138,7 +139,7 @@ void watchdog_interrupt(struct pt_regs *regs, enum exception_code excep)
 	NMICR = NMICR_WDIF;
 
 	nmi_count(cpu)++;
-	kstat_this_cpu.irqs[NMIIRQ]++;
+	kstat_incr_irqs_this_cpu(irq, irq_to_desc(irq));
 	sum = irq_stat[cpu].__irq_count;
 
 	if (last_irq_sums[cpu] == sum) {
arch/parisc/kernel/irq.c +4 -4
index ac2c822..4948280 100644
@@ -120,7 +120,7 @@ int cpu_check_affinity(unsigned int irq, cpumask_t *dest)
 	if (CHECK_IRQ_PER_CPU(irq)) {
 		/* Bad linux design decision.  The mask has already
 		 * been set; we must reset it */
-		irq_desc[irq].affinity = CPU_MASK_ALL;
+		cpumask_setall(irq_desc[irq].affinity);
 		return -EINVAL;
 	}
 
@@ -136,7 +136,7 @@ static void cpu_set_affinity_irq(unsigned int irq, const struct cpumask *dest)
 	if (cpu_check_affinity(irq, dest))
 		return;
 
-	irq_desc[irq].affinity = *dest;
+	cpumask_copy(irq_desc[irq].affinity, dest);
 }
 #endif
 
@@ -295,7 +295,7 @@ int txn_alloc_irq(unsigned int bits_wide)
 unsigned long txn_affinity_addr(unsigned int irq, int cpu)
 {
 #ifdef CONFIG_SMP
-	irq_desc[irq].affinity = cpumask_of_cpu(cpu);
+	cpumask_copy(irq_desc[irq].affinity, cpumask_of(cpu));
 #endif
 
 	return per_cpu(cpu_data, cpu).txn_addr;
@@ -352,7 +352,7 @@ void do_cpu_irq_mask(struct pt_regs *regs)
 	irq = eirr_to_irq(eirr_val);
 
 #ifdef CONFIG_SMP
-	dest = irq_desc[irq].affinity;
+	cpumask_copy(&dest, irq_desc[irq].affinity);
 	if (CHECK_IRQ_PER_CPU(irq_desc[irq].status) &&
 	    !cpu_isset(smp_processor_id(), dest)) {
 		int cpu = first_cpu(dest);
arch/powerpc/include/asm/hw_irq.h +31 -0
index f75a5fc..e10f151 100644
@@ -131,5 +131,36 @@ static inline int irqs_disabled_flags(unsigned long flags)
  */
 struct hw_interrupt_type;
 
+#ifdef CONFIG_PERF_COUNTERS
+static inline unsigned long get_perf_counter_pending(void)
+{
+	unsigned long x;
+
+	asm volatile("lbz %0,%1(13)"
+		: "=r" (x)
+		: "i" (offsetof(struct paca_struct, perf_counter_pending)));
+	return x;
+}
+
+static inline void set_perf_counter_pending(int x)
+{
+	asm volatile("stb %0,%1(13)" : :
+		"r" (x),
+		"i" (offsetof(struct paca_struct, perf_counter_pending)));
+}
+
+extern void perf_counter_do_pending(void);
+
+#else
+
+static inline unsigned long get_perf_counter_pending(void)
+{
+	return 0;
+}
+
+static inline void set_perf_counter_pending(int x) {}
+static inline void perf_counter_do_pending(void) {}
+#endif /* CONFIG_PERF_COUNTERS */
+
 #endif	/* __KERNEL__ */
 #endif	/* _ASM_POWERPC_HW_IRQ_H */
arch/powerpc/include/asm/paca.h +1 -0
index 082b3ae..6ef0557 100644
@@ -99,6 +99,7 @@ struct paca_struct {
 	u8 soft_enabled;		/* irq soft-enable flag */
 	u8 hard_enabled;		/* set if irqs are enabled in MSR */
 	u8 io_sync;			/* writel() needs spin_unlock sync */
+	u8 perf_counter_pending;	/* PM interrupt while soft-disabled */
 
 	/* Stuff for accurate time accounting */
 	u64 user_time;			/* accumulated usermode TB ticks */
arch/powerpc/include/asm/perf_counter.h new file +72 -0

72 lines not on this page · raw message

arch/powerpc/include/asm/systbl.h +1 -0
index 72353f6..4c8095f 100644
@@ -322,3 +322,4 @@ SYSCALL_SPU(epoll_create1)
 SYSCALL_SPU(dup3)
 SYSCALL_SPU(pipe2)
 SYSCALL(inotify_init1)
+SYSCALL(perf_counter_open)
arch/powerpc/include/asm/unistd.h +2 -1
index e07d0c7..7cef5af 100644
@@ -341,10 +341,11 @@
 #define __NR_dup3		316
 #define __NR_pipe2		317
 #define __NR_inotify_init1	318
+#define __NR_perf_counter_open	319
 
 #ifdef __KERNEL__
 
-#define __NR_syscalls		319
+#define __NR_syscalls		320
 
 #define __NR__exit __NR_exit
 #define NR_syscalls	__NR_syscalls
arch/powerpc/kernel/Makefile +1 -0
index 8d1a419..7c941ec 100644
@@ -94,6 +94,7 @@ obj-$(CONFIG_AUDIT)		+= audit.o
 obj64-$(CONFIG_AUDIT)		+= compat_audit.o
 
 obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
+obj-$(CONFIG_PERF_COUNTERS)	+= perf_counter.o ppc970-pmu.o power6-pmu.o
 
 obj-$(CONFIG_8XX_MINIMAL_FPEMU) += softemu8xx.o
 
arch/powerpc/kernel/asm-offsets.c +1 -0
index 19ee491..3734973 100644
@@ -131,6 +131,7 @@ int main(void)
 	DEFINE(PACAKMSR, offsetof(struct paca_struct, kernel_msr));
 	DEFINE(PACASOFTIRQEN, offsetof(struct paca_struct, soft_enabled));
 	DEFINE(PACAHARDIRQEN, offsetof(struct paca_struct, hard_enabled));
+	DEFINE(PACAPERFPEND, offsetof(struct paca_struct, perf_counter_pending));
 	DEFINE(PACASLBCACHE, offsetof(struct paca_struct, slb_cache));
 	DEFINE(PACASLBCACHEPTR, offsetof(struct paca_struct, slb_cache_ptr));
 	DEFINE(PACACONTEXTID, offsetof(struct paca_struct, context.id));
arch/powerpc/kernel/entry_64.S +9 -0
index 383ed6e..f30b4e5 100644
@@ -526,6 +526,15 @@ ALT_FW_FTR_SECTION_END_IFCLR(FW_FEATURE_ISERIES)
 2:
 	TRACE_AND_RESTORE_IRQ(r5);
 
+#ifdef CONFIG_PERF_COUNTERS
+	/* check paca->perf_counter_pending if we're enabling ints */
+	lbz	r3,PACAPERFPEND(r13)
+	and.	r3,r3,r5
+	beq	27f
+	bl	.perf_counter_do_pending
+27:
+#endif /* CONFIG_PERF_COUNTERS */
+
 	/* extract EE bit and use it to restore paca->hard_enabled */
 	ld	r3,_MSR(r1)
 	rldicl	r4,r3,49,63		/* r0 = (r3 >> 15) & 1 */
arch/powerpc/kernel/irq.c +11 -1
index 23b8b5e..7f8e6a9 100644
@@ -104,6 +104,13 @@ static inline notrace void set_soft_enabled(unsigned long enable)
 	: : "r" (enable), "i" (offsetof(struct paca_struct, soft_enabled)));
 }
 
+#ifdef CONFIG_PERF_COUNTERS
+notrace void __weak perf_counter_do_pending(void)
+{
+	set_perf_counter_pending(0);
+}
+#endif
+
 notrace void raw_local_irq_restore(unsigned long en)
 {
 	/*
@@ -135,6 +142,9 @@ notrace void raw_local_irq_restore(unsigned long en)
 			iseries_handle_interrupts();
 	}
 
+	if (get_perf_counter_pending())
+		perf_counter_do_pending();
+
 	/*
 	 * if (get_paca()->hard_enabled) return;
 	 * But again we need to take care that gcc gets hard_enabled directly
@@ -231,7 +241,7 @@ void fixup_irqs(cpumask_t map)
 		if (irq_desc[irq].status & IRQ_PER_CPU)
 			continue;
 
-		cpus_and(mask, irq_desc[irq].affinity, map);
+		cpumask_and(&mask, irq_desc[irq].affinity, &map);
 		if (any_online_cpu(mask) == NR_CPUS) {
 			printk("Breaking affinity for irq %i\n", irq);
 			mask = map;
arch/powerpc/kernel/perf_counter.c new file +785 -0

785 lines not on this page · raw message

arch/powerpc/kernel/power6-pmu.c new file +283 -0

283 lines not on this page · raw message

arch/powerpc/kernel/ppc970-pmu.c new file +375 -0

375 lines not on this page · raw message

arch/powerpc/kernel/vmlinux.lds.S +1 -0
index 161b9b9..295ccc5 100644
@@ -184,6 +184,7 @@ SECTIONS
 	. = ALIGN(PAGE_SIZE);
 	.data.percpu  : AT(ADDR(.data.percpu) - LOAD_OFFSET) {
 		__per_cpu_start = .;
+		*(.data.percpu.page_aligned)
 		*(.data.percpu)
 		*(.data.percpu.shared_aligned)
 		__per_cpu_end = .;
arch/powerpc/platforms/Kconfig.cputype +1 -0
index e868b5c..dc0f3c9 100644
@@ -1,6 +1,7 @@
 config PPC64
 	bool "64-bit kernel"
 	default n
+	select HAVE_PERF_COUNTERS
 	help
 	  This option selects whether a 32-bit or a 64-bit kernel
 	  will be built.
arch/powerpc/platforms/pseries/xics.c +3 -2
index 84e058f..80b5134 100644
@@ -153,9 +153,10 @@ static int get_irq_server(unsigned int virq, unsigned int strict_check)
 {
 	int server;
 	/* For the moment only implement delivery to all cpus or one cpu */
-	cpumask_t cpumask = irq_desc[virq].affinity;
+	cpumask_t cpumask;
 	cpumask_t tmp = CPU_MASK_NONE;
 
+	cpumask_copy(&cpumask, irq_desc[virq].affinity);
 	if (!distribute_irqs)
 		return default_server;
 
@@ -869,7 +870,7 @@ void xics_migrate_irqs_away(void)
 		       virq, cpu);
 
 		/* Reset affinity to all cpus */
-		irq_desc[virq].affinity = CPU_MASK_ALL;
+		cpumask_setall(irq_desc[virq].affinity);
 		desc->chip->set_affinity(virq, cpu_all_mask);
 unlock:
 		spin_unlock_irqrestore(&desc->lock, flags);
arch/powerpc/sysdev/mpic.c +2 -1
index a35297d..532e205 100644
@@ -566,9 +566,10 @@ static void __init mpic_scan_ht_pics(struct mpic *mpic)
 #ifdef CONFIG_SMP
 static int irq_choose_cpu(unsigned int virt_irq)
 {
-	cpumask_t mask = irq_desc[virt_irq].affinity;
+	cpumask_t mask;
 	int cpuid;
 
+	cpumask_copy(&mask, irq_desc[virt_irq].affinity);
 	if (cpus_equal(mask, CPU_MASK_ALL)) {
 		static int irq_rover;
 		static DEFINE_SPINLOCK(irq_rover_lock);
arch/sparc/kernel/irq_64.c +3 -2
index cab8e02..4ac5c65 100644
@@ -247,9 +247,10 @@ struct irq_handler_data {
 #ifdef CONFIG_SMP
 static int irq_choose_cpu(unsigned int virt_irq)
 {
-	cpumask_t mask = irq_desc[virt_irq].affinity;
+	cpumask_t mask;
 	int cpuid;
 
+	cpumask_copy(&mask, irq_desc[virt_irq].affinity);
 	if (cpus_equal(mask, CPU_MASK_ALL)) {
 		static int irq_rover;
 		static DEFINE_SPINLOCK(irq_rover_lock);
@@ -854,7 +855,7 @@ void fixup_irqs(void)
 		    !(irq_desc[irq].status & IRQ_PER_CPU)) {
 			if (irq_desc[irq].chip->set_affinity)
 				irq_desc[irq].chip->set_affinity(irq,
-					&irq_desc[irq].affinity);
+					irq_desc[irq].affinity);
 		}
 		spin_unlock_irqrestore(&irq_desc[irq].lock, flags);
 	}
arch/sparc/kernel/time_64.c +1 -1
index 2db3c22..db310aa 100644
@@ -729,7 +729,7 @@ void timer_interrupt(int irq, struct pt_regs *regs)
 
 	irq_enter();
 
-	kstat_this_cpu.irqs[0]++;
+	kstat_incr_irqs_this_cpu(0, irq_to_desc(0));
 
 	if (unlikely(!evt->event_handler)) {
 		printk(KERN_WARNING
arch/x86/Kconfig +1 -0
index 73f7fe8..1f48445 100644
@@ -685,6 +685,7 @@ config X86_UP_IOAPIC
 config X86_LOCAL_APIC
 	def_bool y
 	depends on X86_64 || (X86_32 && (X86_UP_APIC || (SMP && !X86_VOYAGER) || X86_GENERICARCH))
+	select HAVE_PERF_COUNTERS if (!M386 && !M486)
 
 config X86_IO_APIC
 	def_bool y
arch/x86/ia32/ia32entry.S +6 -5
index 256b00b..01e7c4c 100644
@@ -112,8 +112,8 @@ ENTRY(ia32_sysenter_target)
 	CFI_DEF_CFA	rsp,0
 	CFI_REGISTER	rsp,rbp
 	SWAPGS_UNSAFE_STACK
-	movq	%gs:pda_kernelstack, %rsp
-	addq	$(PDA_STACKOFFSET),%rsp	
+	movq	PER_CPU_VAR(kernel_stack), %rsp
+	addq	$(KERNEL_STACK_OFFSET),%rsp
 	/*
 	 * No need to follow this irqs on/off section: the syscall
 	 * disabled irqs, here we enable it straight after entry:
@@ -273,13 +273,13 @@ ENDPROC(ia32_sysenter_target)
 ENTRY(ia32_cstar_target)
 	CFI_STARTPROC32	simple
 	CFI_SIGNAL_FRAME
-	CFI_DEF_CFA	rsp,PDA_STACKOFFSET
+	CFI_DEF_CFA	rsp,KERNEL_STACK_OFFSET
 	CFI_REGISTER	rip,rcx
 	/*CFI_REGISTER	rflags,r11*/
 	SWAPGS_UNSAFE_STACK
 	movl	%esp,%r8d
 	CFI_REGISTER	rsp,r8
-	movq	%gs:pda_kernelstack,%rsp
+	movq	PER_CPU_VAR(kernel_stack),%rsp
 	/*
 	 * No need to follow this irqs on/off section: the syscall
 	 * disabled irqs and here we enable it straight after entry:
@@ -823,7 +823,8 @@ ia32_sys_call_table:
 	.quad compat_sys_signalfd4
 	.quad sys_eventfd2
 	.quad sys_epoll_create1
-	.quad sys_dup3			/* 330 */
+	.quad sys_dup3				/* 330 */
 	.quad sys_pipe2
 	.quad sys_inotify_init1
+	.quad sys_perf_counter_open
 ia32_syscall_end:
arch/x86/include/asm/apicnum.h new file +12 -0
new file mode 100644
index 0000000..82f613c
@@ -0,0 +1,12 @@
+#ifndef _ASM_X86_APICNUM_H
+#define _ASM_X86_APICNUM_H
+
+/* define MAX_IO_APICS */
+#ifdef CONFIG_X86_32
+# define MAX_IO_APICS 64
+#else
+# define MAX_IO_APICS 128
+# define MAX_LOCAL_APIC 32768
+#endif
+
+#endif /* _ASM_X86_APICNUM_H */
arch/x86/include/asm/atomic_32.h +218 -0

223 lines not on this page · raw message

arch/x86/include/asm/bitops.h +10 -4
index e02a359..02b47a6 100644
@@ -3,6 +3,9 @@
 
 /*
  * Copyright 1992, Linus Torvalds.
+ *
+ * Note: inlines with more than a single statement should be marked
+ * __always_inline to avoid problems with older gcc's inlining heuristics.
  */
 
 #ifndef _LINUX_BITOPS_H
@@ -53,7 +56,8 @@
  * Note that @nr may be almost arbitrarily large; this function is not
  * restricted to acting on a single-word quantity.
  */
-static inline void set_bit(unsigned int nr, volatile unsigned long *addr)
+static __always_inline void
+set_bit(unsigned int nr, volatile unsigned long *addr)
 {
 	if (IS_IMMEDIATE(nr)) {
 		asm volatile(LOCK_PREFIX "orb %1,%0"
@@ -90,7 +94,8 @@ static inline void __set_bit(int nr, volatile unsigned long *addr)
  * you should call smp_mb__before_clear_bit() and/or smp_mb__after_clear_bit()
  * in order to ensure changes are visible on other processors.
  */
-static inline void clear_bit(int nr, volatile unsigned long *addr)
+static __always_inline void
+clear_bit(int nr, volatile unsigned long *addr)
 {
 	if (IS_IMMEDIATE(nr)) {
 		asm volatile(LOCK_PREFIX "andb %1,%0"
@@ -204,7 +209,8 @@ static inline int test_and_set_bit(int nr, volatile unsigned long *addr)
  *
  * This is the same as test_and_set_bit on x86.
  */
-static inline int test_and_set_bit_lock(int nr, volatile unsigned long *addr)
+static __always_inline int
+test_and_set_bit_lock(int nr, volatile unsigned long *addr)
 {
 	return test_and_set_bit(nr, addr);
 }
@@ -300,7 +306,7 @@ static inline int test_and_change_bit(int nr, volatile unsigned long *addr)
 	return oldbit;
 }
 
-static inline int constant_test_bit(unsigned int nr, const volatile unsigned long *addr)
+static __always_inline int constant_test_bit(unsigned int nr, const volatile unsigned long *addr)
 {
 	return ((1UL << (nr % BITS_PER_LONG)) &
 		(((unsigned long *)addr)[nr / BITS_PER_LONG])) != 0;
arch/x86/include/asm/cpu.h +21 -0
index bae482d..f03b23e 100644
@@ -7,6 +7,20 @@
 #include <linux/nodemask.h>
 #include <linux/percpu.h>
 
+#ifdef CONFIG_SMP
+
+extern void prefill_possible_map(void);
+
+#else /* CONFIG_SMP */
+
+static inline void prefill_possible_map(void) {}
+
+#define cpu_physical_id(cpu)			boot_cpu_physical_apicid
+#define safe_smp_processor_id()			0
+#define stack_smp_processor_id()		0
+
+#endif /* CONFIG_SMP */
+
 struct x86_cpu {
 	struct cpu cpu;
 };
@@ -17,4 +31,11 @@ extern void arch_unregister_cpu(int);
 #endif
 
 DECLARE_PER_CPU(int, cpu_state);
+
+#ifdef CONFIG_X86_HAS_BOOT_CPU_ID
+extern unsigned char boot_cpu_id;
+#else
+#define boot_cpu_id				0
+#endif
+
 #endif /* _ASM_X86_CPU_H */
arch/x86/include/asm/cpumask.h new file +28 -0
new file mode 100644
index 0000000..26c6dad
@@ -0,0 +1,28 @@
+#ifndef _ASM_X86_CPUMASK_H
+#define _ASM_X86_CPUMASK_H
+#ifndef __ASSEMBLY__
+#include <linux/cpumask.h>
+
+#ifdef CONFIG_X86_64
+
+extern cpumask_var_t cpu_callin_mask;
+extern cpumask_var_t cpu_callout_mask;
+extern cpumask_var_t cpu_initialized_mask;
+extern cpumask_var_t cpu_sibling_setup_mask;
+
+#else /* CONFIG_X86_32 */
+
+extern cpumask_t cpu_callin_map;
+extern cpumask_t cpu_callout_map;
+extern cpumask_t cpu_initialized;
+extern cpumask_t cpu_sibling_setup_map;
+
+#define cpu_callin_mask		((struct cpumask *)&cpu_callin_map)
+#define cpu_callout_mask	((struct cpumask *)&cpu_callout_map)
+#define cpu_initialized_mask	((struct cpumask *)&cpu_initialized)
+#define cpu_sibling_setup_mask	((struct cpumask *)&cpu_sibling_setup_map)
+
+#endif /* CONFIG_X86_32 */
+
+#endif /* __ASSEMBLY__ */
+#endif /* _ASM_X86_CPUMASK_H */
arch/x86/include/asm/current.h +3 -21
index 0930b4f..c68c361 100644
@@ -1,39 +1,21 @@
 #ifndef _ASM_X86_CURRENT_H
 #define _ASM_X86_CURRENT_H
 
-#ifdef CONFIG_X86_32
 #include <linux/compiler.h>
 #include <asm/percpu.h>
 
+#ifndef __ASSEMBLY__
 struct task_struct;
 
 DECLARE_PER_CPU(struct task_struct *, current_task);
-static __always_inline struct task_struct *get_current(void)
-{
-	return x86_read_percpu(current_task);
-}
-
-#else /* X86_32 */
-
-#ifndef __ASSEMBLY__
-#include <asm/pda.h>
-
-struct task_struct;
 
 static __always_inline struct task_struct *get_current(void)
 {
-	return read_pda(pcurrent);
+	return percpu_read(current_task);
 }
 
-#else /* __ASSEMBLY__ */
-
-#include <asm/asm-offsets.h>
-#define GET_CURRENT(reg) movq %gs:(pda_pcurrent),reg
+#define current get_current()
 
 #endif /* __ASSEMBLY__ */
 
-#endif /* X86_32 */
-
-#define current get_current()
-
 #endif /* _ASM_X86_CURRENT_H */
arch/x86/include/asm/hardirq_32.h +4 -0
index cf7954d..7838276 100644
@@ -9,6 +9,7 @@ typedef struct {
 	unsigned long idle_timestamp;
 	unsigned int __nmi_count;	/* arch dependent */
 	unsigned int apic_timer_irqs;	/* arch dependent */
+	unsigned int apic_perf_irqs;	/* arch dependent */
 	unsigned int irq0_irqs;
 	unsigned int irq_resched_count;
 	unsigned int irq_call_count;
@@ -19,6 +20,9 @@ typedef struct {
 
 DECLARE_PER_CPU(irq_cpustat_t, irq_stat);
 
+/* We can have at most NR_VECTORS irqs routed to a cpu at a time */
+#define MAX_HARDIRQS_PER_CPU NR_VECTORS
+
 #define __ARCH_IRQ_STAT
 #define __IRQ_STAT(cpu, member) (per_cpu(irq_stat, cpu).member)
 
arch/x86/include/asm/hardirq_64.h +20 -5
index b5a6b5d..42930b2 100644
@@ -3,22 +3,37 @@
 
 #include <linux/threads.h>
 #include <linux/irq.h>
-#include <asm/pda.h>
 #include <asm/apic.h>
 
+typedef struct {
+	unsigned int __softirq_pending;
+	unsigned int __nmi_count;	/* arch dependent */
+	unsigned int apic_timer_irqs;	/* arch dependent */
+	unsigned int apic_perf_irqs;	/* arch dependent */
+	unsigned int irq0_irqs;
+	unsigned int irq_resched_count;
+	unsigned int irq_call_count;
+	unsigned int irq_tlb_count;
+	unsigned int irq_thermal_count;
+	unsigned int irq_spurious_count;
+	unsigned int irq_threshold_count;
+} ____cacheline_aligned irq_cpustat_t;
+
+DECLARE_PER_CPU(irq_cpustat_t, irq_stat);
+
 /* We can have at most NR_VECTORS irqs routed to a cpu at a time */
 #define MAX_HARDIRQS_PER_CPU NR_VECTORS
 
 #define __ARCH_IRQ_STAT 1
 
-#define inc_irq_stat(member)	add_pda(member, 1)
+#define inc_irq_stat(member)	percpu_add(irq_stat.member, 1)
 
-#define local_softirq_pending() read_pda(__softirq_pending)
+#define local_softirq_pending() percpu_read(irq_stat.__softirq_pending)
 
 #define __ARCH_SET_SOFTIRQ_PENDING 1
 
-#define set_softirq_pending(x) write_pda(__softirq_pending, (x))
-#define or_softirq_pending(x)  or_pda(__softirq_pending, (x))
+#define set_softirq_pending(x) percpu_write(irq_stat.__softirq_pending, (x))
+#define or_softirq_pending(x)  percpu_or(irq_stat.__softirq_pending, (x))
 
 extern void ack_bad_irq(unsigned int irq);
 
arch/x86/include/asm/hw_irq.h +2 -0
index 8de644b..aa93e53 100644
@@ -30,6 +30,8 @@
 /* Interrupt handlers registered during init_IRQ */
 extern void apic_timer_interrupt(void);
 extern void error_interrupt(void);
+extern void perf_counter_interrupt(void);
+
 extern void spurious_interrupt(void);
 extern void thermal_interrupt(void);
 extern void reschedule_interrupt(void);
arch/x86/include/asm/intel_arch_perfmon.h deleted +0 -31
deleted file mode 100644
index fa0fd06..0000000
@@ -1,31 +0,0 @@
-#ifndef _ASM_X86_INTEL_ARCH_PERFMON_H
-#define _ASM_X86_INTEL_ARCH_PERFMON_H
-
-#define MSR_ARCH_PERFMON_PERFCTR0		0xc1
-#define MSR_ARCH_PERFMON_PERFCTR1		0xc2
-
-#define MSR_ARCH_PERFMON_EVENTSEL0		0x186
-#define MSR_ARCH_PERFMON_EVENTSEL1		0x187
-
-#define ARCH_PERFMON_EVENTSEL0_ENABLE	(1 << 22)
-#define ARCH_PERFMON_EVENTSEL_INT	(1 << 20)
-#define ARCH_PERFMON_EVENTSEL_OS	(1 << 17)
-#define ARCH_PERFMON_EVENTSEL_USR	(1 << 16)
-
-#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_SEL	(0x3c)
-#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_UMASK	(0x00 << 8)
-#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_INDEX (0)
-#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_PRESENT \
-	(1 << (ARCH_PERFMON_UNHALTED_CORE_CYCLES_INDEX))
-
-union cpuid10_eax {
-	struct {
-		unsigned int version_id:8;
-		unsigned int num_counters:8;
-		unsigned int bit_width:8;
-		unsigned int mask_length:8;
-	} split;
-	unsigned int full;
-};
-
-#endif /* _ASM_X86_INTEL_ARCH_PERFMON_H */
arch/x86/include/asm/io_apic.h +2 -24
index 7a1f44a..08ec793 100644
@@ -114,38 +114,16 @@ struct IR_IO_APIC_route_entry {
 extern int nr_ioapics;
 extern int nr_ioapic_registers[MAX_IO_APICS];
 
-/*
- * MP-BIOS irq configuration table structures:
- */
-
 #define MP_MAX_IOAPIC_PIN 127
 
-struct mp_config_ioapic {
-	unsigned long mp_apicaddr;
-	unsigned int mp_apicid;
-	unsigned char mp_type;
-	unsigned char mp_apicver;
-	unsigned char mp_flags;
-};
-
-struct mp_config_intsrc {
-	unsigned int mp_dstapic;
-	unsigned char mp_type;
-	unsigned char mp_irqtype;
-	unsigned short mp_irqflag;
-	unsigned char mp_srcbus;
-	unsigned char mp_srcbusirq;
-	unsigned char mp_dstirq;
-};
-
 /* I/O APIC entries */
-extern struct mp_config_ioapic mp_ioapics[MAX_IO_APICS];
+extern struct mpc_ioapic mp_ioapics[MAX_IO_APICS];
 
 /* # of MP IRQ source entries */
 extern int mp_irq_entries;
 
 /* MP IRQ source entries */
-extern struct mp_config_intsrc mp_irqs[MAX_IRQ_SOURCES];
+extern struct mpc_intsrc mp_irqs[MAX_IRQ_SOURCES];
 
 /* non-0 if default (table-less) MP configuration */
 extern int mpc_default_type;
arch/x86/include/asm/irq_regs_32.h +2 -2
index 86afd74..d7ed33e 100644
@@ -15,7 +15,7 @@ DECLARE_PER_CPU(struct pt_regs *, irq_regs);
 
 static inline struct pt_regs *get_irq_regs(void)
 {
-	return x86_read_percpu(irq_regs);
+	return percpu_read(irq_regs);
 }
 
 static inline struct pt_regs *set_irq_regs(struct pt_regs *new_regs)
@@ -23,7 +23,7 @@ static inline struct pt_regs *set_irq_regs(struct pt_regs *new_regs)
 	struct pt_regs *old_regs;
 
 	old_regs = get_irq_regs();
-	x86_write_percpu(irq_regs, new_regs);
+	percpu_write(irq_regs, new_regs);
 
 	return old_regs;
 }
arch/x86/include/asm/irq_vectors.h +13 -5
index f7ff650..1554d02 100644
@@ -87,6 +87,11 @@
 #define LOCAL_TIMER_VECTOR	0xef
 
 /*
+ * Performance monitoring interrupt vector:
+ */
+#define LOCAL_PERF_VECTOR	0xee
+
+/*
  * First APIC vector available to drivers: (vectors 0x30-0xee) we
  * start at 0x31(0x41) to spread out vectors evenly between priority
  * levels. (0x80 is the syscall vector)
@@ -105,6 +110,8 @@
 
 #if defined(CONFIG_X86_IO_APIC) && !defined(CONFIG_X86_VOYAGER)
 
+#include <asm/apicnum.h>	/* need MAX_IO_APICS */
+
 #ifndef CONFIG_SPARSE_IRQ
 # if NR_CPUS < MAX_IO_APICS
 #  define NR_IRQS (NR_VECTORS + (32 * NR_CPUS))
@@ -112,11 +119,12 @@
 #  define NR_IRQS (NR_VECTORS + (32 * MAX_IO_APICS))
 # endif
 #else
-# if (8 * NR_CPUS) > (32 * MAX_IO_APICS)
-#  define NR_IRQS (NR_VECTORS + (8 * NR_CPUS))
-# else
-#  define NR_IRQS (NR_VECTORS + (32 * MAX_IO_APICS))
-# endif
+
+# define NR_IRQS					\
+	((8 * NR_CPUS) > (32 * MAX_IO_APICS) ?		\
+		(NR_VECTORS + (8 * NR_CPUS)) :		\
+		(NR_VECTORS + (32 * MAX_IO_APICS)))	\
+
 #endif
 
 #elif defined(CONFIG_X86_VOYAGER)
arch/x86/include/asm/mach-default/entry_arch.h +5 -0
index 6b1add8..ad31e5d 100644
@@ -25,10 +25,15 @@ BUILD_INTERRUPT(irq_move_cleanup_interrupt,IRQ_MOVE_CLEANUP_VECTOR)
  * a much simpler SMP time architecture:
  */
 #ifdef CONFIG_X86_LOCAL_APIC
+
 BUILD_INTERRUPT(apic_timer_interrupt,LOCAL_TIMER_VECTOR)
 BUILD_INTERRUPT(error_interrupt,ERROR_APIC_VECTOR)
 BUILD_INTERRUPT(spurious_interrupt,SPURIOUS_APIC_VECTOR)
 
+#ifdef CONFIG_PERF_COUNTERS
+BUILD_INTERRUPT(perf_counter_interrupt, LOCAL_PERF_VECTOR)
+#endif
+
 #ifdef CONFIG_X86_MCE_P4THERMAL
 BUILD_INTERRUPT(thermal_interrupt,THERMAL_APIC_VECTOR)
 #endif
arch/x86/include/asm/mmu_context_32.h +6 -6
index 7e98ce1..08b5345 100644
@@ -4,8 +4,8 @@
 static inline void enter_lazy_tlb(struct mm_struct *mm, struct task_struct *tsk)
 {
 #ifdef CONFIG_SMP
-	if (x86_read_percpu(cpu_tlbstate.state) == TLBSTATE_OK)
-		x86_write_percpu(cpu_tlbstate.state, TLBSTATE_LAZY);
+	if (percpu_read(cpu_tlbstate.state) == TLBSTATE_OK)
+		percpu_write(cpu_tlbstate.state, TLBSTATE_LAZY);
 #endif
 }
 
@@ -19,8 +19,8 @@ static inline void switch_mm(struct mm_struct *prev,
 		/* stop flush ipis for the previous mm */
 		cpu_clear(cpu, prev->cpu_vm_mask);
 #ifdef CONFIG_SMP
-		x86_write_percpu(cpu_tlbstate.state, TLBSTATE_OK);
-		x86_write_percpu(cpu_tlbstate.active_mm, next);
+		percpu_write(cpu_tlbstate.state, TLBSTATE_OK);
+		percpu_write(cpu_tlbstate.active_mm, next);
 #endif
 		cpu_set(cpu, next->cpu_vm_mask);
 
@@ -35,8 +35,8 @@ static inline void switch_mm(struct mm_struct *prev,
 	}
 #ifdef CONFIG_SMP
 	else {
-		x86_write_percpu(cpu_tlbstate.state, TLBSTATE_OK);
-		BUG_ON(x86_read_percpu(cpu_tlbstate.active_mm) != next);
+		percpu_write(cpu_tlbstate.state, TLBSTATE_OK);
+		BUG_ON(percpu_read(cpu_tlbstate.active_mm) != next);
 
 		if (!cpu_test_and_set(cpu, next->cpu_vm_mask)) {
 			/* We were in lazy tlb mode and leave_mm disabled
arch/x86/include/asm/mmu_context_64.h +7 -9
index 677d36e..c457250 100644
@@ -1,13 +1,11 @@
 #ifndef _ASM_X86_MMU_CONTEXT_64_H
 #define _ASM_X86_MMU_CONTEXT_64_H
 
-#include <asm/pda.h>
-
 static inline void enter_lazy_tlb(struct mm_struct *mm, struct task_struct *tsk)
 {
 #ifdef CONFIG_SMP
-	if (read_pda(mmu_state) == TLBSTATE_OK)
-		write_pda(mmu_state, TLBSTATE_LAZY);
+	if (percpu_read(cpu_tlbstate.state) == TLBSTATE_OK)
+		percpu_write(cpu_tlbstate.state, TLBSTATE_LAZY);
 #endif
 }
 
@@ -19,8 +17,8 @@ static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next,
 		/* stop flush ipis for the previous mm */
 		cpu_clear(cpu, prev->cpu_vm_mask);
 #ifdef CONFIG_SMP
-		write_pda(mmu_state, TLBSTATE_OK);
-		write_pda(active_mm, next);
+		percpu_write(cpu_tlbstate.state, TLBSTATE_OK);
+		percpu_write(cpu_tlbstate.active_mm, next);
 #endif
 		cpu_set(cpu, next->cpu_vm_mask);
 		load_cr3(next->pgd);
@@ -30,9 +28,9 @@ static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next,
 	}
 #ifdef CONFIG_SMP
 	else {
-		write_pda(mmu_state, TLBSTATE_OK);
-		if (read_pda(active_mm) != next)
-			BUG();
+		percpu_write(cpu_tlbstate.state, TLBSTATE_OK);
+		BUG_ON(percpu_read(cpu_tlbstate.active_mm) != next);
+
 		if (!cpu_test_and_set(cpu, next->cpu_vm_mask)) {
 			/* We were in lazy tlb mode and leave_mm disabled
 			 * tlb flush IPI delivery. We must reload CR3
arch/x86/include/asm/mpspec_def.h +12 -11
index 59568bc..4a7f96d 100644
@@ -24,17 +24,18 @@
 # endif
 #endif
 
-struct intel_mp_floating {
-	char mpf_signature[4];		/* "_MP_"			*/
-	unsigned int mpf_physptr;	/* Configuration table address	*/
-	unsigned char mpf_length;	/* Our length (paragraphs)	*/
-	unsigned char mpf_specification;/* Specification version	*/
-	unsigned char mpf_checksum;	/* Checksum (makes sum 0)	*/
-	unsigned char mpf_feature1;	/* Standard or configuration ?	*/
-	unsigned char mpf_feature2;	/* Bit7 set for IMCR|PIC	*/
-	unsigned char mpf_feature3;	/* Unused (0)			*/
-	unsigned char mpf_feature4;	/* Unused (0)			*/
-	unsigned char mpf_feature5;	/* Unused (0)			*/
+/* Intel MP Floating Pointer Structure */
+struct mpf_intel {
+	char signature[4];		/* "_MP_"			*/
+	unsigned int physptr;		/* Configuration table address	*/
+	unsigned char length;		/* Our length (paragraphs)	*/
+	unsigned char specification;	/* Specification version	*/
+	unsigned char checksum;		/* Checksum (makes sum 0)	*/
+	unsigned char feature1;		/* Standard or configuration ?	*/
+	unsigned char feature2;		/* Bit7 set for IMCR|PIC	*/
+	unsigned char feature3;		/* Unused (0)			*/
+	unsigned char feature4;		/* Unused (0)			*/
+	unsigned char feature5;		/* Unused (0)			*/
 };
 
 #define MPC_SIGNATURE "PCMP"
arch/x86/include/asm/page_64.h +2 -2
index 5ebca29..e27fdbe 100644
@@ -13,8 +13,8 @@
 #define DEBUG_STACK_ORDER (EXCEPTION_STACK_ORDER + 1)
 #define DEBUG_STKSZ (PAGE_SIZE << DEBUG_STACK_ORDER)
 
-#define IRQSTACK_ORDER 2
-#define IRQSTACKSIZE (PAGE_SIZE << IRQSTACK_ORDER)
+#define IRQ_STACK_ORDER 2
+#define IRQ_STACK_SIZE (PAGE_SIZE << IRQ_STACK_ORDER)
 
 #define STACKFAULT_STACK 1
 #define DOUBLEFAULT_STACK 2
arch/x86/include/asm/paravirt.h +5 -3
index ba3e2ff..c26c6bf 100644
@@ -244,7 +244,8 @@ struct pv_mmu_ops {
 	void (*flush_tlb_user)(void);
 	void (*flush_tlb_kernel)(void);
 	void (*flush_tlb_single)(unsigned long addr);
-	void (*flush_tlb_others)(const cpumask_t *cpus, struct mm_struct *mm,
+	void (*flush_tlb_others)(const struct cpumask *cpus,
+				 struct mm_struct *mm,
 				 unsigned long va);
 
 	/* Hooks for allocating and freeing a pagetable top-level */
@@ -984,10 +985,11 @@ static inline void __flush_tlb_single(unsigned long addr)
 	PVOP_VCALL1(pv_mmu_ops.flush_tlb_single, addr);
 }
 
-static inline void flush_tlb_others(cpumask_t cpumask, struct mm_struct *mm,
+static inline void flush_tlb_others(const struct cpumask *cpumask,
+				    struct mm_struct *mm,
 				    unsigned long va)
 {
-	PVOP_VCALL3(pv_mmu_ops.flush_tlb_others, &cpumask, mm, va);
+	PVOP_VCALL3(pv_mmu_ops.flush_tlb_others, cpumask, mm, va);
 }
 
 static inline int paravirt_pgd_alloc(struct mm_struct *mm)
arch/x86/include/asm/pda.h +16 -108

79 lines not on this page · raw message

arch/x86/include/asm/percpu.h +65 -94

159 lines not on this page · raw message

arch/x86/include/asm/perf_counter.h new file +95 -0

95 lines not on this page · raw message

arch/x86/include/asm/processor.h +3 -0
index 091cd88..f511246 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/setup.h +0 -1
index ebe858c..5369497 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/smp.h +3 -46

49 lines not on this page · raw message

arch/x86/include/asm/system.h +2 -2
index 8e626ea..d1dc27d 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/thread_info.h +11 -13
index 9878964..f384889 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/tlbflush.h +8 -9
index 0e7bbb5..d3539f9 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/topology.h +3 -5
index 4e2f2e0..ffea1fe 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/trampoline.h +1 -0
index 780ba0a..90f06c2 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/unistd_32.h +1 -0
index f2bba78..7e47658 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/unistd_64.h +2 -1
index d2e415e..53025fe 100644

Lines of this file are not shown. View the raw message.

arch/x86/include/asm/uv/uv_bau.h +2 -1
index 50423c7..74e6393 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/acpi/boot.c +47 -49

96 lines not on this page · raw message

arch/x86/kernel/acpi/sleep.c +1 -0
index 707c1f6..4abff45 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/apic.c +14 -12
index 0f830e4..e9af14f 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/asm-offsets_64.c +1 -7
index 1d41d3f..64c834a 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/cpu/Makefile +7 -5
index 82db7f4..c381330 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/cpu/common.c +32 -50

82 lines not on this page · raw message

arch/x86/kernel/cpu/cpufreq/acpi-cpufreq.c +0 -2
index 6f11e02..8f3c95c 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/cpu/intel_cacheinfo.c +44 -19

63 lines not on this page · raw message

arch/x86/kernel/cpu/mcheck/mce_amd_64.c +14 -7
index 8ae8c4f..4772e91 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/cpu/perf_counter.c new file +695 -0

695 lines not on this page · raw message

arch/x86/kernel/cpu/perfctr-watchdog.c +1 -1
index 9abd48b..d6f5b9f 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/crash.c +1 -1
index c689d19..11b93ca 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/dumpstack_64.c +18 -17
index c302d07..d35db59 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/entry_64.S +26 -20

46 lines not on this page · raw message

arch/x86/kernel/head64.c +1 -22
index b9a4d8c..af67d32 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/head_64.S +41 -6

47 lines not on this page · raw message

arch/x86/kernel/io_apic.c +92 -71

163 lines not on this page · raw message

arch/x86/kernel/irq.c +6 -5
index 3973e2d..a6bca1d 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/irq_32.c +1 -1
index 74b9ff7..e0f29be 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/irq_64.c +4 -1
index 63c88e6..1db0524 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/irqinit_32.c +3 -0
index 1507ad4..0bef628 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/irqinit_64.c +5 -0
index da481a1..6a71bfc 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/microcode_intel.c +5 -5
index b7f4c92..5e9f4fc 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/module_32.c +3 -3
index 3db0a54..0edd819 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/module_64.c +16 -16
index 6ba8783..c23880b 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/mpparse.c +71 -71

142 lines not on this page · raw message

arch/x86/kernel/msr.c +1 -1
index 7262666..3cf3413 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/nmi.c +1 -9
index 7228979..23b6d9e 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/process_32.c +1 -4
index a546f55..2c00a57 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/process_64.c +14 -8
index 416fb92..4523ff8 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/reboot.c +1 -0
index 2b46eb4..f8536fe 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/setup.c +1 -1
index ae0d804..f41c448 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/setup_percpu.c +118 -90

208 lines not on this page · raw message

arch/x86/kernel/signal.c +6 -1
index 89bb766..4fa5243 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/smpboot.c +7 -62

69 lines not on this page · raw message

arch/x86/kernel/smpcommon.c +6 -4
index 397e309..add36b4 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/syscall_table_32.S +1 -0
index e2e86a0..0c4d601 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/tlb_32.c +34 -51

85 lines not on this page · raw message

arch/x86/kernel/tlb_64.c +43 -33

76 lines not on this page · raw message

arch/x86/kernel/tlb_uv.c +8 -8
index f885023..690dcf1 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/vmlinux_32.lds.S +1 -8
index 82c6755..3eba7f7 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/vmlinux_64.lds.S +20 -2
index 1a614c0..a09abb8 100644

Lines of this file are not shown. View the raw message.

arch/x86/kernel/x8664_ksyms_64.c +0 -2
index 695e426..3909e3b 100644

Lines of this file are not shown. View the raw message.

arch/x86/mach-voyager/setup.c +1 -0
index a580b95..0ade625 100644

Lines of this file are not shown. View the raw message.

arch/x86/mach-voyager/voyager_smp.c +4 -2
index 9840b7e..96f15b0 100644

Lines of this file are not shown. View the raw message.

arch/x86/mm/init_32.c +0 -1
index 88f1b10..4a6989e 100644

Lines of this file are not shown. View the raw message.

arch/x86/mm/pat.c +27 -10
index 8b08fb9..c948851 100644

Lines of this file are not shown. View the raw message.

arch/x86/oprofile/op_model_ppro.c +1 -1
index e9f80c7..07c9145 100644

Lines of this file are not shown. View the raw message.

arch/x86/xen/enlighten.c +20 -27

47 lines not on this page · raw message

arch/x86/xen/irq.c +4 -4
index bb04260..2e82714 100644

Lines of this file are not shown. View the raw message.

arch/x86/xen/mmu.c +2 -6
index 503c240..98cb986 100644

Lines of this file are not shown. View the raw message.

arch/x86/xen/multicalls.h +1 -1
index 8589382..e786fa7 100644

Lines of this file are not shown. View the raw message.

arch/x86/xen/smp.c +5 -28
index c44e206..72c2eb9 100644

Lines of this file are not shown. View the raw message.

arch/x86/xen/xen-asm_64.S +15 -16
index 05794c5..d6fc51f 100644

Lines of this file are not shown. View the raw message.

drivers/acpi/processor_idle.c +8 -0
index 66a9d81..7acb23f 100644

Lines of this file are not shown. View the raw message.

drivers/base/cpu.c +1 -1
index 719ee5c..5b257a5 100644

Lines of this file are not shown. View the raw message.

drivers/base/topology.c +16 -17
index a778fb5..bf6b132 100644

Lines of this file are not shown. View the raw message.

drivers/char/sysrq.c +2 -0
index 33a9351..fa71b84 100644

Lines of this file are not shown. View the raw message.

drivers/firmware/dcdbas.c +8 -4
index 777fba4..3009e01 100644

Lines of this file are not shown. View the raw message.

drivers/misc/sgi-xp/xpc_main.c +1 -1
index 89218f7..6576170 100644

Lines of this file are not shown. View the raw message.

drivers/net/sfc/efx.c +12 -5
index 7673fd9..101c00a 100644

Lines of this file are not shown. View the raw message.

drivers/oprofile/buffer_sync.c +18 -4
index 9da5a4b..c3ea5fa 100644

Lines of this file are not shown. View the raw message.

drivers/oprofile/buffer_sync.h +4 -0
index 3110732..0ebf5db 100644

Lines of this file are not shown. View the raw message.

drivers/oprofile/oprof.c +8 -1
index 3cffce9..ced39f6 100644

Lines of this file are not shown. View the raw message.

drivers/pci/intr_remapping.c +1 -0
index f78371b..5a57753 100644

Lines of this file are not shown. View the raw message.

drivers/xen/events.c +19 -7
index eb0dfde..3141e14 100644

Lines of this file are not shown. View the raw message.

drivers/xen/manage.c +1 -1
index 9b91617..e7e83b6 100644

Lines of this file are not shown. View the raw message.

fs/exec.c +8 -0
index 0dd60a0..97f9c22 100644

Lines of this file are not shown. View the raw message.

include/asm-generic/bitops/__ffs.h +1 -1
index 9a3274a..937d7c4 100644

Lines of this file are not shown. View the raw message.

include/asm-generic/bitops/__fls.h +1 -1
index be24465..a60a7cc 100644

Lines of this file are not shown. View the raw message.

include/asm-generic/bitops/fls.h +1 -1
index 850859b..0576d1f 100644

Lines of this file are not shown. View the raw message.

include/asm-generic/bitops/fls64.h +2 -2
index 86d403f..b097cf8 100644

Lines of this file are not shown. View the raw message.

include/asm-generic/percpu.h +52 -0

52 lines not on this page · raw message

include/asm-generic/sections.h +1 -1
index 79a7ff9..4ce48e8 100644

Lines of this file are not shown. View the raw message.

include/asm-generic/vmlinux.lds.h +69 -6

75 lines not on this page · raw message

include/linux/init_task.h +11 -0
index 2f3c2d4..49a40fb 100644

Lines of this file are not shown. View the raw message.

include/linux/interrupt.h +1 -0
index 9127f6b..472f117 100644

Lines of this file are not shown. View the raw message.

include/linux/irq.h +83 -3

86 lines not on this page · raw message

include/linux/irqnr.h +1 -0
index 86af92e..887477b 100644

Lines of this file are not shown. View the raw message.

include/linux/kernel_stat.h +8 -0
index 570d204..ecfa668 100644

Lines of this file are not shown. View the raw message.

include/linux/perf_counter.h new file +290 -0

290 lines not on this page · raw message

include/linux/prctl.h +3 -0
index 48d887e..b00df4c 100644

Lines of this file are not shown. View the raw message.

include/linux/sched.h +11 -1
index 4cae9b8..f134a0f 100644

Lines of this file are not shown. View the raw message.

include/linux/syscalls.h +8 -0
index 16875f8..fc81937 100644

Lines of this file are not shown. View the raw message.

include/linux/topology.h +6 -0
index e632d29..a16b9e0 100644

Lines of this file are not shown. View the raw message.

init/Kconfig +30 -0
index 2af8382..6af96b7 100644

Lines of this file are not shown. View the raw message.

kernel/Makefile +1 -0
index 170a921..5537554 100644

Lines of this file are not shown. View the raw message.

kernel/exit.c +9 -4
index f80dec3..29f4b79 100644

Lines of this file are not shown. View the raw message.

kernel/fork.c +1 -0
index bf0cef8..70ca185 100644

Lines of this file are not shown. View the raw message.

kernel/irq/chip.c +4 -1
index f63c706..c248eba 100644

Lines of this file are not shown. View the raw message.

kernel/irq/handle.c +35 -22

57 lines not on this page · raw message

kernel/irq/internals.h +7 -0
index e6d0a43..40416a8 100644

Lines of this file are not shown. View the raw message.

kernel/irq/manage.c +6 -6
index cd0cd8d..b98739a 100644

Lines of this file are not shown. View the raw message.

kernel/irq/migration.c +6 -6
index bd72329..e05ad9b 100644

Lines of this file are not shown. View the raw message.

kernel/irq/numa_migrate.c +16 -3
index ecf765c..666260e 100644

Lines of this file are not shown. View the raw message.

kernel/irq/proc.c +2 -2
index aae3f74..692363d 100644

Lines of this file are not shown. View the raw message.

kernel/perf_counter.c new file +2169 -0

2,169 lines not on this page · raw message

kernel/sched.c +73 -3

76 lines not on this page · raw message

kernel/sched_rt.c +22 -14
index 954e1a8..da932f4 100644

Lines of this file are not shown. View the raw message.

kernel/softirq.c +5 -0
index bdbe9de..0365b48 100644

Lines of this file are not shown. View the raw message.

kernel/sys.c +7 -0
index e7dc0e1..87ca037 100644

Lines of this file are not shown. View the raw message.

kernel/sys_ni.c +3 -0
index 27dad29..68320f6 100644

Lines of this file are not shown. View the raw message.

lib/smp_processor_id.c +1 -1
index 0f8fc22..4689cb0 100644

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Part of the diff is not shown here. View the raw message.

#1 lore raw
Quoted 23 lines · Ingo Molnar

We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:

   http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patch

It's also available in tip/master at:

http://people.redhat.com/mingo/tip.git/README

There are many changes in the v6 release:

- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.

- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc instrumentation and PAPI support.

- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.

Quoted 7 lines

- The 'perfstat' utility (ex 'timec') has been updated:

http://people.redhat.com/mingo/perfcounters/perfstat.c

- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:

http://people.redhat.com/mingo/perfcounters/kerneltop.c

BTW, this kerneltop has nothing to do with that other one??

http://www.xenotime.net/linux/kerneltop/

Thanks,

Signature · 1 line
~Randy
#2 lore raw
Quoted 28 lines · Randy Dunlap
Quoted 27 lines · Ingo Molnar

We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:

   http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patch

It's also available in tip/master at:

http://people.redhat.com/mingo/tip.git/README

There are many changes in the v6 release:

- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.

- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc instrumentation and PAPI support.

- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.

- The 'perfstat' utility (ex 'timec') has been updated:

http://people.redhat.com/mingo/perfcounters/perfstat.c

Quoted 7 lines
Quoted 3 lines

- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:

http://people.redhat.com/mingo/perfcounters/kerneltop.c

BTW, this kerneltop has nothing to do with that other one??

http://www.xenotime.net/linux/kerneltop/

heh, didnt know about that one - there's no connection other than the name :-) The project seems somewhat stale but indeed similar in purpose. Can rename to kerneltop2 i guess.

Ingo

#3 lore raw
Quoted 32 lines · Ingo Molnar
Quoted 30 lines · Randy Dunlap
Quoted 29 lines · Ingo Molnar

We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:

   http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patch

It's also available in tip/master at:

http://people.redhat.com/mingo/tip.git/README

There are many changes in the v6 release:

- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.

- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc instrumentation and PAPI support.

- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.

- The 'perfstat' utility (ex 'timec') has been updated:

http://people.redhat.com/mingo/perfcounters/perfstat.c

- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

http://people.redhat.com/mingo/perfcounters/kerneltop.c

BTW, this kerneltop has nothing to do with that other one??

http://www.xenotime.net/linux/kerneltop/

heh, didnt know about that one - there's no connection other than the name :-) The project seems somewhat stale but indeed similar in purpose. Can rename to kerneltop2 i guess.

Yes, it's stale. I plan to update it sometime this year. :)

I don't care if you rename it or not.

Signature · 1 line
~Randy
#6 lore raw
In reply to Karel Zak
Quoted 7 lines · Karel Zak
Quoted 3 lines · Ingo Molnar

- The 'perfstat' utility (ex 'timec') has been updated:

http://people.redhat.com/mingo/perfcounters/perfstat.c

what are your planning to do with this utility? We can merge it into util-linux-ng.

That would be nice to do, if/once/when the subsystem and the syscall itself is merged upstream. The syscall ABI might change until that happens.

Ingo

#7 lore raw
In reply to Ingo Molnar
Quoted 26 lines · Ingo Molnar

We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:

   http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patch

It's also available in tip/master at:

http://people.redhat.com/mingo/tip.git/README

There are many changes in the v6 release:

- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.

- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc instrumentation and PAPI support.

- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.

- The 'perfstat' utility (ex 'timec') has been updated:

http://people.redhat.com/mingo/perfcounters/perfstat.c

- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:

Quoted 7 lines

http://people.redhat.com/mingo/perfcounters/kerneltop.c

- Merged to latest mainline

- Various fixes and other updates

Ingo

Hi Ingo,

Looking over the latest capabilities of this proposal, I am wondering how it can accommodate performance monitor units which have extra registers which require user-defined data to be loaded into them.

For example, on the Power architecture, there is an Instruction Matching Register which allows the counting of particular instructions. Currently, this is unsupported in perfmon2/3, but we have plans to add it, and it's pretty straight-forward to imagine how this would be done in perfmon.

But I don't see an obvious way to do it with your proposal. Do you have any ideas how Performance Counters for Linux could accommodate this sort of PMU functionality?

One thought would be to change the event code to an event descriptor structure, which has room for lots of bits, including arch-defined bits (in the case of Power, an IMR value, and others). This might also be a way to accommodate unit masks (and enums) as well, which Andi Kleen pointed out as an issue in an earlier LKML posting.

Regards,

- Corey

Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]

#8 lore raw

Hi,

Corey brings up an interesting problem which I wanted to comment on.

The current proposal hinges on the idea that by interpreting a single value the kernel
can understand what the user wants to measure. For instance, if I pass type=0, then
the kernel understands I want to measure CPU_CYCLES. Given that the number of events and their unit mask combinations can be large, the proposal also provides a "raw" mode, where the content of the type field is interpreted as the raw value to
put into a register.

This is where there is an issue because with several PMU models, including on X86, using
the raw bit + 64 value is not enough to figure out what the user wants to measure. This happens
when the PMU has more than counters. Thus, interpreting each raw value has the event code
may be wrong. To remain on familiar territory, the Nehalem uncore PMU has an opcode matcher register,
that uses a 64-bit value. On AMD64 Family 10h, you have IBS. But I could give examples on
Itanium with opcode matchers, range restrictions. Corey provided other examples for Power.
The API has to provide a way to express what the raw value is meant
for: counter, matcher, filter...

There are PMU where programming an event requires writing two config registers. This is the case
for all Netburst-based processors where you have to program CCCR and ESCR. I wonder how,
raw mode is supported for those processors. What if a PMU requires three registers to be programmed?

Quoted 72 lines · Corey Ashford
Quoted 29 lines · Ingo Molnar

We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:

http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patch

It's also available in tip/master at:

http://people.redhat.com/mingo/tip.git/README

There are many changes in the v6 release:

- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.

- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc
instrumentation and PAPI support.

- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.

- The 'perfstat' utility (ex 'timec') has been updated:

http://people.redhat.com/mingo/perfcounters/perfstat.c

Quoted 8 lines

- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:
http://people.redhat.com/mingo/perfcounters/kerneltop.c

- Merged to latest mainline

- Various fixes and other updates

Ingo

Hi Ingo,

Looking over the latest capabilities of this proposal, I am wondering how it can accommodate performance monitor units which have extra registers which require user-defined data to be loaded into them.

For example, on the Power architecture, there is an Instruction Matching Register which allows the counting of particular instructions. Currently, this is unsupported in perfmon2/3, but we have plans to add it, and it's pretty straight-forward to imagine how this would be done in perfmon.

But I don't see an obvious way to do it with your proposal. Do you have any ideas how Performance Counters for Linux could accommodate this sort of PMU functionality?

One thought would be to change the event code to an event descriptor structure, which has room for lots of bits, including arch-defined bits (in the case of Power, an IMR value, and others). This might also be a way to accommodate unit masks (and enums) as well, which Andi Kleen pointed out as an issue in an earlier LKML posting.

Regards,

- Corey

Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]

#9 lore raw
Quoted 12 lines · stephane eranian

Hi,

Corey brings up an interesting problem which I wanted to comment on.

The current proposal hinges on the idea that by interpreting a single value the kernel can understand what the user wants to measure. For instance, if I pass type=0, then the kernel understands I want to measure CPU_CYCLES. Given that the number of events and their unit mask combinations can be large, the proposal also provides a "raw" mode, where the content of the type field is interpreted as the raw value to put into a register.

Quoted 10 lines

This is where there is an issue because with several PMU models, including on X86, using the raw bit + 64 value is not enough to figure out what the user wants to measure. This happens when the PMU has more than counters. Thus, interpreting each raw value has the event code may be wrong. To remain on familiar territory, the Nehalem uncore PMU has an opcode matcher register, that uses a 64-bit value. On AMD64 Family 10h, you have IBS. But I could give examples on Itanium with opcode matchers, range restrictions. Corey provided other examples for Power. The API has to provide a way to express what the raw value is meant for: counter, matcher, filter...

this can be done in a number of ways (in order of increasing levels of abstraction):

- the raw type is kept wide enough. Paul already requested the raw type to be widened to 128 bits to express certain PowerPC features.

- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the write() method or ioctl is extended to express attributes we want to set/change while a counter is running.

- or the highest level counter / hw event data type is extended with new attribute field(s).

My feeling is that we generally want such hw features to start small - i.e. at the raw type level initially. Then we can allow them to climb the ladder, if they prove their utility in practice. We've got space reserved in the ABI to allow for growth like this.

Ingo

#10 lore raw
Quoted 26 lines · Ingo Molnar
Quoted 12 lines · stephane eranian

Hi,

Corey brings up an interesting problem which I wanted to comment on.

The current proposal hinges on the idea that by interpreting a single value the kernel can understand what the user wants to measure. For instance, if I pass type=0, then the kernel understands I want to measure CPU_CYCLES. Given that the number of events and their unit mask combinations can be large, the proposal also provides a "raw" mode, where the content of the type field is interpreted as the raw value to put into a register.

Quoted 10 lines

This is where there is an issue because with several PMU models, including on X86, using the raw bit + 64 value is not enough to figure out what the user wants to measure. This happens when the PMU has more than counters. Thus, interpreting each raw value has the event code may be wrong. To remain on familiar territory, the Nehalem uncore PMU has an opcode matcher register, that uses a 64-bit value. On AMD64 Family 10h, you have IBS. But I could give examples on Itanium with opcode matchers, range restrictions. Corey provided other examples for Power. The API has to provide a way to express what the raw value is meant for: counter, matcher, filter...

Quoted 5 lines

this can be done in a number of ways (in order of increasing levels of abstraction):

- the raw type is kept wide enough. Paul already requested the raw type to be widened to 128 bits to express certain PowerPC features.

Yes, 1 bit is not enough. With 128 that would be enough to encode all the resources I can think of for existing Itanium, X86.

But then, I think fields would have to be renamed to make it clearer. Raw would denote a type of resource and the current type field could be renamed 'code' or 'val' or 'id' which reflects more what the content actually would be.

As for Netburst, both CCCR and ESCR only use the bottom 32-bit so they could be stuffed into the 64-bit 'type' field. But that would not work if a PMU were to require wider values. Those values are part of the event encoding and should not be considered optional nor attributes. That is why using a separate call to program the second value does not seem appropriate to me.

Quoted 15 lines

- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the write() method or ioctl is extended to express attributes we want to set/change while a counter is running.

- or the highest level counter / hw event data type is extended with new attribute field(s).

My feeling is that we generally want such hw features to start small - i.e. at the raw type level initially. Then we can allow them to climb the ladder, if they prove their utility in practice. We've got space reserved in the ABI to allow for growth like this.

Ingo

#11 lore raw
In reply to Ingo Molnar
Quoted 19 lines · Ingo Molnar
Quoted 7 lines · stephane eranian

Hi,

Corey brings up an interesting problem which I wanted to comment on.

The current proposal hinges on the idea that by interpreting a single value the kernel can understand what the user wants to measure. For instance, if I pass type=0, then the kernel understands I want to measure CPU_CYCLES. Given that the number of events and their unit mask combinations can be large, the proposal also provides a "raw" mode, where the content of the type field is interpreted as the raw value to put into a register.

This is where there is an issue because with several PMU models, including on X86, using the raw bit + 64 value is not enough to figure out what the user wants to measure. This happens when the PMU has more than counters. Thus, interpreting each raw value has the event code may be wrong. To remain on familiar territory, the Nehalem uncore PMU has an opcode matcher register, that uses a 64-bit value. On AMD64 Family 10h, you have IBS. But I could give examples on Itanium with opcode matchers, range restrictions. Corey provided other examples for Power. The API has to provide a way to express what the raw value is meant for: counter, matcher, filter...

this can be done in a number of ways (in order of increasing levels of abstraction):

- the raw type is kept wide enough. Paul already requested the raw type to be widened to 128 bits to express certain PowerPC features.

- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the write() method or ioctl is extended to express attributes we want to set/change while a counter is running.

Quoted 6 lines

- or the highest level counter / hw event data type is extended with new attribute field(s).

My feeling is that we generally want such hw features to start small - i.e. at the raw type level initially. Then we can allow them to climb the ladder, if they prove their utility in practice. We've got space reserved in the ABI to allow for growth like this.

Ingo

Hi Ingo and Stephane,

Thanks for the replies.

I think any one of those solutions would work for Power's Instruction Matching Register. If more than one register needs to be programmed, or the values don't fit into the 128-bit raw event types, we could use the "special counter" approach, I think.

I will have another look at the Power PMU description and see if there are other constraints that might cause us to want to go one way or the other, or perhaps a different way.

Regards,

- Corey

Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]

#12 lore raw
Quoted 3 lines

- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the write() method or ioctl is extended to express attributes we want to set/change while a counter is running.

The product of:

        {exotic PMU modes} * {creative performance measurement ideas}

will produce a large number of candidates for these special counters (at least on ia64 ... which has a large number of exotic PMU options).

I don't think that I'm qualified to judge which of them are "useful enough" to warrant a special counter type.

-Tony

#13 lore raw
Quoted 4 lines · Luck, Tony
Quoted 3 lines

- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the write() method or ioctl is extended to express attributes we want to set/change while a counter is running.

Quoted 8 lines

The product of:

        {exotic PMU modes} * {creative performance measurement ideas}

will produce a large number of candidates for these special counters (at least on ia64 ... which has a large number of exotic PMU options).

I don't think that I'm qualified to judge which of them are "useful enough" to warrant a special counter type.

it should certainly be done on a case by case basis. They need to be consciously exposed not just summarily exported to user-space, because PMU hw features have security implications so it has to be done all explicitly.

Ingo

#14 lore raw
In reply to Corey Ashford
Quoted 51 lines · Corey Ashford
Quoted 38 lines · Ingo Molnar
Quoted 12 lines · stephane eranian

Hi,

Corey brings up an interesting problem which I wanted to comment on.

The current proposal hinges on the idea that by interpreting a single value the kernel can understand what the user wants to measure. For instance, if I pass type=0, then the kernel understands I want to measure CPU_CYCLES. Given that the number of events and their unit mask combinations can be large, the proposal also provides a "raw" mode, where the content of the type field is interpreted as the raw value to put into a register.

Quoted 10 lines

This is where there is an issue because with several PMU models, including on X86, using the raw bit + 64 value is not enough to figure out what the user wants to measure. This happens when the PMU has more than counters. Thus, interpreting each raw value has the event code may be wrong. To remain on familiar territory, the Nehalem uncore PMU has an opcode matcher register, that uses a 64-bit value. On AMD64 Family 10h, you have IBS. But I could give examples on Itanium with opcode matchers, range restrictions. Corey provided other examples for Power. The API has to provide a way to express what the raw value is meant for: counter, matcher, filter...

this can be done in a number of ways (in order of increasing levels of abstraction):

- the raw type is kept wide enough. Paul already requested the raw type to be widened to 128 bits to express certain PowerPC features.

- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the write() method or ioctl is extended to express attributes we want to set/change while a counter is running.

- or the highest level counter / hw event data type is extended with new attribute field(s).

Quoted 6 lines

My feeling is that we generally want such hw features to start small - i.e. at the raw type level initially. Then we can allow them to climb the ladder, if they prove their utility in practice. We've got space reserved in the ABI to allow for growth like this.

Ingo

Hi Ingo and Stephane,

Thanks for the replies.

Quoted 8 lines

I think any one of those solutions would work for Power's Instruction Matching Register. If more than one register needs to be programmed, or the values don't fit into the 128-bit raw event types, we could use the "special counter" approach, I think.

I will have another look at the Power PMU description and see if there are other constraints that might cause us to want to go one way or the other, or perhaps a different way.

thanks, that's really appreciated!

One useful approach would be to come up with a bitcount that you think would fit considering even (currently) fringe/odd features - and we'd make sure there's enough space for that in the ABI - should there be a need/desire to expose that in the future.

Ingo

#15 lore raw
Quoted 34 lines · Ingo Molnar
Quoted 32 lines · Corey Ashford
Quoted 18 lines · Ingo Molnar
Quoted 7 lines · stephane eranian

Hi,

Corey brings up an interesting problem which I wanted to comment on.

The current proposal hinges on the idea that by interpreting a single value the kernel can understand what the user wants to measure. For instance, if I pass type=0, then the kernel understands I want to measure CPU_CYCLES. Given that the number of events and their unit mask combinations can be large, the proposal also provides a "raw" mode, where the content of the type field is interpreted as the raw value to put into a register.

This is where there is an issue because with several PMU models, including on X86, using the raw bit + 64 value is not enough to figure out what the user wants to measure. This happens when the PMU has more than counters. Thus, interpreting each raw value has the event code may be wrong. To remain on familiar territory, the Nehalem uncore PMU has an opcode matcher register, that uses a 64-bit value. On AMD64 Family 10h, you have IBS. But I could give examples on Itanium with opcode matchers, range restrictions. Corey provided other examples for Power. The API has to provide a way to express what the raw value is meant for: counter, matcher, filter...

this can be done in a number of ways (in order of increasing levels of abstraction):

- the raw type is kept wide enough. Paul already requested the raw type to be widened to 128 bits to express certain PowerPC features.

- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the write() method or ioctl is extended to express attributes we want to set/change while a counter is running.

Quoted 6 lines

- or the highest level counter / hw event data type is extended with new attribute field(s).

My feeling is that we generally want such hw features to start small - i.e. at the raw type level initially. Then we can allow them to climb the ladder, if they prove their utility in practice. We've got space reserved in the ABI to allow for growth like this.

Ingo

Hi Ingo and Stephane,

Thanks for the replies.

I think any one of those solutions would work for Power's Instruction Matching Register. If more than one register needs to be programmed, or the values don't fit into the 128-bit raw event types, we could use the "special counter" approach, I think.

Quoted 7 lines
Quoted 1 line

I will have another look at the Power PMU description and see if there are other constraints that might cause us to want to go one way or the other, or perhaps a different way.

thanks, that's really appreciated!

One useful approach would be to come up with a bitcount that you think would fit considering even (currently) fringe/odd features - and we'd make sure there's enough space for that in the ABI - should there be a need/desire to expose that in the future.

Ingo

Looking at the Instruction Matching CAM on Power6, it's comprised of two 64-bit values, but there are quite a few reserved bits, and bits that must be programmed in a fixed way. If we were to squeeze out the reserved and fixed bits from the ABI, that leaves 74 real bits of data that a user would like to be able to set.

In addition to that, there is an instruction marking mechanism that requires 2 bits to set the sampling mode.

Lastly, there is a thresholding mechanism that has 6 bits of count, two 3-bit start/end event fields, and a 2-bit granularity field.

In total, that's 90 bits in addition to the event code (9 bits?). There may be a few stragglers that I have missed, and some room should be left for future processors. 128 could be a bit tight for future processor generations.

While reading the Power6 PMU manual, I also had a look at Power5+ PMU manual, and it has five more accessible instruction matching registers (32-bits each). These five are somewhat more special-purpose (they match fewer bits in the instruction), and they probably could be left out, but it would be nice if the ABI had the room for them.

Regards,

- Corey

Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]

#16 lore raw
In reply to Ingo Molnar
Quoted 26 lines · Ingo Molnar

We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:

   http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patch

It's also available in tip/master at:

http://people.redhat.com/mingo/tip.git/README

There are many changes in the v6 release:

- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.

- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc instrumentation and PAPI support.

- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.

- The 'perfstat' utility (ex 'timec') has been updated:

http://people.redhat.com/mingo/perfcounters/perfstat.c

- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:

Quoted 7 lines

http://people.redhat.com/mingo/perfcounters/kerneltop.c

- Merged to latest mainline

- Various fixes and other updates

Ingo

I'm not sure if this is the right place to propose such a thing, but I think it would be very valuable to have a standardized user-side library to accompany this addition to the kernel.

In particular, as a starting place for the discussion, I'd like to see functions in it that are very similar to a subset of what is currently in libpfm. Specifically, I'd like to see the following functions (with the names changed to pcl_* perhaps):

extern pfm_err_t pfm_find_event(const char *str, unsigned int *idx); extern pfm_err_t pfm_find_event_bycode(int code, unsigned int *idx); extern pfm_err_t pfm_find_event_bycode_next(int code, unsigned int start,

                                            unsigned int *next);
extern pfm_err_t pfm_find_event_mask(unsigned int event_idx, const char *str,
                                     unsigned int *mask_idx);

extern pfm_err_t pfm_find_full_event(const char *str, pfmlib_event_t *e);

extern pfm_err_t pfm_get_max_event_name_len(size_t *len);

extern pfm_err_t pfm_get_num_events(unsigned int *count); extern pfm_err_t pfm_get_num_event_masks(unsigned int event_idx,

                                         unsigned int *count);
extern pfm_err_t pfm_get_event_name(unsigned int idx, char *name,
                                    size_t maxlen);
extern pfm_err_t pfm_get_full_event_name(pfmlib_event_t *e, char *name,
                                         size_t maxlen);

extern pfm_err_t pfm_get_event_code(unsigned int idx, int *code); extern pfm_err_t pfm_get_event_mask_code(unsigned int idx,
unsigned int mask_idx, unsigned int *code);
extern pfm_err_t pfm_get_event_description(unsigned int idx, char **str); extern pfm_err_t pfm_get_event_code_counter(unsigned int idx, unsigned int cnt,

                                            int *code);

extern pfm_err_t pfm_get_event_mask_name(unsigned int event_idx,
unsigned int mask_idx, char *name, size_t maxlen);
extern pfm_err_t pfm_get_event_mask_description(unsigned int event_idx,
unsigned int mask_idx, char **desc);

Now, since it's not clear right now how unit masks are going to be handled in your proposal, I'm not sure the that *_event_mask_* functions are applicable, but I think something that fills that function will be needed.

Architectures that have need for additional functionality should be free to add arch-specific functions.

Full descriptions of these functions can be found in the man pages of the libpfm documentation.

Any thoughts on this? Do you already have a user library structure in mind?

Signature · 10 lines
Regards,

- Corey

Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]
#17 lore raw

Corey,

Quoted 86 lines · Corey Ashford
Quoted 29 lines · Ingo Molnar

We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:

http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patch

It's also available in tip/master at:

http://people.redhat.com/mingo/tip.git/README

There are many changes in the v6 release:

- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.

- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc
instrumentation and PAPI support.

- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.

- The 'perfstat' utility (ex 'timec') has been updated:

http://people.redhat.com/mingo/perfcounters/perfstat.c

Quoted 8 lines

- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:
http://people.redhat.com/mingo/perfcounters/kerneltop.c

- Merged to latest mainline

- Various fixes and other updates

Ingo

I'm not sure if this is the right place to propose such a thing, but I think it would be very valuable to have a standardized user-side library to accompany this addition to the kernel.

In particular, as a starting place for the discussion, I'd like to see functions in it that are very similar to a subset of what is currently in libpfm. Specifically, I'd like to see the following functions (with the names changed to pcl_* perhaps):

extern pfm_err_t pfm_find_event(const char *str, unsigned int *idx); extern pfm_err_t pfm_find_event_bycode(int code, unsigned int *idx); extern pfm_err_t pfm_find_event_bycode_next(int code, unsigned int start,

                                           unsigned int *next);
extern pfm_err_t pfm_find_event_mask(unsigned int event_idx, const char
*str,
                                    unsigned int *mask_idx);

extern pfm_err_t pfm_find_full_event(const char *str, pfmlib_event_t *e);

extern pfm_err_t pfm_get_max_event_name_len(size_t *len);

extern pfm_err_t pfm_get_num_events(unsigned int *count); extern pfm_err_t pfm_get_num_event_masks(unsigned int event_idx,

                                        unsigned int *count);
extern pfm_err_t pfm_get_event_name(unsigned int idx, char *name,
                                   size_t maxlen);
extern pfm_err_t pfm_get_full_event_name(pfmlib_event_t *e, char *name,
                                        size_t maxlen);

extern pfm_err_t pfm_get_event_code(unsigned int idx, int *code); extern pfm_err_t pfm_get_event_mask_code(unsigned int idx,
unsigned int mask_idx, unsigned int *code);
extern pfm_err_t pfm_get_event_description(unsigned int idx, char **str); extern pfm_err_t pfm_get_event_code_counter(unsigned int idx, unsigned int cnt,

                                           int *code);

extern pfm_err_t pfm_get_event_mask_name(unsigned int event_idx,
unsigned int mask_idx, char *name, size_t maxlen);
extern pfm_err_t pfm_get_event_mask_description(unsigned int event_idx,
unsigned int mask_idx, char **desc);

Now, since it's not clear right now how unit masks are going to be handled in your proposal, I'm not sure the that *_event_mask_* functions are applicable, but I think something that fills that function will be needed.

Quoted 7 lines

Architectures that have need for additional functionality should be free to add arch-specific functions.

Full descriptions of these functions can be found in the man pages of the libpfm documentation.

Any thoughts on this? Do you already have a us

er library structure in mind?

Quoted 1 line

Yes, I did give some thoughts to all of this. In fact, I have been playing a bit with
libpfm and the LPC proposal.

I think, given that LPC is dealing with event -> counter assignment in the kernel, libpfm
does not have to do it. All it needs to do is event:attributes -> value, and that value is
then passed to the kernel in raw mode.

Event attributes includes on x86, for instance, the edge, invert, counter-mask, plm, field.
I think we could do something more generic than what is currently there. That would not
require PMU specific data structures for attributes. Just pass everything into a string.

To that extent, I have been experimenting with something along those lines:

    int pfm_get_event_encoding(char *event_str, uint64_t **values, int *count);

events are encoded as follows:

event_name:[unit_mask1:unit_mask2:...:unit_maskn][::A1=V1:A2=V2:..:An=Vn]

Attribute names and values depend on each PMU model. Attributes names are strings.
Values can have any type.

For X86, most attributes would be identical, same thing on Itanium because they are
architected.

Some PMU models may need more than one 64-bit value to configure one event, That is
is why there is vector and a count. Libpfm should not be concerned with how those values
are encoded and passed to the kernel. It should be concerned with the event -> value
as described in the PMU documentation.

Given that LPC manages events independently of each other, libpfm does not reallly need
to process multiple events at a time to get a global view of what is being measured.

Here is an example:

$ self inst_retired:any_p::i=1:c=1:u=1:k=1
[0x1d300c0 event_sel=0xc0 umask=0x0 os=1 usr=1 en=1 int=1 inv=1 edge=0
cnt_mask=1] INST_RETIRED
#18 lore raw
Quoted 142 lines · stephane eranian

Corey,

Quoted 84 lines · Corey Ashford
Quoted 28 lines · Ingo Molnar

We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:

http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patch

It's also available in tip/master at:

http://people.redhat.com/mingo/tip.git/README

There are many changes in the v6 release:

- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.

- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc
instrumentation and PAPI support.

- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.

- The 'perfstat' utility (ex 'timec') has been updated:

http://people.redhat.com/mingo/perfcounters/perfstat.c

Quoted 8 lines

- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:
http://people.redhat.com/mingo/perfcounters/kerneltop.c

- Merged to latest mainline

- Various fixes and other updates

Ingo

I'm not sure if this is the right place to propose such a thing, but I think it would be very valuable to have a standardized user-side library to accompany this addition to the kernel.

In particular, as a starting place for the discussion, I'd like to see functions in it that are very similar to a subset of what is currently in libpfm. Specifically, I'd like to see the following functions (with the names changed to pcl_* perhaps):

extern pfm_err_t pfm_find_event(const char *str, unsigned int *idx); extern pfm_err_t pfm_find_event_bycode(int code, unsigned int *idx); extern pfm_err_t pfm_find_event_bycode_next(int code, unsigned int start,

                                           unsigned int *next);
extern pfm_err_t pfm_find_event_mask(unsigned int event_idx, const char
*str,
                                    unsigned int *mask_idx);

extern pfm_err_t pfm_find_full_event(const char *str, pfmlib_event_t *e);

extern pfm_err_t pfm_get_max_event_name_len(size_t *len);

extern pfm_err_t pfm_get_num_events(unsigned int *count); extern pfm_err_t pfm_get_num_event_masks(unsigned int event_idx,

                                        unsigned int *count);
extern pfm_err_t pfm_get_event_name(unsigned int idx, char *name,
                                   size_t maxlen);
extern pfm_err_t pfm_get_full_event_name(pfmlib_event_t *e, char *name,
                                        size_t maxlen);

extern pfm_err_t pfm_get_event_code(unsigned int idx, int *code); extern pfm_err_t pfm_get_event_mask_code(unsigned int idx,
unsigned int mask_idx, unsigned int *code);
extern pfm_err_t pfm_get_event_description(unsigned int idx, char **str); extern pfm_err_t pfm_get_event_code_counter(unsigned int idx, unsigned int cnt,

                                           int *code);

extern pfm_err_t pfm_get_event_mask_name(unsigned int event_idx,
unsigned int mask_idx, char *name, size_t maxlen);
extern pfm_err_t pfm_get_event_mask_description(unsigned int event_idx,
unsigned int mask_idx, char **desc);

Now, since it's not clear right now how unit masks are going to be handled in your proposal, I'm not sure the that *_event_mask_* functions are applicable, but I think something that fills that function will be needed.

Quoted 7 lines

Architectures that have need for additional functionality should be free to add arch-specific functions.

Full descriptions of these functions can be found in the man pages of the libpfm documentation.

Any thoughts on this? Do you already have a us

er library structure in mind?
Yes, I did give some thoughts to all of this. In fact, I have been playing a bit with
libpfm and the LPC proposal.

I think, given that LPC is dealing with event -> counter assignment in the kernel, libpfm
does not have to do it. All it needs to do is event:attributes -> value, and that value is
then passed to the kernel in raw mode.

Event attributes includes on x86, for instance, the edge, invert, counter-mask, plm, field.
I think we could do something more generic than what is currently there. That would not
require PMU specific data structures for attributes. Just pass everything into a string.

To that extent, I have been experimenting with something along those lines:

    int pfm_get_event_encoding(char *event_str, uint64_t **values, int *count);

events are encoded as follows:

   event_name:[unit_mask1:unit_mask2:...:unit_maskn][::A1=V1:A2=V2:..:An=Vn]

Attribute names and values depend on each PMU model. Attributes names are strings.
Values can have any type.

For X86, most attributes would be identical, same thing on Itanium because they are
architected.

Some PMU models may need more than one 64-bit value to configure one event, That is
is why there is vector and a count. Libpfm should not be concerned with how those values
are encoded and passed to the kernel. It should be concerned with the event -> value
as described in the PMU documentation.

Given that LPC manages events independently of each other, libpfm does not reallly need
to process multiple events at a time to get a global view of what is being measured.

Quoted 5 lines

Here is an example:

$ self inst_retired:any_p::i=1:c=1:u=1:k=1
[0x1d300c0 event_sel=0xc0 umask=0x0 os=1 usr=1 en=1 int=1 inv=1 edge=0
cnt_mask=1] INST_RETIRED

This looks encouraging!

I assume the library would still retain the functions that allow us to iterate through the available events, and obtain text description of events. Would it make sense to have similar functions to obtain the available unit masks and attributes for a particular event?

For debugging purposes at least, it might make sense to have a function that does the inverse of pfm_get_event_encoding as well.

Signature · 10 lines
Regards,

- Corey

Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]
#19 lore raw
Quoted 117 lines · Corey Ashford
Quoted 111 lines
Quoted 52 lines

I'm not sure if this is the right place to propose such a thing, but I think
it would be very valuable to have a standardized user-side library to accompany this addition to the kernel.

In particular, as a starting place for the discussion, I'd like to see functions in it that are very similar to a subset of what is currently in libpfm. Specifically, I'd like to see the following functions (with the names changed to pcl_* perhaps):

extern pfm_err_t pfm_find_event(const char *str, unsigned int *idx); extern pfm_err_t pfm_find_event_bycode(int code, unsigned int *idx); extern pfm_err_t pfm_find_event_bycode_next(int code, unsigned int start,

                                          unsigned int *next);
extern pfm_err_t pfm_find_event_mask(unsigned int event_idx, const char
*str,
                                   unsigned int *mask_idx);

extern pfm_err_t pfm_find_full_event(const char *str, pfmlib_event_t *e);

extern pfm_err_t pfm_get_max_event_name_len(size_t *len);

extern pfm_err_t pfm_get_num_events(unsigned int *count); extern pfm_err_t pfm_get_num_event_masks(unsigned int event_idx,

                                       unsigned int *count);
extern pfm_err_t pfm_get_event_name(unsigned int idx, char *name,
                                  size_t maxlen);
extern pfm_err_t pfm_get_full_event_name(pfmlib_event_t *e, char *name,
                                       size_t maxlen);

extern pfm_err_t pfm_get_event_code(unsigned int idx, int *code); extern pfm_err_t pfm_get_event_mask_code(unsigned int idx,
unsigned int mask_idx, unsigned int *code);
extern pfm_err_t pfm_get_event_description(unsigned int idx, char **str); extern pfm_err_t pfm_get_event_code_counter(unsigned int idx, unsigned int
cnt,

                                          int *code);

extern pfm_err_t pfm_get_event_mask_name(unsigned int event_idx,
unsigned int mask_idx, char *name, size_t maxlen);
extern pfm_err_t pfm_get_event_mask_description(unsigned int event_idx,
unsigned int mask_idx, char **desc);

Now, since it's not clear right now how unit masks are going to be handled
in your proposal, I'm not sure the that *_event_mask_* functions are applicable, but I think something that fills that function will be needed.

Quoted 8 lines

Architectures that have need for additional functionality should be free to
add arch-specific functions.

Full descriptions of these functions can be found in the man pages of the libpfm documentation.

Any thoughts on this? Do you already have a us

er library structure in mind?
Yes, I did give some thoughts to all of this. In fact, I have been playing a bit with
libpfm and the LPC proposal.

I think, given that LPC is dealing with event -> counter assignment in the kernel, libpfm
does not have to do it. All it needs to do is event:attributes -> value, and that value is
then passed to the kernel in raw mode.

Event attributes includes on x86, for instance, the edge, invert, counter-mask, plm, field.
I think we could do something more generic than what is currently there. That would not
require PMU specific data structures for attributes. Just pass everything into a string.

To that extent, I have been experimenting with something along those lines:

int pfm_get_event_encoding(char *event_str, uint64_t **values, int
*count);

events are encoded as follows:

event_name:[unit_mask1:unit_mask2:...:unit_maskn][::A1=V1:A2=V2:..:An=Vn]

Attribute names and values depend on each PMU model. Attributes names are strings.
Values can have any type.

For X86, most attributes would be identical, same thing on Itanium because they are
architected.

Some PMU models may need more than one 64-bit value to configure one event, That is
is why there is vector and a count. Libpfm should not be concerned with how those values
are encoded and passed to the kernel. It should be concerned with the event -> value
as described in the PMU documentation.

Given that LPC manages events independently of each other, libpfm does not reallly need
to process multiple events at a time to get a global view of what is being measured.

Quoted 5 lines

Here is an example:

$ self inst_retired:any_p::i=1:c=1:u=1:k=1
[0x1d300c0 event_sel=0xc0 umask=0x0 os=1 usr=1 en=1 int=1 inv=1 edge=0
cnt_mask=1] INST_RETIRED
Quoted 7 lines

This looks encouraging!

I assume the library would still retain the functions that allow us to iterate through the available events, and obtain text description of events.
Would it make sense to have similar functions to obtain the available unit
masks and attributes for a particular event?

Yes, that would most likely stay there, although, I think we could simplify a bit.

Quoted 3 lines

For debugging purposes at least, it might make sense to have a function that does the inverse of pfm_get_event_encoding as well.

Yes, we could provide the opposite function.

I also believe this same scheme could be used to describe non-event features, such as IBS, LBR, Opcode matcher.

#20 lore raw
In reply to Ingo Molnar
Quoted 2 lines · Ingo Molnar

We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:

[snip]

Hi Ingo,

As I was starting to put together a simple implementation of PAPI on top of PCL for Power, I noticed that PCL does not seem to have any sort of versioning and way of ascertaining the current capabilities of what is in the kernel.

This information is needed by tools and libraries built on top of PCL so that they can know what is supported and if any bugs need to be worked around.

Have you thought about how to present this information to the user? I was thinking that since you already have /sys/devices/system/cpu/perf_counter set up, maybe we could add some more files in there. Perhaps a "version" file (perfmon does this), and maybe a "capabilities" file which contains a set of strings exhibiting the supported features. For example,

arch-independent:
MMAPPED_SAMPLE_BUFFER
MMAPPED_SAMPLE_BUFFER_MAX_SIZE=0x20000
CUSTOM_SAMPLING

on powerpc arch:
POWER6_IMC
POWER6_THRESHOLD

or on x86:
NEHALEM_PEBS

etc.

Or it could be a bit mask whose bits are defined in an include file.

Personally, I like the strings approach because it's more extensible, and would not require modifying /usr/include files when you pick up a new kernel. Perhaps the arch-independent and arch-dependent capabilities would be in two different files to make the implementation easier.

Signature · 10 lines
Regards,

- Corey

Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]
#21 lore raw
Quoted 10 lines · Corey Ashford
Quoted 5 lines · Ingo Molnar

We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:

[snip]

Hi Ingo,

Quoted 8 lines

As I was starting to put together a simple implementation of PAPI on top of PCL for Power, I noticed that PCL does not seem to have any sort of versioning and way of ascertaining the current capabilities of what is in the kernel.

This information is needed by tools and libraries built on top of PCL so that they can know what is supported and if any bugs need to be worked around.

I'd prefer to use the standard Linux syscall ABI convention here:

- once upstream, existing functionality is compatible forever

- new functionality is added in a way that it generates a -ENOSYS return from the syscall in an older kernel.

That's why the event structure is sized relatively large for example - to make sure we have space to grow into.

So instead of adding versioning information, it would be very nice if you could check the ABI details for 'traps' that make extensions harder. Try to come up with pie-in-the-sky future items you'd like to see in the ABI, and lets see how supportable it would be.

Example #1 - made up. Say if we had an ABI detail like this:

  struct perf_counter_hw_event {
          u8                     type;

this would limit us to 256 events - which would be clearly stupid as we can easily hit that limit.

Example #2. Not made up:

asmlinkage int
sys_perf_counter_open(struct perf_counter_hw_event *hw_event_uptr __user,
pid_t pid, int cpu, int group_fd)

Those are 5 parameters - we could extend it to 6 and add a 'flags' value that in the current version will return -ENOSYS if the flags value is not zero.

This would add one more dimension of extensibility to the interface.

If you could come up with a list of small details like this, that would be really helpful. Would this work?

Ingo

#22 lore raw
Quoted 37 lines · Ingo Molnar
Quoted 10 lines · Corey Ashford
Quoted 2 lines · Ingo Molnar

We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:

[snip]

Hi Ingo,

As I was starting to put together a simple implementation of PAPI on top of PCL for Power, I noticed that PCL does not seem to have any sort of versioning and way of ascertaining the current capabilities of what is in the kernel.

This information is needed by tools and libraries built on top of PCL so that they can know what is supported and if any bugs need to be worked around.

I'd prefer to use the standard Linux syscall ABI convention here:

- once upstream, existing functionality is compatible forever

- new functionality is added in a way that it generates a -ENOSYS return from the syscall in an older kernel.

That's why the event structure is sized relatively large for example - to make sure we have space to grow into.

So instead of adding versioning information, it would be very nice if you could check the ABI details for 'traps' that make extensions harder. Try to come up with pie-in-the-sky future items you'd like to see in the ABI, and lets see how supportable it would be.

Example #1 - made up. Say if we had an ABI detail like this:

  struct perf_counter_hw_event {
          u8                     type;

this would limit us to 256 events - which would be clearly stupid as we can easily hit that limit.

Example #2. Not made up:

asmlinkage int
sys_perf_counter_open(struct perf_counter_hw_event *hw_event_uptr __user,
pid_t pid, int cpu, int group_fd)

Quoted 7 lines

Those are 5 parameters - we could extend it to 6 and add a 'flags' value that in the current version will return -ENOSYS if the flags value is not zero.

This would add one more dimension of extensibility to the interface.

If you could come up with a list of small details like this, that would be really helpful. Would this work?

Ingo

Thanks for the reply. As I flesh out this PAPI code, I will keep thinking about these issues.

I think the method you describe is good for adding new event types and accessing fancy PMU hardware (instruction matching CAM's for example).

There may be other non-event-related changes that will not be handled quite as well in this way. In the original email I hinted at that we may want an option for mmap'd sample buffers at some point, and so I'm not clear how you'd provide an ABI to request mmap'd buffers (you would probably need to be able to request the size and get back a pointer to the mmap'd buffer). Would this be done through a special sys_perf_counter_open call? Or through a subsequent ioctl call on the group leader after an open (which requires the counters to be initially disabled), etc.

For bugs in the kernel that need to be worked around, I assume you would suggest to the tool programmers that they somehow test for the bug's presence? What if the bug causes a system crash? Perhaps a better solution for that case would be to check the kernel's version number rather than create a separate PCL version?

Thanks for your consideration,

- Corey

Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]

#23 lore raw
Quoted 9 lines · Corey Ashford

There may be other non-event-related changes that will not be handled quite as well in this way. In the original email I hinted at that we may want an option for mmap'd sample buffers at some point, and so I'm not clear how you'd provide an ABI to request mmap'd buffers (you would probably need to be able to request the size and get back a pointer to the mmap'd buffer). Would this be done through a special sys_perf_counter_open call? Or through a subsequent ioctl call on the group leader after an open (which requires the counters to be initially disabled), etc.

What's stopping a regular mmap() call using the fd obtained from sys_perf_counter_open() ?

#24 lore raw
Quoted 5 lines · Peter Zijlstra
Quoted 1 line · Corey Ashford

There may be other non-event-related changes that will not be handled quite as well in this way. In the original email I hinted at that we may want an option for mmap'd sample buffers at some point, and so I'm not clear how you'd provide an ABI to request mmap'd buffers (you would probably need to be able to request the size and get back a pointer to the mmap'd buffer). Would this be done through a special sys_perf_counter_open call? Or through a subsequent ioctl call on the group leader after an open (which requires the counters to be initially disabled), etc.

What's stopping a regular mmap() call using the fd obtained from sys_perf_counter_open() ?

Perhaps nothing. I don't know why it didn't occur to me that this could be done. If PCL supports this, great!

Thanks for your reply!

Signature · 10 lines
Regards,

- Corey

Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]
#25 lore raw
Quoted 15 lines · Corey Ashford
Quoted 12 lines · Peter Zijlstra
Quoted 9 lines · Corey Ashford

There may be other non-event-related changes that will not be handled quite as well in this way. In the original email I hinted at that we may want an option for mmap'd sample buffers at some point, and so I'm not clear how you'd provide an ABI to request mmap'd buffers (you would probably need to be able to request the size and get back a pointer to the mmap'd buffer).
Would this be done through a special sys_perf_counter_open call? Or
through a subsequent ioctl call on the group leader after an open (which requires the counters to be initially disabled), etc.

Quoted 6 lines
Quoted 2 lines

What's stopping a regular mmap() call using the fd obtained from sys_perf_counter_open() ?

Perhaps nothing. I don't know why it didn't occur to me that this could be done. If PCL supports this, great!

That has always been the way, I thought this could be done with PCL. Except that I don't know who decides the size of the mmap().

#26 lore raw
In reply to Peter Zijlstra
Quoted 5 lines · Peter Zijlstra
Quoted 1 line · Corey Ashford

There may be other non-event-related changes that will not be handled quite as well in this way. In the original email I hinted at that we may want an option for mmap'd sample buffers at some point, and so I'm not clear how you'd provide an ABI to request mmap'd buffers (you would probably need to be able to request the size and get back a pointer to the mmap'd buffer). Would this be done through a special sys_perf_counter_open call? Or through a subsequent ioctl call on the group leader after an open (which requires the counters to be initially disabled), etc.

What's stopping a regular mmap() call using the fd obtained from sys_perf_counter_open() ?

Thinking about this some more, I'm not sure it is the right model to be able to read a stream of records up from kernel space.

Let's say I want to read 1MB of records up at once, and then a bit later read another 1MB, and so on, all while the program is still running and counters are counting. Is there a way to use mmap this way? I think I would have to call mmap for each 1MB segment (if my buffer is 1MB). Also how do I indicate to the kernel that I've "consumed" each 1MB chunk and that the kernel buffers for it can be reused? It's easy to understand how to do that with just a read() call, but with mmap'd memory, I don't know.

Another thing that concerns me is this statement in the mmap() man page: "If the size of the mapped file changes after the call to mmap() as a result of some other operation on the mapped file, the effect of references to portions of the mapped region that correspond to added or removed portions of the file is unspecified." Would PCL guarantee the correct contents of memory, despite this caveat?

If you wait till after you've stopped the counters to read up the records, I can see where mmap would work fine. But if you are trying to unload the data from the kernel's space efficiently, over the long execution time of the thread or process being analyzed, I don't see how mmap can be used in this way on PCL's file descriptors.

In perfmon, the model is different, because the mmap'd memory for the buffer is not streamed; it's just a block of memory full of records. Once one buffer fills up, user space is alerted and the kernel inhibits adding new entries (in single-buffering mode), while user space accesses the filled buffer. So the mmap call has to be done only once.
Double-buffering (or more) can be achieved by splitting the buffer
into multiple pieces and filling them up separately and alerting user space when each are full.

Quite possibly I'm missing something obvious, but PCL may need an mmap behavior better suited for efficiently reading data up dynamically over the long run of the program being analyzed.

Regards,

- Corey

Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]

maintainer
#27 lore raw
In reply to Ingo Molnar

arch/powerpc/include/asm/systbl.h SYSCALL_SPU(epoll_create1) · line 325

Quoted 9 lines · Ingo Molnar
arch/powerpc/include/asm/systbl.h +1 -0
index 72353f6..4c8095f 100644
@@ -322,3 +322,4 @@ SYSCALL_SPU(epoll_create1)
 SYSCALL_SPU(dup3)
 SYSCALL_SPU(pipe2)
 SYSCALL(inotify_init1)
+SYSCALL(perf_counter_open)

Is there a reason to forbid this on the SPU? It's probably not particularly useful, but it shouldn't hurt.

arch/powerpc/include/asm/systbl.h

Quoted 10 lines
arch/powerpc/include/asm/systbl.h +10 -0
+
+#ifdef CONFIG_PERF_COUNTERS
+# include <asm/perf_counter.h>
+#endif
+
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/rculist.h>
+#include <linux/rcupdate.h>
+#include <linux/spinlock.h>

I guess all */perf_counter.h files should be exported, but then the #ifdef won't work when the file is included from user space. Also, some of the included headers are not exported.

Quoted 10 lines
arch/powerpc/include/asm/systbl.h +10 -0
+/*
+ * Hardware event to monitor via a performance monitoring counter:
+ */
+struct perf_counter_hw_event {
+	s64			type;
+
+	u64			irq_period;
+	u32			record_type;
+
+	u32			disabled     :  1, /* off by default      */

arch/powerpc/include/asm/systbl.h

Quoted 10 lines
arch/powerpc/include/asm/systbl.h +10 -0
+				nmi	     :  1, /* NMI sampling        */
+				raw	     :  1, /* raw event type      */
+				inherit	     :  1, /* children inherit it */
+				pinned	     :  1, /* must always be on PMU */
+				exclusive    :  1, /* only counter on PMU */
+
+				__reserved_1 : 26;
+
+	u64			__reserved_2;
+};

When exported, the types should be __s64 etc instead of s64.

arch/powerpc/include/asm/systbl.h

Quoted 9 lines
arch/powerpc/include/asm/systbl.h +9 -0
+/*
+ * Ioctls that can be done on a perf counter fd:
+ */
+#define PERF_COUNTER_IOC_ENABLE		_IO('$', 0)
+#define PERF_COUNTER_IOC_DISABLE	_IO('$', 1)
+
+/*
+ * Kernel-internal data types:
+ */

The kernel internal parts of the header should be hidden in #ifdef __KERNEL__.

arch/powerpc/include/asm/systbl.h

Quoted 7 lines
arch/powerpc/include/asm/systbl.h +7 -0
+
+asmlinkage int sys_perf_counter_open(
+
+	struct perf_counter_hw_event	*hw_event_uptr		__user,
+	pid_t				pid,
+	int				cpu,
+	int				group_fd);

For the syscall, I'd suggest sys_perf_counter_fd or sys_perfcounterfd to go along with signalfd, eventfd, etc. The only other _open syscall we have is sys_mq_open(), which is different since it actually performs an open() on a (hidden) file.

All system calls should return a 'long' value to make sure that the errno handling works on all architectures.

arch/powerpc/include/asm/systbl.h

Quoted 7 lines
arch/powerpc/include/asm/systbl.h +7 -0
+static const struct file_operations perf_fops = {
+	.release		= perf_release,
+	.read			= perf_read,
+	.poll			= perf_poll,
+	.unlocked_ioctl		= perf_ioctl,
+	.compat_ioctl		= perf_ioctl,
+};

It feels inconsistent to combine a syscall for getting the fd with ioctl. Assuming the interface is semantically right, I'd suggest using either a chardev with more ioctls or more syscalls but not both:

asmlinkage long sys_perfcounter_fd(

	struct perf_counter_hw_event	*hw_event_uptr		__user,
	pid_t				pid,
	int				cpu,
	int				group_fd);

asmlinkage long sys_perfcounter_setflags(int fd, u32 flags);

or

struct perf_counter_setup {
	struct perf_counter_hw_event event;
	__u64 pid;
	__u32 cpu;
	__u32 group_fd;
};

#define PERF_COUNTER_IOC_SETUP _IOW('$', 1, struct perf_counter_setup) #define PERF_COUNTER_IOC_SETFLAGS _IOW('$', 2, __u32)

	Arnd <><
#28 lore raw
Quoted 14 lines · Arnd Bergmann

arch/powerpc/include/asm/systbl.h SYSCALL_SPU(epoll_create1) · line 325

Quoted 9 lines · Ingo Molnar
arch/powerpc/include/asm/systbl.h +1 -0
index 72353f6..4c8095f 100644
@@ -322,3 +322,4 @@ SYSCALL_SPU(epoll_create1)
 SYSCALL_SPU(dup3)
 SYSCALL_SPU(pipe2)
 SYSCALL(inotify_init1)
+SYSCALL(perf_counter_open)

Is there a reason to forbid this on the SPU? It's probably not particularly useful, but it shouldn't hurt.

No, it's just an oversight, plus the fact that this stuff doesn't support the hardware counters on Cell, which I've heard is rather unusual. Don't you have to hcalls to access it on machines with a hypervisor? Also, is there any facility for counting events on the SPUs?

Quoted 5 lines
Quoted 5 lines
arch/powerpc/include/asm/systbl.h +5 -0
+#ifdef CONFIG_PERF_COUNTERS
+# include <asm/perf_counter.h>
+#endif
+
+#include <linux/list.h>
Quoted 8 lines

arch/powerpc/include/asm/systbl.h

Quoted 4 lines
arch/powerpc/include/asm/systbl.h +4 -0
+#include <linux/mutex.h>
+#include <linux/rculist.h>
+#include <linux/rcupdate.h>
+#include <linux/spinlock.h>

I guess all */perf_counter.h files should be exported, but then the #ifdef won't work when the file is included from user space. Also, some of the included headers are not exported.

You're right, we do need to clean this up a bit. So far, userspace programs have tended to use their own copies of the relevant definitions.

I think all that userspace needs is the hw_event_types enum, the perf_counter_record_type enum, the perf_counter_hw_event struct, and the ioctl definitions. I don't see why userspace would need the asm bits.

Quoted 14 lines
Quoted 14 lines
arch/powerpc/include/asm/systbl.h +14 -0
+/*
+ * Hardware event to monitor via a performance monitoring counter:
+ */
+struct perf_counter_hw_event {
+	s64			type;
+
+	u64			irq_period;
+	u32			record_type;
+
+	u32			disabled     :  1, /* off by default      */
+				nmi	     :  1, /* NMI sampling        */
+				raw	     :  1, /* raw event type      */
+				inherit	     :  1, /* children inherit it */
+				pinned	     :  1, /* must always be on PMU */
Quoted 8 lines

arch/powerpc/include/asm/systbl.h

Quoted 6 lines
arch/powerpc/include/asm/systbl.h +6 -0
+				exclusive    :  1, /* only counter on PMU */
+
+				__reserved_1 : 26;
+
+	u64			__reserved_2;
+};

When exported, the types should be __s64 etc instead of s64.

Yep.

Quoted 4 lines
Quoted 4 lines
arch/powerpc/include/asm/systbl.h +4 -0
+/*
+ * Ioctls that can be done on a perf counter fd:
+ */
+#define PERF_COUNTER_IOC_ENABLE		_IO('$', 0)
Quoted 8 lines

arch/powerpc/include/asm/systbl.h

Quoted 5 lines
arch/powerpc/include/asm/systbl.h +5 -0
+#define PERF_COUNTER_IOC_DISABLE	_IO('$', 1)
+
+/*
+ * Kernel-internal data types:
+ */

The kernel internal parts of the header should be hidden in #ifdef __KERNEL__.

Agreed.

Quoted 4 lines
Quoted 4 lines
arch/powerpc/include/asm/systbl.h +4 -0
+
+asmlinkage int sys_perf_counter_open(
+
+	struct perf_counter_hw_event	*hw_event_uptr		__user,
Quoted 8 lines

arch/powerpc/include/asm/systbl.h

Quoted 3 lines
arch/powerpc/include/asm/systbl.h +3 -0
+	pid_t				pid,
+	int				cpu,
+	int				group_fd);

For the syscall, I'd suggest sys_perf_counter_fd or sys_perfcounterfd to go along with signalfd, eventfd, etc. The only other _open syscall we have is sys_mq_open(), which is different since it actually performs an open() on a (hidden) file.

I don't have a strong opinion on that either way; maybe Ingo does.

Quoted 2 lines

All system calls should return a 'long' value to make sure that the errno handling works on all architectures.

True.

Quoted 9 lines

arch/powerpc/include/asm/systbl.h

Quoted 7 lines
arch/powerpc/include/asm/systbl.h +7 -0
+static const struct file_operations perf_fops = {
+	.release		= perf_release,
+	.read			= perf_read,
+	.poll			= perf_poll,
+	.unlocked_ioctl		= perf_ioctl,
+	.compat_ioctl		= perf_ioctl,
+};

It feels inconsistent to combine a syscall for getting the fd with ioctl.

Why? Once you have an fd, what's inconsistent about doing an ioctl on it?

Quoted 12 lines

Assuming the interface is semantically right, I'd suggest using either a chardev with more ioctls or more syscalls but not both:

asmlinkage long sys_perfcounter_fd(

	struct perf_counter_hw_event	*hw_event_uptr		__user,
	pid_t				pid,
	int				cpu,
	int				group_fd);

asmlinkage long sys_perfcounter_setflags(int fd, u32 flags);

or

Quoted 8 lines
struct perf_counter_setup {
	struct perf_counter_hw_event event;
	__u64 pid;
	__u32 cpu;
	__u32 group_fd;
};

#define PERF_COUNTER_IOC_SETUP _IOW('$', 1, struct perf_counter_setup) #define PERF_COUNTER_IOC_SETFLAGS _IOW('$', 2, __u32)

I don't see any compelling value in either of those suggestions, sorry. Possibly, if it turns out that read() or ioctl() are just too slow (and can't be improved), we might consider adding syscalls.

Paul.

maintainer
#29 lore raw
Quoted 6 lines · Paul Mackerras

Arnd Bergmann writes:
No, it's just an oversight, plus the fact that this stuff doesn't support the hardware counters on Cell, which I've heard is rather unusual. Don't you have to hcalls to access it on machines with a hypervisor? Also, is there any facility for counting events on the SPUs?

Yes, there is, and it's suported in both oprofile and perfmon2. It uses a set of hooks in the spu scheduler to know about what code got loaded into it, and the platform interface is a combination of mmio and rtas/hcall accesses.

Quoted 10 lines
Quoted 9 lines

kernel/perf_counter.c line 1326

Quoted 7 lines
kernel/perf_counter.c new file +7 -0
+static const struct file_operations perf_fops = {
+	.release		= perf_release,
+	.read			= perf_read,
+	.poll			= perf_poll,
+	.unlocked_ioctl		= perf_ioctl,
+	.compat_ioctl		= perf_ioctl,
+};

It feels inconsistent to combine a syscall for getting the fd with ioctl.

Quoted 2 lines

Why? Once you have an fd, what's inconsistent about doing an ioctl on it?

I don't think we have any interface doing that today outside of the traditional chardev, blockdev, socket and regular file descriptors. The first inconsistency would be in having another type of object that you can do ioctl on.

Back in the original spufs interface design, I had similar plans for adding a combination of syscall and ioctl interfaces. The point that convinced me to go with just syscall and regular file operations was that syscall and ioctl are basically two interchangeable ways of doing the same thing (adding an arbitrary function call into the kernel). When you look in /proc/misc, half the stuff in there only exists as a cheap way to sneak a syscall into the kernel without the necessary review.

The inconsistency in combining the two would be the same as having sys_timerfd_settime an ioctl on the fd from sys_timerfd_create, or getdents an ioctl on a directory fd.

The most significant impact of allowing ioctl on the fd is certainly that it allows adding many more interfaces on top of the performance counters as soon as the code is initially merged. Whether you count that as a an advantage or a disadvantage depends on your point of view ;-).

Quoted 21 lines
Quoted 12 lines

Assuming the interface is semantically right, I'd suggest using either a chardev with more ioctls or more syscalls but not both:

asmlinkage long sys_perfcounter_fd(

	struct perf_counter_hw_event	*hw_event_uptr		__user,
	pid_t				pid,
	int				cpu,
	int				group_fd);

asmlinkage long sys_perfcounter_setflags(int fd, u32 flags);

or

Quoted 8 lines
struct perf_counter_setup {
	struct perf_counter_hw_event event;
	__u64 pid;
	__u32 cpu;
	__u32 group_fd;
};

#define PERF_COUNTER_IOC_SETUP _IOW('$', 1, struct perf_counter_setup) #define PERF_COUNTER_IOC_SETFLAGS _IOW('$', 2, __u32)

Quoted 3 lines

I don't see any compelling value in either of those suggestions, sorry. Possibly, if it turns out that read() or ioctl() are just too slow (and can't be improved), we might consider adding syscalls.

I don't think that performance is an issue here, read() is quite well optimized and ioctl() would only be used in the slowpath.

My second example is a bit constructed, if you wanted to go with a chardev, that would probably imply larger interface changes than that.

One way to improve the interface over PERF_COUNTER_IOC_ENABLE might be to use write() for enable/disable, but if sys_perfcounter_fd (or whatever we call it) stays, using other syscalls for operating on it seems to be the most consistant way of doing it.

Another point that now occurred to me is that in sys_perfcounter_fd(), you should probably put the group descriptor argument first, to be more consistent with the openat() family of syscalls which all take the relative position as their first argument.

	Arnd <><
#30 lore raw
In reply to Ingo Molnar

Some points to mention here. This patch set actually introduces two interfaces, a new user/kernel interface and an in-kernel api to access performance counters. These are separate things and sometimes mixed too much. There is a strong need for an in-kernel api. This is the third implementation I am involved (oprofile, perfmon are the others) and the things are always the same way. All these subsystems should be merged to one in-kernel implemenation and share the same code. The different user/kernel i/fs could then coexist and meet the users different needs.

The implementation of the hardware counters is written from scratch again. It is sometimes useful to drop old code, but there is the danger of making errors twice. Implementing performance counters is not trivial, especially buffer handling, SMP and cpu hotplug. For oprofile and perfmon it took years to get stable code. We should benefit from this. (The current x86 code in this patch series seems not to work proper with SMP.) So we should look for a way to better reuse and share code.

See also my comments below.

On 21.01.09 19:50:21, Ingo Molnar wrote:

[...]

Quoted 21 lines
arch/x86/kernel/cpu/perf_counter.c new file +21 -0
+static bool perf_counters_initialized __read_mostly;
+
+/*
+ * Number of (generic) HW counters:
+ */
+static int nr_counters_generic __read_mostly;
+static u64 perf_counter_mask __read_mostly;
+static u64 counter_value_mask __read_mostly;
+
+static int nr_counters_fixed __read_mostly;
+
+struct cpu_hw_counters {
+	struct perf_counter	*counters[X86_PMC_IDX_MAX];
+	unsigned long		used[BITS_TO_LONGS(X86_PMC_IDX_MAX)];
+};
+
+/*
+ * Intel PerfMon v3. Used on Core2 and later.
+ */
+static DEFINE_PER_CPU(struct cpu_hw_counters, cpu_hw_counters);
+

arch/x86/kernel/cpu/perf_counter.c line 52

Quoted 10 lines
arch/x86/kernel/cpu/perf_counter.c new file +10 -0
+static const int intel_perfmon_event_map[] =
+{
+  [PERF_COUNT_CPU_CYCLES]		= 0x003c,
+  [PERF_COUNT_INSTRUCTIONS]		= 0x00c0,
+  [PERF_COUNT_CACHE_REFERENCES]		= 0x4f2e,
+  [PERF_COUNT_CACHE_MISSES]		= 0x412e,
+  [PERF_COUNT_BRANCH_INSTRUCTIONS]	= 0x00c4,
+  [PERF_COUNT_BRANCH_MISSES]		= 0x00c5,
+  [PERF_COUNT_BUS_CYCLES]		= 0x013c,
+};

I would like to define _all_ the behaviour of the architecture and the models in functions instead of parameters and lists. It is hard to explain why, because it is more esthetics, but I believe, only nice things work best. Let me try.

1) The list above seems to be random, there are lots of events and it is hard to define, which event is really important. Surely these events are important, but it is hard to draw a line here.

2) The list assumes/implies the events are available on all architectures and cpus. This is probably not the case, and also, the existence of an event must not be _important_ for a certain architecture. But it has to be there even if it is of no interest.

3) Hard to extend. If an event is added here this could have impact to all other architectures. Data structures are changing.

4) In the kernel the behaviour of a subsystem is offen implemented by functions (e.g. struct device_driver). There are lots of ops structs in the kernel and there are reasons for it.

5) ops structs are more dynamic. The data could be generated dynamically and does not have to be static in some tables and variables.

So, instead of making the list a public data structure, better pass the type to an arch specific function, e.g.:

int arch_xxx_setup_event(int event_type);

If the type is not supported, an error could be returned. There is no more impact. Even the binaries of the builds would be identically if hw_event_types would be extended for a single different architecture.

The same applies also for counters and so on, better implement functions.

-Robert

Signature · 3 lines
Advanced Micro Devices, Inc.
Operating System Research Center
email: [email protected]
#31 lore raw
Quoted 4 lines · Robert Richter

Some points to mention here. This patch set actually introduces two interfaces, a new user/kernel interface and an in-kernel api to access performance counters. These are separate things and sometimes mixed too much. There is a strong need for an in-kernel api. This is the

We have been concentrating more on the user/kernel API since that is the one that cannot be changed in an incompatible way once this stuff goes upstream. The in-kernel API can be changed at any time and is still evolving.

Quoted 5 lines

third implementation I am involved (oprofile, perfmon are the others) and the things are always the same way. All these subsystems should be merged to one in-kernel implemenation and share the same code. The different user/kernel i/fs could then coexist and meet the users different needs.

It would certainly be good to get oprofile to use the same low-level machinery as perf_counters. I'm not sure what the fate of perfmon will be, but it seems unlikely it will go upstream in anything like its present form.

Quoted 16 lines

arch/x86/kernel/cpu/perf_counter.c line 52

Quoted 10 lines
arch/x86/kernel/cpu/perf_counter.c new file +10 -0
+static const int intel_perfmon_event_map[] =
+{
+  [PERF_COUNT_CPU_CYCLES]		= 0x003c,
+  [PERF_COUNT_INSTRUCTIONS]		= 0x00c0,
+  [PERF_COUNT_CACHE_REFERENCES]		= 0x4f2e,
+  [PERF_COUNT_CACHE_MISSES]		= 0x412e,
+  [PERF_COUNT_BRANCH_INSTRUCTIONS]	= 0x00c4,
+  [PERF_COUNT_BRANCH_MISSES]		= 0x00c5,
+  [PERF_COUNT_BUS_CYCLES]		= 0x013c,
+};

I would like to define _all_ the behaviour of the architecture and the models in functions instead of parameters and lists. It is hard to explain why, because it is more esthetics, but I believe, only nice things work best. Let me try.

Quoted 3 lines

1) The list above seems to be random, there are lots of events and it is hard to define, which event is really important. Surely these events are important, but it is hard to draw a line here.

I see that list as a convenience for doing a few simple performance measurements. For any serious in-depth analysis userspace will know what processor it's running on and use raw event codes.

Quoted 12 lines

2) The list assumes/implies the events are available on all architectures and cpus. This is probably not the case, and also, the existence of an event must not be _important_ for a certain architecture. But it has to be there even if it is of no interest.

3) Hard to extend. If an event is added here this could have impact to all other architectures. Data structures are changing.

4) In the kernel the behaviour of a subsystem is offen implemented by functions (e.g. struct device_driver). There are lots of ops structs in the kernel and there are reasons for it.

Quoted 8 lines

5) ops structs are more dynamic. The data could be generated dynamically and does not have to be static in some tables and variables.

So, instead of making the list a public data structure, better pass the type to an arch specific function, e.g.:

int arch_xxx_setup_event(int event_type);

That's exactly what we have, except that it's called hw_perf_counter_init and the event_type you have there is in the struct perf_counter that gets passed in.

Quoted 6 lines

If the type is not supported, an error could be returned. There is no more impact. Even the binaries of the builds would be identically if hw_event_types would be extended for a single different architecture.

The same applies also for counters and so on, better implement functions.

All of that is already done; hw_perf_counter_init gets to interpret the counter->hw_event.type and counter->hw_event.raw fields and decide whether the event is supported, and return an error if not. On x86 it looks like there is a further ops structure (struct pmc_x86_ops) which allows each x86-compatible cpu type to supply its own functions for doing the interpretation of counter->hw_event and other things.

Paul.

#32 lore raw
Quoted 9 lines · Paul Mackerras
Quoted 4 lines

Some points to mention here. This patch set actually introduces two interfaces, a new user/kernel interface and an in-kernel api to access performance counters. These are separate things and sometimes mixed too much. There is a strong need for an in-kernel api. This is the

We have been concentrating more on the user/kernel API since that is the one that cannot be changed in an incompatible way once this stuff goes upstream. The in-kernel API can be changed at any time and is still evolving.

I agree, it is much more easier to change the in-kernel i/f. I just wanted to emphasize the importance of this i/f. Oprofile, Perfmon and also LPC will exist in the future too and should share the same code base. That's what I missed in the discussion until now.

Quoted 11 lines
Quoted 6 lines

third implementation I am involved (oprofile, perfmon are the others) and the things are always the same way. All these subsystems should be merged to one in-kernel implemenation and share the same code. The different user/kernel i/fs could then coexist and meet the users different needs.

It would certainly be good to get oprofile to use the same low-level machinery as perf_counters. I'm not sure what the fate of perfmon will be, but it seems unlikely it will go upstream in anything like its present form.

Right, as I sad above, all should share the same low-level code. And, this code already exists. The question is more how to merge it and bring the things together.

[...]

Quoted 8 lines
Quoted 4 lines

So, instead of making the list a public data structure, better pass the type to an arch specific function, e.g.:

int arch_xxx_setup_event(int event_type);

That's exactly what we have, except that it's called hw_perf_counter_init and the event_type you have there is in the struct perf_counter that gets passed in.

Thanks for pointing this out, I was misinterpreting this as a general hw initialization function, but instead a counter is allocated.

Quoted 8 lines
Quoted 7 lines

If the type is not supported, an error could be returned. There is no more impact. Even the binaries of the builds would be identically if hw_event_types would be extended for a single different architecture.

The same applies also for counters and so on, better implement functions.

Quoted 6 lines

All of that is already done; hw_perf_counter_init gets to interpret the counter->hw_event.type and counter->hw_event.raw fields and decide whether the event is supported, and return an error if not. On x86 it looks like there is a further ops structure (struct pmc_x86_ops) which allows each x86-compatible cpu type to supply its own functions for doing the interpretation of counter->hw_event and other things.

Ok, maybe I mixed too much the architectural with the x86 model specific implementation. My impression is that there is data in generic structures what should be better private for the model or architecture. However, I have to figure out the details here.

-Robert

Signature · 3 lines
Advanced Micro Devices, Inc.
Operating System Research Center
email: [email protected]
#33 lore raw
Quoted 4 lines · Robert Richter

I agree, it is much more easier to change the in-kernel i/f. I just wanted to emphasize the importance of this i/f. Oprofile, Perfmon and also LPC will exist in the future too and should share the same code base. That's what I missed in the discussion until now.

We could implement oprofile on top of lpc for those archs that have LPC support. And afaik only ia64 needs to bother with perfmon as that's the only arch that has support for it anyway.

Now, even on x86 LPC would need a little more arch support before we can fully replace oprofile, but a half-way model would be a LPC oprofile driver that uses LPC on those machines its supported on, while working to provide LPC support for the older machines.

#34 lore raw
In reply to Robert Richter
Quoted 4 lines · Robert Richter

Ok, maybe I mixed too much the architectural with the x86 model specific implementation. My impression is that there is data in generic structures what should be better private for the model or architecture. However, I have to figure out the details here.

The details of the x86 support have changed quite a lot since the v6 patch was posted, I believe. Are you looking at the v6 patch, or at Ingo's tip:perfcounters/core branch?

Ingo - maybe it's time to post a v7 patch?

Paul.

#35 lore raw
Quoted 7 lines · Paul Mackerras
Quoted 4 lines · Robert Richter

Ok, maybe I mixed too much the architectural with the x86 model specific implementation. My impression is that there is data in generic structures what should be better private for the model or architecture. However, I have to figure out the details here.

Quoted 5 lines

The details of the x86 support have changed quite a lot since the v6 patch was posted, I believe. Are you looking at the v6 patch, or at Ingo's tip:perfcounters/core branch?

Ingo - maybe it's time to post a v7 patch?

Yeah - will try to do that later today.

Ingo

#36 lore raw
In reply to Paul Mackerras
Quoted 10 lines · Paul Mackerras
Quoted 4 lines · Robert Richter

Ok, maybe I mixed too much the architectural with the x86 model specific implementation. My impression is that there is data in generic structures what should be better private for the model or architecture. However, I have to figure out the details here.

The details of the x86 support have changed quite a lot since the v6 patch was posted, I believe. Are you looking at the v6 patch, or at Ingo's tip:perfcounters/core branch?

I am using the tip branch, but took the v6 thread to discuss this.

Quoted 2 lines

Ingo - maybe it's time to post a v7 patch?

For me it's fine to work with the branch if patches are posted to the mailing list for review.

-Robert

Signature · 3 lines
Advanced Micro Devices, Inc.
Operating System Research Center
email: [email protected]
#37 lore raw
Quoted 12 lines · Robert Richter
Quoted 10 lines · Paul Mackerras
Quoted 4 lines · Robert Richter

Ok, maybe I mixed too much the architectural with the x86 model specific implementation. My impression is that there is data in generic structures what should be better private for the model or architecture. However, I have to figure out the details here.

The details of the x86 support have changed quite a lot since the v6 patch was posted, I believe. Are you looking at the v6 patch, or at Ingo's tip:perfcounters/core branch?

Quoted 7 lines

I am using the tip branch, but took the v6 thread to discuss this.

Quoted 2 lines

Ingo - maybe it's time to post a v7 patch?

For me it's fine to work with the branch if patches are posted to the mailing list for review.

all perfcounter patches are on lkml and you can see all -tip commits in general by subscribing to the [email protected] mailing list.

Ingo