Performance Counters for Linux, v6
Subsystems (31)
We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:
http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patch
It's also available in tip/master at:
http://people.redhat.com/mingo/tip.git/README
There are many changes in the v6 release:
- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.
- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc instrumentation and PAPI support.
- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.
- The 'perfstat' utility (ex 'timec') has been updated:
http://people.redhat.com/mingo/perfcounters/perfstat.c
- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:
http://people.redhat.com/mingo/perfcounters/kerneltop.c
- Merged to latest mainline
- Various fixes and other updates
Ingo
------------------>
Andi Kleen (1):
x86, generic: mark complex bitops.h inlines as __always_inline
Brian Gerst (12):
x86-64: Move irq stats from PDA to per-cpu and consolidate with 32-bit.
x86-64: Move TLB state from PDA to per-cpu and consolidate with 32-bit.
x86-64: Convert irqstacks to per-cpu
x86-64: Convert exception stacks to per-cpu
x86-64: Move cpu number from PDA to per-cpu and consolidate with 32-bit.
x86-64: Move current task from PDA to per-cpu and consolidate with 32-bit.
x86-64: Move kernelstack from PDA to per-cpu.
x86-64: Move oldrsp from PDA to per-cpu.
x86-64: Move irqcount from PDA to per-cpu.
x86-64: Move nodenumber from PDA to per-cpu.
x86-64: Move isidle from PDA to per-cpu.
x86-64: Use absolute displacements for per-cpu accesses.
Christophe Saout (1):
xen: fix too early kmalloc call
Ingo Molnar (50):
performance counters: documentation
performance counters: x86 support
x86, perfcounters: read out MSR_CORE_PERF_GLOBAL_STATUS with counters disabled
perfcounters: select ANON_INODES
perfcounters, x86: simplify disable/enable of counters perfcounters, x86: clean up debug code
perfcounters: consolidate global-disable codepaths
perf counters: restructure the API
perf counters: add support for group counters
perf counters: group counter, fixes
perf counters: hw driver API
perf counters: implement PERF_COUNT_CPU_CLOCK
perf counters: consolidate hw_perf save/restore APIs
perf counters: implement PERF_COUNT_TASK_CLOCK
perf counters: add prctl interface to disable/enable counters perf counters: clean up state transitions
perf counters: update docs
x86: implement atomic64_t on 32-bit
perfcounters: restructure x86 counter math
perfcounters: implement "counter inheritance"
perfcounters: fix task clock counter
perfcounters: add context switch counter
perfcounters: add task migrations counter
perfcounters: add nr-of-faults counter
perfcounters: fix non-intel-perfmon CPUs
perfcounters, x86: fix sw counters on non-PMC CPUs
perfcounters: fix lapic initialization
perfcounters: release CPU context when exiting task counters
perfcounters: flush on setuid exec
perfcounters: use hw_event.disable flag
perfcounters: remove warnings
perfcounters: tweak group scheduling
x86, perfcounters: rename intel_arch_perfmon.h => perf_counter.h x86, perfcounters: prepare for fixed-mode PMCs
perfcounters: add fixed-mode PMC enumeration
x86, perfcounters: refactor code for fixed-function PMCs
perfcounters: hw ops rename
perfcounters: fix task clock counter
perfcounters: pull inherited counters
perfcounters: fix init context lock
perfcounters: enable lowlevel pmc code to schedule counters
x86, perfcounters: print out the ->used bitmask
perfcounters: remove ->nr_inherited
perfcounters: generalize the counter scheduler
perfcounters: add PERF_COUNT_BUS_CYCLES
x86, perfcounters: add support for fixed-function pmcs
perfcounters: include asm/perf_counter.h only if CONFIG_PERF_COUNTERS=y x86, cpufreq: remove leftover copymask_copy()
x86: fix broken flush_tlb_others_ipi(), fix
percpu: add optimized generic percpu accessors
Jan Beulich (2):
x86: fully honor "nolapic"
x86: avoid early crash in disable_local_APIC()
Jaswinder Singh (1):
x86: perf_counter.c intel_perfmon_event_map and max_intel_perfmon_events should be static
Jaswinder Singh Rajput (20):
x86: perf_counter remove unwanted hw_perf_enable_all
x86: smp.h remove obsolete function declaration
x86: smp.h move zap_low_mappings declartion to tlbflush.h
x86: smp.h move prefill_possible_map declartion to cpu.h
x86: smp.h move stack_processor_id declartion to cpu.h
x86: smp.h move safe_smp_processor_id declartion to cpu.h
x86: smp.h move cpu_physical_id declartion to cpu.h
x86: smp.h move boot_cpu_id declartion to cpu.h
x86: rename intel_mp_floating to mpf_intel
x86: rename all fields of mpf_intel mpf_X to X
x86: smp.h move cpu_callin_mask and cpu_callin_map declartion to cpumask.h
x86: smp.h move cpu_callout_mask and cpu_callout_map declartion to cpumask.h
x86: smp.h move cpu_initialized_mask and cpu_initialized declartion to cpumask.h
x86: smp.h move cpu_sibling_setup_mask and cpu_sibling_setup_map declartion to cpumask.h
x86: microcode_intel.c fix style problems
x86: msr.c fix style problems
x86: module_32.c fix style problems
x86: module_64.c fix style problems
x86: replacing mp_config_ioapic with mpc_ioapic
x86: replacing mp_config_intsrc with mpc_intsrc
Mike Travis (17):
cpumask: update irq_desc to use cpumask_var_t
cpumask: fix bug in use cpumask_var_t in irq_desc SGI UV cpumask: use static temp cpumask in flush_tlb
x86: cleanup remaining cpumask_t code in mce_amd_64.c
x86: reduce stack usage in init_intel_cacheinfo
cpumask: use cpumask_var_t in dcdbas.c
cpumask: reduce stack usage in find_lowest_rq
Xen: reduce memory required for cpu_evtchn_mask
irq: change references from NR_IRQS to nr_irqs
irq: use WARN() instead of WARN_ON().
irq: allocate irq_desc_ptrs array based on nr_irqs
irq: initialize nr_irqs based on nr_cpu_ids
kstat: modify kstat_irqs_legacy to be variable sized cpumask, irq: non-x86 build failures
irq: update all arches for new irq_desc
irq: update all arches for new irq_desc, fix
x86: cleanup early setup_percpu references
Paul Mackerras (13):
perf_counter: Fix return value from dummy hw_perf_counter_init
perf_counter: Fix the cpu_clock software counter
perf_counter: Add optional hw_perf_group_sched_in arch function
perf_counter: Add dummy perf_counter_print_debug function powerpc/perf_counter: Add perf_counter system call on powerpc
powerpc: Provide a way to defer perf counter work until interrupts are enabled powerpc/perf_counter: Add generic support for POWER-family PMU hardware powerpc/perf_counter: Add support for PPC970 family powerpc/perf_counter: Add support for POWER6
perf_counter: Always schedule all software counters in powerpc/perf_counter: Make sure PMU gets enabled properly
perf_counter: Add support for pinned and exclusive counter groups
perf_counter: Add counter enable/disable ioctls
Rusty Russell (5):
cpumask: Use topology_core_cpumask()/topology_thread_cpumask()
cpumask: convert misc driver functions
cpumask: convert drivers/net/sfc
cpumask: convert other misc kernel functions
x86: change flush_tlb_others to take a const struct cpumask
Suresh Siddha (2):
x86: fix broken flush_tlb_others_ipi()
x86, pat: fix reserve_memtype() for legacy 1MB range
Tejun Heo (15):
x86: fix pda_to_op()
x86: make early_per_cpu() a lvalue and use it
x86: make vmlinux_32.lds.S use PERCPU() macro
x86: make percpu symbols zerobased on SMP
x86: load pointer to pda into %gs while brining up a CPU
x86: use static _cpu_pda array
x86: fold pda into percpu area on SMP
x86: merge 64 and 32 SMP percpu handling
x86: make pda a percpu variable
x86: convert pda ops to wrappers around x86 percpu accessors
x86: misc clean up after the percpu update
x86: fix build bug introduced during merge
x86_64: initialize this_cpu_off to __per_cpu_load linker script: add missing VMLINUX_SYMBOL
linker script: add missing .data.percpu.page_aligned
Thomas Gleixner (4):
performance counters: core code
perf counters: protect them against CSTATE transitions perf counters: clean up 'raw' type API
perf counters: expand use of counter->event
Yinghai Lu (3):
perf_counter: more barrier in blank weak function
x86: arch_probe_nr_irqs
x86: make 32bit MAX_HARDIRQS_PER_CPU to be NR_VECTORS
Documentation/cputopology.txt | 6 +- Documentation/perf-counters.txt | 147 ++ arch/alpha/kernel/irq.c | 2 +- arch/arm/kernel/irq.c | 18 +- arch/arm/kernel/vmlinux.lds.S | 1 + arch/arm/oprofile/op_model_mpcore.c | 2 +- arch/blackfin/kernel/irqchip.c | 5 + arch/ia64/kernel/iosapic.c | 2 +- arch/ia64/kernel/irq.c | 4 +- arch/ia64/kernel/irq_ia64.c | 12 +- arch/ia64/kernel/msi_ia64.c | 4 +- arch/ia64/kernel/vmlinux.lds.S | 1 + arch/ia64/sn/kernel/msi_sn.c | 2 +- arch/mips/include/asm/irq.h | 2 +- arch/mips/kernel/irq-gic.c | 2 +- arch/mips/kernel/smtc.c | 6 +- arch/mips/mti-malta/malta-smtc.c | 5 +- arch/mips/sgi-ip22/ip22-int.c | 2 +- arch/mips/sgi-ip22/ip22-time.c | 2 +- arch/mips/sibyte/bcm1480/smp.c | 3 +- arch/mips/sibyte/sb1250/smp.c | 3 +- arch/mn10300/kernel/mn10300-watchdog.c | 3 +- arch/parisc/kernel/irq.c | 8 +- arch/powerpc/include/asm/hw_irq.h | 31 + arch/powerpc/include/asm/paca.h | 1 + arch/powerpc/include/asm/perf_counter.h | 72 + arch/powerpc/include/asm/systbl.h | 1 + arch/powerpc/include/asm/unistd.h | 3 +- arch/powerpc/kernel/Makefile | 1 + arch/powerpc/kernel/asm-offsets.c | 1 + arch/powerpc/kernel/entry_64.S | 9 + arch/powerpc/kernel/irq.c | 12 +- arch/powerpc/kernel/perf_counter.c | 785 +++++++++ arch/powerpc/kernel/power6-pmu.c | 283 +++ arch/powerpc/kernel/ppc970-pmu.c | 375 ++++ arch/powerpc/kernel/vmlinux.lds.S | 1 + arch/powerpc/platforms/Kconfig.cputype | 1 + arch/powerpc/platforms/pseries/xics.c | 5 +- arch/powerpc/sysdev/mpic.c | 3 +- arch/sparc/kernel/irq_64.c | 5 +- arch/sparc/kernel/time_64.c | 2 +- arch/x86/Kconfig | 1 + arch/x86/ia32/ia32entry.S | 11 +- arch/x86/include/asm/apicnum.h | 12 + arch/x86/include/asm/atomic_32.h | 218 +++ arch/x86/include/asm/bitops.h | 14 +- arch/x86/include/asm/cpu.h | 21 + arch/x86/include/asm/cpumask.h | 28 + arch/x86/include/asm/current.h | 24 +- arch/x86/include/asm/hardirq_32.h | 4 + arch/x86/include/asm/hardirq_64.h | 25 +- arch/x86/include/asm/hw_irq.h | 2 + arch/x86/include/asm/intel_arch_perfmon.h | 31 - arch/x86/include/asm/io_apic.h | 26 +- arch/x86/include/asm/irq_regs_32.h | 4 +- arch/x86/include/asm/irq_vectors.h | 18 +- arch/x86/include/asm/mach-default/entry_arch.h | 5 + arch/x86/include/asm/mmu_context_32.h | 12 +- arch/x86/include/asm/mmu_context_64.h | 16 +- arch/x86/include/asm/mpspec_def.h | 23 +- arch/x86/include/asm/page_64.h | 4 +- arch/x86/include/asm/paravirt.h | 8 +- arch/x86/include/asm/pda.h | 124 +-- arch/x86/include/asm/percpu.h | 159 +- arch/x86/include/asm/perf_counter.h | 95 + arch/x86/include/asm/processor.h | 3 + arch/x86/include/asm/setup.h | 1 - arch/x86/include/asm/smp.h | 49 +- arch/x86/include/asm/system.h | 4 +- arch/x86/include/asm/thread_info.h | 24 +- arch/x86/include/asm/tlbflush.h | 17 +- arch/x86/include/asm/topology.h | 8 +- arch/x86/include/asm/trampoline.h | 1 + arch/x86/include/asm/unistd_32.h | 1 + arch/x86/include/asm/unistd_64.h | 3 +- arch/x86/include/asm/uv/uv_bau.h | 3 +- arch/x86/kernel/acpi/boot.c | 96 +- arch/x86/kernel/acpi/sleep.c | 1 + arch/x86/kernel/apic.c | 26 +- arch/x86/kernel/asm-offsets_64.c | 8 +- arch/x86/kernel/cpu/Makefile | 12 +- arch/x86/kernel/cpu/common.c | 82 +- arch/x86/kernel/cpu/cpufreq/acpi-cpufreq.c | 2 - arch/x86/kernel/cpu/intel_cacheinfo.c | 63 +- arch/x86/kernel/cpu/mcheck/mce_amd_64.c | 21 +- arch/x86/kernel/cpu/perf_counter.c | 695 ++++++++ arch/x86/kernel/cpu/perfctr-watchdog.c | 2 +- arch/x86/kernel/crash.c | 2 +- arch/x86/kernel/dumpstack_64.c | 35 +- arch/x86/kernel/entry_64.S | 46 +- arch/x86/kernel/head64.c | 23 +- arch/x86/kernel/head_64.S | 47 +- arch/x86/kernel/io_apic.c | 163 +- arch/x86/kernel/irq.c | 11 +- arch/x86/kernel/irq_32.c | 2 +- arch/x86/kernel/irq_64.c | 5 +- arch/x86/kernel/irqinit_32.c | 3 + arch/x86/kernel/irqinit_64.c | 5 + arch/x86/kernel/microcode_intel.c | 10 +- arch/x86/kernel/module_32.c | 6 +- arch/x86/kernel/module_64.c | 32 +- arch/x86/kernel/mpparse.c | 142 +- arch/x86/kernel/msr.c | 2 +- arch/x86/kernel/nmi.c | 10 +- arch/x86/kernel/process_32.c | 5 +- arch/x86/kernel/process_64.c | 22 +- arch/x86/kernel/reboot.c | 1 + arch/x86/kernel/setup.c | 2 +- arch/x86/kernel/setup_percpu.c | 208 ++- arch/x86/kernel/signal.c | 7 +- arch/x86/kernel/smpboot.c | 69 +- arch/x86/kernel/smpcommon.c | 10 +- arch/x86/kernel/syscall_table_32.S | 1 + arch/x86/kernel/tlb_32.c | 85 +- arch/x86/kernel/tlb_64.c | 76 +- arch/x86/kernel/tlb_uv.c | 16 +- arch/x86/kernel/vmlinux_32.lds.S | 9 +- arch/x86/kernel/vmlinux_64.lds.S | 22 +- arch/x86/kernel/x8664_ksyms_64.c | 2 - arch/x86/mach-voyager/setup.c | 1 + arch/x86/mach-voyager/voyager_smp.c | 6 +- arch/x86/mm/init_32.c | 1 - arch/x86/mm/pat.c | 37 +- arch/x86/oprofile/op_model_ppro.c | 2 +- arch/x86/xen/enlighten.c | 47 +- arch/x86/xen/irq.c | 8 +- arch/x86/xen/mmu.c | 8 +- arch/x86/xen/multicalls.h | 2 +- arch/x86/xen/smp.c | 33 +- arch/x86/xen/xen-asm_64.S | 31 +- drivers/acpi/processor_idle.c | 8 + drivers/base/cpu.c | 2 +- drivers/base/topology.c | 33 +- drivers/char/sysrq.c | 2 + drivers/firmware/dcdbas.c | 12 +- drivers/misc/sgi-xp/xpc_main.c | 2 +- drivers/net/sfc/efx.c | 17 +- drivers/oprofile/buffer_sync.c | 22 +- drivers/oprofile/buffer_sync.h | 4 + drivers/oprofile/oprof.c | 9 +- drivers/pci/intr_remapping.c | 1 + drivers/xen/events.c | 26 +- drivers/xen/manage.c | 2 +- fs/exec.c | 8 + include/asm-generic/bitops/__ffs.h | 2 +- include/asm-generic/bitops/__fls.h | 2 +- include/asm-generic/bitops/fls.h | 2 +- include/asm-generic/bitops/fls64.h | 4 +- include/asm-generic/percpu.h | 52 + include/asm-generic/sections.h | 2 +- include/asm-generic/vmlinux.lds.h | 75 +- include/linux/init_task.h | 11 + include/linux/interrupt.h | 1 + include/linux/irq.h | 86 +- include/linux/irqnr.h | 1 + include/linux/kernel_stat.h | 8 + include/linux/perf_counter.h | 290 ++++ include/linux/prctl.h | 3 + include/linux/sched.h | 12 +- include/linux/syscalls.h | 8 + include/linux/topology.h | 6 + init/Kconfig | 30 + kernel/Makefile | 1 + kernel/exit.c | 13 +- kernel/fork.c | 1 + kernel/irq/chip.c | 5 +- kernel/irq/handle.c | 57 +- kernel/irq/internals.h | 7 + kernel/irq/manage.c | 12 +- kernel/irq/migration.c | 12 +- kernel/irq/numa_migrate.c | 19 +- kernel/irq/proc.c | 4 +- kernel/perf_counter.c | 2169 ++++++++++++++++++++++++ kernel/sched.c | 76 +- kernel/sched_rt.c | 36 +- kernel/softirq.c | 5 + kernel/sys.c | 7 + kernel/sys_ni.c | 3 + lib/smp_processor_id.c | 2 +- 179 files changed, 6925 insertions(+), 1342 deletions(-) create mode 100644 Documentation/perf-counters.txt create mode 100644 arch/powerpc/include/asm/perf_counter.h create mode 100644 arch/powerpc/kernel/perf_counter.c create mode 100644 arch/powerpc/kernel/power6-pmu.c create mode 100644 arch/powerpc/kernel/ppc970-pmu.c create mode 100644 arch/x86/include/asm/apicnum.h create mode 100644 arch/x86/include/asm/cpumask.h delete mode 100644 arch/x86/include/asm/intel_arch_perfmon.h create mode 100644 arch/x86/include/asm/perf_counter.h create mode 100644 arch/x86/kernel/cpu/perf_counter.c create mode 100644 include/linux/perf_counter.h create mode 100644 kernel/perf_counter.c
Documentation/cputopology.txt +3 -3
@@ -18,11 +18,11 @@ For an architecture to support this feature, it must define some of these macros in include/asm-XXX/topology.h: #define topology_physical_package_id(cpu) #define topology_core_id(cpu) -#define topology_thread_siblings(cpu) -#define topology_core_siblings(cpu) +#define topology_thread_cpumask(cpu) +#define topology_core_cpumask(cpu) The type of **_id is int. -The type of siblings is cpumask_t. +The type of siblings is (const) struct cpumask *. To be consistent on all architectures, include/linux/topology.h provides default definitions for any of the above macros that are
Documentation/perf-counters.txt new file +147 -0
147 lines not on this page · raw message
arch/alpha/kernel/irq.c +1 -1
@@ -55,7 +55,7 @@ int irq_select_affinity(unsigned int irq) cpu = (cpu < (NR_CPUS -1 ) ? cpu +1 :0 ); last_cpu = cpu; - irq_desc[irq].affinity =cpumask_of_cpu (cpu); +cpumask_copy (irq_desc[irq].affinity,cpumask_of (cpu)); irq_desc[irq].chip->set_affinity (irq,cpumask_of (cpu));return 0 ; }
arch/arm/kernel/irq.c +12 -6
@@ -104,6 +104,11 @@ static struct irq_desc bad_irq_desc = { .lock = SPIN_LOCK_UNLOCKED }; +#ifdef CONFIG_CPUMASK_OFFSTACK +/* We are not allocating bad_irq_desc.affinity or .pending_mask */ +#error "ARM architecture does not support CONFIG_CPUMASK_OFFSTACK." +#endif +/* * do_IRQ handles all hardware IRQ's. Decoded IRQs should not * come via this function. Instead, they should provide their
@@ -161,7 +166,7 @@ void __init init_IRQ(void) irq_desc[irq].status |=IRQ_NOREQUEST |IRQ_NOPROBE ;#ifdef CONFIG_SMP - bad_irq_desc.affinity =CPU_MASK_ALL ; +cpumask_setall (bad_irq_desc.affinity); bad_irq_desc.cpu =smp_processor_id ();#endif init_arch_irq ();
@@ -191,15 +196,16 @@ void migrate_irqs(void)struct irq_desc *desc = irq_desc + i;if (desc->cpu == cpu) { -unsigned int newcpu =any_online_cpu (desc->affinity); - -if (newcpu ==NR_CPUS ) { +unsigned int newcpu =cpumask_any_and (desc->affinity, + cpu_online_mask); +if (newcpu >= nr_cpu_ids) {if (printk_ratelimit ())printk (KERN_INFO "IRQ%u no longer affine to CPU%u\n" , i, cpu); -cpus_setall (desc->affinity); - newcpu =any_online_cpu (desc->affinity); +cpumask_setall (desc->affinity); + newcpu =cpumask_any_and (desc->affinity, + cpu_online_mask); }route_irq (desc, i, newcpu);
arch/arm/kernel/vmlinux.lds.S +1 -0
@@ -65,6 +65,7 @@ SECTIONS #endif . = ALIGN(4096); __per_cpu_start = .; + *(.data.percpu.page_aligned) *(.data.percpu) *(.data.percpu.shared_aligned) __per_cpu_end = .;
arch/arm/oprofile/op_model_mpcore.c +1 -1
@@ -263,7 +263,7 @@ static void em_route_irq(int irq, unsigned int cpu)const struct cpumask *mask =cpumask_of (cpu);spin_lock_irq (&desc->lock); - desc->affinity = *mask; +cpumask_copy (desc->affinity, mask); desc->chip->set_affinity (irq, mask);spin_unlock_irq (&desc->lock); }
arch/blackfin/kernel/irqchip.c +5 -0
@@ -69,6 +69,11 @@ static struct irq_desc bad_irq_desc = {#endif }; +#ifdef CONFIG_CPUMASK_OFFSTACK +/* We are not allocating a variable-sized bad_irq_desc.affinity */ +#error "Blackfin architecture does not support CONFIG_CPUMASK_OFFSTACK." +#endif +int show_interrupts (struct seq_file *p,void *v) {int i = *(loff_t *) v, j;
arch/ia64/kernel/iosapic.c +1 -1
@@ -880,7 +880,7 @@ iosapic_unregister_intr (unsigned int gsi)if (iosapic_intr_info[irq].count ==0 ) {#ifdef CONFIG_SMP /* Clear affinity */ -cpus_setall (idesc->affinity); +cpumask_setall (idesc->affinity);#endif /* Clear the interrupt information */ iosapic_intr_info[irq].dest =0 ;
arch/ia64/kernel/irq.c +2 -2
@@ -103,7 +103,7 @@ static char irq_redir [NR_IRQS]; // = { [0 ... NR_IRQS-1] = 1 };void set_irq_affinity_info (unsigned int irq,int hwid,int redir) {if (irq <NR_IRQS ) { -cpumask_copy (&irq_desc[irq].affinity, +cpumask_copy (irq_desc[irq].affinity,cpumask_of (cpu_logical_id (hwid))); irq_redir[irq] = (char ) (redir &0xff ); }
@@ -148,7 +148,7 @@ static void migrate_irqs(void)if (desc->status ==IRQ_PER_CPU )continue ; -if (cpumask_any_and (&irq_desc[irq].affinity, cpu_online_mask) +if (cpumask_any_and (irq_desc[irq].affinity, cpu_online_mask) >= nr_cpu_ids) {/* * Save it for phase 2 processing
arch/ia64/kernel/irq_ia64.c +8 -4
@@ -493,11 +493,13 @@ ia64_handle_irq (ia64_vector vector, struct pt_regs *regs) saved_tpr =ia64_getreg (_IA64_REG_CR_TPR);ia64_srlz_d ();while (vector !=IA64_SPURIOUS_INT_VECTOR ) { +struct irq_desc *desc =irq_to_desc (vector); +if (unlikely (IS_LOCAL_TLB_FLUSH (vector))) {smp_local_flush_tlb (); - kstat_this_cpu.irqs[vector]++; +kstat_incr_irqs_this_cpu (vector, desc); }else if (unlikely (IS_RESCHEDULE (vector))) - kstat_this_cpu.irqs[vector]++; +kstat_incr_irqs_this_cpu (vector, desc);else {int irq =local_vector_to_irq (vector);
@@ -551,11 +553,13 @@ void ia64_process_pending_intr(void)* Perform normal interrupt style processing */ while (vector !=IA64_SPURIOUS_INT_VECTOR ) { +struct irq_desc *desc =irq_to_desc (vector); +if (unlikely (IS_LOCAL_TLB_FLUSH (vector))) {smp_local_flush_tlb (); - kstat_this_cpu.irqs[vector]++; +kstat_incr_irqs_this_cpu (vector, desc); }else if (unlikely (IS_RESCHEDULE (vector))) - kstat_this_cpu.irqs[vector]++; +kstat_incr_irqs_this_cpu (vector, desc);else {struct pt_regs *old_regs =set_irq_regs (NULL );int irq =local_vector_to_irq (vector);
arch/ia64/kernel/msi_ia64.c +2 -2
@@ -75,7 +75,7 @@ static void ia64_set_msi_irq_affinity(unsigned int irq, msg.data = data;write_msi_msg (irq, &msg); - irq_desc[irq].affinity = cpumask_of_cpu(cpu); + cpumask_copy(irq_desc[irq].affinity, cpumask_of(cpu)); }#endif /* CONFIG_SMP */
@@ -187,7 +187,7 @@ static void dmar_msi_set_affinity(unsigned int irq, const struct cpumask *mask) msg.address_lo |=MSI_ADDR_DESTID_CPU (cpu_physical_id (cpu));dmar_msi_write (irq, &msg); - irq_desc[irq].affinity = *mask; + cpumask_copy(irq_desc[irq].affinity, mask); }#endif /* CONFIG_SMP */
arch/ia64/kernel/vmlinux.lds.S +1 -0
@@ -219,6 +219,7 @@ SECTIONS .data.percpu PERCPU_ADDR : AT(__phys_per_cpu_start - LOAD_OFFSET) { __per_cpu_start = .; + *(.data.percpu.page_aligned) *(.data.percpu) *(.data.percpu.shared_aligned) __per_cpu_end = .;
arch/ia64/sn/kernel/msi_sn.c +1 -1
@@ -205,7 +205,7 @@ static void sn_set_msi_irq_affinity(unsigned int irq, msg.address_lo = (u32)(bus_addr &0x00000000ffffffff );write_msi_msg (irq, &msg); - irq_desc[irq].affinity = *cpu_mask; + cpumask_copy(irq_desc[irq].affinity, cpu_mask); }#endif /* CONFIG_SMP */
arch/mips/include/asm/irq.h +1 -1
@@ -66,7 +66,7 @@ extern void smtc_forward_irq(unsigned int irq);*/ #define IRQ_AFFINITY_HOOK (irq) \ do { \ - if (!cpu_isset(smp_processor_id(), irq_desc[irq].affinity)) { \ + if (!cpumask_test_cpu(smp_processor_id(), irq_desc[irq].affinity)) {\ smtc_forward_irq(irq); \ irq_exit(); \ return; \
arch/mips/kernel/irq-gic.c +1 -1
@@ -187,7 +187,7 @@ static void gic_set_affinity(unsigned int irq, const struct cpumask *cpumask)set_bit (irq, pcpu_masks[first_cpu (tmp)].pcpu_mask); } - irq_desc[irq].affinity = *cpumask; +cpumask_copy (irq_desc[irq].affinity, cpumask);spin_unlock_irqrestore (&gic_lock, flags); }
arch/mips/kernel/smtc.c +4 -2
@@ -686,7 +686,7 @@ void smtc_forward_irq(unsigned int irq)* and efficiency, we just pick the easiest one to find. */ - target =first_cpu (irq_desc[irq].affinity); + target =cpumask_first (irq_desc[irq].affinity);/* * We depend on the platform code to have correctly processed
@@ -921,11 +921,13 @@ void ipi_decode(struct smtc_ipi *pipi)struct clock_event_device *cd;void *arg_copy = pipi->arg;int type_copy = pipi->type; +int irq =MIPS_CPU_IRQ_BASE +1 ; +smtc_ipi_nq (&freeIPIq, pipi);switch (type_copy) {case SMTC_CLOCK_TICK :irq_enter (); - kstat_this_cpu.irqs[MIPS_CPU_IRQ_BASE +1 ]++; +kstat_incr_irqs_this_cpu (irq,irq_to_desc (irq)); cd = &per_cpu (mips_clockevent_device, cpu); cd->event_handler (cd);irq_exit ();
arch/mips/mti-malta/malta-smtc.c +3 -2
@@ -116,7 +116,7 @@ struct plat_smp_ops msmtc_smp_ops = {void plat_set_irq_affinity (unsigned int irq,const struct cpumask *affinity) { -cpumask_t tmask = *affinity; +cpumask_t tmask;int cpu =0 ;void smtc_set_irq_affinity (unsigned int irq,cpumask_t aff);
@@ -139,11 +139,12 @@ void plat_set_irq_affinity(unsigned int irq, const struct cpumask *affinity)* be made to forward to an offline "CPU". */ +cpumask_copy (&tmask, affinity);for_each_cpu (cpu, affinity) {if ((cpu_data[cpu].vpe_id !=0 ) || !cpu_online (cpu))cpu_clear (cpu, tmask); } - irq_desc[irq].affinity = tmask; +cpumask_copy (irq_desc[irq].affinity, &tmask);if (cpus_empty (tmask))/*
arch/mips/sgi-ip22/ip22-int.c +1 -1
@@ -155,7 +155,7 @@ static void indy_buserror_irq(void)int irq =SGI_BUSERR_IRQ ;irq_enter (); - kstat_this_cpu.irqs[irq]++; +kstat_incr_irqs_this_cpu (irq,irq_to_desc (irq));ip22_be_interrupt (irq);irq_exit (); }
arch/mips/sgi-ip22/ip22-time.c +1 -1
@@ -122,7 +122,7 @@ void indy_8254timer_irq(void)char c;irq_enter (); - kstat_this_cpu.irqs[irq]++; +kstat_incr_irqs_this_cpu (irq,irq_to_desc (irq));printk (KERN_ALERT "Oops, got 8254 interrupt.\n" );ArcRead (0 , &c,1 , &cnt);ArcEnterInteractiveMode ();
arch/mips/sibyte/bcm1480/smp.c +2 -1
@@ -178,9 +178,10 @@ struct plat_smp_ops bcm1480_smp_ops = {void bcm1480_mailbox_interrupt (void ) {int cpu =smp_processor_id (); +int irq =K_BCM1480_INT_MBOX_0_0 ;unsigned int action; - kstat_this_cpu.irqs[K_BCM1480_INT_MBOX_0_0 ]++; +kstat_incr_irqs_this_cpu (irq,irq_to_desc (irq));/* Load the mailbox register to figure out what we're supposed to do */ action = (__raw_readq (mailbox_0_regs[cpu]) >>48 ) &0xffff ;
arch/mips/sibyte/sb1250/smp.c +2 -1
@@ -166,9 +166,10 @@ struct plat_smp_ops sb_smp_ops = {void sb1250_mailbox_interrupt (void ) {int cpu =smp_processor_id (); +int irq =K_INT_MBOX_0 ;unsigned int action; - kstat_this_cpu.irqs[K_INT_MBOX_0 ]++; +kstat_incr_irqs_this_cpu (irq,irq_to_desc (irq));/* Load the mailbox register to figure out what we're supposed to do */ action = (____raw_readq (mailbox_regs[cpu]) >>48 ) &0xffff ;
arch/mn10300/kernel/mn10300-watchdog.c +2 -1
@@ -130,6 +130,7 @@ void watchdog_interrupt(struct pt_regs *regs, enum exception_code excep)* the stack NMI-atomically, it's safe to use smp_processor_id(). */ int sum, cpu =smp_processor_id (); +int irq =NMIIRQ ;u8 wdt, tmp; wdt =WDCTR & ~WDCTR_WDCNE ;
@@ -138,7 +139,7 @@ void watchdog_interrupt(struct pt_regs *regs, enum exception_code excep)NMICR =NMICR_WDIF ;nmi_count (cpu)++; - kstat_this_cpu.irqs[NMIIRQ ]++; +kstat_incr_irqs_this_cpu (irq,irq_to_desc (irq)); sum = irq_stat[cpu].__irq_count;if (last_irq_sums[cpu] == sum) {
arch/parisc/kernel/irq.c +4 -4
@@ -120,7 +120,7 @@ int cpu_check_affinity(unsigned int irq, cpumask_t *dest)if (CHECK_IRQ_PER_CPU (irq)) {/* Bad linux design decision. The mask has already * been set; we must reset it */ - irq_desc[irq].affinity =CPU_MASK_ALL ; +cpumask_setall (irq_desc[irq].affinity);return -EINVAL ; }
@@ -136,7 +136,7 @@ static void cpu_set_affinity_irq(unsigned int irq, const struct cpumask *dest)if (cpu_check_affinity (irq, dest))return ; - irq_desc[irq].affinity = *dest; + cpumask_copy(irq_desc[irq].affinity, dest); }#endif
@@ -295,7 +295,7 @@ int txn_alloc_irq(unsigned int bits_wide)unsigned long txn_affinity_addr (unsigned int irq,int cpu) {#ifdef CONFIG_SMP - irq_desc[irq].affinity =cpumask_of_cpu (cpu); +cpumask_copy (irq_desc[irq].affinity,cpumask_of (cpu));#endif return per_cpu (cpu_data, cpu).txn_addr;
@@ -352,7 +352,7 @@ void do_cpu_irq_mask(struct pt_regs *regs) irq =eirr_to_irq (eirr_val);#ifdef CONFIG_SMP - dest = irq_desc[irq].affinity; +cpumask_copy (&dest, irq_desc[irq].affinity);if (CHECK_IRQ_PER_CPU (irq_desc[irq].status) && !cpu_isset (smp_processor_id (), dest)) {int cpu =first_cpu (dest);
arch/powerpc/include/asm/hw_irq.h +31 -0
@@ -131,5 +131,36 @@ static inline int irqs_disabled_flags(unsigned long flags)*/ struct hw_interrupt_type ; +#ifdef CONFIG_PERF_COUNTERS +static inline unsigned long get_perf_counter_pending (void ) +{ +unsigned long x; + + asmvolatile ("lbz %0,%1(13)" + :"=r" (x) + :"i" (offsetof(struct paca_struct , perf_counter_pending))); +return x; +} + +static inline void set_perf_counter_pending (int x) +{ + asmvolatile ("stb %0,%1(13)" : : +"r" (x), +"i" (offsetof(struct paca_struct , perf_counter_pending))); +} + +extern void perf_counter_do_pending (void ); + +#else + +static inline unsigned long get_perf_counter_pending (void ) +{ +return 0 ; +} + +static inline void set_perf_counter_pending (int x) {} +static inline void perf_counter_do_pending (void ) {} +#endif /* CONFIG_PERF_COUNTERS */ +#endif /* __KERNEL__ */ #endif /* _ASM_POWERPC_HW_IRQ_H */
arch/powerpc/include/asm/paca.h +1 -0
@@ -99,6 +99,7 @@ struct paca_struct {u8 soft_enabled;/* irq soft-enable flag */ u8 hard_enabled;/* set if irqs are enabled in MSR */ u8 io_sync;/* writel() needs spin_unlock sync */ +u8 perf_counter_pending;/* PM interrupt while soft-disabled */ /* Stuff for accurate time accounting */ u64 user_time;/* accumulated usermode TB ticks */
arch/powerpc/include/asm/perf_counter.h new file +72 -0
72 lines not on this page · raw message
arch/powerpc/include/asm/systbl.h +1 -0
@@ -322,3 +322,4 @@ SYSCALL_SPU(epoll_create1) SYSCALL_SPU(dup3) SYSCALL_SPU(pipe2) SYSCALL(inotify_init1) +SYSCALL(perf_counter_open)
arch/powerpc/include/asm/unistd.h +2 -1
@@ -341,10 +341,11 @@#define __NR_dup3 316#define __NR_pipe2 317#define __NR_inotify_init1 318 +#define __NR_perf_counter_open 319#ifdef __KERNEL__ -#define __NR_syscalls 319 +#define __NR_syscalls 320#define __NR__exit __NR_exit#define NR_syscalls __NR_syscalls
arch/powerpc/kernel/Makefile +1 -0
@@ -94,6 +94,7 @@ obj-$(CONFIG_AUDIT) += audit.o obj64-$(CONFIG_AUDIT) += compat_audit.o obj-$(CONFIG_DYNAMIC_FTRACE) += ftrace.o +obj-$(CONFIG_PERF_COUNTERS) += perf_counter.o ppc970-pmu.o power6-pmu.o obj-$(CONFIG_8XX_MINIMAL_FPEMU) += softemu8xx.o
arch/powerpc/kernel/asm-offsets.c +1 -0
@@ -131,6 +131,7 @@ int main(void)DEFINE (PACAKMSR , offsetof(struct paca_struct , kernel_msr));DEFINE (PACASOFTIRQEN , offsetof(struct paca_struct , soft_enabled));DEFINE (PACAHARDIRQEN , offsetof(struct paca_struct , hard_enabled)); +DEFINE (PACAPERFPEND , offsetof(struct paca_struct , perf_counter_pending));DEFINE (PACASLBCACHE , offsetof(struct paca_struct , slb_cache));DEFINE (PACASLBCACHEPTR , offsetof(struct paca_struct , slb_cache_ptr)); DEFINE(PACACONTEXTID, offsetof(struct paca_struct, context.id));
arch/powerpc/kernel/entry_64.S +9 -0
@@ -526,6 +526,15 @@ ALT_FW_FTR_SECTION_END_IFCLR(FW_FEATURE_ISERIES) 2: TRACE_AND_RESTORE_IRQ(r5); +#ifdef CONFIG_PERF_COUNTERS + /* check paca->perf_counter_pending if we're enabling ints */ + lbz r3,PACAPERFPEND(r13) + and. r3,r3,r5 + beq 27f + bl .perf_counter_do_pending +27: +#endif /* CONFIG_PERF_COUNTERS */ + /* extract EE bit and use it to restore paca->hard_enabled */ ld r3,_MSR(r1) rldicl r4,r3,49,63 /* r0 = (r3 >> 15) & 1 */
arch/powerpc/kernel/irq.c +11 -1
@@ -104,6 +104,13 @@ static inline notrace void set_soft_enabled(unsigned long enable) : : "r" (enable), "i" (offsetof(struct paca_struct, soft_enabled))); } +#ifdef CONFIG_PERF_COUNTERS +notrace void __weak perf_counter_do_pending(void) +{ +set_perf_counter_pending (0 ); +} +#endif + notrace void raw_local_irq_restore(unsigned long en) {/*
@@ -135,6 +142,9 @@ notrace void raw_local_irq_restore(unsigned long en)iseries_handle_interrupts (); } +if (get_perf_counter_pending ()) +perf_counter_do_pending (); +/* * if (get_paca()->hard_enabled) return; * But again we need to take care that gcc gets hard_enabled directly
@@ -231,7 +241,7 @@ void fixup_irqs(cpumask_t map)if (irq_desc[irq].status &IRQ_PER_CPU )continue ; -cpus_and (mask, irq_desc[irq].affinity, map); +cpumask_and (&mask, irq_desc[irq].affinity, &map);if (any_online_cpu (mask) ==NR_CPUS ) {printk ("Breaking affinity for irq %i\n" , irq); mask = map;
arch/powerpc/kernel/perf_counter.c new file +785 -0
785 lines not on this page · raw message
arch/powerpc/kernel/power6-pmu.c new file +283 -0
283 lines not on this page · raw message
arch/powerpc/kernel/ppc970-pmu.c new file +375 -0
375 lines not on this page · raw message
arch/powerpc/kernel/vmlinux.lds.S +1 -0
@@ -184,6 +184,7 @@ SECTIONS . = ALIGN(PAGE_SIZE); .data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) { __per_cpu_start = .; + *(.data.percpu.page_aligned) *(.data.percpu) *(.data.percpu.shared_aligned) __per_cpu_end = .;
arch/powerpc/platforms/Kconfig.cputype +1 -0
@@ -1,6 +1,7 @@ config PPC64 bool "64-bit kernel" default n + select HAVE_PERF_COUNTERS help This option selects whether a 32-bit or a 64-bit kernel will be built.
arch/powerpc/platforms/pseries/xics.c +3 -2
@@ -153,9 +153,10 @@ static int get_irq_server(unsigned int virq, unsigned int strict_check) {int server;/* For the moment only implement delivery to all cpus or one cpu */ -cpumask_t cpumask = irq_desc[virq].affinity; +cpumask_t cpumask;cpumask_t tmp =CPU_MASK_NONE ; +cpumask_copy (&cpumask, irq_desc[virq].affinity);if (!distribute_irqs)return default_server;
@@ -869,7 +870,7 @@ void xics_migrate_irqs_away(void) virq, cpu);/* Reset affinity to all cpus */ - irq_desc[virq].affinity =CPU_MASK_ALL ; +cpumask_setall (irq_desc[virq].affinity); desc->chip->set_affinity (virq, cpu_all_mask); unlock:spin_unlock_irqrestore (&desc->lock, flags);
arch/powerpc/sysdev/mpic.c +2 -1
@@ -566,9 +566,10 @@ static void __init mpic_scan_ht_pics(struct mpic *mpic)#ifdef CONFIG_SMP static int irq_choose_cpu (unsigned int virt_irq) { -cpumask_t mask = irq_desc[virt_irq].affinity; +cpumask_t mask;int cpuid; +cpumask_copy (&mask, irq_desc[virt_irq].affinity);if (cpus_equal (mask,CPU_MASK_ALL )) {static int irq_rover;static DEFINE_SPINLOCK (irq_rover_lock);
arch/sparc/kernel/irq_64.c +3 -2
@@ -247,9 +247,10 @@ struct irq_handler_data {#ifdef CONFIG_SMP static int irq_choose_cpu (unsigned int virt_irq) { -cpumask_t mask = irq_desc[virt_irq].affinity; +cpumask_t mask;int cpuid; +cpumask_copy (&mask, irq_desc[virt_irq].affinity);if (cpus_equal (mask,CPU_MASK_ALL )) {static int irq_rover;static DEFINE_SPINLOCK (irq_rover_lock);
@@ -854,7 +855,7 @@ void fixup_irqs(void) !(irq_desc[irq].status & IRQ_PER_CPU)) {if (irq_desc[irq].chip->set_affinity) irq_desc[irq].chip->set_affinity (irq, - &irq_desc[irq].affinity); + irq_desc[irq].affinity); }spin_unlock_irqrestore (&irq_desc[irq].lock, flags); }
arch/sparc/kernel/time_64.c +1 -1
@@ -729,7 +729,7 @@ void timer_interrupt(int irq, struct pt_regs *regs)irq_enter (); - kstat_this_cpu.irqs[0 ]++; +kstat_incr_irqs_this_cpu (0 ,irq_to_desc (0 ));if (unlikely (!evt->event_handler)) { printk(KERN_WARNING
arch/x86/Kconfig +1 -0
@@ -685,6 +685,7 @@ config X86_UP_IOAPIC config X86_LOCAL_APIC def_bool y depends on X86_64 || (X86_32 && (X86_UP_APIC || (SMP && !X86_VOYAGER) || X86_GENERICARCH)) + select HAVE_PERF_COUNTERS if (!M386 && !M486) config X86_IO_APIC def_bool y
arch/x86/ia32/ia32entry.S +6 -5
@@ -112,8 +112,8 @@ ENTRY(ia32_sysenter_target) CFI_DEF_CFA rsp,0 CFI_REGISTER rsp,rbp SWAPGS_UNSAFE_STACK - movq %gs:pda_kernelstack, %rsp - addq $(PDA_STACKOFFSET),%rsp + movq PER_CPU_VAR(kernel_stack), %rsp + addq $(KERNEL_STACK_OFFSET),%rsp /* * No need to follow this irqs on/off section: the syscall * disabled irqs, here we enable it straight after entry:
@@ -273,13 +273,13 @@ ENDPROC(ia32_sysenter_target) ENTRY(ia32_cstar_target) CFI_STARTPROC32 simple CFI_SIGNAL_FRAME - CFI_DEF_CFA rsp,PDA_STACKOFFSET + CFI_DEF_CFA rsp,KERNEL_STACK_OFFSET CFI_REGISTER rip,rcx /*CFI_REGISTER rflags,r11*/ SWAPGS_UNSAFE_STACK movl %esp,%r8d CFI_REGISTER rsp,r8 - movq %gs:pda_kernelstack,%rsp + movq PER_CPU_VAR(kernel_stack),%rsp /* * No need to follow this irqs on/off section: the syscall * disabled irqs and here we enable it straight after entry:
@@ -823,7 +823,8 @@ ia32_sys_call_table: .quad compat_sys_signalfd4 .quad sys_eventfd2 .quad sys_epoll_create1 - .quad sys_dup3 /* 330 */ + .quad sys_dup3 /* 330 */ .quad sys_pipe2 .quad sys_inotify_init1 + .quad sys_perf_counter_open ia32_syscall_end:
arch/x86/include/asm/apicnum.h new file +12 -0
@@ -0,0 +1,12 @@ +#ifndef _ASM_X86_APICNUM_H +#define _ASM_X86_APICNUM_H + +/* define MAX_IO_APICS */ +#ifdef CONFIG_X86_32 +# define MAX_IO_APICS 64 +#else +# define MAX_IO_APICS 128 +# define MAX_LOCAL_APIC 32768 +#endif + +#endif /* _ASM_X86_APICNUM_H */
arch/x86/include/asm/atomic_32.h +218 -0
223 lines not on this page · raw message
arch/x86/include/asm/bitops.h +10 -4
@@ -3,6 +3,9 @@/* * Copyright 1992, Linus Torvalds. +* +* Note: inlines with more than a single statement should be marked +* __always_inline to avoid problems with older gcc's inlining heuristics. */ #ifndef _LINUX_BITOPS_H
@@ -53,7 +56,8 @@* Note that @nr may be almost arbitrarily large; this function is not * restricted to acting on a single-word quantity. */ -static inline void set_bit (unsigned int nr,volatile unsigned long *addr) +static __always_inline void +set_bit (unsigned int nr,volatile unsigned long *addr) {if (IS_IMMEDIATE (nr)) { asm volatile(LOCK_PREFIX "orb %1,%0"
@@ -90,7 +94,8 @@ static inline void __set_bit(int nr, volatile unsigned long *addr)* you should call smp_mb__before_clear_bit() and/or smp_mb__after_clear_bit() * in order to ensure changes are visible on other processors. */ -static inline void clear_bit (int nr,volatile unsigned long *addr) +static __always_inline void +clear_bit (int nr,volatile unsigned long *addr) {if (IS_IMMEDIATE (nr)) { asm volatile(LOCK_PREFIX "andb %1,%0"
@@ -204,7 +209,8 @@ static inline int test_and_set_bit(int nr, volatile unsigned long *addr)* * This is the same as test_and_set_bit on x86. */ -static inline int test_and_set_bit_lock (int nr,volatile unsigned long *addr) +static __always_inline int +test_and_set_bit_lock (int nr,volatile unsigned long *addr) {return test_and_set_bit (nr, addr); }
@@ -300,7 +306,7 @@ static inline int test_and_change_bit(int nr, volatile unsigned long *addr)return oldbit; } -static inline int constant_test_bit (unsigned int nr,const volatile unsigned long *addr) +static __always_inline int constant_test_bit(unsigned int nr, const volatile unsigned long *addr) {return ((1UL << (nr %BITS_PER_LONG )) & (((unsigned long *)addr)[nr /BITS_PER_LONG ])) !=0 ;
arch/x86/include/asm/cpu.h +21 -0
@@ -7,6 +7,20 @@#include <linux/nodemask.h> #include <linux/percpu.h> +#ifdef CONFIG_SMP + +extern void prefill_possible_map (void ); + +#else /* CONFIG_SMP */ + +static inline void prefill_possible_map (void ) {} + +#define cpu_physical_id (cpu) boot_cpu_physical_apicid +#define safe_smp_processor_id () 0 +#define stack_smp_processor_id () 0 + +#endif /* CONFIG_SMP */ +struct x86_cpu {struct cpu cpu; };
@@ -17,4 +31,11 @@ extern void arch_unregister_cpu(int);#endif DECLARE_PER_CPU (int, cpu_state); + +#ifdef CONFIG_X86_HAS_BOOT_CPU_ID +extern unsigned char boot_cpu_id; +#else +#define boot_cpu_id 0 +#endif +#endif /* _ASM_X86_CPU_H */
arch/x86/include/asm/cpumask.h new file +28 -0
@@ -0,0 +1,28 @@ +#ifndef _ASM_X86_CPUMASK_H +#define _ASM_X86_CPUMASK_H +#ifndef __ASSEMBLY__ +#include <linux/cpumask.h> + +#ifdef CONFIG_X86_64 + +extern cpumask_var_t cpu_callin_mask; +extern cpumask_var_t cpu_callout_mask; +extern cpumask_var_t cpu_initialized_mask; +extern cpumask_var_t cpu_sibling_setup_mask; + +#else /* CONFIG_X86_32 */ + +extern cpumask_t cpu_callin_map; +extern cpumask_t cpu_callout_map; +extern cpumask_t cpu_initialized; +extern cpumask_t cpu_sibling_setup_map; + +#define cpu_callin_mask ((struct cpumask *)&cpu_callin_map) +#define cpu_callout_mask ((struct cpumask *)&cpu_callout_map) +#define cpu_initialized_mask ((struct cpumask *)&cpu_initialized) +#define cpu_sibling_setup_mask ((struct cpumask *)&cpu_sibling_setup_map) + +#endif /* CONFIG_X86_32 */ + +#endif /* __ASSEMBLY__ */ +#endif /* _ASM_X86_CPUMASK_H */
arch/x86/include/asm/current.h +3 -21
@@ -1,39 +1,21 @@#ifndef _ASM_X86_CURRENT_H#define _ASM_X86_CURRENT_H -#ifdef CONFIG_X86_32 #include <linux/compiler.h> #include <asm/percpu.h> +#ifndef __ASSEMBLY__struct task_struct ; DECLARE_PER_CPU(struct task_struct *, current_task); -static __always_inline struct task_struct *get_current(void) -{ -return x86_read_percpu (current_task); -} - -#else /* X86_32 */ - -#ifndef __ASSEMBLY__ -#include <asm/pda.h> - -struct task_struct ; static __always_inline struct task_struct *get_current(void) { -return read_pda (pcurrent); +return percpu_read (current_task); } -#else /* __ASSEMBLY__ */ - -#include <asm/asm-offsets.h> -#define GET_CURRENT (reg) movq %gs:(pda_pcurrent),reg +#define current get_current()#endif /* __ASSEMBLY__ */ -#endif /* X86_32 */ - -#define current get_current() -#endif /* _ASM_X86_CURRENT_H */
arch/x86/include/asm/hardirq_32.h +4 -0
@@ -9,6 +9,7 @@ typedef struct {unsigned long idle_timestamp;unsigned int __nmi_count;/* arch dependent */ unsigned int apic_timer_irqs;/* arch dependent */ +unsigned int apic_perf_irqs;/* arch dependent */ unsigned int irq0_irqs;unsigned int irq_resched_count;unsigned int irq_call_count;
@@ -19,6 +20,9 @@ typedef struct {DECLARE_PER_CPU (irq_cpustat_t, irq_stat); +/* We can have at most NR_VECTORS irqs routed to a cpu at a time */ +#define MAX_HARDIRQS_PER_CPU NR_VECTORS +#define __ARCH_IRQ_STAT#define __IRQ_STAT (cpu, member) (per_cpu(irq_stat, cpu).member)
arch/x86/include/asm/hardirq_64.h +20 -5
@@ -3,22 +3,37 @@#include <linux/threads.h> #include <linux/irq.h> -#include <asm/pda.h> #include <asm/apic.h> +typedef struct { +unsigned int __softirq_pending; +unsigned int __nmi_count;/* arch dependent */ +unsigned int apic_timer_irqs;/* arch dependent */ +unsigned int apic_perf_irqs;/* arch dependent */ +unsigned int irq0_irqs; +unsigned int irq_resched_count; +unsigned int irq_call_count; +unsigned int irq_tlb_count; +unsigned int irq_thermal_count; +unsigned int irq_spurious_count; +unsigned int irq_threshold_count; +}____cacheline_aligned irq_cpustat_t ; + +DECLARE_PER_CPU (irq_cpustat_t, irq_stat); +/* We can have at most NR_VECTORS irqs routed to a cpu at a time */ #define MAX_HARDIRQS_PER_CPU NR_VECTORS#define __ARCH_IRQ_STAT 1 -#define inc_irq_stat (member) add_pda(member, 1) +#define inc_irq_stat (member) percpu_add(irq_stat.member, 1) -#define local_softirq_pending () read_pda(__softirq_pending) +#define local_softirq_pending () percpu_read(irq_stat.__softirq_pending)#define __ARCH_SET_SOFTIRQ_PENDING 1 -#define set_softirq_pending (x) write_pda(__softirq_pending, (x)) -#define or_softirq_pending (x) or_pda(__softirq_pending, (x)) +#define set_softirq_pending (x) percpu_write(irq_stat.__softirq_pending, (x)) +#define or_softirq_pending (x) percpu_or(irq_stat.__softirq_pending, (x))extern void ack_bad_irq (unsigned int irq);
arch/x86/include/asm/hw_irq.h +2 -0
@@ -30,6 +30,8 @@/* Interrupt handlers registered during init_IRQ */ extern void apic_timer_interrupt (void );extern void error_interrupt (void ); +extern void perf_counter_interrupt (void ); +extern void spurious_interrupt (void );extern void thermal_interrupt (void );extern void reschedule_interrupt (void );
arch/x86/include/asm/intel_arch_perfmon.h deleted +0 -31
@@ -1,31 +0,0 @@ -#ifndef _ASM_X86_INTEL_ARCH_PERFMON_H -#define _ASM_X86_INTEL_ARCH_PERFMON_H - -#define MSR_ARCH_PERFMON_PERFCTR0 0xc1 -#define MSR_ARCH_PERFMON_PERFCTR1 0xc2 - -#define MSR_ARCH_PERFMON_EVENTSEL0 0x186 -#define MSR_ARCH_PERFMON_EVENTSEL1 0x187 - -#define ARCH_PERFMON_EVENTSEL0_ENABLE (1 << 22) -#define ARCH_PERFMON_EVENTSEL_INT (1 << 20) -#define ARCH_PERFMON_EVENTSEL_OS (1 << 17) -#define ARCH_PERFMON_EVENTSEL_USR (1 << 16) - -#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_SEL (0x3c) -#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_UMASK (0x00 << 8) -#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_INDEX (0) -#define ARCH_PERFMON_UNHALTED_CORE_CYCLES_PRESENT \ - (1 << (ARCH_PERFMON_UNHALTED_CORE_CYCLES_INDEX)) - -union cpuid10_eax { -struct { -unsigned int version_id:8 ; -unsigned int num_counters:8 ; -unsigned int bit_width:8 ; -unsigned int mask_length:8 ; - } split; -unsigned int full; -}; - -#endif /* _ASM_X86_INTEL_ARCH_PERFMON_H */
arch/x86/include/asm/io_apic.h +2 -24
@@ -114,38 +114,16 @@ struct IR_IO_APIC_route_entry {extern int nr_ioapics;extern int nr_ioapic_registers[MAX_IO_APICS ]; -/* -* MP-BIOS irq configuration table structures: -*/ -#define MP_MAX_IOAPIC_PIN 127 -struct mp_config_ioapic { -unsigned long mp_apicaddr; -unsigned int mp_apicid; -unsigned char mp_type; -unsigned char mp_apicver; -unsigned char mp_flags; -}; - -struct mp_config_intsrc { -unsigned int mp_dstapic; -unsigned char mp_type; -unsigned char mp_irqtype; -unsigned short mp_irqflag; -unsigned char mp_srcbus; -unsigned char mp_srcbusirq; -unsigned char mp_dstirq; -}; -/* I/O APIC entries */ -extern struct mp_config_ioapic mp_ioapics[MAX_IO_APICS ]; +extern struct mpc_ioapic mp_ioapics[MAX_IO_APICS ];/* # of MP IRQ source entries */ extern int mp_irq_entries;/* MP IRQ source entries */ -extern struct mp_config_intsrc mp_irqs[MAX_IRQ_SOURCES ]; +extern struct mpc_intsrc mp_irqs[MAX_IRQ_SOURCES ];/* non-0 if default (table-less) MP configuration */ extern int mpc_default_type;
arch/x86/include/asm/irq_regs_32.h +2 -2
@@ -15,7 +15,7 @@ DECLARE_PER_CPU(struct pt_regs *, irq_regs);static inline struct pt_regs *get_irq_regs (void ) { -return x86_read_percpu (irq_regs); +return percpu_read (irq_regs); }static inline struct pt_regs *set_irq_regs (struct pt_regs *new_regs)
@@ -23,7 +23,7 @@ static inline struct pt_regs *set_irq_regs(struct pt_regs *new_regs)struct pt_regs *old_regs; old_regs =get_irq_regs (); -x86_write_percpu (irq_regs, new_regs); +percpu_write (irq_regs, new_regs);return old_regs; }
arch/x86/include/asm/irq_vectors.h +13 -5
@@ -87,6 +87,11 @@#define LOCAL_TIMER_VECTOR 0xef/* +* Performance monitoring interrupt vector: +*/ +#define LOCAL_PERF_VECTOR 0xee + +/* * First APIC vector available to drivers: (vectors 0x30-0xee) we * start at 0x31(0x41) to spread out vectors evenly between priority * levels. (0x80 is the syscall vector)
@@ -105,6 +110,8 @@#if defined(CONFIG_X86_IO_APIC ) && !defined(CONFIG_X86_VOYAGER ) +#include <asm/apicnum.h> /* need MAX_IO_APICS */ +#ifndef CONFIG_SPARSE_IRQ # if NR_CPUS <MAX_IO_APICS # define NR_IRQS (NR_VECTORS + (32 * NR_CPUS))
@@ -112,11 +119,12 @@# define NR_IRQS (NR_VECTORS + (32 * MAX_IO_APICS))# endif #else -# if (8 *NR_CPUS ) > (32 *MAX_IO_APICS ) -# define NR_IRQS (NR_VECTORS + (8 * NR_CPUS)) -# else -# define NR_IRQS (NR_VECTORS + (32 * MAX_IO_APICS)) -# endif + +# define NR_IRQS \ + ((8 * NR_CPUS) > (32 * MAX_IO_APICS) ? \ + (NR_VECTORS + (8 * NR_CPUS)) : \ + (NR_VECTORS + (32 * MAX_IO_APICS))) \ +#endif #elif defined(CONFIG_X86_VOYAGER)
arch/x86/include/asm/mach-default/entry_arch.h +5 -0
@@ -25,10 +25,15 @@ BUILD_INTERRUPT(irq_move_cleanup_interrupt,IRQ_MOVE_CLEANUP_VECTOR)* a much simpler SMP time architecture: */ #ifdef CONFIG_X86_LOCAL_APIC + BUILD_INTERRUPT(apic_timer_interrupt,LOCAL_TIMER_VECTOR) BUILD_INTERRUPT(error_interrupt,ERROR_APIC_VECTOR) BUILD_INTERRUPT(spurious_interrupt,SPURIOUS_APIC_VECTOR) +#ifdef CONFIG_PERF_COUNTERS +BUILD_INTERRUPT(perf_counter_interrupt, LOCAL_PERF_VECTOR) +#endif +#ifdef CONFIG_X86_MCE_P4THERMAL BUILD_INTERRUPT(thermal_interrupt,THERMAL_APIC_VECTOR)#endif
arch/x86/include/asm/mmu_context_32.h +6 -6
@@ -4,8 +4,8 @@static inline void enter_lazy_tlb (struct mm_struct *mm,struct task_struct *tsk) {#ifdef CONFIG_SMP -if (x86_read_percpu (cpu_tlbstate.state) ==TLBSTATE_OK ) -x86_write_percpu (cpu_tlbstate.state,TLBSTATE_LAZY ); +if (percpu_read (cpu_tlbstate.state) ==TLBSTATE_OK ) +percpu_write (cpu_tlbstate.state,TLBSTATE_LAZY );#endif }
@@ -19,8 +19,8 @@ static inline void switch_mm(struct mm_struct *prev,/* stop flush ipis for the previous mm */ cpu_clear (cpu, prev->cpu_vm_mask);#ifdef CONFIG_SMP -x86_write_percpu (cpu_tlbstate.state,TLBSTATE_OK ); -x86_write_percpu (cpu_tlbstate.active_mm, next); +percpu_write (cpu_tlbstate.state,TLBSTATE_OK ); +percpu_write (cpu_tlbstate.active_mm, next);#endif cpu_set (cpu, next->cpu_vm_mask);
@@ -35,8 +35,8 @@ static inline void switch_mm(struct mm_struct *prev, }#ifdef CONFIG_SMP else { -x86_write_percpu (cpu_tlbstate.state,TLBSTATE_OK ); -BUG_ON (x86_read_percpu (cpu_tlbstate.active_mm) != next); +percpu_write (cpu_tlbstate.state,TLBSTATE_OK ); +BUG_ON (percpu_read (cpu_tlbstate.active_mm) != next);if (!cpu_test_and_set (cpu, next->cpu_vm_mask)) {/* We were in lazy tlb mode and leave_mm disabled
arch/x86/include/asm/mmu_context_64.h +7 -9
@@ -1,13 +1,11 @@#ifndef _ASM_X86_MMU_CONTEXT_64_H#define _ASM_X86_MMU_CONTEXT_64_H -#include <asm/pda.h> -static inline void enter_lazy_tlb (struct mm_struct *mm,struct task_struct *tsk) {#ifdef CONFIG_SMP -if (read_pda (mmu_state) ==TLBSTATE_OK ) -write_pda (mmu_state,TLBSTATE_LAZY ); +if (percpu_read (cpu_tlbstate.state) ==TLBSTATE_OK ) +percpu_write (cpu_tlbstate.state,TLBSTATE_LAZY );#endif }
@@ -19,8 +17,8 @@ static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next,/* stop flush ipis for the previous mm */ cpu_clear (cpu, prev->cpu_vm_mask);#ifdef CONFIG_SMP -write_pda (mmu_state,TLBSTATE_OK ); -write_pda (active_mm, next); +percpu_write (cpu_tlbstate.state,TLBSTATE_OK ); +percpu_write (cpu_tlbstate.active_mm, next);#endif cpu_set (cpu, next->cpu_vm_mask);load_cr3 (next->pgd);
@@ -30,9 +28,9 @@ static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next, }#ifdef CONFIG_SMP else { -write_pda (mmu_state,TLBSTATE_OK ); -if (read_pda (active_mm) != next) -BUG (); +percpu_write (cpu_tlbstate.state,TLBSTATE_OK ); +BUG_ON (percpu_read (cpu_tlbstate.active_mm) != next); +if (!cpu_test_and_set (cpu, next->cpu_vm_mask)) {/* We were in lazy tlb mode and leave_mm disabled * tlb flush IPI delivery. We must reload CR3
arch/x86/include/asm/mpspec_def.h +12 -11
@@ -24,17 +24,18 @@# endif #endif -struct intel_mp_floating { -char mpf_signature[4 ];/* "_MP_" */ -unsigned int mpf_physptr;/* Configuration table address */ -unsigned char mpf_length;/* Our length (paragraphs) */ -unsigned char mpf_specification;/* Specification version */ -unsigned char mpf_checksum;/* Checksum (makes sum 0) */ -unsigned char mpf_feature1;/* Standard or configuration ? */ -unsigned char mpf_feature2;/* Bit7 set for IMCR|PIC */ -unsigned char mpf_feature3;/* Unused (0) */ -unsigned char mpf_feature4;/* Unused (0) */ -unsigned char mpf_feature5;/* Unused (0) */ +/* Intel MP Floating Pointer Structure */ +struct mpf_intel { +char signature[4 ];/* "_MP_" */ +unsigned int physptr;/* Configuration table address */ +unsigned char length;/* Our length (paragraphs) */ +unsigned char specification;/* Specification version */ +unsigned char checksum;/* Checksum (makes sum 0) */ +unsigned char feature1;/* Standard or configuration ? */ +unsigned char feature2;/* Bit7 set for IMCR|PIC */ +unsigned char feature3;/* Unused (0) */ +unsigned char feature4;/* Unused (0) */ +unsigned char feature5;/* Unused (0) */ };#define MPC_SIGNATURE "PCMP"
arch/x86/include/asm/page_64.h +2 -2
@@ -13,8 +13,8 @@#define DEBUG_STACK_ORDER (EXCEPTION_STACK_ORDER + 1)#define DEBUG_STKSZ (PAGE_SIZE << DEBUG_STACK_ORDER) -#define IRQSTACK_ORDER 2 -#define IRQSTACKSIZE (PAGE_SIZE << IRQSTACK_ORDER) +#define IRQ_STACK_ORDER 2 +#define IRQ_STACK_SIZE (PAGE_SIZE << IRQ_STACK_ORDER)#define STACKFAULT_STACK 1#define DOUBLEFAULT_STACK 2
arch/x86/include/asm/paravirt.h +5 -3
@@ -244,7 +244,8 @@ struct pv_mmu_ops {void (*flush_tlb_user)(void );void (*flush_tlb_kernel)(void );void (*flush_tlb_single)(unsigned long addr); -void (*flush_tlb_others)(const cpumask_t *cpus,struct mm_struct *mm, +void (*flush_tlb_others)(const struct cpumask *cpus, +struct mm_struct *mm,unsigned long va);/* Hooks for allocating and freeing a pagetable top-level */
@@ -984,10 +985,11 @@ static inline void __flush_tlb_single(unsigned long addr)PVOP_VCALL1 (pv_mmu_ops.flush_tlb_single, addr); } -static inline void flush_tlb_others (cpumask_t cpumask,struct mm_struct *mm, +static inline void flush_tlb_others (const struct cpumask *cpumask, +struct mm_struct *mm,unsigned long va) { -PVOP_VCALL3 (pv_mmu_ops.flush_tlb_others, &cpumask, mm, va); +PVOP_VCALL3 (pv_mmu_ops.flush_tlb_others, cpumask, mm, va); }static inline int paravirt_pgd_alloc (struct mm_struct *mm)
arch/x86/include/asm/pda.h +16 -108
79 lines not on this page · raw message
arch/x86/include/asm/percpu.h +65 -94
159 lines not on this page · raw message
arch/x86/include/asm/perf_counter.h new file +95 -0
95 lines not on this page · raw message
arch/x86/include/asm/smp.h +3 -46
49 lines not on this page · raw message
arch/x86/kernel/acpi/boot.c +47 -49
96 lines not on this page · raw message
arch/x86/kernel/cpu/common.c +32 -50
82 lines not on this page · raw message
arch/x86/kernel/cpu/cpufreq/acpi-cpufreq.c +0 -2
Lines of this file are not shown. View the raw message.
arch/x86/kernel/cpu/intel_cacheinfo.c +44 -19
63 lines not on this page · raw message
arch/x86/kernel/cpu/mcheck/mce_amd_64.c +14 -7
Lines of this file are not shown. View the raw message.
arch/x86/kernel/cpu/perf_counter.c new file +695 -0
695 lines not on this page · raw message
arch/x86/kernel/entry_64.S +26 -20
46 lines not on this page · raw message
arch/x86/kernel/head_64.S +41 -6
47 lines not on this page · raw message
arch/x86/kernel/io_apic.c +92 -71
163 lines not on this page · raw message
arch/x86/kernel/mpparse.c +71 -71
142 lines not on this page · raw message
arch/x86/kernel/setup_percpu.c +118 -90
208 lines not on this page · raw message
arch/x86/kernel/smpboot.c +7 -62
69 lines not on this page · raw message
arch/x86/kernel/tlb_32.c +34 -51
85 lines not on this page · raw message
arch/x86/kernel/tlb_64.c +43 -33
76 lines not on this page · raw message
arch/x86/xen/enlighten.c +20 -27
47 lines not on this page · raw message
include/asm-generic/percpu.h +52 -0
52 lines not on this page · raw message
include/asm-generic/vmlinux.lds.h +69 -6
75 lines not on this page · raw message
include/linux/irq.h +83 -3
86 lines not on this page · raw message
include/linux/perf_counter.h new file +290 -0
290 lines not on this page · raw message
kernel/irq/handle.c +35 -22
57 lines not on this page · raw message
kernel/perf_counter.c new file +2169 -0
2,169 lines not on this page · raw message
kernel/sched.c +73 -3
76 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.
Quoted 23 lines · Ingo Molnar
We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:
http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patchIt's also available in tip/master at:
http://people.redhat.com/mingo/tip.git/README
There are many changes in the v6 release:
- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.
- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc instrumentation and PAPI support.
- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.
Quoted 7 lines
- The 'perfstat' utility (ex 'timec') has been updated:
http://people.redhat.com/mingo/perfcounters/perfstat.c
- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:
BTW, this kerneltop has nothing to do with that other one??
http://www.xenotime.net/linux/kerneltop/
Thanks,
Signature · 1 line
~Randy
Quoted 28 lines · Randy Dunlap
Quoted 27 lines · Ingo Molnar
We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:
http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patchIt's also available in tip/master at:
http://people.redhat.com/mingo/tip.git/README
There are many changes in the v6 release:
- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.
- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc instrumentation and PAPI support.
- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.
- The 'perfstat' utility (ex 'timec') has been updated:
Quoted 7 lines
Quoted 3 lines
- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:
BTW, this kerneltop has nothing to do with that other one??
heh, didnt know about that one - there's no connection other than the name :-) The project seems somewhat stale but indeed similar in purpose. Can rename to kerneltop2 i guess.
Ingo
Quoted 32 lines · Ingo Molnar
Quoted 30 lines · Randy Dunlap
Quoted 29 lines · Ingo Molnar
We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:
http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patchIt's also available in tip/master at:
http://people.redhat.com/mingo/tip.git/README
There are many changes in the v6 release:
- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.
- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc instrumentation and PAPI support.
- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.
- The 'perfstat' utility (ex 'timec') has been updated:
http://people.redhat.com/mingo/perfcounters/perfstat.c
- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
BTW, this kerneltop has nothing to do with that other one??
heh, didnt know about that one - there's no connection other than the name :-) The project seems somewhat stale but indeed similar in purpose. Can rename to kerneltop2 i guess.
Yes, it's stale. I plan to update it sometime this year. :)
I don't care if you rename it or not.
Signature · 1 line
~Randy
Hi,
Quoted 3 lines · Ingo Molnar
- The 'perfstat' utility (ex 'timec') has been updated:
what are your planning to do with this utility? We can merge it into util-linux-ng.
Karel
Signature · 1 line
Karel Zak <[email protected]>
Quoted 6 lines · Karel Zak
Quoted 3 lines · Ingo Molnar
- The 'perfstat' utility (ex 'timec') has been updated:
what are your planning to do with this utility? We can merge it into
grr.. s/your/you/
Quoted 3 lines
util-linux-ng.
Karel
Signature · 1 line
Karel Zak <[email protected]>
Quoted 7 lines · Karel Zak
Quoted 3 lines · Ingo Molnar
- The 'perfstat' utility (ex 'timec') has been updated:
what are your planning to do with this utility? We can merge it into util-linux-ng.
That would be nice to do, if/once/when the subsystem and the syscall itself is merged upstream. The syscall ABI might change until that happens.
Ingo
Quoted 26 lines · Ingo Molnar
We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:
http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patchIt's also available in tip/master at:
http://people.redhat.com/mingo/tip.git/README
There are many changes in the v6 release:
- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.
- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc instrumentation and PAPI support.
- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.
- The 'perfstat' utility (ex 'timec') has been updated:
http://people.redhat.com/mingo/perfcounters/perfstat.c
- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:
Quoted 7 lines
http://people.redhat.com/mingo/perfcounters/kerneltop.c
- Merged to latest mainline
- Various fixes and other updates
Ingo
Hi Ingo,
Looking over the latest capabilities of this proposal, I am wondering how it can accommodate performance monitor units which have extra registers which require user-defined data to be loaded into them.
For example, on the Power architecture, there is an Instruction Matching Register which allows the counting of particular instructions. Currently, this is unsupported in perfmon2/3, but we have plans to add it, and it's pretty straight-forward to imagine how this would be done in perfmon.
But I don't see an obvious way to do it with your proposal. Do you have any ideas how Performance Counters for Linux could accommodate this sort of PMU functionality?
One thought would be to change the event code to an event descriptor structure, which has room for lots of bits, including arch-defined bits (in the case of Power, an IMR value, and others). This might also be a way to accommodate unit masks (and enums) as well, which Andi Kleen pointed out as an issue in an earlier LKML posting.
Regards,
- Corey
Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]
Hi,
Corey brings up an interesting problem which I wanted to comment on.
The current proposal hinges on the idea that by interpreting a single value the kernel
can understand what the user wants to measure. For instance, if I pass type=0, then
the kernel understands I want to measure CPU_CYCLES. Given that the number of events and their unit mask combinations can be large, the proposal also provides a "raw" mode, where the content of the type field is interpreted as the raw value to
put into a register.
This is where there is an issue because with several PMU models, including on X86, using
the raw bit + 64 value is not enough to figure out what the user wants to measure. This happens
when the PMU has more than counters. Thus, interpreting each raw value has the event code
may be wrong. To remain on familiar territory, the Nehalem uncore PMU has an opcode matcher register,
that uses a 64-bit value. On AMD64 Family 10h, you have IBS. But I could give examples on
Itanium with opcode matchers, range restrictions. Corey provided other examples for Power.
The API has to provide a way to express what the raw value is meant
for: counter, matcher, filter...
There are PMU where programming an event requires writing two config registers. This is the case
for all Netburst-based processors where you have to program CCCR and ESCR. I wonder how,
raw mode is supported for those processors. What if a PMU requires three registers to be programmed?
Quoted 72 lines · Corey Ashford
Quoted 29 lines · Ingo Molnar
We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:
http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patch
It's also available in tip/master at:
http://people.redhat.com/mingo/tip.git/README
There are many changes in the v6 release:
- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.
- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc
instrumentation and PAPI support.- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.
- The 'perfstat' utility (ex 'timec') has been updated:
Quoted 8 lines
- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:
http://people.redhat.com/mingo/perfcounters/kerneltop.c- Merged to latest mainline
- Various fixes and other updates
Ingo
Hi Ingo,
Looking over the latest capabilities of this proposal, I am wondering how it can accommodate performance monitor units which have extra registers which require user-defined data to be loaded into them.
For example, on the Power architecture, there is an Instruction Matching Register which allows the counting of particular instructions. Currently, this is unsupported in perfmon2/3, but we have plans to add it, and it's pretty straight-forward to imagine how this would be done in perfmon.
But I don't see an obvious way to do it with your proposal. Do you have any ideas how Performance Counters for Linux could accommodate this sort of PMU functionality?
One thought would be to change the event code to an event descriptor structure, which has room for lots of bits, including arch-defined bits (in the case of Power, an IMR value, and others). This might also be a way to accommodate unit masks (and enums) as well, which Andi Kleen pointed out as an issue in an earlier LKML posting.
Regards,
- Corey
Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]
Quoted 12 lines · stephane eranian
Hi,
Corey brings up an interesting problem which I wanted to comment on.
The current proposal hinges on the idea that by interpreting a single value the kernel can understand what the user wants to measure. For instance, if I pass type=0, then the kernel understands I want to measure
CPU_CYCLES. Given that the number of events and their unit mask combinations can be large, the proposal also provides a "raw" mode, where the content of the type field is interpreted as the raw value to put into a register.
Quoted 10 lines
This is where there is an issue because with several PMU models, including on X86, using the raw bit + 64 value is not enough to figure out what the user wants to measure. This happens when the PMU has more than counters. Thus, interpreting each raw value has the event code may be wrong. To remain on familiar territory, the Nehalem uncore PMU has an opcode matcher register, that uses a 64-bit value. On AMD64 Family 10h, you have IBS. But I could give examples on Itanium with opcode matchers, range restrictions. Corey provided other examples for Power. The API has to provide a way to express what the raw value is meant for: counter, matcher, filter...
this can be done in a number of ways (in order of increasing levels of abstraction):
- the raw type is kept wide enough. Paul already requested the raw type to be widened to 128 bits to express certain PowerPC features.
- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the write() method or ioctl is extended to express attributes we want to set/change while a counter is running.
- or the highest level counter / hw event data type is extended with new attribute field(s).
My feeling is that we generally want such hw features to start small - i.e. at the raw type level initially. Then we can allow them to climb the ladder, if they prove their utility in practice. We've got space reserved in the ABI to allow for growth like this.
Ingo
Quoted 26 lines · Ingo Molnar
Quoted 12 lines · stephane eranian
Hi,
Corey brings up an interesting problem which I wanted to comment on.
The current proposal hinges on the idea that by interpreting a single value the kernel can understand what the user wants to measure. For instance, if I pass type=0, then the kernel understands I want to measure
CPU_CYCLES. Given that the number of events and their unit mask combinations can be large, the proposal also provides a "raw" mode, where the content of the type field is interpreted as the raw value to put into a register.Quoted 10 lines
This is where there is an issue because with several PMU models, including on X86, using the raw bit + 64 value is not enough to figure out what the user wants to measure. This happens when the PMU has more than counters. Thus, interpreting each raw value has the event code may be wrong. To remain on familiar territory, the Nehalem uncore PMU has an opcode matcher register, that uses a 64-bit value. On AMD64 Family 10h, you have IBS. But I could give examples on Itanium with opcode matchers, range restrictions. Corey provided other examples for Power. The API has to provide a way to express what the raw value is meant for: counter, matcher, filter...
Quoted 5 lines
this can be done in a number of ways (in order of increasing levels of abstraction):
- the raw type is kept wide enough. Paul already requested the raw type to be widened to 128 bits to express certain PowerPC features.
Yes, 1 bit is not enough. With 128 that would be enough to encode all the resources I can think of for existing Itanium, X86.
But then, I think fields would have to be renamed to make it clearer. Raw would denote a type of resource and the current type field could be renamed 'code' or 'val' or 'id' which reflects more what the content actually would be.
As for Netburst, both CCCR and ESCR only use the bottom 32-bit so they could be stuffed into the 64-bit 'type' field. But that would not work if a PMU were to require wider values. Those values are part of the event encoding and should not be considered optional nor attributes. That is why using a separate call to program the second value does not seem appropriate to me.
Quoted 15 lines
- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the
write()method or ioctl is extended to express attributes we want to set/change while a counter is running.- or the highest level counter / hw event data type is extended with new attribute field(s).
My feeling is that we generally want such hw features to start small - i.e. at the raw type level initially. Then we can allow them to climb the ladder, if they prove their utility in practice. We've got space reserved in the ABI to allow for growth like this.
Ingo
Quoted 19 lines · Ingo Molnar
Quoted 7 lines · stephane eranian
Hi,
Corey brings up an interesting problem which I wanted to comment on.
The current proposal hinges on the idea that by interpreting a single value the kernel can understand what the user wants to measure. For instance, if I pass type=0, then the kernel understands I want to measure
CPU_CYCLES. Given that the number of events and their unit mask combinations can be large, the proposal also provides a "raw" mode, where the content of the type field is interpreted as the raw value to put into a register.This is where there is an issue because with several PMU models, including on X86, using the raw bit + 64 value is not enough to figure out what the user wants to measure. This happens when the PMU has more than counters. Thus, interpreting each raw value has the event code may be wrong. To remain on familiar territory, the Nehalem uncore PMU has an opcode matcher register, that uses a 64-bit value. On AMD64 Family 10h, you have IBS. But I could give examples on Itanium with opcode matchers, range restrictions. Corey provided other examples for Power. The API has to provide a way to express what the raw value is meant for: counter, matcher, filter...
this can be done in a number of ways (in order of increasing levels of abstraction):
- the raw type is kept wide enough. Paul already requested the raw type to be widened to 128 bits to express certain PowerPC features.
- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the
write()method or ioctl is extended to express attributes we want to set/change while a counter is running.
Quoted 6 lines
- or the highest level counter / hw event data type is extended with new attribute field(s).
My feeling is that we generally want such hw features to start small - i.e. at the raw type level initially. Then we can allow them to climb the ladder, if they prove their utility in practice. We've got space reserved in the ABI to allow for growth like this.
Ingo
Hi Ingo and Stephane,
Thanks for the replies.
I think any one of those solutions would work for Power's Instruction Matching Register. If more than one register needs to be programmed, or the values don't fit into the 128-bit raw event types, we could use the "special counter" approach, I think.
I will have another look at the Power PMU description and see if there are other constraints that might cause us to want to go one way or the other, or perhaps a different way.
Regards,
- Corey
Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]
Quoted 3 lines
- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the
write()method or ioctl is extended to express attributes we want to set/change while a counter is running.
The product of:
{exotic PMU modes} * {creative performance measurement ideas}will produce a large number of candidates for these special counters (at least on ia64 ... which has a large number of exotic PMU options).
I don't think that I'm qualified to judge which of them are "useful enough" to warrant a special counter type.
-Tony
Quoted 4 lines · Luck, Tony
Quoted 3 lines
- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the
write()method or ioctl is extended to express attributes we want to set/change while a counter is running.
Quoted 8 lines
The product of:
{exotic PMU modes} * {creative performance measurement ideas}will produce a large number of candidates for these special counters (at least on ia64 ... which has a large number of exotic PMU options).
I don't think that I'm qualified to judge which of them are "useful enough" to warrant a special counter type.
it should certainly be done on a case by case basis. They need to be consciously exposed not just summarily exported to user-space, because PMU hw features have security implications so it has to be done all explicitly.
Ingo
Quoted 51 lines · Corey Ashford
Quoted 38 lines · Ingo Molnar
Quoted 12 lines · stephane eranian
Hi,
Corey brings up an interesting problem which I wanted to comment on.
The current proposal hinges on the idea that by interpreting a single value the kernel can understand what the user wants to measure. For instance, if I pass type=0, then the kernel understands I want to measure
CPU_CYCLES. Given that the number of events and their unit mask combinations can be large, the proposal also provides a "raw" mode, where the content of the type field is interpreted as the raw value to put into a register.Quoted 10 lines
This is where there is an issue because with several PMU models, including on X86, using the raw bit + 64 value is not enough to figure out what the user wants to measure. This happens when the PMU has more than counters. Thus, interpreting each raw value has the event code may be wrong. To remain on familiar territory, the Nehalem uncore PMU has an opcode matcher register, that uses a 64-bit value. On AMD64 Family 10h, you have IBS. But I could give examples on Itanium with opcode matchers, range restrictions. Corey provided other examples for Power. The API has to provide a way to express what the raw value is meant for: counter, matcher, filter...
this can be done in a number of ways (in order of increasing levels of abstraction):
- the raw type is kept wide enough. Paul already requested the raw type to be widened to 128 bits to express certain PowerPC features.
- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the
write()method or ioctl is extended to express attributes we want to set/change while a counter is running.- or the highest level counter / hw event data type is extended with new attribute field(s).
Quoted 6 lines
My feeling is that we generally want such hw features to start small - i.e. at the raw type level initially. Then we can allow them to climb the ladder, if they prove their utility in practice. We've got space reserved in the ABI to allow for growth like this.
Ingo
Hi Ingo and Stephane,
Thanks for the replies.
Quoted 8 lines
I think any one of those solutions would work for Power's Instruction Matching Register. If more than one register needs to be programmed, or the values don't fit into the 128-bit raw event types, we could use the "special counter" approach, I think.
I will have another look at the Power PMU description and see if there are other constraints that might cause us to want to go one way or the other, or perhaps a different way.
thanks, that's really appreciated!
One useful approach would be to come up with a bitcount that you think would fit considering even (currently) fringe/odd features - and we'd make sure there's enough space for that in the ABI - should there be a need/desire to expose that in the future.
Ingo
Quoted 34 lines · Ingo Molnar
Quoted 32 lines · Corey Ashford
Quoted 18 lines · Ingo Molnar
Quoted 7 lines · stephane eranian
Hi,
Corey brings up an interesting problem which I wanted to comment on.
The current proposal hinges on the idea that by interpreting a single value the kernel can understand what the user wants to measure. For instance, if I pass type=0, then the kernel understands I want to measure
CPU_CYCLES. Given that the number of events and their unit mask combinations can be large, the proposal also provides a "raw" mode, where the content of the type field is interpreted as the raw value to put into a register.This is where there is an issue because with several PMU models, including on X86, using the raw bit + 64 value is not enough to figure out what the user wants to measure. This happens when the PMU has more than counters. Thus, interpreting each raw value has the event code may be wrong. To remain on familiar territory, the Nehalem uncore PMU has an opcode matcher register, that uses a 64-bit value. On AMD64 Family 10h, you have IBS. But I could give examples on Itanium with opcode matchers, range restrictions. Corey provided other examples for Power. The API has to provide a way to express what the raw value is meant for: counter, matcher, filter...
this can be done in a number of ways (in order of increasing levels of abstraction):
- the raw type is kept wide enough. Paul already requested the raw type to be widened to 128 bits to express certain PowerPC features.
- or the PMU capability is expressed as a special counter type (if it's useful enough) - and then either the
write()method or ioctl is extended to express attributes we want to set/change while a counter is running.Quoted 6 lines
- or the highest level counter / hw event data type is extended with new attribute field(s).
My feeling is that we generally want such hw features to start small - i.e. at the raw type level initially. Then we can allow them to climb the ladder, if they prove their utility in practice. We've got space reserved in the ABI to allow for growth like this.
Ingo
Hi Ingo and Stephane,
Thanks for the replies.
I think any one of those solutions would work for Power's Instruction Matching Register. If more than one register needs to be programmed, or the values don't fit into the 128-bit raw event types, we could use the "special counter" approach, I think.
Quoted 7 lines
Quoted 1 line
I will have another look at the Power PMU description and see if there are other constraints that might cause us to want to go one way or the other, or perhaps a different way.
thanks, that's really appreciated!
One useful approach would be to come up with a bitcount that you think would fit considering even (currently) fringe/odd features - and we'd make sure there's enough space for that in the ABI - should there be a need/desire to expose that in the future.
Ingo
Looking at the Instruction Matching CAM on Power6, it's comprised of two 64-bit values, but there are quite a few reserved bits, and bits that must be programmed in a fixed way. If we were to squeeze out the reserved and fixed bits from the ABI, that leaves 74 real bits of data that a user would like to be able to set.
In addition to that, there is an instruction marking mechanism that requires 2 bits to set the sampling mode.
Lastly, there is a thresholding mechanism that has 6 bits of count, two 3-bit start/end event fields, and a 2-bit granularity field.
In total, that's 90 bits in addition to the event code (9 bits?). There may be a few stragglers that I have missed, and some room should be left for future processors. 128 could be a bit tight for future processor generations.
While reading the Power6 PMU manual, I also had a look at Power5+ PMU manual, and it has five more accessible instruction matching registers (32-bits each). These five are somewhat more special-purpose (they match fewer bits in the instruction), and they probably could be left out, but it would be nice if the ABI had the room for them.
Regards,
- Corey
Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]
Quoted 26 lines · Ingo Molnar
We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:
http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patchIt's also available in tip/master at:
http://people.redhat.com/mingo/tip.git/README
There are many changes in the v6 release:
- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.
- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc instrumentation and PAPI support.
- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.
- The 'perfstat' utility (ex 'timec') has been updated:
http://people.redhat.com/mingo/perfcounters/perfstat.c
- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:
Quoted 7 lines
http://people.redhat.com/mingo/perfcounters/kerneltop.c
- Merged to latest mainline
- Various fixes and other updates
Ingo
I'm not sure if this is the right place to propose such a thing, but I think it would be very valuable to have a standardized user-side library to accompany this addition to the kernel.
In particular, as a starting place for the discussion, I'd like to see functions in it that are very similar to a subset of what is currently in libpfm. Specifically, I'd like to see the following functions (with the names changed to pcl_* perhaps):
extern pfm_err_t pfm_find_event(const char *str, unsigned int *idx); extern pfm_err_t pfm_find_event_bycode(int code, unsigned int *idx); extern pfm_err_t pfm_find_event_bycode_next(int code, unsigned int start,
unsigned int *next);
extern pfm_err_t pfm_find_event_mask(unsigned int event_idx, const char *str,
unsigned int *mask_idx);extern pfm_err_t pfm_find_full_event(const char *str, pfmlib_event_t *e);
extern pfm_err_t pfm_get_max_event_name_len(size_t *len);
extern pfm_err_t pfm_get_num_events(unsigned int *count); extern pfm_err_t pfm_get_num_event_masks(unsigned int event_idx,
unsigned int *count);
extern pfm_err_t pfm_get_event_name(unsigned int idx, char *name,
size_t maxlen);
extern pfm_err_t pfm_get_full_event_name(pfmlib_event_t *e, char *name,
size_t maxlen);extern pfm_err_t pfm_get_event_code(unsigned int idx, int *code); extern pfm_err_t pfm_get_event_mask_code(unsigned int idx,
unsigned int mask_idx, unsigned int *code);
extern pfm_err_t pfm_get_event_description(unsigned int idx, char **str); extern pfm_err_t pfm_get_event_code_counter(unsigned int idx, unsigned int cnt,
int *code);
extern pfm_err_t pfm_get_event_mask_name(unsigned int event_idx,
unsigned int mask_idx, char *name, size_t maxlen);
extern pfm_err_t pfm_get_event_mask_description(unsigned int event_idx,
unsigned int mask_idx, char **desc);
Now, since it's not clear right now how unit masks are going to be handled in your proposal, I'm not sure the that *_event_mask_* functions are applicable, but I think something that fills that function will be needed.
Architectures that have need for additional functionality should be free to add arch-specific functions.
Full descriptions of these functions can be found in the man pages of the libpfm documentation.
Any thoughts on this? Do you already have a user library structure in mind?
Signature · 10 lines
Regards, - Corey Corey Ashford Software Engineer IBM Linux Technology Center, Linux Toolchain Beaverton, OR 503-578-3507 [email protected]
Corey,
Quoted 86 lines · Corey Ashford
Quoted 29 lines · Ingo Molnar
We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:
http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patch
It's also available in tip/master at:
http://people.redhat.com/mingo/tip.git/README
There are many changes in the v6 release:
- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.
- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc
instrumentation and PAPI support.- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.
- The 'perfstat' utility (ex 'timec') has been updated:
Quoted 8 lines
- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:
http://people.redhat.com/mingo/perfcounters/kerneltop.c- Merged to latest mainline
- Various fixes and other updates
Ingo
I'm not sure if this is the right place to propose such a thing, but I think it would be very valuable to have a standardized user-side library to accompany this addition to the kernel.
In particular, as a starting place for the discussion, I'd like to see functions in it that are very similar to a subset of what is currently in libpfm. Specifically, I'd like to see the following functions (with the names changed to pcl_* perhaps):
extern
pfm_err_tpfm_find_event(const char *str, unsigned int *idx); externpfm_err_tpfm_find_event_bycode(int code, unsigned int *idx); externpfm_err_tpfm_find_event_bycode_next(int code, unsigned int start,unsigned int *next); extern pfm_err_t pfm_find_event_mask(unsigned int event_idx, const char *str, unsigned int *mask_idx);extern
pfm_err_tpfm_find_full_event(const char *str, pfmlib_event_t *e);extern
pfm_err_tpfm_get_max_event_name_len(size_t *len);extern
pfm_err_tpfm_get_num_events(unsigned int *count); externpfm_err_tpfm_get_num_event_masks(unsigned intevent_idx,unsigned int *count); extern pfm_err_t pfm_get_event_name(unsigned int idx, char *name, size_t maxlen); extern pfm_err_t pfm_get_full_event_name(pfmlib_event_t *e, char *name, size_t maxlen);extern
pfm_err_tpfm_get_event_code(unsigned int idx, int *code); externpfm_err_tpfm_get_event_mask_code(unsigned int idx,
unsigned intmask_idx, unsigned int *code);
externpfm_err_tpfm_get_event_description(unsigned int idx, char **str); externpfm_err_tpfm_get_event_code_counter(unsigned int idx, unsigned int cnt,int *code);extern
pfm_err_tpfm_get_event_mask_name(unsigned intevent_idx,
unsigned intmask_idx, char *name,size_tmaxlen);
externpfm_err_tpfm_get_event_mask_description(unsigned intevent_idx,
unsigned intmask_idx, char **desc);Now, since it's not clear right now how unit masks are going to be handled in your proposal, I'm not sure the that *_event_mask_* functions are applicable, but I think something that fills that function will be needed.
Quoted 7 lines
Architectures that have need for additional functionality should be free to add arch-specific functions.
Full descriptions of these functions can be found in the man pages of the libpfm documentation.
Any thoughts on this? Do you already have a us
er library structure in mind?
Quoted 1 line
Yes, I did give some thoughts to all of this. In fact, I have been playing a bit with
libpfm and the LPC proposal.
I think, given that LPC is dealing with event -> counter assignment in the kernel, libpfm
does not have to do it. All it needs to do is event:attributes -> value, and that value is
then passed to the kernel in raw mode.
Event attributes includes on x86, for instance, the edge, invert, counter-mask, plm, field.
I think we could do something more generic than what is currently there. That would not
require PMU specific data structures for attributes. Just pass everything into a string.
To that extent, I have been experimenting with something along those lines:
int pfm_get_event_encoding(char *event_str, uint64_t **values, int *count);
events are encoded as follows:
event_name:[unit_mask1:unit_mask2:...:unit_maskn][::A1=V1:A2=V2:..:An=Vn]
Attribute names and values depend on each PMU model. Attributes names are strings.
Values can have any type.
For X86, most attributes would be identical, same thing on Itanium because they are
architected.
Some PMU models may need more than one 64-bit value to configure one event, That is
is why there is vector and a count. Libpfm should not be concerned with how those values
are encoded and passed to the kernel. It should be concerned with the event -> value
as described in the PMU documentation.
Given that LPC manages events independently of each other, libpfm does not reallly need
to process multiple events at a time to get a global view of what is being measured.
Here is an example:
$ self inst_retired:any_p::i=1:c=1:u=1:k=1 [0x1d300c0 event_sel=0xc0 umask=0x0 os=1 usr=1 en=1 int=1 inv=1 edge=0 cnt_mask=1] INST_RETIRED
Quoted 142 lines · stephane eranian
Corey,
Quoted 84 lines · Corey Ashford
Quoted 28 lines · Ingo Molnar
We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:
http://people.redhat.com/mingo/perfcounters/perfcounters-v6-v2.6.29-rc2.patch
It's also available in tip/master at:
http://people.redhat.com/mingo/tip.git/README
There are many changes in the v6 release:
- PowerPC performance counters support from Paul Mackerras, for POWER6 and for the PPC970 family.
- ioctl API to disable/enable individual counters and groups without closing their fd. This can be useful for libraries, ad-hoc
instrumentation and PAPI support.- 'pinned' and 'exclusive' counter attributes - for those applications that want to influence counter scheduling explicitly.
- The 'perfstat' utility (ex 'timec') has been updated:
Quoted 8 lines
- 'kerneltop' (easy-to-use text mode NMI profiler) has been updated:
http://people.redhat.com/mingo/perfcounters/kerneltop.c- Merged to latest mainline
- Various fixes and other updates
Ingo
I'm not sure if this is the right place to propose such a thing, but I think it would be very valuable to have a standardized user-side library to accompany this addition to the kernel.
In particular, as a starting place for the discussion, I'd like to see functions in it that are very similar to a subset of what is currently in libpfm. Specifically, I'd like to see the following functions (with the names changed to pcl_* perhaps):
extern
pfm_err_tpfm_find_event(const char *str, unsigned int *idx); externpfm_err_tpfm_find_event_bycode(int code, unsigned int *idx); externpfm_err_tpfm_find_event_bycode_next(int code, unsigned int start,unsigned int *next); extern pfm_err_t pfm_find_event_mask(unsigned int event_idx, const char *str, unsigned int *mask_idx);extern
pfm_err_tpfm_find_full_event(const char *str, pfmlib_event_t *e);extern
pfm_err_tpfm_get_max_event_name_len(size_t *len);extern
pfm_err_tpfm_get_num_events(unsigned int *count); externpfm_err_tpfm_get_num_event_masks(unsigned intevent_idx,unsigned int *count); extern pfm_err_t pfm_get_event_name(unsigned int idx, char *name, size_t maxlen); extern pfm_err_t pfm_get_full_event_name(pfmlib_event_t *e, char *name, size_t maxlen);extern
pfm_err_tpfm_get_event_code(unsigned int idx, int *code); externpfm_err_tpfm_get_event_mask_code(unsigned int idx,
unsigned intmask_idx, unsigned int *code);
externpfm_err_tpfm_get_event_description(unsigned int idx, char **str); externpfm_err_tpfm_get_event_code_counter(unsigned int idx, unsigned int cnt,int *code);extern
pfm_err_tpfm_get_event_mask_name(unsigned intevent_idx,
unsigned intmask_idx, char *name,size_tmaxlen);
externpfm_err_tpfm_get_event_mask_description(unsigned intevent_idx,
unsigned intmask_idx, char **desc);Now, since it's not clear right now how unit masks are going to be handled in your proposal, I'm not sure the that *_event_mask_* functions are applicable, but I think something that fills that function will be needed.
Quoted 7 lines
Architectures that have need for additional functionality should be free to add arch-specific functions.
Full descriptions of these functions can be found in the man pages of the libpfm documentation.
Any thoughts on this? Do you already have a us
er library structure in mind?
Yes, I did give some thoughts to all of this. In fact, I have been playing a bit with
libpfm and the LPC proposal.I think, given that LPC is dealing with event -> counter assignment in the kernel, libpfm
does not have to do it. All it needs to do is event:attributes -> value, and that value is
then passed to the kernel in raw mode.Event attributes includes on x86, for instance, the edge, invert, counter-mask, plm, field.
I think we could do something more generic than what is currently there. That would not
require PMU specific data structures for attributes. Just pass everything into a string.To that extent, I have been experimenting with something along those lines:
int pfm_get_event_encoding(char *event_str, uint64_t **values, int *count);events are encoded as follows:
event_name:[unit_mask1:unit_mask2:...:unit_maskn][::A1=V1:A2=V2:..:An=Vn]Attribute names and values depend on each PMU model. Attributes names are strings.
Values can have any type.For X86, most attributes would be identical, same thing on Itanium because they are
architected.Some PMU models may need more than one 64-bit value to configure one event, That is
is why there is vector and a count. Libpfm should not be concerned with how those values
are encoded and passed to the kernel. It should be concerned with the event -> value
as described in the PMU documentation.Given that LPC manages events independently of each other, libpfm does not reallly need
to process multiple events at a time to get a global view of what is being measured.
Quoted 5 lines
Here is an example:
$ self inst_retired:any_p::i=1:c=1:u=1:k=1 [0x1d300c0 event_sel=0xc0 umask=0x0 os=1 usr=1 en=1 int=1 inv=1 edge=0 cnt_mask=1] INST_RETIRED
This looks encouraging!
I assume the library would still retain the functions that allow us to iterate through the available events, and obtain text description of events. Would it make sense to have similar functions to obtain the available unit masks and attributes for a particular event?
For debugging purposes at least, it might make sense to have a function that does the inverse of pfm_get_event_encoding as well.
Signature · 10 lines
Regards, - Corey Corey Ashford Software Engineer IBM Linux Technology Center, Linux Toolchain Beaverton, OR 503-578-3507 [email protected]
Quoted 117 lines · Corey Ashford
Quoted 111 lines
Quoted 52 lines
I'm not sure if this is the right place to propose such a thing, but I think
it would be very valuable to have a standardized user-side library to accompany this addition to the kernel.In particular, as a starting place for the discussion, I'd like to see functions in it that are very similar to a subset of what is currently in libpfm. Specifically, I'd like to see the following functions (with the names changed to pcl_* perhaps):
extern
pfm_err_tpfm_find_event(const char *str, unsigned int *idx); externpfm_err_tpfm_find_event_bycode(int code, unsigned int *idx); externpfm_err_tpfm_find_event_bycode_next(int code, unsigned int start,unsigned int *next); extern pfm_err_t pfm_find_event_mask(unsigned int event_idx, const char *str, unsigned int *mask_idx);extern
pfm_err_tpfm_find_full_event(const char *str, pfmlib_event_t *e);extern
pfm_err_tpfm_get_max_event_name_len(size_t *len);extern
pfm_err_tpfm_get_num_events(unsigned int *count); externpfm_err_tpfm_get_num_event_masks(unsigned intevent_idx,unsigned int *count); extern pfm_err_t pfm_get_event_name(unsigned int idx, char *name, size_t maxlen); extern pfm_err_t pfm_get_full_event_name(pfmlib_event_t *e, char *name, size_t maxlen);extern
pfm_err_tpfm_get_event_code(unsigned int idx, int *code); externpfm_err_tpfm_get_event_mask_code(unsigned int idx,
unsigned intmask_idx, unsigned int *code);
externpfm_err_tpfm_get_event_description(unsigned int idx, char **str); externpfm_err_tpfm_get_event_code_counter(unsigned int idx, unsigned int
cnt,int *code);extern
pfm_err_tpfm_get_event_mask_name(unsigned intevent_idx,
unsigned intmask_idx, char *name,size_tmaxlen);
externpfm_err_tpfm_get_event_mask_description(unsigned intevent_idx,
unsigned intmask_idx, char **desc);Now, since it's not clear right now how unit masks are going to be handled
in your proposal, I'm not sure the that *_event_mask_* functions are applicable, but I think something that fills that function will be needed.Quoted 8 lines
Architectures that have need for additional functionality should be free to
add arch-specific functions.Full descriptions of these functions can be found in the man pages of the libpfm documentation.
Any thoughts on this? Do you already have a us
er library structure in mind?
Yes, I did give some thoughts to all of this. In fact, I have been playing a bit with
libpfm and the LPC proposal.I think, given that LPC is dealing with event -> counter assignment in the kernel, libpfm
does not have to do it. All it needs to do is event:attributes -> value, and that value is
then passed to the kernel in raw mode.Event attributes includes on x86, for instance, the edge, invert, counter-mask, plm, field.
I think we could do something more generic than what is currently there. That would not
require PMU specific data structures for attributes. Just pass everything into a string.To that extent, I have been experimenting with something along those lines:
int pfm_get_event_encoding(char
*event_str,uint64_t**values, int
*count);events are encoded as follows:
event_name:[unit_mask1:unit_mask2:...:unit_maskn][::A1=V1:A2=V2:..:An=Vn]
Attribute names and values depend on each PMU model. Attributes names are strings.
Values can have any type.For X86, most attributes would be identical, same thing on Itanium because they are
architected.Some PMU models may need more than one 64-bit value to configure one event, That is
is why there is vector and a count. Libpfm should not be concerned with how those values
are encoded and passed to the kernel. It should be concerned with the event -> value
as described in the PMU documentation.Given that LPC manages events independently of each other, libpfm does not reallly need
to process multiple events at a time to get a global view of what is being measured.Quoted 5 lines
Here is an example:
$ self inst_retired:any_p::i=1:c=1:u=1:k=1 [0x1d300c0 event_sel=0xc0 umask=0x0 os=1 usr=1 en=1 int=1 inv=1 edge=0 cnt_mask=1] INST_RETIRED
Quoted 7 lines
This looks encouraging!
I assume the library would still retain the functions that allow us to iterate through the available events, and obtain text description of events.
Would it make sense to have similar functions to obtain the available unit
masks and attributes for a particular event?
Yes, that would most likely stay there, although, I think we could simplify a bit.
Quoted 3 lines
For debugging purposes at least, it might make sense to have a function that does the inverse of
pfm_get_event_encodingas well.
Yes, we could provide the opposite function.
I also believe this same scheme could be used to describe non-event features, such as IBS, LBR, Opcode matcher.
Quoted 2 lines · Ingo Molnar
We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:
[snip]
Hi Ingo,
As I was starting to put together a simple implementation of PAPI on top of PCL for Power, I noticed that PCL does not seem to have any sort of versioning and way of ascertaining the current capabilities of what is in the kernel.
This information is needed by tools and libraries built on top of PCL so that they can know what is supported and if any bugs need to be worked around.
Have you thought about how to present this information to the user? I was thinking that since you already have /sys/devices/system/cpu/perf_counter set up, maybe we could add some more files in there. Perhaps a "version" file (perfmon does this), and maybe a "capabilities" file which contains a set of strings exhibiting the supported features. For example,
arch-independent:MMAPPED_SAMPLE_BUFFER
MMAPPED_SAMPLE_BUFFER_MAX_SIZE=0x20000CUSTOM_SAMPLING
on powerpc arch:POWER6_IMCPOWER6_THRESHOLD
or on x86:NEHALEM_PEBS
etc.
Or it could be a bit mask whose bits are defined in an include file.
Personally, I like the strings approach because it's more extensible, and would not require modifying /usr/include files when you pick up a new kernel. Perhaps the arch-independent and arch-dependent capabilities would be in two different files to make the implementation easier.
Signature · 10 lines
Regards, - Corey Corey Ashford Software Engineer IBM Linux Technology Center, Linux Toolchain Beaverton, OR 503-578-3507 [email protected]
Quoted 10 lines · Corey Ashford
Quoted 5 lines · Ingo Molnar
We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:
[snip]
Hi Ingo,
Quoted 8 lines
As I was starting to put together a simple implementation of PAPI on top of PCL for Power, I noticed that PCL does not seem to have any sort of versioning and way of ascertaining the current capabilities of what is in the kernel.
This information is needed by tools and libraries built on top of PCL so that they can know what is supported and if any bugs need to be worked around.
I'd prefer to use the standard Linux syscall ABI convention here:
- once upstream, existing functionality is compatible forever
- new functionality is added in a way that it generates a -ENOSYS return from the syscall in an older kernel.
That's why the event structure is sized relatively large for example - to make sure we have space to grow into.
So instead of adding versioning information, it would be very nice if you could check the ABI details for 'traps' that make extensions harder. Try to come up with pie-in-the-sky future items you'd like to see in the ABI, and lets see how supportable it would be.
Example #1 - made up. Say if we had an ABI detail like this:
struct perf_counter_hw_event {
u8 type;this would limit us to 256 events - which would be clearly stupid as we can easily hit that limit.
Example #2. Not made up:
asmlinkage int
sys_perf_counter_open(struct perf_counter_hw_event *hw_event_uptr __user,
pid_t pid, int cpu, int group_fd)
Those are 5 parameters - we could extend it to 6 and add a 'flags' value that in the current version will return -ENOSYS if the flags value is not zero.
This would add one more dimension of extensibility to the interface.
If you could come up with a list of small details like this, that would be really helpful. Would this work?
Ingo
Quoted 37 lines · Ingo Molnar
Quoted 10 lines · Corey Ashford
Quoted 2 lines · Ingo Molnar
We are pleased to announce version 6 of our performance counters subsystem implementation. The shortlog, diffstat and the combo patch can be found below. The combo patch against latest -git (2.6.29-rc2) can be also found at:
[snip]
Hi Ingo,
As I was starting to put together a simple implementation of PAPI on top of PCL for Power, I noticed that PCL does not seem to have any sort of versioning and way of ascertaining the current capabilities of what is in the kernel.
This information is needed by tools and libraries built on top of PCL so that they can know what is supported and if any bugs need to be worked around.
I'd prefer to use the standard Linux syscall ABI convention here:
- once upstream, existing functionality is compatible forever
- new functionality is added in a way that it generates a
-ENOSYSreturn from the syscall in an older kernel.That's why the event structure is sized relatively large for example - to make sure we have space to grow into.
So instead of adding versioning information, it would be very nice if you could check the ABI details for 'traps' that make extensions harder. Try to come up with pie-in-the-sky future items you'd like to see in the ABI, and lets see how supportable it would be.
Example #1 - made up. Say if we had an ABI detail like this:
struct perf_counter_hw_event { u8 type;this would limit us to 256 events - which would be clearly stupid as we can easily hit that limit.
Example #2. Not made up:
asmlinkage int
sys_perf_counter_open(structperf_counter_hw_event*hw_event_uptr__user,
pid_tpid, int cpu, intgroup_fd)
Quoted 7 lines
Those are 5 parameters - we could extend it to 6 and add a 'flags' value that in the current version will return
-ENOSYSif the flags value is not zero.This would add one more dimension of extensibility to the interface.
If you could come up with a list of small details like this, that would be really helpful. Would this work?
Ingo
Thanks for the reply. As I flesh out this PAPI code, I will keep thinking about these issues.
I think the method you describe is good for adding new event types and accessing fancy PMU hardware (instruction matching CAM's for example).
There may be other non-event-related changes that will not be handled quite as well in this way. In the original email I hinted at that we may want an option for mmap'd sample buffers at some point, and so I'm not clear how you'd provide an ABI to request mmap'd buffers (you would probably need to be able to request the size and get back a pointer to the mmap'd buffer). Would this be done through a special sys_perf_counter_open call? Or through a subsequent ioctl call on the group leader after an open (which requires the counters to be initially disabled), etc.
For bugs in the kernel that need to be worked around, I assume you would suggest to the tool programmers that they somehow test for the bug's presence? What if the bug causes a system crash? Perhaps a better solution for that case would be to check the kernel's version number rather than create a separate PCL version?
Thanks for your consideration,
- Corey
Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]
Quoted 9 lines · Corey Ashford
There may be other non-event-related changes that will not be handled quite as well in this way. In the original email I hinted at that we may want an option for mmap'd sample buffers at some point, and so I'm not clear how you'd provide an ABI to request mmap'd buffers (you would probably need to be able to request the size and get back a pointer to the mmap'd buffer). Would this be done through a special
sys_perf_counter_opencall? Or through a subsequent ioctl call on the group leader after an open (which requires the counters to be initially disabled), etc.
What's stopping a regular mmap() call using the fd obtained from sys_perf_counter_open() ?
Quoted 5 lines · Peter Zijlstra
Quoted 1 line · Corey Ashford
There may be other non-event-related changes that will not be handled quite as well in this way. In the original email I hinted at that we may want an option for mmap'd sample buffers at some point, and so I'm not clear how you'd provide an ABI to request mmap'd buffers (you would probably need to be able to request the size and get back a pointer to the mmap'd buffer). Would this be done through a special
sys_perf_counter_opencall? Or through a subsequent ioctl call on the group leader after an open (which requires the counters to be initially disabled), etc.What's stopping a regular
mmap()call using the fd obtained fromsys_perf_counter_open()?
Perhaps nothing. I don't know why it didn't occur to me that this could be done. If PCL supports this, great!
Thanks for your reply!
Signature · 10 lines
Regards, - Corey Corey Ashford Software Engineer IBM Linux Technology Center, Linux Toolchain Beaverton, OR 503-578-3507 [email protected]
Quoted 15 lines · Corey Ashford
Quoted 12 lines · Peter Zijlstra
Quoted 9 lines · Corey Ashford
There may be other non-event-related changes that will not be handled quite as well in this way. In the original email I hinted at that we may want an option for mmap'd sample buffers at some point, and so I'm not clear how you'd provide an ABI to request mmap'd buffers (you would probably need to be able to request the size and get back a pointer to the mmap'd buffer).
Would this be done through a specialsys_perf_counter_opencall? Or
through a subsequent ioctl call on the group leader after an open (which requires the counters to be initially disabled), etc.
Quoted 6 lines
Quoted 2 lines
What's stopping a regular
mmap()call using the fd obtained fromsys_perf_counter_open()?Perhaps nothing. I don't know why it didn't occur to me that this could be done. If PCL supports this, great!
That has always been the way, I thought this could be done with PCL. Except that I don't know who decides the size of the mmap().
Quoted 5 lines · Peter Zijlstra
Quoted 1 line · Corey Ashford
There may be other non-event-related changes that will not be handled quite as well in this way. In the original email I hinted at that we may want an option for mmap'd sample buffers at some point, and so I'm not clear how you'd provide an ABI to request mmap'd buffers (you would probably need to be able to request the size and get back a pointer to the mmap'd buffer). Would this be done through a special
sys_perf_counter_opencall? Or through a subsequent ioctl call on the group leader after an open (which requires the counters to be initially disabled), etc.What's stopping a regular
mmap()call using the fd obtained fromsys_perf_counter_open()?
Thinking about this some more, I'm not sure it is the right model to be able to read a stream of records up from kernel space.
Let's say I want to read 1MB of records up at once, and then a bit later read another 1MB, and so on, all while the program is still running and counters are counting. Is there a way to use mmap this way? I think I would have to call mmap for each 1MB segment (if my buffer is 1MB). Also how do I indicate to the kernel that I've "consumed" each 1MB chunk and that the kernel buffers for it can be reused? It's easy to understand how to do that with just a read() call, but with mmap'd memory, I don't know.
Another thing that concerns me is this statement in the mmap() man page: "If the size of the mapped file changes after the call to mmap() as a result of some other operation on the mapped file, the effect of references to portions of the mapped region that correspond to added or removed portions of the file is unspecified." Would PCL guarantee the correct contents of memory, despite this caveat?
If you wait till after you've stopped the counters to read up the records, I can see where mmap would work fine. But if you are trying to unload the data from the kernel's space efficiently, over the long execution time of the thread or process being analyzed, I don't see how mmap can be used in this way on PCL's file descriptors.
In perfmon, the model is different, because the mmap'd memory for the buffer is not streamed; it's just a block of memory full of records. Once one buffer fills up, user space is alerted and the kernel inhibits adding new entries (in single-buffering mode), while user space accesses the filled buffer. So the mmap call has to be done only once.
Double-buffering (or more) can be achieved by splitting the buffer
into multiple pieces and filling them up separately and alerting user space when each are full.
Quite possibly I'm missing something obvious, but PCL may need an mmap behavior better suited for efficiently reading data up dynamically over the long run of the program being analyzed.
Regards,
- Corey
Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain
Beaverton, OR
503-578-3507
[email protected]
Arnd Bergmann
maintainerarch/powerpc/include/asm/systbl.h SYSCALL_SPU(epoll_create1) · line 325
Quoted 9 lines · Ingo Molnar
arch/powerpc/include/asm/systbl.h +1
-0@@ -322,3 +322,4 @@ SYSCALL_SPU(epoll_create1) SYSCALL_SPU(dup3) SYSCALL_SPU(pipe2) SYSCALL(inotify_init1) +SYSCALL(perf_counter_open)
Is there a reason to forbid this on the SPU? It's probably not particularly useful, but it shouldn't hurt.
arch/powerpc/include/asm/systbl.h
Quoted 10 lines
arch/powerpc/include/asm/systbl.h +10
-0+ +#ifdef CONFIG_PERF_COUNTERS +# include <asm/perf_counter.h> +#endif + +#include <linux/list.h> +#include <linux/mutex.h> +#include <linux/rculist.h> +#include <linux/rcupdate.h> +#include <linux/spinlock.h>
I guess all */perf_counter.h files should be exported, but then the #ifdef won't work when the file is included from user space. Also, some of the included headers are not exported.
Quoted 10 lines
arch/powerpc/include/asm/systbl.h +10
-0+/* +* Hardware event to monitor via a performance monitoring counter: +*/ +struct perf_counter_hw_event { +s64 type; + +u64 irq_period; +u32 record_type; + + u32 disabled : 1, /* off by default */
arch/powerpc/include/asm/systbl.h
Quoted 10 lines
arch/powerpc/include/asm/systbl.h +10
-0+ nmi : 1, /* NMI sampling */ + raw : 1, /* raw event type */ + inherit : 1, /* children inherit it */ + pinned : 1, /* must always be on PMU */ + exclusive : 1, /* only counter on PMU */ + + __reserved_1 : 26; + +u64 __reserved_2; +};
When exported, the types should be __s64 etc instead of s64.
arch/powerpc/include/asm/systbl.h
Quoted 9 lines
arch/powerpc/include/asm/systbl.h +9
-0+/* +* Ioctls that can be done on a perf counter fd: +*/ +#define PERF_COUNTER_IOC_ENABLE _IO('$', 0) +#define PERF_COUNTER_IOC_DISABLE _IO('$', 1) + +/* +* Kernel-internal data types: +*/
The kernel internal parts of the header should be hidden in #ifdef __KERNEL__.
arch/powerpc/include/asm/systbl.h
Quoted 7 lines
arch/powerpc/include/asm/systbl.h +7
-0+ +asmlinkage int sys_perf_counter_open( + +struct perf_counter_hw_event *hw_event_uptr__user , +pid_t pid, +int cpu, +int group_fd);
For the syscall, I'd suggest sys_perf_counter_fd or sys_perfcounterfd to go along with signalfd, eventfd, etc. The only other _open syscall we have is sys_mq_open(), which is different since it actually performs an open() on a (hidden) file.
All system calls should return a 'long' value to make sure that the errno handling works on all architectures.
arch/powerpc/include/asm/systbl.h
Quoted 7 lines
arch/powerpc/include/asm/systbl.h +7
-0+static const struct file_operations perf_fops = { + .release = perf_release, + .read = perf_read, + .poll = perf_poll, + .unlocked_ioctl = perf_ioctl, + .compat_ioctl = perf_ioctl, +};
It feels inconsistent to combine a syscall for getting the fd with ioctl. Assuming the interface is semantically right, I'd suggest using either a chardev with more ioctls or more syscalls but not both:
asmlinkage long sys_perfcounter_fd(
struct perf_counter_hw_event *hw_event_uptr __user, pid_t pid, int cpu, int group_fd);
asmlinkage long sys_perfcounter_setflags(int fd, u32 flags);
or
struct perf_counter_setup {
struct perf_counter_hw_event event;
__u64 pid;
__u32 cpu;
__u32 group_fd;
};#define PERF_COUNTER_IOC_SETUP _IOW('$', 1, struct perf_counter_setup) #define PERF_COUNTER_IOC_SETFLAGS _IOW('$', 2, __u32)
Arnd <><
Quoted 14 lines · Arnd Bergmann
arch/powerpc/include/asm/systbl.h SYSCALL_SPU(epoll_create1) · line 325
Quoted 9 lines · Ingo Molnar
arch/powerpc/include/asm/systbl.h +1
-0@@ -322,3 +322,4 @@ SYSCALL_SPU(epoll_create1) SYSCALL_SPU(dup3) SYSCALL_SPU(pipe2) SYSCALL(inotify_init1) +SYSCALL(perf_counter_open)Is there a reason to forbid this on the SPU? It's probably not particularly useful, but it shouldn't hurt.
No, it's just an oversight, plus the fact that this stuff doesn't support the hardware counters on Cell, which I've heard is rather unusual. Don't you have to hcalls to access it on machines with a hypervisor? Also, is there any facility for counting events on the SPUs?
Quoted 5 lines
Quoted 5 lines
arch/powerpc/include/asm/systbl.h +5
-0+#ifdef CONFIG_PERF_COUNTERS +# include <asm/perf_counter.h> +#endif + +#include <linux/list.h>
Quoted 8 lines
arch/powerpc/include/asm/systbl.h
Quoted 4 lines
arch/powerpc/include/asm/systbl.h +4
-0+#include <linux/mutex.h> +#include <linux/rculist.h> +#include <linux/rcupdate.h> +#include <linux/spinlock.h> I guess all
*/perf_counter.hfiles should be exported, but then the #ifdef won't work when the file is included from user space. Also, some of the included headers are not exported.
You're right, we do need to clean this up a bit. So far, userspace programs have tended to use their own copies of the relevant definitions.
I think all that userspace needs is the hw_event_types enum, the perf_counter_record_type enum, the perf_counter_hw_event struct, and the ioctl definitions. I don't see why userspace would need the asm bits.
Quoted 14 lines
Quoted 14 lines
arch/powerpc/include/asm/systbl.h +14
-0+/* +* Hardware event to monitor via a performance monitoring counter: +*/ +struct perf_counter_hw_event { +s64 type; + +u64 irq_period; +u32 record_type; + +u32 disabled :1 ,/* off by default */ + nmi :1 ,/* NMI sampling */ + raw :1 ,/* raw event type */ + inherit :1 ,/* children inherit it */ + pinned : 1, /* must always be on PMU */
Quoted 8 lines
arch/powerpc/include/asm/systbl.h
Quoted 6 lines
arch/powerpc/include/asm/systbl.h +6
-0+ exclusive :1 ,/* only counter on PMU */ + + __reserved_1 : 26; + +u64 __reserved_2; +};When exported, the types should be
__s64etc instead of s64.
Yep.
Quoted 4 lines
Quoted 4 lines
arch/powerpc/include/asm/systbl.h +4
-0+/* +* Ioctls that can be done on a perf counter fd: +*/ +#define PERF_COUNTER_IOC_ENABLE _IO('$', 0)
Quoted 8 lines
arch/powerpc/include/asm/systbl.h
Quoted 5 lines
arch/powerpc/include/asm/systbl.h +5
-0+#define PERF_COUNTER_IOC_DISABLE _IO('$', 1) + +/* +* Kernel-internal data types: +*/ The kernel internal parts of the header should be hidden in #ifdef
__KERNEL__.
Agreed.
Quoted 4 lines
Quoted 4 lines
arch/powerpc/include/asm/systbl.h +4
-0+ +asmlinkage int sys_perf_counter_open( + + struct perf_counter_hw_event *hw_event_uptr __user,
Quoted 8 lines
arch/powerpc/include/asm/systbl.h
Quoted 3 lines
arch/powerpc/include/asm/systbl.h +3
-0+pid_t pid, + int cpu, + int group_fd);For the syscall, I'd suggest
sys_perf_counter_fdorsys_perfcounterfdto go along with signalfd, eventfd, etc. The only other_opensyscall we have issys_mq_open(), which is different since it actually performs anopen()on a (hidden) file.
I don't have a strong opinion on that either way; maybe Ingo does.
Quoted 2 lines
All system calls should return a 'long' value to make sure that the errno handling works on all architectures.
True.
Quoted 9 lines
arch/powerpc/include/asm/systbl.h
Quoted 7 lines
arch/powerpc/include/asm/systbl.h +7
-0+static const struct file_operations perf_fops = { + .release = perf_release, + .read = perf_read, + .poll = perf_poll, + .unlocked_ioctl = perf_ioctl, + .compat_ioctl = perf_ioctl, +};It feels inconsistent to combine a syscall for getting the fd with ioctl.
Why? Once you have an fd, what's inconsistent about doing an ioctl on it?
Quoted 12 lines
Assuming the interface is semantically right, I'd suggest using either a chardev with more ioctls or more syscalls but not both:
asmlinkage long sys_perfcounter_fd(
struct perf_counter_hw_event *hw_event_uptr __user, pid_t pid, int cpu, int group_fd);asmlinkage long
sys_perfcounter_setflags(int fd, u32 flags);or
Quoted 8 lines
struct perf_counter_setup { struct perf_counter_hw_event event; __u64 pid; __u32 cpu; __u32 group_fd; };#define
PERF_COUNTER_IOC_SETUP_IOW('$', 1, struct perf_counter_setup)#definePERF_COUNTER_IOC_SETFLAGS_IOW('$', 2, __u32)
I don't see any compelling value in either of those suggestions, sorry. Possibly, if it turns out that read() or ioctl() are just too slow (and can't be improved), we might consider adding syscalls.
Paul.
Arnd Bergmann
maintainerQuoted 6 lines · Paul Mackerras
Arnd Bergmann writes:
No, it's just an oversight, plus the fact that this stuff doesn't support the hardware counters on Cell, which I've heard is rather unusual. Don't you have to hcalls to access it on machines with a hypervisor? Also, is there any facility for counting events on the SPUs?
Yes, there is, and it's suported in both oprofile and perfmon2. It uses a set of hooks in the spu scheduler to know about what code got loaded into it, and the platform interface is a combination of mmio and rtas/hcall accesses.
Quoted 10 lines
Quoted 9 lines
kernel/perf_counter.c line 1326
Quoted 7 lines
kernel/perf_counter.c new file +7
-0+static const struct file_operations perf_fops = { + .release = perf_release, + .read = perf_read, + .poll = perf_poll, + .unlocked_ioctl = perf_ioctl, + .compat_ioctl = perf_ioctl, +};It feels inconsistent to combine a syscall for getting the fd with ioctl.
Quoted 2 lines
Why? Once you have an fd, what's inconsistent about doing an ioctl on it?
I don't think we have any interface doing that today outside of the traditional chardev, blockdev, socket and regular file descriptors. The first inconsistency would be in having another type of object that you can do ioctl on.
Back in the original spufs interface design, I had similar plans for adding a combination of syscall and ioctl interfaces. The point that convinced me to go with just syscall and regular file operations was that syscall and ioctl are basically two interchangeable ways of doing the same thing (adding an arbitrary function call into the kernel). When you look in /proc/misc, half the stuff in there only exists as a cheap way to sneak a syscall into the kernel without the necessary review.
The inconsistency in combining the two would be the same as having sys_timerfd_settime an ioctl on the fd from sys_timerfd_create, or getdents an ioctl on a directory fd.
The most significant impact of allowing ioctl on the fd is certainly that it allows adding many more interfaces on top of the performance counters as soon as the code is initially merged. Whether you count that as a an advantage or a disadvantage depends on your point of view ;-).
Quoted 21 lines
Quoted 12 lines
Assuming the interface is semantically right, I'd suggest using either a chardev with more ioctls or more syscalls but not both:
asmlinkage long sys_perfcounter_fd(
struct perf_counter_hw_event *hw_event_uptr __user, pid_t pid, int cpu, int group_fd);asmlinkage long
sys_perfcounter_setflags(int fd, u32 flags);or
Quoted 8 lines
struct perf_counter_setup { struct perf_counter_hw_event event; __u64 pid; __u32 cpu; __u32 group_fd; };#define
PERF_COUNTER_IOC_SETUP_IOW('$', 1, struct perf_counter_setup)#definePERF_COUNTER_IOC_SETFLAGS_IOW('$', 2, __u32)
Quoted 3 lines
I don't see any compelling value in either of those suggestions, sorry. Possibly, if it turns out that
read()orioctl()are just too slow (and can't be improved), we might consider adding syscalls.
I don't think that performance is an issue here, read() is quite well optimized and ioctl() would only be used in the slowpath.
My second example is a bit constructed, if you wanted to go with a chardev, that would probably imply larger interface changes than that.
One way to improve the interface over PERF_COUNTER_IOC_ENABLE might be to use write() for enable/disable, but if sys_perfcounter_fd (or whatever we call it) stays, using other syscalls for operating on it seems to be the most consistant way of doing it.
Another point that now occurred to me is that in sys_perfcounter_fd(), you should probably put the group descriptor argument first, to be more consistent with the openat() family of syscalls which all take the relative position as their first argument.
Arnd <><
Some points to mention here. This patch set actually introduces two interfaces, a new user/kernel interface and an in-kernel api to access performance counters. These are separate things and sometimes mixed too much. There is a strong need for an in-kernel api. This is the third implementation I am involved (oprofile, perfmon are the others) and the things are always the same way. All these subsystems should be merged to one in-kernel implemenation and share the same code. The different user/kernel i/fs could then coexist and meet the users different needs.
The implementation of the hardware counters is written from scratch again. It is sometimes useful to drop old code, but there is the danger of making errors twice. Implementing performance counters is not trivial, especially buffer handling, SMP and cpu hotplug. For oprofile and perfmon it took years to get stable code. We should benefit from this. (The current x86 code in this patch series seems not to work proper with SMP.) So we should look for a way to better reuse and share code.
See also my comments below.
On 21.01.09 19:50:21, Ingo Molnar wrote:
[...]
Quoted 21 lines
arch/x86/kernel/cpu/perf_counter.c new file +21
-0+static bool perf_counters_initialized__read_mostly ; + +/* +* Number of (generic) HW counters: +*/ +static int nr_counters_generic__read_mostly ; +static u64 perf_counter_mask__read_mostly ; +static u64 counter_value_mask__read_mostly ; + +static int nr_counters_fixed__read_mostly ; + +struct cpu_hw_counters { +struct perf_counter *counters[X86_PMC_IDX_MAX ]; +unsigned long used[BITS_TO_LONGS (X86_PMC_IDX_MAX )]; +}; + +/* +* Intel PerfMon v3. Used on Core2 and later. +*/ +static DEFINE_PER_CPU(struct cpu_hw_counters, cpu_hw_counters); +
arch/x86/kernel/cpu/perf_counter.c line 52
Quoted 10 lines
arch/x86/kernel/cpu/perf_counter.c new file +10
-0+static const int intel_perfmon_event_map[] = +{ + [PERF_COUNT_CPU_CYCLES ] =0x003c , + [PERF_COUNT_INSTRUCTIONS ] =0x00c0 , + [PERF_COUNT_CACHE_REFERENCES ] =0x4f2e , + [PERF_COUNT_CACHE_MISSES ] =0x412e , + [PERF_COUNT_BRANCH_INSTRUCTIONS ] =0x00c4 , + [PERF_COUNT_BRANCH_MISSES ] =0x00c5 , + [PERF_COUNT_BUS_CYCLES ] =0x013c , +};
I would like to define _all_ the behaviour of the architecture and the models in functions instead of parameters and lists. It is hard to explain why, because it is more esthetics, but I believe, only nice things work best. Let me try.
1) The list above seems to be random, there are lots of events and it is hard to define, which event is really important. Surely these events are important, but it is hard to draw a line here.
2) The list assumes/implies the events are available on all architectures and cpus. This is probably not the case, and also, the existence of an event must not be _important_ for a certain architecture. But it has to be there even if it is of no interest.
3) Hard to extend. If an event is added here this could have impact to all other architectures. Data structures are changing.
4) In the kernel the behaviour of a subsystem is offen implemented by functions (e.g. struct device_driver). There are lots of ops structs in the kernel and there are reasons for it.
5) ops structs are more dynamic. The data could be generated dynamically and does not have to be static in some tables and variables.
So, instead of making the list a public data structure, better pass the type to an arch specific function, e.g.:
int arch_xxx_setup_event(int event_type);
If the type is not supported, an error could be returned. There is no more impact. Even the binaries of the builds would be identically if hw_event_types would be extended for a single different architecture.
The same applies also for counters and so on, better implement functions.
-Robert
Signature · 3 lines
Advanced Micro Devices, Inc. Operating System Research Center email: [email protected]
Quoted 4 lines · Robert Richter
Some points to mention here. This patch set actually introduces two interfaces, a new user/kernel interface and an in-kernel api to access performance counters. These are separate things and sometimes mixed too much. There is a strong need for an in-kernel api. This is the
We have been concentrating more on the user/kernel API since that is the one that cannot be changed in an incompatible way once this stuff goes upstream. The in-kernel API can be changed at any time and is still evolving.
Quoted 5 lines
third implementation I am involved (oprofile, perfmon are the others) and the things are always the same way. All these subsystems should be merged to one in-kernel implemenation and share the same code. The different user/kernel i/fs could then coexist and meet the users different needs.
It would certainly be good to get oprofile to use the same low-level machinery as perf_counters. I'm not sure what the fate of perfmon will be, but it seems unlikely it will go upstream in anything like its present form.
Quoted 16 lines
arch/x86/kernel/cpu/perf_counter.c line 52
Quoted 10 lines
arch/x86/kernel/cpu/perf_counter.c new file +10
-0+static const int intel_perfmon_event_map[] = +{ + [PERF_COUNT_CPU_CYCLES ] =0x003c , + [PERF_COUNT_INSTRUCTIONS ] =0x00c0 , + [PERF_COUNT_CACHE_REFERENCES ] =0x4f2e , + [PERF_COUNT_CACHE_MISSES ] =0x412e , + [PERF_COUNT_BRANCH_INSTRUCTIONS ] =0x00c4 , + [PERF_COUNT_BRANCH_MISSES ] =0x00c5 , + [PERF_COUNT_BUS_CYCLES ] =0x013c , +};I would like to define _all_ the behaviour of the architecture and the models in functions instead of parameters and lists. It is hard to explain why, because it is more esthetics, but I believe, only nice things work best. Let me try.
Quoted 3 lines
1) The list above seems to be random, there are lots of events and it is hard to define, which event is really important. Surely these events are important, but it is hard to draw a line here.
I see that list as a convenience for doing a few simple performance measurements. For any serious in-depth analysis userspace will know what processor it's running on and use raw event codes.
Quoted 12 lines
2) The list assumes/implies the events are available on all architectures and cpus. This is probably not the case, and also, the existence of an event must not be _important_ for a certain architecture. But it has to be there even if it is of no interest.
3) Hard to extend. If an event is added here this could have impact to all other architectures. Data structures are changing.
4) In the kernel the behaviour of a subsystem is offen implemented by functions (e.g. struct
device_driver). There are lots of ops structs in the kernel and there are reasons for it.
Quoted 8 lines
5) ops structs are more dynamic. The data could be generated dynamically and does not have to be static in some tables and variables.
So, instead of making the list a public data structure, better pass the type to an arch specific function, e.g.:
int
arch_xxx_setup_event(int event_type);
That's exactly what we have, except that it's called hw_perf_counter_init and the event_type you have there is in the struct perf_counter that gets passed in.
Quoted 6 lines
If the type is not supported, an error could be returned. There is no more impact. Even the binaries of the builds would be identically if
hw_event_typeswould be extended for a single different architecture.The same applies also for counters and so on, better implement functions.
All of that is already done; hw_perf_counter_init gets to interpret the counter->hw_event.type and counter->hw_event.raw fields and decide whether the event is supported, and return an error if not. On x86 it looks like there is a further ops structure (struct pmc_x86_ops) which allows each x86-compatible cpu type to supply its own functions for doing the interpretation of counter->hw_event and other things.
Paul.
Quoted 9 lines · Paul Mackerras
Quoted 4 lines
Some points to mention here. This patch set actually introduces two interfaces, a new user/kernel interface and an in-kernel api to access performance counters. These are separate things and sometimes mixed too much. There is a strong need for an in-kernel api. This is the
We have been concentrating more on the user/kernel API since that is the one that cannot be changed in an incompatible way once this stuff goes upstream. The in-kernel API can be changed at any time and is still evolving.
I agree, it is much more easier to change the in-kernel i/f. I just wanted to emphasize the importance of this i/f. Oprofile, Perfmon and also LPC will exist in the future too and should share the same code base. That's what I missed in the discussion until now.
Quoted 11 lines
Quoted 6 lines
third implementation I am involved (oprofile, perfmon are the others) and the things are always the same way. All these subsystems should be merged to one in-kernel implemenation and share the same code. The different user/kernel i/fs could then coexist and meet the users different needs.
It would certainly be good to get oprofile to use the same low-level machinery as
perf_counters. I'm not sure what the fate of perfmon will be, but it seems unlikely it will go upstream in anything like its present form.
Right, as I sad above, all should share the same low-level code. And, this code already exists. The question is more how to merge it and bring the things together.
[...]
Quoted 8 lines
Quoted 4 lines
So, instead of making the list a public data structure, better pass the type to an arch specific function, e.g.:
int
arch_xxx_setup_event(int event_type);That's exactly what we have, except that it's called
hw_perf_counter_initand theevent_typeyou have there is in the structperf_counterthat gets passed in.
Thanks for pointing this out, I was misinterpreting this as a general hw initialization function, but instead a counter is allocated.
Quoted 8 lines
Quoted 7 lines
If the type is not supported, an error could be returned. There is no more impact. Even the binaries of the builds would be identically if
hw_event_typeswould be extended for a single different architecture.The same applies also for counters and so on, better implement functions.
Quoted 6 lines
All of that is already done;
hw_perf_counter_initgets to interpret thecounter->hw_event.typeandcounter->hw_event.rawfields and decide whether the event is supported, and return an error if not. On x86 it looks like there is a further ops structure (structpmc_x86_ops) which allows each x86-compatible cpu type to supply its own functions for doing the interpretation ofcounter->hw_eventand other things.
Ok, maybe I mixed too much the architectural with the x86 model specific implementation. My impression is that there is data in generic structures what should be better private for the model or architecture. However, I have to figure out the details here.
-Robert
Signature · 3 lines
Advanced Micro Devices, Inc. Operating System Research Center email: [email protected]
Quoted 4 lines · Robert Richter
I agree, it is much more easier to change the in-kernel i/f. I just wanted to emphasize the importance of this i/f. Oprofile, Perfmon and also LPC will exist in the future too and should share the same code base. That's what I missed in the discussion until now.
We could implement oprofile on top of lpc for those archs that have LPC support. And afaik only ia64 needs to bother with perfmon as that's the only arch that has support for it anyway.
Now, even on x86 LPC would need a little more arch support before we can fully replace oprofile, but a half-way model would be a LPC oprofile driver that uses LPC on those machines its supported on, while working to provide LPC support for the older machines.
Quoted 4 lines · Robert Richter
Ok, maybe I mixed too much the architectural with the x86 model specific implementation. My impression is that there is data in generic structures what should be better private for the model or architecture. However, I have to figure out the details here.
The details of the x86 support have changed quite a lot since the v6 patch was posted, I believe. Are you looking at the v6 patch, or at Ingo's tip:perfcounters/core branch?
Ingo - maybe it's time to post a v7 patch?
Paul.
Quoted 7 lines · Paul Mackerras
Quoted 4 lines · Robert Richter
Ok, maybe I mixed too much the architectural with the x86 model specific implementation. My impression is that there is data in generic structures what should be better private for the model or architecture. However, I have to figure out the details here.
Quoted 5 lines
The details of the x86 support have changed quite a lot since the v6 patch was posted, I believe. Are you looking at the v6 patch, or at Ingo's tip:perfcounters/core branch?
Ingo - maybe it's time to post a v7 patch?
Yeah - will try to do that later today.
Ingo
Quoted 10 lines · Paul Mackerras
Quoted 4 lines · Robert Richter
Ok, maybe I mixed too much the architectural with the x86 model specific implementation. My impression is that there is data in generic structures what should be better private for the model or architecture. However, I have to figure out the details here.
The details of the x86 support have changed quite a lot since the v6 patch was posted, I believe. Are you looking at the v6 patch, or at Ingo's tip:perfcounters/core branch?
I am using the tip branch, but took the v6 thread to discuss this.
Quoted 2 lines
Ingo - maybe it's time to post a v7 patch?
For me it's fine to work with the branch if patches are posted to the mailing list for review.
-Robert
Signature · 3 lines
Advanced Micro Devices, Inc. Operating System Research Center email: [email protected]
Quoted 12 lines · Robert Richter
Quoted 10 lines · Paul Mackerras
Quoted 4 lines · Robert Richter
Ok, maybe I mixed too much the architectural with the x86 model specific implementation. My impression is that there is data in generic structures what should be better private for the model or architecture. However, I have to figure out the details here.
The details of the x86 support have changed quite a lot since the v6 patch was posted, I believe. Are you looking at the v6 patch, or at Ingo's tip:perfcounters/core branch?
Quoted 7 lines
I am using the tip branch, but took the v6 thread to discuss this.
Quoted 2 lines
Ingo - maybe it's time to post a v7 patch?
For me it's fine to work with the branch if patches are posted to the mailing list for review.
all perfcounter patches are on lkml and you can see all -tip commits in general by subscribing to the [email protected] mailing list.
Ingo