Skip to contentSkip to conversation

perf_counters: powerpc: Add support for POWER7 processors

PATCH In review Paul Mackerras 3 participants 8 messages Jun 11, 2009 – Jun 12, 2009 PowerPC

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 perf_counters: powerpc: Add support for POWER7 processors

This adds the back-end for the PMU on POWER7 processors. POWER7 has 4 fully-programmable counters and two fixed-function counters (which do respect the freeze conditions, can generate interrupts, and are writable, unlike PMC5/6 on POWER5+/6).

---
 arch/powerpc/kernel/Makefile       |    3 +-
 arch/powerpc/kernel/perf_counter.c |    4 +
 arch/powerpc/kernel/power7-pmu.c   |  316 ++++++++++++++++++++++++++++++++++++
 3 files changed, 322 insertions(+), 1 deletions(-)
 create mode 100644 arch/powerpc/kernel/power7-pmu.c
arch/powerpc/kernel/Makefile +2 -1
index 9ba1bb7..a2c6834 100644
@@ -95,7 +95,8 @@ obj64-$(CONFIG_AUDIT)		+= compat_audit.o
 obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
 obj-$(CONFIG_FUNCTION_GRAPH_TRACER)	+= ftrace.o
 obj-$(CONFIG_PERF_COUNTERS)	+= perf_counter.o power4-pmu.o ppc970-pmu.o \
-				   power5-pmu.o power5+-pmu.o power6-pmu.o
+				   power5-pmu.o power5+-pmu.o power6-pmu.o \
+				   power7-pmu.o
 
 obj-$(CONFIG_8XX_MINIMAL_FPEMU) += softemu8xx.o
 
arch/powerpc/kernel/perf_counter.c +4 -0
index 4990ce2..5d12e68 100644
@@ -1181,6 +1181,7 @@ extern struct power_pmu ppc970_pmu;
 extern struct power_pmu power5_pmu;
 extern struct power_pmu power5p_pmu;
 extern struct power_pmu power6_pmu;
+extern struct power_pmu power7_pmu;
 
 static int init_perf_counters(void)
 {
@@ -1207,6 +1208,9 @@ static int init_perf_counters(void)
 	case 0x3e:
 		ppmu = &power6_pmu;
 		break;
+	case 0x3f:
+		ppmu = &power7_pmu;
+		break;
 	}
 
 	/*
arch/powerpc/kernel/power7-pmu.c new file +316 -0

316 lines not on this page · raw message

Signature · 1 line
1.5.5.rc3.7.gba13
#1 lore raw

PATCH 2/2 perf_counter: powerpc: Implement generalized cache events for POWER processors

This adds tables of event codes for the generalized cache events for all the currently supported powerpc processors: POWER{4,5,5+,6,7} and PPC970*, plus powerpc-specific code to use these tables when a generalized cache event is requested.

---
 arch/powerpc/include/asm/perf_counter.h |    3 ++
 arch/powerpc/kernel/perf_counter.c      |   42 +++++++++++++++++++++++++++-
 arch/powerpc/kernel/power4-pmu.c        |   41 +++++++++++++++++++++++++++
 arch/powerpc/kernel/power5+-pmu.c       |   45 +++++++++++++++++++++++++++++-
 arch/powerpc/kernel/power5-pmu.c        |   41 +++++++++++++++++++++++++++
 arch/powerpc/kernel/power6-pmu.c        |   46 +++++++++++++++++++++++++++++-
 arch/powerpc/kernel/power7-pmu.c        |   41 +++++++++++++++++++++++++++
 arch/powerpc/kernel/ppc970-pmu.c        |   41 +++++++++++++++++++++++++++
 8 files changed, 294 insertions(+), 6 deletions(-)
arch/powerpc/include/asm/perf_counter.h +3 -0
index 1c60f0c..cc7c887 100644
@@ -33,6 +33,9 @@ struct power_pmu {
 	u32	flags;
 	int	n_generic;
 	int	*generic_events;
+	int	(*cache_events)[PERF_COUNT_HW_CACHE_MAX]
+			       [PERF_COUNT_HW_CACHE_OP_MAX]
+			       [PERF_COUNT_HW_CACHE_RESULT_MAX];
 };
 
 extern struct power_pmu *ppmu;
arch/powerpc/kernel/perf_counter.c +40 -2

59 lines not on this page · raw message

arch/powerpc/kernel/power4-pmu.c +41 -0

51 lines not on this page · raw message

arch/powerpc/kernel/power5+-pmu.c +43 -2

57 lines not on this page · raw message

arch/powerpc/kernel/power5-pmu.c +41 -0

51 lines not on this page · raw message

arch/powerpc/kernel/power6-pmu.c +44 -2

58 lines not on this page · raw message

arch/powerpc/kernel/power7-pmu.c +41 -0

51 lines not on this page · raw message

arch/powerpc/kernel/ppc970-pmu.c +41 -0

51 lines not on this page · raw message

Signature · 1 line
1.5.5.rc3.7.gba13
#2 lore raw

Re: [PATCH 2/2] perf_counter: powerpc: Implement generalized cache events for POWER processors

Quoted 6 lines · Paul Mackerras

This adds tables of event codes for the generalized cache events for all the currently supported powerpc processors:
POWER{4,5,5+,6,7} and PPC970*, plus powerpc-specific code to use these tables when a generalized cache event is requested.

Quoted 10 lines
---
 arch/powerpc/include/asm/perf_counter.h |    3 ++
 arch/powerpc/kernel/perf_counter.c      |   42 +++++++++++++++++++++++++++-
 arch/powerpc/kernel/power4-pmu.c        |   41 +++++++++++++++++++++++++++
 arch/powerpc/kernel/power5+-pmu.c       |   45 +++++++++++++++++++++++++++++-
 arch/powerpc/kernel/power5-pmu.c        |   41 +++++++++++++++++++++++++++
 arch/powerpc/kernel/power6-pmu.c        |   46 +++++++++++++++++++++++++++++-
 arch/powerpc/kernel/power7-pmu.c        |   41 +++++++++++++++++++++++++++
 arch/powerpc/kernel/ppc970-pmu.c        |   41 +++++++++++++++++++++++++++
 8 files changed, 294 insertions(+), 6 deletions(-)

Ah, cool! I tried to construct the table so that Power would be able to fill it in a meaningful way - it seems like that was indeed possible.

Any particular observations you have about the cache events generalization? Would you do more of them (which ones?), fewer of them?

We can also add transparent fallback logic to the tools perhaps: for example a 'hits == total-misses' combo counter.

This can be expressed in the sampling space too: the latest tools do weighted samples, so we can actually do _negative_, weighted
sampling: the misses can subtract from a function's ->count value.

I dont know whether we should do such combo counters in the kernel itself - i'm slightly against that notion. (seems complex)

One last-minute change we are thinking about is to change 'L2' to 'LLC'. This matters on systems which have a L3 cache. The first level and the last level cache are generally the most important ones. What do you think?

arch/powerpc/kernel/power6-pmu.c static int power6_generic_events[] = { · line 514

Quoted 4 lines
arch/powerpc/kernel/power6-pmu.c +4 -0
+	[C(BPU)] = {		/* 	RESULT_ACCESS	RESULT_MISS */
+		[C(OP_READ)] = {	0x430e6,	0x400052	},
+		[C(OP_WRITE)] = {	-1,		-1		},
+		[C(OP_PREFETCH)] = {	-1,		-1		},

Ah, the RESULT_ACCESS/RESULT_MISS tabularization is a nice aesthetic touch - will do that for x86 too.

arch/powerpc/kernel/power6-pmu.c struct power_pmu power6_pmu = { · line 531

Quoted 11 lines
arch/powerpc/kernel/power6-pmu.c +3 -2
@@ -483,8 +524,9 @@ struct power_pmu power6_pmu = {
 	.get_constraint = p6_get_constraint,
 	.get_alternatives = p6_get_alternatives,
 	.disable_pmc = p6_disable_pmc,
+	.limited_pmc_event = p6_limited_pmc_event,
+	.flags = PPMU_LIMITED_PMC5_6 | PPMU_ALT_SIPR,
 	.n_generic = ARRAY_SIZE(power6_generic_events),
 	.generic_events = power6_generic_events,
-	.flags = PPMU_LIMITED_PMC5_6 | PPMU_ALT_SIPR,
-	.limited_pmc_event = p6_limited_pmc_event,
+	.cache_events = &power6_cache_events,

Btw., a very small nit, any way i could convince you to do such mass-initializations in the Power code, in the way we do elsewhere in perfcounters, by using vertical spacing:

.get_constraint = p6_get_constraint,
.get_alternatives = p6_get_alternatives,
.disable_pmc = p6_disable_pmc,

	.limited_pmc_event	= p6_limited_pmc_event,
	.flags			= PPMU_LIMITED_PMC5_6 | PPMU_ALT_SIPR,
  	.n_generic		= ARRAY_SIZE(power6_generic_events),
  	.generic_events		= power6_generic_events,
	.cache_events		= &power6_cache_events,

IMHO that form is infinitely more readable.

Thanks,

Ingo

#3 lore raw

Re: [PATCH 2/2] perf_counter: powerpc: Implement generalized cache events for POWER processors

Quoted 3 lines · Ingo Molnar

Ah, cool! I tried to construct the table so that Power would be able to fill it in a meaningful way - it seems like that was indeed possible.

Yes, by and large. The coverage is a little spotty on some processors but there's enough there to be useful IMO.

Quoted 3 lines

Any particular observations you have about the cache events generalization? Would you do more of them (which ones?), fewer of them?

One thing I noticed is that most of our processors have events for counting how many times data for a load comes from each of various sources. On our larger machines it's not a simple hierarchy because data can come from an L2 or L3 cache in another chip or another node, or from memory. On POWER6 for example there are separate events for data being loaded from each possible source, further divided up by the cacheline state (shared or modified) for the cache sources. So we have ~ 18 separate data source events for POWER6 (not counting the L1 hit case). And similarly for events counting where instructions are fetched from and where PTEs are fetched from.

It's a slightly different way of looking at things, I guess, looking at the distribution of where a processor is getting its data from instead of focusing on a particular cache and counting how often it does or doesn't supply data on request.

Does x86 have anything similar?

Quoted 6 lines

We can also add transparent fallback logic to the tools perhaps: for example a 'hits == total-misses' combo counter.

This can be expressed in the sampling space too: the latest tools do weighted samples, so we can actually do _negative_, weighted
sampling: the misses can subtract from a function's ->count value.

Cute, I hadn't noticed that.

Quoted 2 lines

I dont know whether we should do such combo counters in the kernel itself - i'm slightly against that notion. (seems complex)

Yeah.

When thinking about having "composite" events, i.e. a counter whose value is computed from two or more hardware counters, I couldn't see how to do sampling in the general case. It's easy if we're just adding multiple counters, but sampling when subtracting counters is hard. For example, if you want to sample every N cache hits, and you're computing hits as accesses - misses, I couldn't see a decent way to know when to take the sample, not without having to take an interrupt on every access in some circumstances.

Quoted 4 lines

One last-minute change we are thinking about is to change 'L2' to 'LLC'. This matters on systems which have a L3 cache. The first level and the last level cache are generally the most important ones. What do you think?

It's probably a good idea. I'll have to put in code to detect whether the system has L3 caches and adjust the table (or switch to a different table), but that's doable.

There aren't "last level cache" events on POWER processors, except to the extent that the "data loaded from memory" events imply that no cache had the data. But there's 3 separate memory-source events on POWER6, for instance, for memory attached to this core, another core in this node, or another node.

Actually, it looks like the L3 miss event we have on POWER6 for instance just refers to the local L3. It could be a miss in the local L3 but a hit in the L3 in another node, so the data will come from the remote L3 but still be counted as an L3 miss.

Quoted 7 lines

arch/powerpc/kernel/power6-pmu.c static int power6_generic_events[] = { · line 514

Quoted 4 lines
arch/powerpc/kernel/power6-pmu.c +4 -0
+	[C(BPU)] = {		/* 	RESULT_ACCESS	RESULT_MISS */
+		[C(OP_READ)] = {	0x430e6,	0x400052	},
+		[C(OP_WRITE)] = {	-1,		-1		},
+		[C(OP_PREFETCH)] = {	-1,		-1		},

Ah, the RESULT_ACCESS/RESULT_MISS tabularization is a nice aesthetic touch - will do that for x86 too.

Yeah, it is quite clear while using only 1/4 of the vertical space.

Quoted 3 lines

Btw., a very small nit, any way i could convince you to do such mass-initializations in the Power code, in the way we do elsewhere in perfcounters, by using vertical spacing:

Sure.

Paul.

#4 lore raw

Re: [PATCH 2/2] perf_counter: powerpc: Implement generalized cache events for POWER processors

Quoted 6 lines · Paul Mackerras
Quoted 3 lines

I dont know whether we should do such combo counters in the kernel itself - i'm slightly against that notion. (seems complex)

Yeah.

Quoted 9 lines

When thinking about having "composite" events, i.e. a counter whose value is computed from two or more hardware counters, I couldn't see how to do sampling in the general case. It's easy if we're just adding multiple counters, but sampling when subtracting counters is hard. For example, if you want to sample every N cache hits, and you're computing hits as accesses - misses, I couldn't see a decent way to know when to take the sample, not without having to take an interrupt on every access in some circumstances.

We now have a period field - and that could be negative and be subtracted by the profiler automatically.

It's still statistical and a given instruction can go 'negative' sporadically, but in terms of total function averages and for any high-traffic place it's still pretty expressive IMO.

Ingo

#5 lore raw

Re: [PATCH 2/2] perf_counter: powerpc: Implement generalized cache events for POWER processors

In reply to Paul Mackerras
Quoted 5 lines · Paul Mackerras

Ingo Molnar writes:

Yeah.

Quoted 8 lines

When thinking about having "composite" events, i.e. a counter whose value is computed from two or more hardware counters, I couldn't see how to do sampling in the general case. It's easy if we're just adding multiple counters, but sampling when subtracting counters is hard. For example, if you want to sample every N cache hits, and you're computing hits as accesses - misses, I couldn't see a decent way to know when to take the sample, not without having to take an interrupt on every access in some circumstances.

The PAPI equivalent of this, its preset aka standard events, do not allow profiling or interrupt on overflow for "derived" events. "derived events" has the same meaning as your composite events. So there is precedent for not allowing sampling on them.

Regards,

- Corey

Corey Ashford
Software Engineer
IBM Linux Technology Center, Linux Toolchain

#6 lore raw

[tip:branch?] perf_counter: powerpc: Implement generalized cache events for POWER processors

In reply to Paul Mackerras
Commit-ID:  905de321d7499be1b5d1ae135ea390604a52d32d
Gitweb:     http://git.kernel.org/tip/905de321d7499be1b5d1ae135ea390604a52d32d
Author:     Paul Mackerras <[email protected]>
AuthorDate: Thu, 11 Jun 2009 14:55:42 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu, 11 Jun 2009 12:07:37 +0200

perf_counter: powerpc: Implement generalized cache events for POWER processors

This adds tables of event codes for the generalized cache events for all the currently supported powerpc processors: POWER{4,5,5+,6,7} and PPC970*, plus powerpc-specific code to use these tables when a generalized cache event is requested.

LKML-Reference: <[email protected]>

---
 arch/powerpc/include/asm/perf_counter.h |    3 ++
 arch/powerpc/kernel/perf_counter.c      |   42 +++++++++++++++++++++++++++-
 arch/powerpc/kernel/power4-pmu.c        |   41 +++++++++++++++++++++++++++
 arch/powerpc/kernel/power5+-pmu.c       |   45 +++++++++++++++++++++++++++++-
 arch/powerpc/kernel/power5-pmu.c        |   41 +++++++++++++++++++++++++++
 arch/powerpc/kernel/power6-pmu.c        |   46 +++++++++++++++++++++++++++++-
 arch/powerpc/kernel/power7-pmu.c        |   41 +++++++++++++++++++++++++++
 arch/powerpc/kernel/ppc970-pmu.c        |   41 +++++++++++++++++++++++++++
 8 files changed, 294 insertions(+), 6 deletions(-)
arch/powerpc/include/asm/perf_counter.h +3 -0
index 1c60f0c..cc7c887 100644
@@ -33,6 +33,9 @@ struct power_pmu {
 	u32	flags;
 	int	n_generic;
 	int	*generic_events;
+	int	(*cache_events)[PERF_COUNT_HW_CACHE_MAX]
+			       [PERF_COUNT_HW_CACHE_OP_MAX]
+			       [PERF_COUNT_HW_CACHE_RESULT_MAX];
 };
 
 extern struct power_pmu *ppmu;
arch/powerpc/kernel/perf_counter.c +40 -2

59 lines not on this page · raw message

arch/powerpc/kernel/power4-pmu.c +41 -0

51 lines not on this page · raw message

arch/powerpc/kernel/power5+-pmu.c +43 -2

57 lines not on this page · raw message

arch/powerpc/kernel/power5-pmu.c +41 -0

51 lines not on this page · raw message

arch/powerpc/kernel/power6-pmu.c +44 -2

58 lines not on this page · raw message

arch/powerpc/kernel/power7-pmu.c +41 -0

51 lines not on this page · raw message

arch/powerpc/kernel/ppc970-pmu.c +41 -0

51 lines not on this page · raw message

#7 lore raw
In reply to Paul Mackerras
Commit-ID:  526e9a09d2e2dcb6c75c68710465678bb647e5fe
Gitweb:     http://git.kernel.org/tip/526e9a09d2e2dcb6c75c68710465678bb647e5fe
Author:     Paul Mackerras <[email protected]>
AuthorDate: Thu, 11 Jun 2009 14:54:01 +1000
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu, 11 Jun 2009 12:07:37 +0200

perf_counters: powerpc: Add support for POWER7 processors

This adds the back-end for the PMU on POWER7 processors. POWER7 has 4 fully-programmable counters and two fixed-function counters (which do respect the freeze conditions, can generate interrupts, and are writable, unlike PMC5/6 on POWER5+/6).

LKML-Reference: <[email protected]>

---
 arch/powerpc/kernel/Makefile       |    3 +-
 arch/powerpc/kernel/perf_counter.c |    4 +
 arch/powerpc/kernel/power7-pmu.c   |  316 ++++++++++++++++++++++++++++++++++++
 3 files changed, 322 insertions(+), 1 deletions(-)
arch/powerpc/kernel/Makefile +2 -1
index 9ba1bb7..a2c6834 100644
@@ -95,7 +95,8 @@ obj64-$(CONFIG_AUDIT)		+= compat_audit.o
 obj-$(CONFIG_DYNAMIC_FTRACE)	+= ftrace.o
 obj-$(CONFIG_FUNCTION_GRAPH_TRACER)	+= ftrace.o
 obj-$(CONFIG_PERF_COUNTERS)	+= perf_counter.o power4-pmu.o ppc970-pmu.o \
-				   power5-pmu.o power5+-pmu.o power6-pmu.o
+				   power5-pmu.o power5+-pmu.o power6-pmu.o \
+				   power7-pmu.o
 
 obj-$(CONFIG_8XX_MINIMAL_FPEMU) += softemu8xx.o
 
arch/powerpc/kernel/perf_counter.c +4 -0
index 4990ce2..5d12e68 100644
@@ -1181,6 +1181,7 @@ extern struct power_pmu ppc970_pmu;
 extern struct power_pmu power5_pmu;
 extern struct power_pmu power5p_pmu;
 extern struct power_pmu power6_pmu;
+extern struct power_pmu power7_pmu;
 
 static int init_perf_counters(void)
 {
@@ -1207,6 +1208,9 @@ static int init_perf_counters(void)
 	case 0x3e:
 		ppmu = &power6_pmu;
 		break;
+	case 0x3f:
+		ppmu = &power7_pmu;
+		break;
 	}
 
 	/*
arch/powerpc/kernel/power7-pmu.c new file +316 -0

316 lines not on this page · raw message