Skip to contentSkip to conversation

perf: add memory access sampling support

PATCH Superseded Stephane Eranian 3 participants 33 messages Jan 15, 2013 – Jan 24, 2013

Subsystems (2)

v6 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v6 cover perf: add memory access sampling support

This patch series had a new feature to the kernel perf_events interface and corresponding user level tool, perf.

With this patch, it is possible to sample (not trace) memory accesses (load, store). For loads, the instruction and data addresses are captured along with the latency and data source. For stores, the instruction and data addresses are capture along with limited cache and TLB information.

For load data source, the memory hierarchy level, the tlb, snoop and lock information is captured.

Although the perf_event interface is extended in a generic manner, sampling memory accesses requires HW support. The current patches implement the feature on Intel processors starting with Nehalem. The patches leverage the PEBS Load Latency and Precise Store mechanisms. Precise Store is present only on Sandy Bridge and Ivy Bridge based processors.

The perf tool is extended to make capturing and analyzing the data easier with a new command: perf mem.

$ perf mem -t load rec triad
$ perf mem -t load rep --stdio
# Samples: 19K of event 'cpu/mem-loads/pp'
# Total cost : 1013994
# Sort order : cost,mem,sym,dso,symbol_daddr,dso_daddr,snoop,tlb,locked
#
# Overhead      Samples     Cost  Memory access Symbol      Shared Obj  Data Symbol             Data Object Snoop  TLB access   Locked
# ........  ...........  .......  ............. ..........  ........... ......................  ........... ...... ............ ......
#
     0.10%            1      986  LFB hit       [.] triad   triad       [.] 0x00007f67dffe8038  [unknown]    None  L1 or L2 hit  No    
     0.09%            1      890  LFB hit       [.] triad   triad       [.] 0x00007f67df91a750  [unknown]    None  L1 or L2 hit  No    
     0.08%            1      826  LFB hit       [.] triad   triad       [.] 0x00007f67e288fba8  [unknown]    None  L1 or L2 hit  No    
     0.08%            1      825  LFB hit       [.] triad   triad       [.] 0x00007f67dea28c80  [unknown]    None  L1 or L2 hit  No    
     0.08%            1      787  LFB hit       [.] triad   triad       [.] 0x00007f67df055a60  [unknown]    None  L1 or L2 hit  No    

The perf mem command is a wrapper around perf record/report. It passes the right options to the report and record commands. Note that the TUI mode is supported.

One powerful feature of perf is that users can toy with sort order to display the information in different format or from a different angle. This is particularly useful with memory sampling:

$ perf mem -t load rep --sort=mem
# Samples: 19K of event 'cpu/mem-loads/pp'
# Total cost : 1013994
# Sort order : mem
#
# Overhead      Samples             Memory access
# ........  ...........  ........................
#
    85.26%        10633  LFB hit                 
     7.35%         8151  L1 hit                  
     3.13%          383  L3 hit                  
     3.09%          195  Local RAM hit           
     1.16%          259  L2 hit                  
     0.00%            4  Uncached hit            

Or if one is interested in the data view:

$ perf mem -t load rep --sort=symbol_daddr,cost
# Samples: 19K of event 'cpu/mem-loads/pp'
# Total cost : 1013994
# Sort order : symbol_daddr,cost
#
# Overhead      Samples             Data Symbol     Cost
# ........  ...........  ......................  .......
#
     0.10%            1  [.] 0x00007f67dffe8038      986
     0.09%            1  [.] 0x00007f67df91a750      890
     0.08%            1  [.] 0x00007f67e288fba8      826

One note on the cost displayed: On Intel processors with PEBS Load Latency, as described in the SDM, the cost encompasses the number of cycles from dispatch to Globally Observable (GO) state. That means, that it includes OOO execution. It is not usual to see L1D Hits with a cost of > 100 cycles. Always look at the memory level for an approximation of the access penalty, then interpret the cost value accordingly.

Data symbolization is working for initialized global variables. Dynamically allocated data and bss symbolization is currently non-functional.

There is no cost associated with stores.

In v2, we leverage some of Andi Kleen's Haswell patches, namely the weighted samples and perf tool event parser fixes. We also introduce PERF_RECORD_MISC_DATA_MMAP to tag mmaps for data vs. code. This helps the perf tool distinguish data. vs. code mmaps (and therefore symbols). We have also integrated the feedback from v1. Note that in v2 data symbol resolution is not yet fully operational, but there is a slight improvement.

In v3, we rebased the patch on 3.7.0-rc6 which includes certain of Nahyung's patches.

In v4, we rebase to v3.7 tip and also included the fixes from Namhyung Kim for symbolization of data addresses. We now have accesses to global variables working.

In v5, we rebase to 3.8.0-rc1. We also updated the WEIGHT patches from Andi to fix a couple of issues. Integrated the feedback from jolsa@. Reintegrated the man page.

In v6, we rebased to 3.8.0-rc3 and fixed the issue reported by jolsa@ related to hist_entry->mem_info maps not being marked as referenced.

Andi Kleen (2):
perf, x86: Support CPU specific sysfs events perf, core: Add a concept of a weightened sample

Namhyung Kim (2):
perf tools: Ignore ABS symbols when loading data maps perf tools: Fix output of symbol_daddr offset

Stephane Eranian (14):
perf/x86: improve sysfs event mapping with event string perf/x86: add flags to event constraints
perf: add minimal support for PERF_SAMPLE_WEIGHT
perf: add support for PERF_SAMPLE_ADDR in dump_sampple()
perf: add generic memory sampling interface perf/x86: add memory profiling via PEBS Load Latency perf/x86: export PEBS load latency threshold register to sysfs perf/x86: add support for PEBS Precise Store
perf tools: add mem access sampling core support perf report: add support for mem access profiling perf record: add support for mem access profiling perf tools: add new mem command for memory access profiling
perf: add PERF_RECORD_MISC_MMAP_DATA to RECORD_MMAP perf tools: detect data vs. text mappings

 arch/x86/include/uapi/asm/msr-index.h         |    1 +
 arch/x86/kernel/cpu/perf_event.c              |   67 +++--
 arch/x86/kernel/cpu/perf_event.h              |   62 ++++-
 arch/x86/kernel/cpu/perf_event_intel.c        |   34 ++-
 arch/x86/kernel/cpu/perf_event_intel_ds.c     |  182 +++++++++++++-
 arch/x86/kernel/cpu/perf_event_intel_uncore.c |    2 +-
 include/linux/perf_event.h                    |    5 +
 include/uapi/linux/perf_event.h               |   70 +++++-
 kernel/events/core.c                          |   15 ++
 tools/perf/Documentation/perf-mem.txt         |   48 ++++
 tools/perf/Makefile                           |    1 +
 tools/perf/builtin-mem.c                      |  238 ++++++++++++++++++
 tools/perf/builtin-record.c                   |    2 +
 tools/perf/builtin-report.c                   |  131 +++++++++-
 tools/perf/builtin.h                          |    1 +
 tools/perf/command-list.txt                   |    1 +
 tools/perf/perf.c                             |    1 +
 tools/perf/perf.h                             |    1 +
 tools/perf/util/event.h                       |    2 +
 tools/perf/util/evsel.c                       |   16 ++
 tools/perf/util/hist.c                        |   77 +++++-
 tools/perf/util/hist.h                        |   13 +
 tools/perf/util/machine.c                     |   10 +-
 tools/perf/util/session.c                     |   45 ++++
 tools/perf/util/session.h                     |    4 +
 tools/perf/util/sort.c                        |  325 ++++++++++++++++++++++++-
 tools/perf/util/sort.h                        |   10 +-
 tools/perf/util/symbol-elf.c                  |    3 +
 tools/perf/util/symbol.h                      |    7 +
 29 files changed, 1326 insertions(+), 48 deletions(-)
 create mode 100644 tools/perf/Documentation/perf-mem.txt
 create mode 100644 tools/perf/builtin-mem.c
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH v6 1/18 perf, x86: Support CPU specific sysfs events

From: Andi Kleen <[email protected]>

Add a way for the CPU initialization code to register additional events, and merge them into the events attribute directory. Used in the next patch.

---
 arch/x86/kernel/cpu/perf_event.c |   33 +++++++++++++++++++++++++++++++++
 arch/x86/kernel/cpu/perf_event.h |    1 +
 2 files changed, 34 insertions(+)
arch/x86/kernel/cpu/perf_event.c +33 -0
index 4428fd1..26df58b 100644
@@ -1341,6 +1341,30 @@ static void __init filter_events(struct attribute **attrs)
 	}
 }
 
+/* Merge two pointer arrays */
+static __init struct attribute **merge_attr(struct attribute **a,
+					    struct attribute **b)
+{
+	struct attribute **new;
+	int j, i;
+
+	for (j = 0; a[j]; j++)
+		;
+	for (i = 0; b[i]; i++)
+		j++;
+	j++;
+	new = kmalloc(sizeof(struct attribute *) * j, GFP_KERNEL);
+	if (!new)
+		return NULL;
+	j = 0;
+	for (i = 0; a[i]; i++)
+		new[j++] = a[i];
+	for (i = 0; b[i]; i++)
+		new[j++] = b[i];
+	new[j] = NULL;
+	return new;
+}
+
 static ssize_t events_sysfs_show(struct device *dev, struct device_attribute *attr,
 			  char *page)
 {
@@ -1482,6 +1506,15 @@ static int __init init_hw_perf_events(void)
 	else
 		filter_events(x86_pmu_events_group.attrs);
 
+	if (x86_pmu.cpu_events) {
+		struct attribute *tmp;
+
+		tmp = merge_attr(x86_pmu_events_group.attrs,
+				 x86_pmu.cpu_events);
+		if (!WARN_ON(!tmp))
+			x86_pmu_events_group.attrs = tmp;
+	}
+
 	pr_info("... version:                %d\n",     x86_pmu.version);
 	pr_info("... bit width:              %d\n",     x86_pmu.cntval_bits);
 	pr_info("... generic registers:      %d\n",     x86_pmu.num_counters);
arch/x86/kernel/cpu/perf_event.h +1 -0
index 115c1ea..4170043 100644
@@ -355,6 +355,7 @@ struct x86_pmu {
 	struct attribute **format_attrs;
 
 	ssize_t		(*events_sysfs_show)(char *page, u64 config);
+	struct attribute **cpu_events;
 
 	/*
 	 * CPU Hotplug hooks
Signature · 1 line
1.7.9.5
#2 lore raw

PATCH v6 2/18 perf/x86: improve sysfs event mapping with event string

In reply to Stephane Eranian

This patch extends Jiri's changes to make generic events mapping visible via sysfs. The patch extends the mechanism to non-generic events by allowing the mappings to be hardcoded in strings.

This mechanism will be used by the PEBS-LL patch later on.

---
 arch/x86/kernel/cpu/perf_event.c |   27 ++++++++++++---------------
 arch/x86/kernel/cpu/perf_event.h |   23 +++++++++++++++++++++++
 2 files changed, 35 insertions(+), 15 deletions(-)
arch/x86/kernel/cpu/perf_event.c +12 -15
index 26df58b..5744dc9 100644
@@ -1316,20 +1316,22 @@ static struct attribute_group x86_pmu_format_group = {
 	.attrs = NULL,
 };
 
-struct perf_pmu_events_attr {
-	struct device_attribute attr;
-	u64 id;
-};
-
 /*
  * Remove all undefined events (x86_pmu.event_map(id) == 0)
  * out of events_attr attributes.
  */
 static void __init filter_events(struct attribute **attrs)
 {
+	struct device_attribute *d;
+	struct perf_pmu_events_attr *pmu_attr;
 	int i, j;
 
 	for (i = 0; attrs[i]; i++) {
+		d = (struct device_attribute *)attrs[i];
+		pmu_attr = container_of(d, struct perf_pmu_events_attr, attr);
+		/* str trumps id */
+		if (pmu_attr->event_str)
+			continue;
 		if (x86_pmu.event_map(i))
 			continue;
 
@@ -1370,19 +1372,14 @@ static ssize_t events_sysfs_show(struct device *dev, struct device_attribute *at
 {
 	struct perf_pmu_events_attr *pmu_attr = \
 		container_of(attr, struct perf_pmu_events_attr, attr);
-
 	u64 config = x86_pmu.event_map(pmu_attr->id);
-	return x86_pmu.events_sysfs_show(page, config);
-}
 
-#define EVENT_VAR(_id)  event_attr_##_id
-#define EVENT_PTR(_id) &event_attr_##_id.attr.attr
+	/* string trumps id */
+	if (pmu_attr->event_str)
+		return sprintf(page, "%s", pmu_attr->event_str);
 
-#define EVENT_ATTR(_name, _id)					\
-static struct perf_pmu_events_attr EVENT_VAR(_id) = {		\
-	.attr = __ATTR(_name, 0444, events_sysfs_show, NULL),	\
-	.id   =  PERF_COUNT_HW_##_id,				\
-};
+	return x86_pmu.events_sysfs_show(page, config);
+}
 
 EVENT_ATTR(cpu-cycles,			CPU_CYCLES		);
 EVENT_ATTR(instructions,		INSTRUCTIONS		);
arch/x86/kernel/cpu/perf_event.h +23 -0
index 4170043..3f4380c 100644
@@ -420,6 +420,29 @@ do {									\
 #define ERF_NO_HT_SHARING	1
 #define ERF_HAS_RSP_1		2
 
+#define EVENT_VAR(_id)  event_attr_##_id
+#define EVENT_PTR(_id) &event_attr_##_id.attr.attr
+
+#define EVENT_ATTR(_name, _id)					\
+static struct perf_pmu_events_attr EVENT_VAR(_id) = {		\
+	.attr = __ATTR(_name, 0444, events_sysfs_show, NULL),	\
+	.id   =  PERF_COUNT_HW_##_id,				\
+	.event_str = NULL,					\
+};
+
+#define EVENT_ATTR_STR(_name, v, str)				  \
+static struct perf_pmu_events_attr event_attr_##v = {		  \
+	.attr      = __ATTR(_name, 0444, events_sysfs_show, NULL),\
+	.id        =  0,					  \
+	.event_str =  str,					  \
+};
+
+struct perf_pmu_events_attr {
+	struct device_attribute attr;
+	u64 id;
+	const char *event_str;
+};
+
 extern struct x86_pmu x86_pmu __read_mostly;
 
 DECLARE_PER_CPU(struct cpu_hw_events, cpu_hw_events);
Signature · 1 line
1.7.9.5
Acked-by on v6 2/18
#3 lore raw

Re: [PATCH v6 02/18] perf/x86: improve sysfs event mapping with event string

Quoted 9 lines · Stephane Eranian

This patch extends Jiri's changes to make generic events mapping visible via sysfs. The patch extends the mechanism to non-generic events by allowing the mappings to be hardcoded in strings.

This mechanism will be used by the PEBS-LL patch later on.

Looks good to me.

#4 lore raw

PATCH v6 3/18 perf/x86: add flags to event constraints

In reply to Stephane Eranian

This patch adds a flags field to each event constraint. It can be used to store event specific features which can then later be used by scheduling code or low-level x86 code.

The flags are propagated into event->hw.flags during the get_event_constraint() call. They are cleared during the put_event_constraint() call.

This mechanism is going to be used by the PEBS-LL patches. It avoids defining yet another table to hold event specific information.

---
 arch/x86/kernel/cpu/perf_event.c              |    2 +-
 arch/x86/kernel/cpu/perf_event.h              |    8 +++++---
 arch/x86/kernel/cpu/perf_event_intel.c        |    5 ++++-
 arch/x86/kernel/cpu/perf_event_intel_ds.c     |    4 +++-
 arch/x86/kernel/cpu/perf_event_intel_uncore.c |    2 +-
 include/linux/perf_event.h                    |    1 +
 6 files changed, 15 insertions(+), 7 deletions(-)
arch/x86/kernel/cpu/perf_event.c +1 -1
index 5744dc9..35b516a 100644
@@ -1493,7 +1493,7 @@ static int __init init_hw_perf_events(void)
 
 	unconstrained = (struct event_constraint)
 		__EVENT_CONSTRAINT(0, (1ULL << x86_pmu.num_counters) - 1,
-				   0, x86_pmu.num_counters, 0);
+				   0, x86_pmu.num_counters, 0, 0);
 
 	x86_pmu.attr_rdpmc = 1; /* enable userspace RDPMC usage by default */
 	x86_pmu_format_group.attrs = x86_pmu.format_attrs;
arch/x86/kernel/cpu/perf_event.h +5 -3
index 3f4380c..3f10cfe 100644
@@ -59,6 +59,7 @@ struct event_constraint {
 	u64	cmask;
 	int	weight;
 	int	overlap;
+	int	flags;
 };
 
 struct amd_nb {
@@ -170,16 +171,17 @@ struct cpu_hw_events {
 	void				*kfree_on_online;
 };
 
-#define __EVENT_CONSTRAINT(c, n, m, w, o) {\
+#define __EVENT_CONSTRAINT(c, n, m, w, o, f) {\
 	{ .idxmsk64 = (n) },		\
 	.code = (c),			\
 	.cmask = (m),			\
 	.weight = (w),			\
 	.overlap = (o),			\
+	.flags = f,			\
 }
 
 #define EVENT_CONSTRAINT(c, n, m)	\
-	__EVENT_CONSTRAINT(c, n, m, HWEIGHT(n), 0)
+	__EVENT_CONSTRAINT(c, n, m, HWEIGHT(n), 0, 0)
 
 /*
  * The overlap flag marks event constraints with overlapping counter
@@ -203,7 +205,7 @@ struct cpu_hw_events {
  * and its counter masks must be kept at a minimum.
  */
 #define EVENT_CONSTRAINT_OVERLAP(c, n, m)	\
-	__EVENT_CONSTRAINT(c, n, m, HWEIGHT(n), 1)
+	__EVENT_CONSTRAINT(c, n, m, HWEIGHT(n), 1, 0)
 
 /*
  * Constraint on the Event code.
arch/x86/kernel/cpu/perf_event_intel.c +4 -1
index 93b9e11..57d6527 100644
@@ -1367,8 +1367,10 @@ x86_get_event_constraints(struct cpu_hw_events *cpuc, struct perf_event *event)
 
 	if (x86_pmu.event_constraints) {
 		for_each_event_constraint(c, x86_pmu.event_constraints) {
-			if ((event->hw.config & c->cmask) == c->code)
+			if ((event->hw.config & c->cmask) == c->code) {
+				event->hw.flags |= c->flags;
 				return c;
+			}
 		}
 	}
 
@@ -1413,6 +1415,7 @@ intel_put_shared_regs_event_constraints(struct cpu_hw_events *cpuc,
 static void intel_put_event_constraints(struct cpu_hw_events *cpuc,
 					struct perf_event *event)
 {
+	event->hw.flags = 0;
 	intel_put_shared_regs_event_constraints(cpuc, event);
 }
 
arch/x86/kernel/cpu/perf_event_intel_ds.c +3 -1
index 826054a..f30d85b 100644
@@ -430,8 +430,10 @@ struct event_constraint *intel_pebs_constraints(struct perf_event *event)
 
 	if (x86_pmu.pebs_constraints) {
 		for_each_event_constraint(c, x86_pmu.pebs_constraints) {
-			if ((event->hw.config & c->cmask) == c->code)
+			if ((event->hw.config & c->cmask) == c->code) {
+				event->hw.flags |= c->flags;
 				return c;
+			}
 		}
 	}
 
arch/x86/kernel/cpu/perf_event_intel_uncore.c +1 -1
index b43200d..75da9e1 100644
@@ -2438,7 +2438,7 @@ static int __init uncore_type_init(struct intel_uncore_type *type)
 
 	type->unconstrainted = (struct event_constraint)
 		__EVENT_CONSTRAINT(0, (1ULL << type->num_counters) - 1,
-				0, type->num_counters, 0);
+				0, type->num_counters, 0, 0);
 
 	for (i = 0; i < type->num_boxes; i++) {
 		pmus[i].func_id = -1;
include/linux/perf_event.h +1 -0
index 6bfb2faa..484cfbc 100644
@@ -128,6 +128,7 @@ struct hw_perf_event {
 			int		event_base_rdpmc;
 			int		idx;
 			int		last_cpu;
+			int		flags;
 
 			struct hw_perf_event_extra extra_reg;
 			struct hw_perf_event_extra branch_reg;
Signature · 1 line
1.7.9.5
#5 lore raw

Re: [PATCH v6 03/18] perf/x86: add flags to event constraints

arch/x86/kernel/cpu/perf_event_intel.c x86_get_event_constraints(struct cpu_hw_events *cpuc, struct perf_event *event) · line 1373

Quoted 11 lines
arch/x86/kernel/cpu/perf_event_intel.c +3 -1
@@ -1367,8 +1367,10 @@ x86_get_event_constraints(struct cpu_hw_events *cpuc, struct perf_event *event)
 
 	if (x86_pmu.event_constraints) {
 		for_each_event_constraint(c, x86_pmu.event_constraints) {
-			if ((event->hw.config & c->cmask) == c->code)
+			if ((event->hw.config & c->cmask) == c->code) {
+				event->hw.flags |= c->flags;
 				return c;
+			}

It's not fully clear where that hw.flags field gets initially zeroed. Is that implicit in the allocation? Some comments would be good about its live cycle.

Or just use a = instead of |=? Why would you have multiple flags in different places?

-Andi

#6 lore raw

Re: [PATCH v6 03/18] perf/x86: add flags to event constraints

Quoted 4 lines · Andi Kleen
Quoted 4 lines
arch/x86/kernel/cpu/perf_event_intel.c +0 -0
@@ -1367,8 +1367,10 @@ x86_get_event_constraints(struct cpu_hw_events *cpuc, struct perf_event *event)
 
Quoted 11 lines

arch/x86/kernel/cpu/perf_event_intel.c x86_get_event_constraints(struct cpu_hw_events *cpuc, struct perf_event *event) · line 1373

Quoted 7 lines
arch/x86/kernel/cpu/perf_event_intel.c +3 -1
      if (x86_pmu.event_constraints) {
              for_each_event_constraint(c, x86_pmu.event_constraints) {
-                     if ((event->hw.config & c->cmask) == c->code)
+                     if ((event->hw.config & c->cmask) == c->code) {
+                             event->hw.flags |= c->flags;
                              return c;
+                     }

It's not fully clear where that hw.flags field gets initially zeroed. Is that implicit in the allocation? Some comments would be good about its live cycle.

Yes, this is by allocation.
I used |= in case we need to add more flags in the future. I will add a comment.

Quoted 3 lines

Or just use a = instead of |=? Why would you have multiple flags in different places?

-Andi

#7 lore raw

PATCH v6 4/18 perf, core: Add a concept of a weightened sample

In reply to Stephane Eranian

From: Andi Kleen <[email protected]>

For some events it's useful to weight sample with a hardware provided number. This expresses how expensive the action the sample represent was. This allows the profiler to scale the samples to be more informative to the programmer.

There is already the period which is used similarly, but it means something different, so I chose to not overload it. Instead a new sample type for WEIGHT is added.

Can be used for multiple things. Initially it is used for TSX abort costs and profiling by memory latencies (so to make expensive load appear higher up in the histograms) The concept is quite generic and can be extended to many other kinds of events or architectures, as long as the hardware provides suitable auxillary values. In principle it could be also used for software tracpoints.

This adds the generic glue. A new optional sample format for a 64bit weight value.

---
 include/linux/perf_event.h      |    2 ++
 include/uapi/linux/perf_event.h |    5 ++++-
 kernel/events/core.c            |    6 ++++++
 3 files changed, 12 insertions(+), 1 deletion(-)
include/linux/perf_event.h +2 -0
index 484cfbc..bb2429d 100644
@@ -584,6 +584,7 @@ struct perf_sample_data {
 	struct perf_branch_stack	*br_stack;
 	struct perf_regs_user		regs_user;
 	u64				stack_user_size;
+	u64				weight;
 };
 
 static inline void perf_sample_data_init(struct perf_sample_data *data,
@@ -597,6 +598,7 @@ static inline void perf_sample_data_init(struct perf_sample_data *data,
 	data->regs_user.abi = PERF_SAMPLE_REGS_ABI_NONE;
 	data->regs_user.regs = NULL;
 	data->stack_user_size = 0;
+	data->weight = 0;
 }
 
 extern void perf_output_sample(struct perf_output_handle *handle,
include/uapi/linux/perf_event.h +4 -1
index 4f63c05..7e24641 100644
@@ -132,8 +132,10 @@ enum perf_event_sample_format {
 	PERF_SAMPLE_BRANCH_STACK		= 1U << 11,
 	PERF_SAMPLE_REGS_USER			= 1U << 12,
 	PERF_SAMPLE_STACK_USER			= 1U << 13,
+	PERF_SAMPLE_WEIGHT			= 1U << 14,
+
+	PERF_SAMPLE_MAX = 1U << 15,		/* non-ABI */
 
-	PERF_SAMPLE_MAX = 1U << 14,		/* non-ABI */
 };
 
 /*
@@ -587,6 +589,7 @@ enum perf_event_type {
 	 * 	{ u64			size;
 	 * 	  char			data[size];
 	 * 	  u64			dyn_size; } && PERF_SAMPLE_STACK_USER
+	 *	{ u64			weight;   } && PERF_SAMPLE_WEIGHT
 	 * };
 	 */
 	PERF_RECORD_SAMPLE			= 9,
kernel/events/core.c +6 -0
index 301079d..bc2ce07 100644
@@ -955,6 +955,9 @@ static void perf_event__header_size(struct perf_event *event)
 	if (sample_type & PERF_SAMPLE_READ)
 		size += event->read_size;
 
+	if (sample_type & PERF_SAMPLE_WEIGHT)
+		size += sizeof(data->weight);
+
 	event->header_size = size;
 }
 
@@ -4169,6 +4172,9 @@ void perf_output_sample(struct perf_output_handle *handle,
 		perf_output_sample_ustack(handle,
 					  data->stack_user_size,
 					  data->regs_user.regs);
+
+	if (sample_type & PERF_SAMPLE_WEIGHT)
+		perf_output_put(handle, data->weight);
 }
 
 void perf_prepare_sample(struct perf_event_header *header,
Signature · 1 line
1.7.9.5
#8 lore raw

PATCH v6 5/18 perf: add minimal support for PERF_SAMPLE_WEIGHT

In reply to Stephane Eranian

Ensure we grab the weight from raw sample struct
and that we can dump it via perf report -D.

---
 tools/perf/util/event.h   |    1 +
 tools/perf/util/evsel.c   |    5 +++++
 tools/perf/util/session.c |    3 +++
 3 files changed, 9 insertions(+)
tools/perf/util/event.h +1 -0
index 0d573ff..cf52977 100644
@@ -90,6 +90,7 @@ struct perf_sample {
 	u64 period;
 	u32 cpu;
 	u32 raw_size;
+	u64 weight;
 	void *raw_data;
 	struct ip_callchain *callchain;
 	struct branch_stack *branch_stack;
tools/perf/util/evsel.c +5 -0
index 1b16dd1..e08ce12 100644
@@ -1058,6 +1058,11 @@ int perf_evsel__parse_sample(struct perf_evsel *evsel, union perf_event *event,
 		}
 	}
 
+	if (type & PERF_SAMPLE_WEIGHT) {
+		data->weight = *array;
+		array++;
+	}
+
 	return 0;
 }
 
tools/perf/util/session.c +3 -0
index ce6f511..117983e 100644
@@ -1006,6 +1006,9 @@ static void dump_sample(struct perf_evsel *evsel, union perf_event *event,
 
 	if (sample_type & PERF_SAMPLE_STACK_USER)
 		stack_user__printf(&sample->user_stack);
+
+	if (sample_type & PERF_SAMPLE_WEIGHT)
+		printf(" ... weight: %"PRIu64"\n", sample->weight);
 }
 
 static struct machine *
Signature · 1 line
1.7.9.5
#9 lore raw

Re: [PATCH v6 05/18] perf: add minimal support for PERF_SAMPLE_WEIGHT

Quoted 2 lines · Stephane Eranian

Ensure we grab the weight from raw sample struct
and that we can dump it via perf report -D.

AFAIK i have a similar patch in the haswell patchkit.

It's straight forward of course, just have to chose one. I think mine does slightly more.

-Andi

#10 lore raw

Re: [PATCH v6 05/18] perf: add minimal support for PERF_SAMPLE_WEIGHT

Quoted 9 lines · Andi Kleen
Quoted 2 lines · Stephane Eranian

Ensure we grab the weight from raw sample struct
and that we can dump it via perf report -D.

AFAIK i have a similar patch in the haswell patchkit.

It's straight forward of course, just have to chose one. I think mine does slightly more.

Yes, mine is a tiny subset of yours. Should use yours instead. Will swap with yours if that's easy to do.

#11 lore raw

PATCH v6 6/18 perf: add support for PERF_SAMPLE_ADDR in dump_sampple()

In reply to Stephane Eranian

Was missing from current code yet PERF_SAMPLE_ADDR has been present for a long time. Needed for PEBS-LL mode.

---
 tools/perf/util/session.c |    4 ++++
 1 file changed, 4 insertions(+)
tools/perf/util/session.c +4 -0
index 117983e..9900778 100644
@@ -1009,6 +1009,10 @@ static void dump_sample(struct perf_evsel *evsel, union perf_event *event,
 
 	if (sample_type & PERF_SAMPLE_WEIGHT)
 		printf(" ... weight: %"PRIu64"\n", sample->weight);
+
+	if (sample_type & PERF_SAMPLE_ADDR)
+		printf(" ..... data: 0x%"PRIx64"\n", sample->addr);
+
 }
 
 static struct machine *
Signature · 1 line
1.7.9.5
#12 lore raw

PATCH v6 7/18 perf: add generic memory sampling interface

In reply to Stephane Eranian

This patch adds PERF_SAMPLE_DSRC.

PERF_SAMPLE_DSRC collects the data source, i.e., where did the data associated with the sampled instruction come from. Information is stored in a perf_mem_dsrc structure. It contains opcode, mem level, tlb, snoop, lock information, subject to availability in hardware.

---
 include/linux/perf_event.h      |    2 ++
 include/uapi/linux/perf_event.h |   68 +++++++++++++++++++++++++++++++++++++--
 kernel/events/core.c            |    6 ++++
 3 files changed, 74 insertions(+), 2 deletions(-)
include/linux/perf_event.h +2 -0
index bb2429d..8fe4610 100644
@@ -579,6 +579,7 @@ struct perf_sample_data {
 		u32	reserved;
 	}				cpu_entry;
 	u64				period;
+	union  perf_mem_dsrc		dsrc;
 	struct perf_callchain_entry	*callchain;
 	struct perf_raw_record		*raw;
 	struct perf_branch_stack	*br_stack;
@@ -599,6 +600,7 @@ static inline void perf_sample_data_init(struct perf_sample_data *data,
 	data->regs_user.regs = NULL;
 	data->stack_user_size = 0;
 	data->weight = 0;
+	data->dsrc.val = 0;
 }
 
 extern void perf_output_sample(struct perf_output_handle *handle,
include/uapi/linux/perf_event.h +66 -2

85 lines not on this page · raw message

kernel/events/core.c +6 -0
index bc2ce07..fd4ceea 100644
@@ -958,6 +958,9 @@ static void perf_event__header_size(struct perf_event *event)
 	if (sample_type & PERF_SAMPLE_WEIGHT)
 		size += sizeof(data->weight);
 
+	if (sample_type & PERF_SAMPLE_DSRC)
+		size += sizeof(data->dsrc.val);
+
 	event->header_size = size;
 }
 
@@ -4175,6 +4178,9 @@ void perf_output_sample(struct perf_output_handle *handle,
 
 	if (sample_type & PERF_SAMPLE_WEIGHT)
 		perf_output_put(handle, data->weight);
+
+	if (sample_type & PERF_SAMPLE_DSRC)
+		perf_output_put(handle, data->dsrc.val);
 }
 
 void perf_prepare_sample(struct perf_event_header *header,
Signature · 1 line
1.7.9.5
#13 lore raw

Re: [PATCH v6 07/18] perf: add generic memory sampling interface

include/uapi/linux/perf_event.h enum perf_event_sample_format { · line 136

Quoted 10 lines

extern void perf_output_sample(struct perf_output_handle *handle,

include/uapi/linux/perf_event.h +1 -0
index 7e24641..8283218 100644
@@ -133,9 +133,9 @@ enum perf_event_sample_format {
 	PERF_SAMPLE_REGS_USER			= 1U << 12,
 	PERF_SAMPLE_STACK_USER			= 1U << 13,
 	PERF_SAMPLE_WEIGHT			= 1U << 14,
+	PERF_SAMPLE_DSRC			= 1U << 15,

This conflicts with similar extensions in the Haswell patchkit, but that can be worked out by just moving some numbers (and making sure the input/output calls are still in the right place)

Quoted 9 lines
include/uapi/linux/perf_event.h +9 -0
+union perf_mem_dsrc {
+	__u64 val;
+	struct {
+		__u64   mem_op:5,	/* type of opcode */
+			mem_lvl:14,	/* memory hierarchy level */
+			mem_snoop:5,	/* snoop mode */
+			mem_lock:2,	/* lock instr */
+			mem_dtlb:7,	/* tlb access */
+			mem_rsvd:31;

include/uapi/linux/perf_event.h enum perf_callchain_context { · line 637

Quoted 10 lines
include/uapi/linux/perf_event.h +10 -0
+	};
+};
+
+/* type of opcode (load/store/prefetch,code) */
+#define PERF_MEM_OP_NA		0x01 /* not available */
+#define PERF_MEM_OP_LOAD	0x02 /* load instruction */
+#define PERF_MEM_OP_STORE	0x04 /* store instruction */
+#define PERF_MEM_OP_PFETCH	0x08 /* prefetch */
+#define PERF_MEM_OP_EXEC	0x10 /* code (execution) */
+#define PERF_MEM_OP_SHIFT	0

Do we really need the shift? it's implicit in the bitfield right?

Quoted 5 lines
include/uapi/linux/perf_event.h +5 -0
+/* memory hierarchy (memory level, hit or miss) */
+#define PERF_MEM_LVL_NA		0x01  /* not available */
+#define PERF_MEM_LVL_HIT	0x02  /* hit level */
+#define PERF_MEM_LVL_MISS	0x04  /* miss level  */
+#define PERF_MEM_LVL_L1		0x08  /* L1 */

include/uapi/linux/perf_event.h enum perf_callchain_context { · line 653

Quoted 10 lines
include/uapi/linux/perf_event.h +10 -0
+#define PERF_MEM_LVL_LFB	0x10  /* Line Fill Buffer */
+#define PERF_MEM_LVL_L2		0x20  /* L2 hit */
+#define PERF_MEM_LVL_L3		0x40  /* L3 hit */
+#define PERF_MEM_LVL_LOC_RAM	0x80  /* Local DRAM */
+#define PERF_MEM_LVL_REM_RAM1	0x100 /* Remote DRAM (1 hop) */
+#define PERF_MEM_LVL_REM_RAM2	0x200 /* Remote DRAM (2 hops) */
+#define PERF_MEM_LVL_REM_CCE1	0x400 /* Remote Cache (1 hop) */
+#define PERF_MEM_LVL_REM_CCE2	0x800 /* Remote Cache (2 hops) */
+#define PERF_MEM_LVL_IO		0x1000 /* I/O memory */
+#define PERF_MEM_LVL_UNC	0x2000 /* Uncached memory */

I would leave some free bits here, obviously this doesn't cover all that may be possible in system architecture. Also why is this a bit mask, you can only hit one level right? So perhaps a number.

include/uapi/linux/perf_event.h enum perf_callchain_context { · line 676

Quoted 8 lines
include/uapi/linux/perf_event.h +8 -0
+/* TLB access */
+#define PERF_MEM_TLB_NA		0x01 /* not available */
+#define PERF_MEM_TLB_HIT	0x02 /* hit level */
+#define PERF_MEM_TLB_MISS	0x04 /* miss level */
+#define PERF_MEM_TLB_L1		0x08 /* L1 */
+#define PERF_MEM_TLB_L2		0x10 /* L2 */
+#define PERF_MEM_TLB_WK		0x20 /* Hardware Walker*/
+#define PERF_MEM_TLB_OS		0x40 /* OS fault handler */

Same

include/uapi/linux/perf_event.h enum perf_callchain_context { · line 680

Quoted 4 lines
include/uapi/linux/perf_event.h +4 -0
+#define PERF_MEM_TLB_SHIFT	26
+
+#define PERF_MEM_S(a, s) \
+	(((u64)PERF_MEM_##a##_##s) << PERF_MEM_##a##_SHIFT)

Is that used by anything?

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only
#14 lore raw

Re: [PATCH v6 07/18] perf: add generic memory sampling interface

Quoted 6 lines · Andi Kleen
Quoted 6 lines

extern void perf_output_sample(struct perf_output_handle *handle,

include/uapi/linux/perf_event.h +0 -0
index 7e24641..8283218 100644
@@ -133,9 +133,9 @@ enum perf_event_sample_format {
Quoted 9 lines

include/uapi/linux/perf_event.h enum perf_event_sample_format { · line 136

Quoted 4 lines
include/uapi/linux/perf_event.h +1 -0
      PERF_SAMPLE_REGS_USER                   = 1U << 12,
      PERF_SAMPLE_STACK_USER                  = 1U << 13,
      PERF_SAMPLE_WEIGHT                      = 1U << 14,
+     PERF_SAMPLE_DSRC                        = 1U << 15,

This conflicts with similar extensions in the Haswell patchkit, but that can be worked out by just moving some numbers (and making sure the input/output calls are still in the right place)

Yes, it all depends on which patch goes in first. No big deal.

Quoted 14 lines
Quoted 14 lines
include/uapi/linux/perf_event.h +13 -0
+union perf_mem_dsrc {
+     __u64 val;
+     struct {
+             __u64   mem_op:5,       /* type of opcode */
+                     mem_lvl:14,     /* memory hierarchy level */
+                     mem_snoop:5,    /* snoop mode */
+                     mem_lock:2,     /* lock instr */
+                     mem_dtlb:7,     /* tlb access */
+                     mem_rsvd:31;
+     };
+};
+
+/* type of opcode (load/store/prefetch,code) */
Quoted 9 lines

include/uapi/linux/perf_event.h enum perf_callchain_context { · line 637

Quoted 6 lines
include/uapi/linux/perf_event.h +6 -0
+#define PERF_MEM_OP_NA               0x01 /* not available */
+#define PERF_MEM_OP_LOAD     0x02 /* load instruction */
+#define PERF_MEM_OP_STORE    0x04 /* store instruction */
+#define PERF_MEM_OP_PFETCH   0x08 /* prefetch */
+#define PERF_MEM_OP_EXEC     0x10 /* code (execution) */
+#define PERF_MEM_OP_SHIFT    0

Do we really need the shift? it's implicit in the bitfield right?

The bitfield is provided for reference for user code. It is not used by the kernel code. We use plain u64 instead thus we need the shift. This is used for the static pebs_data_source[] table.

Quoted 11 lines
Quoted 11 lines
include/uapi/linux/perf_event.h +11 -0
+/* memory hierarchy (memory level, hit or miss) */
+#define PERF_MEM_LVL_NA              0x01  /* not available */
+#define PERF_MEM_LVL_HIT     0x02  /* hit level */
+#define PERF_MEM_LVL_MISS    0x04  /* miss level  */
+#define PERF_MEM_LVL_L1              0x08  /* L1 */
+#define PERF_MEM_LVL_LFB     0x10  /* Line Fill Buffer */
+#define PERF_MEM_LVL_L2              0x20  /* L2 hit */
+#define PERF_MEM_LVL_L3              0x40  /* L3 hit */
+#define PERF_MEM_LVL_LOC_RAM 0x80  /* Local DRAM */
+#define PERF_MEM_LVL_REM_RAM1        0x100 /* Remote DRAM (1 hop) */
+#define PERF_MEM_LVL_REM_RAM2        0x200 /* Remote DRAM (2 hops) */
Quoted 9 lines

include/uapi/linux/perf_event.h enum perf_callchain_context { · line 653

Quoted 4 lines
include/uapi/linux/perf_event.h +4 -0
+#define PERF_MEM_LVL_REM_CCE1        0x400 /* Remote Cache (1 hop) */
+#define PERF_MEM_LVL_REM_CCE2        0x800 /* Remote Cache (2 hops) */
+#define PERF_MEM_LVL_IO              0x1000 /* I/O memory */
+#define PERF_MEM_LVL_UNC     0x2000 /* Uncached memory */

I would leave some free bits here, obviously this doesn't cover all that may be possible in system architecture. Also why is this a bit mask, you can only hit one level right? So perhaps a number.

Yeah, I have been going back and forth on how to best define this to leave some room for extensions. For now, we have 31 bits left at the in the MSB part of the u64. Could either leave them there or try to commandeer some for the memory hierarchy field.

This can be a bitmask on architectures where the HW cannot determine for sure where the line came from. It may provided best effort such as missed L2 or L3.

Quoted 10 lines

include/uapi/linux/perf_event.h enum perf_callchain_context { · line 676

Quoted 8 lines
include/uapi/linux/perf_event.h +8 -0
+/* TLB access */
+#define PERF_MEM_TLB_NA              0x01 /* not available */
+#define PERF_MEM_TLB_HIT     0x02 /* hit level */
+#define PERF_MEM_TLB_MISS    0x04 /* miss level */
+#define PERF_MEM_TLB_L1              0x08 /* L1 */
+#define PERF_MEM_TLB_L2              0x10 /* L2 */
+#define PERF_MEM_TLB_WK              0x20 /* Hardware Walker*/
+#define PERF_MEM_TLB_OS              0x40 /* OS fault handler */

The current x86 PEBS-LL is a example of HW that cannot disambiguate where the TLB access actually was. It can for instance return that the access did not miss the 2nd level TLB which means: hit L1 TLB or L2 TLB. That's why you need a bitmask.

Quoted 11 lines

Same

include/uapi/linux/perf_event.h enum perf_callchain_context { · line 680

Quoted 4 lines
include/uapi/linux/perf_event.h +4 -0
+#define PERF_MEM_TLB_SHIFT   26
+
+#define PERF_MEM_S(a, s) \
+     (((u64)PERF_MEM_##a##_##s) << PERF_MEM_##a##_SHIFT)

Is that used by anything?

Yes, it is used to populate the pebs_data_source[] in perf_event_intel_ds.c

#15 lore raw

PATCH v6 8/18 perf/x86: add memory profiling via PEBS Load Latency

In reply to Stephane Eranian

This patch adds support for memory profiling using the PEBS Load Latency facility.

Load accesses are sampled by HW and the instruction address, data address, load latency, data source, tlb, locked information can be saved in the sampling buffer if using the PERF_SAMPLE_COST (for latency), PERF_SAMPLE_ADDR, PERF_SAMPLE_DSRC types.

To enable PEBS Load Latency, users have to use the model specific event:
- on NHM/WSM: MEM_INST_RETIRED:LATENCY_ABOVE_THRESHOLD
- on SNB/IVB: MEM_TRANS_RETIRED:LATENCY_ABOVE_THRESHOLD

To make things easier, this patch also exports a generic alias via sysfs: mem-loads. It export the right event encoding based on the host CPU and can be used directly by the perf tool.

Loosely based on Intel's Lin Ming patch posted on LKML in July 2011.

---
 arch/x86/include/uapi/asm/msr-index.h     |    1 +
 arch/x86/kernel/cpu/perf_event.c          |    5 +-
 arch/x86/kernel/cpu/perf_event.h          |   25 +++++-
 arch/x86/kernel/cpu/perf_event_intel.c    |   24 ++++++
 arch/x86/kernel/cpu/perf_event_intel_ds.c |  133 +++++++++++++++++++++++++++--
 5 files changed, 178 insertions(+), 10 deletions(-)
arch/x86/include/uapi/asm/msr-index.h +1 -0
index 433a59f..1031604 100644
@@ -71,6 +71,7 @@
 #define MSR_IA32_PEBS_ENABLE		0x000003f1
 #define MSR_IA32_DS_AREA		0x00000600
 #define MSR_IA32_PERF_CAPABILITIES	0x00000345
+#define MSR_PEBS_LD_LAT_THRESHOLD	0x000003f6
 
 #define MSR_MTRRfix64K_00000		0x00000250
 #define MSR_MTRRfix16K_80000		0x00000258
arch/x86/kernel/cpu/perf_event.c +4 -1
index 35b516a..1a2b337 100644
@@ -1367,7 +1367,7 @@ static __init struct attribute **merge_attr(struct attribute **a,
 	return new;
 }
 
-static ssize_t events_sysfs_show(struct device *dev, struct device_attribute *attr,
+ssize_t events_sysfs_show(struct device *dev, struct device_attribute *attr,
 			  char *page)
 {
 	struct perf_pmu_events_attr *pmu_attr = \
@@ -1498,6 +1498,9 @@ static int __init init_hw_perf_events(void)
 	x86_pmu.attr_rdpmc = 1; /* enable userspace RDPMC usage by default */
 	x86_pmu_format_group.attrs = x86_pmu.format_attrs;
 
+	if (x86_pmu.event_attrs)
+		x86_pmu_events_group.attrs = x86_pmu.event_attrs;
+
 	if (!x86_pmu.events_sysfs_show)
 		x86_pmu_events_group.attrs = &empty_attrs;
 	else
arch/x86/kernel/cpu/perf_event.h +24 -1
index 3f10cfe..3f91411 100644
@@ -46,6 +46,7 @@ enum extra_reg_type {
 	EXTRA_REG_RSP_0 = 0,	/* offcore_response_0 */
 	EXTRA_REG_RSP_1 = 1,	/* offcore_response_1 */
 	EXTRA_REG_LBR   = 2,	/* lbr_select */
+	EXTRA_REG_LDLAT = 3,	/* ld_lat_threshold */
 
 	EXTRA_REG_MAX		/* number of entries needed */
 };
@@ -61,6 +62,10 @@ struct event_constraint {
 	int	overlap;
 	int	flags;
 };
+/*
+ * struct event_constraint flags
+ */
+#define PERF_X86_EVENT_PEBS_LDLAT	0x1 /* ld+ldlat data address sampling */
 
 struct amd_nb {
 	int nb_id;  /* NorthBridge id */
@@ -233,6 +238,10 @@ struct cpu_hw_events {
 #define INTEL_UEVENT_CONSTRAINT(c, n)	\
 	EVENT_CONSTRAINT(c, n, INTEL_ARCH_EVENT_MASK)
 
+#define INTEL_PLD_CONSTRAINT(c, n)	\
+	__EVENT_CONSTRAINT(c, n, INTEL_ARCH_EVENT_MASK, \
+			   HWEIGHT(n), 0, PERF_X86_EVENT_PEBS_LDLAT)
+
 #define EVENT_CONSTRAINT_END		\
 	EVENT_CONSTRAINT(0, 0, 0)
 
@@ -262,12 +271,22 @@ struct extra_reg {
 	.msr = (ms),		\
 	.config_mask = (m),	\
 	.valid_mask = (vm),	\
-	.idx = EXTRA_REG_##i	\
+	.idx = EXTRA_REG_##i,	\
 	}
 
 #define INTEL_EVENT_EXTRA_REG(event, msr, vm, idx)	\
 	EVENT_EXTRA_REG(event, msr, ARCH_PERFMON_EVENTSEL_EVENT, vm, idx)
 
+#define INTEL_UEVENT_EXTRA_REG(event, msr, vm, idx) \
+	EVENT_EXTRA_REG(event, msr, ARCH_PERFMON_EVENTSEL_EVENT | \
+			ARCH_PERFMON_EVENTSEL_UMASK, vm, idx)
+
+#define INTEL_UEVENT_PEBS_LDLAT_EXTRA_REG(c) \
+	INTEL_UEVENT_EXTRA_REG(c, \
+			       MSR_PEBS_LD_LAT_THRESHOLD, \
+			       0xffff, \
+			       LDLAT)
+
 #define EVENT_EXTRA_END EVENT_EXTRA_REG(0, 0, 0, 0, RSP_0)
 
 union perf_capabilities {
@@ -355,6 +374,7 @@ struct x86_pmu {
 	 */
 	int		attr_rdpmc;
 	struct attribute **format_attrs;
+	struct attribute **event_attrs;
 
 	ssize_t		(*events_sysfs_show)(char *page, u64 config);
 	struct attribute **cpu_events;
@@ -659,6 +679,9 @@ int p6_pmu_init(void);
 
 int knc_pmu_init(void);
 
+ssize_t events_sysfs_show(struct device *dev, struct device_attribute *attr,
+			  char *page);
+
 #else /* CONFIG_CPU_SUP_INTEL */
 
 static inline void reserve_ds_buffers(void)
arch/x86/kernel/cpu/perf_event_intel.c +24 -0
index 57d6527..1d2a7ca 100644
@@ -81,6 +81,7 @@ static struct event_constraint intel_nehalem_event_constraints[] __read_mostly =
 static struct extra_reg intel_nehalem_extra_regs[] __read_mostly =
 {
 	INTEL_EVENT_EXTRA_REG(0xb7, MSR_OFFCORE_RSP_0, 0xffff, RSP_0),
+	INTEL_UEVENT_PEBS_LDLAT_EXTRA_REG(0x100b),
 	EVENT_EXTRA_END
 };
 
@@ -111,6 +112,7 @@ static struct extra_reg intel_westmere_extra_regs[] __read_mostly =
 {
 	INTEL_EVENT_EXTRA_REG(0xb7, MSR_OFFCORE_RSP_0, 0xffff, RSP_0),
 	INTEL_EVENT_EXTRA_REG(0xbb, MSR_OFFCORE_RSP_1, 0xffff, RSP_1),
+	INTEL_UEVENT_PEBS_LDLAT_EXTRA_REG(0x100b),
 	EVENT_EXTRA_END
 };
 
@@ -130,9 +132,23 @@ static struct event_constraint intel_gen_event_constraints[] __read_mostly =
 static struct extra_reg intel_snb_extra_regs[] __read_mostly = {
 	INTEL_EVENT_EXTRA_REG(0xb7, MSR_OFFCORE_RSP_0, 0x3fffffffffull, RSP_0),
 	INTEL_EVENT_EXTRA_REG(0xbb, MSR_OFFCORE_RSP_1, 0x3fffffffffull, RSP_1),
+	INTEL_UEVENT_PEBS_LDLAT_EXTRA_REG(0x01cd),
 	EVENT_EXTRA_END
 };
 
+EVENT_ATTR_STR(mem-loads, mem_ld_nhm, "event=0x0b,umask=0x10,ldlat=3");
+EVENT_ATTR_STR(mem-loads, mem_ld_snb, "event=0xcd,umask=0x1,ldlat=3");
+
+struct attribute *nhm_events_attrs[] = {
+	EVENT_PTR(mem_ld_nhm),
+	NULL,
+};
+
+struct attribute *snb_events_attrs[] = {
+	EVENT_PTR(mem_ld_snb),
+	NULL,
+};
+
 static u64 intel_pmu_event_map(int hw_event)
 {
 	return intel_perfmon_event_map[hw_event];
@@ -2009,6 +2025,8 @@ __init int intel_pmu_init(void)
 		x86_pmu.enable_all = intel_pmu_nhm_enable_all;
 		x86_pmu.extra_regs = intel_nehalem_extra_regs;
 
+		x86_pmu.cpu_events = nhm_events_attrs;
+
 		/* UOPS_ISSUED.STALLED_CYCLES */
 		intel_perfmon_event_map[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] =
 			X86_CONFIG(.event=0x0e, .umask=0x01, .inv=1, .cmask=1);
@@ -2049,6 +2067,8 @@ __init int intel_pmu_init(void)
 		x86_pmu.extra_regs = intel_westmere_extra_regs;
 		x86_pmu.er_flags |= ERF_HAS_RSP_1;
 
+		x86_pmu.cpu_events = nhm_events_attrs;
+
 		/* UOPS_ISSUED.STALLED_CYCLES */
 		intel_perfmon_event_map[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] =
 			X86_CONFIG(.event=0x0e, .umask=0x01, .inv=1, .cmask=1);
@@ -2077,6 +2097,8 @@ __init int intel_pmu_init(void)
 		x86_pmu.er_flags |= ERF_HAS_RSP_1;
 		x86_pmu.er_flags |= ERF_NO_HT_SHARING;
 
+		x86_pmu.cpu_events = snb_events_attrs;
+
 		/* UOPS_ISSUED.ANY,c=1,i=1 to count stall cycles */
 		intel_perfmon_event_map[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] =
 			X86_CONFIG(.event=0x0e, .umask=0x01, .inv=1, .cmask=1);
@@ -2102,6 +2124,8 @@ __init int intel_pmu_init(void)
 		x86_pmu.er_flags |= ERF_HAS_RSP_1;
 		x86_pmu.er_flags |= ERF_NO_HT_SHARING;
 
+		x86_pmu.cpu_events = snb_events_attrs;
+
 		/* UOPS_ISSUED.ANY,c=1,i=1 to count stall cycles */
 		intel_perfmon_event_map[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] =
 			X86_CONFIG(.event=0x0e, .umask=0x01, .inv=1, .cmask=1);
arch/x86/kernel/cpu/perf_event_intel_ds.c +125 -8

185 lines not on this page · raw message

Signature · 1 line
1.7.9.5
Acked-by on v6 8/18
#16 lore raw

Re: [PATCH v6 08/18] perf/x86: add memory profiling via PEBS Load Latency

arch/x86/kernel/cpu/perf_event_intel_ds.c static void __intel_pmu_pebs_event(struct perf_event *event, · line 682

Quoted 13 lines
arch/x86/kernel/cpu/perf_event_intel_ds.c +13 -0
+	sample_type = event->attr.sample_type;
+
+	/*
+	 * if PEBS-LL or PreciseStore
+	 */
+	if (fll) {
+		if (sample_type & PERF_SAMPLE_ADDR)
+			data.addr = pebs->dla;
+
+		/*
+		 * Use latency for weight (only avail with PEBS-LL)
+		 */
+		if (fll && (sample_type & PERF_SAMPLE_WEIGHT))

The extra fll tests here don't make sense because it's always true inside the if. You could remove the variable and the tests and only check once in the if.

The rest looks good to me. There will be some conflicts with the Haswell patches, but either of us can rebase.

-Andi

#17 lore raw

PATCH v6 9/18 perf/x86: export PEBS load latency threshold register to sysfs

In reply to Stephane Eranian

Make the PEBS Load Latency threshold register layout and encoding visible to user level tools.

---
 arch/x86/kernel/cpu/perf_event_intel.c |    3 +++
 1 file changed, 3 insertions(+)
arch/x86/kernel/cpu/perf_event_intel.c +3 -0
index 1d2a7ca..8eced4c 100644
@@ -1755,6 +1755,8 @@ static void intel_pmu_flush_branch_stack(void)
 
 PMU_FORMAT_ATTR(offcore_rsp, "config1:0-63");
 
+PMU_FORMAT_ATTR(ldlat, "config1:0-15");
+
 static struct attribute *intel_arch3_formats_attr[] = {
 	&format_attr_event.attr,
 	&format_attr_umask.attr,
@@ -1765,6 +1767,7 @@ static struct attribute *intel_arch3_formats_attr[] = {
 	&format_attr_cmask.attr,
 
 	&format_attr_offcore_rsp.attr, /* XXX do NHM/WSM + SNB breakout */
+	&format_attr_ldlat.attr, /* PEBS load latency */
 	NULL,
 };
 
Signature · 1 line
1.7.9.5
#18 lore raw

PATCH v6 10/18 perf/x86: add support for PEBS Precise Store

In reply to Stephane Eranian

This patch adds support for PEBS Precise Store which is available on Intel Sandy Bridge and Ivy Bridge processors.

To use Precise store, the proper PEBS event must be used: mem_trans_retired:precise_stores. For the perf tool, the generic mem-stores event exported via sysfs can be used directly.

---
 arch/x86/kernel/cpu/perf_event.h          |    5 +++
 arch/x86/kernel/cpu/perf_event_intel.c    |    2 ++
 arch/x86/kernel/cpu/perf_event_intel_ds.c |   49 +++++++++++++++++++++++++++--
 3 files changed, 54 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/perf_event.h +5 -0
index 3f91411..645b864 100644
@@ -66,6 +66,7 @@ struct event_constraint {
  * struct event_constraint flags
  */
 #define PERF_X86_EVENT_PEBS_LDLAT	0x1 /* ld+ldlat data address sampling */
+#define PERF_X86_EVENT_PEBS_ST		0x2 /* st data address sampling */
 
 struct amd_nb {
 	int nb_id;  /* NorthBridge id */
@@ -242,6 +243,10 @@ struct cpu_hw_events {
 	__EVENT_CONSTRAINT(c, n, INTEL_ARCH_EVENT_MASK, \
 			   HWEIGHT(n), 0, PERF_X86_EVENT_PEBS_LDLAT)
 
+#define INTEL_PST_CONSTRAINT(c, n)	\
+	__EVENT_CONSTRAINT(c, n, INTEL_ARCH_EVENT_MASK, \
+			  HWEIGHT(n), 0, PERF_X86_EVENT_PEBS_ST)
+
 #define EVENT_CONSTRAINT_END		\
 	EVENT_CONSTRAINT(0, 0, 0)
 
arch/x86/kernel/cpu/perf_event_intel.c +2 -0
index 8eced4c..665e26c 100644
@@ -138,6 +138,7 @@ static struct extra_reg intel_snb_extra_regs[] __read_mostly = {
 
 EVENT_ATTR_STR(mem-loads, mem_ld_nhm, "event=0x0b,umask=0x10,ldlat=3");
 EVENT_ATTR_STR(mem-loads, mem_ld_snb, "event=0xcd,umask=0x1,ldlat=3");
+EVENT_ATTR_STR(mem-stores, mem_st_snb, "event=0xcd,umask=0x2");
 
 struct attribute *nhm_events_attrs[] = {
 	EVENT_PTR(mem_ld_nhm),
@@ -146,6 +147,7 @@ struct attribute *nhm_events_attrs[] = {
 
 struct attribute *snb_events_attrs[] = {
 	EVENT_PTR(mem_ld_snb),
+	EVENT_PTR(mem_st_snb),
 	NULL,
 };
 
arch/x86/kernel/cpu/perf_event_intel_ds.c +47 -2

96 lines not on this page · raw message

Signature · 1 line
1.7.9.5
Acked-by on v6 10/18
#19 lore raw

Re: [PATCH v6 10/18] perf/x86: add support for PEBS Precise Store

Quoted 13 lines
arch/x86/kernel/cpu/perf_event_intel_ds.c +2 -1
 	u64 sample_type;
-	int fll;
+	int fll, fst;
 
 	if (!intel_pmu_save_and_restart(event))
 		return;
 
 	fll = event->hw.flags & PERF_X86_EVENT_PEBS_LDLAT;
+	fst = event->hw.flags & PERF_X86_EVENT_PEBS_ST;
 
 	perf_sample_data_init(&data, 0, event->hw.last_period);

arch/x86/kernel/cpu/perf_event_intel_ds.c static void __intel_pmu_pebs_event(struct perf_event *event, · line 718

Quoted 5 lines
arch/x86/kernel/cpu/perf_event_intel_ds.c +1 -1
@@ -672,7 +715,7 @@ static void __intel_pmu_pebs_event(struct perf_event *event,
 	/*
 	 * if PEBS-LL or PreciseStore
 	 */
-	if (fll) {
+	if (fll || fst) {

Ok I understand now why the other patch looked so strange. it makes sense and you can disregard those comments.

Looks good

-Andi

#20 lore raw

PATCH v6 11/18 perf tools: add mem access sampling core support

In reply to Stephane Eranian

This patch adds the sorting and histogram support
functions to enable profiling of memory accesses.

The following sorting orders are added:
- symbol_daddr: data address symbol (or raw address)
- dso_daddr: data address shared object
- weight: access cost
- locked: access uses locked transaction
- tlb : TLB access
- mem : memory level of the access (L1, L2, L3, RAM, ...)
- snoop: access snoop mode

---
 tools/perf/util/event.h   |    1 +
 tools/perf/util/evsel.c   |    5 +
 tools/perf/util/hist.c    |   76 ++++++++++-
 tools/perf/util/hist.h    |   13 ++
 tools/perf/util/session.c |   38 ++++++
 tools/perf/util/session.h |    4 +
 tools/perf/util/sort.c    |  325 ++++++++++++++++++++++++++++++++++++++++++++-
 tools/perf/util/sort.h    |   10 +-
 tools/perf/util/symbol.h  |    7 +
 9 files changed, 469 insertions(+), 10 deletions(-)
tools/perf/util/event.h +1 -0
index cf52977..ad66b44 100644
@@ -91,6 +91,7 @@ struct perf_sample {
 	u32 cpu;
 	u32 raw_size;
 	u64 weight;
+	u64 dsrc;
 	void *raw_data;
 	struct ip_callchain *callchain;
 	struct branch_stack *branch_stack;
tools/perf/util/evsel.c +5 -0
index e08ce12..9f334dd 100644
@@ -1063,6 +1063,11 @@ int perf_evsel__parse_sample(struct perf_evsel *evsel, union perf_event *event,
 		array++;
 	}
 
+	if (type & PERF_SAMPLE_DSRC) {
+		data->dsrc = *array;
+		array++;
+	}
+
 	return 0;
 }
 
tools/perf/util/hist.c +73 -3

130 lines not on this page · raw message

tools/perf/util/hist.h +13 -0
index 8b091a5..464d9d2 100644
@@ -49,6 +49,13 @@ enum hist_column {
 	HISTC_DSO_FROM,
 	HISTC_DSO_TO,
 	HISTC_SRCLINE,
+	HISTC_MEM_DADDR_SYMBOL,
+	HISTC_MEM_DADDR_DSO,
+	HISTC_MEM_COST,
+	HISTC_MEM_LOCKED,
+	HISTC_MEM_TLB,
+	HISTC_MEM_LVL,
+	HISTC_MEM_SNOOP,
 	HISTC_NR_COLS, /* Last entry */
 };
 
@@ -86,6 +93,12 @@ struct hist_entry *__hists__add_branch_entry(struct hists *self,
 					     struct branch_info *bi,
 					     u64 period);
 
+struct hist_entry *__hists__add_mem_entry(struct hists *self,
+					  struct addr_location *al,
+					  struct symbol *sym_parent,
+					  struct mem_info *mi,
+					  u64 period);
+
 void hists__output_resort(struct hists *self);
 void hists__output_resort_threaded(struct hists *hists);
 void hists__collapse_resort(struct hists *self);
tools/perf/util/session.c +38 -0
index 9900778..49ca66b 100644
@@ -273,6 +273,42 @@ static void ip__resolve_ams(struct machine *self, struct thread *thread,
 	ams->map = al.map;
 }
 
+static void ip__resolve_data(struct machine *self, struct thread *thread,
+			     u8 m,
+			    struct addr_map_symbol *ams,
+			    u64 addr)
+{
+	struct addr_location al;
+
+	memset(&al, 0, sizeof(al));
+
+	thread__find_addr_location(thread, self, m, MAP__VARIABLE, addr, &al,
+				   NULL);
+	ams->addr = addr;
+	ams->al_addr = al.addr;
+	ams->sym = al.sym;
+	ams->map = al.map;
+}
+
+struct mem_info *machine__resolve_mem(struct machine *self,
+				      struct thread *thr,
+				      struct perf_sample *sample,
+				      u8 cpumode)
+{
+	struct mem_info *mi;
+
+	mi = calloc(1, sizeof(struct mem_info));
+	if (!mi)
+		return NULL;
+
+	ip__resolve_ams(self, thr, &mi->iaddr, sample->ip);
+	ip__resolve_data(self, thr, cpumode, &mi->daddr, sample->addr);
+	mi->cost = sample->weight;
+	mi->dsrc.val = sample->dsrc;
+
+	return mi;
+}
+
 struct branch_info *machine__resolve_bstack(struct machine *self,
 					    struct thread *thr,
 					    struct branch_stack *bs)
@@ -1013,6 +1049,8 @@ static void dump_sample(struct perf_evsel *evsel, union perf_event *event,
 	if (sample_type & PERF_SAMPLE_ADDR)
 		printf(" ..... data: 0x%"PRIx64"\n", sample->addr);
 
+	if (sample_type & PERF_SAMPLE_DSRC)
+		printf(" . data_src: 0x%"PRIx64"\n", sample->dsrc);
 }
 
 static struct machine *
tools/perf/util/session.h +4 -0
index cea133a..f3ea026 100644
@@ -69,6 +69,10 @@ int perf_session__resolve_callchain(struct perf_session *self, struct perf_evsel
 				    struct ip_callchain *chain,
 				    struct symbol **parent);
 
+struct mem_info *machine__resolve_mem(struct machine *self,
+				      struct thread *thread,
+				      struct perf_sample *sample, u8 cpumode);
+
 bool perf_session__has_traces(struct perf_session *self, const char *msg);
 
 void mem_bswap_64(void *src, int byte_size);
tools/perf/util/sort.c +319 -6

361 lines not on this page · raw message

tools/perf/util/sort.h +9 -1
index b4e8c3b..adee5eb 100644
@@ -103,7 +103,8 @@ struct hist_entry {
 	struct rb_root		sorted_chain;
 	struct branch_info	*branch_info;
 	struct hists		*hists;
-	struct callchain_root	callchain[0];
+	struct mem_info		*mem_info;
+	struct callchain_root	callchain[0]; /* must be last member */
 };
 
 static inline bool hist_entry__has_pairs(struct hist_entry *he)
@@ -137,6 +138,13 @@ enum sort_type {
 	SORT_SYM_TO,
 	SORT_MISPREDICT,
 	SORT_SRCLINE,
+	SORT_MEM_DADDR_SYMBOL,
+	SORT_MEM_DADDR_DSO,
+	SORT_MEM_COST,
+	SORT_MEM_LOCKED,
+	SORT_MEM_TLB,
+	SORT_MEM_LVL,
+	SORT_MEM_SNOOP,
 };
 
 /*
tools/perf/util/symbol.h +7 -0
index de68f98..fdf6371 100644
@@ -152,6 +152,13 @@ struct branch_info {
 	struct branch_flags flags;
 };
 
+struct mem_info {
+	struct addr_map_symbol iaddr;
+	struct addr_map_symbol daddr;
+	u64	cost;
+	union perf_mem_dsrc dsrc;
+};
+
 struct addr_location {
 	struct thread *thread;
 	struct map    *map;
Signature · 1 line
1.7.9.5
#21 lore raw

Re: [PATCH v6 11/18] perf tools: add mem access sampling core support

Quoted 11 lines · Stephane Eranian

This patch adds the sorting and histogram support
functions to enable profiling of memory accesses.

The following sorting orders are added:
- symbol_daddr: data address symbol (or raw address)
- dso_daddr: data address shared object
- weight: access cost
- locked: access uses locked transaction
- tlb : TLB access
- mem : memory level of the access (L1, L2, L3, RAM, ...)
- snoop: access snoop mode

Do they actually sort? I found that the sort keys do not actually sort in most cases. This is mostly an issue with numbers like "weight" and of course not added by your patch.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only
#22 lore raw

PATCH v6 12/18 perf report: add support for mem access profiling

In reply to Stephane Eranian

This patch adds the --mem-mode option to perf report.

This mode requires a perf.data file created with memory access samples.

---
 tools/perf/builtin-report.c |  131 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 127 insertions(+), 4 deletions(-)
tools/perf/builtin-report.c +127 -4
index fc25100..231165a 100644
@@ -46,6 +46,7 @@ struct perf_report {
 	bool			show_full_info;
 	bool			show_threads;
 	bool			inverted_callchain;
+	bool			mem_mode;
 	struct perf_read_values	show_threads_values;
 	const char		*pretty_printing_style;
 	symbol_filter_t		annotate_init;
@@ -54,6 +55,96 @@ struct perf_report {
 	DECLARE_BITMAP(cpu_bitmap, MAX_NR_CPUS);
 };
 
+static int perf_report__add_mem_hist_entry(struct perf_tool *tool,
+					   struct addr_location *al,
+					   struct perf_sample *sample,
+					   struct perf_evsel *evsel,
+					   struct machine *machine,
+					   union perf_event *event)
+{
+	struct perf_report *rep = container_of(tool, struct perf_report, tool);
+	struct symbol *parent = NULL;
+	u8 cpumode = event->header.misc & PERF_RECORD_MISC_CPUMODE_MASK;
+	int err = 0;
+	struct hist_entry *he;
+	struct mem_info *mi, *mx;
+	uint64_t cost;
+
+	if ((sort__has_parent || symbol_conf.use_callchain)
+	    && sample->callchain) {
+		err = machine__resolve_callchain(machine, evsel, al->thread,
+						 sample, &parent);
+		if (err)
+			return err;
+	}
+
+	mi = machine__resolve_mem(machine, al->thread, sample, cpumode);
+	if (!mi)
+		return -ENOMEM;
+
+	if (rep->hide_unresolved && !al->sym)
+		return 0;
+
+	cost = mi->cost;
+	if (!cost)
+		cost = 1;
+
+	/*
+	 * The report shows the percentage of total branches captured
+	 * and not events sampled. Thus we use a pseudo period of 1.
+	 * Only in the newt browser we are doing integrated annotation,
+	 * so we don't allocated the extra space needed because the stdio
+	 * code will not use it.
+	 */
+	he = __hists__add_mem_entry(&evsel->hists, al, parent, mi,
+				    cost);
+	if (!he)
+		return -ENOMEM;
+
+	if (sort__has_sym && he->ms.sym && use_browser > 0) {
+		struct annotation *notes = symbol__annotation(he->ms.sym);
+
+		assert(evsel != NULL);
+
+		if (notes->src == NULL && symbol__alloc_hist(he->ms.sym) < 0)
+			goto out;
+
+		err = hist_entry__inc_addr_samples(he, evsel->idx, al->addr);
+		if (err)
+			goto out;
+	}
+
+	if (sort__has_sym && he->mem_info->daddr.sym && use_browser > 0) {
+		struct annotation *notes;
+
+		mx = he->mem_info;
+
+		notes = symbol__annotation(mx->daddr.sym);
+		if (!notes->src
+		    && symbol__alloc_hist(mx->daddr.sym) < 0)
+			goto out;
+
+		err = symbol__inc_addr_samples(mx->daddr.sym,
+					       mx->daddr.map,
+					       evsel->idx,
+					       mx->daddr.al_addr);
+		if (err)
+			goto out;
+	}
+
+	evsel->hists.stats.total_period += sample->period;
+	hists__inc_nr_events(&evsel->hists, PERF_RECORD_SAMPLE);
+	err = 0;
+
+	if (symbol_conf.use_callchain) {
+		err = callchain_append(he->callchain,
+				       &callchain_cursor,
+				       sample->period);
+	}
+out:
+	return err;
+}
+
 static int perf_report__add_branch_hist_entry(struct perf_tool *tool,
 					struct addr_location *al,
 					struct perf_sample *sample,
@@ -209,6 +300,12 @@ static int process_sample_event(struct perf_tool *tool,
 			pr_debug("problem adding lbr entry, skipping event\n");
 			return -1;
 		}
+	} else if (rep->mem_mode == 1) {
+		if (perf_report__add_mem_hist_entry(tool, &al, sample,
+						    evsel, machine, event)) {
+			pr_debug("problem adding mem entry, skipping event\n");
+			return -1;
+		}
 	} else {
 		if (al.map != NULL)
 			al.map->dso->hit = 1;
@@ -292,7 +389,8 @@ static void sig_handler(int sig __maybe_unused)
 	session_done = 1;
 }
 
-static size_t hists__fprintf_nr_sample_events(struct hists *self,
+static size_t hists__fprintf_nr_sample_events(struct perf_report *rep,
+					      struct hists *self,
 					      const char *evname, FILE *fp)
 {
 	size_t ret;
@@ -305,7 +403,11 @@ static size_t hists__fprintf_nr_sample_events(struct hists *self,
 	if (evname != NULL)
 		ret += fprintf(fp, " of event '%s'", evname);
 
-	ret += fprintf(fp, "\n# Event count (approx.): %" PRIu64, nr_events);
+	if (rep->mem_mode) {
+		ret += fprintf(fp, "\n# Total cost : %" PRIu64, nr_events);
+		ret += fprintf(fp, "\n# Sort order : %s", sort_order);
+	} else
+		ret += fprintf(fp, "\n# Event count (approx.): %" PRIu64, nr_events);
 	return ret + fprintf(fp, "\n#\n");
 }
 
@@ -319,7 +421,7 @@ static int perf_evlist__tty_browse_hists(struct perf_evlist *evlist,
 		struct hists *hists = &pos->hists;
 		const char *evname = perf_evsel__name(pos);
 
-		hists__fprintf_nr_sample_events(hists, evname, stdout);
+		hists__fprintf_nr_sample_events(rep, hists, evname, stdout);
 		hists__fprintf(hists, true, 0, 0, stdout);
 		fprintf(stdout, "\n\n");
 	}
@@ -596,7 +698,7 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)
 		    "Use the stdio interface"),
 	OPT_STRING('s', "sort", &sort_order, "key[,key2...]",
 		   "sort by key(s): pid, comm, dso, symbol, parent, dso_to,"
-		   " dso_from, symbol_to, symbol_from, mispredict"),
+		   " dso_from, symbol_to, symbol_from, mispredict, mem, cost, symbol_daddr, dso_daddr, tlb, snoop, locked"),
 	OPT_BOOLEAN(0, "showcpuutilization", &symbol_conf.show_cpu_utilization,
 		    "Show sample percentage for different cpu modes"),
 	OPT_STRING('p', "parent", &parent_pattern, "regex",
@@ -642,6 +744,7 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)
 		    "use branch records for histogram filling", parse_branch_mode),
 	OPT_STRING(0, "objdump", &objdump_path, "path",
 		   "objdump binary to use for disassembly and annotations"),
+	OPT_BOOLEAN(0, "mem-mode", &report.mem_mode, "mem access profile"),
 	OPT_END()
 	};
 
@@ -687,6 +790,18 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)
 				     "dso_to,symbol_to";
 
 	}
+	if (report.mem_mode) {
+		if (sort__branch_mode == 1) {
+			fprintf(stderr, "branch and mem mode incompatible\n");
+			goto error;
+		}
+		/*
+		 * if no sort_order is provided, then specify
+		 * branch-mode specific order
+		 */
+		if (sort_order == default_sort_order)
+			sort_order = "cost,mem,sym,dso,symbol_daddr,dso_daddr,snoop,tlb,locked";
+	}
 
 	if (strcmp(input_name, "-") != 0)
 		setup_browser(true);
@@ -758,6 +873,14 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)
 		sort_entry__setup_elide(&sort_sym_from, symbol_conf.sym_from_list, "sym_from", stdout);
 		sort_entry__setup_elide(&sort_sym_to, symbol_conf.sym_to_list, "sym_to", stdout);
 	} else {
+		if (report.mem_mode) {
+			sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "symbol_daddr", stdout);
+			sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "dso_daddr", stdout);
+			sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "mem", stdout);
+			sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "cost", stdout);
+			sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "tlb", stdout);
+			sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "snoop", stdout);
+		}
 		sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "dso", stdout);
 		sort_entry__setup_elide(&sort_sym, symbol_conf.sym_list, "symbol", stdout);
 	}
Signature · 1 line
1.7.9.5
#23 lore raw

PATCH v6 13/18 perf record: add support for mem access profiling

In reply to Stephane Eranian

Add the -l option to perf record to enable sampling access cost sampling.

Data address sampling is obtained via the -d option.

---
 tools/perf/builtin-record.c |    2 ++
 tools/perf/perf.h           |    1 +
 tools/perf/util/evsel.c     |    6 ++++++
 3 files changed, 9 insertions(+)
tools/perf/builtin-record.c +2 -0
index f3151d3..341f9c1 100644
@@ -1059,6 +1059,8 @@ const struct option record_options[] = {
 	OPT_CALLBACK('j', "branch-filter", &record.opts.branch_stack,
 		     "branch filter mask", "branch stack filter modes",
 		     parse_branch_stack),
+	OPT_BOOLEAN('l', "cost", &record.opts.weight,
+		    "event cost"),
 	OPT_END()
 };
 
tools/perf/perf.h +1 -0
index 2c340e7..c976553 100644
@@ -240,6 +240,7 @@ struct perf_record_opts {
 	bool	     sample_id_all_missing;
 	bool	     exclude_guest_missing;
 	bool	     period;
+	bool         weight;
 	unsigned int freq;
 	unsigned int mmap_pages;
 	unsigned int user_freq;
tools/perf/util/evsel.c +6 -0
index 9f334dd..ad0c649 100644
@@ -501,6 +501,12 @@ void perf_evsel__config(struct perf_evsel *evsel,
 		attr->sample_type	|= PERF_SAMPLE_CPU;
 	}
 
+	if (opts->weight)
+		attr->sample_type	|= PERF_SAMPLE_WEIGHT;
+
+	if (opts->sample_address)
+		attr->sample_type	|= PERF_SAMPLE_DSRC;
+
 	if (opts->no_delay) {
 		attr->watermark = 0;
 		attr->wakeup_events = 1;
Signature · 1 line
1.7.9.5
#24 lore raw

PATCH v6 14/18 perf tools: add new mem command for memory access profiling

In reply to Stephane Eranian

This new command is a wrapper on top of perf record and perf report to make it easier to configure for memory access profiling.

To record loads:

$ perf mem -t load rec .....

To record stores:

$ perf mem -t store rec .....

To get the report:

$ perf mem -t load rep
---
 tools/perf/Documentation/perf-mem.txt |   48 +++++++
 tools/perf/Makefile                   |    1 +
 tools/perf/builtin-mem.c              |  242 +++++++++++++++++++++++++++++++++
 tools/perf/builtin.h                  |    1 +
 tools/perf/command-list.txt           |    1 +
 tools/perf/perf.c                     |    1 +
 tools/perf/util/hist.c                |    1 +
 7 files changed, 295 insertions(+)
 create mode 100644 tools/perf/Documentation/perf-mem.txt
 create mode 100644 tools/perf/builtin-mem.c
tools/perf/Documentation/perf-mem.txt new file +48 -0
new file mode 100644
index 0000000..888d511
@@ -0,0 +1,48 @@
+perf-mem(1)
+===========
+
+NAME
+----
+perf-mem - Profile memory accesses
+
+SYNOPSIS
+--------
+[verse]
+'perf mem' [<options>] (record [<command>] | report)
+
+DESCRIPTION
+-----------
+"perf mem -t <TYPE> record" runs a command and gathers memory operation data
+from it, into perf.data. Perf record options are accepted and are passed through.
+
+"perf mem -t <TYPE> report" displays the result. It invokes perf report with the
+right set of options to display a memory access profile.
+
+OPTIONS
+-------
+<command>...::
+	Any command you can specify in a shell.
+
+-t::
+--type=::
+	Select the memory operation type: load or store (default: load)
+
+-D::
+--dump-raw-samples=::
+	Dump the raw decoded samples on the screen in a format that is easy to parse with
+	one sample per line.
+
+-x::
+--field-separator::
+	Specify the field separator used when dump raw samples (-D option). By default,
+	The separator is the space character.
+
+-C::
+--cpu-list::
+	Restrict dump of raw samples to those provided via this option. Note that the same
+	option can be passed in record mode. It will be interpreted the same way as perf
+	record.
+
+SEE ALSO
+--------
+linkperf:perf-record[1], linkperf:perf-report[1]
tools/perf/Makefile +1 -0
index 994f0f6..be8f4697 100644
@@ -505,6 +505,7 @@ BUILTIN_OBJS += $(OUTPUT)builtin-lock.o
 BUILTIN_OBJS += $(OUTPUT)builtin-kvm.o
 BUILTIN_OBJS += $(OUTPUT)builtin-inject.o
 BUILTIN_OBJS += $(OUTPUT)tests/builtin-test.o
+BUILTIN_OBJS += $(OUTPUT)builtin-mem.o
 
 PERFLIBS = $(LIB_FILE) $(LIBTRACEEVENT)
 
tools/perf/builtin-mem.c new file +242 -0

242 lines not on this page · raw message

tools/perf/builtin.h +1 -0
index 08143bd..b210d62 100644
@@ -36,6 +36,7 @@ extern int cmd_kvm(int argc, const char **argv, const char *prefix);
 extern int cmd_test(int argc, const char **argv, const char *prefix);
 extern int cmd_trace(int argc, const char **argv, const char *prefix);
 extern int cmd_inject(int argc, const char **argv, const char *prefix);
+extern int cmd_mem(int argc, const char **argv, const char *prefix);
 
 extern int find_scripts(char **scripts_array, char **scripts_path_array);
 #endif
tools/perf/command-list.txt +1 -0
index 3e86bbd..2c5b621 100644
@@ -24,3 +24,4 @@ perf-kmem			mainporcelain common
 perf-lock			mainporcelain common
 perf-kvm			mainporcelain common
 perf-test			mainporcelain common
+perf-mem			mainporcelain common
tools/perf/perf.c +1 -0
index 0f661fb..682340e 100644
@@ -60,6 +60,7 @@ static struct cmd_struct commands[] = {
 	{ "trace",	cmd_trace,	0 },
 #endif
 	{ "inject",	cmd_inject,	0 },
+	{ "mem",	cmd_mem,	0 },
 };
 
 struct pager_config {
tools/perf/util/hist.c +1 -0
index b5259c9..fc05b9f 100644
@@ -483,6 +483,7 @@ hist_entry__collapse(struct hist_entry *left, struct hist_entry *right)
 void hist_entry__free(struct hist_entry *he)
 {
 	free(he->branch_info);
+	free(he->mem_info);
 	free(he);
 }
 
Signature · 1 line
1.7.9.5
#25 lore raw

PATCH v6 15/18 perf: add PERF_RECORD_MISC_MMAP_DATA to RECORD_MMAP

In reply to Stephane Eranian

Type of mapping was lost and made it hard for a tool to distinguish code vs. data mmaps. Perf has the ability to distinguish the two.

Use a bit in the header->misc bitmask to keep track of the mmap type. If PERF_RECORD_MISC_MMAP_DATA is set then the mapping is not executable (!VM_EXEC). If not set, then the mapping is executable.

---
 include/uapi/linux/perf_event.h |    1 +
 kernel/events/core.c            |    3 +++
 2 files changed, 4 insertions(+)
include/uapi/linux/perf_event.h +1 -0
index 8283218..5b80f14 100644
@@ -445,6 +445,7 @@ struct perf_event_mmap_page {
 #define PERF_RECORD_MISC_GUEST_KERNEL		(4 << 0)
 #define PERF_RECORD_MISC_GUEST_USER		(5 << 0)
 
+#define PERF_RECORD_MISC_MMAP_DATA		(1 << 13)
 /*
  * Indicates that the content of PERF_SAMPLE_IP points to
  * the actual instruction that triggered the event. See also
kernel/events/core.c +3 -0
index fd4ceea..8292f15 100644
@@ -4764,6 +4764,9 @@ static void perf_event_mmap_event(struct perf_mmap_event *mmap_event)
 	mmap_event->file_name = name;
 	mmap_event->file_size = size;
 
+	if (!(vma->vm_flags & VM_EXEC))
+		mmap_event->event_id.header.misc |= PERF_RECORD_MISC_MMAP_DATA;
+
 	mmap_event->event_id.header.size = sizeof(mmap_event->event_id) + size;
 
 	rcu_read_lock();
Signature · 1 line
1.7.9.5
Acked-by on v6 15/18
#26 lore raw

Re: [PATCH v6 15/18] perf: add PERF_RECORD_MISC_MMAP_DATA to RECORD_MMAP

Quoted 10 lines · Stephane Eranian

Type of mapping was lost and made it hard for a tool to distinguish code vs. data mmaps. Perf has the ability to distinguish the two.

Use a bit in the header->misc bitmask to keep track of the mmap type. If PERF_RECORD_MISC_MMAP_DATA is set then the mapping is not executable (!VM_EXEC). If not set, then the mapping is executable.

-Andi

#27 lore raw

PATCH v6 16/18 perf tools: detect data vs. text mappings

In reply to Stephane Eranian

Leverages the PERF_RECORD_MISC_MMAP_DATA bit in the RECORD_MMAP record header. When the bit is set then the mapping type is set to MAP__VARIABLE.

---
 tools/perf/util/machine.c |   10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
tools/perf/util/machine.c +9 -1
index 1f09d05..d1c3e48 100644
@@ -379,6 +379,7 @@ int machine__process_mmap_event(struct machine *machine, union perf_event *event
 	u8 cpumode = event->header.misc & PERF_RECORD_MISC_CPUMODE_MASK;
 	struct thread *thread;
 	struct map *map;
+	enum map_type type;
 	int ret = 0;
 
 	if (dump_trace)
@@ -395,10 +396,17 @@ int machine__process_mmap_event(struct machine *machine, union perf_event *event
 	thread = machine__findnew_thread(machine, event->mmap.pid);
 	if (thread == NULL)
 		goto out_problem;
+
+	if (event->header.misc & PERF_RECORD_MISC_MMAP_DATA)
+		type = MAP__VARIABLE;
+	else
+		type = MAP__FUNCTION;
+
 	map = map__new(&machine->user_dsos, event->mmap.start,
 			event->mmap.len, event->mmap.pgoff,
 			event->mmap.pid, event->mmap.filename,
-			MAP__FUNCTION);
+			type);
+
 	if (map == NULL)
 		goto out_problem;
 
Signature · 1 line
1.7.9.5
#28 lore raw

PATCH v6 17/18 perf tools: Ignore ABS symbols when loading data maps

In reply to Stephane Eranian

From: Namhyung Kim <[email protected]>

When loading symbols in a data mapping, ABS symbols (which has a value of SHN_ABS in its st_shndx) failed at elf_getscn(). And it marks the loading as a failure so already loaded symbols cannot be fixed up.

I'm not sure what should be done. Just ignore them for now. :)

Cc Stephane Eranian <[email protected]>
---
 tools/perf/util/symbol-elf.c |    3 +++
 1 file changed, 3 insertions(+)
tools/perf/util/symbol-elf.c +3 -0
index db0cc92..00cf128 100644
@@ -719,6 +719,9 @@ int dso__load_sym(struct dso *dso, struct map *map,
 			used_opd = true;
 		}
 
+		if (sym.st_shndx == SHN_ABS)
+			continue;
+
 		sec = elf_getscn(runtime_ss->elf, sym.st_shndx);
 		if (!sec)
 			goto out_elf_end;
Signature · 1 line
1.7.9.5
#29 lore raw

PATCH v6 18/18 perf tools: Fix output of symbol_daddr offset

In reply to Stephane Eranian

From: Namhyung Kim <[email protected]>

The symbol addresses in a dso have relative offsets from the start of a mapping. So in order to ouput correct offset value from @ip, one of them should be converted.

Cc Stephane Eranian <[email protected]>
---
 tools/perf/util/sort.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/sort.c +1 -1
index 8164544..d9cb158 100644
@@ -186,7 +186,7 @@ static int _hist_entry__sym_snprintf(struct map *map, struct symbol *sym,
 		if (map->type == MAP__VARIABLE) {
 			ret += repsep_snprintf(bf + ret, size - ret, "%s", sym->name);
 			ret += repsep_snprintf(bf + ret, size - ret, "+0x%llx",
-					ip - sym->start);
+					ip - map->unmap_ip(map, sym->start));
 			ret += repsep_snprintf(bf + ret, size - ret, "%-*s",
 				       width - ret, "");
 		} else {
Signature · 1 line
1.7.9.5
#30 lore raw
In reply to Stephane Eranian
Quoted 2 lines · Stephane Eranian

This patch series had a new feature to the kernel perf_events interface and corresponding user level tool, perf.

Would be nice to merge this with the overlapping parts of Andi's Haswell series.

Thanks,

Ingo

#31 lore raw
Quoted 9 lines · Ingo Molnar
Quoted 2 lines · Stephane Eranian

This patch series had a new feature to the kernel perf_events interface and corresponding user level tool, perf.

Would be nice to merge this with the overlapping parts of Andi's Haswell series.

That's what I have now. Merged with a couple of patches from Andi's. Will post update today.

#32 lore raw
Quoted 4 lines · Stephane Eranian
Quoted 3 lines · Ingo Molnar

* Stephane Eranian <[email protected]> wrote:

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

This patch series had a new feature to the kernel perf_events interface and corresponding user level tool, perf.

Would be nice to merge this with the overlapping parts of Andi's Haswell series.

That's what I have now. Merged with a couple of patches from Andi's. Will post update today.

Ok, great!

Thanks,

Ingo