Skip to contentSkip to conversation

perf script: add script to profile and resolve physical mem type

PATCH In review [email protected] 3 participants 3 messages Jan 4, 2018 – Jan 17, 2018 perf events

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 perf script: add script to profile and resolve physical mem type

From: Kan Liang <[email protected]>

There could be different types of memory in the system. E.g normal System Memory, Persistent Memory. To understand how the workload maps to those memories, it's important to know the I/O statistics of them. Perf can collect physical addresses, but those are raw data. It still needs extra work to resolve the physical addresses. Provide a script to facilitate the physical addresses resolving and I/O statistics.

Profile with MEM_INST_RETIRED.ALL_LOADS or MEM_UOPS_RETIRED.ALL_LOADS event if any of them is available.
Look up the /proc/iomem and resolve the physical address. Provide memory type summary.

Here is an example output.
#perf script report mem-phys-addr
Event: mem_inst_retired.all_loads:P
Memory type count percentage
---------------------------------------- ----------- -----------

System RAM                                        74        53.2%
Persistent Memory                                 55        39.6%

N/A

---

Changes since V2:
- Apply the new license rules.
- Add comments for globals

Changes since V1:
- Do not mix DLA and Load Latency. Do not compare the loads and stores. Only profile the loads.
- Use event name to replace the RAW event

 tools/perf/scripts/python/bin/mem-phys-addr-record | 19 +++++
 tools/perf/scripts/python/bin/mem-phys-addr-report |  3 +
 tools/perf/scripts/python/mem-phys-addr.py         | 95 ++++++++++++++++++++++
 .../util/scripting-engines/trace-event-python.c    |  2 +
 4 files changed, 119 insertions(+)
 create mode 100644 tools/perf/scripts/python/bin/mem-phys-addr-record
 create mode 100644 tools/perf/scripts/python/bin/mem-phys-addr-report
 create mode 100644 tools/perf/scripts/python/mem-phys-addr.py
tools/perf/scripts/python/bin/mem-phys-addr-record new file +19 -0
new file mode 100644
index 0000000..5a87512
@@ -0,0 +1,19 @@
+#!/bin/bash
+
+#
+# Profiling physical memory by all retired load instructions/uops event
+# MEM_INST_RETIRED.ALL_LOADS or MEM_UOPS_RETIRED.ALL_LOADS
+#
+
+load=`perf list | grep mem_inst_retired.all_loads`
+if [ -z "$load" ]; then
+	load=`perf list | grep mem_uops_retired.all_loads`
+fi
+if [ -z "$load" ]; then
+	echo "There is no event to count all retired load instructions/uops."
+	exit 1
+fi
+
+arg=$(echo $load | tr -d ' ')
+arg="$arg:P"
+perf record --phys-data -e $arg $@
tools/perf/scripts/python/bin/mem-phys-addr-report new file +3 -0
new file mode 100644
index 0000000..3f2b847
@@ -0,0 +1,3 @@
+#!/bin/bash
+# description: resolve physical address samples
+perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/mem-phys-addr.py
tools/perf/scripts/python/mem-phys-addr.py new file +95 -0

95 lines not on this page · raw message

tools/perf/util/scripting-engines/trace-event-python.c +2 -0
index c1848b5..ea07088 100644
@@ -499,6 +499,8 @@ static PyObject *get_perf_sample_dict(struct perf_sample *sample,
 			PyLong_FromUnsignedLongLong(sample->time));
 	pydict_set_item_string_decref(dict_sample, "period",
 			PyLong_FromUnsignedLongLong(sample->period));
+	pydict_set_item_string_decref(dict_sample, "phys_addr",
+			PyLong_FromUnsignedLongLong(sample->phys_addr));
 	set_sample_read_in_dict(dict_sample, sample, evsel);
 	pydict_set_item_string_decref(dict, "sample", dict_sample);
 
Signature · 1 line
2.7.4
Reviewed-by on v3 1/1
#1 lore raw
Quoted 9 lines · On Thu, Jan 04, 2018 at 12:59:55PM -0800, [email protected] wrote:

From: Kan Liang <[email protected]>

There could be different types of memory in the system. E.g normal System Memory, Persistent Memory. To understand how the workload maps to those memories, it's important to know the I/O statistics of them. Perf can collect physical addresses, but those are raw data. It still needs extra work to resolve the physical addresses. Provide a script to facilitate the physical addresses resolving and I/O statistics.

-Andi

#2 lore raw

[tip:perf/core] perf script python: Add script to profile and resolve physical mem type

In reply to [email protected]
perf script python: Add script to profile and resolve physical mem type
Commit-ID:  41013f0c095980775e0746272873891ca7c28fb1
Gitweb:     https://git.kernel.org/tip/41013f0c095980775e0746272873891ca7c28fb1
Author:     Kan Liang <[email protected]>
AuthorDate: Thu, 4 Jan 2018 12:59:55 -0800
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Fri, 12 Jan 2018 11:06:57 -0300

perf script python: Add script to profile and resolve physical mem type

There could be different types of memory in the system. E.g normal System Memory, Persistent Memory. To understand how the workload maps to those memories, it's important to know the I/O statistics of them. Perf can collect physical addresses, but those are raw data. It still needs extra work to resolve the physical addresses. Provide a script to facilitate the physical addresses resolving and I/O statistics.

Profile with MEM_INST_RETIRED.ALL_LOADS or MEM_UOPS_RETIRED.ALL_LOADS event if any of them is available.

Look up the /proc/iomem and resolve the physical address. Provide memory type summary.

Here is an example output:

  # perf script report mem-phys-addr
  Event: mem_inst_retired.all_loads:P
  Memory type                                    count   percentage
  ----------------------------------------  -----------  -----------
  System RAM                                        74        53.2%
  Persistent Memory                                 55        39.6%
  N/A

---

Changes since V2:
- Apply the new license rules.
- Add comments for globals

Changes since V1:
- Do not mix DLA and Load Latency. Do not compare the loads and stores. Only profile the loads.
- Use event name to replace the RAW event

---
 tools/perf/scripts/python/bin/mem-phys-addr-record | 19 +++++
 tools/perf/scripts/python/bin/mem-phys-addr-report |  3 +
 tools/perf/scripts/python/mem-phys-addr.py         | 95 ++++++++++++++++++++++
 .../util/scripting-engines/trace-event-python.c    |  2 +
 4 files changed, 119 insertions(+)
tools/perf/scripts/python/bin/mem-phys-addr-record new file +19 -0
new file mode 100644
index 0000000..5a87512
@@ -0,0 +1,19 @@
+#!/bin/bash
+
+#
+# Profiling physical memory by all retired load instructions/uops event
+# MEM_INST_RETIRED.ALL_LOADS or MEM_UOPS_RETIRED.ALL_LOADS
+#
+
+load=`perf list | grep mem_inst_retired.all_loads`
+if [ -z "$load" ]; then
+	load=`perf list | grep mem_uops_retired.all_loads`
+fi
+if [ -z "$load" ]; then
+	echo "There is no event to count all retired load instructions/uops."
+	exit 1
+fi
+
+arg=$(echo $load | tr -d ' ')
+arg="$arg:P"
+perf record --phys-data -e $arg $@
tools/perf/scripts/python/bin/mem-phys-addr-report new file +3 -0
new file mode 100644
index 0000000..3f2b847
@@ -0,0 +1,3 @@
+#!/bin/bash
+# description: resolve physical address samples
+perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/mem-phys-addr.py
tools/perf/scripts/python/mem-phys-addr.py new file +95 -0

95 lines not on this page · raw message

tools/perf/util/scripting-engines/trace-event-python.c +2 -0
index c1848b5..ea07088 100644
@@ -499,6 +499,8 @@ static PyObject *get_perf_sample_dict(struct perf_sample *sample,
 			PyLong_FromUnsignedLongLong(sample->time));
 	pydict_set_item_string_decref(dict_sample, "period",
 			PyLong_FromUnsignedLongLong(sample->period));
+	pydict_set_item_string_decref(dict_sample, "phys_addr",
+			PyLong_FromUnsignedLongLong(sample->phys_addr));
 	set_sample_read_in_dict(dict_sample, sample, evsel);
 	pydict_set_item_string_decref(dict, "sample", dict_sample);