Skip to contentSkip to conversation

perf: add support for profiling jitted code

PATCH Superseded Stephane Eranian 5 participants 13 messages Mar 24, 2015 – Mar 26, 2015

Subsystems (2)
Documentation perf events

v4 of 8 Superseded Latest is v8 →

lore
Share & tools
lore

Stephane Eranian

author
#0 lore raw

v4 cover perf: add support for profiling jitted code

This patch series extends perf record/report/annotate to enable profiling of jitted (just-in-time compiled) code. The current perf tool provides very limited support for profiling jitted code for some runtime environments. But the support is experimental and cannot be used in complex environments. It relies on files in /tmp, for instance. It does not support annotate mode or rejitted code.

This patch series adds a better way of profiling jitted code with the following advantages:
- support any jitted code environment (some with modifications)
- support Java runtime with JVMTI interface with no modifications
- provides a portable JVMTI agent library
- known to support V8 runtime
- known to support DART runtime
- supports code rejitting and movements
- no files in /tmp
- meta-data file is unique to each run
- no changes to perf report/annotate
- support per-thread and system-wide profiling
- support monitoring of multiple simultaneous Jit runtimes

The support is based on cooperation with the runtime. For Java runtimes, supporting the JVMTI interface, there is no change necessary. For other runtimes, modifications are necessary to emit the meta-data necessary to support symbolization and annotation of the samples. Those modifications are fairly straighforward and already tested on V8 and DART.

The jit environment emits a binary dump file which contains the jitted code (in raw format) and meta-data describing the mapping of functions. The binary format is documented in the jitdump.h header file. It is adapted from the OProfile jitdump format.

To enable synchronization of the runtime MMAPs with those recorded by the kernel on behalf of the perf tool, the runtime needs to timestamp any record in the dump file using the same time source. This third version is using Pawel Moll's CLOCK_MONOTONIC patch posted

	https://lkml.org/lkml/2015/1/21/711

The patch is packaged in this series for convenience. Without this patch installed, records emitted by the runtime cannot be properly synchronized (inserted in the flow of MMAPS) and symbolization is not possible.

The current support only works when the runtime is monitored from start to finish: perf record java --agentpath:libpfmjvmti.so my_class.

Once the run is completed, the jitdump file needs to be injected into the perf.data file. This is accomplished by using the perf inject command. This will also generate an ELF image for each jitted function. The inject MMAP records will point to those ELF images. The reasoning behind using ELF images is that it makes processing for perf report and annotate automatic and transparent. It also makes it easier to package and analyze on a remote machine.

The reporting is unchanged, simply invoke perf report or perf annotate on the modified perf.data file. The jitted code will appear symbolized and the assembly view will display the instruction level profile!

As an added bonus, the series includes support for demangling function signature from OpenJDK.

The current patch series does not include support for source line information. Therefore, the source code cannot yet be displayed in perf annotate. This will come in a later update.

Furthermore, we believe there is a way to skip the perf inject phase and have perf report/annotate directly inject the MMAP records on the fly during processing of the perf.data file. Perf report would also generate the ELF files if necessary. Such optimization, would make using this extension seamless in system-wide mode and larger environments. This will be added in a later update as well.

In V2, we have switchde to Pawell Moll and David Ahern posix clock kernel module instead. We have dropped the patch which modified the arguments to map_init() because the change was not used. We are not printing the return type of Java methods anymore and have made the Java demangler a separate module. We also rebased to 3.19.0+ from tip.git.

In V3, we switched to Pawel Moll's CLOCK_MONOTONIC perf clock patches. This patch switch perf_events from sched_clock to CLOCK_MONOTONIC, a clock source which is available to users.

In V4, we rebased to 4.0-rc5. We also simplified the process by getting rid of the requirement to pass the jitdump file name to perf inject. Now, perf injects automtically detects if jitdumps were generated and it merges the relevant meta-data. This is accomplished by having the jit runtime mmap the jitdump file for the purpose of creating a MMAP record in the perf.data file. That MMAP contains all the info to locate the jitdump file and generate the ELF images for jitted functions.

To use the new feature:
- compile and install Pawel's CLOCK_MONOTONIC patch:
- compile the patched kernel
- install the patched (and modules)
- reboot the machine to this patched kernel
- check that you have /proc/sys/kerne/perf_sample_time_clk_id
- compile perf
- cd tools/perf/jvmti; make; install wherever is appropriate

Example using openJDK:

   $ perf record java -agentpath:libjvmti.so my_class
   $ perf inject -i perf.data -jit -o perf.data.jitted
   $ perf report -i perf.data.jitted

Thanks to all the contributors and testers.

Enjoy,

Pawel Moll (1):
perf: Use monotonic clock as a source for timestamps

Stephane Eranian (3):
perf tools: add Java demangling support
perf inject: add jitdump mmap injection support
perf tools: add JVMTI agent library

 Documentation/kernel-parameters.txt      |   9 +
 kernel/events/core.c                     |  43 ++-
 tools/build/Makefile.feature             |   2 +
 tools/build/feature/Makefile             |   4 +
 tools/build/feature/test-libcrypto.c     |   9 +
 tools/perf/Documentation/perf-inject.txt |   6 +
 tools/perf/builtin-inject.c              |  60 +++-
 tools/perf/config/Makefile               |  11 +
 tools/perf/jvmti/Makefile                |  70 ++++
 tools/perf/jvmti/jvmti_agent.c           | 394 +++++++++++++++++++++
 tools/perf/jvmti/jvmti_agent.h           |  23 ++
 tools/perf/jvmti/libjvmti.c              | 149 ++++++++
 tools/perf/util/Build                    |   3 +
 tools/perf/util/demangle-java.c          | 199 +++++++++++
 tools/perf/util/demangle-java.h          |  10 +
 tools/perf/util/genelf.c                 | 479 +++++++++++++++++++++++++
 tools/perf/util/genelf.h                 |   6 +
 tools/perf/util/jitdump.c                | 588 +++++++++++++++++++++++++++++++
 tools/perf/util/jitdump.h                |  92 +++++
 tools/perf/util/symbol-elf.c             |   3 +
 20 files changed, 2146 insertions(+), 14 deletions(-)
 create mode 100644 tools/build/feature/test-libcrypto.c
 create mode 100644 tools/perf/jvmti/Makefile
 create mode 100644 tools/perf/jvmti/jvmti_agent.c
 create mode 100644 tools/perf/jvmti/jvmti_agent.h
 create mode 100644 tools/perf/jvmti/libjvmti.c
 create mode 100644 tools/perf/util/demangle-java.c
 create mode 100644 tools/perf/util/demangle-java.h
 create mode 100644 tools/perf/util/genelf.c
 create mode 100644 tools/perf/util/genelf.h
 create mode 100644 tools/perf/util/jitdump.c
 create mode 100644 tools/perf/util/jitdump.h
Signature · 1 line
1.9.1
author
#1 lore raw

PATCH v4 1/4 perf: Use monotonic clock as a source for timestamps

From: Pawel Moll <[email protected]>

Until now, perf framework never defined the meaning of the timestamps captured as PERF_SAMPLE_TIME sample type. The values were obtaining from local (sched) clock, which is unavailable in userspace. This made it impossible to correlate perf data with any other events. Other tracing solutions have the source configurable (ftrace) or just share a common time domain between kernel and userspace (LTTng).

Follow the trend by using monotonic clock, which is readily available as POSIX CLOCK_MONOTONIC.

Also add a sysctl "perf_sample_time_clk_id" attribute (usually available as "/proc/sys/kernel/perf_sample_time_clk_id") which can be used by the user to obtain the clk_id to be used with POSIX clock API (eg. clock_gettime()) to obtain a time value comparable with perf samples.

Old behaviour can be restored by using "perf_use_local_clock" kernel parameter.

Signed-off-by Pawel Moll <[email protected]>
---
 Documentation/kernel-parameters.txt |  9 ++++++++
 kernel/events/core.c                | 43 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 51 insertions(+), 1 deletion(-)
Documentation/kernel-parameters.txt +9 -0
index bfcb1a6..e80da0b 100644
@@ -91,6 +91,7 @@ the beginning of each description states the restrictions within which a
 	NUMA	NUMA support is enabled.
 	NFS	Appropriate NFS support is enabled.
 	OSS	OSS sound support is enabled.
+	PERF	Performance events and counters support is enabled.
 	PV_OPS	A paravirtualized kernel is enabled.
 	PARIDE	The ParIDE (parallel port IDE) subsystem is enabled.
 	PARISC	The PA-RISC architecture is enabled.

@@ -2813,6 +2814,14 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
 			allocator.  This parameter is primarily	for debugging
 			and performance comparison.
 
+	perf_use_local_clock
+			[PERF]
+			Use local_clock() as a source for perf timestamps
+			generation. This was be the default behaviour and
+			this parameter can be used to maintain backward
+			compatibility or on older hardware with expensive
+			monotonic clock source.
+
 	pf.		[PARIDE]
 			See Documentation/blockdev/paride.txt.
kernel/events/core.c +42 -1

63 lines not on this page · raw message

Signature · 1 line
1.9.1
author
#2 lore raw

PATCH v4 2/4 perf tools: add Java demangling support

In reply to Stephane Eranian

Add Java function descriptor demangling support.
Something bfd cannot do.

Use the JAVA_DEMANGLE_NORET flag to avoid decoding the return type of functions.

Signed-off-by Stephane Eranian <[email protected]>
---
 tools/perf/util/Build           |   1 +
 tools/perf/util/demangle-java.c | 199 ++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/demangle-java.h |  10 ++
 tools/perf/util/symbol-elf.c    |   3 +
 4 files changed, 213 insertions(+)
 create mode 100644 tools/perf/util/demangle-java.c
 create mode 100644 tools/perf/util/demangle-java.h
tools/perf/util/Build +1 -0
index 797490a..9bff65e 100644
@@ -74,6 +74,7 @@ libperf-y += data.o
 libperf-$(CONFIG_X86) += tsc.o
 libperf-y += cloexec.o
 libperf-y += thread-stack.o
+libperf-y += demangle-java.o
 
 libperf-$(CONFIG_LIBELF) += symbol-elf.o
 libperf-$(CONFIG_LIBELF) += probe-event.o
tools/perf/util/demangle-java.c new file +199 -0

199 lines not on this page · raw message

tools/perf/util/demangle-java.h new file +10 -0
new file mode 100644
index 0000000..a981c1f
@@ -0,0 +1,10 @@
+#ifndef __PERF_DEMANGLE_JAVA
+#define __PERF_DEMANGLE_JAVA 1
+/*
+ * demangle function flags
+ */
+#define JAVA_DEMANGLE_NORET	0x1 /* do not process return type */
+
+char * java_demangle_sym(const char *str, int flags);
+
+#endif /* __PERF_DEMANGLE_JAVA */
tools/perf/util/symbol-elf.c +3 -0
index 476268c..2174bf4 100644
@@ -6,6 +6,7 @@
 #include <inttypes.h>
 
 #include "symbol.h"
+#include "demangle-java.h"
 #include "machine.h"
 #include "vdso.h"
 #include <symbol/kallsyms.h>

@@ -1046,6 +1047,8 @@ int dso__load_sym(struct dso *dso, struct map *map,
 				demangle_flags = DMGL_PARAMS | DMGL_ANSI;
 
 			demangled = bfd_demangle(NULL, elf_name, demangle_flags);
+			if (demangled == NULL)
+				demangled = java_demangle_sym(elf_name, JAVA_DEMANGLE_NORET);
 			if (demangled != NULL)
 				elf_name = demangled;
 		}
Signature · 1 line
1.9.1
#3 lore raw

Re: [PATCH v4 2/4] perf tools: add Java demangling support

Quoted 6 lines · Stephane Eranian

Add Java function descriptor demangling support.
Something bfd cannot do.

Use the JAVA_DEMANGLE_NORET flag to avoid decoding the return type of functions.

Quoted 7 lines
Signed-off-by Stephane Eranian <[email protected]>
---
 tools/perf/util/Build           |   1 +
 tools/perf/util/demangle-java.c | 199
++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/demangle-java.h |  10 ++
 tools/perf/util/symbol-elf.c    |   3 +

Stephane:

Your patch makes a change to tools/perf/util/Build. I don't find that file in the linux kernel source tree for 4.0-rc5. I built the tool just to see if that file gets created by the build but still did not find it. I am wondering where the file comes from?

Carl Love

author
#4 lore raw

Re: [PATCH v4 2/4] perf tools: add Java demangling support

Quoted 15 lines · Carl E. Love
Quoted 6 lines · Stephane Eranian

Add Java function descriptor demangling support.
Something bfd cannot do.

Use the JAVA_DEMANGLE_NORET flag to avoid decoding the return type of functions.

Quoted 7 lines
Signed-off-by Stephane Eranian <[email protected]>
---
 tools/perf/util/Build           |   1 +
 tools/perf/util/demangle-java.c | 199
++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/demangle-java.h |  10 ++
 tools/perf/util/symbol-elf.c    |   3 +
Quoted 7 lines

Stephane:

Your patch makes a change to tools/perf/util/Build. I don't find that file in the linux kernel source tree for 4.0-rc5. I built the tool just to see if that file gets created by the build but still did not find it. I am wondering where the file comes from?

Im on tip.git and at commit:
f05bf44 Merge branch 'perf/core'

I had to do this to get the features-tests to work. This setup keeps changing on me.

Quoted 2 lines

Carl Love

#5 lore raw

Re: [PATCH v4 2/4] perf tools: add Java demangling support

Quoted 5 lines · Stephane Eranian

Im on tip.git and at commit:
f05bf44 Merge branch 'perf/core'

I had to do this to get the features-tests to work. This setup keeps changing on me.

perf patches go through Arnaldo, so new patch sets like this should be against acme's tree, perf/core branch.

David

#6 lore raw

Re: [PATCH v4 2/4] perf tools: add Java demangling support

In reply to Stephane Eranian

Em Tue, Mar 24, 2015 at 11:25:31PM +0100, Stephane Eranian escreveu:

<SNIP>

Quoted 15 lines
tools/perf/util/demangle-java.c new file +15 -0
+static char *
+__demangle_java_sym(const char *str, const char *end, char *buf, int maxlen, int mode)
+{
+	int rlen = 0;
+	int array = 0;
+	int narg = 0;
+	const char *q;
+
+	if (!end)
+		end = str + strlen(str);
+
+	for (q = str; q != end; q++) {
+
+		if (rlen == (maxlen - 1))
+			break;

tools/perf/util/demangle-java.c line 65

Quoted 10 lines
tools/perf/util/demangle-java.c new file +10 -0
+
+		switch (*q) {
+		case 'L':
+			if (mode == MODE_PREFIX || mode == MODE_CTYPE) {
+				if (mode == MODE_CTYPE) {
+					if (narg)
+						rlen += snprintf(buf+rlen, maxlen - rlen, ", ");
+					narg++;
+				}
+				rlen += snprintf(buf+rlen, maxlen - rlen, "class ");

I think you should replace all these snprintf with scnprintf to avoid that trap described in its man page:

-----
Return value:

The functions snprintf() and vsnprintf() do not write more than
size bytes (including the terminating null byte ('\0')). If the output was truncated due to this limit then the return value is the number of characters (excluding the terminating null byte) which would have been written to the final string if enough space had been available. Thus, a return value of size or more means that the output was truncated. (See also below under NOTES.)
-----

I.e. it will not return the number of bytes _printed_, which is what you expect, no? I.e. you don't seem to be checking for truncation.

Nit:

please have spaces separating operators consistently, i.e. sometimes you use 'a+b', sometimes 'a + b', please use consistently 'a + b'.

- Arnaldo

Quoted 173 lines
tools/perf/util/demangle-java.c new file +134 -0
+				if (mode == MODE_PREFIX)
+					mode = MODE_CLASS;
+			} else
+				buf[rlen++] = *q;
+			break;
+		case 'B':
+		case 'C':
+		case 'D':
+		case 'F':
+		case 'I':
+		case 'J':
+		case 'S':
+		case 'Z':
+			if (mode == MODE_TYPE) {
+				if (narg)
+					rlen += snprintf(buf+rlen, maxlen - rlen, ", ");
+				rlen += snprintf(buf+rlen, maxlen - rlen, "%s", base_types[*q - 'A']);
+				while(array--)
+					rlen += snprintf(buf+rlen, maxlen - rlen, "[]");
+				array = 0;
+				narg++;
+			} else
+				buf[rlen++] = *q;
+			break;
+		case 'V':
+			if (mode == MODE_TYPE) {
+				rlen += snprintf(buf+rlen, maxlen - rlen, "void");
+				while(array--)
+					rlen += snprintf(buf+rlen, maxlen - rlen, "[]");
+				array = 0;
+			} else
+				buf[rlen++] = *q;
+			break;
+		case '[':
+			if (mode != MODE_TYPE)
+				goto error;
+			array++;
+			break;
+		case '(':
+			if (mode != MODE_FUNC)
+				goto error;
+			buf[rlen++] = *q;
+			mode = MODE_TYPE;
+			break;
+		case ')':
+			if (mode != MODE_TYPE)
+				goto error;
+			buf[rlen++] = *q;
+			narg = 0;
+			break;
+		case ';':
+			if (mode != MODE_CLASS && mode != MODE_CTYPE)
+				goto error;
+			/* safe because at least one other char to process */
+			if (isalpha(*(q+1)))
+				rlen += snprintf(buf+rlen, maxlen - rlen, ".");
+			if (mode == MODE_CLASS)
+				mode = MODE_FUNC;
+			else if (mode == MODE_CTYPE)
+				mode = MODE_TYPE;
+			break;
+		case '/':
+			if (mode != MODE_CLASS && mode != MODE_CTYPE)
+				goto error;
+			rlen += snprintf(buf+rlen, maxlen - rlen, ".");
+			break;
+		default :
+			buf[rlen++] = *q;
+		}
+	}
+	buf[rlen] = '\0';
+	return buf;
+error:
+	return NULL;
+}
+
+/*
+ * Demangle Java function signature (openJDK, not GCJ)
+ * input:
+ * 	str: string to parse. String is not modified
+ *    flags: comobination of JAVA_DEMANGLE_* flags to modify demangling
+ * return:
+ *	if input can be demangled, then a newly allocated string is returned.
+ *	if input cannot be demangled, then NULL is returned
+ *
+ * Note: caller is responsible for freeing demangled string
+ */
+char *
+java_demangle_sym(const char *str, int flags)
+{
+	char *buf, *ptr;
+	char *p;
+	size_t len, l1 = 0;
+
+	if (!str)
+		return NULL;
+
+	/* find start of retunr type */
+	p = strrchr(str, ')');
+	if (!p)
+		return NULL;
+
+	/*
+	 * expansion factor estimated to 3x
+	 */
+	len = strlen(str) * 3 + 1;
+	buf = malloc(len);
+	if (!buf)
+		return NULL;
+
+	buf[0] = '\0';
+	if (!(flags & JAVA_DEMANGLE_NORET)) {
+		/*
+		 * get return type first
+		 */
+		ptr = __demangle_java_sym(p+1, NULL, buf, len, MODE_TYPE);
+		if (!ptr)
+			goto error;
+
+		/* add space between return type and function prototype */
+		l1 = strlen(buf);
+		buf[l1++] = ' ';
+	}
+
+	/* process function up to return type */
+	ptr = __demangle_java_sym(str, p + 1, buf + l1, len - l1, MODE_PREFIX);
+	if (!ptr)
+		goto error;
+
+	return buf;
+error:
+	free(buf);
+	return NULL;
+}
tools/perf/util/demangle-java.h new file +10 -0
new file mode 100644
index 0000000..a981c1f
@@ -0,0 +1,10 @@
+#ifndef __PERF_DEMANGLE_JAVA
+#define __PERF_DEMANGLE_JAVA 1
+/*
+ * demangle function flags
+ */
+#define JAVA_DEMANGLE_NORET	0x1 /* do not process return type */
+
+char * java_demangle_sym(const char *str, int flags);
+
+#endif /* __PERF_DEMANGLE_JAVA */
tools/perf/util/symbol-elf.c +3 -0
index 476268c..2174bf4 100644
@@ -6,6 +6,7 @@
 #include <inttypes.h>
 
 #include "symbol.h"
+#include "demangle-java.h"
 #include "machine.h"
 #include "vdso.h"
 #include <symbol/kallsyms.h>

@@ -1046,6 +1047,8 @@ int dso__load_sym(struct dso *dso, struct map *map,
 				demangle_flags = DMGL_PARAMS | DMGL_ANSI;
 
 			demangled = bfd_demangle(NULL, elf_name, demangle_flags);
+			if (demangled == NULL)
+				demangled = java_demangle_sym(elf_name, JAVA_DEMANGLE_NORET);
 			if (demangled != NULL)
 				elf_name = demangled;
 		}
Signature · 1 line
1.9.1
author
#7 lore raw

PATCH v4 3/4 perf inject: add jitdump mmap injection support

In reply to Stephane Eranian

This patch adds a --jit option to perf inject.

This options injects MMAP records into the perf.data file to cover the jitted code mmaps. It also emits ELF images for each function in the jidump file. Those images are created where the jitdump file is. The MMAP records point to that location as well.

Typical flow:

$ java -agentpath:libpjvmti.so java_class
$ perf inject --jit -i perf.data -o perf.data.jitted
$ perf report -i perf.data.jitted

Note that jitdump.h support is not limited to Java, it works with any jitted environment modified to emit the jitdump file format, include those where code can be jitted multiple times and moved around.

The jitdump.h format is adapted from the Oprofile project.

The genelf.c (ELF binary generation) depends on MD5 hash encoding for the buildid. To enable this, libssl-dev must be installed. If not, then genelf.c defaults to using urandom to generate the buildid, which is not ideal. The Makefile auto-detects the presence on libssl-dev.

This version mmaps the jitdump file to create a marker MMAP record in the perf.data file. The marker is used to detect jitdump and cause perf inject to inject the jitted mmaps and generate ELF images for jitted functions.

Signed-off-by Stephane Eranian <[email protected]>
---
 tools/build/Makefile.feature             |   2 +
 tools/build/feature/Makefile             |   4 +
 tools/build/feature/test-libcrypto.c     |   9 +
 tools/perf/Documentation/perf-inject.txt |   6 +
 tools/perf/builtin-inject.c              |  60 +++-
 tools/perf/config/Makefile               |  11 +
 tools/perf/util/Build                    |   2 +
 tools/perf/util/genelf.c                 | 479 +++++++++++++++++++++++++
 tools/perf/util/genelf.h                 |   6 +
 tools/perf/util/jitdump.c                | 588 +++++++++++++++++++++++++++++++
 tools/perf/util/jitdump.h                |  92 +++++
 11 files changed, 1246 insertions(+), 13 deletions(-)
 create mode 100644 tools/build/feature/test-libcrypto.c
 create mode 100644 tools/perf/util/genelf.c
 create mode 100644 tools/perf/util/genelf.h
 create mode 100644 tools/perf/util/jitdump.c
 create mode 100644 tools/perf/util/jitdump.h
tools/build/Makefile.feature +2 -0
index 3a0b0ca..8b468a3 100644
@@ -45,6 +45,7 @@ FEATURE_TESTS =			\
 	libpython			\
 	libpython-version		\
 	libslang			\
+	libcrypto			\
 	libunwind			\
 	pthread-attr-setaffinity-np	\
 	stackprotector-all		\

@@ -64,6 +65,7 @@ FEATURE_DISPLAY =			\
 	libperl				\
 	libpython			\
 	libslang			\
+	libcrypto			\
 	libunwind			\
 	libdw-dwarf-unwind		\
 	zlib				\
tools/build/feature/Makefile +4 -0
index 463ed8f..ed86700 100644
@@ -23,6 +23,7 @@ FILES=					\
 	test-libpython.bin		\
 	test-libpython-version.bin	\
 	test-libslang.bin		\
+	test-libcrypto.bin		\
 	test-libunwind.bin		\
 	test-libunwind-debug-frame.bin	\
 	test-pthread-attr-setaffinity-np.bin	\

@@ -93,6 +94,9 @@ __BUILD = $(CC) $(CFLAGS) -Wall -Werror -o $(OUTPUT)$@ $(patsubst %.bin,%.c,$@)
 test-libslang.bin:
 	$(BUILD) -I/usr/include/slang -lslang
 
+test-libcrypto.bin:
+	$(BUILD) -lcrypto
+
 test-gtk2.bin:
 	$(BUILD) $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null)
tools/build/feature/test-libcrypto.c new file +9 -0
new file mode 100644
index 0000000..8580c40
@@ -0,0 +1,9 @@
+#include <openssl/md5.h>
+
+int main(void)
+{
+        MD5_CTX context;
+
+        MD5_Init(&context);
+	return 0;
+}
tools/perf/Documentation/perf-inject.txt +6 -0
index dc7442c..e9790dc 100644
@@ -44,6 +44,12 @@ OPTIONS
 --kallsyms=<file>::
 	kallsyms pathname
 
+-j::
+--jit::
+	Process jitdump files by injecting the mmap records corresponding to jitted
+	functions. This option also generates the ELF images for each jitted function
+	found in the jitdumps files captured in the input perf.data file. Use this option
+	if you are monitoring environment using JIT runtimes, such as Java, DART or V8.
 SEE ALSO
 --------
 linkperf:perf-record[1], linkperf:perf-report[1], linkperf:perf-archive[1]
tools/perf/builtin-inject.c +47 -13

102 lines not on this page · raw message

tools/perf/config/Makefile +11 -0
index cd121df..e3cf5e4 100644
@@ -366,6 +366,17 @@ ifndef NO_LIBAUDIT
   endif
 endif
 
+ifndef NO_LIBCRYPTO
+  ifneq ($(feature-libcrypto), 1)
+    msg := $(warning No libcrypto.h found, disables jitted code injection, please install libssl-devel or libssl-dev);
+    NO_LIBCRYPTO := 1
+  else
+    CFLAGS += -DHAVE_LIBCRYPTO_SUPPORT
+    EXTLIBS += -lcrypto
+    $(call detected,CONFIG_CRYPTO)
+  endif
+endif
+
 ifdef NO_NEWT
   NO_SLANG=1
 endif
tools/perf/util/Build +2 -0
index 9bff65e..8bc62b4 100644
@@ -75,6 +75,8 @@ libperf-$(CONFIG_X86) += tsc.o
 libperf-y += cloexec.o
 libperf-y += thread-stack.o
 libperf-y += demangle-java.o
+libperf-y += jitdump.o
+libperf-y += genelf.o
 
 libperf-$(CONFIG_LIBELF) += symbol-elf.o
 libperf-$(CONFIG_LIBELF) += probe-event.o
tools/perf/util/genelf.c new file +479 -0

479 lines not on this page · raw message

tools/perf/util/genelf.h new file +6 -0
new file mode 100644
index 0000000..11307a2
@@ -0,0 +1,6 @@
+#ifndef __GENELF_H__
+#define __GENELF_H__
+
+extern int jit_write_elf(int fd, unsigned long code_addr, const char *sym, const void *code, int csize);
+
+#endif
tools/perf/util/jitdump.c new file +588 -0

588 lines not on this page · raw message

tools/perf/util/jitdump.h new file +92 -0

92 lines not on this page · raw message

Signature · 1 line
1.9.1
#8 lore raw

Re: [PATCH v4 3/4] perf inject: add jitdump mmap injection support

Quoted 11 lines · Stephane Eranian
 tools/build/Makefile.feature             |   2 +
 tools/build/feature/Makefile             |   4 +
 tools/build/feature/test-libcrypto.c     |   9 +
 tools/perf/Documentation/perf-inject.txt |   6 +
 tools/perf/builtin-inject.c              |  60 +++-
 tools/perf/config/Makefile               |  11 +
 tools/perf/util/Build                    |   2 +
 tools/perf/util/genelf.c                 | 479
+++++++++++++++++++++++++
 tools/perf/util/genelf.h                 |   6 +
 tools/perf/util/jitdump.c                | 588
Quoted 9 lines

+++++++++++++++++++++++++++++++

 tools/perf/util/jitdump.h                |  92 +++++
 11 files changed, 1246 insertions(+), 13 deletions(-)
 create mode 100644 tools/build/feature/test-libcrypto.c
 create mode 100644 tools/perf/util/genelf.c
 create mode 100644 tools/perf/util/genelf.h
 create mode 100644 tools/perf/util/jitdump.c
 create mode 100644 tools/perf/util/jitdump.h

Stephane:

I do not fine the directory tools/build in the linux kernel tree. I am wondering if the Build file and the tools/build directory might be part of your source tree build environment?

Carl Love

#9 lore raw

Re: [PATCH v4 3/4] perf inject: add jitdump mmap injection support

Quoted 3 lines · Carl E. Love

I do not fine the directory tools/build in the linux kernel tree. I am wondering if the Build file and the tools/build directory might be part of your source tree build environment?

you want to use acme's perf/core branch from
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux

#10 lore raw

Re: [PATCH v4 3/4] perf inject: add jitdump mmap injection support

In reply to Stephane Eranian

tools/perf/builtin-inject.c line 21

Quoted 13 lines · Stephane Eranian

linkperf:perf-record[1], linkperf:perf-report[1],
linkperf:perf-archive[1]

tools/perf/builtin-inject.c +1 -0
index ea46df25..af25d1a 100644
@@ -16,6 +16,7 @@
 #include "util/debug.h"
 #include "util/build-id.h"
 #include "util/data.h"
+#include "util/jit.h"

Stephane:

The patch references the file util/jit.h. I can't find this file. Did it get missed in the patch?

Carl Love

reviewer
#11 lore raw

Re: [PATCH v4 3/4] perf inject: add jitdump mmap injection support

In reply to Stephane Eranian

<SNIP>

Quoted 27 lines
tools/perf/util/jitdump.c new file +27 -0
+static int
+jit_open(struct jit_buf_desc *jd, const char *name)
+{
+	struct jitheader header;
+	struct jr_prefix *prefix;
+	ssize_t bs, bsz = 0;
+	void *n, *buf = NULL;
+	int ret, retval = -1;
+
+	jd->in = fopen(name, "r");
+	if (!jd->in)
+		return -1;
+
+	bsz = hmax(sizeof(header), sizeof(*prefix));
+
+	buf = malloc(bsz);
+	if (!buf)
+		goto error;
+
+	ret = fread(buf, sizeof(header), 1, jd->in);
+	if (ret != 1)
+		goto error;
+
+	memcpy(&header, buf, sizeof(header));
+
+	if (header.magic != JITHEADER_MAGIC) {
+		if (header.magic != JITHEADER_MAGIC_SW)

tools/perf/util/jitdump.c line 130

Quoted 10 lines
tools/perf/util/jitdump.c new file +10 -0
+			goto error;
+		jd->needs_bswap = 1;
+	}
+
+	if (jd->needs_bswap) {
+		header.version    = bswap_32(header.version);
+		header.total_size = bswap_32(header.total_size);
+		header.pid	  = bswap_32(header.pid);
+		header.elf_mach   = bswap_32(header.elf_mach);
+		header.timestamp  = bswap_32(header.timestamp);

timestamp is 64-bit

author
#12 lore raw

PATCH v4 4/4 perf tools: add JVMTI agent library

In reply to Stephane Eranian

This is a standalone JVMTI library to help profile Java jitted code with perf record/perf report. The library is not installed or compiled automatically by perf Makefile. It is not used directly by perf. It is arch agnostic and has been tested on X86 and ARM. It needs to be used with a Java runtime, such as OpenJDK, as follows:

$ java -agentpath:libjvmti.so .......

When used this way, java will generate a jitdump binary file in $HOME/.debug/java/jit/java-jit-*

This binary dump file contains information to help symbolize and annotate jitted code.

The next step is to inject the jitdump information into the perf.data file:

$ perf inject --jit -i perf.data -o perf.data.jitted

This injects the MMAP records to cover the jitted code and also generates one ELF image for each jitted function. The ELF images are created in the same subdir as the jitdump file. The MMAP records point there too.

Then to visualize the function or asm profile, simply use the regular perf commands:

$ perf report -i perf.data.jitted
or
$ perf annotate -i perf.data.jitted

JVMTI agent code adapted from the OProfile's opagent code.

This version of the JVMTI agent is using Pawel Moll's CLOCK_MONOTONIC perf clock patch posted here:

	https://lkml.org/lkml/2015/1/21/711
Signed-off-by Stephane Eranian <[email protected]>
---
 tools/perf/jvmti/Makefile      |  70 ++++++++
 tools/perf/jvmti/jvmti_agent.c | 394 +++++++++++++++++++++++++++++++++++++++++
 tools/perf/jvmti/jvmti_agent.h |  23 +++
 tools/perf/jvmti/libjvmti.c    | 149 ++++++++++++++++
 4 files changed, 636 insertions(+)
 create mode 100644 tools/perf/jvmti/Makefile
 create mode 100644 tools/perf/jvmti/jvmti_agent.c
 create mode 100644 tools/perf/jvmti/jvmti_agent.h
 create mode 100644 tools/perf/jvmti/libjvmti.c
tools/perf/jvmti/Makefile new file +70 -0
new file mode 100644
index 0000000..22efd13
@@ -0,0 +1,70 @@
+ARCH=$(shell uname -m)
+
+ifeq ($(ARCH), x86_64)
+JARCH=amd64
+endif
+ifeq ($(ARCH), armv7l)
+JARCH=armhf
+endif
+ifeq ($(ARCH), armv6l)
+JARCH=armhf
+endif
+ifeq ($(ARCH), ppc64)
+JARCH=powerpc
+endif
+ifeq ($(ARCH), ppc64le)
+JARCH=powerpc
+endif
+
+DESTDIR=/usr/local
+
+VERSION=1
+REVISION=0
+AGE=0
+
+LN=ln -sf
+RM=rm
+
+SLIBJVMTI=libjvmti.so.$(VERSION).$(REVISION).$(AGE)
+VLIBJVMTI=libjvmti.so.$(VERSION)
+SLDFLAGS=-shared -Wl,-soname -Wl,$(VLIBJVMTI)
+SOLIBEXT=so
+
+JDIR=$(shell /usr/sbin/update-java-alternatives -l | head -1 | cut -d ' ' -f 3)
+# -lrt required in 32-bit mode for clock_gettime()
+LIBS=-lelf -lrt
+INCDIR=-I $(JDIR)/include -I $(JDIR)/include/linux
+
+TARGETS=$(SLIBJVMTI)
+
+SRCS=libjvmti.c jvmti_agent.c
+OBJS=$(SRCS:.c=.o)
+SOBJS=$(OBJS:.o=.lo)
+OPT=-O2 -g -Werror -Wall
+
+CFLAGS=$(INCDIR) $(OPT)
+
+all: $(TARGETS)
+
+.c.o:
+	$(CC) $(CFLAGS) -c $*.c
+.c.lo:
+	$(CC) -fPIC -DPIC $(CFLAGS) -c $*.c -o $*.lo
+
+$(OBJS) $(SOBJS): Makefile jvmti_agent.h ../util/jitdump.h
+
+$(SLIBJVMTI):  $(SOBJS)
+	$(CC) $(CFLAGS) $(SLDFLAGS)  -o $@ $(SOBJS) $(LIBS)
+	$(LN) $@ libjvmti.$(SOLIBEXT)
+
+clean:
+	$(RM) -f *.o *.so.* *.so *.lo
+
+install:
+	-mkdir -p $(DESTDIR)/lib
+	install -m 755 $(SLIBJVMTI) $(DESTDIR)/lib/
+	(cd $(DESTDIR)/lib; $(LN) $(SLIBJVMTI) $(VLIBJVMTI))
+	(cd $(DESTDIR)/lib; $(LN) $(SLIBJVMTI) libjvmti.$(SOLIBEXT))
+	ldconfig
+
+.SUFFIXES: .c .S .o .lo
tools/perf/jvmti/jvmti_agent.c new file +394 -0

394 lines not on this page · raw message

tools/perf/jvmti/jvmti_agent.h new file +23 -0
new file mode 100644
index 0000000..54e5c5e
@@ -0,0 +1,23 @@
+#ifndef __JVMTI_AGENT_H__
+#define __JVMTI_AGENT_H__
+
+#include <sys/types.h>
+#include <stdint.h>
+
+#define __unused __attribute__((unused))
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+void *jvmti_open(void);
+int   jvmti_close(void *agent);
+int   jvmti_write_code(void *agent, char const *symbol_name,
+		       uint64_t vma, void const *code,
+		       const unsigned int code_size);
+
+#if defined(__cplusplus)
+}
+
+#endif
+#endif /* __JVMTI_H__ */
tools/perf/jvmti/libjvmti.c new file +149 -0

149 lines not on this page · raw message

Signature · 1 line
1.9.1