perf tools: Add perf data CTF conversion
Jiri Olsa
reviewerv3 cover perf tools: Add perf data CTF conversion
hi,
this is follow up on original RFC patchset:
http://marc.info/?t=140732735600004&r=1&w=2
Basically we are adding 'perf data convert' command to allow conversion of perf data file into CTF [1] data.
v3 changes:
- rebased to latest acme's perf/core
v2 changes:
- addressed comments from Namhyung
- rebased to latest acme's perf/core
Changes from RFC:
- able to generate CTF data, that are possible to be displayed under tracecompas GUI [3], please check several screenshots in here [4]
- storing CTF data streams per cpu
- several cleanups
Examples:
- Catch default perf data (cycles event):
$ perf record ls [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.012 MB perf.data (~546 samples) ]
- To display converted CTF data run [2]:
$ babeltrace ./ctf-data/
[03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
[03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
[03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 }
[03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 }
[03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 }
[03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 }
[03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 }
[03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 }
[03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }- To get some nice output in tracecompas GUI [3], please capture sched:* and syscall tracepoints like:
# perf record -e 'sched:*,raw_syscalls:*' -a ^C[ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 412.347 MB perf.data (~18015721 samples) ]
- To convert perf data file run:
# perf data convert --to-ctf=./ctf [ perf data convert: Converted 'perf.data' into CTF data './ctf' ] [ perf data convert: Converted and wrote 408.421 MB (3964792 samples) ]
- To display converted CTF data run [2]:
# babeltrace ./ctf-data/
[23:32:20.165354855] (+0.000000507) sched:sched_wakeup: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810BCA72, perf_tid = 0, perf_pid = 0, perf_id = 462554, perf_period = 1, common_type = 265, ...
[23:32:20.165359078] (+0.000001181) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 0, perf_pid = 0, perf_id = 462562, perf_period = 1, common_type = 263, ...
[23:32:20.165364686] (+0.000000328) sched:sched_stat_runtime: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 5326, perf_pid = 5326, perf_id = 462610, perf_period = 11380, ...
[23:32:20.165366067] (+0.000001205) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 5326, perf_pid = 5326, perf_id = 462562, perf_period = 1, common_type ...
[23:32:20.165723312] (+0.000001479) sched:sched_stat_runtime: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 11821, perf_pid = 11821, perf_id = 462612, perf_period = 1000265, ...
[23:32:20.065282391] (+?.?????????) raw_syscalls:sys_enter: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810230AF, perf_tid = 26155, perf_pid = 26155, perf_id = 462635, perf_period = 1, ...
[23:32:20.065286422] (+0.000004031) raw_syscalls:sys_exit: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810231D8, perf_tid = 26155, perf_pid = 26155, perf_id = 462639, perf_period = 1, ...- Or run tracecompass and open the CTF data ;-)
Changes are also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/core_ctf_convert
thanks,
jirka
[1] Common Trace Format - http://www.efficios.com/ctf
[2] babeltrace - http://www.efficios.com/babeltrace
[3] Trace compass - http://projects.eclipse.org/projects/tools.tracecompass
[4] screenshots - http://people.redhat.com/~jolsa/tracecompass-perf/
---
Jiri Olsa (5):
perf tools: Add feature check for libbabeltrace perf tools: Add new perf data command
perf data: Add perf data to CTF conversion support perf data: Enable stream flush within processing perf data: Add support for setting ordered_events queue size
Sebastian Andrzej Siewior (3):
perf data: Add a 'perf' prefix to the generic fields perf data: Add tracepoint events fields CTF conversion support perf data: Switch to multiple cpu stream files
tools/perf/Documentation/perf-data.txt | 40 ++ tools/perf/Documentation/perf.txt | 7 +- tools/perf/Makefile.perf | 9 +- tools/perf/builtin-data.c | 119 ++++++ tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/config/Makefile | 24 ++ tools/perf/config/feature-checks/Makefile | 8 +- tools/perf/config/feature-checks/test-all.c | 5 + tools/perf/config/feature-checks/test-libbabeltrace.c | 8 + tools/perf/perf.c | 1 + tools/perf/util/data-convert-bt.c | 1057 +++++++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data-convert-bt.h | 8 + tools/perf/util/debug.c | 2 + tools/perf/util/debug.h | 1 + 15 files changed, 1287 insertions(+), 4 deletions(-) create mode 100644 tools/perf/Documentation/perf-data.txt create mode 100644 tools/perf/builtin-data.c create mode 100644 tools/perf/config/feature-checks/test-libbabeltrace.c create mode 100644 tools/perf/util/data-convert-bt.c create mode 100644 tools/perf/util/data-convert-bt.h
Jiri Olsa
reviewerPATCH v3 1/8 perf tools: Add feature check for libbabeltrace
Adding feature check for babeltrace library [1], which will be used for perf data file CTF [2] conversion in following patches.
The babeltrace library is now automatically detected as standard feature. It's possible to specify LIBBABELTRACE_DIR make variable to specify location of installed libbabeltrace, like:
$ make LIBBABELTRACE_DIR=/opt/libbabeltrace/
BUILD: Doing 'make -j4' parallel buildAuto-detecting system features:
... dwarf: [ on ] ... glibc: [ on ] ... gtk2: [ on ] ... libaudit: [ on ] ... libbfd: [ on ] ... libelf: [ on ] ... libnuma: [ on ] ... libperl: [ on ] ... libpython: [ on ] ... libslang: [ on ] ... libunwind: [ on ] ... libbabeltrace: [ on ] ... libdw-dwarf-unwind: [ on ] ... zlib: [ on ]
... DWARF post unwind library: libunwind
[1] babeltrace - http://www.efficios.com/babeltrace
[2] Common Trace Format - http://www.efficios.com/ctf
--- tools/perf/Makefile.perf | 4 +++- tools/perf/config/Makefile | 24 ++++++++++++++++++++++ tools/perf/config/feature-checks/Makefile | 8 ++++++-- tools/perf/config/feature-checks/test-all.c | 5 +++++ .../config/feature-checks/test-libbabeltrace.c | 8 ++++++++ 5 files changed, 46 insertions(+), 3 deletions(-) create mode 100644 tools/perf/config/feature-checks/test-libbabeltrace.c
tools/perf/Makefile.perf +3 -1
@@ -68,7 +68,9 @@ include config/utilities.mak # for reading the x32 mode 32-bit compatibility VDSO in 64-bit mode # # Define NO_ZLIB if you do not want to support compressed kernel modules - +# +# Define NO_LIBBABELTRACE if you do not want libbabeltrace support +# for CTF data format. ifeq ($(srctree),) srctree := $(patsubst %/,%,$(dir $(shell pwd)))
tools/perf/config/Makefile +24 -0
@@ -84,6 +84,17 @@ ifndef NO_LIBELF FEATURE_CHECK_LDFLAGS-libdw-dwarf-unwind := $(LIBDW_LDFLAGS) -ldw endif +ifndef NO_LIBBABELTRACE + # for linking with debug library, run like: + # make DEBUG=1 LIBBABELTRACE_DIR=/opt/libbabeltrace/ + ifdef LIBBABELTRACE_DIR + LIBBABELTRACE_CFLAGS := -I$(LIBBABELTRACE_DIR)/include + LIBBABELTRACE_LDFLAGS := -L$(LIBBABELTRACE_DIR)/lib + endif + FEATURE_CHECK_CFLAGS-libbabeltrace := $(LIBBABELTRACE_CFLAGS) + FEATURE_CHECK_LDFLAGS-libbabeltrace := $(LIBBABELTRACE_LDFLAGS) -lbabeltrace-ctf +endif + # include ARCH specific config -include $(src-perf)/arch/$(ARCH)/Makefile
@@ -201,6 +212,7 @@ CORE_FEATURE_TESTS = \ stackprotector-all \ timerfd \ libdw-dwarf-unwind \ + libbabeltrace \ zlib LIB_FEATURE_TESTS = \
@@ -216,6 +228,7 @@ LIB_FEATURE_TESTS = \ libslang \ libunwind \ libdw-dwarf-unwind \ + libbabeltrace \ zlib VF_FEATURE_TESTS = \
@@ -661,6 +674,17 @@ else NO_PERF_READ_VDSOX32 := 1 endif +ifndef NO_LIBBABELTRACE + ifeq ($(feature-libbabeltrace), 0) + msg := $(warning No libbabeltrace found, disables 'perf data' CTF format support, please install libbabeltrace-devel/libbabeltrace-ctf-dev); + NO_LIBBABELTRACE := 1 + else + CFLAGS += -DHAVE_LIBBABELTRACE_SUPPORT $(LIBBABELTRACE_CFLAGS) + LDFLAGS += $(LIBBABELTRACE_LDFLAGS) + EXTLIBS += -lbabeltrace-ctf + endif +endif + # Among the variables below, these: # perfexecdir # template_dir
tools/perf/config/feature-checks/Makefile +6 -2
@@ -28,6 +28,7 @@ FILES= \ test-stackprotector-all.bin \ test-timerfd.bin \ test-libdw-dwarf-unwind.bin \ + test-libbabeltrace.bin \ test-compile-32.bin \ test-compile-x32.bin \ test-zlib.bin
@@ -42,7 +43,7 @@ BUILD = $(CC) $(CFLAGS) -o $(OUTPUT)$@ $(patsubst %.bin,%.c,$@) $(LDFLAGS) ############################### test-all.bin: - $(BUILD) -Werror -fstack-protector-all -O2 -Werror -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma -lelf -laudit -I/usr/include/slang -lslang $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null) $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -DPACKAGE='"perf"' -lbfd -ldl -lz + $(BUILD) -Werror -fstack-protector-all -O2 -Werror -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma -lelf -laudit -I/usr/include/slang -lslang $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null) $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -DPACKAGE='"perf"' -lbfd -ldl -lz -lbabeltrace test-hello.bin: $(BUILD)
@@ -129,7 +130,10 @@ test-timerfd.bin: $(BUILD) test-libdw-dwarf-unwind.bin: - $(BUILD) + $(BUILD) # -ldw provided by $(FEATURE_CHECK_LDFLAGS-libdw-dwarf-unwind) + +test-libbabeltrace.bin: + $(BUILD) # -lbabeltrace provided by $(FEATURE_CHECK_LDFLAGS-libbabeltrace) test-sync-compare-and-swap.bin: $(BUILD) -Werror
tools/perf/config/feature-checks/test-all.c +5 -0
@@ -97,6 +97,10 @@# include "test-zlib.c" #undef main +#define main main_test_libbabeltrace +# include "test-libbabeltrace.c" +#undef main +int main (int argc,char *argv[]) {main_test_libpython ();
@@ -121,6 +125,7 @@ int main(int argc, char *argv[])main_test_libdw_dwarf_unwind ();main_test_sync_compare_and_swap (argc, argv);main_test_zlib (); +main_test_libbabeltrace ();return 0 ; }
tools/perf/config/feature-checks/test-libbabeltrace.c new file +8 -0
@@ -0,0 +1,8 @@ + +#include <babeltrace/ctf-writer/writer.h> + +int main (void ) +{ +bt_ctf_stream_class_get_packet_context_type ((void *)0 ); +return 0 ; +}
Signature · 1 line
1.9.3
Jiri Olsa
reviewerPATCH v3 2/8 perf tools: Add new perf data command
In reply to Jiri OlsaAdding new 'perf data' command to provide operations over data files.
The 'perf data convert' sub command is coming in following patch, but there's possibility for other useful commands like 'perf data ls' (to display perf data file in directory in ls style).
--- tools/perf/Documentation/perf-data.txt | 15 +++++++ tools/perf/Makefile.perf | 1 + tools/perf/builtin-data.c | 75 ++++++++++++++++++++++++++++++++++ tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/perf.c | 1 + 6 files changed, 94 insertions(+) create mode 100644 tools/perf/Documentation/perf-data.txt create mode 100644 tools/perf/builtin-data.c
tools/perf/Documentation/perf-data.txt new file +15 -0
@@ -0,0 +1,15 @@ +perf-data(1) +============== + +NAME +---- +perf-data - Data file related processing + +SYNOPSIS +-------- +[verse] +'perf data' [<common options>] <command> [<options>]", + +DESCRIPTION +----------- +Data file related processing.
tools/perf/Makefile.perf +1 -0
@@ -495,6 +495,7 @@ BUILTIN_OBJS += $(OUTPUT)builtin-kvm.o BUILTIN_OBJS += $(OUTPUT)builtin-inject.o BUILTIN_OBJS += $(OUTPUT)tests/builtin-test.o BUILTIN_OBJS += $(OUTPUT)builtin-mem.o +BUILTIN_OBJS += $(OUTPUT)builtin-data.o PERFLIBS = $(LIB_FILE) $(LIBAPIKFS) $(LIBTRACEEVENT)
tools/perf/builtin-data.c new file +75 -0
75 lines not on this page · raw message
tools/perf/builtin.h +1 -0
@@ -37,6 +37,7 @@ extern int cmd_test(int argc, const char **argv, const char *prefix);extern int cmd_trace (int argc,const char **argv,const char *prefix);extern int cmd_inject (int argc,const char **argv,const char *prefix);extern int cmd_mem (int argc,const char **argv,const char *prefix); +extern int cmd_data (int argc,const char **argv,const char *prefix);extern int find_scripts (char **scripts_array,char **scripts_path_array);#endif
tools/perf/command-list.txt +1 -0
@@ -7,6 +7,7 @@ perf-archive mainporcelain common perf-bench mainporcelain common perf-buildid-cache mainporcelain common perf-buildid-list mainporcelain common +perf-data mainporcelain common perf-diff mainporcelain common perf-evlist mainporcelain common perf-inject mainporcelain common
tools/perf/perf.c +1 -0
@@ -62,6 +62,7 @@ static struct cmd_struct commands[] = {#endif { "inject", cmd_inject, 0 }, { "mem", cmd_mem, 0 }, + { "data", cmd_data, 0 }, }; struct pager_config {
Signature · 1 line
1.9.3
Jiri Olsa
reviewerPATCH v3 3/8 perf data: Add perf data to CTF conversion support
In reply to Jiri OlsaAdding 'perf data convert' to convert perf data file into different format. This patch adds support for CTF format conversion.
To convert perf.data into CTF run:
$ perf data convert --to-ctf=./ctf-data/ [ perf data convert: Converted 'perf.data' into CTF data './ctf-data/' ] [ perf data convert: Converted and wrote 11.268 MB (100230 samples) ]
The command will create CTF metadata out of perf.data file (or one specified via -i option) and then convert all sample events into single CTF stream.
Each sample_type bit is translated into separated CTF event field apart from following exceptions:
PERF_SAMPLE_RAW - added in next patch PERF_SAMPLE_READ - TODO PERF_SAMPLE_CALLCHAIN - TODO
PERF_SAMPLE_BRANCH_STACK - TODO
PERF_SAMPLE_REGS_USER - TODO
PERF_SAMPLE_STACK_USER - TODO
$ perf --debug=data-convert=2 data convert ...
The converted CTF data could be analyzed by CTF tools, like babletrace or tracecompass [1].
$ babeltrace ./ctf-data/
[03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
[03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
[03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 }
[03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 }
[03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 }
[03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 }
[03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 }
[03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 }
[03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }The following members to the ctf-environment were decided to be added to distinguish and specify perf CTF data:
- domain
It says "kernel" because it contains a kernel trace (not to be confused with a user space like lttng-ust does)
- tracer_name
It says perf. This can be used to distinguish between lttng and perf CTF based trace.
- version
The kernel version from stream. In addition to release, this is what it looks like on a Debian kernel:
release = "3.14-1-amd64";
version = "3.14.0";
[1] http://projects.eclipse.org/projects/tools.tracecompass
--- tools/perf/Documentation/perf-data.txt | 25 ++ tools/perf/Documentation/perf.txt | 7 +- tools/perf/Makefile.perf | 4 + tools/perf/builtin-data.c | 44 +++ tools/perf/util/data-convert-bt.c | 612 +++++++++++++++++++++++++++++++++ tools/perf/util/data-convert-bt.h | 8 + tools/perf/util/debug.c | 2 + tools/perf/util/debug.h | 1 + 8 files changed, 702 insertions(+), 1 deletion(-) create mode 100644 tools/perf/util/data-convert-bt.c create mode 100644 tools/perf/util/data-convert-bt.h
tools/perf/Documentation/perf-data.txt +25 -0
@@ -13,3 +13,28 @@ SYNOPSIS DESCRIPTION ----------- Data file related processing. + +COMMANDS +-------- +convert:: + Converts perf data file into another format (only CTF [1] format is support by now). + It's possible to set data-convert debug variable to get debug messages from conversion, + like: + perf --debug data-convert data convert ... + +OPTIONS for 'convert' +--------------------- +--to-ctf:: + Triggers the CTF conversion, specify the path of CTF data directory. + +-i:: + Specify input perf data file path. + +-v:: +--verbose:: + Be more verbose (show counter open errors, etc). + +SEE ALSO +-------- +linkperf:perf[1] +[1] Common Trace Format - http://www.efficios.com/ctf
tools/perf/Documentation/perf.txt +6 -1
@@ -13,11 +13,16 @@ SYNOPSIS OPTIONS ------- --debug:: - Setup debug variable (just verbose for now) in value + Setup debug variable (see list below) in value range (0, 10). Use like: --debug verbose # sets verbose = 1 --debug verbose=2 # sets verbose = 2 + List of debug variables allowed to set: + verbose - general debug messages + ordered-events - ordered events object debug messages + data-convert - data convert command debug messages + --buildid-dir:: Setup buildid cache directory. It has higher priority than buildid.dir config file option.
tools/perf/Makefile.perf +4 -0
@@ -411,6 +411,10 @@ LIB_OBJS += $(OUTPUT)util/tsc.o LIB_OBJS += $(OUTPUT)util/cloexec.o LIB_OBJS += $(OUTPUT)util/thread-stack.o +ifndef NO_LIBBABELTRACE +LIB_OBJS += $(OUTPUT)util/data-convert-bt.o +endif + LIB_OBJS += $(OUTPUT)ui/setup.o LIB_OBJS += $(OUTPUT)ui/helpline.o LIB_OBJS += $(OUTPUT)ui/progress.o
tools/perf/builtin-data.c +44 -0
57 lines not on this page · raw message
tools/perf/util/data-convert-bt.c new file +612 -0
612 lines not on this page · raw message
tools/perf/util/data-convert-bt.h new file +8 -0
@@ -0,0 +1,8 @@ +#ifndef __DATA_CONVERT_BT_H +#define __DATA_CONVERT_BT_H +#ifdef HAVE_LIBBABELTRACE_SUPPORT + +int bt_convert__perf2ctf (const char *input_name,const char *to_ctf); + +#endif /* HAVE_LIBBABELTRACE_SUPPORT */ +#endif /* __DATA_CONVERT_BT_H */
tools/perf/util/debug.c +2 -0
@@ -20,6 +20,7 @@ int verbose;bool dump_trace = false, quiet = false;int debug_ordered_events;static int redirect_to_stderr; +int debug_data_convert;static int _eprintf (int level,int var,const char *fmt,va_list args) {
@@ -147,6 +148,7 @@ static struct debug_variable { { .name = "verbose", .ptr = &verbose }, { .name = "ordered-events", .ptr = &debug_ordered_events}, { .name = "stderr", .ptr = &redirect_to_stderr}, + { .name = "data-convert", .ptr = &debug_data_convert }, { .name = NULL, } };
tools/perf/util/debug.h +1 -0
@@ -12,6 +12,7 @@extern int verbose;extern bool quiet, dump_trace;extern int debug_ordered_events; +extern int debug_data_convert;#ifndef pr_fmt#define pr_fmt (fmt) fmt
Signature · 1 line
1.9.3
Jiri Olsa
reviewerPATCH v3 4/8 perf data: Add a 'perf' prefix to the generic fields
In reply to Jiri OlsaFrom: Sebastian Andrzej Siewior <[email protected]>
Some of the tracers bring their own id or pid fields and we can end up having two of them. This patch adds a "perf_" prefix to the 'generic' fields so we avoid a clash of the member names.
The change is visible in the babeltrace output:
Before:
$ babeltrace ./ctf-data/
[03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 }
[03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 }
...Now:
$ babeltrace ./ctf-data/
[03:19:13.962131936] (+0.000001935) cycles: { }, { perf_ip = 0xFFFFFFFF8105443A, perf_tid = 20714, perf_pid = 20714, perf_period = 8 }
[03:19:13.962133732] (+0.000001796) cycles: { }, { perf_ip = 0xFFFFFFFF8105443A, perf_tid = 20714, perf_pid = 20714, perf_period = 114 }
...--- tools/perf/util/data-convert-bt.c | 42 ++++++++++++++++++++------------------- 1 file changed, 22 insertions(+), 20 deletions(-)
tools/perf/util/data-convert-bt.c +22 -20
@@ -147,60 +147,62 @@ static int add_generic_values(struct ctf_writer *cw,*/ if (type &PERF_SAMPLE_IP ) { - ret =value_set_u64_hex (cw, event,"ip" , sample->ip); + ret =value_set_u64_hex (cw, event,"perf_ip" , sample->ip);if (ret)return -1 ; }if (type &PERF_SAMPLE_TID ) { - ret =value_set_s32 (cw, event,"tid" , sample->tid); + ret =value_set_s32 (cw, event,"perf_tid" , sample->tid);if (ret)return -1 ; - ret =value_set_s32 (cw, event,"pid" , sample->pid); + ret =value_set_s32 (cw, event,"perf_pid" , sample->pid);if (ret)return -1 ; }if ((type &PERF_SAMPLE_ID ) || (type &PERF_SAMPLE_IDENTIFIER )) { - ret =value_set_u64 (cw, event,"id" , sample->id); + ret =value_set_u64 (cw, event,"perf_id" , sample->id);if (ret)return -1 ; }if (type &PERF_SAMPLE_STREAM_ID ) { - ret =value_set_u64 (cw, event,"stream_id" , sample->stream_id); + ret =value_set_u64 (cw, event,"perf_stream_id" , sample->stream_id);if (ret)return -1 ; }if (type &PERF_SAMPLE_CPU ) { - ret =value_set_u32 (cw, event,"cpu" , sample->cpu); + ret =value_set_u32 (cw, event,"perf_cpu" , sample->cpu);if (ret)return -1 ; }if (type &PERF_SAMPLE_PERIOD ) { - ret =value_set_u64 (cw, event,"period" , sample->period); + ret =value_set_u64 (cw, event,"perf_period" , sample->period);if (ret)return -1 ; }if (type &PERF_SAMPLE_WEIGHT ) { - ret =value_set_u64 (cw, event,"weight" , sample->weight); + ret =value_set_u64 (cw, event,"perf_weight" , sample->weight);if (ret)return -1 ; }if (type &PERF_SAMPLE_DATA_SRC ) { - ret =value_set_u64 (cw, event,"data_src" , sample->data_src); + ret =value_set_u64 (cw, event,"perf_data_src" , + sample->data_src);if (ret)return -1 ; }if (type &PERF_SAMPLE_TRANSACTION ) { - ret =value_set_u64 (cw, event,"transaction" , sample->transaction); + ret =value_set_u64 (cw, event,"perf_transaction" , + sample->transaction);if (ret)return -1 ; }
@@ -276,34 +278,34 @@ static int add_generic_types(struct ctf_writer *cw, struct perf_evsel *evsel, }while (0 )if (type &PERF_SAMPLE_IP ) -ADD_FIELD (event_class, cw->data.u64_hex,"ip" ); +ADD_FIELD (event_class, cw->data.u64_hex,"perf_ip" );if (type &PERF_SAMPLE_TID ) { -ADD_FIELD (event_class, cw->data.s32,"tid" ); -ADD_FIELD (event_class, cw->data.s32,"pid" ); +ADD_FIELD (event_class, cw->data.s32,"perf_tid" ); +ADD_FIELD (event_class, cw->data.s32,"perf_pid" ); }if ((type &PERF_SAMPLE_ID ) || (type &PERF_SAMPLE_IDENTIFIER )) -ADD_FIELD (event_class, cw->data.u64,"id" ); +ADD_FIELD (event_class, cw->data.u64,"perf_id" );if (type &PERF_SAMPLE_STREAM_ID ) -ADD_FIELD (event_class, cw->data.u64,"stream_id" ); +ADD_FIELD (event_class, cw->data.u64,"perf_stream_id" );if (type &PERF_SAMPLE_CPU ) -ADD_FIELD (event_class, cw->data.u32,"cpu" ); +ADD_FIELD (event_class, cw->data.u32,"perf_cpu" );if (type &PERF_SAMPLE_PERIOD ) -ADD_FIELD (event_class, cw->data.u64,"period" ); +ADD_FIELD (event_class, cw->data.u64,"perf_period" );if (type &PERF_SAMPLE_WEIGHT ) -ADD_FIELD (event_class, cw->data.u64,"weight" ); +ADD_FIELD (event_class, cw->data.u64,"perf_weight" );if (type &PERF_SAMPLE_DATA_SRC ) -ADD_FIELD (event_class, cw->data.u64,"data_src" ); +ADD_FIELD (event_class, cw->data.u64,"perf_data_src" );if (type &PERF_SAMPLE_TRANSACTION ) -ADD_FIELD (event_class, cw->data.u64,"transaction" ); +ADD_FIELD (event_class, cw->data.u64,"perf_transaction" );#undef ADD_FIELDreturn 0 ;
Signature · 1 line
1.9.3
Jiri Olsa
reviewerPATCH v3 5/8 perf data: Add tracepoint events fields CTF conversion support
In reply to Jiri OlsaFrom: Sebastian Andrzej Siewior <[email protected]>
Adding support to convert tracepoint event fields into CTF event fields.
We parse each tracepoint event for CTF conversion and add tracepoint fields as regular CTF event fields, so they appear in babeltrace output like:
$ babeltrace ./ctf-data/
...
[09:02:00.950703057] (+?.?????????) sched:sched_stat_runtime: { }, { perf_ip = ... SNIP ... common_type = 298, common_flags = 1, \
common_preempt_count = 0, common_pid = 31813, comm = "perf", pid = 31813, runtime = 458800, vruntime = 52059858071 }
...--- tools/perf/util/data-convert-bt.c | 242 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 242 insertions(+)
tools/perf/util/data-convert-bt.c +242 -0
@@ -126,6 +126,177 @@ FUNC_VALUE_SET(s64)FUNC_VALUE_SET (u64)__FUNC_VALUE_SET (u64_hex, u64) +static struct bt_ctf_field_type* +get_tracepoint_field_type(struct ctf_writer *cw, struct format_field *field) +{ +unsigned long flags = field->flags; + + if (flags & FIELD_IS_STRING) + return cw->data.string; + + if (!(flags & FIELD_IS_SIGNED)) { +/* unsigned long are mostly pointers */ +if (flags &FIELD_IS_LONG || flags &FIELD_IS_POINTER ) +return cw->data.u64_hex; + } + +if (flags &FIELD_IS_SIGNED ) { +if (field->size ==8 ) +return cw->data.s64; +else +return cw->data.s32; + } + +if (field->size ==8 ) +return cw->data.u64; +else +return cw->data.u32; +} + +static int add_tracepoint_field_value (struct ctf_writer *cw, +struct bt_ctf_event_class *event_class, +struct bt_ctf_event *event, +struct perf_sample *sample, +struct format_field *fmtf) +{ +struct bt_ctf_field_type *type; +struct bt_ctf_field *array_field; +struct bt_ctf_field *field; +const char *name = fmtf->name; +void *data = sample->raw_data; +unsigned long long value_int; +unsigned long flags = fmtf->flags; +unsigned int n_items; +unsigned int i; +unsigned int offset; +unsigned int len; +int ret; + + offset = fmtf->offset; + len = fmtf->size; +if (flags &FIELD_IS_STRING ) + flags &= ~FIELD_IS_ARRAY ; + +if (flags &FIELD_IS_DYNAMIC ) { +unsigned long long tmp_val; + + tmp_val =pevent_read_number (fmtf->event->pevent, + data + offset, len); + offset = tmp_val; + len = offset >>16 ; + offset &=0xffff ; + } + +if (flags &FIELD_IS_ARRAY ) { + + type =bt_ctf_event_class_get_field_by_name ( + event_class, name); + array_field =bt_ctf_field_create (type); +bt_ctf_field_type_put (type); +if (!array_field) { +pr_err ("Failed to create array type %s\n" , name); +return -1 ; + } + + len = fmtf->size / fmtf->arraylen; + n_items = fmtf->arraylen; + }else { + n_items =1 ; + array_field =NULL ; + } + + type =get_tracepoint_field_type (cw, fmtf); + +for (i =0 ; i < n_items; i++) { +if (!(flags &FIELD_IS_STRING )) + value_int =pevent_read_number ( + fmtf->event->pevent, + data + offset + i * len, len); + +if (flags &FIELD_IS_ARRAY ) + field =bt_ctf_field_array_get_field (array_field, i); +else + field =bt_ctf_field_create (type); + +if (!field) { +pr_err ("failed to create a field %s\n" , name); +return -1 ; + } + +if (flags &FIELD_IS_STRING ) + ret =bt_ctf_field_string_set_value (field, + data + offset + i * len); +else if (!(flags &FIELD_IS_SIGNED )) + ret =bt_ctf_field_unsigned_integer_set_value ( + field, value_int); +else + ret =bt_ctf_field_signed_integer_set_value ( + field, value_int); +if (ret) { +pr_err ("failed to set file value %s\n" , name); + goto err_put_field; + } +if (!(flags &FIELD_IS_ARRAY )) { + ret =bt_ctf_event_set_payload (event, name, field); +if (ret) { +pr_err ("failed to set payload %s\n" , name); + goto err_put_field; + } + } +bt_ctf_field_put (field); + } +if (flags &FIELD_IS_ARRAY ) { + ret =bt_ctf_event_set_payload (event, name, array_field); +if (ret) { +pr_err ("Failed add payload array %s\n" , name); +return -1 ; + } +bt_ctf_field_put (array_field); + } +return 0 ; + +err_put_field: +bt_ctf_field_put (field); +return -1 ; +} + +static int add_tracepoint_fields_values (struct ctf_writer *cw, +struct bt_ctf_event_class *event_class, +struct bt_ctf_event *event, +struct format_field *fields, +struct perf_sample *sample) +{ +struct format_field *field; +int ret; + +for (field = fields; field; field = field->next) { + ret =add_tracepoint_field_value (cw, event_class, event, sample, + field); +if (ret) +return -1 ; + } +return 0 ; +} + +static int add_tracepoint_values (struct ctf_writer *cw, +struct bt_ctf_event_class *event_class, +struct bt_ctf_event *event, +struct perf_evsel *evsel, +struct perf_sample *sample) +{ +struct format_field *common_fields = evsel->tp_format->format.common_fields; +struct format_field *fields = evsel->tp_format->format.fields; +int ret; + + ret =add_tracepoint_fields_values (cw, event_class, event, + common_fields, sample); +if (!ret) + ret =add_tracepoint_fields_values (cw, event_class, event, + fields, sample); + +return ret; +} + static int add_generic_values(struct ctf_writer *cw, struct bt_ctf_event *event, struct perf_evsel *evsel,
@@ -246,11 +417,76 @@ static int process_sample_event(struct perf_tool *tool,if (ret)return -1 ; +if (evsel->attr.type ==PERF_TYPE_TRACEPOINT ) { + ret =add_tracepoint_values (cw, event_class, event, + evsel, sample); +if (ret) +return -1 ; + } +bt_ctf_stream_append_event (cw->stream, event);bt_ctf_event_put (event);return 0 ; } +static int add_tracepoint_fields_types (struct ctf_writer *cw, +struct format_field *fields, +struct bt_ctf_event_class *event_class) +{ +struct format_field *field; +int ret; + +for (field = fields; field; field = field->next) { +struct bt_ctf_field_type *type; +unsigned long flags = field->flags; + +pr2 (" field '%s'\n" , field->name); + + type =get_tracepoint_field_type (cw, field); +if (!type) +return -1 ; + +/* +* A string is an array of chars. For this we use the string +* type and don't care that it is an array. What we don't +* support is an array of strings. +*/ +if (flags &FIELD_IS_STRING ) + flags &= ~FIELD_IS_ARRAY ; + +if (flags &FIELD_IS_ARRAY ) + type =bt_ctf_field_type_array_create (type, field->arraylen); + + ret =bt_ctf_event_class_add_field (event_class, type, + field->name); + +if (flags &FIELD_IS_ARRAY ) +bt_ctf_field_type_put (type); + +if (ret) { +pr_err ("Failed to add field '%s\n" , field->name); +return -1 ; + } + } + +return 0 ; +} + +static int add_tracepoint_types (struct ctf_writer *cw, +struct perf_evsel *evsel, +struct bt_ctf_event_class *class) +{ +struct format_field *common_fields = evsel->tp_format->format.common_fields; +struct format_field *fields = evsel->tp_format->format.fields; +int ret; + + ret =add_tracepoint_fields_types (cw, common_fields, class); +if (!ret) + ret =add_tracepoint_fields_types (cw, fields, class); + +return ret; +} +static int add_generic_types (struct ctf_writer *cw,struct perf_evsel *evsel,struct bt_ctf_event_class *event_class) {
@@ -328,6 +564,12 @@ static int add_event(struct ctf_writer *cw, struct perf_evsel *evsel)if (ret) goto err; +if (evsel->attr.type ==PERF_TYPE_TRACEPOINT ) { + ret =add_tracepoint_types (cw, evsel, event_class); +if (ret) + goto err; + } + ret =bt_ctf_stream_class_add_event_class (cw->stream_class, event_class);if (ret) {pr ("Failed to add event class into stream.\n" );
Signature · 1 line
1.9.3
Jiri Olsa
reviewerPATCH v3 6/8 perf data: Switch to multiple cpu stream files
In reply to Jiri OlsaFrom: Sebastian Andrzej Siewior <[email protected]>
Currently we store the data into single data strea/file. The cpu if data is stored within the event sample. The lttng puts the CPU number that belongs to the event into the packet context instead into the event.
This patch makes sure that the trace produce by perf does look the same way. We now use one stream per-CPU. Having it all in one stream increased the total size of the resulting file. The test went from 416KiB (with perf_cpu event member) to 24MiB due to the required (and pointless) flush. With the per-cpu streams the total size went up to 588KiB.
--- tools/perf/util/data-convert-bt.c | 205 +++++++++++++++++++++++++++++++++----- 1 file changed, 181 insertions(+), 24 deletions(-)
tools/perf/util/data-convert-bt.c +181 -24
@@ -38,12 +38,20 @@ struct evsel_priv {struct bt_ctf_event_class *event_class; }; +#define MAX_CPUS 4096 + +struct ctf_stream { +struct bt_ctf_stream *stream; +int cpu; +}; +struct ctf_writer {/* writer primitives */ -struct bt_ctf_writer *writer; -struct bt_ctf_stream *stream; -struct bt_ctf_stream_class *stream_class; -struct bt_ctf_clock *clock; +struct bt_ctf_writer *writer; +struct ctf_stream **stream; +int stream_cnt; +struct bt_ctf_stream_class *stream_class; +struct bt_ctf_clock *clock;/* data types */ union {
@@ -346,12 +354,6 @@ static int add_generic_values(struct ctf_writer *cw,return -1 ; } -if (type &PERF_SAMPLE_CPU ) { - ret =value_set_u32 (cw, event,"perf_cpu" , sample->cpu); -if (ret) -return -1 ; - } -if (type &PERF_SAMPLE_PERIOD ) { ret =value_set_u64 (cw, event,"perf_period" , sample->period);if (ret)
@@ -381,6 +383,113 @@ static int add_generic_values(struct ctf_writer *cw,return 0 ; } +static int ctf_stream__flush (struct ctf_stream *cs) +{ +int err =0 ; + +if (cs) { + err =bt_ctf_stream_flush (cs->stream); +if (err) +pr_err ("CTF stream %d flush failed\n" , cs->cpu); + +pr ("Flush stream for cpu %d\n" , cs->cpu); + } + +return err; +} + +static struct ctf_stream *ctf_stream__create (struct ctf_writer *cw,int cpu) +{ +struct ctf_stream *cs; +struct bt_ctf_field *pkt_ctx =NULL ; +struct bt_ctf_field *cpu_field =NULL ; +struct bt_ctf_stream *stream =NULL ; +int ret; + + cs =zalloc (sizeof (*cs)); +if (!cs) { +pr_err ("Failed to allocate ctf stream\n" ); +return NULL ; + } + + stream =bt_ctf_writer_create_stream (cw->writer, cw->stream_class); +if (!stream) { +pr_err ("Failed to create CTF stream\n" ); + goto out; + } + + pkt_ctx =bt_ctf_stream_get_packet_context (stream); +if (!pkt_ctx) { +pr_err ("Failed to obtain packet context\n" ); + goto out; + } + + cpu_field =bt_ctf_field_structure_get_field (pkt_ctx,"cpu_id" ); +bt_ctf_field_put (pkt_ctx); +if (!cpu_field) { +pr_err ("Failed to obtain cpu field\n" ); + goto out; + } + + ret =bt_ctf_field_unsigned_integer_set_value (cpu_field, (u32 ) cpu); +if (ret) { +pr_err ("Failed to update CPU number\n" ); + goto out; + } + +bt_ctf_field_put (cpu_field); + + cs->cpu = cpu; + cs->stream = stream; +return cs; + +out: +if (cpu_field) +bt_ctf_field_put (cpu_field); +if (stream) +bt_ctf_stream_put (stream); + +free (cs); +return NULL ; +} + +static void ctf_stream__delete (struct ctf_stream *cs) +{ +if (cs) { +bt_ctf_stream_put (cs->stream); +free (cs); + } +} + +static struct ctf_stream *ctf_stream (struct ctf_writer *cw,int cpu) +{ +struct ctf_stream *cs = cw->stream[cpu]; + +if (!cs) { + cs =ctf_stream__create (cw, cpu); + cw->stream[cpu] = cs; + } + +return cs; +} + +static int get_sample_cpu (struct ctf_writer *cw,struct perf_sample *sample, +struct perf_evsel *evsel) +{ +int cpu =0 ; + +if (evsel->attr.sample_type &PERF_SAMPLE_CPU ) + cpu = sample->cpu; + +if (cpu > cw->stream_cnt) { +pr_err ("Event was recorded for CPU %d, limit is at %d.\n" , + cpu, cw->stream_cnt); + cpu =0 ; + } + +return cpu; +} + static int process_sample_event(struct perf_tool *tool, union perf_event *_event __maybe_unused, struct perf_sample *sample,
@@ -390,6 +499,7 @@ static int process_sample_event(struct perf_tool *tool, struct convert *c = container_of(tool, struct convert, tool);struct evsel_priv *priv = evsel->priv;struct ctf_writer *cw = &c->writer; +struct ctf_stream *cs;struct bt_ctf_event_class *event_class;struct bt_ctf_event *event;int ret;
@@ -424,9 +534,12 @@ static int process_sample_event(struct perf_tool *tool,return -1 ; } -bt_ctf_stream_append_event (cw->stream, event); + cs =ctf_stream (cw,get_sample_cpu (cw, sample, evsel)); +if (cs) +bt_ctf_stream_append_event (cs->stream, event); +bt_ctf_event_put (event); -return 0 ; +return cs ?0 :-1 ; } static int add_tracepoint_fields_types(struct ctf_writer *cw,
@@ -528,9 +641,6 @@ static int add_generic_types(struct ctf_writer *cw, struct perf_evsel *evsel,if (type &PERF_SAMPLE_STREAM_ID )ADD_FIELD (event_class, cw->data.u64,"perf_stream_id" ); -if (type &PERF_SAMPLE_CPU ) -ADD_FIELD (event_class, cw->data.u32,"perf_cpu" ); -if (type &PERF_SAMPLE_PERIOD )ADD_FIELD (event_class, cw->data.u64,"perf_period" );
@@ -604,6 +714,39 @@ static int setup_events(struct ctf_writer *cw, struct perf_session *session)return 0 ; } +static int setup_streams (struct ctf_writer *cw,struct perf_session *session) +{ +struct ctf_stream **stream; +struct perf_header *ph = &session->header; +int ncpus; + +/* +* Try to get the number of cpus used in the data file, +* if not present fallback to the MAX_CPUS. +*/ + ncpus = ph->env.nr_cpus_avail ?:MAX_CPUS ; + + stream =zalloc (sizeof (*stream) * ncpus); +if (!stream) { +pr_err ("Failed to allocate streams.\n" ); +return -ENOMEM ; + } + + cw->stream = stream; + cw->stream_cnt = ncpus; +return 0 ; +} + +static void free_streams (struct ctf_writer *cw) +{ +int cpu; + +for (cpu =0 ; cpu < cw->stream_cnt; cpu++) +ctf_stream__delete (cw->stream[cpu]); + +free (cw->stream); +} +static int ctf_writer__setup_env (struct ctf_writer *cw,struct perf_session *session) {
@@ -713,7 +856,7 @@ static void ctf_writer__cleanup(struct ctf_writer *cw)ctf_writer__cleanup_data (cw);bt_ctf_clock_put (cw->clock); -bt_ctf_stream_put (cw->stream); +free_streams (cw);bt_ctf_stream_class_put (cw->stream_class);bt_ctf_writer_put (cw->writer);
@@ -725,8 +868,9 @@ static int ctf_writer__init(struct ctf_writer *cw, const char *path) {struct bt_ctf_writer *writer;struct bt_ctf_stream_class *stream_class; -struct bt_ctf_stream *stream;struct bt_ctf_clock *clock; +struct bt_ctf_field_type *pkt_ctx_type; +int ret;/* CTF writer */ writer =bt_ctf_writer_create (path);
@@ -767,14 +911,15 @@ static int ctf_writer__init(struct ctf_writer *cw, const char *path)if (ctf_writer__init_data (cw)) goto err_cleanup; -/* CTF stream instance */ - stream =bt_ctf_writer_create_stream (writer, stream_class); -if (!stream) { -pr ("Failed to create CTF stream.\n" ); +/* Add cpu_id for packet context */ + pkt_ctx_type =bt_ctf_stream_class_get_packet_context_type (stream_class); +if (!pkt_ctx_type) goto err_cleanup; - } - cw->stream = stream; + ret =bt_ctf_field_type_structure_add_field (pkt_ctx_type, cw->data.u32,"cpu_id" ); +bt_ctf_field_type_put (pkt_ctx_type); +if (ret) + goto err_cleanup;/* CTF clock writer setup */ if (bt_ctf_writer_add_clock (writer, clock)) {
@@ -791,6 +936,14 @@ err:return -1 ; } +static void ctf_writer__flush_streams (struct ctf_writer *cw) +{ +int cpu; + +for (cpu =0 ; cpu < cw->stream_cnt; cpu++) +ctf_stream__flush (cw->stream[cpu]); +} +int bt_convert__perf2ctf (const char *input,const char *path) {struct perf_session *session;
@@ -833,9 +986,13 @@ int bt_convert__perf2ctf(const char *input, const char *path)if (setup_events (cw, session)) goto free_session; +if (setup_streams (cw, session)) + goto free_session; + err =perf_session__process_events (session, &c.tool);if (!err) - err =bt_ctf_stream_flush (cw->stream); +ctf_writer__flush_streams (cw); + fprintf(stderr, "[ perf data convert: Converted '%s' into CTF data '%s' ]\n",
Signature · 1 line
1.9.3
Jiri Olsa
reviewerPATCH v3 7/8 perf data: Enable stream flush within processing
In reply to Jiri OlsaFor big data files the size of data allocated for stream instance could get really high. It's needed to flush the data out of the stream once in a while.
Unfortunately there's no size indication in the stream object, so we govern the flush based on the number of stored events. Current flush limit is set ot 100000 events.
--- tools/perf/util/data-convert-bt.c | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-)
tools/perf/util/data-convert-bt.c +24 -2
@@ -43,6 +43,7 @@ struct evsel_priv {struct ctf_stream {struct bt_ctf_stream *stream;int cpu; +u32 count; };struct ctf_writer {
@@ -392,7 +393,10 @@ static int ctf_stream__flush(struct ctf_stream *cs)if (err)pr_err ("CTF stream %d flush failed\n" , cs->cpu); -pr ("Flush stream for cpu %d\n" , cs->cpu); +pr ("Flush stream for cpu %d (%u samples)\n" , + cs->cpu, cs->count); + + cs->count =0 ; }return err;
@@ -490,6 +494,19 @@ static int get_sample_cpu(struct ctf_writer *cw, struct perf_sample *sample,return cpu; } +#define STREAM_FLUSH_COUNT 100000 + +/* +* Currently we have no other way to determine the +* time for the stream flush other than keep track +* of the number of events and check it against +* threshold. +*/ +static bool is_flush_needed (struct ctf_stream *cs) +{ +return cs->count >=STREAM_FLUSH_COUNT ; +} + static int process_sample_event(struct perf_tool *tool, union perf_event *_event __maybe_unused, struct perf_sample *sample,
@@ -535,8 +552,13 @@ static int process_sample_event(struct perf_tool *tool, } cs =ctf_stream (cw,get_sample_cpu (cw, sample, evsel)); -if (cs) +if (cs) { +if (is_flush_needed (cs)) +ctf_stream__flush (cs); + + cs->count++;bt_ctf_stream_append_event (cs->stream, event); + }bt_ctf_event_put (event);return cs ?0 :-1 ;
Signature · 1 line
1.9.3
Jiri Olsa
reviewerPATCH v3 8/8 perf data: Add support for setting ordered_events queue size
In reply to Jiri OlsaAdding support to limit the size of ordered_events queue, so we could control allocation size of perf data files without proper finished round events.
--- tools/perf/util/data-convert-bt.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
tools/perf/util/data-convert-bt.c +22 -0
@@ -74,6 +74,9 @@ struct convert {u64 events_size;u64 events_count; + +/* Ordered events configured queue size. */ +u64 queue_size; }; static int value_set(struct bt_ctf_field_type *type,
@@ -966,6 +969,18 @@ static void ctf_writer__flush_streams(struct ctf_writer *cw)ctf_stream__flush (cw->stream[cpu]); } +static int convert__config (const char *var,const char *value,void *cb) +{ +struct convert *c = cb; + +if (!strcmp (var,"convert.queue-size" )) { + c->queue_size =perf_config_u64 (var, value); +return 0 ; + } + +return perf_default_config (var, value, cb); +} +int bt_convert__perf2ctf (const char *input,const char *path) {struct perf_session *session;
@@ -991,6 +1006,8 @@ int bt_convert__perf2ctf(const char *input, const char *path)struct ctf_writer *cw = &c.writer;int err =-1 ; +perf_config (convert__config, &c); +/* CTF writer */ if (ctf_writer__init (cw, path))return -1 ;
@@ -1000,6 +1017,11 @@ int bt_convert__perf2ctf(const char *input, const char *path)if (!session) goto free_writer; +if (c.queue_size) { +ordered_events__set_alloc_size (&session->ordered_events, + c.queue_size); + } +/* CTF writer env/clock setup */ if (ctf_writer__setup_env (cw, session)) goto free_session;
Signature · 1 line
1.9.3
Hi,
I'm a developer for the Trace Compass tool (see links [3], [4] in Jiri's email). I can confirm that the generated CTF can be read correctly by our tool, which enables many views and analyses (Control Flow, CPU usage view, etc.) that were previously only available for LTTng traces.
Some of our users also use perf extensively, and are looking forward to this feature! Is there any ETA as to when this will be merged upstream?
Thanks,
Alexandre
Quoted 120 lines · Jiri Olsa
hi,
this is follow up on original RFC patchset:http://marc.info/?t=140732735600004&r=1&w=2Basically we are adding 'perf data convert' command to allow conversion of perf data file into CTF [1] data.
v3 changes:
- rebased to latest acme's perf/corev2 changes:
- addressed comments from Namhyung
- rebased to latest acme's perf/coreChanges from RFC:
- able to generate CTF data, that are possible to be displayed under tracecompas GUI [3], please check several screenshots in here [4]
- storing CTF data streams per cpu
- several cleanupsExamples:
- Catch default perf data (cycles event):$ perf record ls [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.012 MB perf.data (~546 samples) ]- To display converted CTF data run [2]:
$ babeltrace ./ctf-data/ [03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 } [03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 } [03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 } [03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 } [03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 } [03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 } [03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }- To get some nice output in tracecompas GUI [3], please capture sched:* and syscall tracepoints like:
# perf record -e 'sched:*,raw_syscalls:*' -a ^C[ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 412.347 MB perf.data (~18015721 samples) ]- To convert perf data file run:
# perf data convert --to-ctf=./ctf [ perf data convert: Converted 'perf.data' into CTF data './ctf' ] [ perf data convert: Converted and wrote 408.421 MB (3964792 samples) ]- To display converted CTF data run [2]:
# babeltrace ./ctf-data/ [23:32:20.165354855] (+0.000000507) sched:sched_wakeup: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810BCA72, perf_tid = 0, perf_pid = 0, perf_id = 462554, perf_period = 1, common_type = 265, ... [23:32:20.165359078] (+0.000001181) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 0, perf_pid = 0, perf_id = 462562, perf_period = 1, common_type = 263, ... [23:32:20.165364686] (+0.000000328) sched:sched_stat_runtime: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 5326, perf_pid = 5326, perf_id = 462610, perf_period = 11380, ... [23:32:20.165366067] (+0.000001205) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 5326, perf_pid = 5326, perf_id = 462562, perf_period = 1, common_type ... [23:32:20.165723312] (+0.000001479) sched:sched_stat_runtime: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 11821, perf_pid = 11821, perf_id = 462612, perf_period = 1000265, ... [23:32:20.065282391] (+?.?????????) raw_syscalls:sys_enter: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810230AF, perf_tid = 26155, perf_pid = 26155, perf_id = 462635, perf_period = 1, ... [23:32:20.065286422] (+0.000004031) raw_syscalls:sys_exit: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810231D8, perf_tid = 26155, perf_pid = 26155, perf_id = 462639, perf_period = 1, ...- Or run tracecompass and open the CTF data ;-)
Changes are also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/core_ctf_convertthanks,
jirka[1] Common Trace Format - http://www.efficios.com/ctf
[2] babeltrace - http://www.efficios.com/babeltrace
[3] Trace compass - http://projects.eclipse.org/projects/tools.tracecompass
[4] screenshots - http://people.redhat.com/~jolsa/tracecompass-perf/Cc Arnaldo Carvalho de Melo <[email protected]>Cc David Ahern <[email protected]>Cc Dominique Toupin <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Jeremie Galarneau <[email protected]>Cc Jiri Olsa <[email protected]>Cc Mathieu Desnoyers <[email protected]>Cc Namhyung Kim <[email protected]>Cc Paul Mackerras <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Sebastian Andrzej Siewior <[email protected]>Cc Tom Zanussi <[email protected]>---
Jiri Olsa (5):
perf tools: Add feature check for libbabeltrace perf tools: Add new perf data command
perf data: Add perf data to CTF conversion support perf data: Enable stream flush within processing perf data: Add support for settingordered_eventsqueue sizeSebastian Andrzej Siewior (3):
perf data: Add a 'perf' prefix to the generic fields perf data: Add tracepoint events fields CTF conversion support perf data: Switch to multiple cpu stream filestools/perf/Documentation/perf-data.txt | 40 ++ tools/perf/Documentation/perf.txt | 7 +- tools/perf/Makefile.perf | 9 +- tools/perf/builtin-data.c | 119 ++++++ tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/config/Makefile | 24 ++ tools/perf/config/feature-checks/Makefile | 8 +- tools/perf/config/feature-checks/test-all.c | 5 + tools/perf/config/feature-checks/test-libbabeltrace.c | 8 + tools/perf/perf.c | 1 + tools/perf/util/data-convert-bt.c | 1057 +++++++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data-convert-bt.h | 8 + tools/perf/util/debug.c | 2 + tools/perf/util/debug.h | 1 + 15 files changed, 1287 insertions(+), 4 deletions(-)create mode 100644
tools/perf/Documentation/perf-data.txtcreate mode 100644tools/perf/builtin-data.c
create mode 100644tools/perf/config/feature-checks/test-libbabeltrace.ccreate mode 100644tools/perf/util/data-convert-bt.c
create mode 100644tools/perf/util/data-convert-bt.h
Quoted 8 lines · Alexandre Montplaisir
Hi,
I'm a developer for the Trace Compass tool (see links [3], [4] in Jiri's email). I can confirm that the generated CTF can be read correctly by our tool, which enables many views and analyses (Control Flow, CPU usage view, etc.) that were previously only available for LTTng traces.
Some of our users also use perf extensively, and are looking forward to this feature! Is there any ETA as to when this will be merged upstream?
Thanks,
Alexandre
That was a bit too fast, it seems there are issues with very recent versions of Babeltrace. You can follow the discussion at
http://lists.linuxfoundation.org/pipermail/diamon-discuss/2015-January/000007.html
Cheers,
Alex
Quoted 144 lines · Jiri Olsa
Quoted 140 lines
hi,
this is follow up on original RFC patchset:http://marc.info/?t=140732735600004&r=1&w=2Basically we are adding 'perf data convert' command to allow conversion of perf data file into CTF [1] data.
v3 changes:
- rebased to latest acme's perf/corev2 changes:
- addressed comments from Namhyung
- rebased to latest acme's perf/coreChanges from RFC:
- able to generate CTF data, that are possible to be displayed under tracecompas GUI [3], please check several screenshots in here [4]
- storing CTF data streams per cpu
- several cleanupsExamples:
- Catch default perf data (cycles event):$ perf record ls [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.012 MB perf.data (~546 samples) ]- To display converted CTF data run [2]:
$ babeltrace ./ctf-data/ [03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 } [03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 } [03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 } [03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 } [03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 } [03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 } [03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }- To get some nice output in tracecompas GUI [3], please capture sched:* and syscall tracepoints like:
# perf record -e 'sched:*,raw_syscalls:*' -a ^C[ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 412.347 MB perf.data (~18015721 samples) ]- To convert perf data file run:
# perf data convert --to-ctf=./ctf [ perf data convert: Converted 'perf.data' into CTF data './ctf' ] [ perf data convert: Converted and wrote 408.421 MB (3964792 samples) ]- To display converted CTF data run [2]:
# babeltrace ./ctf-data/ [23:32:20.165354855] (+0.000000507) sched:sched_wakeup: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810BCA72, perf_tid = 0, perf_pid = 0, perf_id = 462554, perf_period = 1, common_type = 265, ... [23:32:20.165359078] (+0.000001181) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 0, perf_pid = 0, perf_id = 462562, perf_period = 1, common_type = 263, ... [23:32:20.165364686] (+0.000000328) sched:sched_stat_runtime: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 5326, perf_pid = 5326, perf_id = 462610, perf_period = 11380, ... [23:32:20.165366067] (+0.000001205) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 5326, perf_pid = 5326, perf_id = 462562, perf_period = 1, common_type ... [23:32:20.165723312] (+0.000001479) sched:sched_stat_runtime: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 11821, perf_pid = 11821, perf_id = 462612, perf_period = 1000265, ... [23:32:20.065282391] (+?.?????????) raw_syscalls:sys_enter: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810230AF, perf_tid = 26155, perf_pid = 26155, perf_id = 462635, perf_period = 1, ... [23:32:20.065286422] (+0.000004031) raw_syscalls:sys_exit: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810231D8, perf_tid = 26155, perf_pid = 26155, perf_id = 462639, perf_period = 1, ...- Or run tracecompass and open the CTF data ;-)
Changes are also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/core_ctf_convertthanks,
jirka[1] Common Trace Format - http://www.efficios.com/ctf
[2] babeltrace - http://www.efficios.com/babeltrace
[3] Trace compass - http://projects.eclipse.org/projects/tools.tracecompass
[4] screenshots - http://people.redhat.com/~jolsa/tracecompass-perf/Cc Arnaldo Carvalho de Melo <[email protected]>Cc David Ahern <[email protected]>Cc Dominique Toupin <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Jeremie Galarneau <[email protected]>Cc Jiri Olsa <[email protected]>Cc Mathieu Desnoyers <[email protected]>Cc Namhyung Kim <[email protected]>Cc Paul Mackerras <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Sebastian Andrzej Siewior <[email protected]>Cc Tom Zanussi <[email protected]>---
Jiri Olsa (5):
perf tools: Add feature check for libbabeltrace perf tools: Add new perf data command
perf data: Add perf data to CTF conversion support perf data: Enable stream flush within processing perf data: Add support for settingordered_eventsqueue sizeSebastian Andrzej Siewior (3):
perf data: Add a 'perf' prefix to the generic fields perf data: Add tracepoint events fields CTF conversion support perf data: Switch to multiple cpu stream filestools/perf/Documentation/perf-data.txt | 40 ++ tools/perf/Documentation/perf.txt | 7 +- tools/perf/Makefile.perf | 9 +- tools/perf/builtin-data.c | 119 ++++++ tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/config/Makefile | 24 ++ tools/perf/config/feature-checks/Makefile | 8 +- tools/perf/config/feature-checks/test-all.c | 5 + tools/perf/config/feature-checks/test-libbabeltrace.c | 8 + tools/perf/perf.c | 1 + tools/perf/util/data-convert-bt.c | 1057 +++++++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data-convert-bt.h | 8 + tools/perf/util/debug.c | 2 + tools/perf/util/debug.h | 1 + 15 files changed, 1287 insertions(+), 4 deletions(-)create mode 100644
tools/perf/Documentation/perf-data.txtcreate mode 100644tools/perf/builtin-data.ccreate mode 100644tools/perf/config/feature-checks/test-libbabeltrace.c
create mode 100644tools/perf/util/data-convert-bt.ccreate mode 100644tools/perf/util/data-convert-bt.h
Quoted 12 lines · Alexandre Montplaisir
Quoted 11 lines · Alexandre Montplaisir
Hi,
I'm a developer for the Trace Compass tool (see links [3], [4] in Jiri's email). I can confirm that the generated CTF can be read correctly by our tool, which enables many views and analyses (Control Flow, CPU usage view, etc.) that were previously only available for LTTng traces.
Some of our users also use perf extensively, and are looking forward to this feature! Is there any ETA as to when this will be merged upstream?
Quoted 8 lines
Quoted 2 lines
Thanks,
AlexandreThat was a bit too fast, it seems there are issues with very recent versions of Babeltrace. You can follow the discussion at
http://lists.linuxfoundation.org/pipermail/diamon-discuss/2015-January/000007.html
It's a regression introduced by a commit that was pushed to master yesterday. I'm working on a fix.
In the meantime, testing against Babeltrace master 3baf0856 should be alright.
Jérémie
Quoted 159 lines
Cheers,
AlexQuoted 154 lines · Jiri Olsa
Quoted 149 lines
hi,
this is follow up on original RFC patchset:http://marc.info/?t=140732735600004&r=1&w=2Basically we are adding 'perf data convert' command to allow conversion of perf data file into CTF [1] data.
v3 changes:
- rebased to latest acme's perf/corev2 changes:
- addressed comments from Namhyung
- rebased to latest acme's perf/coreChanges from RFC:
- able to generate CTF data, that are possible to be displayed under tracecompas GUI [3], please check several screenshots in here [4]
- storing CTF data streams per cpu
- several cleanupsExamples:
- Catch default perf data (cycles event):$ perf record ls [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.012 MB perf.data (~546 samples) ]- To display converted CTF data run [2]:
$ babeltrace ./ctf-data/ [03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 } [03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 } [03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 } [03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 } [03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 } [03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 } [03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }- To get some nice output in tracecompas GUI [3], please capture sched:* and syscall tracepoints like:
# perf record -e 'sched:*,raw_syscalls:*' -a ^C[ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 412.347 MB perf.data (~18015721 samples) ]- To convert perf data file run:
# perf data convert --to-ctf=./ctf [ perf data convert: Converted 'perf.data' into CTF data './ctf' ] [ perf data convert: Converted and wrote 408.421 MB (3964792 samples) ]- To display converted CTF data run [2]:
# babeltrace ./ctf-data/ [23:32:20.165354855] (+0.000000507) sched:sched_wakeup: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810BCA72, perf_tid = 0, perf_pid = 0, perf_id = 462554, perf_period = 1, common_type = 265, ... [23:32:20.165359078] (+0.000001181) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 0, perf_pid = 0, perf_id = 462562, perf_period = 1, common_type = 263, ... [23:32:20.165364686] (+0.000000328) sched:sched_stat_runtime: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 5326, perf_pid = 5326, perf_id = 462610, perf_period = 11380, ... [23:32:20.165366067] (+0.000001205) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 5326, perf_pid = 5326, perf_id = 462562, perf_period = 1, common_type ... [23:32:20.165723312] (+0.000001479) sched:sched_stat_runtime: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 11821, perf_pid = 11821, perf_id = 462612, perf_period = 1000265, ... [23:32:20.065282391] (+?.?????????) raw_syscalls:sys_enter: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810230AF, perf_tid = 26155, perf_pid = 26155, perf_id = 462635, perf_period = 1, ... [23:32:20.065286422] (+0.000004031) raw_syscalls:sys_exit: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810231D8, perf_tid = 26155, perf_pid = 26155, perf_id = 462639, perf_period = 1, ...- Or run tracecompass and open the CTF data ;-)
Changes are also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/core_ctf_convertthanks,
jirka[1] Common Trace Format - http://www.efficios.com/ctf
[2] babeltrace - http://www.efficios.com/babeltrace
[3] Trace compass -
http://projects.eclipse.org/projects/tools.tracecompass
[4] screenshots - http://people.redhat.com/~jolsa/tracecompass-perf/Cc Arnaldo Carvalho de Melo <[email protected]>Cc David Ahern <[email protected]>Cc Dominique Toupin <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Jeremie Galarneau <[email protected]>Cc Jiri Olsa <[email protected]>Cc Mathieu Desnoyers <[email protected]>Cc Namhyung Kim <[email protected]>Cc Paul Mackerras <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Sebastian Andrzej Siewior <[email protected]>Cc Tom Zanussi <[email protected]>---
Jiri Olsa (5):
perf tools: Add feature check for libbabeltrace perf tools: Add new perf data command
perf data: Add perf data to CTF conversion support perf data: Enable stream flush within processing perf data: Add support for settingordered_eventsqueue sizeSebastian Andrzej Siewior (3):
perf data: Add a 'perf' prefix to the generic fields perf data: Add tracepoint events fields CTF conversion support perf data: Switch to multiple cpu stream filestools/perf/Documentation/perf-data.txt | 40 ++ tools/perf/Documentation/perf.txt | 7 +- tools/perf/Makefile.perf | 9 +- tools/perf/builtin-data.c | 119 ++++++ tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/config/Makefile | 24 ++ tools/perf/config/feature-checks/Makefile | 8 +- tools/perf/config/feature-checks/test-all.c | 5 + tools/perf/config/feature-checks/test-libbabeltrace.c | 8 + tools/perf/perf.c | 1 + tools/perf/util/data-convert-bt.c | 1057 +++++++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data-convert-bt.h | 8 + tools/perf/util/debug.c | 2 + tools/perf/util/debug.h | 1 + 15 files changed, 1287 insertions(+), 4 deletions(-)create mode 100644
tools/perf/Documentation/perf-data.txtcreate mode 100644tools/perf/builtin-data.ccreate mode 100644tools/perf/config/feature-checks/test-libbabeltrace.c
create mode 100644tools/perf/util/data-convert-bt.ccreate mode 100644tools/perf/util/data-convert-bt.h
Signature · 3 lines
Jérémie Galarneau EfficiOS Inc. http://www.efficios.com
Quoted 18 lines · Jérémie Galarneau
Quoted 16 lines · Alexandre Montplaisir
Quoted 13 lines · Alexandre Montplaisir
Hi,
I'm a developer for the Trace Compass tool (see links [3], [4] in Jiri's email). I can confirm that the generated CTF can be read correctly by our tool, which enables many views and analyses (Control Flow, CPU usage view, etc.) that were previously only available for LTTng traces.
Some of our users also use perf extensively, and are looking forward to this feature! Is there any ETA as to when this will be merged upstream?
Thanks,
Alexandre
Quoted 8 lines
Quoted 4 lines
That was a bit too fast, it seems there are issues with very recent versions of Babeltrace. You can follow the discussion at
http://lists.linuxfoundation.org/pipermail/diamon-discuss/2015-January/000007.htmlIt's a regression introduced by a commit that was pushed to master yesterday. I'm working on a fix.
In the meantime, testing against Babeltrace master3baf0856should be alright.
yep.. works for me ;-)
thanks,
jirka
Quoted 18 lines · Jérémie Galarneau
Quoted 16 lines · Alexandre Montplaisir
Quoted 13 lines · Alexandre Montplaisir
Hi,
I'm a developer for the Trace Compass tool (see links [3], [4] in Jiri's email). I can confirm that the generated CTF can be read correctly by our tool, which enables many views and analyses (Control Flow, CPU usage view, etc.) that were previously only available for LTTng traces.
Some of our users also use perf extensively, and are looking forward to this feature! Is there any ETA as to when this will be merged upstream?
Thanks,
Alexandre
Quoted 8 lines
Quoted 4 lines
That was a bit too fast, it seems there are issues with very recent versions of Babeltrace. You can follow the discussion at
http://lists.linuxfoundation.org/pipermail/diamon-discuss/2015-January/000007.htmlIt's a regression introduced by a commit that was pushed to master yesterday. I'm working on a fix.
In the meantime, testing against Babeltrace master3baf0856should be alright.
even on this HEAD I'm still getting the 3rd issue I described in earlier email:
---
the bt_ctf_trace_create_stream function tries to set ID on stream_class which gets frozen just few moments ago in bt_ctf_stream_create call..
this trigers failure and close up of stream fd which is not set yet:
if (close(stream->pos.fd)) {
perror("close");
}
hence the 'close: Bad file descriptor' message
so this one looks like a bug to me, but we might be misusing the API also..
---
I need attached patch in perf code.. any idea?
thanks,
jirka
---
tools/perf/util/data-convert-bt.c +5 -0
@@ -960,6 +960,11 @@ static int ctf_writer__init(struct ctf_writer *cw, const char *path) goto err_cleanup; } +if (bt_ctf_stream_class_set_id (stream_class,0 )) { +pr ("Failed to set CTF stream class ID.\n" ); + goto err_cleanup; + } + cw->stream_class = stream_class;/* CTF clock stream setup */
Quoted 41 lines · Jiri Olsa
Quoted 18 lines · Jérémie Galarneau
Quoted 16 lines · Alexandre Montplaisir
Quoted 13 lines · Alexandre Montplaisir
Hi,
I'm a developer for the Trace Compass tool (see links [3], [4] in Jiri's email). I can confirm that the generated CTF can be read correctly by our tool, which enables many views and analyses (Control Flow, CPU usage view, etc.) that were previously only available for LTTng traces.
Some of our users also use perf extensively, and are looking forward to this feature! Is there any ETA as to when this will be merged upstream?
Thanks,
AlexandreQuoted 8 lines
Quoted 4 lines
That was a bit too fast, it seems there are issues with very recent versions of Babeltrace. You can follow the discussion at
http://lists.linuxfoundation.org/pipermail/diamon-discuss/2015-January/000007.htmlIt's a regression introduced by a commit that was pushed to master yesterday. I'm working on a fix.
In the meantime, testing against Babeltrace master3baf0856should be alright.even on this HEAD I'm still getting the 3rd issue I described in earlier email:
---
thebt_ctf_trace_create_streamfunction tries to set ID onstream_classwhich gets frozen just few moments ago inbt_ctf_stream_createcall..this trigers failure and close up of stream fd which is not set yet:
if (
close(stream->pos.fd)) {
perror("close");
}
Quoted 6 lines
hence the 'close: Bad file descriptor' message
so this one looks like a bug to me, but we might be misusing the API also..
---I need attached patch in perf code.. any idea?
I have merged a patch addressing this problem. Thanks for reporting!
commit 29d9d76c476cbf3fdf6fa709bfbdc24309974f06
Author: Jérémie Galarneau <[email protected]> Date: Wed Jan 28 16:52:28 2015 -0500
Fix: lock stream class after assigning stream id
Fixes a bug that was introduced by 2f100782 which made it possible to set custom stream class IDs.
The stream class is frozen when a stream of its type is instanciated. However, the trace or writer must still assign a unique ID to the stream class if none were set prior. This modification moves the stream class locking (freeze) after the ID assignment check.
Jérémie
Quoted 21 lines
thanks,
jirka---
tools/perf/util/data-convert-bt.c +5
-0@@ -960,6 +960,11 @@ static int ctf_writer__init(struct ctf_writer *cw, const char *path) goto err_cleanup; } +if (bt_ctf_stream_class_set_id (stream_class,0 )) { +pr ("Failed to set CTF stream class ID.\n" ); + goto err_cleanup; + } + cw->stream_class = stream_class;/* CTF clock stream setup */
Signature · 3 lines
Jérémie Galarneau EfficiOS Inc. http://www.efficios.com
Quoted 42 lines · Jérémie Galarneau
Quoted 41 lines · Jiri Olsa
Quoted 18 lines · Jérémie Galarneau
Quoted 16 lines · Alexandre Montplaisir
Quoted 13 lines · Alexandre Montplaisir
Hi,
I'm a developer for the Trace Compass tool (see links [3], [4] in Jiri's email). I can confirm that the generated CTF can be read correctly by our tool, which enables many views and analyses (Control Flow, CPU usage view, etc.) that were previously only available for LTTng traces.
Some of our users also use perf extensively, and are looking forward to this feature! Is there any ETA as to when this will be merged upstream?
Thanks,
AlexandreQuoted 8 lines
Quoted 4 lines
That was a bit too fast, it seems there are issues with very recent versions of Babeltrace. You can follow the discussion at
http://lists.linuxfoundation.org/pipermail/diamon-discuss/2015-January/000007.htmlIt's a regression introduced by a commit that was pushed to master yesterday. I'm working on a fix.
In the meantime, testing against Babeltrace master3baf0856should be alright.even on this HEAD I'm still getting the 3rd issue I described in earlier email:
--- the bt_ctf_trace_create_stream function tries to set ID on stream_class which gets frozen just few moments ago in bt_ctf_stream_create call.. this trigers failure and close up of stream fd which is not set yet: if (close(stream->pos.fd)) { perror("close"); }
Quoted 8 lines
Quoted 6 lines
hence the 'close: Bad file descriptor' message
so this one looks like a bug to me, but we might be misusing the API also..
---I need attached patch in perf code.. any idea?
I have merged a patch addressing this problem. Thanks for reporting!
cool, seems to work perfectly.. thanks
I'll resend converter v4 shortly
jirka
Quoted 115 lines · Jiri Olsa
hi,
this is follow up on original RFC patchset:http://marc.info/?t=140732735600004&r=1&w=2Basically we are adding 'perf data convert' command to allow conversion of perf data file into CTF [1] data.
v3 changes:
- rebased to latest acme's perf/corev2 changes:
- addressed comments from Namhyung
- rebased to latest acme's perf/coreChanges from RFC:
- able to generate CTF data, that are possible to be displayed under tracecompas GUI [3], please check several screenshots in here [4]
- storing CTF data streams per cpu
- several cleanupsExamples:
- Catch default perf data (cycles event):$ perf record ls [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.012 MB perf.data (~546 samples) ]- To display converted CTF data run [2]:
$ babeltrace ./ctf-data/ [03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 } [03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 } [03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 } [03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 } [03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 } [03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 } [03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }- To get some nice output in tracecompas GUI [3], please capture sched:* and syscall tracepoints like:
# perf record -e 'sched:*,raw_syscalls:*' -a ^C[ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 412.347 MB perf.data (~18015721 samples) ]- To convert perf data file run:
# perf data convert --to-ctf=./ctf [ perf data convert: Converted 'perf.data' into CTF data './ctf' ] [ perf data convert: Converted and wrote 408.421 MB (3964792 samples) ]- To display converted CTF data run [2]:
# babeltrace ./ctf-data/ [23:32:20.165354855] (+0.000000507) sched:sched_wakeup: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810BCA72, perf_tid = 0, perf_pid = 0, perf_id = 462554, perf_period = 1, common_type = 265, ... [23:32:20.165359078] (+0.000001181) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 0, perf_pid = 0, perf_id = 462562, perf_period = 1, common_type = 263, ... [23:32:20.165364686] (+0.000000328) sched:sched_stat_runtime: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 5326, perf_pid = 5326, perf_id = 462610, perf_period = 11380, ... [23:32:20.165366067] (+0.000001205) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 5326, perf_pid = 5326, perf_id = 462562, perf_period = 1, common_type ... [23:32:20.165723312] (+0.000001479) sched:sched_stat_runtime: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 11821, perf_pid = 11821, perf_id = 462612, perf_period = 1000265, ... [23:32:20.065282391] (+?.?????????) raw_syscalls:sys_enter: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810230AF, perf_tid = 26155, perf_pid = 26155, perf_id = 462635, perf_period = 1, ... [23:32:20.065286422] (+0.000004031) raw_syscalls:sys_exit: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810231D8, perf_tid = 26155, perf_pid = 26155, perf_id = 462639, perf_period = 1, ...- Or run tracecompass and open the CTF data ;-)
Changes are also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/core_ctf_convertthanks,
jirka[1] Common Trace Format - http://www.efficios.com/ctf
[2] babeltrace - http://www.efficios.com/babeltrace
[3] Trace compass - http://projects.eclipse.org/projects/tools.tracecompass
[4] screenshots - http://people.redhat.com/~jolsa/tracecompass-perf/Cc Arnaldo Carvalho de Melo <[email protected]>Cc David Ahern <[email protected]>Cc Dominique Toupin <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Jeremie Galarneau <[email protected]>Cc Jiri Olsa <[email protected]>Cc Mathieu Desnoyers <[email protected]>Cc Namhyung Kim <[email protected]>Cc Paul Mackerras <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Sebastian Andrzej Siewior <[email protected]>Cc Tom Zanussi <[email protected]>---
Jiri Olsa (5):
perf tools: Add feature check for libbabeltrace perf tools: Add new perf data command
perf data: Add perf data to CTF conversion support perf data: Enable stream flush within processing perf data: Add support for settingordered_eventsqueue sizeSebastian Andrzej Siewior (3):
perf data: Add a 'perf' prefix to the generic fields perf data: Add tracepoint events fields CTF conversion support perf data: Switch to multiple cpu stream filestools/perf/Documentation/perf-data.txt | 40 ++ tools/perf/Documentation/perf.txt | 7 +- tools/perf/Makefile.perf | 9 +- tools/perf/builtin-data.c | 119 ++++++ tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/config/Makefile | 24 ++ tools/perf/config/feature-checks/Makefile | 8 +- tools/perf/config/feature-checks/test-all.c | 5 + tools/perf/config/feature-checks/test-libbabeltrace.c | 8 + tools/perf/perf.c | 1 + tools/perf/util/data-convert-bt.c | 1057 +++++++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data-convert-bt.h | 8 + tools/perf/util/debug.c | 2 + tools/perf/util/debug.h | 1 + 15 files changed, 1287 insertions(+), 4 deletions(-)
Quoted 6 lines
create mode 100644
tools/perf/Documentation/perf-data.txtcreate mode 100644tools/perf/builtin-data.c
create mode 100644tools/perf/config/feature-checks/test-libbabeltrace.ccreate mode 100644tools/perf/util/data-convert-bt.c
create mode 100644tools/perf/util/data-convert-bt.h
All of the patches look good to me:
Seems like a great start point to a cool feature.
Hi Jiri,
Quoted 6 lines · Jiri Olsa
hi,
this is follow up on original RFC patchset:
http://marc.info/?t=140732735600004&r=1&w=2Basically we are adding 'perf data convert' command to allow conversion of perf data file into CTF [1] data.
I think I already gave my ACKs to this series, but let me repeat it.
Thanks,
Namhyung
Quoted 119 lines
v3 changes:
- rebased to latest acme's perf/corev2 changes:
- addressed comments from Namhyung
- rebased to latest acme's perf/coreChanges from RFC:
- able to generate CTF data, that are possible to be displayed under tracecompas GUI [3], please check several screenshots in here [4]
- storing CTF data streams per cpu
- several cleanupsExamples:
- Catch default perf data (cycles event):$ perf record ls [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.012 MB perf.data (~546 samples) ]- To display converted CTF data run [2]:
$ babeltrace ./ctf-data/ [03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 } [03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 } [03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 } [03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 } [03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 } [03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 } [03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }- To get some nice output in tracecompas GUI [3], please capture sched:* and syscall tracepoints like:
# perf record -e 'sched:*,raw_syscalls:*' -a ^C[ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 412.347 MB perf.data (~18015721 samples) ]- To convert perf data file run:
# perf data convert --to-ctf=./ctf [ perf data convert: Converted 'perf.data' into CTF data './ctf' ] [ perf data convert: Converted and wrote 408.421 MB (3964792 samples) ]- To display converted CTF data run [2]:
# babeltrace ./ctf-data/ [23:32:20.165354855] (+0.000000507) sched:sched_wakeup: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810BCA72, perf_tid = 0, perf_pid = 0, perf_id = 462554, perf_period = 1, common_type = 265, ... [23:32:20.165359078] (+0.000001181) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 0, perf_pid = 0, perf_id = 462562, perf_period = 1, common_type = 263, ... [23:32:20.165364686] (+0.000000328) sched:sched_stat_runtime: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 5326, perf_pid = 5326, perf_id = 462610, perf_period = 11380, ... [23:32:20.165366067] (+0.000001205) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 5326, perf_pid = 5326, perf_id = 462562, perf_period = 1, common_type ... [23:32:20.165723312] (+0.000001479) sched:sched_stat_runtime: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 11821, perf_pid = 11821, perf_id = 462612, perf_period = 1000265, ... [23:32:20.065282391] (+?.?????????) raw_syscalls:sys_enter: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810230AF, perf_tid = 26155, perf_pid = 26155, perf_id = 462635, perf_period = 1, ... [23:32:20.065286422] (+0.000004031) raw_syscalls:sys_exit: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810231D8, perf_tid = 26155, perf_pid = 26155, perf_id = 462639, perf_period = 1, ...- Or run tracecompass and open the CTF data ;-)
Changes are also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/core_ctf_convertthanks,
jirka[1] Common Trace Format - http://www.efficios.com/ctf
[2] babeltrace - http://www.efficios.com/babeltrace
[3] Trace compass - http://projects.eclipse.org/projects/tools.tracecompass
[4] screenshots - http://people.redhat.com/~jolsa/tracecompass-perf/Cc Arnaldo Carvalho de Melo <[email protected]>Cc David Ahern <[email protected]>Cc Dominique Toupin <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Jeremie Galarneau <[email protected]>Cc Jiri Olsa <[email protected]>Cc Mathieu Desnoyers <[email protected]>Cc Namhyung Kim <[email protected]>Cc Paul Mackerras <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Sebastian Andrzej Siewior <[email protected]>Cc Tom Zanussi <[email protected]>---
Jiri Olsa (5):
perf tools: Add feature check for libbabeltrace perf tools: Add new perf data command
perf data: Add perf data to CTF conversion support perf data: Enable stream flush within processing perf data: Add support for settingordered_eventsqueue sizeSebastian Andrzej Siewior (3):
perf data: Add a 'perf' prefix to the generic fields perf data: Add tracepoint events fields CTF conversion support perf data: Switch to multiple cpu stream filestools/perf/Documentation/perf-data.txt | 40 ++ tools/perf/Documentation/perf.txt | 7 +- tools/perf/Makefile.perf | 9 +- tools/perf/builtin-data.c | 119 ++++++ tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/config/Makefile | 24 ++ tools/perf/config/feature-checks/Makefile | 8 +- tools/perf/config/feature-checks/test-all.c | 5 + tools/perf/config/feature-checks/test-libbabeltrace.c | 8 + tools/perf/perf.c | 1 + tools/perf/util/data-convert-bt.c | 1057 +++++++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data-convert-bt.h | 8 + tools/perf/util/debug.c | 2 + tools/perf/util/debug.h | 1 + 15 files changed, 1287 insertions(+), 4 deletions(-) create mode 100644 tools/perf/Documentation/perf-data.txt create mode 100644 tools/perf/builtin-data.c create mode 100644 tools/perf/config/feature-checks/test-libbabeltrace.c create mode 100644 tools/perf/util/data-convert-bt.c create mode 100644 tools/perf/util/data-convert-bt.h--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
As a Trace Compass developper, this opens a lot of possibilities for us, like correlating PCAP, Perf and LTTng traces...
I hope it gets in soon as we already support it. :)
Quoted 120 lines · Jiri Olsa
hi,
this is follow up on original RFC patchset:
http://marc.info/?t=140732735600004&r=1&w=2Basically we are adding 'perf data convert' command to allow conversion of perf data file into CTF [1] data.
v3 changes:
- rebased to latest acme's perf/corev2 changes:
- addressed comments from Namhyung
- rebased to latest acme's perf/coreChanges from RFC:
- able to generate CTF data, that are possible to be displayed under tracecompas GUI [3], please check several screenshots in here [4]
- storing CTF data streams per cpu
- several cleanupsExamples:
- Catch default perf data (cycles event):$ perf record ls [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.012 MB perf.data (~546 samples) ]- To display converted CTF data run [2]:
$ babeltrace ./ctf-data/ [03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 } [03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 } [03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 } [03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 } [03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 } [03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 } [03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }- To get some nice output in tracecompas GUI [3], please capture sched:* and syscall tracepoints like:
# perf record -e 'sched:*,raw_syscalls:*' -a ^C[ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 412.347 MB perf.data (~18015721 samples) ]- To convert perf data file run:
# perf data convert --to-ctf=./ctf [ perf data convert: Converted 'perf.data' into CTF data './ctf' ] [ perf data convert: Converted and wrote 408.421 MB (3964792 samples) ]- To display converted CTF data run [2]:
# babeltrace ./ctf-data/ [23:32:20.165354855] (+0.000000507) sched:sched_wakeup: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810BCA72, perf_tid = 0, perf_pid = 0, perf_id = 462554, perf_period = 1, common_type = 265, ... [23:32:20.165359078] (+0.000001181) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 0, perf_pid = 0, perf_id = 462562, perf_period = 1, common_type = 263, ... [23:32:20.165364686] (+0.000000328) sched:sched_stat_runtime: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 5326, perf_pid = 5326, perf_id = 462610, perf_period = 11380, ... [23:32:20.165366067] (+0.000001205) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 5326, perf_pid = 5326, perf_id = 462562, perf_period = 1, common_type ... [23:32:20.165723312] (+0.000001479) sched:sched_stat_runtime: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 11821, perf_pid = 11821, perf_id = 462612, perf_period = 1000265, ... [23:32:20.065282391] (+?.?????????) raw_syscalls:sys_enter: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810230AF, perf_tid = 26155, perf_pid = 26155, perf_id = 462635, perf_period = 1, ... [23:32:20.065286422] (+0.000004031) raw_syscalls:sys_exit: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810231D8, perf_tid = 26155, perf_pid = 26155, perf_id = 462639, perf_period = 1, ...- Or run tracecompass and open the CTF data ;-)
Changes are also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/core_ctf_convertthanks,
jirka[1] Common Trace Format - http://www.efficios.com/ctf
[2] babeltrace - http://www.efficios.com/babeltrace
[3] Trace compass - http://projects.eclipse.org/projects/tools.tracecompass
[4] screenshots - http://people.redhat.com/~jolsa/tracecompass-perf/Cc Arnaldo Carvalho de Melo <[email protected]>Cc David Ahern <[email protected]>Cc Dominique Toupin <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Jeremie Galarneau <[email protected]>Cc Jiri Olsa <[email protected]>Cc Mathieu Desnoyers <[email protected]>Cc Namhyung Kim <[email protected]>Cc Paul Mackerras <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Sebastian Andrzej Siewior <[email protected]>Cc Tom Zanussi <[email protected]>---
Jiri Olsa (5):
perf tools: Add feature check for libbabeltrace perf tools: Add new perf data command
perf data: Add perf data to CTF conversion support perf data: Enable stream flush within processing perf data: Add support for settingordered_eventsqueue sizeSebastian Andrzej Siewior (3):
perf data: Add a 'perf' prefix to the generic fields perf data: Add tracepoint events fields CTF conversion support perf data: Switch to multiple cpu stream filestools/perf/Documentation/perf-data.txt | 40 ++ tools/perf/Documentation/perf.txt | 7 +- tools/perf/Makefile.perf | 9 +- tools/perf/builtin-data.c | 119 ++++++ tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/config/Makefile | 24 ++ tools/perf/config/feature-checks/Makefile | 8 +- tools/perf/config/feature-checks/test-all.c | 5 + tools/perf/config/feature-checks/test-libbabeltrace.c | 8 + tools/perf/perf.c | 1 + tools/perf/util/data-convert-bt.c | 1057 +++++++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data-convert-bt.h | 8 + tools/perf/util/debug.c | 2 + tools/perf/util/debug.h | 1 + 15 files changed, 1287 insertions(+), 4 deletions(-) create mode 100644 tools/perf/Documentation/perf-data.txt create mode 100644 tools/perf/builtin-data.c create mode 100644 tools/perf/config/feature-checks/test-libbabeltrace.c create mode 100644 tools/perf/util/data-convert-bt.c create mode 100644 tools/perf/util/data-convert-bt.h