perf tools: Add perf data CTF conversion
Jiri Olsa
reviewercover perf tools: Add perf data CTF conversion
hi,
this is initial post of the CTF converter for perf. Basically adding 'perf data convert' command to allow conversion of perf data file into CTF [1] data.
To convert perf data file run:
$ perf data convert --to-ctf=./ctf-data/ [ perf data convert: Converted 'perf.data' into CTF data './ctf-data/' ] [ perf data convert: Converted and wrote 11.268 MB (100230 samples) ]
To display converted CTF data run [2]:
$ babeltrace ./ctf-data/
[03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
[03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
[03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 }
[03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 }
[03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 }
[03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 }
[03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 }
[03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 }
[03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }It has still some limitations (noted in patches), but it has reached the point of the useful perf feature. More features and supports are about to come and it'll be better to continue publicly.
Also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/core_ctf_convert
thanks for comments,
jirka
[1] Common Trace Format - http://www.efficios.com/ctf
[2] babeltrace - http://www.efficios.com/babeltrace
---
Jiri Olsa (3):
perf tools: Add feature check for libbabeltrace perf tools: Add new perf data command
perf tools data: Add perf data to CTF convertion support
Sebastian Andrzej Siewior (2):
perf tools data: Add a 'perf' prefix to the generic fields perf tools data: Add tracepoint events fields CTF convertion support
tools/perf/Documentation/perf-data.txt | 40 ++++ tools/perf/Documentation/perf.txt | 7 +- tools/perf/Makefile.perf | 8 + tools/perf/builtin-data.c | 112 ++++++++++ tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/config/Makefile | 28 ++- tools/perf/config/feature-checks/Makefile | 10 +- tools/perf/config/feature-checks/test-all.c | 5 + tools/perf/config/feature-checks/test-libbabeltrace.c | 8 + tools/perf/perf.c | 1 + tools/perf/util/data-bt.c | 859 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data-bt.h | 8 + tools/perf/util/debug.c | 4 +- tools/perf/util/debug.h | 1 + 15 files changed, 1086 insertions(+), 7 deletions(-) create mode 100644 tools/perf/Documentation/perf-data.txt create mode 100644 tools/perf/builtin-data.c create mode 100644 tools/perf/config/feature-checks/test-libbabeltrace.c create mode 100644 tools/perf/util/data-bt.c create mode 100644 tools/perf/util/data-bt.h
Jiri Olsa
reviewerPATCH 1/5 perf tools: Add feature check for libbabeltrace
Adding feature check for babeltrace library [1], which will be used for perf data file CTF [2] conversion in following patches.
The babeltrace library is now automatically detected as standard feature. It's possible to specify LIBBABELTRACE_DIR make variable to specify location of installed libbabeltrace, like:
$ make LIBBABELTRACE_DIR=/opt/libbabeltrace/
BUILD: Doing 'make -j4' parallel buildAuto-detecting system features:
... dwarf: [ on ] ... glibc: [ on ] ... gtk2: [ on ] ... libaudit: [ on ] ... libbfd: [ on ] ... libelf: [ on ] ... libnuma: [ on ] ... libperl: [ on ] ... libpython: [ on ] ... libslang: [ on ] ... libunwind: [ on ] ... libdw-dwarf-unwind: [ on ] ... libbabeltrace: [ on ]
... DWARF post unwind library: libunwind
[1] babeltrace - http://www.efficios.com/babeltrace
[2] Common Trace Format - http://www.efficios.com/ctf
--- tools/perf/Makefile.perf | 3 +++ tools/perf/config/Makefile | 27 ++++++++++++++++++++-- tools/perf/config/feature-checks/Makefile | 10 +++++--- tools/perf/config/feature-checks/test-all.c | 5 ++++ .../config/feature-checks/test-libbabeltrace.c | 8 +++++++ 5 files changed, 48 insertions(+), 5 deletions(-) create mode 100644 tools/perf/config/feature-checks/test-libbabeltrace.c
tools/perf/Makefile.perf +3 -0
@@ -60,6 +60,9 @@ include config/utilities.mak # # Define NO_LIBDW_DWARF_UNWIND if you do not want libdw support # for dwarf backtrace post unwind. +# +# Define NO_LIBBABELTRACE if you do not want libbabeltrace support +# for CTF data format. ifeq ($(srctree),) srctree := $(patsubst %/,%,$(dir $(shell pwd)))
tools/perf/config/Makefile +25 -2
@@ -88,6 +88,17 @@ ifndef NO_LIBELF FEATURE_CHECK_LDFLAGS-libdw-dwarf-unwind := $(LIBDW_LDFLAGS) -ldw endif +ifndef NO_LIBBABELTRACE + # for linking with debug library, run like: + # make DEBUG=1 LIBBABELTRACE_DIR=/opt/libbabeltrace + ifdef LIBBABELTRACE_DIR + LIBBABELTRACE_CFLAGS := -I$(LIBBABELTRACE_DIR)/include + LIBBABELTRACE_LDFLAGS := -L$(LIBBABELTRACE_DIR)/lib + endif + FEATURE_CHECK_CFLAGS-libbabeltrace := $(LIBBABELTRACE_CFLAGS) + FEATURE_CHECK_LDFLAGS-libbabeltrace := $(LIBBABELTRACE_LDFLAGS) -lbabeltrace-ctf +endif + # include ARCH specific config -include $(src-perf)/arch/$(ARCH)/Makefile
@@ -204,7 +215,8 @@ CORE_FEATURE_TESTS = \ libunwind \ stackprotector-all \ timerfd \ - libdw-dwarf-unwind + libdw-dwarf-unwind \ + libbabeltrace LIB_FEATURE_TESTS = \ dwarf \
@@ -218,7 +230,8 @@ LIB_FEATURE_TESTS = \ libpython \ libslang \ libunwind \ - libdw-dwarf-unwind + libdw-dwarf-unwind \ + libbabeltrace VF_FEATURE_TESTS = \ backtrace \
@@ -620,6 +633,16 @@ ifdef HAVE_KVM_STAT_SUPPORT CFLAGS += -DHAVE_KVM_STAT_SUPPORT endif +ifndef NO_LIBBABELTRACE + ifeq ($(feature-libbabeltrace), 0) + msg := $(warning No libbabeltrace found, disables 'perf data' CTF format support, please install libbabeltrace-devel/libbabeltrace-dev); + NO_LIBBABELTRACE := 1 + else + CFLAGS += -DHAVE_LIBBABELTRACE_SUPPORT + EXTLIBS += -lbabeltrace + endif +endif + # Among the variables below, these: # perfexecdir # template_dir
tools/perf/config/feature-checks/Makefile +7 -3
@@ -27,7 +27,8 @@ FILES= \ test-libunwind-debug-frame.bin \ test-stackprotector-all.bin \ test-timerfd.bin \ - test-libdw-dwarf-unwind.bin + test-libdw-dwarf-unwind.bin \ + test-libbabeltrace.bin CC := $(CROSS_COMPILE)gcc -MD PKG_CONFIG := $(CROSS_COMPILE)pkg-config
@@ -39,7 +40,7 @@ BUILD = $(CC) $(CFLAGS) -o $(OUTPUT)$@ $(patsubst %.bin,%.c,$@) $(LDFLAGS) ############################### test-all.bin: - $(BUILD) -Werror -fstack-protector-all -O2 -Werror -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma -lelf -laudit -I/usr/include/slang -lslang $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null) $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -DPACKAGE='"perf"' -lbfd -ldl + $(BUILD) -Werror -fstack-protector-all -O2 -Werror -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma -lelf -laudit -I/usr/include/slang -lslang $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null) $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -DPACKAGE='"perf"' -lbfd -ldl -lbabeltrace test-hello.bin: $(BUILD)
@@ -126,7 +127,10 @@ test-timerfd.bin: $(BUILD) test-libdw-dwarf-unwind.bin: - $(BUILD) + $(BUILD) # -ldw provided by $(FEATURE_CHECK_LDFLAGS-libdw-dwarf-unwind) + +test-libbabeltrace.bin: + $(BUILD) # -lbabeltrace provided by $(FEATURE_CHECK_LDFLAGS-libbabeltrace) test-sync-compare-and-swap.bin: $(BUILD) -Werror
tools/perf/config/feature-checks/test-all.c +5 -0
@@ -93,6 +93,10 @@# include "test-sync-compare-and-swap.c" #undef main +#define main main_test_libbabeltrace +# include "test-libbabeltrace.c" +#undef main +int main (int argc,char *argv[]) {main_test_libpython ();
@@ -116,6 +120,7 @@ int main(int argc, char *argv[])main_test_stackprotector_all ();main_test_libdw_dwarf_unwind ();main_test_sync_compare_and_swap (argc, argv); +main_test_libbabeltrace ();return 0 ; }
tools/perf/config/feature-checks/test-libbabeltrace.c new file +8 -0
@@ -0,0 +1,8 @@ + +#include <babeltrace/ctf-writer/writer.h> + +int main (void ) +{ +bt_ctf_writer_create (NULL ); +return 0 ; +}
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 2/5 perf tools: Add new perf data command
In reply to Jiri OlsaAdding new 'perf data' command to provide operations over data files.
The 'perf data convert' sub command is coming in following patch, but there's possibility for other useful commands like 'perf data ls' (to display perf data file in directory in ls style).
--- tools/perf/Documentation/perf-data.txt | 15 ++++++ tools/perf/Makefile.perf | 1 + tools/perf/builtin-data.c | 98 ++++++++++++++++++++++++++++++++++ tools/perf/builtin.h | 1 + tools/perf/command-list.txt | 1 + tools/perf/perf.c | 1 + 6 files changed, 117 insertions(+) create mode 100644 tools/perf/Documentation/perf-data.txt create mode 100644 tools/perf/builtin-data.c
tools/perf/Documentation/perf-data.txt new file +15 -0
@@ -0,0 +1,15 @@ +perf-data(1) +============== + +NAME +---- +perf-data - Data file related processing + +SYNOPSIS +-------- +[verse] +'perf data' [<common options>] <command> [<options>]", + +DESCRIPTION +----------- +Data file related processing.
tools/perf/Makefile.perf +1 -0
@@ -464,6 +464,7 @@ BUILTIN_OBJS += $(OUTPUT)builtin-kvm.o BUILTIN_OBJS += $(OUTPUT)builtin-inject.o BUILTIN_OBJS += $(OUTPUT)tests/builtin-test.o BUILTIN_OBJS += $(OUTPUT)builtin-mem.o +BUILTIN_OBJS += $(OUTPUT)builtin-data.o PERFLIBS = $(LIB_FILE) $(LIBAPIKFS) $(LIBTRACEEVENT)
tools/perf/builtin-data.c new file +98 -0
98 lines not on this page · raw message
tools/perf/builtin.h +1 -0
@@ -37,6 +37,7 @@ extern int cmd_test(int argc, const char **argv, const char *prefix);extern int cmd_trace (int argc,const char **argv,const char *prefix);extern int cmd_inject (int argc,const char **argv,const char *prefix);extern int cmd_mem (int argc,const char **argv,const char *prefix); +extern int cmd_data (int argc,const char **argv,const char *prefix);extern int find_scripts (char **scripts_array,char **scripts_path_array);#endif
tools/perf/command-list.txt +1 -0
@@ -7,6 +7,7 @@ perf-archive mainporcelain common perf-bench mainporcelain common perf-buildid-cache mainporcelain common perf-buildid-list mainporcelain common +perf-data mainporcelain common perf-diff mainporcelain common perf-evlist mainporcelain common perf-inject mainporcelain common
tools/perf/perf.c +1 -0
@@ -62,6 +62,7 @@ static struct cmd_struct commands[] = {#endif { "inject", cmd_inject, 0 }, { "mem", cmd_mem, 0 }, + { "data", cmd_data, 0 }, }; struct pager_config {
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 3/5 perf tools data: Add perf data to CTF convertion support
In reply to Jiri OlsaAdding 'perf data convert' to convert perf data file into different format. This patch adds support for CTF format conversion.
To convert perf.data into CTF run:
$ perf data convert --to-ctf=./ctf-data/ [ perf data convert: Converted 'perf.data' into CTF data './ctf-data/' ] [ perf data convert: Converted and wrote 11.268 MB (100230 samples) ]
The command will create CTF metadata out of perf.data file (or one specified via -i option) and then convert all sample events into single CTF stream.
Each sample_type bit is translated into separated CTF event field apart from following exceptions:
PERF_SAMPLE_RAW - added in next patch PERF_SAMPLE_READ - TODO PERF_SAMPLE_CALLCHAIN - TODO
PERF_SAMPLE_BRANCH_STACK - TODO
PERF_SAMPLE_REGS_USER - TODO
PERF_SAMPLE_STACK_USER - TODO
$ perf --debug=data-convert=2 data convert ...
The converted CTF data could be analyzed by CTF tools, like:
$ babeltrace ./ctf-data/
[03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
[03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
[03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 }
[03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 }
[03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 }
[03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 }
[03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 }
[03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 }
[03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }--- tools/perf/Documentation/perf-data.txt | 25 ++ tools/perf/Documentation/perf.txt | 7 +- tools/perf/Makefile.perf | 4 + tools/perf/builtin-data.c | 14 + tools/perf/config/Makefile | 5 +- tools/perf/util/data-bt.c | 611 +++++++++++++++++++++++++++++++++ tools/perf/util/data-bt.h | 8 + tools/perf/util/debug.c | 4 +- tools/perf/util/debug.h | 1 + 9 files changed, 675 insertions(+), 4 deletions(-) create mode 100644 tools/perf/util/data-bt.c create mode 100644 tools/perf/util/data-bt.h
tools/perf/Documentation/perf-data.txt +25 -0
@@ -13,3 +13,28 @@ SYNOPSIS DESCRIPTION ----------- Data file related processing. + +COMMANDS +-------- +convert:: + Converts perf data file into another format (only CTF [1] format is support by now). + It's possible to set data-convert debug variable to get debug messages from conversion, + like: + perf --debug data-convert data convert ... + +OPTIONS for 'convert' +--------------------- +--to-ctf:: + Triggers the CTF conversion, specify the path of CTF data directory. + +-i:: + Specify input perf data file path. + +-v:: +--verbose:: + Be more verbose (show counter open errors, etc). + +SEE ALSO +-------- +linkperf:perf[1] +[1] Common Trace Format - http://www.efficios.com/ctf
tools/perf/Documentation/perf.txt +6 -1
@@ -13,11 +13,16 @@ SYNOPSIS OPTIONS ------- --debug:: - Setup debug variable (just verbose for now) in value + Setup debug variable (see list below) in value range (0, 10). Use like: --debug verbose # sets verbose = 1 --debug verbose=2 # sets verbose = 2 + List of debug variables allowed to set: + verbose - general debug messages + ordered-events - ordered events object debug messages + data-convert - data convert command debug messages + DESCRIPTION ----------- Performance counters for Linux are a new kernel-based subsystem
tools/perf/Makefile.perf +4 -0
@@ -383,6 +383,10 @@ LIB_OBJS += $(OUTPUT)util/data.o LIB_OBJS += $(OUTPUT)util/tsc.o LIB_OBJS += $(OUTPUT)util/cloexec.o +ifneq ($(NO_LIBBABELTRACE),1) +LIB_OBJS += $(OUTPUT)util/data-bt.o +endif + LIB_OBJS += $(OUTPUT)ui/setup.o LIB_OBJS += $(OUTPUT)ui/helpline.o LIB_OBJS += $(OUTPUT)ui/progress.o
tools/perf/builtin-data.c +14 -0
@@ -3,6 +3,7 @@#include "perf.h" #include "debug.h" #include "parse-options.h" +#include "data-bt.h" typedef int (*data_cmd_fn_t )(int argc,const char **argv,const char *prefix);
@@ -49,10 +50,14 @@ static const char * const data_convert_usage[] = {static int cmd_data_convert (int argc,const char **argv,const char *prefix__maybe_unused ) { +const char *to_ctf =NULL ;const struct option options[] = {OPT_INCR ('v' ,"verbose" , &verbose,"be more verbose" ),OPT_STRING ('i' ,"input" , &input_name,"file" ,"input file name" ), +#ifdef HAVE_LIBBABELTRACE_SUPPORT +OPT_STRING (0 ,"to-ctf" , &to_ctf,NULL ,"Convert to CTF format" ), +#endifOPT_END () };
@@ -63,6 +68,15 @@ static int cmd_data_convert(int argc, const char **argv,return -1 ; } +if (to_ctf) { +#ifdef HAVE_LIBBABELTRACE_SUPPORT +return bt_convert__perf2ctf (input_name, to_ctf); +#else +pr_err ("The libbabeltrace support is not compiled in\n" ); +return -1 ; +#endif + } +return 0 ; }
tools/perf/config/Makefile +3 -2
@@ -638,8 +638,9 @@ ifndef NO_LIBBABELTRACE msg := $(warning No libbabeltrace found, disables 'perf data' CTF format support, please install libbabeltrace-devel/libbabeltrace-dev); NO_LIBBABELTRACE := 1 else - CFLAGS += -DHAVE_LIBBABELTRACE_SUPPORT - EXTLIBS += -lbabeltrace + CFLAGS += -DHAVE_LIBBABELTRACE_SUPPORT $(LIBBABELTRACE_CFLAGS) + LDFLAGS += $(LIBBABELTRACE_LDFLAGS) + EXTLIBS += -lbabeltrace-ctf endif endif
tools/perf/util/data-bt.c new file +611 -0
611 lines not on this page · raw message
tools/perf/util/data-bt.h new file +8 -0
@@ -0,0 +1,8 @@ +#ifndef __DATA_BT_H +#define __DATA_BT_H +#ifdef HAVE_LIBBABELTRACE_SUPPORT + +int bt_convert__perf2ctf (const char *input_name,const char *to_ctf); + +#endif /* HAVE_LIBBABELTRACE_SUPPORT */ +#endif /* __DATA_BT_H */
tools/perf/util/debug.c +3 -1
@@ -19,6 +19,7 @@int verbose;bool dump_trace = false, quiet = false;int debug_ordered_events; +int debug_data_convert;static int _eprintf (int level,int var,const char *fmt,va_list args) {
@@ -144,7 +145,8 @@ static struct debug_variable {int *ptr; } debug_variables[] = { { .name = "verbose", .ptr = &verbose }, - { .name = "ordered-events", .ptr = &debug_ordered_events}, + { .name = "ordered-events", .ptr = &debug_ordered_events }, + { .name = "data-convert", .ptr = &debug_data_convert }, { .name = NULL, } };
tools/perf/util/debug.h +1 -0
@@ -11,6 +11,7 @@extern int verbose;extern bool quiet, dump_trace;extern int debug_ordered_events; +extern int debug_data_convert;#ifndef pr_fmt#define pr_fmt (fmt) fmt
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 4/5 perf tools data: Add a 'perf' prefix to the generic fields
In reply to Jiri OlsaFrom: Sebastian Andrzej Siewior <[email protected]>
Some of the tracers bring their own id or pid fields and we can end up having two of them. This patch adds a "perf_" prefix to the 'generic' fields so we avoid a clash of the member names.
The change is visible in the babeltrace output:
Before:
$ babeltrace ./ctf-data/
[03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 }
[03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 }
...Now:
$ babeltrace ./ctf-data/
[03:19:13.962131936] (+0.000001935) cycles: { }, { perf_ip = 0xFFFFFFFF8105443A, perf_tid = 20714, perf_pid = 20714, perf_period = 8 }
[03:19:13.962133732] (+0.000001796) cycles: { }, { perf_ip = 0xFFFFFFFF8105443A, perf_tid = 20714, perf_pid = 20714, perf_period = 114 }
...--- tools/perf/util/data-bt.c | 42 ++++++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 20 deletions(-)
tools/perf/util/data-bt.c +22 -20
@@ -147,60 +147,62 @@ static int add_generic_values(struct ctf_writer *cw,*/ if (type &PERF_SAMPLE_IP ) { - ret =value_set_u64_hex (cw, event,"ip" , sample->ip); + ret =value_set_u64_hex (cw, event,"perf_ip" , sample->ip);if (ret)return -1 ; }if (type &PERF_SAMPLE_TID ) { - ret =value_set_s32 (cw, event,"tid" , sample->tid); + ret =value_set_s32 (cw, event,"perf_tid" , sample->tid);if (ret)return -1 ; - ret =value_set_s32 (cw, event,"pid" , sample->tid); + ret =value_set_s32 (cw, event,"perf_pid" , sample->tid);if (ret)return -1 ; }if ((type &PERF_SAMPLE_ID ) || (type &PERF_SAMPLE_IDENTIFIER )) { - ret =value_set_u64 (cw, event,"id" , sample->id); + ret =value_set_u64 (cw, event,"perf_id" , sample->id);if (ret)return -1 ; }if (type &PERF_SAMPLE_STREAM_ID ) { - ret =value_set_u64 (cw, event,"stream_id" , sample->id); + ret =value_set_u64 (cw, event,"perf_stream_id" , sample->id);if (ret)return -1 ; }if (type &PERF_SAMPLE_CPU ) { - ret =value_set_u32 (cw, event,"cpu" , sample->cpu); + ret =value_set_u32 (cw, event,"perf_cpu" , sample->cpu);if (ret)return -1 ; }if (type &PERF_SAMPLE_PERIOD ) { - ret =value_set_u64 (cw, event,"period" , sample->period); + ret =value_set_u64 (cw, event,"perf_period" , sample->period);if (ret)return -1 ; }if (type &PERF_SAMPLE_WEIGHT ) { - ret =value_set_u64 (cw, event,"weight" , sample->weight); + ret =value_set_u64 (cw, event,"perf_weight" , sample->weight);if (ret)return -1 ; }if (type &PERF_SAMPLE_DATA_SRC ) { - ret =value_set_u64 (cw, event,"data_src" , sample->data_src); + ret =value_set_u64 (cw, event,"perf_data_src" , + sample->data_src);if (ret)return -1 ; }if (type &PERF_SAMPLE_TRANSACTION ) { - ret =value_set_u64 (cw, event,"transaction" , sample->transaction); + ret =value_set_u64 (cw, event,"perf_transaction" , + sample->transaction);if (ret)return -1 ; }
@@ -276,34 +278,34 @@ static int add_generic_types(struct ctf_writer *cw, struct perf_evsel *evsel, }while (0 )if (type &PERF_SAMPLE_IP ) -ADD_FIELD (event_class, cw->data.u64_hex,"ip" ); +ADD_FIELD (event_class, cw->data.u64_hex,"perf_ip" );if (type &PERF_SAMPLE_TID ) { -ADD_FIELD (event_class, cw->data.s32,"tid" ); -ADD_FIELD (event_class, cw->data.s32,"pid" ); +ADD_FIELD (event_class, cw->data.s32,"perf_tid" ); +ADD_FIELD (event_class, cw->data.s32,"perf_pid" ); }if ((type &PERF_SAMPLE_ID ) || (type &PERF_SAMPLE_IDENTIFIER )) -ADD_FIELD (event_class, cw->data.u64,"id" ); +ADD_FIELD (event_class, cw->data.u64,"perf_id" );if (type &PERF_SAMPLE_STREAM_ID ) -ADD_FIELD (event_class, cw->data.u64,"stream_id" ); +ADD_FIELD (event_class, cw->data.u64,"perf_stream_id" );if (type &PERF_SAMPLE_CPU ) -ADD_FIELD (event_class, cw->data.u32,"cpu" ); +ADD_FIELD (event_class, cw->data.u32,"perf_cpu" );if (type &PERF_SAMPLE_PERIOD ) -ADD_FIELD (event_class, cw->data.u64,"period" ); +ADD_FIELD (event_class, cw->data.u64,"perf_period" );if (type &PERF_SAMPLE_WEIGHT ) -ADD_FIELD (event_class, cw->data.u64,"weight" ); +ADD_FIELD (event_class, cw->data.u64,"perf_weight" );if (type &PERF_SAMPLE_DATA_SRC ) -ADD_FIELD (event_class, cw->data.u64,"data_src" ); +ADD_FIELD (event_class, cw->data.u64,"perf_data_src" );if (type &PERF_SAMPLE_TRANSACTION ) -ADD_FIELD (event_class, cw->data.u64,"transaction" ); +ADD_FIELD (event_class, cw->data.u64,"perf_transaction" );#undef ADD_FIELDreturn 0 ;
Signature · 1 line
1.8.3.1
Jiri Olsa
reviewerPATCH 5/5 perf tools data: Add tracepoint events fields CTF convertion support
In reply to Jiri OlsaFrom: Sebastian Andrzej Siewior <[email protected]>
Adding support to convert tracepoint event fields into CTF event fields.
We parse each tracepoint event for CTF conversion and add tracepoint fields as regular CTF event fields, so they appear in babeltrace output like:
$ babeltrace ./ctf-data/
...
[09:02:00.950703057] (+?.?????????) sched:sched_stat_runtime: { }, { perf_ip = ... SNIP ... common_type = 298, common_flags = 1, \
common_preempt_count = 0, common_pid = 31813, comm = "perf", pid = 31813, runtime = 458800, vruntime = 52059858071 }
...--- tools/perf/util/data-bt.c | 246 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 246 insertions(+)
tools/perf/util/data-bt.c +246 -0
@@ -126,6 +126,179 @@ FUNC_VALUE_SET(s64)FUNC_VALUE_SET (u64)__FUNC_VALUE_SET (u64_hex, u64) +static struct bt_ctf_field_type* +get_tracepoint_field_type(struct ctf_writer *cw, struct format_field *field) +{ +unsigned long flags = field->flags; + + if (flags & FIELD_IS_STRING) + return cw->data.string; + + if (!(flags & FIELD_IS_SIGNED)) { +/* unsigned long are mostly pointers */ +if (flags &FIELD_IS_LONG || flags &FIELD_IS_POINTER ) +return cw->data.u64_hex; + } + +if (flags &FIELD_IS_SIGNED ) { +if (field->size ==8 ) +return cw->data.s64; +else +return cw->data.s32; + } + +if (field->size ==8 ) +return cw->data.u64; +else +return cw->data.u32; +} + +static int add_tracepoint_field_value (struct ctf_writer *cw, +struct bt_ctf_event_class *event_class, +struct bt_ctf_event *event, +struct perf_sample *sample, +struct format_field *fmtf) +{ +struct bt_ctf_field_type *type; +struct bt_ctf_field *array_field; +struct bt_ctf_field *field; +const char *name = fmtf->name; +void *data = sample->raw_data; +unsigned long long value_int; +unsigned long flags = fmtf->flags; +unsigned int n_items; +unsigned int i; +unsigned int offset; +unsigned int len; +int ret; + + offset = fmtf->offset; + len = fmtf->size; +if (flags &FIELD_IS_STRING ) + flags &= ~FIELD_IS_ARRAY ; + +if (flags &FIELD_IS_DYNAMIC ) { +unsigned long long tmp_val; + + tmp_val =pevent_read_number (fmtf->event->pevent, + data + offset, len); + offset = tmp_val; + len = offset >>16 ; + offset &=0xffff ; + } + +if (flags &FIELD_IS_ARRAY ) { + + type =bt_ctf_event_class_get_field_by_name ( + event_class, name); + array_field =bt_ctf_field_create (type); +bt_ctf_field_type_put (type); +if (!array_field) { +pr_err ("Failed to create array type %s\n" , name); +return -1 ; + } + + len = fmtf->size / fmtf->arraylen; + n_items = fmtf->arraylen; + }else { + n_items =1 ; + array_field =NULL ; + } + + type =get_tracepoint_field_type (cw, fmtf); + +for (i =0 ; i < n_items; i++) { +if (!(flags &FIELD_IS_STRING )) + value_int =pevent_read_number ( + fmtf->event->pevent, + data + offset + i * len, len); + +if (flags &FIELD_IS_ARRAY ) + field =bt_ctf_field_array_get_field (array_field, i); +else + field =bt_ctf_field_create (type); + +if (!field) { +pr_err ("failed to create a field %s\n" , name); +return -1 ; + } + +if (flags &FIELD_IS_STRING ) + ret =bt_ctf_field_string_set_value (field, + data + offset + i * len); +else if (!(flags &FIELD_IS_SIGNED )) + ret =bt_ctf_field_unsigned_integer_set_value ( + field, value_int); +else + ret =bt_ctf_field_signed_integer_set_value ( + field, value_int); +if (ret) { +pr_err ("failed to set file value %s\n" , name); + goto err_put_field; + } +if (!(flags &FIELD_IS_ARRAY )) { + ret =bt_ctf_event_set_payload (event, name, field); +if (ret) { +pr_err ("failed to set payload %s\n" , name); + goto err_put_field; + } + } +bt_ctf_field_put (field); + } +if (flags &FIELD_IS_ARRAY ) { + ret =bt_ctf_event_set_payload (event, name, array_field); +if (ret) { +pr_err ("Failed add payload array %s\n" , name); +return -1 ; + } +bt_ctf_field_put (array_field); + } +return 0 ; + +err_put_field: +bt_ctf_field_put (field); +return -1 ; +} + +static int add_tracepoint_fields_values (struct ctf_writer *cw, +struct bt_ctf_event_class *event_class, +struct bt_ctf_event *event, +struct format_field *fields, +struct perf_sample *sample) +{ +struct format_field *field; +int ret; + +for (field = fields; field; field = field->next) { + ret =add_tracepoint_field_value (cw, event_class, event, sample, + field); +if (ret) +return -1 ; + } +return 0 ; +} + +static int add_tracepoint_values (struct ctf_writer *cw, +struct bt_ctf_event_class *event_class, +struct bt_ctf_event *event, +struct perf_evsel *evsel, +struct perf_sample *sample) +{ +struct format_field *common_fields, *fields; +int ret; + + common_fields = evsel->tp_format->format.common_fields; + fields = evsel->tp_format->format.fields; + + ret =add_tracepoint_fields_values (cw, event_class, event, + common_fields, sample); +if (!ret) + ret =add_tracepoint_fields_values (cw, event_class, event, + fields, sample); + +return ret; +} + static int add_generic_values(struct ctf_writer *cw, struct bt_ctf_event *event, struct perf_evsel *evsel,
@@ -246,11 +419,78 @@ static int process_sample_event(struct perf_tool *tool,if (ret)return -1 ; +if (evsel->attr.type ==PERF_TYPE_TRACEPOINT ) { + ret =add_tracepoint_values (cw, event_class, event, + evsel, sample); +if (ret) +return -1 ; + } +bt_ctf_stream_append_event (cw->stream, event);bt_ctf_event_put (event);return 0 ; } +static int add_tracepoint_fields_types (struct ctf_writer *cw, +struct format_field *fields, +struct bt_ctf_event_class *event_class) +{ +struct format_field *field; +int ret; + +for (field = fields; field; field = field->next) { +struct bt_ctf_field_type *type; +unsigned long flags = field->flags; + +pr2 (" field '%s'\n" , field->name); + + type =get_tracepoint_field_type (cw, field); +if (!type) +return -1 ; + +/* +* A string is an array of chars. For this we use the string +* type and don't care that it is an array. What we don't +* support is an array of strings. +*/ +if (flags &FIELD_IS_STRING ) + flags &= ~FIELD_IS_ARRAY ; + +if (flags &FIELD_IS_ARRAY ) + type =bt_ctf_field_type_array_create (type, field->arraylen); + + ret =bt_ctf_event_class_add_field (event_class, type, + field->name); + +if (flags &FIELD_IS_ARRAY ) +bt_ctf_field_type_put (type); + +if (ret) { +pr_err ("Failed to add field '%s\n" , field->name); +return -1 ; + } + } + +return 0 ; +} + +static int add_tracepoint_types (struct ctf_writer *cw, +struct perf_evsel *evsel, +struct bt_ctf_event_class *class) +{ +struct format_field *common_fields, *fields; +int ret; + + common_fields = evsel->tp_format->format.common_fields; + fields = evsel->tp_format->format.fields; + + ret =add_tracepoint_fields_types (cw, common_fields, class); +if (!ret) + ret =add_tracepoint_fields_types (cw, fields, class); + +return ret; +} +static int add_generic_types (struct ctf_writer *cw,struct perf_evsel *evsel,struct bt_ctf_event_class *event_class) {
@@ -328,6 +568,12 @@ static int add_event(struct ctf_writer *cw, struct perf_evsel *evsel)if (ret) goto err; +if (evsel->attr.type ==PERF_TYPE_TRACEPOINT ) { + ret =add_tracepoint_types (cw, evsel, event_class); +if (ret) + goto err; + } + ret =bt_ctf_stream_class_add_event_class (cw->stream_class, event_class);if (ret) {pr ("Failed to add event class into stream.\n" );
Signature · 1 line
1.8.3.1
Quoted 22 lines · Jiri Olsa
hi,
this is initial post of the CTF converter for perf. Basically adding 'perf data convert' command to allow conversion of perf data file into CTF [1] data.To convert perf data file run:
$ perf data convert --to-ctf=./ctf-data/ [ perf data convert: Converted 'perf.data' into CTF data './ctf-data/' ] [ perf data convert: Converted and wrote 11.268 MB (100230 samples) ]To display converted CTF data run [2]:
$ babeltrace ./ctf-data/ [03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 } [03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 } [03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 } [03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 } [03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 } [03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 } [03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 } [03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }
Quoted 8 lines
It has still some limitations (noted in patches), but it has reached the point of the useful perf feature. More features and supports are about to come and it'll be better to continue publicly.
Also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/core_ctf_convert
so I've put perf converted CTF stream into the eclipse CTF trace viewer and got it displayed with the 'Generic CTF trace' type
The other type 'LTTng kernel trace' seems to set some rules for the CTF fields/format the trace has to obey. Is this described somewhere?
thanks,
jirka