perf tools: Add perf data CTF conversion

Commit-ID:  53d0a57343949b2af9b27229db534b98e5a0c4d0
Gitweb:     http://git.kernel.org/tip/53d0a57343949b2af9b27229db534b98e5a0c4d0
Author:     Jiri Olsa <[email protected]>
AuthorDate: Fri, 20 Feb 2015 23:16:58 +0100
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Wed, 25 Feb 2015 12:42:24 -0300

perf tools: Add feature check for libbabeltrace

Adding feature check for babeltrace library [1], which will be used for perf data file CTF [2] conversion in following patches.

Quoted 10 lines · tip-bot for Jiri Olsa

Commit-ID:  53d0a57343949b2af9b27229db534b98e5a0c4d0
Gitweb:     http://git.kernel.org/tip/53d0a57343949b2af9b27229db534b98e5a0c4d0
Author:     Jiri Olsa <[email protected]>
AuthorDate: Fri, 20 Feb 2015 23:16:58 +0100
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Wed, 25 Feb 2015 12:42:24 -0300

perf tools: Add feature check for libbabeltrace > >
Adding feature check for babeltrace library [1], which will be used for perf data file CTF [2] conversion in following patches.

So this doesn't work on recent Ubuntu systems:

... libbabeltrace: [ OFF ]

triton:~/tip/tools/perf/config/feature-checks> make
test-libbabeltrace.bin
gcc -MD -Wall -Werror -o test-libbabeltrace.bin test-libbabeltrace.c
# -lbabeltrace provided by
test-libbabeltrace.c: In function ‘main’:test-libbabeltrace.c:6:2: error: implicit declaration of function ‘bt_ctf_stream_class_get_packet_context_type’ [-Werror=implicit-function-declaration]
bt_ctf_stream_class_get_packet_context_type((void *) 0);
^
cc1: all warnings being treated as errors
Makefile:140: recipe for target 'test-libbabeltrace.bin' failed

I have all the requirements, libbabeltrace-ctf-dev and libbabeltrace-dev installed, yet the writer.h file:

-rw-r--r-- 1 root root 5283 Oct 7 14:44 /usr/include/babeltrace/ctf-writer/writer.h

does not include a prototype for
bt_ctf_stream_class_get_packet_context_type().

If Ubuntu's writer.h is too old then either our API use should match the ones available:

triton:~/tip/tools/perf/config/feature-checks> grep extern /usr/include/babeltrace/ctf-writer/writer.h
extern "C" {
extern struct bt_ctf_writer *bt_ctf_writer_create(const char *path); extern struct bt_ctf_stream *bt_ctf_writer_create_stream(
extern int bt_ctf_writer_add_environment_field(struct bt_ctf_writer *writer, extern int bt_ctf_writer_add_clock(struct bt_ctf_writer *writer, extern char *bt_ctf_writer_get_metadata_string(struct bt_ctf_writer *writer); extern void bt_ctf_writer_flush_metadata(struct bt_ctf_writer *writer); extern int bt_ctf_writer_set_byte_order(struct bt_ctf_writer *writer, extern void bt_ctf_writer_get(struct bt_ctf_writer *writer); extern void bt_ctf_writer_put(struct bt_ctf_writer *writer);

or CTF support should be removed from the default set of perf features, because right now it blocks the 'all' build speedup.

From: Sebastian Andrzej Siewior <[email protected]> Adding support to convert tracepoint event fields into CTF event fields.

We parse each tracepoint event for CTF conversion and add tracepoint fields as regular CTF event fields, so they appear in babeltrace output like:

  $ babeltrace ./ctf-data/
  ...
  [09:02:00.950703057] (+?.?????????) sched:sched_stat_runtime: { }, { perf_ip = ... SNIP ... common_type = 298, common_flags = 1, \
  common_preempt_count = 0, common_pid = 31813, comm = "perf", pid = 31813, runtime = 458800, vruntime = 52059858071 }
  ...

Em Fri, Feb 20, 2015 at 11:17:02PM +0100, Jiri Olsa escreveu:

Quoted 8 lines

From: Sebastian Andrzej Siewior <[email protected]> Adding support to convert tracepoint event fields into CTF event fields.

We parse each tracepoint event for CTF conversion and add tracepoint fields as regular CTF event fields, so they appear in babeltrace output like:

Quoted 5 lines

  $ babeltrace ./ctf-data/
  ...
  [09:02:00.950703057] (+?.?????????) sched:sched_stat_runtime: { }, { perf_ip = ... SNIP ... common_type = 298, common_flags = 1, \
  common_preempt_count = 0, common_pid = 31813, comm = "perf", pid = 31813, runtime = 458800, vruntime = 52059858071 }
  ...

Processed the previous patches, everything ok:

[acme@ssdandy linux]$ ls -la perf.data
ls: cannot access perf.data: No such file or directory
[acme@ssdandy linux]$ trace record usleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.029 MB perf.data (88 samples) ]
[acme@ssdandy linux]$ ls -la perf.data
-rw-------. 1 acme acme 5399896 Fev 25 16:19 perf.data
[acme@ssdandy linux]$ perf evlist
raw_syscalls:sys_enter
raw_syscalls:sys_exit
[acme@ssdandy linux]$ perf evlist -v
raw_syscalls:sys_enter: sample_freq=1, type: 2, config: 75, size: 104, sample_type: IP|TID|TIME|ID|CPU|PERIOD|RAW, read_format: ID, disabled: 1, inherit: 1, mmap: 1, mmap2: 1, comm: 1, comm_exec: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, exclude_guest: 1 raw_syscalls:sys_exit: sample_freq=1, type: 2, config: 74, size: 104, sample_type: IP|TID|TIME|ID|CPU|PERIOD|RAW, read_format: ID, disabled: 1, inherit: 1, enable_on_exec: 1, sample_id_all: 1, exclude_guest: 1 [acme@ssdandy linux]$ perf data convert --to-ctf=./ctf-data/
[ perf data convert: Converted 'perf.data' into CTF data './ctf-data/' ]
[ perf data convert: Converted and wrote 0.009 MB (88 samples) ]
[acme@ssdandy linux]$ babeltrace ./ctf-data/
[23:48:47.557933780] (+?.?????????) raw_syscalls:sys_exit: { }, { perf_ip = 0xFFFFFFFF81020FBA, perf_tid = 5093, perf_pid = 5093, perf_id = 1512, perf_cpu = 3, perf_period = 1 }
[23:48:47.557957461] (+0.000023681) raw_syscalls:sys_enter: { }, { perf_ip = 0xFFFFFFFF81020EA6, perf_tid = 5093, perf_pid = 5093, perf_id = 1504, perf_cpu = 3, perf_period = 1 }
[23:48:47.557958406] (+0.000000945) raw_syscalls:sys_exit: { }, { perf_ip = 0xFFFFFFFF81020FBA, perf_tid = 5093, perf_pid = 5093, perf_id = 1512, perf_cpu = 3, perf_period = 1 }
[23:48:47.557973567] (+0.000015161) raw_syscalls:sys_enter: { }, { perf_ip = 0xFFFFFFFF81020EA6, perf_tid = 5093, perf_pid = 5093, perf_id = 1504, perf_cpu = 3, perf_period = 1 }
[23:48:47.557976047] (+0.000002480) raw_syscalls:sys_exit: { }, { perf_ip = 0xFFFFFFFF81020FBA, perf_tid = 5093, perf_pid = 5093, perf_id = 1512, perf_cpu = 3, perf_period = 1 }
[23:48:47.557985774] (+0.000009727) raw_syscalls:sys_enter: { }, { perf_ip = 0xFFFFFFFF81020EA6, perf_tid = 5093, perf_pid = 5093, perf_id = 1504, perf_cpu = 3, perf_period = 1 }
[23:48:47.557990826] (+0.000005052) raw_syscalls:sys_exit: { }, { perf_ip = 0xFFFFFFFF81020FBA, perf_tid = 5093, perf_pid = 5093, perf_id = 1512, perf_cpu = 3, perf_period = 1 }
<SNIP>

But then I apply this patch (convert tracepoint events fields into CTF event fields) and:

[acme@ssdandy linux]$ perf data convert --to-ctf=./ctf-data/
[ perf data convert: Converted 'perf.data' into CTF data './ctf-data/' ]
[ perf data convert: Converted and wrote 0.009 MB (88 samples) ]
[acme@ssdandy linux]$ babeltrace ./ctf-data/
[error] Packet size (18446744073709551615 bits) is larger than remaining file size (262144 bits). [error] Stream index creation error.
[error] Open file stream error.
[warning] [Context] Cannot open_trace of format ctf at path ./ctf-data.
[warning] [Context] cannot open trace "./ctf-data" from ./ctf-data/ for reading. [error] Cannot open any trace for reading.

[error] opening trace "./ctf-data/" for reading.

[error] none of the specified trace paths could be opened.

[acme@ssdandy linux]$

It stops working.

[acme@ssdandy linux]$ ls -la ctf-data/
total 44
drwxrwx---. 2 acme acme 41 Fev 25 16:12 .
drwxrwxr-x. 28 acme acme 4096 Fev 25 16:19 ..

(quoted diff) +0 -2

-rw-rw----.  1 acme acme  4666 Fev 25 16:21 metadata
-rw-rw----.  1 acme acme 32768 Fev 25 16:21 perf_stream_0

[acme@ssdandy linux]$

Can you try to reproduce this? The ctf-data/metadata file is below:

Quoted 72 lines

Em Fri, Feb 20, 2015 at 11:17:02PM +0100, Jiri Olsa escreveu:

Quoted 8 lines

From: Sebastian Andrzej Siewior <[email protected]> Adding support to convert tracepoint event fields into CTF event fields.

We parse each tracepoint event for CTF conversion and add tracepoint fields as regular CTF event fields, so they appear in babeltrace output like:

Quoted 5 lines

  $ babeltrace ./ctf-data/
  ...
  [09:02:00.950703057] (+?.?????????) sched:sched_stat_runtime: { }, { perf_ip = ... SNIP ... common_type = 298, common_flags = 1, \
  common_preempt_count = 0, common_pid = 31813, comm = "perf", pid = 31813, runtime = 458800, vruntime = 52059858071 }
  ...

Processed the previous patches, everything ok:

[acme@ssdandy linux]$ ls -la perf.data
ls: cannot access perf.data: No such file or directory
[acme@ssdandy linux]$ trace record usleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.029 MB perf.data (88 samples) ]
[acme@ssdandy linux]$ ls -la perf.data
-rw-------. 1 acme acme 5399896 Fev 25 16:19 perf.data
[acme@ssdandy linux]$ perf evlist
raw_syscalls:sys_enter
raw_syscalls:sys_exit
[acme@ssdandy linux]$ perf evlist -v
raw_syscalls:sys_enter: sample_freq=1, type: 2, config: 75, size: 104, sample_type: IP|TID|TIME|ID|CPU|PERIOD|RAW, read_format: ID, disabled: 1, inherit: 1, mmap: 1, mmap2: 1, comm: 1, comm_exec: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, exclude_guest: 1 raw_syscalls:sys_exit: sample_freq=1, type: 2, config: 74, size: 104, sample_type: IP|TID|TIME|ID|CPU|PERIOD|RAW, read_format: ID, disabled: 1, inherit: 1, enable_on_exec: 1, sample_id_all: 1, exclude_guest: 1 [acme@ssdandy linux]$ perf data convert --to-ctf=./ctf-data/
[ perf data convert: Converted 'perf.data' into CTF data './ctf-data/' ]
[ perf data convert: Converted and wrote 0.009 MB (88 samples) ]
[acme@ssdandy linux]$ babeltrace ./ctf-data/
[23:48:47.557933780] (+?.?????????) raw_syscalls:sys_exit: { }, { perf_ip = 0xFFFFFFFF81020FBA, perf_tid = 5093, perf_pid = 5093, perf_id = 1512, perf_cpu = 3, perf_period = 1 }
[23:48:47.557957461] (+0.000023681) raw_syscalls:sys_enter: { }, { perf_ip = 0xFFFFFFFF81020EA6, perf_tid = 5093, perf_pid = 5093, perf_id = 1504, perf_cpu = 3, perf_period = 1 }
[23:48:47.557958406] (+0.000000945) raw_syscalls:sys_exit: { }, { perf_ip = 0xFFFFFFFF81020FBA, perf_tid = 5093, perf_pid = 5093, perf_id = 1512, perf_cpu = 3, perf_period = 1 }
[23:48:47.557973567] (+0.000015161) raw_syscalls:sys_enter: { }, { perf_ip = 0xFFFFFFFF81020EA6, perf_tid = 5093, perf_pid = 5093, perf_id = 1504, perf_cpu = 3, perf_period = 1 }
[23:48:47.557976047] (+0.000002480) raw_syscalls:sys_exit: { }, { perf_ip = 0xFFFFFFFF81020FBA, perf_tid = 5093, perf_pid = 5093, perf_id = 1512, perf_cpu = 3, perf_period = 1 }
[23:48:47.557985774] (+0.000009727) raw_syscalls:sys_enter: { }, { perf_ip = 0xFFFFFFFF81020EA6, perf_tid = 5093, perf_pid = 5093, perf_id = 1504, perf_cpu = 3, perf_period = 1 }
[23:48:47.557990826] (+0.000005052) raw_syscalls:sys_exit: { }, { perf_ip = 0xFFFFFFFF81020FBA, perf_tid = 5093, perf_pid = 5093, perf_id = 1512, perf_cpu = 3, perf_period = 1 }
<SNIP>

But then I apply this patch (convert tracepoint events fields into CTF event fields) and:

[acme@ssdandy linux]$ perf data convert --to-ctf=./ctf-data/
[ perf data convert: Converted 'perf.data' into CTF data './ctf-data/' ]
[ perf data convert: Converted and wrote 0.009 MB (88 samples) ]
[acme@ssdandy linux]$ babeltrace ./ctf-data/
[error] Packet size (18446744073709551615 bits) is larger than remaining file size (262144 bits). [error] Stream index creation error.
[error] Open file stream error.
[warning] [Context] Cannot open_trace of format ctf at path ./ctf-data.
[warning] [Context] cannot open trace "./ctf-data" from ./ctf-data/ for reading. [error] Cannot open any trace for reading.

[error] opening trace "./ctf-data/" for reading.

[error] none of the specified trace paths could be opened.

[acme@ssdandy linux]$

It stops working.

[acme@ssdandy linux]$ ls -la ctf-data/
total 44
drwxrwx---. 2 acme acme 41 Fev 25 16:12 .
drwxrwxr-x. 28 acme acme 4096 Fev 25 16:19 ..

(quoted diff) +0 -2

-rw-rw----.  1 acme acme  4666 Fev 25 16:21 metadata
-rw-rw----.  1 acme acme 32768 Fev 25 16:21 perf_stream_0

[acme@ssdandy linux]$

Quoted 1 line

Can you try to reproduce this? The ctf-data/metadata file is below:

hum, i just tried and can't reproduce this one.. anychance of mixed babeltrace installations? How did you install babeltrace sources?

I assume this was some standard Fedora latest something system right?

jirka

git HEAD:

[root@krava perf]# git log --oneline | head -3
9534f89 perf data: Add tracepoint events fields CTF conversion support bb848d1 perf stat: Report unsupported events properly ffdfa43 perf tools: Compare JOBS to 0 after grep

test:

[root@krava perf]# LD_LIBRARY_PATH=/opt/libbabeltrace/lib/ ./perf trace record usleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.023 MB perf.data (88 samples) ]
[root@krava perf]# LD_LIBRARY_PATH=/opt/libbabeltrace/lib/ ./perf data convert --to-ctf=./ctf-data
[ perf data convert: Converted 'perf.data' into CTF data './ctf-data' ]
[ perf data convert: Converted and wrote 0.009 MB (88 samples) ]
[root@krava perf]# babeltrace ./ctf-data/
[10:04:03.320544407] (+?.?????????) raw_syscalls:sys_exit: { }, { perf_ip = 0xFFFFFFFF810235AC, perf_tid = 23899, perf_pid = 23899, perf_id = 82, perf_cpu = 3, perf_period = 1, common_type = 75, common_flags = 0, common_preempt_count = 0, common_pid = 23899, id = 59, ret = 0 } [10:04:03.320568054] (+0.000023647) raw_syscalls:sys_enter: { }, { perf_ip = 0xFFFFFFFF81023420, perf_tid = 23899, perf_pid = 23899, perf_id = 78, perf_cpu = 3, perf_period = 1, common_type = 76, common_flags = 0, common_preempt_count = 0, common_pid = 23899, id = 12, args = [ [0] = 0x0, [1] = 0x0, [2] = 0x0, [3] = 0x0, [4] = 0x37F, [5] = 0x64 ] } [10:04:03.320568993] (+0.000000939) raw_syscalls:sys_exit: { }, { perf_ip = 0xFFFFFFFF810235AC, perf_tid = 23899, perf_pid = 23899, perf_id = 82, perf_cpu = 3, perf_period = 1, common_type = 75, common_flags = 0, common_preempt_count = 0, common_pid = 23899, id = 12, ret = 24440832 }
...

Quoted 5 lines

Quoted 2 lines

Can you try to reproduce this? The ctf-data/metadata file is below:

hum, i just tried and can't reproduce this one.. anychance of mixed babeltrace installations? How did you install babeltrace sources?

I will recheck

Quoted 1 line

I assume this was some standard Fedora latest something system right?

This was on RHEL7

SNIP

Quoted 12 lines

Quoted 5 lines

Quoted 2 lines

Can you try to reproduce this? The ctf-data/metadata file is below:

hum, i just tried and can't reproduce this one.. anychance of mixed babeltrace installations? How did you install babeltrace sources?

I will recheck

Quoted 1 line

I assume this was some standard Fedora latest something system right?

This was on RHEL7

hum, how did you compile babeltrace on RHEL7 im getting following error:

[root@intel-s3ea2-09 babeltrace]# ./bootstrap
+ '[' '!' -e config ']'
+ libtoolize --force --copy
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `config'.
libtoolize: copying file `config/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'

libtoolize: copying file `m4/lt~obsolete.m4'
+ autoreconf -i
configure.ac:48: warning: macro 'AM_PATH_GLIB_2_0' not found in library configure.ac:48: warning: macro 'AM_PATH_GLIB_2_0' not found in library configure.ac:48: error: possibly undefined macro: AM_PATH_GLIB_2_0
If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation.
autoreconf: /usr/bin/autoconf failed with exit status: 1

Doesn't work as well...

:-\

Will try debugging...

Em Mon, Mar 09, 2015 at 06:51:21PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 4 lines

Doesn't work as well...

:-\

Quoted 1 line

Will try debugging...

So I tried checkout out before Ingo's changes to libbabeltrace detection, i.e.:

[acme@zoo linux]$ git checkout -b ttmp 9a75606ca06d94aab1ed0dbe96935e3f89dfb81c

And it works:

[acme@zoo linux]$ git bisect bad b49f1a4be701c2386ccc7496dc8442cf26424d5c is the first bad commit commit b49f1a4be701c2386ccc7496dc8442cf26424d5c

Author: Ingo Molnar <[email protected]>
Date:   Sat Feb 28 10:16:27 2015 +0100

perf tools: Improve feature test debuggability

Certain feature tests fail with link errors:

--------------------------------------------------------------

Ran out of time, will continue later/tomorrow.

Em Mon, Mar 09, 2015 at 08:11:19PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 1 line

Em Mon, Mar 09, 2015 at 06:51:21PM -0300, Arnaldo Carvalho de Melo escreveu:

[root@zoo ~]# find /opt/libbabeltrace/include/babeltrace/ -type f | xargs grep bt_ctf_stream_class_get_packet_context_type /opt/libbabeltrace/include/babeltrace/ctf-ir/stream-class.h: * bt_ctf_stream_class_get_packet_context_type: get the stream class' packet /opt/libbabeltrace/include/babeltrace/ctf-ir/stream-class.h:extern struct bt_ctf_field_type *bt_ctf_stream_class_get_packet_context_type( [root@zoo ~]#

That seems to be included from the file included in the feature test code :-\

Really ran out of time now...

Quoted 7 lines

Em Mon, Mar 09, 2015 at 08:11:19PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 1 line

Em Mon, Mar 09, 2015 at 06:51:21PM -0300, Arnaldo Carvalho de Melo escreveu:

[root@zoo ~]# find /opt/libbabeltrace/include/babeltrace/ -type f | xargs grep bt_ctf_stream_class_get_packet_context_type /opt/libbabeltrace/include/babeltrace/ctf-ir/stream-class.h:extern struct bt_ctf_field_type *bt_ctf_stream_class_get_packet_context_type(

That seems to be included from the file included in the feature test code :-\
Really ran out of time now...

ouch, thanks a lot for debugging.. the reason is indeed the -Werror added by:
b49f1a4be701 perf tools: Improve feature test debuggability

following patch seems to fix this for me
---
Following patch added -Werror for feature builds:
b49f1a4be701 perf tools: Improve feature test debuggability

and broke libbabeltrace feature build, because it was including wrong header and gcc couldn't find the used symbol definition.

Adding proper header and keeping the old one as it is needed also (libbabeltrace quirk).

+++ b/tools/perf/config/feature-checks/test-libbabeltrace.c
#include <babeltrace/ctf-writer/writer.h>
+#include <babeltrace/ctf-ir/stream-class.h>

int main(void)
{

Signature · 1 line

1.9.3

Em Tue, Mar 10, 2015 at 01:00:35PM +0100, Jiri Olsa escreveu:

Quoted 22 lines · Arnaldo Carvalho de Melo

Quoted 7 lines

Em Mon, Mar 09, 2015 at 08:11:19PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 1 line

Em Mon, Mar 09, 2015 at 06:51:21PM -0300, Arnaldo Carvalho de Melo escreveu:

[root@zoo ~]# find /opt/libbabeltrace/include/babeltrace/ -type f | xargs grep bt_ctf_stream_class_get_packet_context_type /opt/libbabeltrace/include/babeltrace/ctf-ir/stream-class.h:extern struct bt_ctf_field_type *bt_ctf_stream_class_get_packet_context_type(

That seems to be included from the file included in the feature test code :-\
Really ran out of time now...

ouch, thanks a lot for debugging.. the reason is indeed the -Werror added by:
b49f1a4be701 perf tools: Improve feature test debuggability

following patch seems to fix this for me
---
Following patch added -Werror for feature builds:
b49f1a4be701 perf tools: Improve feature test debuggability

and broke libbabeltrace feature build, because it was including wrong header and gcc couldn't find the used symbol definition.

Adding proper header and keeping the old one as it is needed also (libbabeltrace quirk).

Quoted 3 lines

+++ b/tools/perf/config/feature-checks/test-libbabeltrace.c
#include <babeltrace/ctf-writer/writer.h>
+#include <babeltrace/ctf-ir/stream-class.h>

Right, that was the fix I found as well, well, I was trying to include _just_ babeltrace/ctf-ir/stream-class.h, but that is buggy in that it doesn't include the header file with the uint32_t and int64_t types:

[acme@zoo linux]$ cat /tmp/build/perf/config/feature-checks/make-libbabeltrace
make[1]: Entering directory `/home/git/linux/tools/perf/config/feature-checks'
gcc -MD -I/opt/libbabeltrace//include -Wall -Werror -o /tmp/build/perf/config/feature-checks/test-libbabeltrace.bin test-libbabeltrace.c -Wl,-z,noexecstack -L/opt/libbabeltrace//lib -lbabeltrace-ctf # -lbabeltrace provided by In file included from test-libbabeltrace.c:2:0:
/opt/libbabeltrace//include/babeltrace/ctf-ir/stream-class.h:118:1: error: unknown type name ‘int64_t’
extern int64_t bt_ctf_stream_class_get_id(
^
/opt/libbabeltrace//include/babeltrace/ctf-ir/stream-class.h:134:45: error: unknown type name ‘uint32_t’

   struct bt_ctf_stream_class *stream_class, uint32_t id);
                                             ^

make[1]: *** [test-libbabeltrace.bin] Error 1
make[1]: Leaving directory `/home/git/linux/tools/perf/config/feature-checks' [acme@zoo linux]$

Quoted 5 lines

int main(void)
{

Signature · 1 line

1.9.3

Ah, I found the following patch useful in debugging this, if you're ok with it, I'll put it in my tree with your Acked-by, ok?

- Arnaldo

[acme@zoo linux]$ git diff tools/perf/config/Makefile

tools/perf/config/Makefile +0 -0

index e2350ada6ccd..4992c39652e6 100644

Quoted 24 lines · Arnaldo Carvalho de Melo

Em Tue, Mar 10, 2015 at 01:00:35PM +0100, Jiri Olsa escreveu:

Quoted 22 lines · Arnaldo Carvalho de Melo

Quoted 7 lines

Em Mon, Mar 09, 2015 at 08:11:19PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 1 line

Em Mon, Mar 09, 2015 at 06:51:21PM -0300, Arnaldo Carvalho de Melo escreveu:

[root@zoo ~]# find /opt/libbabeltrace/include/babeltrace/ -type f | xargs grep bt_ctf_stream_class_get_packet_context_type /opt/libbabeltrace/include/babeltrace/ctf-ir/stream-class.h:extern struct bt_ctf_field_type *bt_ctf_stream_class_get_packet_context_type(

That seems to be included from the file included in the feature test code :-\
Really ran out of time now...

ouch, thanks a lot for debugging.. the reason is indeed the -Werror added by:
b49f1a4be701 perf tools: Improve feature test debuggability

following patch seems to fix this for me
---
Following patch added -Werror for feature builds:
b49f1a4be701 perf tools: Improve feature test debuggability

and broke libbabeltrace feature build, because it was including wrong header and gcc couldn't find the used symbol definition.

Adding proper header and keeping the old one as it is needed also (libbabeltrace quirk).

Quoted 7 lines

Quoted 3 lines

+++ b/tools/perf/config/feature-checks/test-libbabeltrace.c
#include <babeltrace/ctf-writer/writer.h>
+#include <babeltrace/ctf-ir/stream-class.h>

Right, that was the fix I found as well, well, I was trying to include _just_ babeltrace/ctf-ir/stream-class.h, but that is buggy in that it doesn't include the header file with the uint32_t and int64_t types:

yep, thats the 'libbabeltrace quirk' I mentioned in the changelog ;-)

Quoted 32 lines

[acme@zoo linux]$ cat /tmp/build/perf/config/feature-checks/make-libbabeltrace
make[1]: Entering directory `/home/git/linux/tools/perf/config/feature-checks'
gcc -MD -I/opt/libbabeltrace//include -Wall -Werror -o /tmp/build/perf/config/feature-checks/test-libbabeltrace.bin test-libbabeltrace.c -Wl,-z,noexecstack -L/opt/libbabeltrace//lib -lbabeltrace-ctf # -lbabeltrace provided by In file included from test-libbabeltrace.c:2:0:
/opt/libbabeltrace//include/babeltrace/ctf-ir/stream-class.h:118:1: error: unknown type name ‘int64_t’
extern int64_t bt_ctf_stream_class_get_id(
^
/opt/libbabeltrace//include/babeltrace/ctf-ir/stream-class.h:134:45: error: unknown type name ‘uint32_t’

   struct bt_ctf_stream_class *stream_class, uint32_t id);
                                             ^

make[1]: *** [test-libbabeltrace.bin] Error 1
make[1]: Leaving directory `/home/git/linux/tools/perf/config/feature-checks' [acme@zoo linux]$

Quoted 5 lines

int main(void)
{
--
1.9.3

Ah, I found the following patch useful in debugging this, if you're ok with it, I'll put it in my tree with your Acked-by, ok?

- Arnaldo

[acme@zoo linux]$ git diff tools/perf/config/Makefile

tools/perf/config/Makefile +0 -0

index e2350ada6ccd..4992c39652e6 100644

tools/perf/config/Makefile endif · line 188

Quoted 6 lines

tools/perf/config/Makefile +1 -1

@@ -184,7 +184,7 @@ endif
 
 feature_check = $(eval $(feature_check_code))
 define feature_check_code
-  feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C config/feature-checks test-$1.bin >/dev/null 2>/dev/null && echo 1 ||
+  feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C config/feature-checks test-$1.bin > $(OUTPUT_FEATURES)make-$(1) 2>&1
 endef

ack, that seem usefull.. I always redirected this to '>> /tmp/krava' never got enough will to make it nice and generic ;-)

thanks,
jirka

Quoted 7 lines · Arnaldo Carvalho de Melo

Quoted 6 lines

Em Tue, Mar 10, 2015 at 01:00:35PM +0100, Jiri Olsa escreveu:

Quoted 4 lines

Adding proper header and keeping the old one as it is needed also (libbabeltrace quirk).

+#include <babeltrace/ctf-ir/stream-class.h>

Quoted 5 lines

Quoted 3 lines

Right, that was the fix I found as well, well, I was trying to include _just_ babeltrace/ctf-ir/stream-class.h, but that is buggy in that it doesn't include the header file with the uint32_t and int64_t types:

yep, thats the 'libbabeltrace quirk' I mentioned in the changelog ;-)

Seems I missed that being discussed. Fixed in Babeltrace master as of 48d711aa2.

Quoted 33 lines · Jiri Olsa

Quoted 31 lines · Arnaldo Carvalho de Melo

Em Tue, Mar 10, 2015 at 01:00:35PM +0100, Jiri Olsa escreveu:

Quoted 20 lines · Arnaldo Carvalho de Melo

Quoted 7 lines

Em Mon, Mar 09, 2015 at 08:11:19PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 1 line

Em Mon, Mar 09, 2015 at 06:51:21PM -0300, Arnaldo Carvalho de Melo escreveu:

[root@zoo ~]# find /opt/libbabeltrace/include/babeltrace/ -type f | xargs grep bt_ctf_stream_class_get_packet_context_type /opt/libbabeltrace/include/babeltrace/ctf-ir/stream-class.h:extern struct bt_ctf_field_type *bt_ctf_stream_class_get_packet_context_type(

That seems to be included from the file included in the feature test code :-\
Really ran out of time now...

ouch, thanks a lot for debugging.. the reason is indeed the -Werror added by:
b49f1a4be701 perf tools: Improve feature test debuggability

following patch seems to fix this for me
---
Following patch added -Werror for feature builds:
b49f1a4be701 perf tools: Improve feature test debuggability

and broke libbabeltrace feature build, because it was including wrong header and gcc couldn't find the used symbol definition.

Quoted 6 lines

Adding proper header and keeping the old one as it is needed also (libbabeltrace quirk).

+++ b/tools/perf/config/feature-checks/test-libbabeltrace.c
#include <babeltrace/ctf-writer/writer.h>
+#include <babeltrace/ctf-ir/stream-class.h>

Right, that was the fix I found as well, well, I was trying to include _just_ babeltrace/ctf-ir/stream-class.h, but that is buggy in that it doesn't include the header file with the uint32_t and int64_t types:

Quoted 2 lines

yep, thats the 'libbabeltrace quirk' I mentioned in the changelog ;-)

Seems I missed that being discussed. Fixed in Babeltrace master as of 48d711aa2.

Quoted 5 lines · Jiri Olsa

Quoted 2 lines · Arnaldo Carvalho de Melo

yep, thats the 'libbabeltrace quirk' I mentioned in the changelog ;-)

Seems I missed that being discussed. Fixed in Babeltrace master as of 48d711aa2.

that seems to work, but I could now load ctf trace data only in babeltrace. I've got following error in tracecompas while trying to load the trace:

"No trace types found to match location /tmp/ctf-data/metadata"

but I might have old version of tracecompas and Im little fuzzy on how to upgrade to new one.. CC-ing tracecompas guys ;-)

for anyone interested, the latest perf ctf code is in:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/ctf branch

Quoted 12 lines

Ah, I found the following patch useful in debugging this, if you're ok with it, I'll put it in my tree with your Acked-by, ok?

- Arnaldo

[acme@zoo linux]$ git diff tools/perf/config/Makefile

tools/perf/config/Makefile +0 -0

index e2350ada6ccd..4992c39652e6 100644
@@ -184,7 +184,7 @@ endif
 

tools/perf/config/Makefile endif · line 188

Quoted 5 lines

tools/perf/config/Makefile +1 -1

 feature_check = $(eval $(feature_check_code))
 define feature_check_code
-  feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C config/feature-checks test-$1.bin >/dev/null 2>/dev/null && echo 1 ||
+  feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C config/feature-checks test-$1.bin > $(OUTPUT_FEATURES)make-$(1) 2>&1 
 endef

ack, that seem usefull.. I always redirected this to '>> /tmp/krava' never got enough will to make it nice and generic ;-)

SNIP

Quoted 2 lines

endef

tools/perf/config/Makefile $(call detected_var,LIBDIR) · line 888

Quoted 11 lines

feature_set = $(eval $(feature_set_code))

tools/perf/config/Makefile +6 -0

@@ -880,3 +880,9 @@ $(call detected_var,LIBDIR)
 $(call detected_var,GTK_CFLAGS)
 $(call detected_var,PERL_EMBED_CCOPTS)
 $(call detected_var,PYTHON_EMBED_CCOPTS)
+
+CLEAN_FILES = $(OUTPUT_FEATURES)make-*.feature-test.output
+
+clean:
+	$(call QUIET_CLEAN, config) $(RM) $(CLEAN_FILES)
+

hum, I think it'd be better if it stay in feature-checks dir, I'll try to send patch for this as part of the v2 build patchset

Quoted 8 lines

Em Tue, Mar 10, 2015 at 01:00:35PM +0100, Jiri Olsa escreveu:

Quoted 3 lines · Arnaldo Carvalho de Melo

Quoted 1 line

Em Mon, Mar 09, 2015 at 08:11:19PM -0300, Arnaldo Carvalho de Melo escreveu:

+#include <babeltrace/ctf-ir/stream-class.h>

Right, that was the fix I found as well, well, I was trying to include _just_ babeltrace/ctf-ir/stream-class.h, but that is buggy in that it doesn't include the header file with the uint32_t and int64_t types:

yep, thats the 'libbabeltrace quirk' I mentioned in the changelog ;-)

tools/perf/config/Makefile endif · line 188

Quoted 8 lines

+++ b/tools/perf/config/Makefile

tools/perf/config/Makefile +1 -1

@@ -184,7 +184,7 @@ endif
 
 feature_check = $(eval $(feature_check_code))
 define feature_check_code
-  feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C config/feature-checks test-$1.bin >/dev/null 2>/dev/null && echo 1 ||
+  feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C config/feature-checks test-$1.bin > $(OUTPUT_FEATURES)make-$(1) 2>&1 
 endef

ack, that seem usefull.. I always redirected this to '>> /tmp/krava' never got enough will to make it nice and generic ;-)

From: Sebastian Andrzej Siewior <[email protected]>

Currently we store the data into single data strea/file. The cpu if data is stored within the event sample. The lttng puts the CPU number that belongs to the event into the packet context instead into the event.

This patch makes sure that the trace produce by perf does look the same way. We now use one stream per-CPU. Having it all in one stream increased the total size of the resulting file. The test went from 416KiB (with perf_cpu event member) to 24MiB due to the required (and pointless) flush. With the per-cpu streams the total size went up to 588KiB.

Cc Arnaldo Carvalho de Melo <[email protected]>

Cc David Ahern <[email protected]>

Cc Frederic Weisbecker <[email protected]>

Cc Jeremie Galarneau <[email protected]>

Cc Jiri Olsa <[email protected]>

Cc Namhyung Kim <[email protected]>

Cc Paul Mackerras <[email protected]>

Cc Peter Zijlstra <[email protected]>

Cc Sebastian Andrzej Siewior <[email protected]>

Cc Tom Zanussi <[email protected]>

Cc Wang Nan <[email protected]>

Signed-off-by Jiri Olsa <[email protected]>

Signed-off-by Sebastian Andrzej Siewior <[email protected]>

---
 tools/perf/util/data-convert-bt.c | 205 +++++++++++++++++++++++++++++++++-----
 1 file changed, 181 insertions(+), 24 deletions(-)

tools/perf/util/data-convert-bt.c +181 -24

index 6fa5c3ef336b..4bb769e081a8 100644
@@ -38,12 +38,20 @@ struct evsel_priv {
 	struct bt_ctf_event_class *event_class;
 };
 
+#define MAX_CPUS	4096
+
+struct ctf_stream {
+	struct bt_ctf_stream *stream;
+	int cpu;
+};
+
 struct ctf_writer {
 	/* writer primitives */
-	struct bt_ctf_writer		*writer;
-	struct bt_ctf_stream		*stream;
-	struct bt_ctf_stream_class	*stream_class;
-	struct bt_ctf_clock		*clock;
+	struct bt_ctf_writer		 *writer;
+	struct ctf_stream		**stream;
+	int				  stream_cnt;
+	struct bt_ctf_stream_class	 *stream_class;
+	struct bt_ctf_clock		 *clock;
 
 	/* data types */
 	union {

@@ -346,12 +354,6 @@ static int add_generic_values(struct ctf_writer *cw,
 			return -1;
 	}
 
-	if (type & PERF_SAMPLE_CPU) {
-		ret = value_set_u32(cw, event, "perf_cpu", sample->cpu);
-		if (ret)
-			return -1;
-	}
-
 	if (type & PERF_SAMPLE_PERIOD) {
 		ret = value_set_u64(cw, event, "perf_period", sample->period);
 		if (ret)

@@ -381,6 +383,113 @@ static int add_generic_values(struct ctf_writer *cw,
 	return 0;
 }
 
+static int ctf_stream__flush(struct ctf_stream *cs)
+{
+	int err = 0;
+
+	if (cs) {
+		err = bt_ctf_stream_flush(cs->stream);
+		if (err)
+			pr_err("CTF stream %d flush failed\n", cs->cpu);
+
+		pr("Flush stream for cpu %d\n", cs->cpu);
+	}
+
+	return err;
+}
+
+static struct ctf_stream *ctf_stream__create(struct ctf_writer *cw, int cpu)
+{
+	struct ctf_stream *cs;
+	struct bt_ctf_field *pkt_ctx   = NULL;
+	struct bt_ctf_field *cpu_field = NULL;
+	struct bt_ctf_stream *stream   = NULL;
+	int ret;
+
+	cs = zalloc(sizeof(*cs));
+	if (!cs) {
+		pr_err("Failed to allocate ctf stream\n");
+		return NULL;
+	}
+
+	stream = bt_ctf_writer_create_stream(cw->writer, cw->stream_class);
+	if (!stream) {
+		pr_err("Failed to create CTF stream\n");
+		goto out;
+	}
+
+	pkt_ctx = bt_ctf_stream_get_packet_context(stream);
+	if (!pkt_ctx) {
+		pr_err("Failed to obtain packet context\n");
+		goto out;
+	}
+
+	cpu_field = bt_ctf_field_structure_get_field(pkt_ctx, "cpu_id");
+	bt_ctf_field_put(pkt_ctx);
+	if (!cpu_field) {
+		pr_err("Failed to obtain cpu field\n");
+		goto out;
+	}
+
+	ret = bt_ctf_field_unsigned_integer_set_value(cpu_field, (u32) cpu);
+	if (ret) {
+		pr_err("Failed to update CPU number\n");
+		goto out;
+	}
+
+	bt_ctf_field_put(cpu_field);
+
+	cs->cpu    = cpu;
+	cs->stream = stream;
+	return cs;
+
+out:
+	if (cpu_field)
+		bt_ctf_field_put(cpu_field);
+	if (stream)
+		bt_ctf_stream_put(stream);
+
+	free(cs);
+	return NULL;
+}
+
+static void ctf_stream__delete(struct ctf_stream *cs)
+{
+	if (cs) {
+		bt_ctf_stream_put(cs->stream);
+		free(cs);
+	}
+}
+
+static struct ctf_stream *ctf_stream(struct ctf_writer *cw, int cpu)
+{
+	struct ctf_stream *cs = cw->stream[cpu];
+
+	if (!cs) {
+		cs = ctf_stream__create(cw, cpu);
+		cw->stream[cpu] = cs;
+	}
+
+	return cs;
+}
+
+static int get_sample_cpu(struct ctf_writer *cw, struct perf_sample *sample,
+			  struct perf_evsel *evsel)
+{
+	int cpu = 0;
+
+	if (evsel->attr.sample_type & PERF_SAMPLE_CPU)
+		cpu = sample->cpu;
+
+	if (cpu > cw->stream_cnt) {
+		pr_err("Event was recorded for CPU %d, limit is at %d.\n",
+			cpu, cw->stream_cnt);
+		cpu = 0;
+	}
+
+	return cpu;
+}
+
 static int process_sample_event(struct perf_tool *tool,
 				union perf_event *_event __maybe_unused,
 				struct perf_sample *sample,

@@ -390,6 +499,7 @@ static int process_sample_event(struct perf_tool *tool,
 	struct convert *c = container_of(tool, struct convert, tool);
 	struct evsel_priv *priv = evsel->priv;
 	struct ctf_writer *cw = &c->writer;
+	struct ctf_stream *cs;
 	struct bt_ctf_event_class *event_class;
 	struct bt_ctf_event *event;
 	int ret;

@@ -424,9 +534,12 @@ static int process_sample_event(struct perf_tool *tool,
 			return -1;
 	}
 
-	bt_ctf_stream_append_event(cw->stream, event);
+	cs = ctf_stream(cw, get_sample_cpu(cw, sample, evsel));
+	if (cs)
+		bt_ctf_stream_append_event(cs->stream, event);
+
 	bt_ctf_event_put(event);
-	return 0;
+	return cs ? 0 : -1;
 }
 
 static int add_tracepoint_fields_types(struct ctf_writer *cw,

@@ -528,9 +641,6 @@ static int add_generic_types(struct ctf_writer *cw, struct perf_evsel *evsel,
 	if (type & PERF_SAMPLE_STREAM_ID)
 		ADD_FIELD(event_class, cw->data.u64, "perf_stream_id");
 
-	if (type & PERF_SAMPLE_CPU)
-		ADD_FIELD(event_class, cw->data.u32, "perf_cpu");
-
 	if (type & PERF_SAMPLE_PERIOD)
 		ADD_FIELD(event_class, cw->data.u64, "perf_period");
 

@@ -604,6 +714,39 @@ static int setup_events(struct ctf_writer *cw, struct perf_session *session)
 	return 0;
 }
 
+static int setup_streams(struct ctf_writer *cw, struct perf_session *session)
+{
+	struct ctf_stream **stream;
+	struct perf_header *ph = &session->header;
+	int ncpus;
+
+	/*
+	 * Try to get the number of cpus used in the data file,
+	 * if not present fallback to the MAX_CPUS.
+	 */
+	ncpus = ph->env.nr_cpus_avail ?: MAX_CPUS;
+
+	stream = zalloc(sizeof(*stream) * ncpus);
+	if (!stream) {
+		pr_err("Failed to allocate streams.\n");
+		return -ENOMEM;
+	}
+
+	cw->stream     = stream;
+	cw->stream_cnt = ncpus;
+	return 0;
+}
+
+static void free_streams(struct ctf_writer *cw)
+{
+	int cpu;
+
+	for (cpu = 0; cpu < cw->stream_cnt; cpu++)
+		ctf_stream__delete(cw->stream[cpu]);
+
+	free(cw->stream);
+}
+
 static int ctf_writer__setup_env(struct ctf_writer *cw,
 				 struct perf_session *session)
 {

@@ -713,7 +856,7 @@ static void ctf_writer__cleanup(struct ctf_writer *cw)
 	ctf_writer__cleanup_data(cw);
 
 	bt_ctf_clock_put(cw->clock);
-	bt_ctf_stream_put(cw->stream);
+	free_streams(cw);
 	bt_ctf_stream_class_put(cw->stream_class);
 	bt_ctf_writer_put(cw->writer);
 

@@ -725,8 +868,9 @@ static int ctf_writer__init(struct ctf_writer *cw, const char *path)
 {
 	struct bt_ctf_writer		*writer;
 	struct bt_ctf_stream_class	*stream_class;
-	struct bt_ctf_stream		*stream;
 	struct bt_ctf_clock		*clock;
+	struct bt_ctf_field_type	*pkt_ctx_type;
+	int				ret;
 
 	/* CTF writer */
 	writer = bt_ctf_writer_create(path);

@@ -767,14 +911,15 @@ static int ctf_writer__init(struct ctf_writer *cw, const char *path)
 	if (ctf_writer__init_data(cw))
 		goto err_cleanup;
 
-	/* CTF stream instance */
-	stream = bt_ctf_writer_create_stream(writer, stream_class);
-	if (!stream) {
-		pr("Failed to create CTF stream.\n");
+	/* Add cpu_id for packet context */
+	pkt_ctx_type = bt_ctf_stream_class_get_packet_context_type(stream_class);
+	if (!pkt_ctx_type)
 		goto err_cleanup;
-	}
 
-	cw->stream = stream;
+	ret = bt_ctf_field_type_structure_add_field(pkt_ctx_type, cw->data.u32, "cpu_id");
+	bt_ctf_field_type_put(pkt_ctx_type);
+	if (ret)
+		goto err_cleanup;
 
 	/* CTF clock writer setup */
 	if (bt_ctf_writer_add_clock(writer, clock)) {

@@ -791,6 +936,14 @@ err:
 	return -1;
 }
 
+static void ctf_writer__flush_streams(struct ctf_writer *cw)
+{
+	int cpu;
+
+	for (cpu = 0; cpu < cw->stream_cnt; cpu++)
+		ctf_stream__flush(cw->stream[cpu]);
+}
+
 int bt_convert__perf2ctf(const char *input, const char *path)
 {
 	struct perf_session *session;

@@ -833,9 +986,13 @@ int bt_convert__perf2ctf(const char *input, const char *path)
 	if (setup_events(cw, session))
 		goto free_session;
 
+	if (setup_streams(cw, session))
+		goto free_session;
+
 	err = perf_session__process_events(session, &c.tool);
 	if (!err)
-		err = bt_ctf_stream_flush(cw->stream);
+		ctf_writer__flush_streams(cw);
+
 
 	fprintf(stderr,
 		"[ perf data convert: Converted '%s' into CTF data '%s' ]\n",

Hi Jiri,

Have you noticed that this patch causes a endianess problem?

Without this patch:

$ perf  data convert --to-ctf ./out.ctf
[ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ]
[ perf data convert: Converted and wrote 0.000 MB (11 samples) ]

With this patch:

$ perf  data convert --to-ctf ./out.ctf
perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order: Assertion `byte_order == 1234 || byte_order == 4321' failed.
Aborted

I'll look into this problem if you haven't solved it yet. Please let me know if you have already have some solutions.

Quoted 12 lines · Wang Nan

Hi Jiri,

Have you noticed that this patch causes a endianess problem?

Without this patch:

$ perf  data convert --to-ctf ./out.ctf
[ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ]
[ perf data convert: Converted and wrote 0.000 MB (11 samples) ]

With this patch:

Quoted 6 lines

$ perf  data convert --to-ctf ./out.ctf
perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order: Assertion `byte_order == 1234 || byte_order == 4321' failed.
Aborted

I'll look into this problem if you haven't solved it yet. Please let me know if you have already have some solutions.

hum, never seen this.. any other details?

through different endian server or something like that?

jirka

Quoted 13 lines · Wang Nan

Quoted 12 lines

Hi Jiri,

Have you noticed that this patch causes a endianess problem?

Without this patch:

$ perf  data convert --to-ctf ./out.ctf
[ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ]
[ perf data convert: Converted and wrote 0.000 MB (11 samples) ]

With this patch:

Quoted 8 lines

Quoted 6 lines

$ perf  data convert --to-ctf ./out.ctf
perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order: Assertion `byte_order == 1234 || byte_order == 4321' failed.
Aborted

I'll look into this problem if you haven't solved it yet. Please let me know if you have already have some solutions.

hum, never seen this.. any other details?

My perf is -tip tree (commit 4b52a8d), with patch 'perf data: Add tracepoint events fields CTF conversion support' applied. Babeltrace is fresh new git clone (commit 48d711a).

you moved perf.data

Quoted 2 lines

through different endian server or something like that?

Never.

Em Thu, Mar 12, 2015 at 09:40:09PM +0800, Wang Nan escreveu:

Quoted 25 lines · Jiri Olsa

Quoted 13 lines · Wang Nan

Quoted 12 lines

Hi Jiri,

Have you noticed that this patch causes a endianess problem?

Without this patch:

$ perf  data convert --to-ctf ./out.ctf
[ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ]
[ perf data convert: Converted and wrote 0.000 MB (11 samples) ]

With this patch:

Quoted 8 lines

Quoted 6 lines

$ perf  data convert --to-ctf ./out.ctf
perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order: Assertion `byte_order == 1234 || byte_order == 4321' failed.
Aborted

I'll look into this problem if you haven't solved it yet. Please let me know if you have already have some solutions.

hum, never seen this.. any other details?

Quoted 8 lines

My perf is -tip tree (commit 4b52a8d), with patch 'perf data: Add tracepoint events fields CTF conversion support' applied. Babeltrace is fresh new git clone (commit 48d711a).

you moved perf.data

Quoted 2 lines

through different endian server or something like that?

Never.

Just reproduced this:

[acme@ssdandy linux]$ perf data convert --to-ctf=./ctf-data/
perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order:
Assertion `byte_order == 1234 || byte_order == 4321' failed. Aborted (core dumped)
[acme@ssdandy linux]$

perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order: Assertion `byte_order == 1234 || byte_order == 4321' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff5b345d7 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install audit-libs-2.4.1-5.el7.x86_64 bzip2-libs-1.0.6-12.el7.x86_64 elfutils-libelf-0.160-1.el7.x86_64 elfutils-libs-0.160-1.el7.x86_64 glib2-2.40.0-4.el7.x86_64 glibc-2.17-78.el7.x86_64 libgcc-4.8.3-9.el7.x86_64 libuuid-2.23.2-21.el7.x86_64 nss-softokn-freebl-3.16.2.3-9.el7.x86_64 numactl-libs-2.0.9-4.el7.x86_64 perl-libs-5.16.3-285.el7.x86_64 popt-1.13-16.el7.x86_64 python-libs-2.7.5-16.el7.x86_64 slang-2.2.4-11.el7.x86_64 xz-libs-5.1.2-9alpha.el7.x86_64 zlib-1.2.7-13.el7.x86_64 (gdb) bt

#0  0x00007ffff5b345d7 in raise () from /lib64/libc.so.6
#1  0x00007ffff5b35cc8 in abort () from /lib64/libc.so.6
#2  0x00007ffff5b2d546 in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007ffff5b2d5f2 in __assert_fail () from /lib64/libc.so.6
#4  0x00007ffff510034a in bt_ctf_field_type_set_native_byte_order (type=<optimized out>, byte_order=<optimized out>) at event-types.c:1855
#5  0x00007ffff5101b59 in bt_ctf_stream_class_freeze (stream_class=0x8acc20) at stream-class.c:478
#6  0x00007ffff50fbdc5 in bt_ctf_event_create (event_class=event_class@entry=0x133d590) at event.c:324
#7  0x00000000004e158d in process_sample_event (tool=0x7fffffffdc40, _event=<optimized out>, sample=0x7fffffffd840, evsel=0x8b1040, machine=<optimized out>) at util/data-convert-bt.c:518
#8  0x00000000004b011d in __ordered_events__flush (oe=oe@entry=0x8b0210) at util/ordered-events.c:214
#9  0x00000000004b043a in ordered_events__flush (oe=oe@entry=0x8b0210, how=how@entry=OE_FLUSH__FINAL) at util/ordered-events.c:279

#10 0x00000000004aee18 in __perf_session__process_events (file_size=139704, data_size=<optimized out>, data_offset=<optimized out>, session=0x8b0020) at util/session.c:1394 #11 perf_session__process_events (session=session@entry=0x8b0020) at util/session.c:1412
#12 0x00000000004e270e in bt_convert__perf2ctf (input=<optimized out>, path=0x7fffffffe59d "./ctf-data/") at util/data-convert-bt.c:992
#13 0x00000000004479cf in cmd_data_convert (argc=<optimized out>, argv=<optimized out>, prefix=<optimized out>) at builtin-data.c:77
#14 0x000000000046f365 in run_builtin (p=p@entry=0x8345a0 <commands+576>, argc=argc@entry=3, argv=argv@entry=0x7fffffffe2d0) at perf.c:370
#15 0x000000000041def0 in handle_internal_command (argv=0x7fffffffe2d0, argc=3) at perf.c:429
#16 run_argv (argv=0x7fffffffe050, argcp=0x7fffffffe05c) at perf.c:473
#17 main (argc=3, argv=0x7fffffffe2d0) at perf.c:588
(gdb)

Stopping at this to process other patches, will be back to this after reducing the queue,

- Arnaldo

Quoted 61 lines · Arnaldo Carvalho de Melo

Em Thu, Mar 12, 2015 at 09:40:09PM +0800, Wang Nan escreveu:

Quoted 25 lines · Jiri Olsa

Quoted 13 lines · Wang Nan

Quoted 12 lines

Hi Jiri,

Have you noticed that this patch causes a endianess problem?

Without this patch:

$ perf  data convert --to-ctf ./out.ctf
[ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ]
[ perf data convert: Converted and wrote 0.000 MB (11 samples) ]

With this patch:

Quoted 8 lines

Quoted 6 lines

$ perf  data convert --to-ctf ./out.ctf
perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order: Assertion `byte_order == 1234 || byte_order == 4321' failed.
Aborted

I'll look into this problem if you haven't solved it yet. Please let me know if you have already have some solutions.

hum, never seen this.. any other details?

Quoted 8 lines

My perf is -tip tree (commit 4b52a8d), with patch 'perf data: Add tracepoint events fields CTF conversion support' applied. Babeltrace is fresh new git clone (commit 48d711a).

you moved perf.data

Quoted 2 lines

through different endian server or something like that?

Never.

Just reproduced this:

[acme@ssdandy linux]$ perf data convert --to-ctf=./ctf-data/
perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order:
Assertion `byte_order == 1234 || byte_order == 4321' failed. Aborted (core dumped)
[acme@ssdandy linux]$

perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order: Assertion `byte_order == 1234 || byte_order == 4321' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff5b345d7 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install audit-libs-2.4.1-5.el7.x86_64 bzip2-libs-1.0.6-12.el7.x86_64 elfutils-libelf-0.160-1.el7.x86_64 elfutils-libs-0.160-1.el7.x86_64 glib2-2.40.0-4.el7.x86_64 glibc-2.17-78.el7.x86_64 libgcc-4.8.3-9.el7.x86_64 libuuid-2.23.2-21.el7.x86_64 nss-softokn-freebl-3.16.2.3-9.el7.x86_64 numactl-libs-2.0.9-4.el7.x86_64 perl-libs-5.16.3-285.el7.x86_64 popt-1.13-16.el7.x86_64 python-libs-2.7.5-16.el7.x86_64 slang-2.2.4-11.el7.x86_64 xz-libs-5.1.2-9alpha.el7.x86_64 zlib-1.2.7-13.el7.x86_64 (gdb) bt

#0  0x00007ffff5b345d7 in raise () from /lib64/libc.so.6
#1  0x00007ffff5b35cc8 in abort () from /lib64/libc.so.6
#2  0x00007ffff5b2d546 in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007ffff5b2d5f2 in __assert_fail () from /lib64/libc.so.6
#4  0x00007ffff510034a in bt_ctf_field_type_set_native_byte_order (type=<optimized out>, byte_order=<optimized out>) at event-types.c:1855
#5  0x00007ffff5101b59 in bt_ctf_stream_class_freeze (stream_class=0x8acc20) at stream-class.c:478
#6  0x00007ffff50fbdc5 in bt_ctf_event_create (event_class=event_class@entry=0x133d590) at event.c:324
#7  0x00000000004e158d in process_sample_event (tool=0x7fffffffdc40, _event=<optimized out>, sample=0x7fffffffd840, evsel=0x8b1040, machine=<optimized out>) at util/data-convert-bt.c:518
#8  0x00000000004b011d in __ordered_events__flush (oe=oe@entry=0x8b0210) at util/ordered-events.c:214
#9  0x00000000004b043a in ordered_events__flush (oe=oe@entry=0x8b0210, how=how@entry=OE_FLUSH__FINAL) at util/ordered-events.c:279

#10 0x00000000004aee18 in __perf_session__process_events (file_size=139704, data_size=<optimized out>, data_offset=<optimized out>, session=0x8b0020) at util/session.c:1394

Quoted 8 lines

#11 perf_session__process_events (session=session@entry=0x8b0020) at util/session.c:1412
#12 0x00000000004e270e in bt_convert__perf2ctf (input=<optimized out>, path=0x7fffffffe59d "./ctf-data/") at util/data-convert-bt.c:992 #13 0x00000000004479cf in cmd_data_convert (argc=<optimized out>, argv=<optimized out>, prefix=<optimized out>) at builtin-data.c:77 #14 0x000000000046f365 in run_builtin (p=p@entry=0x8345a0 <commands+576>, argc=argc@entry=3, argv=argv@entry=0x7fffffffe2d0) at perf.c:370 #15 0x000000000041def0 in handle_internal_command (argv=0x7fffffffe2d0, argc=3) at perf.c:429
#16 run_argv (argv=0x7fffffffe050, argcp=0x7fffffffe05c) at perf.c:473
#17 main (argc=3, argv=0x7fffffffe2d0) at perf.c:588
(gdb)

Thanks for providing this info, I'm looking into it and will keep you posted.

Jérémie

Quoted 4 lines

Stopping at this to process other patches, will be back to this after reducing the queue,

- Arnaldo

Em Thu, Mar 12, 2015 at 09:40:09PM +0800, Wang Nan escreveu:

Quoted 25 lines · Jiri Olsa

Quoted 13 lines · Wang Nan

Quoted 12 lines

Hi Jiri,

Have you noticed that this patch causes a endianess problem?

Without this patch:

$ perf  data convert --to-ctf ./out.ctf
[ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ]
[ perf data convert: Converted and wrote 0.000 MB (11 samples) ]

With this patch:

Quoted 8 lines

Quoted 6 lines

$ perf  data convert --to-ctf ./out.ctf
perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order: Assertion `byte_order == 1234 || byte_order == 4321' failed.
Aborted

I'll look into this problem if you haven't solved it yet. Please let me know if you have already have some solutions.

hum, never seen this.. any other details?

Quoted 8 lines

My perf is -tip tree (commit 4b52a8d), with patch 'perf data: Add tracepoint events fields CTF conversion support' applied. Babeltrace is fresh new git clone (commit 48d711a).

you moved perf.data

Quoted 2 lines

through different endian server or something like that?

Never.

Just reproduced this:

[acme@ssdandy linux]$ perf data convert --to-ctf=./ctf-data/
perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order:
Assertion `byte_order == 1234 || byte_order == 4321' failed. Aborted (core dumped)
[acme@ssdandy linux]$

perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order: Assertion `byte_order == 1234 || byte_order == 4321' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff5b345d7 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install audit-libs-2.4.1-5.el7.x86_64 bzip2-libs-1.0.6-12.el7.x86_64 elfutils-libelf-0.160-1.el7.x86_64 elfutils-libs-0.160-1.el7.x86_64 glib2-2.40.0-4.el7.x86_64 glibc-2.17-78.el7.x86_64 libgcc-4.8.3-9.el7.x86_64 libuuid-2.23.2-21.el7.x86_64 nss-softokn-freebl-3.16.2.3-9.el7.x86_64 numactl-libs-2.0.9-4.el7.x86_64 perl-libs-5.16.3-285.el7.x86_64 popt-1.13-16.el7.x86_64 python-libs-2.7.5-16.el7.x86_64 slang-2.2.4-11.el7.x86_64 xz-libs-5.1.2-9alpha.el7.x86_64 zlib-1.2.7-13.el7.x86_64 (gdb) bt

#0  0x00007ffff5b345d7 in raise () from /lib64/libc.so.6
#1  0x00007ffff5b35cc8 in abort () from /lib64/libc.so.6
#2  0x00007ffff5b2d546 in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007ffff5b2d5f2 in __assert_fail () from /lib64/libc.so.6
#4  0x00007ffff510034a in bt_ctf_field_type_set_native_byte_order (type=<optimized out>, byte_order=<optimized out>) at event-types.c:1855
#5  0x00007ffff5101b59 in bt_ctf_stream_class_freeze (stream_class=0x8acc20) at stream-class.c:478
#6  0x00007ffff50fbdc5 in bt_ctf_event_create (event_class=event_class@entry=0x133d590) at event.c:324
#7  0x00000000004e158d in process_sample_event (tool=0x7fffffffdc40, _event=<optimized out>, sample=0x7fffffffd840, evsel=0x8b1040, machine=<optimized out>) at util/data-convert-bt.c:518
#8  0x00000000004b011d in __ordered_events__flush (oe=oe@entry=0x8b0210) at util/ordered-events.c:214
#9  0x00000000004b043a in ordered_events__flush (oe=oe@entry=0x8b0210, how=how@entry=OE_FLUSH__FINAL) at util/ordered-events.c:279

#10 0x00000000004aee18 in __perf_session__process_events (file_size=139704, data_size=<optimized out>, data_offset=<optimized out>, session=0x8b0020) at util/session.c:1394 #11 perf_session__process_events (session=session@entry=0x8b0020) at util/session.c:1412
#12 0x00000000004e270e in bt_convert__perf2ctf (input=<optimized out>, path=0x7fffffffe59d "./ctf-data/") at util/data-convert-bt.c:992
#13 0x00000000004479cf in cmd_data_convert (argc=<optimized out>, argv=<optimized out>, prefix=<optimized out>) at builtin-data.c:77
#14 0x000000000046f365 in run_builtin (p=p@entry=0x8345a0 <commands+576>, argc=argc@entry=3, argv=argv@entry=0x7fffffffe2d0) at perf.c:370
#15 0x000000000041def0 in handle_internal_command (argv=0x7fffffffe2d0, argc=3) at perf.c:429
#16 run_argv (argv=0x7fffffffe050, argcp=0x7fffffffe05c) at perf.c:473
#17 main (argc=3, argv=0x7fffffffe2d0) at perf.c:588
(gdb)

Stopping at this to process other patches, will be back to this after reducing the queue,

- Arnaldo