perf data convert to ctf fail.
Hi Jiri Olsa,
We are working on your perf convert to ctf patches and trying to convert perf.data into CTF trace. By introducing 8 patches from your repository (the last 8 patches of branch perf/core_ctf_convert) we can use 'perf data convert --to-ctf' command. However, it doesn't work properly:
$ perf record ls $ perf data convert --to-ctf ./out.ctf close: Bad file descriptor Failed to create CTF stream [ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ] [ perf data convert: Converted and wrote 0.000 MB (1 samples) ]
After some debug we found that the problem raises from libbaleltrace:
in bt_ctf_writer_create_stream() --> bt_ctf_trace_create_stream(), it tries to create a stream from stream_class, but doesn't provide a stream_id. When bt_ctf_trace_create_stream() tries to allocate an id for it, the stream_class has been freezed.
Could you please have a look at this problem?
Thank you!
Hi Wang,
I can reproduce your issue, using Babeltrace commit 36336d933 (from Jan 6th). I had it working at some point before, so there must have been some changes to the Babeltrace API since then.
If I update to Babeltrace master (9f476966), I then hit https://bugs.lttng.org/issues/874 . Fixing it manually I can get it building, but then the "perf data convert" command now fails with a new error:
perf data convert --to-ctf=./ctf
perf: event.c:292: bt_ctf_event_create: Assertion `event_class->stream_class->event_header_type' failed. Aborted
I've CC'ed Mathieu and Jérémie, perhaps they can give us some pointers as to what should be updated in the perf patch set?
Cheers,
Alex
Quoted 24 lines · Wang Nan
Hi Jiri Olsa,
We are working on your perf convert to ctf patches and trying to convert perf.data into CTF trace. By introducing 8 patches from your repository (the last 8 patches of branch perf/core_ctf_convert) we can use 'perf data convert --to-ctf' command. However, it doesn't work properly:
$ perf record ls $ perf data convert --to-ctf ./out.ctf close: Bad file descriptor Failed to create CTF stream [ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ] [ perf data convert: Converted and wrote 0.000 MB (1 samples) ]After some debug we found that the problem raises from libbaleltrace:
inbt_ctf_writer_create_stream()-->bt_ctf_trace_create_stream(), it tries to create a stream fromstream_class, but doesn't provide astream_id. Whenbt_ctf_trace_create_stream()tries to allocate an id for it, thestream_classhas been freezed.Could you please have a look at this problem?
Thank you!
Quoted 11 lines · Alexandre Montplaisir
Hi Wang,
I can reproduce your issue, using Babeltrace commit
36336d933(from Jan 6th). I had it working at some point before, so there must have been some changes to the Babeltrace API since then.If I update to Babeltrace master (
9f476966), I then hit
https://bugs.lttng.org/issues/874 . Fixing it manually I can get it building, but then the "perf data convert" command now fails with a new error:
Quoted 4 lines
perf data convert --to-ctf=./ctf
perf: event.c:292:bt_ctf_event_create: Assertion `event_class->stream_class->event_header_type' failed. Aborted
hum, haven't seen this one before.. I'll check with latest babeltrace
Quoted 22 lines
I've CC'ed Mathieu and Jérémie, perhaps they can give us some pointers as to what should be updated in the perf patch set?
Cheers,
AlexQuoted 13 lines · Wang Nan
Hi Jiri Olsa,
We are working on your perf convert to ctf patches and trying to convert perf.data into CTF trace. By introducing 8 patches from your repository (the last 8 patches of branch perf/core_ctf_convert) we can use 'perf data convert --to-ctf' command. However, it doesn't work properly:
$ perf record ls $ perf data convert --to-ctf ./out.ctf close: Bad file descriptor Failed to create CTF stream [ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ] [ perf data convert: Converted and wrote 0.000 MB (1 samples) ]
Any chance you're not using proper library? I'm always lazy to properly install that and need to run perf with LD_LIBRARY_PATH set, like:
LD_LIBRARY_PATH=/opt/libbabeltrace/lib ./perf ....
Quoted 6 lines
Quoted 6 lines
After some debug we found that the problem raises from libbaleltrace:
inbt_ctf_writer_create_stream()-->bt_ctf_trace_create_stream(), it tries to create a stream fromstream_class, but doesn't provide astream_id. Whenbt_ctf_trace_create_stream()tries to allocate an id for it, thestream_classhas been freezed.
probably it's some issue wrt the babeltrace changes Alexandre mentioned.. I'll check
thanks,
jirka
Quoted 12 lines · Jiri Olsa
Quoted 11 lines · Alexandre Montplaisir
Hi Wang,
I can reproduce your issue, using Babeltrace commit
36336d933(from Jan 6th). I had it working at some point before, so there must have been some changes to the Babeltrace API since then.If I update to Babeltrace master (
9f476966), I then hit
https://bugs.lttng.org/issues/874 . Fixing it manually I can get it building, but then the "perf data convert" command now fails with a new error:
Quoted 30 lines
Quoted 4 lines
perf data convert --to-ctf=./ctf
perf: event.c:292:bt_ctf_event_create: Assertion `event_class->stream_class->event_header_type' failed. Abortedhum, haven't seen this one before.. I'll check with latest babeltrace
Quoted 22 lines
I've CC'ed Mathieu and Jérémie, perhaps they can give us some pointers as to what should be updated in the perf patch set?
Cheers,
AlexQuoted 13 lines · Wang Nan
Hi Jiri Olsa,
We are working on your perf convert to ctf patches and trying to convert perf.data into CTF trace. By introducing 8 patches from your repository (the last 8 patches of branch perf/core_ctf_convert) we can use 'perf data convert --to-ctf' command. However, it doesn't work properly:
$ perf record ls $ perf data convert --to-ctf ./out.ctf close: Bad file descriptor Failed to create CTF stream [ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ] [ perf data convert: Converted and wrote 0.000 MB (1 samples) ]
so I've got both errors in the end ;-)
looks like the babeltrace writer API got changed or were not using it right..
I put some fixes on top of my perf/core_ctf_convert branch and it's working for me now, but I need to discuss the API usage
Jérémie,
my babeltrace git sources HEAD is 9f476966aa40.
I needed to do following changes/commits in perf:
(please check my git tree [1] branch perf/core_ctf_convert)
- perf data: Define custom event_header_type
define our own event_header_type because it's no longer generated by default by the library.. is it correct behaviour?
- perf data: Disable Werror convert object
this is probably babeltrace bug, because if I enable -Werror I'll get: In file included from /opt/libbabeltrace//include/babeltrace/ctf-writer/stream.h:30:0, from util/data-convert-bt.c:13: /opt/libbabeltrace//include/babeltrace/ctf-ir/stream.h:196:12: error: redundant redeclaration of ‘bt_ctf_stream_set_event_context’ [-Werror=redundant-decls] extern int bt_ctf_stream_set_event_context( ^ /opt/libbabeltrace//include/babeltrace/ctf-ir/stream.h:171:12: note: previous declaration of ‘bt_ctf_stream_set_event_context’ was here extern int bt_ctf_stream_set_event_context( ^
- perf data: Init stram_class with ID
the bt_ctf_trace_create_stream function tries to set ID on stream_class which gets frozen just few moments ago in bt_ctf_stream_create call..
this trigers failure and close up of stream fd which is not set yet:
if (close(stream->pos.fd)) {
perror("close");
}
hence the 'close: Bad file descriptor' message
so this one looks like a bug to me, but we might be misusing the API also..
feel free to move this discussion to babeltrace-devel list ;-)
thanks for comments,
jirka
[1] git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
and adding Sebastian to the loop ;-)
jirka
Quoted 96 lines · Jiri Olsa
Quoted 12 lines · Jiri Olsa
Quoted 11 lines · Alexandre Montplaisir
Hi Wang,
I can reproduce your issue, using Babeltrace commit
36336d933(from Jan 6th). I had it working at some point before, so there must have been some changes to the Babeltrace API since then.If I update to Babeltrace master (
9f476966), I then hit
https://bugs.lttng.org/issues/874 . Fixing it manually I can get it building, but then the "perf data convert" command now fails with a new error:Quoted 30 lines
Quoted 4 lines
perf data convert --to-ctf=./ctf
perf: event.c:292:bt_ctf_event_create: Assertion `event_class->stream_class->event_header_type' failed. Abortedhum, haven't seen this one before.. I'll check with latest babeltrace
Quoted 22 lines
I've CC'ed Mathieu and Jérémie, perhaps they can give us some pointers as to what should be updated in the perf patch set?
Cheers,
AlexQuoted 13 lines · Wang Nan
Hi Jiri Olsa,
We are working on your perf convert to ctf patches and trying to convert perf.data into CTF trace. By introducing 8 patches from your repository (the last 8 patches of branch perf/core_ctf_convert) we can use 'perf data convert --to-ctf' command. However, it doesn't work properly:
$ perf record ls $ perf data convert --to-ctf ./out.ctf close: Bad file descriptor Failed to create CTF stream [ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ] [ perf data convert: Converted and wrote 0.000 MB (1 samples) ]so I've got both errors in the end ;-)
looks like the babeltrace writer API got changed or were not using it right..
I put some fixes on top of my perf/core_ctf_convert branch and it's working for me now, but I need to discuss the API usage
Jérémie,
my babeltrace git sources HEAD is9f476966aa40.I needed to do following changes/commits in perf:
(please check my git tree [1] branch perf/core_ctf_convert)- perf data: Define custom
event_header_type
define our ownevent_header_typebecause it's no longer generated by default by the library.. is it correct behaviour?- perf data: Disable Werror convert object
this is probably babeltrace bug, because if I enable -Werror I'll get: In file included from /opt/libbabeltrace//include/babeltrace/ctf-writer/stream.h:30:0, from util/data-convert-bt.c:13: /opt/libbabeltrace//include/babeltrace/ctf-ir/stream.h:196:12: error: redundant redeclaration of ‘bt_ctf_stream_set_event_context’ [-Werror=redundant-decls] extern int bt_ctf_stream_set_event_context( ^ /opt/libbabeltrace//include/babeltrace/ctf-ir/stream.h:171:12: note: previous declaration of ‘bt_ctf_stream_set_event_context’ was here extern int bt_ctf_stream_set_event_context( ^- perf data: Init
stram_classwith ID
thebt_ctf_trace_create_streamfunction tries to set ID onstream_classwhich gets frozen just few moments ago inbt_ctf_stream_createcall..this trigers failure and close up of stream fd which is not set yet:
if (
close(stream->pos.fd)) {
perror("close");
}hence the 'close: Bad file descriptor' message
so this one looks like a bug to me, but we might be misusing the API also..feel free to move this discussion to babeltrace-devel list ;-)
thanks for comments,
jirka[1]
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
Quoted 86 lines · Jiri Olsa
Quoted 12 lines · Jiri Olsa
Quoted 11 lines · Alexandre Montplaisir
Hi Wang,
I can reproduce your issue, using Babeltrace commit
36336d933(from Jan 6th). I had it working at some point before, so there must have been some changes to the Babeltrace API since then.If I update to Babeltrace master (
9f476966), I then hit
https://bugs.lttng.org/issues/874 . Fixing it manually I can get it building, but then the "perf data convert" command now fails with a new error:Quoted 30 lines
Quoted 4 lines
perf data convert --to-ctf=./ctf
perf: event.c:292:bt_ctf_event_create: Assertion `event_class->stream_class->event_header_type' failed. Abortedhum, haven't seen this one before.. I'll check with latest babeltrace
Quoted 22 lines
I've CC'ed Mathieu and Jérémie, perhaps they can give us some pointers as to what should be updated in the perf patch set?
Cheers,
AlexQuoted 13 lines · Wang Nan
Hi Jiri Olsa,
We are working on your perf convert to ctf patches and trying to convert perf.data into CTF trace. By introducing 8 patches from your repository (the last 8 patches of branch perf/core_ctf_convert) we can use 'perf data convert --to-ctf' command. However, it doesn't work properly:
$ perf record ls $ perf data convert --to-ctf ./out.ctf close: Bad file descriptor Failed to create CTF stream [ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ] [ perf data convert: Converted and wrote 0.000 MB (1 samples) ]so I've got both errors in the end ;-)
looks like the babeltrace writer API got changed or were not using it right..
I put some fixes on top of my perf/core_ctf_convert branch and it's working for me now, but I need to discuss the API usage
Jérémie,
my babeltrace git sources HEAD is9f476966aa40.I needed to do following changes/commits in perf:
(please check my git tree [1] branch perf/core_ctf_convert)- perf data: Define custom
event_header_type
define our ownevent_header_typebecause it's no longer generated by default by the library.. is it correct behaviour?- perf data: Disable Werror convert object
this is probably babeltrace bug, because if I enable -Werror I'll get: In file included from /opt/libbabeltrace//include/babeltrace/ctf-writer/stream.h:30:0, from util/data-convert-bt.c:13: /opt/libbabeltrace//include/babeltrace/ctf-ir/stream.h:196:12: error: redundant redeclaration of ‘bt_ctf_stream_set_event_context’ [-Werror=redundant-decls] extern int bt_ctf_stream_set_event_context( ^ /opt/libbabeltrace//include/babeltrace/ctf-ir/stream.h:171:12: note: previous declaration of ‘bt_ctf_stream_set_event_context’ was here extern int bt_ctf_stream_set_event_context( ^- perf data: Init
stram_classwith ID
thebt_ctf_trace_create_streamfunction tries to set ID onstream_classwhich gets frozen just few moments ago inbt_ctf_stream_createcall..this trigers failure and close up of stream fd which is not set yet:
if (
close(stream->pos.fd)) {
perror("close");
}
Quoted 5 lines
hence the 'close: Bad file descriptor' message
so this one looks like a bug to me, but we might be misusing the API also..feel free to move this discussion to babeltrace-devel list ;-)
CC-ing lttng-dev.
Thanks for the quick report! I'm on it and I'll keep you updated as soon as I have pushed a fix.
Jérémie
Quoted 5 lines
thanks for comments,
jirka[1]
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
Signature · 3 lines
Jérémie Galarneau EfficiOS Inc. http://www.efficios.com
Quoted 86 lines · Jiri Olsa
Quoted 12 lines · Jiri Olsa
Quoted 11 lines · Alexandre Montplaisir
Hi Wang,
I can reproduce your issue, using Babeltrace commit
36336d933(from Jan 6th). I had it working at some point before, so there must have been some changes to the Babeltrace API since then.If I update to Babeltrace master (
9f476966), I then hit
https://bugs.lttng.org/issues/874 . Fixing it manually I can get it building, but then the "perf data convert" command now fails with a new error:Quoted 30 lines
Quoted 4 lines
perf data convert --to-ctf=./ctf
perf: event.c:292:bt_ctf_event_create: Assertion `event_class->stream_class->event_header_type' failed. Abortedhum, haven't seen this one before.. I'll check with latest babeltrace
Quoted 22 lines
I've CC'ed Mathieu and Jérémie, perhaps they can give us some pointers as to what should be updated in the perf patch set?
Cheers,
AlexQuoted 13 lines · Wang Nan
Hi Jiri Olsa,
We are working on your perf convert to ctf patches and trying to convert perf.data into CTF trace. By introducing 8 patches from your repository (the last 8 patches of branch perf/core_ctf_convert) we can use 'perf data convert --to-ctf' command. However, it doesn't work properly:
$ perf record ls $ perf data convert --to-ctf ./out.ctf close: Bad file descriptor Failed to create CTF stream [ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ] [ perf data convert: Converted and wrote 0.000 MB (1 samples) ]so I've got both errors in the end ;-)
looks like the babeltrace writer API got changed or were not using it right..
I put some fixes on top of my perf/core_ctf_convert branch and it's working for me now, but I need to discuss the API usage
Jérémie,
my babeltrace git sources HEAD is9f476966aa40.I needed to do following changes/commits in perf:
(please check my git tree [1] branch perf/core_ctf_convert)- perf data: Define custom
event_header_type
define our ownevent_header_typebecause it's no longer generated by default by the library.. is it correct behaviour?- perf data: Disable Werror convert object
this is probably babeltrace bug, because if I enable -Werror I'll get: In file included from /opt/libbabeltrace//include/babeltrace/ctf-writer/stream.h:30:0, from util/data-convert-bt.c:13: /opt/libbabeltrace//include/babeltrace/ctf-ir/stream.h:196:12: error: redundant redeclaration of ‘bt_ctf_stream_set_event_context’ [-Werror=redundant-decls] extern int bt_ctf_stream_set_event_context( ^ /opt/libbabeltrace//include/babeltrace/ctf-ir/stream.h:171:12: note: previous declaration of ‘bt_ctf_stream_set_event_context’ was here extern int bt_ctf_stream_set_event_context( ^- perf data: Init
stram_classwith ID
thebt_ctf_trace_create_streamfunction tries to set ID onstream_classwhich gets frozen just few moments ago inbt_ctf_stream_createcall..this trigers failure and close up of stream fd which is not set yet:
if (
close(stream->pos.fd)) {
perror("close");
}
Quoted 5 lines
hence the 'close: Bad file descriptor' message
so this one looks like a bug to me, but we might be misusing the API also..feel free to move this discussion to babeltrace-devel list ;-)
CC-ing lttng-dev.
Thanks for the quick report! I'm on it and I'll keep you updated as soon as I have pushed a fix.
Jérémie
Quoted 5 lines
thanks for comments,
jirka[1]
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
Signature · 8 lines
Jérémie Galarneau EfficiOS Inc. http://www.efficios.com _______________________________________________ lttng-dev mailing list [email protected] http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Quoted 56 lines · Jiri Olsa
Quoted 12 lines · Jiri Olsa
Quoted 11 lines · Alexandre Montplaisir
Hi Wang,
I can reproduce your issue, using Babeltrace commit
36336d933(from Jan 6th). I had it working at some point before, so there must have been some changes to the Babeltrace API since then.If I update to Babeltrace master (
9f476966), I then hit
https://bugs.lttng.org/issues/874 . Fixing it manually I can get it building, but then the "perf data convert" command now fails with a new error:Quoted 30 lines
Quoted 4 lines
perf data convert --to-ctf=./ctf
perf: event.c:292:bt_ctf_event_create: Assertion `event_class->stream_class->event_header_type' failed. Abortedhum, haven't seen this one before.. I'll check with latest babeltrace
Quoted 22 lines
I've CC'ed Mathieu and Jérémie, perhaps they can give us some pointers as to what should be updated in the perf patch set?
Cheers,
AlexQuoted 13 lines · Wang Nan
Hi Jiri Olsa,
We are working on your perf convert to ctf patches and trying to convert perf.data into CTF trace. By introducing 8 patches from your repository (the last 8 patches of branch perf/core_ctf_convert) we can use 'perf data convert --to-ctf' command. However, it doesn't work properly:
$ perf record ls $ perf data convert --to-ctf ./out.ctf close: Bad file descriptor Failed to create CTF stream [ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ] [ perf data convert: Converted and wrote 0.000 MB (1 samples) ]so I've got both errors in the end ;-)
looks like the babeltrace writer API got changed or were not using it right..
I put some fixes on top of my perf/core_ctf_convert branch and it's working for me now, but I need to discuss the API usage
Jérémie,
my babeltrace git sources HEAD is9f476966aa40.
Quoted 7 lines
I needed to do following changes/commits in perf:
(please check my git tree [1] branch perf/core_ctf_convert)- perf data: Define custom
event_header_type
define our ownevent_header_typebecause it's no longer generated by default by the library.. is it correct behaviour?
The event header is now "override-able" but, by default, should still be initialized by the library. This is definitely a bug. You don't need to use this new API if the default event header layout (uint32_t id; uint64_t timestamp) suited your use-case.
Quoted 17 lines
- perf data: Disable Werror convert object
this is probably babeltrace bug, because if I enable -Werror I'll get: In file included from /opt/libbabeltrace//include/babeltrace/ctf-writer/stream.h:30:0, from util/data-convert-bt.c:13: /opt/libbabeltrace//include/babeltrace/ctf-ir/stream.h:196:12: error: redundant redeclaration of ‘bt_ctf_stream_set_event_context’ [-Werror=redundant-decls] extern int bt_ctf_stream_set_event_context( ^ /opt/libbabeltrace//include/babeltrace/ctf-ir/stream.h:171:12: note: previous declaration of ‘bt_ctf_stream_set_event_context’ was here extern int bt_ctf_stream_set_event_context( ^- perf data: Init
stram_classwith ID
thebt_ctf_trace_create_streamfunction tries to set ID onstream_classwhich gets frozen just few moments ago inbt_ctf_stream_createcall..
Quoted 9 lines
this trigers failure and close up of stream fd which is not set yet:
if (
close(stream->pos.fd)) {
perror("close");
}hence the 'close: Bad file descriptor' message
so this one looks like a bug to me, but we might be misusing the API also..
I think that was all introduced by 9f476966aa40bd0de2cd0654623ea03f8a3254eb. I'll keep you posted.
Jérémie
Quoted 7 lines
feel free to move this discussion to babeltrace-devel list ;-)
thanks for comments,
jirka[1]
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
Signature · 3 lines
Jérémie Galarneau EfficiOS Inc. http://www.efficios.com