Skip to contentSkip to conversation

Human read-writeable format for CTF traces

DISCUSSION Mathieu Desnoyers 2 participants 2 messages Feb 3, 2014 1 message not in the archive

lore
Share & tools
lore
#1 lore raw
In reply to a message not in the archive

----- Original Message -----

Quoted 64 lines

From: "Geneviève Bastien" <[email protected]>
To: [email protected]
Sent: Monday, February 3, 2014 9:59:34 AM
Subject: [lttng-dev] Human read-writeable format for CTF traces

Hello all,

Short question:

As we develop analyses in TMF for LTTng traces, we'd like to unit test them with very specific sequences of events, something no real-life trace can provide. We'd like to build our own traces manually.

Does a human read-writeable format already exist to work with CTF traces or convert from/to CTF traces? I heard of a few CTF trace generators, some which haven't been rebased in a while. What's their status? Anything that answers our need there?

Thanks,

==================
Detailed specification of what I'm looking for and why:

* In TMF, analyses use event requests and they handle events. It does not matter at all whether the event came from a CTF binary stream or from a text file, as long as they have the same name and fields, it will be handled the same.

* Also, while we may want to hand make a full test trace, we may also wish to take (part of) a real life CTF trace, convert it to a human read-writeable format, tweak it to our needs and... as far as TMF is concerned, we'll be perfectly happy with this format, but it may eventually be desirable to convert it back to a CTF trace.

* XML-defined analyses are about to make it into TMF master. That means users will be able to develop in XML their own state providers, but eventually also their own views, filters, analyses, etc. Before trying them on real traces, it must be possible to test them on controlled hand-made traces. So that trace format is not just a feature for unit tests, it should be made available in the main TMF for users to use.

* We must keep in mind that users of this format may not have access to the complete LTTng toolchain, or even to a linux command line as they may be working... on Windows! That's even more true now that work is being done to convert Windows traces to CTF format. So the solution must be completely standalone in TMF.

* In a quick prototype last week, I came up with this format for the human read-writeable test traces:

<trace>
<event>
<timestamp value="1" />
<eventName value="A" />
<source value="0" /> /** the cpu */
<field name="b" value="1" type="int" />
<field name="c" value="abcdef" type="string" />
</event>
</trace>

Why XML? Because it's easy to validate with a XSD and we can use existing tool to automatically generate from this XSD a user interface to edit the events. But that's just a suggestion.

Quoted 5 lines

* Lastly and not least, whatever we end up implementing in TMF must use or at least play nice with other tools around CTF that do more or less the same thing. I wouldn't want to reinvent the wheel here.

Any thoughts? Any thing I should look at?

Hi Geneviève,

Jérémie will probably have more to add when he gets back from FOSDEM, but I would expect that the ctf writer API recently added to babeltrace (currently in master branch), along with the Python bindings that cover trace read and write APIs, should allow you to implement things like:

- A plugin to read a CTF trace, and output it in an intermediate format to facilitate edits (e.g. XML as you propose),
- A plugin to read this XML format and output a CTF trace.

You could also generate the XML trace completely by hand if you like, and then convert it to CTF with the second plugin I'm relating to above.

Another possibility is that the XML description also allows describing what the trace contains at a slightly higher level. For instance, if you have a periodic event happening for a certain amount of time, it would be described in XML, and then "generated" by the XML-to-CTF converter.

Indeed, having this kind of infrastructure for unit-testing is going to be *extremely* useful to validate the state system.

Thanks!

Mathieu

Quoted 9 lines

Thanks,
Geneviève

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Signature · 8 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
#2 lore raw
In reply to a message not in the archive

Hi Genevieve,
Would it be an interesting option to make a "metadata" that would parse the xml? Just throwing another option into the mix.
Matthew

Quoted 73 lines · Geneviève Bastien

Hello all,

Short question:

As we develop analyses in TMF for LTTng traces, we'd like to unit test them with very specific sequences of events, something no real-life trace can provide. We'd like to build our own traces manually.

Does a human read-writeable format already exist to work with CTF traces or convert from/to CTF traces? I heard of a few CTF trace generators, some which haven't been rebased in a while. What's their status? Anything that answers our need there?

Thanks,

==================
Detailed specification of what I'm looking for and why:

* In TMF, analyses use event requests and they handle events. It does not matter at all whether the event came from a CTF binary stream or from a text file, as long as they have the same name and fields, it will be handled the same.

* Also, while we may want to hand make a full test trace, we may also wish to take (part of) a real life CTF trace, convert it to a human read-writeable format, tweak it to our needs and... as far as TMF is concerned, we'll be perfectly happy with this format, but it may eventually be desirable to convert it back to a CTF trace.

* XML-defined analyses are about to make it into TMF master. That means users will be able to develop in XML their own state providers, but eventually also their own views, filters, analyses, etc. Before trying them on real traces, it must be possible to test them on controlled hand-made traces. So that trace format is not just a feature for unit tests, it should be made available in the main TMF for users to use.

* We must keep in mind that users of this format may not have access to the complete LTTng toolchain, or even to a linux command line as they may be working... on Windows! That's even more true now that work is being done to convert Windows traces to CTF format. So the solution must be completely standalone in TMF.

* In a quick prototype last week, I came up with this format for the human read-writeable test traces:

<trace>
<event>
<timestamp value="1" />
<eventName value="A" />
<source value="0" /> /** the cpu */
<field name="b" value="1" type="int" />
<field name="c" value="abcdef" type="string" />
</event>
</trace>

Why XML? Because it's easy to validate with a XSD and we can use existing tool to automatically generate from this XSD a user interface to edit the events. But that's just a suggestion.

* Lastly and not least, whatever we end up implementing in TMF must use or at least play nice with other tools around CTF that do more or less the same thing. I wouldn't want to reinvent the wheel here.

Any thoughts? Any thing I should look at?

Thanks,
Geneviève

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev