Skip to contentSkip to conversation

Merge two LTTng CTF traces into one

DISCUSSION Matthew Khouzam 3 participants 9 messages Dec 22, 2014 – Jan 9, 2015 21 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

Michel has hit the nail on the head there.

I would just like to add that one can export an experiment in trace compass. This means that the two traces are shipped in a zip with the synchronization. You can also apply a constant clock offset if network events are unavailable. If you really really want to make a CTF trace as an output, perhaps look at the python bound babeltrace?

BR

Matthew

Quoted 34 lines · Michel Dagenais

Normally, the two traces are stored separately and they are "merged" at display time in TraceCompass. If they were taken on the same host, they use the same clock and should be naturally aligned. If they come from different hosts and you want very fine symchronization, you need some communication events from which to synchronize the traces. TCP packet send and receive events are typically used. If you want to look at two traces simultaneously, it normally implies that the two are related, the two nodes communicating through the network.

------------------------------------------------------------------------

Hi,

Is there any way to merge the two CTF traces and see them in one with trace synchronization ?

How it is related to trace synchronization using network events ?

Thanks,
Divya

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw

Sorry, I was a little hasty in my reply. You can export traces, but not experiments yet, this is a feature to expect soon if people want it.

Quoted 53 lines · Matthew Khouzam

Michel has hit the nail on the head there.

I would just like to add that one can export an experiment in trace compass. This means that the two traces are shipped in a zip with the synchronization. You can also apply a constant clock offset if network events are unavailable. If you really really want to make a CTF trace as an output, perhaps look at the python bound babeltrace?

BR

Matthew

Quoted 30 lines · Michel Dagenais

Normally, the two traces are stored separately and they are "merged" at display time in TraceCompass. If they were taken on the same host, they use the same clock and should be naturally aligned. If they come from different hosts and you want very fine symchronization, you need some communication events from which to synchronize the traces. TCP packet send and receive events are typically used. If you want to look at two traces simultaneously, it normally implies that the two are related, the two nodes communicating through the network.

------------------------------------------------------------------------

Hi,

Is there any way to merge the two CTF traces and see them in one with trace synchronization ?

How it is related to trace synchronization using network events ?

Thanks,
Divya

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Quoted 4 lines

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): CA+=dQ-_sjNg3jj11Psm+wrKbbyS1Ox1kHZyoFwQ2MSfP+SqvuQ@mail.gmail.com

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): CA+=dQ-9zwf=0Qnf8mCwWionoOo-BxRyScb607-0wfoi2hBO8XA@mail.gmail.com

A message that is not in the archive (sent off-list or not yet received): CA+=dQ-8U4i-x1Ffs_R6t2iL1jYmTbJT4PK-CB7b-Sc_MisidQg@mail.gmail.com

A message that is not in the archive (sent off-list or not yet received): CA+=dQ-8+ZWXna=kMxQ0o6ypu2=cTURjE3A4h_uEFnbNctFdfxA@mail.gmail.com

#11 lore raw

Hi,

I followed all the steps in the below link but I am not getting trace synchronized either of the way

http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.linuxtools.lttng.help%2Fdoc%2FTrace-synchronization.html

I have confirmed that net_dev and netif , inet* all tracepoint are enabled . I have used ssh and scp and ping as my workloads.

But still my synchronization quality is absent . I am attaching some screenshots.

I tried synchronization of two traces of same machine. I am getting synchronization as absent .

Can you please tell me what can I do to debug it?

Thanks,
Divya

Quoted 1 line

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): CA+=dQ--wqUbkbW3MXh-4X1NrJ-qp=8B4dh6ydutgGzEcOXHxUA@mail.gmail.com

A message that is not in the archive (sent off-list or not yet received): CA+=dQ-9ZmQJJKZOcoL6kQX5kSMKDZ9HMS2xkjstMiw-5bz79EQ@mail.gmail.com

A message that is not in the archive (sent off-list or not yet received): CA+=dQ-8oGmyKDfNEfLwu_WLeouu0XXxQVigciVBDV3T6dQJ+pg@mail.gmail.com

#16 lore raw

Hi Divya,

I'm bringing this discussion back to lttng-dev, as it may benefit others in the future.

You now have the right fields to synchronize traces. It uses packets of type TCP and fields seq, ack_seq and flags to match packets.

But what are you trying to achieve exactly? The traces you sent me are taken on the same machine at different times. They can never be synchronized. Trace synchronization allows you to synchronize traces taken at the same time on different machines (clocks) so that their events can have the same reference clock.

You can apply a time offset to one of the traces so that their beginning matches (Right-click on the experiment -> Apply time offset... More documentation on this feature is available in the master code). If that's what you want. Otherwise, please explain your specific need.

Geneviève

Quoted 58 lines · Divya Vyas

Hi,

I am attaching the traces . I tried to synchronize the traces of same machine but still quality is absent .

I am in urgent need to get these traces synchronized .

On Tue, Jan 6, 2015 at 11:00 AM, Divya Vyas <[email protected] <mailto:[email protected]>> wrote:

Hi ,

I tried again the steps.

net_dev_queue: { cpu_id = 0 }, { skbaddr = 18446612135892807424, len = 98, name = "eth0", network_header = ( _nhtype_ip : container = 1 ), network_fields = { nhtype_ip = { ihl_version = 0x45, tos = 0, tot_len = 84, id = 0x560A, frag_off = 16384, ttl = 64, protocol = 0x1, checksum = 0xFFF4, saddr = 0xAA267D8, daddr = 0xAA2678E } }, transport_header = ( _thtype_none : container = 0 ), transport_fields = { thtype_none = { } } }

net_dev_queue: { cpu_id = 0 }, { skbaddr = 18446612135892807424, len = 98, name = "eth0", network_header = ( _nhtype_ip : container = 1 ), network_fields = { nhtype_ip = { ihl_version = 0x45, tos = 0, tot_len = 84, id = 0x560A, frag_off = 16384, ttl = 64, protocol = 0x1, checksum = 0xFFF4, saddr = 0xAA267D8, daddr = 0xAA2678E } }, transport_header = ( _thtype_none : container = 0 ), transport_fields = { thtype_none = { } } }

Is this kind of data should be added in the trace log ?

On Tue, Jan 6, 2015 at 3:21 AM, Divya Vyas <[email protected] <mailto:[email protected]>> wrote:

Hi,

Can you please help me what kind of extra fields will be added in trace data. Because I think I followed all the steps.

The branch I was not able to got ,getting some git error. So I downloaded snapshot from website.

On Jan 5, 2015 9:14 PM, "Geneviève Bastien" <[email protected] <mailto:[email protected]>> wrote:

Hi Divya,

I looked at your traces and you do not seem to be using the 'net_data_experimental' special branch to get the extra fields for synchronization. You have the net_dev_queue and netif_receive_skb events, but they do not have the required fields to be able to synchronize. See the section "Obtain synchronizable traces" in
http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.linuxtools.lttng.help%2Fdoc%2FTrace-synchronization.html
for how to compile with that branch.

Quoted 222 lines

Geneviève

Quoted 215 lines · Divya Vyas

Hi,

I am getting Synchronization quality is absent. The workload I am using is ssh and scp .

I have attached the traces . Can you please look at it and give em some advise where I am going wrong?

Thanks,
Divya

On Wed, Dec 24, 2014 at 1:54 AM, Geneviève Bastien <[email protected] <mailto:[email protected]>> wrote:

Hi Divya,

If you followed the instructions in the help, then you have compiled
LTTng-modules with the right tracepoints enabled on all the machines you
want to synchronize. Which method did you choose?

Make sure you enabled either net_dev_queue/netif_receive_skb or inet_sock_local_in/inet_sock_local_out on all machines. Also make sure
all machines exchange TCP packets during the trace, otherwise there will
be nothing to synchronize them on.

When you say you are not getting anything, what do you mean? Do you have
output in the Synchronization view? If the synchronization was OK, but
nothing was found, you should get a Synchronization Quality of "Absent".
If the output is empty, either you have long traces and you didn't get
anything yet (there is no progress bar for the synchronization yet). Or
you may have gotten an exception.

Can you provide us with your traces so we can take a closer look?

Geneviève

On 14-12-23 02:40 PM, Matthew Khouzam wrote:
> So you have traces with network stuff, I would suggest seeing what
> events are enabled in your trace. Maybe you are missing some key events,
> maybe the synchronization worked and we're chasing a ghost...
>
> I am ccing Genevieve Bastien, and Masoume Jabbarifar, the two people who
> designed and implemented synchronisation.
>
> I am not as much an expert as I would like to be in that field.
>
> For offsetting though, it's in tracecompass, the next version of the
> Eclipse LTTng viewer
>
> Here is a link to the page.
> http://projects.eclipse.org/projects/tools.tracecompass
>
> On 14-12-23 12:19 AM, Divya Vyas wrote:
>> Hi,
>>
>> Is there any document which describes how to synchronize the traces on
>> same machine as well as different machine ( setting offset also ) .
>>
>> I exported the two trace in experiment and clicked on synchronize
>> traces . But not getting anything.
>>
>> I followed this page
>>
>>
http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.linuxtools.lttng.help%2Fdoc%2FTrace-synchronization.html
>>
>> Thanks,
>> Divya
>>
>>
>>
>> On Mon, Dec 22, 2014 at 11:24 PM, Matthew Khouzam >> <[email protected] <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>>
>> wrote:
>>
>> Sorry, I was a little hasty in my reply. You
can export traces,
>> but not
>> experiments yet, this is a feature to expect
soon if people want it.
>>
>>
>> On 14-12-22 10:17 AM, Matthew Khouzam wrote:
>> > Michel has hit the nail on the head there.
>> >
>> > I would just like to add that one can export
an experiment in trace
>> > compass. This means that the two traces are
shipped in a zip
>> with the
>> > synchronization. You can also apply a
constant clock offset if
>> network
>> > events are unavailable. If you really really
want to make a CTF
>> trace as
>> > an output, perhaps look at the python bound
babeltrace?
>> >
>> > BR
>> >
>> > Matthew
>> >
>> >
>> > On 14-12-22 08:07 AM, Michel Dagenais wrote:
>> >> Normally, the two traces are stored
separately and they are
>> "merged"
>> >> at display time in TraceCompass. If they
were taken on the same
>> host,
>> >> they use the same clock and should be
naturally aligned. If
>> they come
>> >> from different hosts and you want very fine
symchronization,
>> you need
>> >> some communication events from which to
synchronize the traces. TCP
>> >> packet send and receive events are
typically used. If you want
>> to look
>> >> at two traces simultaneously, it normally
implies that the two are
>> >> related, the two nodes communicating
through the network.
>> >>
>> >>
>>
------------------------------------------------------------------------
>> >>
>> >> Hi,
>> >>
>> >> Is there any way to merge the two CTF
traces and see them
>> in one
>> >> with trace synchronization ?
>> >>
>> >> How it is related to trace synchronization
using network
>> events ?
>> >>
>> >>
>> >> Thanks,
>> >> Divya
>> >>
>> >>
_______________________________________________
>> >> lttng-dev mailing list
>> >> [email protected]
<mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>
>> >>
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>> >>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> lttng-dev mailing list
>> >> [email protected]
<mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>
>> >>
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>> >
>> > _______________________________________________
>> > lttng-dev mailing list
>> > [email protected]
<mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>
>> >
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>>
>> _______________________________________________
>> lttng-dev mailing list
>> [email protected] <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>
>>
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>>

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

#18 lore raw

Hi Genevieve,

My target to is merge the traces of two different machines taken on different time or same time .

Actually synchronization was not working on different machine so I tried on same machine. But now I got it correctly I think so :)

For different time we need to apply offset so that their beginning matches. I think I need to do this .

I tried applying offset on eclipse luna ( I added linux tools in install
new software) but I am not getting this option " Apply time offset ".

I tried on tracecompass also, where I am getting time offset option but the synchronization view is empty . The window is not showing anything. The tracecompass version is 0.1.0

Is there any other plugin for synchronization ?

Quoted 25 lines · Geneviève Bastien

Hi Divya,

I'm bringing this discussion back to lttng-dev, as it may benefit others in the future.

You now have the right fields to synchronize traces. It uses packets of type TCP and fields seq, ack_seq and flags to match packets.

But what are you trying to achieve exactly? The traces you sent me are taken on the same machine at different times. They can never be synchronized. Trace synchronization allows you to synchronize traces taken at the same time on different machines (clocks) so that their events can have the same reference clock.

You can apply a time offset to one of the traces so that their beginning matches (Right-click on the experiment -> Apply time offset... More documentation on this feature is available in the master code). If that's what you want. Otherwise, please explain your specific need.

Geneviève

On 01/06/2015 01:33 AM, Divya Vyas wrote:

Quoted 239 lines

Hi,

I am attaching the traces . I tried to synchronize the traces of same
machine but still quality is absent .

I am in urgent need to get these traces synchronized .

Quoted 228 lines · Divya Vyas

Hi ,

I tried again the steps.

net_dev_queue: { cpu_id = 0 }, { skbaddr = 18446612135892807424, len = 98, name = "eth0", network_header = ( _nhtype_ip : container = 1 ), network_fields = { nhtype_ip = { ihl_version = 0x45, tos = 0, tot_len = 84, id = 0x560A, frag_off = 16384, ttl = 64, protocol = 0x1, checksum = 0xFFF4, saddr = 0xAA267D8, daddr = 0xAA2678E } }, transport_header = ( _thtype_none : container = 0 ), transport_fields = { thtype_none = { } } }

net_dev_queue: { cpu_id = 0 }, { skbaddr = 18446612135892807424, len = 98, name = "eth0", network_header = ( _nhtype_ip : container = 1 ), network_fields = { nhtype_ip = { ihl_version = 0x45, tos = 0, tot_len = 84, id = 0x560A, frag_off = 16384, ttl = 64, protocol = 0x1, checksum = 0xFFF4, saddr = 0xAA267D8, daddr = 0xAA2678E } }, transport_header = ( _thtype_none : container = 0 ), transport_fields = { thtype_none = { } } }

Is this kind of data should be added in the trace log ?

Quoted 202 lines · Divya Vyas

Hi,

Can you please help me what kind of extra fields will be added in trace data. Because I think I followed all the steps.

The branch I was not able to got ,getting some git error. So I downloaded snapshot from website.

Quoted 192 lines · Geneviève Bastien

Hi Divya,

I looked at your traces and you do not seem to be using the 'net_data_experimental' special branch to get the extra fields for synchronization. You have the net_dev_queue and netif_receive_skb events, but they do not have the required fields to be able to synchronize. See the section "Obtain synchronizable traces" in
http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.linuxtools.lttng.help%2Fdoc%2FTrace-synchronization.html
for how to compile with that branch.

Geneviève

On 12/28/2014 12:44 AM, Divya Vyas wrote:

Hi,

I am getting Synchronization quality is absent. The workload I am
using is ssh and scp .

I have attached the traces . Can you please look at it and give em
some advise where I am going wrong?

Thanks,
Divya

Quoted 161 lines · Geneviève Bastien

Hi Divya,

If you followed the instructions in the help, then you have compiled LTTng-modules with the right tracepoints enabled on all the machines you
want to synchronize. Which method did you choose?

Make sure you enabled either net_dev_queue/netif_receive_skb or inet_sock_local_in/inet_sock_local_out on all machines. Also make sure all machines exchange TCP packets during the trace, otherwise there will
be nothing to synchronize them on.

When you say you are not getting anything, what do you mean? Do you have
output in the Synchronization view? If the synchronization was OK, but nothing was found, you should get a Synchronization Quality of "Absent".
If the output is empty, either you have long traces and you didn't get anything yet (there is no progress bar for the synchronization yet). Or you may have gotten an exception.

Can you provide us with your traces so we can take a closer look?

Geneviève

Quoted 2 lines · Matthew Khouzam

So you have traces with network stuff, I would suggest seeing what events are enabled in your trace. Maybe you are missing some key

events,

Quoted 3 lines

maybe the synchronization worked and we're chasing a ghost...

I am ccing Genevieve Bastien, and Masoume Jabbarifar, the two people

who

Quoted 4 lines

designed and implemented synchronisation.

I am not as much an expert as I would like to be in that field.

Quoted 10 lines

For offsetting though, it's in tracecompass, the next version of the Eclipse LTTng viewer

Here is a link to the page.
http://projects.eclipse.org/projects/tools.tracecompass

Quoted 3 lines · Divya Vyas

Hi,

Is there any document which describes how to synchronize the traces

on

Quoted 8 lines
Quoted 8 lines

same machine as well as different machine ( setting offset also ) .

I exported the two trace in experiment and clicked on synchronize traces . But not getting anything.

I followed this page

http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.linuxtools.lttng.help%2Fdoc%2FTrace-synchronization.html

Quoted 6 lines
Quoted 6 lines

Thanks,
Divya

Quoted 8 lines
Quoted 8 lines

On Mon, Dec 22, 2014 at 11:24 PM, Matthew Khouzam <[email protected] <mailto:[email protected]

wrote:

Sorry, I was a little hasty in my reply. You can export traces, but not
experiments yet, this is a feature to expect soon if people

want it.

Quoted 6 lines
Quoted 6 lines · Matthew Khouzam
Quoted 3 lines

Michel has hit the nail on the head there.

I would just like to add that one can export an experiment in

trace

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

compass. This means that the two traces are shipped in a zip

with the
> synchronization. You can also apply a constant clock offset if network
> events are unavailable. If you really really want to make a

CTF

Quoted 5 lines
Quoted 5 lines
    trace as
    > an output, perhaps look at the python bound babeltrace?
    >
    > BR
    >
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

Matthew

Quoted 1 line · Michel Dagenais

Normally, the two traces are stored separately and they are

"merged"
>> at display time in TraceCompass. If they were taken on the

same

Quoted 6 lines
Quoted 6 lines

host,
>> they use the same clock and should be naturally aligned. If they come
>> from different hosts and you want very fine symchronization, you need
>> some communication events from which to synchronize the

traces. TCP

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

packet send and receive events are typically used. If you

want

Quoted 2 lines
Quoted 2 lines

to look
>> at two traces simultaneously, it normally implies that the

two are

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

related, the two nodes communicating through the network.

------------------------------------------------------------------------

Quoted 11 lines
Quoted 11 lines
Quoted 4 lines
Quoted 4 lines

Hi,

Is there any way to merge the two CTF traces and see them

    in one
    >>     with trace synchronization ?
    >>
    >>     How it is related to trace synchronization using network
    events ?
    >>
    >>
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Thanks,
Divya

_______________________________________________
lttng-dev mailing list
[email protected] <mailto:

[email protected]>

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Quoted 21 lines
Quoted 21 lines
Quoted 9 lines
Quoted 8 lines

_______________________________________________
lttng-dev mailing list
[email protected] <mailto:[email protected]>
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Quoted 4 lines

_______________________________________________
lttng-dev mailing list
[email protected] <mailto:[email protected]>
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

_______________________________________________
lttng-dev mailing list
[email protected] <mailto:[email protected]>
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): CA+=dQ-_CaEvWV6uF+01tZFmfhSoHE0g=OGBKoKNC5cwMaWDSew@mail.gmail.com

#20 lore raw

Hi Divya,

Quoted 5 lines · Divya Vyas

Hi Genevieve,

My target to is merge the traces of two different machines taken on different time or same time .

Trace synchronization works by first matching events corresponding to the sending and the reception of a TCP packet from both traces, and then taking those matched events and computing a formula that will transform timestamps from one trace to the same clock as another trace. So only traces taken at the same time (meaning together, not the same clock time of course) will work.

Quoted 3 lines

Actually synchronization was not working on different machine so I tried on same machine. But now I got it correctly I think so :)

Indeed, you have the right events now, with the net_dev_queue and netif_receive_skb. So you should be OK. But I *think* this method works only when on a LAN. If your computers communicate through internet, you're better off with the inet_sock_local_[in|out] method

Quoted 7 lines

For different time we need to apply offset so that their beginning matches. I think I need to do this .

I tried applying offset on eclipse luna ( I added linux tools in
install new software) but I am not getting this option " Apply time
offset ".

It is in master only so far, so you should use tracecompass to have this option.

Quoted 4 lines

I tried on tracecompass also, where I am getting time offset option but the synchronization view is empty . The window is not showing anything. The tracecompass version is 0.1.0

The time offset does not involve synchronization at all, it is just an offset, so you would not see anything in the synchronization view. But in the "Properties" view, when you select the trace, the time offset is displayed.

Geneviève

Quoted 27 lines

Is there any other plugin for synchronization ?

On Tue, Jan 6, 2015 at 9:58 PM, Geneviève Bastien <[email protected] <mailto:[email protected]>> wrote:

Hi Divya,

I'm bringing this discussion back to lttng-dev, as it may benefit others in the future.

You now have the right fields to synchronize traces. It uses packets of type TCP and fields seq, ack_seq and flags to match packets.

But what are you trying to achieve exactly? The traces you sent me are taken on the same machine at different times. They can never be synchronized. Trace synchronization allows you to synchronize traces taken at the same time on different machines (clocks) so that their events can have the same reference clock.

You can apply a time offset to one of the traces so that their beginning matches (Right-click on the experiment -> Apply time offset... More documentation on this feature is available in the master code). If that's what you want. Otherwise, please explain your specific need.

Quoted 299 lines

Geneviève

Quoted 292 lines · Divya Vyas

Hi,

I am attaching the traces . I tried to synchronize the traces of same machine but still quality is absent .

I am in urgent need to get these traces synchronized .

On Tue, Jan 6, 2015 at 11:00 AM, Divya Vyas <[email protected] <mailto:[email protected]>> wrote:

Hi ,

I tried again the steps.

net_dev_queue: { cpu_id = 0 }, { skbaddr = 18446612135892807424, len = 98, name = "eth0", network_header = ( _nhtype_ip : container = 1 ), network_fields = { nhtype_ip = { ihl_version = 0x45, tos = 0, tot_len = 84, id = 0x560A, frag_off = 16384, ttl = 64, protocol = 0x1, checksum = 0xFFF4, saddr = 0xAA267D8, daddr = 0xAA2678E } }, transport_header = ( _thtype_none : container = 0 ), transport_fields = { thtype_none = { } } }

net_dev_queue: { cpu_id = 0 }, { skbaddr = 18446612135892807424, len = 98, name = "eth0", network_header = ( _nhtype_ip : container = 1 ), network_fields = { nhtype_ip = { ihl_version = 0x45, tos = 0, tot_len = 84, id = 0x560A, frag_off = 16384, ttl = 64, protocol = 0x1, checksum = 0xFFF4, saddr = 0xAA267D8, daddr = 0xAA2678E } }, transport_header = ( _thtype_none : container = 0 ), transport_fields = { thtype_none = { } } }

Is this kind of data should be added in the trace log ?

On Tue, Jan 6, 2015 at 3:21 AM, Divya Vyas <[email protected] <mailto:[email protected]>> wrote:

Hi,

Can you please help me what kind of extra fields will be added in trace data. Because I think I followed all the steps.

The branch I was not able to got ,getting some git error. So I downloaded snapshot from website.

On Jan 5, 2015 9:14 PM, "Geneviève Bastien" <[email protected] <mailto:[email protected]>> wrote:

Hi Divya,

I looked at your traces and you do not seem to be using the 'net_data_experimental' special branch to get the extra fields for synchronization. You have the net_dev_queue and netif_receive_skb events, but they do not have the required fields to be able to synchronize. See the section "Obtain synchronizable traces" in
http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.linuxtools.lttng.help%2Fdoc%2FTrace-synchronization.html
for how to compile with that branch.

Geneviève

Quoted 221 lines · Divya Vyas

Hi,

I am getting Synchronization quality is absent. The workload I am using is ssh and scp .

I have attached the traces . Can you please look at it and give em some advise where I am going wrong?

Thanks,
Divya

On Wed, Dec 24, 2014 at 1:54 AM, Geneviève Bastien <[email protected] <mailto:[email protected]>> wrote:

Hi Divya,

If you followed the instructions in the help, then you have compiled
LTTng-modules with the right tracepoints enabled on all the machines you
want to synchronize. Which method did you choose?

Make sure you enabled either net_dev_queue/netif_receive_skb or inet_sock_local_in/inet_sock_local_out on all machines. Also make sure
all machines exchange TCP packets during the trace, otherwise there will
be nothing to synchronize them on.

When you say you are not getting anything, what do you mean? Do you have
output in the Synchronization view? If the synchronization was OK, but
nothing was found, you should get a Synchronization Quality of "Absent". If the output is empty, either you have long traces and you didn't get
anything yet (there is no progress bar for the synchronization yet). Or
you may have gotten an exception.

Can you provide us with your traces so we can take a closer look?

Geneviève

On 14-12-23 02:40 PM, Matthew Khouzam wrote:
> So you have traces with network stuff, I would suggest seeing what
> events are enabled in your trace. Maybe you are missing some key events,
> maybe the synchronization worked and we're chasing a ghost...
>
> I am ccing Genevieve Bastien, and Masoume Jabbarifar, the two people who
> designed and implemented synchronisation. >
> I am not as much an expert as I would like to be in that field.
>
> For offsetting though, it's in tracecompass, the next version of the
> Eclipse LTTng viewer
>
> Here is a link to the page.
>
http://projects.eclipse.org/projects/tools.tracecompass
>
> On 14-12-23 12:19 AM, Divya Vyas wrote:
>> Hi,
>>
>> Is there any document which describes how to synchronize the traces on
>> same machine as well as different machine ( setting offset also ) .
>>
>> I exported the two trace in experiment and clicked on synchronize
>> traces . But not getting anything.
>>
>> I followed this page
>>
>>
http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.linuxtools.lttng.help%2Fdoc%2FTrace-synchronization.html
>>
>> Thanks,
>> Divya
>>
>>
>>
>> On Mon, Dec 22, 2014 at 11:24 PM, Matthew Khouzam >> <[email protected] <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>> >> wrote:
>>
>> Sorry, I was a little hasty in my reply.
You can export traces,
>> but not
>> experiments yet, this is a feature to
expect soon if people want it.
>>
>>
>> On 14-12-22 10:17 AM, Matthew Khouzam wrote:
>> > Michel has hit the nail on the head there.
>> >
>> > I would just like to add that one can
export an experiment in trace
>> > compass. This means that the two traces
are shipped in a zip
>> with the
>> > synchronization. You can also apply a
constant clock offset if
>> network
>> > events are unavailable. If you really
really want to make a CTF
>> trace as
>> > an output, perhaps look at the python
bound babeltrace?
>> >
>> > BR
>> >
>> > Matthew
>> >
>> >
>> > On 14-12-22 08:07 AM, Michel Dagenais
wrote:
>> >> Normally, the two traces are stored
separately and they are
>> "merged"
>> >> at display time in TraceCompass. If
they were taken on the same
>> host,
>> >> they use the same clock and should be
naturally aligned. If
>> they come
>> >> from different hosts and you want very
fine symchronization,
>> you need
>> >> some communication events from which
to synchronize the traces. TCP
>> >> packet send and receive events are
typically used. If you want
>> to look
>> >> at two traces simultaneously, it
normally implies that the two are
>> >> related, the two nodes communicating
through the network.
>> >>
>> >>
>>
------------------------------------------------------------------------
>> >>
>> >> Hi,
>> >>
>> >> Is there any way to merge the two
CTF traces and see them
>> in one
>> >> with trace synchronization ?
>> >>
>> >> How it is related to trace
synchronization using network
>> events ?
>> >>
>> >>
>> >> Thanks,
>> >> Divya
>> >>
>> >>
_______________________________________________
>> >> lttng-dev mailing list
>> >> [email protected]
<mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>
>> >>
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>> >>
>> >>
>> >>
>> >>
>> >>
_______________________________________________
>> >> lttng-dev mailing list
>> >> [email protected]
<mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>
>> >>
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>> >
>> >
_______________________________________________
>> > lttng-dev mailing list
>> > [email protected]
<mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>
>> >
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>>
>>
_______________________________________________
>> lttng-dev mailing list
>> [email protected]
<mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>
>>
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>>

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): CA+=dQ--GJj1rt4jTFOc-sV4YMNb+jpWXnWT_7oULab-FiXRJaQ@mail.gmail.com

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): CA+=dQ-_u2D47dkEt_20YeGBZYBOTiADX6BWq8FR-7XWaOko10A@mail.gmail.com

#25 lore raw

Hi Divya,

I'm putting you back on the lttng-dev mailing list, please reply to that list in the future so other people can answer, because I think that trace synchronization is not exactly what you are looking for here...

Quoted 2 lines

I tried the sessions at same time and communication events in between and it worked :) I am getting quality as accurate and I can see both traces merged .

Excellent!

Quoted 2 lines

Now I come at my second point i.e taking traces from lttng sessions at different machines and at different time and may be communication events or might be not.

If the sessions are taken are different moments, they can _never_ synchronize. Trace synchronization first matches events such that one single event from a trace (for example "sending a packet") can be correlated to exactly one event in the second trace (the reception of that packet). If that is the case, the first event can have a timestamp of now and the second be timed to January 1st 1970, trace synchronization will work and the 2 traces will have a common time reference after synchronization.

Quoted 8 lines

So In my opinion I need to apply time offset on one of the trace to match the beginning so that they can be synchronized .

So like first trace timestamp starts at 07:13:59 .499 222 870 and other starts at 11:21:35.537 682 885 . Then I can apply timeoffset ( How much offset ? In apply time offset advanced option If reference time is 7:13.... then target time will be 11:21:35... it will calculate the offset ) on one of the trace to match traces .

Is it possible to synchronize the traces then ? Actually I tried these steps but not getting the synchronized traces but then both traces beginning is exactly same ( obviously it will as applied by the offset)

Please let me know how can I synchronize the traces from different machines and at different time

I'm not sure what your end goal is exactly. Do you want to compare trace executions? I am cc'ing François Doray who is currently working on a trace comparison utility. Here's a presentation he did recently: http://ahls.dorsal.polymtl.ca/system/files/Comparison%20of%20Traces%20to%20Diagnose%20Performance%20Variations.pdf

His work is still experimental and available on github. If that's what you're interested in, he might give you some information on how to use it.

Otherwise, please explain what you are trying to achieve.

Geneviève

Quoted 110 lines · On Thu, Jan 8, 2015 at 12:39 AM, Geneviève Bastien <[email protected]…
    On 15-01-07 11:32 AM, Divya Vyas wrote:
Quoted 27 lines

Hi ,

I have some comments below and thanks a lot for your help

On Jan 7, 2015 7:55 PM, "Geneviève Bastien" <[email protected] <mailto:[email protected]>> wrote:
>
> Hi Divya,
>
>
> On 15-01-07 01:24 AM, Divya Vyas wrote:
>>
>>
>> Hi Genevieve,
>>
>> My target to is merge the traces of two different machines taken on different time or same time .
>
> Trace synchronization works by first matching events corresponding to the sending and the reception of a TCP packet from both traces, and then taking those matched events and computing a formula that will transform timestamps from one trace to the same clock as another trace. So only traces taken at the same time (meaning together, not the same clock time of course) will work.

Quoted 7 lines

I am not getting this last point , where the traces are taken together. You mean that we start lttng session on two machines at around same time and then during session use some communication events . Then take the traces into tracecompass for synchronization in experiment. I am not getting in which cases time offset is needed?

Yes, two sessions on two machines at the same time and they communicate during those sessions. Then you can synchronize traces. The following traces are used in unit tests to test trace synchronization if you wish to see what it looks like:
http://archive.eclipse.org/tracecompass/test-traces/ctf/synctraces.tar.gz

Quoted 4 lines

In what traces I need to apply time offset so that their beginning matches. Is it applicable on the above case which I am thinking above.

You never _need_ to apply time offset. It is if you have traces that do not contain communication events, and you want one of them to be offsetted by a certain time. For example, you have a trace taken GST-5 and one at GST, you cannot synchronize them because they don't have communication events but you want to analyze them with the same time reference, you would offset the one at GST-5 by +5 hours.

Or if you have a custom event that you know happened at the same time on both traces, but synchronization does not support that event, then you can offset one trace so that both events from both traces have the same timestamp.

Time offsetting is a manual process by the user, while trace synchronization is an automatic analysis using pre-defined events to compute a formula to transform timestamps from one trace to the same clock as the other.

Quoted 31 lines
Quoted 4 lines
Quoted 4 lines

Actually synchronization was not working on different machine

so I tried on same machine. But now I got it correctly I think so :) >
> Indeed, you have the right events now, with the net_dev_queue and netif_receive_skb. So you should be OK. But I *think* this method works only when on a LAN. If your computers communicate through internet, you're better off with the inet_sock_local_[in|out] method

They are connected through LAN only.

>
>>
>> For different time we need to apply offset so that their beginning matches. I think I need to do this .
>>
>> I tried applying offset on eclipse luna ( I added linux tools
in install new software) but I am not getting this option "
Apply time offset ".
>
> It is in master only so far, so you should use tracecompass to have this option.
>
>>
>> I tried on tracecompass also, where I am getting time offset option but the synchronization view is empty . The window is not showing anything. The tracecompass version is 0.1.0
>

Quoted 9 lines

> The time offset does not involve synchronization at all, it is just an offset, so you would not see anything in the synchronization view. But in the "Properties" view, when you select the trace, the time offset is displayed.

I meant to say that I tried synchronizing the traces in tracecompass without time offset . The synchronizing view is empty. It should show at least quality absent etc.

Quoted 8 lines

Quality absent would show only if the traces are from different machines. If you are trying to synchronize traces from the same machine, that view will stay empty.

Geneviève

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): [email protected]

#27 lore raw
Quoted 4 lines · Geneviève Bastien

Hi Divya,

I'm putting you back on the lttng-dev mailing list, please reply to that

list in the future so other people can answer, because I think that trace synchronization is not exactly what you are looking for here...

I am sorry I clicked reply instead of reply to all.

I have a host and on top of that I have multiple guests. Suppose like from 9 to 9:30 I want to analyze the host information as well as guest information in single view and in a synchronized way. As the guest time is different from host . I need the guest traces referenced at host time . and I tracecompass I can see the host traces as well as guest traces in a synchronized way.

I hope now I gave the clear picture of what I am doing. Please let me know if u need some explanation.

Quoted 2 lines
Quoted 2 lines

I tried the sessions at same time and communication events in between

and it worked :)

Quoted 6 lines
Quoted 1 line

I am getting quality as accurate and I can see both traces merged .

Excellent!

Quoted 2 lines

Now I come at my second point i.e taking traces from lttng sessions at

different machines and at different time and may be communication events or might be not.

Quoted 2 lines

If the sessions are taken are different moments, they can _never_

synchronize.

Yes I think you told that they can be analyzed by applying offset but not synchronized.

Trace synchronization first matches events such that one single event from a trace (for example "sending a packet") can be correlated to exactly one event in the second trace (the reception of that packet). If that is the case, the first event can have a timestamp of now and the second be timed to January 1st 1970, trace synchronization will work and the 2 traces will have a common time reference after synchronization.

What is the criteria of timestamps to be synchronized? How much two timestamps matched for synchronization

Quoted 4 lines
Quoted 4 lines

So In my opinion I need to apply time offset on one of the trace to

match the beginning so that they can be synchronized .

Quoted 2 lines
Quoted 2 lines

So like first trace timestamp starts at 07:13:59 .499 222 870 and other

starts at 11:21:35.537 682 885 . Then I can apply timeoffset ( How much offset ? In apply time offset advanced option If reference time is 7:13.... then target time will be 11:21:35... it will calculate the offset ) on one of the trace to match traces .

Quoted 2 lines
Quoted 2 lines

Is it possible to synchronize the traces then ? Actually I tried these

steps but not getting the synchronized traces but then both traces beginning is exactly same ( obviously it will as applied by the offset)

Quoted 2 lines
Quoted 2 lines

Please let me know how can I synchronize the traces from different

machines and at different time

Quoted 2 lines

I'm not sure what your end goal is exactly. Do you want to compare trace

executions? I am cc'ing François Doray who is currently working on a trace comparison utility. Here's a presentation he did recently:
http://ahls.dorsal.polymtl.ca/system/files/Comparison%20of%20Traces%20to%20Diagnose%20Performance%20Variations.pdf

Quoted 2 lines

His work is still experimental and available on github. If that's what

you're interested in, he might give you some information on how to use it.

Quoted 11 lines

Otherwise, please explain what you are trying to achieve.

Geneviève

Quoted 4 lines

On Thu, Jan 8, 2015 at 12:39 AM, Geneviève Bastien <[email protected]>

Quoted 8 lines · wrote:
Quoted 8 lines
Quoted 8 lines · Divya Vyas
Quoted 6 lines

Hi ,

I have some comments below and thanks a lot for your help

On Jan 7, 2015 7:55 PM, "Geneviève Bastien" <[email protected]>

Quoted 10 lines · wrote:
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

Hi Divya,

Quoted 5 lines · Divya Vyas

Hi Genevieve,

My target to is merge the traces of two different machines taken on

different time or same time .

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Trace synchronization works by first matching events corresponding

to the sending and the reception of a TCP packet from both traces, and then taking those matched events and computing a formula that will transform timestamps from one trace to the same clock as another trace. So only traces taken at the same time (meaning together, not the same clock time of course) will work.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

I am not getting this last point , where the traces are taken

together. You mean that we start lttng session on two machines at around same time and then during session use some communication events . Then take the traces into tracecompass for synchronization in experiment. I am not getting in which cases time offset is needed?

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Yes, two sessions on two machines at the same time and they communicate

during those sessions. Then you can synchronize traces. The following traces are used in unit tests to test trace synchronization if you wish to see what it looks like:
http://archive.eclipse.org/tracecompass/test-traces/ctf/synctraces.tar.gz

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

In what traces I need to apply time offset so that their beginning

matches. Is it applicable on the above case which I am thinking above.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

You never _need_ to apply time offset. It is if you have traces that do

not contain communication events, and you want one of them to be offsetted by a certain time. For example, you have a trace taken GST-5 and one at GST, you cannot synchronize them because they don't have communication events but you want to analyze them with the same time reference, you would offset the one at GST-5 by +5 hours.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Or if you have a custom event that you know happened at the same time

on both traces, but synchronization does not support that event, then you can offset one trace so that both events from both traces have the same timestamp.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Time offsetting is a manual process by the user, while trace

synchronization is an automatic analysis using pre-defined events to compute a formula to transform timestamps from one trace to the same clock as the other.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Actually synchronization was not working on different machine so I

tried on same machine. But now I got it correctly I think so :)

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Indeed, you have the right events now, with the net_dev_queue and

netif_receive_skb. So you should be OK. But I *think* this method works only when on a LAN. If your computers communicate through internet, you're better off with the inet_sock_local_[in|out] method

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

They are connected through LAN only.

Quoted 3 lines
Quoted 3 lines

For different time we need to apply offset so that their beginning

matches. I think I need to do this .

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

I tried applying offset on eclipse luna ( I added linux tools in

install new software) but I am not getting this option " Apply time
offset ".

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

It is in master only so far, so you should use tracecompass to have

this option.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I tried on tracecompass also, where I am getting time offset option

but the synchronization view is empty . The window is not showing anything. The tracecompass version is 0.1.0

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

The time offset does not involve synchronization at all, it is just

an offset, so you would not see anything in the synchronization view. But in the "Properties" view, when you select the trace, the time offset is displayed.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

I meant to say that I tried synchronizing the traces in tracecompass

without time offset . The synchronizing view is empty.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line

It should show at least quality absent etc.

Quality absent would show only if the traces are from different

machines. If you are trying to synchronize traces from the same machine, that view will stay empty.

Quoted 6 lines
Quoted 5 lines
Quoted 4 lines

Geneviève

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): CA+=dQ-_DGAtobYX8W=mQUyJMPD8LD4b6FQDrnC9SqmQr_0gWAA@mail.gmail.com

#29 lore raw
Quoted 4 lines · Divya Vyas
Quoted 1 line · On Jan 8, 2015 10:12 PM, "Geneviève Bastien" <[email protected]…
Quoted 9 lines
Quoted 3 lines

Hi Divya,

I'm putting you back on the lttng-dev mailing list, please reply to

that list in the future so other people can answer, because I think that trace synchronization is not exactly what you are looking for here...

I am sorry I clicked reply instead of reply to all.

I have a host and on top of that I have multiple guests. Suppose like from 9 to 9:30 I want to analyze the host information as well as guest information in single view and in a synchronized way. As the guest time is different from host . I need the guest traces referenced at host time . and I tracecompass I can see the host traces as well as guest traces in a synchronized way.

Then your traces are taken at the same time. The ideal in your case is to have your machines communicate, at least a few TCP packets during the duration of the trace so they can be synchronized. If you trace a virtual machine guest and host for 30 minutes, chances are the time drift between the clocks is important so simple time offsetting at a point that you know (either the beginnings of the trace or a precise point in the trace) will correlate justly the traces at this point but the further you go from there, the less accurate the time will be. Trace synchronization accounts for the time drift so if you have accurate synchronization, then the same accuracy applies to the whole trace.

Also, I may point you to a recently contributed Virtual Machine Analysis. Part of it is in master, some still on gerrit (the view for instance). The documentation is still unexistent. It works with the Qemu/KVM virtualizer, and needs some custom compilation of lttng-modules on both guest and host. It is still experimental. It allows to associate a host process with a guest CPU so that you can see on the guest whether the CPU was actually running or preempted on the host. If you're using Qemu/KVM and are interested in this analysis, I can work on the documentation sooner than later.

Quoted 12 lines

I hope now I gave the clear picture of what I am doing. Please let me know if u need some explanation.

Quoted 2 lines
Quoted 2 lines

I tried the sessions at same time and communication events in

between and it worked :)

Quoted 6 lines
Quoted 1 line

I am getting quality as accurate and I can see both traces merged .

Excellent!

Quoted 2 lines

Now I come at my second point i.e taking traces from lttng

sessions at different machines and at different time and may be communication events or might be not.

Quoted 1 line
Quoted 9 lines
Quoted 1 line

If the sessions are taken are different moments, they can _never_

synchronize.

Yes I think you told that they can be analyzed by applying offset but not synchronized.

Trace synchronization first matches events such that one single event from a trace (for example "sending a packet") can be correlated to exactly one event in the second trace (the reception of that packet). If that is the case, the first event can have a timestamp of now and the second be timed to January 1st 1970, trace synchronization will work and the 2 traces will have a common time reference after synchronization.

What is the criteria of timestamps to be synchronized? How much two timestamps matched for synchronization

There is no criteria for the timestamp. Matching events is the important part of the process. If you are interested in trace synchronization, I invite you to read this article by Masoume Jabbarifar [1]. It is at the base of what is implemented in Trace Compass.

[1] http://link.springer.com/article/10.1007/s10922-014-9331-7#page-1

Geneviève

Quoted 105 lines
Quoted 4 lines
Quoted 4 lines

So In my opinion I need to apply time offset on one of the trace to

match the beginning so that they can be synchronized .

Quoted 2 lines
Quoted 2 lines

So like first trace timestamp starts at 07:13:59 .499 222 870 and

other starts at 11:21:35.537 682 885 . Then I can apply timeoffset ( How much offset ? In apply time offset advanced option If reference time is 7:13.... then target time will be 11:21:35... it will calculate the offset ) on one of the trace to match traces .

Quoted 2 lines
Quoted 2 lines

Is it possible to synchronize the traces then ? Actually I tried

these steps but not getting the synchronized traces but then both traces beginning is exactly same ( obviously it will as applied by the offset)

Quoted 2 lines
Quoted 2 lines

Please let me know how can I synchronize the traces from different

machines and at different time

Quoted 2 lines

I'm not sure what your end goal is exactly. Do you want to compare

trace executions? I am cc'ing François Doray who is currently working on a trace comparison utility. Here's a presentation he did recently: http://ahls.dorsal.polymtl.ca/system/files/Comparison%20of%20Traces%20to%20Diagnose%20Performance%20Variations.pdf

Quoted 2 lines

His work is still experimental and available on github. If that's

what you're interested in, he might give you some information on how to use it.

Quoted 11 lines

Otherwise, please explain what you are trying to achieve.

Geneviève

Quoted 4 lines

On Thu, Jan 8, 2015 at 12:39 AM, Geneviève Bastien

Quoted 8 lines · <[email protected] <mailto:[email protected]>> wrote:
Quoted 8 lines
Quoted 8 lines · Divya Vyas
Quoted 6 lines

Hi ,

I have some comments below and thanks a lot for your help

On Jan 7, 2015 7:55 PM, "Geneviève Bastien"

Quoted 10 lines · <[email protected] <mailto:[email protected]>> wrote:
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

Hi Divya,

Quoted 5 lines · Divya Vyas

Hi Genevieve,

My target to is merge the traces of two different machines

taken on different time or same time .

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Trace synchronization works by first matching events

corresponding to the sending and the reception of a TCP packet from both traces, and then taking those matched events and computing a formula that will transform timestamps from one trace to the same clock as another trace. So only traces taken at the same time (meaning together, not the same clock time of course) will work.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

I am not getting this last point , where the traces are taken

together. You mean that we start lttng session on two machines at around same time and then during session use some communication events . Then take the traces into tracecompass for synchronization in experiment. I am not getting in which cases time offset is needed?

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Yes, two sessions on two machines at the same time and they

communicate during those sessions. Then you can synchronize traces. The following traces are used in unit tests to test trace synchronization if you wish to see what it looks like: http://archive.eclipse.org/tracecompass/test-traces/ctf/synctraces.tar.gz

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

In what traces I need to apply time offset so that their

beginning matches. Is it applicable on the above case which I am thinking above.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

You never _need_ to apply time offset. It is if you have traces

that do not contain communication events, and you want one of them to be offsetted by a certain time. For example, you have a trace taken GST-5 and one at GST, you cannot synchronize them because they don't have communication events but you want to analyze them with the same time reference, you would offset the one at GST-5 by +5 hours.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Or if you have a custom event that you know happened at the same

time on both traces, but synchronization does not support that event, then you can offset one trace so that both events from both traces have the same timestamp.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Time offsetting is a manual process by the user, while trace

synchronization is an automatic analysis using pre-defined events to compute a formula to transform timestamps from one trace to the same clock as the other.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Actually synchronization was not working on different machine

so I tried on same machine. But now I got it correctly I think so :)

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Indeed, you have the right events now, with the net_dev_queue

and netif_receive_skb. So you should be OK. But I *think* this method works only when on a LAN. If your computers communicate through internet, you're better off with the inet_sock_local_[in|out] method

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

They are connected through LAN only.

Quoted 3 lines
Quoted 3 lines

For different time we need to apply offset so that their

beginning matches. I think I need to do this .

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

I tried applying offset on eclipse luna ( I added linux tools

in install new software) but I am not getting this option " Apply time offset ".

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

It is in master only so far, so you should use tracecompass to

have this option.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I tried on tracecompass also, where I am getting time offset

option but the synchronization view is empty . The window is not showing anything. The tracecompass version is 0.1.0

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

The time offset does not involve synchronization at all, it is

just an offset, so you would not see anything in the synchronization view. But in the "Properties" view, when you select the trace, the time offset is displayed.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 14 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

I meant to say that I tried synchronizing the traces in

tracecompass without time offset . The synchronizing view is empty.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line

It should show at least quality absent etc.

Quality absent would show only if the traces are from different

machines. If you are trying to synchronize traces from the same machine, that view will stay empty.

Quoted 6 lines
Quoted 5 lines
Quoted 4 lines

Geneviève

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev