Parsing external datasources in trace compass for regions of interest in a trace
Hello,
I am right now looking getting external datasources to supply information for a trace, like a region of interest.
Background: we can run an analysis on a trace manually, in python, excel or what have you and come up with something interesting, like: around 12:30pm something happened to a trace. Let's dig deeper with tracecompass.
To make this work: we need to find a way to import the data, so we should agree on a standard data standard. I personally like the LTTng analysis outputs so I am picking that for now. :)
Here are some formats that could be acceptable as regions of interest
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
boookmark
* [ 2015-01-15 12:18:37.216484041] name boookmark
* [12:18:37.216484041] name boookmark
* [ 12:18:37.216484041, 12:18:53.821580313] name boookmark
* [ 2014-12-12 17:29:43.802588035] name with space
* [ 2014-12-12 17:29:43] irrational title
* [17:29:43.802588035] rational title
* [ 17:29:43] test test test
* [17:29:43,17:29:44] thing
* [12:18:37.216484041 ] (no text)
When there is one timestamp, we will create one bookmark. When there is a range, we would create two bookmarks in a given trace, the first one being appended with " start" the second with " end"
This should allow tracecompass to marginally benefit from the lttng analyses that are looking pretty nice, and later perhaps from tcp analyses from tshark and syslog stuff.
Does anyone have an objection to this way of working?
----- Original Message -----
Quoted 37 lines
From: "Matthew Khouzam" <[email protected]>
To: "tracecompass developer discussions" <[email protected]>, [email protected], [email protected]
Sent: Friday, February 6, 2015 5:13:42 PM
Subject: [diamon-discuss] Parsing external datasources in trace compass for regions of interest in a traceHello,
I am right now looking getting external datasources to supply information for a trace, like a region of interest.
Background: we can run an analysis on a trace manually, in python, excel or what have you and come up with something interesting, like: around 12:30pm something happened to a trace. Let's dig deeper with tracecompass.
To make this work: we need to find a way to import the data, so we should agree on a standard data standard. I personally like the LTTng analysis outputs so I am picking that for now. :)
Here are some formats that could be acceptable as regions of interest
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
boookmark
* [ 2015-01-15 12:18:37.216484041] name boookmark
* [12:18:37.216484041] name boookmark
* [ 12:18:37.216484041, 12:18:53.821580313] name boookmark
* [ 2014-12-12 17:29:43.802588035] name with space
* [ 2014-12-12 17:29:43] irrational title
* [17:29:43.802588035] rational title
* [ 17:29:43] test test test
* [17:29:43,17:29:44] thing
* [12:18:37.216484041 ] (no text)When there is one timestamp, we will create one bookmark. When there is a range, we would create two bookmarks in a given trace, the first one being appended with " start" the second with " end"
Quoted 5 lines
This should allow tracecompass to marginally benefit from the lttng analyses that are looking pretty nice, and later perhaps from tcp analyses from tshark and syslog stuff.
Does anyone have an objection to this way of working?
Do you intend on supporting timestamps that appear elsewhere than the beginning of lines ? Do you consider all text at the right of the timestamp or timestamp range to be part of the naming of the region of interest linked to the timestamp/timestamp range ?
Julien, did we want to reserve [] for only timestamp range or we also want them for single timestamps ?
Thanks,
Mathieu
Quoted 6 lines
_______________________________________________
diamon-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Quoted 44 lines · Mathieu Desnoyers
----- Original Message -----
Quoted 37 lines
From: "Matthew Khouzam" <[email protected]>
To: "tracecompass developer discussions" <[email protected]>, [email protected], [email protected]
Sent: Friday, February 6, 2015 5:13:42 PM
Subject: [diamon-discuss] Parsing external datasources in trace compass for regions of interest in a traceHello,
I am right now looking getting external datasources to supply information for a trace, like a region of interest.
Background: we can run an analysis on a trace manually, in python, excel or what have you and come up with something interesting, like: around 12:30pm something happened to a trace. Let's dig deeper with tracecompass.
To make this work: we need to find a way to import the data, so we should agree on a standard data standard. I personally like the LTTng analysis outputs so I am picking that for now. :)
Here are some formats that could be acceptable as regions of interest
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
boookmark
* [ 2015-01-15 12:18:37.216484041] name boookmark
* [12:18:37.216484041] name boookmark
* [ 12:18:37.216484041, 12:18:53.821580313] name boookmark
* [ 2014-12-12 17:29:43.802588035] name with space
* [ 2014-12-12 17:29:43] irrational title
* [17:29:43.802588035] rational title
* [ 17:29:43] test test test
* [17:29:43,17:29:44] thing
* [12:18:37.216484041 ] (no text)When there is one timestamp, we will create one bookmark. When there is a range, we would create two bookmarks in a given trace, the first one being appended with " start" the second with " end"
Quoted 5 lines
This should allow tracecompass to marginally benefit from the lttng analyses that are looking pretty nice, and later perhaps from tcp analyses from tshark and syslog stuff.
Does anyone have an objection to this way of working?
Quoted 7 lines
Do you intend on supporting timestamps that appear elsewhere than the beginning of lines ? Do you consider all text at the right of the timestamp or timestamp range to be part of the naming of the region of interest linked to the timestamp/timestamp range ?
Julien, did we want to reserve [] for only timestamp range or we also want them for single timestamps ?
Right now the [] are only valid with time ranges (they are not accepted with --begin and --end). I don't mind allowing the brackets for all timestamps (like in dmesg), we still have the "," to identify if it's a range or not.
But for convenience, they will remain optional for --begin and --end.
We just have to keep in mind, that [32,64] could be a range of values/frequency and not necessarily timestamps.
Julien
Great feedback. I think for a first draft, this will work out swimmingly.
Right now, I was actually finding the first group of [], splitting with "," [1], then for every timestamp, creating a bookmark.
so [12:20:23 , 12:23:22] some range
would give
bookmark 1-> 12:20:23 -> some range start
bookmark 2-> 12:23:22 -> some range end
and
[22:33:44 ] [key : value] [name = value] [id -> ref ] ... would parse as everything after the timestamp would be read as a string.
text [22:33:44] text2
would discard text.
Is this ok?
[1]note, it will not work with https://github.com/lttng/lttng-analyses
where dash is used.
Quoted 62 lines · Julien Desfossez
Quoted 42 lines · Mathieu Desnoyers
----- Original Message -----
Quoted 41 lines
From: "Matthew Khouzam" <[email protected]>
To: "tracecompass developer discussions" <[email protected]>, [email protected], [email protected]
Sent: Friday, February 6, 2015 5:13:42 PM
Subject: [diamon-discuss] Parsing external datasources in trace compass for regions of interest in a traceHello,
I am right now looking getting external datasources to supply information for a trace, like a region of interest.
Background: we can run an analysis on a trace manually, in python, excel or what have you and come up with something interesting, like: around 12:30pm something happened to a trace. Let's dig deeper with tracecompass.
To make this work: we need to find a way to import the data, so we should agree on a standard data standard. I personally like the LTTng analysis outputs so I am picking that for now. :)
Here are some formats that could be acceptable as regions of interest
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
boookmark
* [ 2015-01-15 12:18:37.216484041] name boookmark
* [12:18:37.216484041] name boookmark
* [ 12:18:37.216484041, 12:18:53.821580313] name boookmark
* [ 2014-12-12 17:29:43.802588035] name with space
* [ 2014-12-12 17:29:43] irrational title
* [17:29:43.802588035] rational title
* [ 17:29:43] test test test
* [17:29:43,17:29:44] thing
* [12:18:37.216484041 ] (no text)When there is one timestamp, we will create one bookmark. When there is a range, we would create two bookmarks in a given trace, the first one being appended with " start" the second with " end"
This should allow tracecompass to marginally benefit from the lttng analyses that are looking pretty nice, and later perhaps from tcp analyses from tshark and syslog stuff.
Quoted 8 lines
Quoted 1 line
Does anyone have an objection to this way of working?
Do you intend on supporting timestamps that appear elsewhere than the beginning of lines ? Do you consider all text at the right of the timestamp or timestamp range to be part of the naming of the region of interest linked to the timestamp/timestamp range ?
Julien, did we want to reserve [] for only timestamp range or we also want them for single timestamps ?
Right now the [] are only valid with time ranges (they are not accepted with --begin and --end). I don't mind allowing the brackets for all timestamps (like in dmesg), we still have the "," to identify if it's a range or not.
But for convenience, they will remain optional for --begin and --end.We just have to keep in mind, that [32,64] could be a range of values/frequency and not necessarily timestamps.
Julien
Quoted 4 lines · Matthew Khouzam
Great feedback. I think for a first draft, this will work out swimmingly.
Right now, I was actually finding the first group of [], splitting with "," [1], then for every timestamp, creating a bookmark.
That's a good first start I think.
Quoted 5 lines
so [12:20:23 , 12:23:22] some range
would give
bookmark 1-> 12:20:23 -> some range start
bookmark 2-> 12:23:22 -> some range end
I'm not sure exactly how clear it will be with overlapping ranges in the TraceCompass list of bookmarks, but I guess it will be handled in a next phase.
Quoted 9 lines
and
[22:33:44 ] [key : value] [name = value] [id -> ref ] ... would parse as everything after the timestamp would be read as a string.text [22:33:44] text2
would discard text.Is this ok?
Yes, let's start with that. As we discussed, if we need to output differently from the analyses, we will make the changes.
Quoted 2 lines
[1]note, it will not work with https://github.com/lttng/lttng-analyses
where dash is used.
Oh if you are referring to the "Work in progress" section, you are right, don't worry about this, it is a prototype and cannot even be used from the scripts in the master branch. When this is ready it will use the standard format.
Thanks,
Julien
Quoted 65 lines · Julien Desfossez
Quoted 62 lines · Mathieu Desnoyers
Quoted 42 lines
----- Original Message -----
Quoted 41 lines
From: "Matthew Khouzam" <[email protected]>
To: "tracecompass developer discussions" <[email protected]>, [email protected], [email protected]
Sent: Friday, February 6, 2015 5:13:42 PM
Subject: [diamon-discuss] Parsing external datasources in trace compass for regions of interest in a traceHello,
I am right now looking getting external datasources to supply information for a trace, like a region of interest.
Background: we can run an analysis on a trace manually, in python, excel or what have you and come up with something interesting, like: around 12:30pm something happened to a trace. Let's dig deeper with tracecompass.
To make this work: we need to find a way to import the data, so we should agree on a standard data standard. I personally like the LTTng analysis outputs so I am picking that for now. :)
Here are some formats that could be acceptable as regions of interest
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
boookmark
* [ 2015-01-15 12:18:37.216484041] name boookmark
* [12:18:37.216484041] name boookmark
* [ 12:18:37.216484041, 12:18:53.821580313] name boookmark
* [ 2014-12-12 17:29:43.802588035] name with space
* [ 2014-12-12 17:29:43] irrational title
* [17:29:43.802588035] rational title
* [ 17:29:43] test test test
* [17:29:43,17:29:44] thing
* [12:18:37.216484041 ] (no text)When there is one timestamp, we will create one bookmark. When there is a range, we would create two bookmarks in a given trace, the first one being appended with " start" the second with " end"
This should allow tracecompass to marginally benefit from the lttng analyses that are looking pretty nice, and later perhaps from tcp analyses from tshark and syslog stuff.
Quoted 8 lines
Quoted 1 line
Does anyone have an objection to this way of working?
Do you intend on supporting timestamps that appear elsewhere than the beginning of lines ? Do you consider all text at the right of the timestamp or timestamp range to be part of the naming of the region of interest linked to the timestamp/timestamp range ?
Julien, did we want to reserve [] for only timestamp range or we also want them for single timestamps ?
Right now the [] are only valid with time ranges (they are not accepted with --begin and --end). I don't mind allowing the brackets for all timestamps (like in dmesg), we still have the "," to identify if it's a range or not.
But for convenience, they will remain optional for --begin and --end.We just have to keep in mind, that [32,64] could be a range of values/frequency and not necessarily timestamps.
Julien
Hi Matthew
See below for some comments.
/Bernd
Quoted 7 lines · Julien Desfossez
Quoted 4 lines · Matthew Khouzam
Great feedback. I think for a first draft, this will work out swimmingly.
Right now, I was actually finding the first group of [], splitting with "," [1], then for every timestamp, creating a bookmark.
That's a good first start I think.
Quoted 7 lines
Quoted 4 lines
so [12:20:23 , 12:23:22] some range
would give
bookmark 1-> 12:20:23 -> some range start
bookmark 2-> 12:23:22 -> some range endI'm not sure exactly how clear it will be with overlapping ranges in the TraceCompass list of bookmarks, but I guess it will be handled in a next phase.
I think, having 2 bookmarks for a time range would not be very intuitive for the user. There should be only one bookmark for each area of interest regardless if it is for a single timestamp or a time range. In the bookmark implementation you can save various attributes. You can store the start and end time for a range. If the user opens a bookmark with end time, Trace Compass will open the trace, seek to the start time and then set the current active time range between start time and end time. If the end time isn't set the last step is not done. The Trace Compass framework has all the means to do these steps.
Quoted 88 lines
Quoted 8 lines
and
[22:33:44 ] [key : value] [name = value] [id -> ref ] ... would parse as everything after the timestamp would be read as a string.text [22:33:44] text2
would discard text.Is this ok?
Yes, let's start with that. As we discussed, if we need to output differently from the analyses, we will make the changes.
Quoted 2 lines
[1]note, it will not work with https://github.com/lttng/lttng-analyses
where dash is used.Oh if you are referring to the "Work in progress" section, you are right, don't worry about this, it is a prototype and cannot even be used from the scripts in the master branch. When this is ready it will use the standard format.
Thanks,
Julien
Quoted 58 lines · Julien Desfossez
Quoted 57 lines · Mathieu Desnoyers
Quoted 42 lines
----- Original Message -----
Quoted 41 lines
From: "Matthew Khouzam" <[email protected]>
To: "tracecompass developer discussions" <[email protected]>, [email protected], [email protected]
Sent: Friday, February 6, 2015 5:13:42 PM
Subject: [diamon-discuss] Parsing external datasources in trace compass for regions of interest in a traceHello,
I am right now looking getting external datasources to supply information for a trace, like a region of interest.
Background: we can run an analysis on a trace manually, in python, excel or what have you and come up with something interesting, like: around 12:30pm something happened to a trace. Let's dig deeper with tracecompass.
To make this work: we need to find a way to import the data, so we should agree on a standard data standard. I personally like the LTTng analysis outputs so I am picking that for now. :)
Here are some formats that could be acceptable as regions of interest
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
boookmark
* [ 2015-01-15 12:18:37.216484041] name boookmark
* [12:18:37.216484041] name boookmark
* [ 12:18:37.216484041, 12:18:53.821580313] name boookmark
* [ 2014-12-12 17:29:43.802588035] name with space
* [ 2014-12-12 17:29:43] irrational title
* [17:29:43.802588035] rational title
* [ 17:29:43] test test test
* [17:29:43,17:29:44] thing
* [12:18:37.216484041 ] (no text)When there is one timestamp, we will create one bookmark. When there is a range, we would create two bookmarks in a given trace, the first one being appended with " start" the second with " end"
This should allow tracecompass to marginally benefit from the lttng analyses that are looking pretty nice, and later perhaps from tcp analyses from tshark and syslog stuff.
Quoted 8 lines
Quoted 1 line
Does anyone have an objection to this way of working?
Do you intend on supporting timestamps that appear elsewhere than the beginning of lines ? Do you consider all text at the right of the timestamp or timestamp range to be part of the naming of the region of interest linked to the timestamp/timestamp range ?
Julien, did we want to reserve [] for only timestamp range or we also want them for single timestamps ?
Right now the [] are only valid with time ranges (they are not accepted with --begin and --end). I don't mind allowing the brackets for all timestamps (like in dmesg), we still have the "," to identify if it's a range or not.
But for convenience, they will remain optional for --begin and --end.Quoted 4 lines
Quoted 4 lines
We just have to keep in mind, that [32,64] could be a range of values/frequency and not necessarily timestamps.
Julien
_______________________________________________
diamon-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss
Hi Matthew
See below for some comments.
/Bernd
Quoted 7 lines · Julien Desfossez
Quoted 4 lines · Matthew Khouzam
Great feedback. I think for a first draft, this will work out swimmingly.
Right now, I was actually finding the first group of [], splitting with "," [1], then for every timestamp, creating a bookmark.
That's a good first start I think.
Quoted 7 lines
Quoted 4 lines
so [12:20:23 , 12:23:22] some range
would give
bookmark 1-> 12:20:23 -> some range start
bookmark 2-> 12:23:22 -> some range endI'm not sure exactly how clear it will be with overlapping ranges in the TraceCompass list of bookmarks, but I guess it will be handled in a next phase.
I think, having 2 bookmarks for a time range would not be very intuitive for the user. There should be only one bookmark for each area of interest regardless if it is for a single timestamp or a time range. In the bookmark implementation you can save various attributes. You can store the start and end time for a range. If the user opens a bookmark with end time, Trace Compass will open the trace, seek to the start time and then set the current active time range between start time and end time. If the end time isn't set the last step is not done. The Trace Compass framework has all the means to do these steps.
Quoted 88 lines
Quoted 8 lines
and
[22:33:44 ] [key : value] [name = value] [id -> ref ] ... would parse as everything after the timestamp would be read as a string.text [22:33:44] text2
would discard text.Is this ok?
Yes, let's start with that. As we discussed, if we need to output differently from the analyses, we will make the changes.
Quoted 2 lines
[1]note, it will not work with https://github.com/lttng/lttng-analyses
where dash is used.Oh if you are referring to the "Work in progress" section, you are right, don't worry about this, it is a prototype and cannot even be used from the scripts in the master branch. When this is ready it will use the standard format.
Thanks,
Julien
Quoted 58 lines · Julien Desfossez
Quoted 57 lines · Mathieu Desnoyers
Quoted 42 lines
----- Original Message -----
Quoted 41 lines
From: "Matthew Khouzam" <[email protected]>
To: "tracecompass developer discussions" <[email protected]>, [email protected], [email protected]
Sent: Friday, February 6, 2015 5:13:42 PM
Subject: [diamon-discuss] Parsing external datasources in trace compass for regions of interest in a traceHello,
I am right now looking getting external datasources to supply information for a trace, like a region of interest.
Background: we can run an analysis on a trace manually, in python, excel or what have you and come up with something interesting, like: around 12:30pm something happened to a trace. Let's dig deeper with tracecompass.
To make this work: we need to find a way to import the data, so we should agree on a standard data standard. I personally like the LTTng analysis outputs so I am picking that for now. :)
Here are some formats that could be acceptable as regions of interest
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
boookmark
* [ 2015-01-15 12:18:37.216484041] name boookmark
* [12:18:37.216484041] name boookmark
* [ 12:18:37.216484041, 12:18:53.821580313] name boookmark
* [ 2014-12-12 17:29:43.802588035] name with space
* [ 2014-12-12 17:29:43] irrational title
* [17:29:43.802588035] rational title
* [ 17:29:43] test test test
* [17:29:43,17:29:44] thing
* [12:18:37.216484041 ] (no text)When there is one timestamp, we will create one bookmark. When there is a range, we would create two bookmarks in a given trace, the first one being appended with " start" the second with " end"
This should allow tracecompass to marginally benefit from the lttng analyses that are looking pretty nice, and later perhaps from tcp analyses from tshark and syslog stuff.
Quoted 8 lines
Quoted 1 line
Does anyone have an objection to this way of working?
Do you intend on supporting timestamps that appear elsewhere than the beginning of lines ? Do you consider all text at the right of the timestamp or timestamp range to be part of the naming of the region of interest linked to the timestamp/timestamp range ?
Julien, did we want to reserve [] for only timestamp range or we also want them for single timestamps ?
Right now the [] are only valid with time ranges (they are not accepted with --begin and --end). I don't mind allowing the brackets for all timestamps (like in dmesg), we still have the "," to identify if it's a range or not.
But for convenience, they will remain optional for --begin and --end.Quoted 4 lines
Quoted 4 lines
We just have to keep in mind, that [32,64] could be a range of values/frequency and not necessarily timestamps.
Julien
_______________________________________________
diamon-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss
Quoted 4 lines · Matthew Khouzam
Great feedback. I think for a first draft, this will work out swimmingly.
Right now, I was actually finding the first group of [], splitting with "," [1], then for every timestamp, creating a bookmark.
That's a good first start I think.
Quoted 5 lines
so [12:20:23 , 12:23:22] some range
would give
bookmark 1-> 12:20:23 -> some range start
bookmark 2-> 12:23:22 -> some range end
I'm not sure exactly how clear it will be with overlapping ranges in the TraceCompass list of bookmarks, but I guess it will be handled in a next phase.
Quoted 9 lines
and
[22:33:44 ] [key : value] [name = value] [id -> ref ] ... would parse as everything after the timestamp would be read as a string.text [22:33:44] text2
would discard text.Is this ok?
Yes, let's start with that. As we discussed, if we need to output differently from the analyses, we will make the changes.
Quoted 2 lines
[1]note, it will not work with https://github.com/lttng/lttng-analyses
where dash is used.
Oh if you are referring to the "Work in progress" section, you are right, don't worry about this, it is a prototype and cannot even be used from the scripts in the master branch. When this is ready it will use the standard format.
Thanks,
Julien
Quoted 65 lines · Julien Desfossez
Quoted 62 lines · Mathieu Desnoyers
Quoted 42 lines
----- Original Message -----
Quoted 41 lines
From: "Matthew Khouzam" <[email protected]>
To: "tracecompass developer discussions" <[email protected]>, [email protected], [email protected]
Sent: Friday, February 6, 2015 5:13:42 PM
Subject: [diamon-discuss] Parsing external datasources in trace compass for regions of interest in a traceHello,
I am right now looking getting external datasources to supply information for a trace, like a region of interest.
Background: we can run an analysis on a trace manually, in python, excel or what have you and come up with something interesting, like: around 12:30pm something happened to a trace. Let's dig deeper with tracecompass.
To make this work: we need to find a way to import the data, so we should agree on a standard data standard. I personally like the LTTng analysis outputs so I am picking that for now. :)
Here are some formats that could be acceptable as regions of interest
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
boookmark
* [ 2015-01-15 12:18:37.216484041] name boookmark
* [12:18:37.216484041] name boookmark
* [ 12:18:37.216484041, 12:18:53.821580313] name boookmark
* [ 2014-12-12 17:29:43.802588035] name with space
* [ 2014-12-12 17:29:43] irrational title
* [17:29:43.802588035] rational title
* [ 17:29:43] test test test
* [17:29:43,17:29:44] thing
* [12:18:37.216484041 ] (no text)When there is one timestamp, we will create one bookmark. When there is a range, we would create two bookmarks in a given trace, the first one being appended with " start" the second with " end"
This should allow tracecompass to marginally benefit from the lttng analyses that are looking pretty nice, and later perhaps from tcp analyses from tshark and syslog stuff.
Quoted 8 lines
Quoted 1 line
Does anyone have an objection to this way of working?
Do you intend on supporting timestamps that appear elsewhere than the beginning of lines ? Do you consider all text at the right of the timestamp or timestamp range to be part of the naming of the region of interest linked to the timestamp/timestamp range ?
Julien, did we want to reserve [] for only timestamp range or we also want them for single timestamps ?
Right now the [] are only valid with time ranges (they are not accepted with --begin and --end). I don't mind allowing the brackets for all timestamps (like in dmesg), we still have the "," to identify if it's a range or not.
But for convenience, they will remain optional for --begin and --end.We just have to keep in mind, that [32,64] could be a range of values/frequency and not necessarily timestamps.
Julien
Great feedback. I think for a first draft, this will work out swimmingly.
Right now, I was actually finding the first group of [], splitting with "," [1], then for every timestamp, creating a bookmark.
so [12:20:23 , 12:23:22] some range
would give
bookmark 1-> 12:20:23 -> some range start
bookmark 2-> 12:23:22 -> some range end
and
[22:33:44 ] [key : value] [name = value] [id -> ref ] ... would parse as everything after the timestamp would be read as a string.
text [22:33:44] text2
would discard text.
Is this ok?
[1]note, it will not work with https://github.com/lttng/lttng-analyses
where dash is used.
Quoted 62 lines · Julien Desfossez
Quoted 42 lines · Mathieu Desnoyers
----- Original Message -----
Quoted 41 lines
From: "Matthew Khouzam" <[email protected]>
To: "tracecompass developer discussions" <[email protected]>, [email protected], [email protected]
Sent: Friday, February 6, 2015 5:13:42 PM
Subject: [diamon-discuss] Parsing external datasources in trace compass for regions of interest in a traceHello,
I am right now looking getting external datasources to supply information for a trace, like a region of interest.
Background: we can run an analysis on a trace manually, in python, excel or what have you and come up with something interesting, like: around 12:30pm something happened to a trace. Let's dig deeper with tracecompass.
To make this work: we need to find a way to import the data, so we should agree on a standard data standard. I personally like the LTTng analysis outputs so I am picking that for now. :)
Here are some formats that could be acceptable as regions of interest
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
boookmark
* [ 2015-01-15 12:18:37.216484041] name boookmark
* [12:18:37.216484041] name boookmark
* [ 12:18:37.216484041, 12:18:53.821580313] name boookmark
* [ 2014-12-12 17:29:43.802588035] name with space
* [ 2014-12-12 17:29:43] irrational title
* [17:29:43.802588035] rational title
* [ 17:29:43] test test test
* [17:29:43,17:29:44] thing
* [12:18:37.216484041 ] (no text)When there is one timestamp, we will create one bookmark. When there is a range, we would create two bookmarks in a given trace, the first one being appended with " start" the second with " end"
This should allow tracecompass to marginally benefit from the lttng analyses that are looking pretty nice, and later perhaps from tcp analyses from tshark and syslog stuff.
Quoted 8 lines
Quoted 1 line
Does anyone have an objection to this way of working?
Do you intend on supporting timestamps that appear elsewhere than the beginning of lines ? Do you consider all text at the right of the timestamp or timestamp range to be part of the naming of the region of interest linked to the timestamp/timestamp range ?
Julien, did we want to reserve [] for only timestamp range or we also want them for single timestamps ?
Right now the [] are only valid with time ranges (they are not accepted with --begin and --end). I don't mind allowing the brackets for all timestamps (like in dmesg), we still have the "," to identify if it's a range or not.
But for convenience, they will remain optional for --begin and --end.We just have to keep in mind, that [32,64] could be a range of values/frequency and not necessarily timestamps.
Julien
Quoted 44 lines · Mathieu Desnoyers
----- Original Message -----
Quoted 37 lines
From: "Matthew Khouzam" <[email protected]>
To: "tracecompass developer discussions" <[email protected]>, [email protected], [email protected]
Sent: Friday, February 6, 2015 5:13:42 PM
Subject: [diamon-discuss] Parsing external datasources in trace compass for regions of interest in a traceHello,
I am right now looking getting external datasources to supply information for a trace, like a region of interest.
Background: we can run an analysis on a trace manually, in python, excel or what have you and come up with something interesting, like: around 12:30pm something happened to a trace. Let's dig deeper with tracecompass.
To make this work: we need to find a way to import the data, so we should agree on a standard data standard. I personally like the LTTng analysis outputs so I am picking that for now. :)
Here are some formats that could be acceptable as regions of interest
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
boookmark
* [ 2015-01-15 12:18:37.216484041] name boookmark
* [12:18:37.216484041] name boookmark
* [ 12:18:37.216484041, 12:18:53.821580313] name boookmark
* [ 2014-12-12 17:29:43.802588035] name with space
* [ 2014-12-12 17:29:43] irrational title
* [17:29:43.802588035] rational title
* [ 17:29:43] test test test
* [17:29:43,17:29:44] thing
* [12:18:37.216484041 ] (no text)When there is one timestamp, we will create one bookmark. When there is a range, we would create two bookmarks in a given trace, the first one being appended with " start" the second with " end"
Quoted 5 lines
This should allow tracecompass to marginally benefit from the lttng analyses that are looking pretty nice, and later perhaps from tcp analyses from tshark and syslog stuff.
Does anyone have an objection to this way of working?
Quoted 7 lines
Do you intend on supporting timestamps that appear elsewhere than the beginning of lines ? Do you consider all text at the right of the timestamp or timestamp range to be part of the naming of the region of interest linked to the timestamp/timestamp range ?
Julien, did we want to reserve [] for only timestamp range or we also want them for single timestamps ?
Right now the [] are only valid with time ranges (they are not accepted with --begin and --end). I don't mind allowing the brackets for all timestamps (like in dmesg), we still have the "," to identify if it's a range or not.
But for convenience, they will remain optional for --begin and --end.
We just have to keep in mind, that [32,64] could be a range of values/frequency and not necessarily timestamps.
Julien
----- Original Message -----
Quoted 37 lines
From: "Matthew Khouzam" <[email protected]>
To: "tracecompass developer discussions" <[email protected]>, [email protected], [email protected]
Sent: Friday, February 6, 2015 5:13:42 PM
Subject: [diamon-discuss] Parsing external datasources in trace compass for regions of interest in a traceHello,
I am right now looking getting external datasources to supply information for a trace, like a region of interest.
Background: we can run an analysis on a trace manually, in python, excel or what have you and come up with something interesting, like: around 12:30pm something happened to a trace. Let's dig deeper with tracecompass.
To make this work: we need to find a way to import the data, so we should agree on a standard data standard. I personally like the LTTng analysis outputs so I am picking that for now. :)
Here are some formats that could be acceptable as regions of interest
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
* [2015-01-15 12:18:37.216484041, 2015-01-15 12:18:53.821580313] name
boookmark
* [ 2015-01-15 12:18:37.216484041] name boookmark
* [12:18:37.216484041] name boookmark
* [ 12:18:37.216484041, 12:18:53.821580313] name boookmark
* [ 2014-12-12 17:29:43.802588035] name with space
* [ 2014-12-12 17:29:43] irrational title
* [17:29:43.802588035] rational title
* [ 17:29:43] test test test
* [17:29:43,17:29:44] thing
* [12:18:37.216484041 ] (no text)When there is one timestamp, we will create one bookmark. When there is a range, we would create two bookmarks in a given trace, the first one being appended with " start" the second with " end"
Quoted 5 lines
This should allow tracecompass to marginally benefit from the lttng analyses that are looking pretty nice, and later perhaps from tcp analyses from tshark and syslog stuff.
Does anyone have an objection to this way of working?
Do you intend on supporting timestamps that appear elsewhere than the beginning of lines ? Do you consider all text at the right of the timestamp or timestamp range to be part of the naming of the region of interest linked to the timestamp/timestamp range ?
Julien, did we want to reserve [] for only timestamp range or we also want them for single timestamps ?
Thanks,
Mathieu
Quoted 6 lines
_______________________________________________
diamon-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/diamon-discuss
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com