Machine Interface
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 13 lines · Jonathan Rajotte
Hello all,
After speaking with Michel Dagenais, Geneviève Bastien, folks over at EfficiOs and Ericsson, a machine interface for lttng-tool would be a nice feature to have. Olivier Cotte and me will be working on MI for the
next few weeks.The main objective would be to reduce parsing complexity
in control tools.
ex:Regex used in TMF control service. (org.eclipse.linuxtools.internal.lttng2.ui.views.control.service.LTTngControlService.java)Multiples solutions come to mind:
Review
Quoted 3 lines
(quoted diff) +0
-3-XML -JSON -YAML
YAML sounds very attractive in this context.
It's easily parseable even from a shell script. If you need to automate LTTng tracing tasks on a small embedded target busybox is all you have!
Best,
Paul
Quoted 21 lines
-Custom syntax (like GDB Machine interface* as proposed by Alexandre
Montplaisir)XML/JSON/YAML promote extensibility and ease of development for new tools dependant on lttng-tools by being established standards.
On the other hand, they add dependencies to lttng-tools where a custom syntax would not.Machine interface output would be generated only with --mi option and could contain additional information than regular output.
We are focusing on output right now but we may eventually add a 'mi' command for input if necessary.
What do you think ?
Thanks
Signature · 3 lines
Paul Woegerer, SW Development Engineer Sourcery Analyzer <http://go.mentor.com/sourceryanalyzer> Mentor Graphics, Embedded Software Division
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 13 lines · Paul
Quoted 12 lines · Jonathan Rajotte
Hello all,
After speaking with Michel Dagenais, Geneviève Bastien, folks over at EfficiOs and Ericsson, a machine interface for lttng-tool would be a nice feature to have. Olivier Cotte and me will be working on MI for the
next few weeks.The main objective would be to reduce parsing complexity
in control tools.
ex:Regex used in TMF control service. (org.eclipse.linuxtools.internal.lttng2.ui.views.control.service.LTTngControlService.java)
Quoted 6 lines
Review
Quoted 4 lines
Multiples solutions come to mind:
(quoted diff) +0
-3-XML -JSON -YAMLYAML sounds very attractive in this context.
I am also fond of the yaml syntax. It is very easy to read/write by a human. Not sure how much this matters though, since is a machine interface.
Quoted 2 lines
It's easily parseable even from a shell script. If you need to automate LTTng tracing tasks on a small embedded target busybox is all you have!
This point is relevant for me, since I actually intend to use this to enhance bash completion, which is done with a bash script.
I never thought I would say that one day, but I think that XML could have an advantage here. Parsing yaml with regexp (such as in a shell script) is easier than XML, but it's still not bulletproof. Consider this eventual yaml structure I just made up for the output of the session list.
sessions:
- name: my_session
domain: kernel
events:
- name: sched_switch
filter: prev_pid == 1234
- name: sched_migrate_task
otherproperty: "something"
- name: another_session
domain: ust
events:
- name: event_hello
- name: event_bonjour
loglevel: 3
If you want to get the list of sessions, you'll probably want to grep the "name: " entries, but then event names would match as well. Of course, names could be different, or you could grep for the appropriate amount of spaces/tabs at the beginning of the line. But then it also depends if the name dictionary entry is the first, then it will have a "-" in front... It's possible, but it will also be very easy to break scripts.
With XML, there are tools (xmlstarlet, xmllint, probably others) that allow you to evaluate XPath expressions and thus not depend of the formatting. For example, an equivalent XML schema could be
<lttng>
<session name="my_session" domain="kernel">
<event name="sched_switch" filter="prev_pid == 1234" /> <event name="sched_migrate_task" otherproperty="something" />
</session>
<session name="another_session" domain="ust">
<event name="event_hello" />
<event name="event_bonjour" loglevel="3" />
</session>
</lttng>
To get a plain list of sessions, you could then do
$ lttng list --mi | xmlstarlet sel -T -t -v "/lttng/session/@name" -n my_session another_session
Maybe such tools exist for yaml?
Also, I understand your point for embedded devices though, where space is limited and you don't want to load an XML lib/tool just for that. If the design of the MI is somewhat modular though, it could be possible to support more than one output format, with the different options available at configure time.
Quoted 5 lines
Best,
PaulQuoted 2 lines
-Custom syntax (like GDB Machine interface* as proposed by Alexandre
Montplaisir)
I would suggest not going for a custom syntax... that sounds like more work for more chances of bugs.
Quoted 4 lines
Quoted 4 lines
XML/JSON/YAML promote extensibility and ease of development for new tools dependant on lttng-tools by being established standards.
On the other hand, they add dependencies to lttng-tools where a custom syntax would not.
I would suggest making the dependency optional. If you don't have the lib, or don't want to depend on the lib, have a --disable-something at configure time.
Quoted 25 lines
Quoted 6 lines
Machine interface output would be generated only with --mi option and could contain additional information than regular output.
We are focusing on output right now but we may eventually add a 'mi' command for input if necessary.
Quoted 7 lines
What do you think ?
Thanks
--
Paul Woegerer, SW Development Engineer
Sourcery Analyzer <http://go.mentor.com/sourceryanalyzer>
Mentor Graphics, Embedded Software Division_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
A message that is not in the archive (sent off-list or not yet received): CAFXXi0=enX3ERpH1Q3R-P4ii2wU7TE2wX_NEcXYF0iEXKCW6yA@mail.gmail.com
Hi Simon,
Quoted 19 lines · Simon Marchi
script) is easier than XML, but it's still not bulletproof. Consider this eventual yaml structure I just made up for the output of the session list.
sessions:
- name:my_session
domain: kernel
events:
- name:sched_switch
filter:prev_pid== 1234
- name:sched_migrate_task
otherproperty: "something"
- name:another_session
domain: ust
events:
- name:event_hello
- name:event_bonjour
loglevel: 3
Quoted 7 lines
If you want to get the list of sessions, you'll probably want to grep the "name: " entries, but then event names would match as well. Of course, names could be different, or you could grep for the appropriate amount of spaces/tabs at the beginning of the line. But then it also depends if the name dictionary entry is the first, then it will have a "-" in front... It's possible, but it will also be very easy to break scripts.
The following simple script below would e.g. give you the list of sessions for you example above. To demonstrate the flexibility it also generates the event list for session my_session. This is just a quick prototype but it should make clear that YAML is considerably easier to parse from a shell script than any other option (and also reasonably robust). Also notice that no external helpers (like grep) are needed.
list_of_sessions=''
current_session=''
my_session_events=''
while IFS= read -r; do
line=$REPLY
case "${line}" in
" - "*)
;&
" "*)
L2_key=${line:8}
L2_key=${L2_key%:*}
let vpos=2+8+${#L2_key}
L2_value=${line:${vpos}}
;;
" - "*)
# New L1 list elem -> reset L2 state
L2_key=''
L2_value=''
;&
" "*)
L1_key=${line:4}
L1_key=${L1_key%:*}
let vpos=2+4+${#L1_key}
L1_value=${line:${vpos}}
;;
esac
echo "($L1_key,$L1_value), ($L2_key,$L2_value)"
# Collect the session names
if [ "$L1_key" = "name" ]; then
current_session="$L1_value"
list_of_sessions="$list_of_sessions $current_session"
fi
# Which events are in session my_session ?
if [ "$current_session" = "my_session" ] && [ "$L1_key" = "events" ] && [ "$L2_key" = "name" ]; then
my_session_events="$my_session_events $L2_value"
fidone
echo
echo List of sessions: $list_of_sessions
echo my_session events: $my_session_events
echo
Thanks,
Paul
Signature · 3 lines
Paul Woegerer, SW Development Engineer Sourcery Analyzer <http://go.mentor.com/sourceryanalyzer> Mentor Graphics, Embedded Software Division
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 30 lines · Paul
Hi Simon,
Quoted 19 lines · Simon Marchi
script) is easier than XML, but it's still not bulletproof. Consider this eventual yaml structure I just made up for the output of the session list.
sessions:
- name:my_session
domain: kernel
events:
- name:sched_switch
filter:prev_pid== 1234
- name:sched_migrate_task
otherproperty: "something"
- name:another_session
domain: ust
events:
- name:event_hello
- name:event_bonjour
loglevel: 3Quoted 7 lines
If you want to get the list of sessions, you'll probably want to grep the "name: " entries, but then event names would match as well. Of course, names could be different, or you could grep for the appropriate amount of spaces/tabs at the beginning of the line. But then it also depends if the name dictionary entry is the first, then it will have a "-" in front... It's possible, but it will also be very easy to break scripts.
Quoted 7 lines
The following simple script below would e.g. give you the list of sessions for you example above. To demonstrate the flexibility it also generates the event list for session
my_session. This is just a quick prototype but it should make clear that YAML is considerably easier to parse from a shell script than any other option (and also reasonably robust). Also notice that no external helpers (like grep) are needed.
This is impressive scripting! Although I can see this growing a little bit out of control for more complex yaml trees. I agree with you that yaml is the easiest of them to parse in a pure shell script since the formatting is more restricted. But if you're on a machine that can take it, using a solution based on an XPath tool (or equivalent) will make it so much more maintainable.
Quoted 52 lines
list_of_sessions=''
current_session=''
my_session_events=''while IFS= read -r; do
line=$REPLY case "${line}" in " - "*) ;& " "*) L2_key=${line:8} L2_key=${L2_key%:*} let vpos=2+8+${#L2_key} L2_value=${line:${vpos}} ;; " - "*) # New L1 list elem -> reset L2 state L2_key='' L2_value='' ;& " "*) L1_key=${line:4} L1_key=${L1_key%:*} let vpos=2+4+${#L1_key} L1_value=${line:${vpos}} ;; esac echo "($L1_key,$L1_value), ($L2_key,$L2_value)" # Collect the session names if [ "$L1_key" = "name" ]; then current_session="$L1_value" list_of_sessions="$list_of_sessions $current_session" fi # Which events are in session my_session ? if [ "$current_session" = "my_session" ] && [ "$L1_key" = "events" ] && [ "$L2_key" = "name" ]; then my_session_events="$my_session_events $L2_value" fidone
echo
echo List of sessions: $list_of_sessions
echomy_sessionevents: $my_session_events
echoThanks,
Paul--
Paul Woegerer, SW Development Engineer
Sourcery Analyzer <http://go.mentor.com/sourceryanalyzer>
Mentor Graphics, Embedded Software Division
Hi Jonathan,
Review
Quoted 6 lines
Multiples solutions come to mind:
(quoted diff) +0
-4-XML -JSON -YAML -Custom syntax (like GDB Machine interface* as proposed by Alexandre Montplaisir)
I would advocate for the use of XML for the machine interface. Two reasons come to mind:
1 - The expression of hierarchy (session, events, etc.) is much easier with XML.
2 - In a recent post [1], it was decided that the configuration load and restore (for sessions) would also use XML. This will introduce a new dependency on libxml. If we were to choose yet another format for the machine interface, we would probably need to introduce another dependency to some library.
Quoted 2 lines
Machine interface output would be generated only with --mi option and could contain additional information than regular output.
Perhaps adding --xml option to the relevant command? This would be similar to svn machine interface [2].
Thanks,
Christian
[1] - http://lists.lttng.org/pipermail/lttng-dev/2013-December/022068.html
[2] - http://svnbook.red-bean.com/en/1.7/svn-book.html#svn.ref.svn.sw
Quoted 43 lines · Jonathan Rajotte
Hello all,
After speaking with Michel Dagenais, Geneviève Bastien, folks over at EfficiOs and Ericsson, a machine interface for lttng-tool would be a nice feature to have. Olivier Cotte and me will be working on MI for the
next few weeks.The main objective would be to reduce parsing complexity
in control tools.
ex:Regex used in TMF control service. (org.eclipse.linuxtools.internal.lttng2.ui.views.control.service.LTTngControlService.java)Multiples solutions come to mind:
(quoted diff) +0
-4-XML -JSON -YAML -Custom syntax (like GDB Machine interface* as proposed by Alexandre Montplaisir)XML/JSON/YAML promote extensibility and ease of development for new tools dependant on lttng-tools by being established standards.
On the other hand, they add dependencies to lttng-tools where a custom syntax would not.Machine interface output would be generated only with --mi option and could contain additional information than regular output.
We are focusing on output right now but we may eventually add a 'mi' command for input if necessary.
What do you think ?
Thanks
*https://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI.html
--
Jonathan Rajotte Julien
Chargé de laboratoire INF1995
Membre MD6
Polytechnique Montréal
Quoted 4 lines
_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
A message that is not in the archive (sent off-list or not yet received): CAGDH53mHYJg-uVd+4Cv=X6ST5Ly+nmE_BQSVmdN3Hb4k7ZKeSw@mail.gmail.com
Quoted 12 lines · Christian Babeux
Hi Jonathan,
Review
Quoted 6 lines
Multiples solutions come to mind:
(quoted diff) +0
-4-XML -JSON -YAML -Custom syntax (like GDB Machine interface* as proposed by Alexandre Montplaisir)I would advocate for the use of XML for the machine interface. Two reasons come to mind:
Quoted 7 lines
1 - The expression of hierarchy (session, events, etc.) is much easier with XML.
2 - In a recent post [1], it was decided that the configuration load and restore (for sessions) would also use XML. This will introduce a new dependency on libxml. If we were to choose yet another format for the machine interface, we would probably need to introduce another dependency to some library.
Good point
+1 for XML
Quoted 6 lines
Quoted 3 lines
Machine interface output would be generated only with --mi option and could contain additional information than regular output.
Perhaps adding --xml option to the relevant command? This would be similar to svn machine interface [2].
How about --machine-interface <output-type>? By default <output-type> would be the chosen format.
This open opportunities for other MI type and have an unified syntax.
--xml option is clear and efficient.We could add an option for each type (--json,--yaml) if needed.
Quoted 57 lines
Thanks,
Christian
[1] - http://lists.lttng.org/pipermail/lttng-dev/2013-December/022068.html
[2] - http://svnbook.red-bean.com/en/1.7/svn-book.html#svn.ref.svn.swQuoted 47 lines · Jonathan Rajotte
Hello all,
After speaking with Michel Dagenais, Geneviève Bastien, folks over at EfficiOs and Ericsson, a machine interface for lttng-tool would be a nice feature to have. Olivier Cotte and me will be working on MI for the
next few weeks.The main objective would be to reduce parsing complexity
in control tools.
ex:Regex used in TMF control service. (org.eclipse.linuxtools.internal.lttng2.ui.views.control.service.LTTngControlService.java)Multiples solutions come to mind:
(quoted diff) +0
-4-XML -JSON -YAML -Custom syntax (like GDB Machine interface* as proposed by Alexandre Montplaisir)XML/JSON/YAML promote extensibility and ease of development for new tools dependant on lttng-tools by being established standards.
On the other hand, they add dependencies to lttng-tools where a custom syntax would not.Machine interface output would be generated only with --mi option and could contain additional information than regular output.
We are focusing on output right now but we may eventually add a 'mi' command for input if necessary.
What do you think ?
Thanks
*https://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI.html
--
Jonathan Rajotte Julien
Chargé de laboratoire INF1995
Membre MD6
Polytechnique Montréal_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Signature · 4 lines
Jonathan Rajotte Juliana Chargé de laboratoire INF1995 Membre MD6 Polytechnique Montréal
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 7 lines
How about --machine-interface <output-type>? By default <output-type> would be the chosen format.
This open opportunities for other MI type and have an unified syntax.
--xml option is clear and efficient.We could add an option for each type (--json,--yaml) if needed.
IMO, supporting multiple output formats would be a tad bit overkill. Moreover, the issue of bringing in additional library dependencies to support multiple output format remains.
Thanks,
Christian
Quoted 92 lines · Jonathan Rajotte
Quoted 12 lines · Christian Babeux
Hi Jonathan,
Review
Quoted 6 lines
Multiples solutions come to mind:
(quoted diff) +0
-4-XML -JSON -YAML -Custom syntax (like GDB Machine interface* as proposed by Alexandre Montplaisir)I would advocate for the use of XML for the machine interface. Two reasons come to mind:
Quoted 7 lines
1 - The expression of hierarchy (session, events, etc.) is much easier with XML.
2 - In a recent post [1], it was decided that the configuration load and restore (for sessions) would also use XML. This will introduce a new dependency on libxml. If we were to choose yet another format for the machine interface, we would probably need to introduce another dependency to some library.
Good point
+1 for XMLQuoted 6 lines
Quoted 3 lines
Machine interface output would be generated only with --mi option and could contain additional information than regular output.
Perhaps adding --xml option to the relevant command? This would be similar to svn machine interface [2].
How about --machine-interface <output-type>? By default <output-type> would be the chosen format.
This open opportunities for other MI type and have an unified syntax.
--xml option is clear and efficient.We could add an option for each type (--json,--yaml) if needed.
Quoted 53 lines
Thanks,
Christian
[1] - http://lists.lttng.org/pipermail/lttng-dev/2013-December/022068.html
[2] - http://svnbook.red-bean.com/en/1.7/svn-book.html#svn.ref.svn.swQuoted 43 lines · Jonathan Rajotte
Hello all,
After speaking with Michel Dagenais, Geneviève Bastien, folks over at EfficiOs and Ericsson, a machine interface for lttng-tool would be a nice feature to have. Olivier Cotte and me will be working on MI for the
next few weeks.The main objective would be to reduce parsing complexity
in control tools.
ex:Regex used in TMF control service. (org.eclipse.linuxtools.internal.lttng2.ui.views.control.service.LTTngControlService.java)Multiples solutions come to mind:
(quoted diff) +0
-4-XML -JSON -YAML -Custom syntax (like GDB Machine interface* as proposed by Alexandre Montplaisir)XML/JSON/YAML promote extensibility and ease of development for new tools dependant on lttng-tools by being established standards.
On the other hand, they add dependencies to lttng-tools where a custom syntax would not.Machine interface output would be generated only with --mi option and could contain additional information than regular output.
We are focusing on output right now but we may eventually add a 'mi' command for input if necessary.
What do you think ?
Thanks
*https://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI.html
--
Jonathan Rajotte Julien
Chargé de laboratoire INF1995
Membre MD6
Polytechnique Montréal
Quoted 10 lines
Quoted 4 lines
Quoted 4 lines
_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev--
Jonathan Rajotte Juliana
Chargé de laboratoire INF1995
Membre MD6
Polytechnique Montréal
_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev