Skip to contentSkip to conversation

ocfs2: add trace event support for ocfs2

PATCH New Wengang Wang 3 participants 7 messages Sep 16, 2009 – Oct 15, 2009 OCFS2

lore
Share & tools
lore
#0 lore raw

cover ocfs2: add trace event support for ocfs2

following patches add trace event support for ocfs2. for tracepoint, see Documentation/trace/tracepoints.txt for trace events, see Documentation/trace/events.txt

the first patch add file include/trace/events/ocfs2.h to kernel tree. it is used to store trace declarations. it's contents are self-explained.
NOTE that each ocfs2 header file should be above "#include <linux/tracepoint.h>"

the second one is an example which adds trace event to ocfs2_file_aio_read(). NOTE that CREATE_TRACE_POINTS must be defined before including trace/event/ocfs2.h when trace event is used.

enable/disable the trace event:
1) debugfs is need to be mount for using trace event. mount -t debugfs debugfs /sys/kernel/debug(or somewhere else)
2) enable/disable some specific event,

	echo '1'/'0'>/sys/kernel/debug/tracing/events/ocfs2/ocfs2_file_aio_read/enable

3) to see the trace log,
cat /sys/kernel/debug/tracing/trace

an output example is:
[root at cool ocfs2_file_aio_read]# cat /sys/kernel/debug/tracing/trace # tracer: nop
#
# TASK-PID CPU# TIMESTAMP FUNCTION
# | | | | |

             cat-2587  [001]   708.929889: ocfs2_file_aio_read: 0xf544bed8 0xf544bf5c 1 0

             cat-2587  [000]   708.944128: ocfs2_file_aio_read: 0xf544bed8 0xf544bf5c 1 43

examples of using filter.
1) if pid is 0, do the trace:

	echo "common_pid == 0" >/sys/kernel/debug/tracing/events/ocfs2/ocfs2_file_aio_read/filter
2) if group id not 0, do the trace:
	echo "common_tgid != 0" >filter

for filter expression, so far, see http://article.gmane.org/gmane.linux.kernel/887690?

#1 lore raw

cover ocfs2: add trace event support for ocfs2

Quoted 11 lines · Wengang Wang

following patches add trace event support for ocfs2. for tracepoint, see Documentation/trace/tracepoints.txt for trace events, see Documentation/trace/events.txt

the first patch add file include/trace/events/ocfs2.h to kernel tree. it is used to store trace declarations. it's contents are self-explained.
NOTE that each ocfs2 header file should be above "#include <linux/tracepoint.h>"

the second one is an example which adds trace event to ocfs2_file_aio_read(). NOTE that CREATE_TRACE_POINTS must be defined before including trace/event/ocfs2.h when trace event is used.

and CREATE_TRACE_POINTS should be defined in only one .c file please.

regards,
wengang.

Quoted 28 lines

enable/disable the trace event:
1) debugfs is need to be mount for using trace event. mount -t debugfs debugfs /sys/kernel/debug(or somewhere else)
2) enable/disable some specific event,

	echo '1'/'0'>/sys/kernel/debug/tracing/events/ocfs2/ocfs2_file_aio_read/enable

3) to see the trace log,
cat /sys/kernel/debug/tracing/trace

an output example is:
[root at cool ocfs2_file_aio_read]# cat /sys/kernel/debug/tracing/trace # tracer: nop
#
# TASK-PID CPU# TIMESTAMP FUNCTION
# | | | | |

             cat-2587  [001]   708.929889: ocfs2_file_aio_read: 0xf544bed8 0xf544bf5c 1 0

             cat-2587  [000]   708.944128: ocfs2_file_aio_read: 0xf544bed8 0xf544bf5c 1 43

examples of using filter.
1) if pid is 0, do the trace:

	echo "common_pid == 0" >/sys/kernel/debug/tracing/events/ocfs2/ocfs2_file_aio_read/filter
2) if group id not 0, do the trace:
	echo "common_tgid != 0" >filter

for filter expression, so far, see http://article.gmane.org/gmane.linux.kernel/887690?

#2 lore raw

cover ocfs2: add trace event support for ocfs2

In reply to Wengang Wang
Quoted 1 line · Wengang Wang

following patches add trace event support for ocfs2.

Wengang,
This looks like a nice start. Thank you for getting this going.
I'm sorry I haven't gotten back until now, I've been busy.
What I'd like to do next is get a small starting set of
tracepoints. You have one tracepoint in this patch as a good example, but we should get a few more. I don't want every mlog changed into a tracepoint. Some of them are old and useless :-) Instead, I'd like to find information we often want while debugging.
Tao, Tiger, and Sunil, what are parts of the code you often come
across when debugging? Certain mlog() prints you always find really important to see. If you point them out, Wengang can cook up tracepoints for them.

Joel

Signature · 10 lines
"But all my words come back to me
 In shades of mediocrity.
 Like emptiness in harmony
 I need someone to comfort me."

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker at oracle.com
Phone: (650) 506-8127
#3 lore raw

cover ocfs2: add trace event support for ocfs2

Hi all,

Quoted 6 lines · Joel Becker
Quoted 1 line · Wengang Wang

following patches add trace event support for ocfs2.

Wengang,
This looks like a nice start. Thank you for getting this going.
I'm sorry I haven't gotten back until now, I've been busy.

Joel, no problem!

Quoted 5 lines

What I'd like to do next is get a small starting set of
tracepoints. You have one tracepoint in this patch as a good example, but we should get a few more. I don't want every mlog changed into a tracepoint. Some of them are old and useless :-) Instead, I'd like to find information we often want while debugging.

yes, I understand that.

Quoted 4 lines

Tao, Tiger, and Sunil, what are parts of the code you often come
across when debugging? Certain mlog() prints you always find really important to see. If you point them out, Wengang can cook up tracepoints for them.

and yes, welcome to give me suggests for the trace points!!

regards,
wengang.

#4 lore raw

cover ocfs2: add trace event support for ocfs2

In reply to Joel Becker

Hi Joel/Wengang,

Quoted 15 lines · Joel Becker
Quoted 1 line · Wengang Wang

following patches add trace event support for ocfs2.

Wengang,
This looks like a nice start. Thank you for getting this going.
I'm sorry I haven't gotten back until now, I've been busy.
What I'd like to do next is get a small starting set of
tracepoints. You have one tracepoint in this patch as a good example, but we should get a few more. I don't want every mlog changed into a tracepoint. Some of them are old and useless :-) Instead, I'd like to find information we often want while debugging.
Tao, Tiger, and Sunil, what are parts of the code you often come
across when debugging? Certain mlog() prints you always find really important to see. If you point them out, Wengang can cook up tracepoints for them.

aops.c maybe? And recovery/dlm(The most awesome part)?

btw, I think the whole mechanism of b-tree(alloc.c) is quite stable now after tristan's stress test case for refcount tree. :) So no urgent need for this file by now.

Regards,
Tao

Quoted 3 lines

Joel

#5 lore raw

cover ocfs2: add trace event support for ocfs2

Quoted 18 lines · Tao Ma

Hi Joel/Wengang,

Quoted 15 lines · Joel Becker
Quoted 1 line · Wengang Wang

following patches add trace event support for ocfs2.

Wengang,
This looks like a nice start. Thank you for getting this going.
I'm sorry I haven't gotten back until now, I've been busy.
What I'd like to do next is get a small starting set of
tracepoints. You have one tracepoint in this patch as a good example, but we should get a few more. I don't want every mlog changed into a tracepoint. Some of them are old and useless :-) Instead, I'd like to find information we often want while debugging.
Tao, Tiger, and Sunil, what are parts of the code you often come
across when debugging? Certain mlog() prints you always find really important to see. If you point them out, Wengang can cook up tracepoints for them.

Quoted 1 line

aops.c maybe? And recovery/dlm(The most awesome part)?

But what things? We're not just going to replace every mlog,
and we're not going to do every function. We really want to figure out what information is actually useful and add tracepoints for it. So if you have particular functions you need data on, or particular mlog() calls you always like to see, we'd love to hear about it.
Wengang, dlmglue.c is definitely a place to look at. I'd love
to see events for the inc/dec of l_ro_holders and l_ex_holders. I'd also love to see the mlogs for 'BAST fired' and 'UNLOCK AST fired' converted, and a similar event for the regular AST.

Joel

Signature · 8 lines
"To fall in love is to create a religion that has a fallible god."
        -Jorge Luis Borges

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker at oracle.com
Phone: (650) 506-8127
#6 lore raw

cover ocfs2: add trace event support for ocfs2

Quoted 23 lines · Joel Becker
Quoted 21 lines · Tao Ma

Hi Joel/Wengang,

Quoted 17 lines · Joel Becker
Quoted 2 lines · Wengang Wang

following patches add trace event support for ocfs2.

Wengang,
This looks like a nice start. Thank you for getting this going.
I'm sorry I haven't gotten back until now, I've been busy.
What I'd like to do next is get a small starting set of
tracepoints. You have one tracepoint in this patch as a good example, but we should get a few more. I don't want every mlog changed into a tracepoint. Some of them are old and useless :-) Instead, I'd like to find information we often want while debugging.
Tao, Tiger, and Sunil, what are parts of the code you often come
across when debugging? Certain mlog() prints you always find really important to see. If you point them out, Wengang can cook up tracepoints for them.

Quoted 9 lines
Quoted 2 lines

aops.c maybe? And recovery/dlm(The most awesome part)?

But what things? We're not just going to replace every mlog,
and we're not going to do every function. We really want to figure out what information is actually useful and add tracepoints for it. So if you have particular functions you need data on, or particular mlog() calls you always like to see, we'd love to hear about it.

No idea of recovery/dlm since I don't have much experience in it. As for aops.c, I think the most important part is ocfs2_write_begin_no_lock. It is very complicated, sometimes it will write inline_data, sometimes CoW a refcount record, sometimes makes a extent written. It is helpful for us to know some detail information about it.

Regards,
Tao

Quoted 8 lines

Wengang, dlmglue.c is definitely a place to look at. I'd love
to see events for the inc/dec of l_ro_holders and l_ex_holders. I'd also love to see the mlogs for 'BAST fired' and 'UNLOCK AST fired' converted, and a similar event for the regular AST.

Joel