Benchmarks of kernel tracing options 2 (ftrace, lttng and perf)
Hi all,
This is a response to a benchmark, submitted a few weeks ago, comparing kernel tracing options.
http://lkml.org/lkml/2010/10/28/422
We followed the methodology described in the link bellow, but using the shellscripts posted there to reproduce autotest scripts.
http://lkml.org/lkml/2010/10/28/261
We disabled the extra syscall tracing on lttng, for a fair comparison.
http://lkml.org/lkml/2010/10/28/290
Average results with tracing "on":
lttng: 220 ns ftrace: 260 ns perf: 740 ns
E5405 system
kernel 2.6.36
-lttng 0.239 + 0.19.2modules + sys_getuid tracepoint + sys_getuid probe
+ remove syscall_trace
-ftrace and perf + sys_getuid tracepoint
Quoted 10 lines · Douglas Santos
Hi all,
This is a response to a benchmark, submitted a few weeks ago, comparing kernel tracing options.
http://lkml.org/lkml/2010/10/28/422We followed the methodology described in the link bellow, but using the shellscripts posted there to reproduce autotest scripts.
http://lkml.org/lkml/2010/10/28/261
Quoted 7 lines
We disabled the extra syscall tracing on lttng, for a fair comparison.
http://lkml.org/lkml/2010/10/28/290Average results with tracing "on":
lttng: 220 ns ftrace: 260 ns
Heh, so ftrace got worse with the new kernel?
-- Steve
Quoted 10 lines
perf: 740 ns
E5405 system
kernel 2.6.36(quoted diff) +0
-2-lttng 0.239 + 0.19.2modules + sys_getuid tracepoint + sys_getuid probe + remove syscall_trace -ftrace and perf + sys_getuid tracepoint
Quoting Steven Rostedt <[email protected]>:
Quoted 15 lines · Douglas Santos
Quoted 3 lines
Hi all,
This is a response to a benchmark, submitted a few weeks ago, comparing
kernel
Quoted 10 lines
tracing options.
http://lkml.org/lkml/2010/10/28/422We followed the methodology described in the link bellow, but using the shellscripts posted there to reproduce autotest scripts.
http://lkml.org/lkml/2010/10/28/261We disabled the extra syscall tracing on lttng, for a fair comparison.
http://lkml.org/lkml/2010/10/28/290
Quoted 6 lines
Quoted 4 lines
Average results with tracing "on":
lttng: 220 ns ftrace: 260 nsHeh, so ftrace got worse with the new kernel?
The previous bench was doing tracing "on" minus "off" average results. They also used autotest scripts, not sure if it does exactly the same thing.
I'll check if we missed something.
Quoted 22 lines · Douglas Santos
Quoting Steven Rostedt <[email protected]>:
Quoted 21 lines · Douglas Santos
Quoted 3 lines
Hi all,
This is a response to a benchmark, submitted a few weeks ago, comparing
kernel
Quoted 7 lines
tracing options.
http://lkml.org/lkml/2010/10/28/422We followed the methodology described in the link bellow, but using the shellscripts posted there to reproduce autotest scripts.
http://lkml.org/lkml/2010/10/28/261Quoted 7 lines
We disabled the extra syscall tracing on lttng, for a fair comparison.
http://lkml.org/lkml/2010/10/28/290Average results with tracing "on":
lttng: 220 ns
ftrace: 260 nsHeh, so ftrace got worse with the new kernel?
Steve, can you explain how you're drawing that conclusion? Did Douglas run this benchmark before on a previous kernel (I didn't see it if so)?
- You can't directly compare to my results because of different hardware.
- The methodology for lttng is different (syscall tracing was removed).
- My results were also on 2.6.36
Quoted 3 lines
The previous bench was doing tracing "on" minus "off" average results. They also used autotest scripts, not sure if it does exactly the same thing.
I think the subtraction is important, or it is at least important to see what the "off" result is as a baseline of comparison. Otherwise, a huge portion of the measurement is the cost of making the syscall itself.
Quoted 5 lines
I'll check if we missed something.
Quoted 20 lines · David Sharp
Quoted 18 lines · Douglas Santos
Quoting Steven Rostedt <[email protected]>:
Quoted 17 lines · Douglas Santos
Quoted 3 lines
Hi all,
This is a response to a benchmark, submitted a few weeks ago, comparing
kernel
Quoted 12 lines
tracing options.
http://lkml.org/lkml/2010/10/28/422We followed the methodology described in the link bellow, but using the shellscripts posted there to reproduce autotest scripts.
http://lkml.org/lkml/2010/10/28/261We disabled the extra syscall tracing on lttng, for a fair comparison.
http://lkml.org/lkml/2010/10/28/290Average results with tracing "on":
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
lttng: 220 ns ftrace: 260 nsHeh, so ftrace got worse with the new kernel?
Steve, can you explain how you're drawing that conclusion? Did Douglas run this benchmark before on a previous kernel (I didn't see it if so)?
Oops, no, I was thinking that this was from your tests. I remember asking you to try the new kernel. I think I got you and Douglas confused :-)
-- Steve