ftracetest: Add a couple of ftrace test cases
Steven Rostedt
maintainerRFC PATCH 1/1 ftracetest: Add a couple of ftrace test cases
[
Masami, I took two of my test scripts and added some basic comments to them and copied them pretty much unchanged into a ftrace directory under test.d. Is this fine, or is there more massaging I need to do to them?
I know the echos don't show up, but I kept them anyway. What should happen with them?
]
Added two test cases to get the feel of adding tests to ftracetest. The two cases are:
function profiling test, to make sure function profiling still works
with function tracing (was a regression)
function graph filter test to make sure that the function graph filter
does filter and also continues to filter when another function tracer is running (like the stack tracer)
--- .../ftrace/test.d/ftrace/fgraph-filter.tc | 111 +++++++++++++++++++++ .../ftrace/test.d/ftrace/func_profiler.tc | 78 +++++++++++++++ 2 files changed, 189 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc create mode 100644 tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +111 -0
@@ -0,0 +1,111 @@ +#!/bin/sh +# description: ftrace - function graph filters + +# Make sure that function graph filtering works, and is not +# affected by other tracers enabled (like stack tracer) + +if ! grep -q function_graph available_tracers; then + echo "no function graph tracer configured" + exit 0; +fi + +if [ ! -f set_ftrace_filter ]; then + echo "set_ftrace_filter not found? Is dynamic ftrace not set?" + exit 0 +fi + +do_reset() { + echo nop > current_tracer + echo 0 > /proc/sys/kernel/stack_tracer_enabled + echo 1 > tracing_on + echo > trace + echo > set_ftrace_filter +} + +echo 0 > tracing_on +# clear trace +echo > trace +# filter something, schedule is always good +if ! echo "schedule" > set_ftrace_filter; then + # test for powerpc 64 + if ! echo ".schedule" > set_ftrace_filter; then + echo "can not enable schedule filter" + exit -1 + fi +fi + +echo function_graph > current_tracer +echo 1 > tracing_on +sleep 1 +# search for functions (has "()" on the line), and make sure +# that only the schedule function was found +count=`cat trace | grep '()' | grep -v schedule | wc -l` +if [ $count -ne 0 ]; then + echo "Graph filtering not working by itself?" + exit -1; +fi + +# Make sure we did find something +count=`cat trace | grep 'schedule()' | wc -l` +if [ $count -eq 0 ]; then + echo "No schedule traces found?" + exit -1 +fi + +echo "Graph filtering works by itself" + +if [ ! -f stack_trace ]; then + echo "Stack tracer not configured, can't continue test" + # stack tracer not configured in this kernel? pass test anyway + do_reset + exit 0; +fi + +echo "Now testing with stack tracer" + +echo 1 > /proc/sys/kernel/stack_tracer_enabled + +echo 0 > tracing_on +echo > trace +echo 1 > tracing_on +sleep 1 + +count=`cat trace | grep '()' | grep -v schedule | wc -l` + +if [ $count -ne 0 ]; then + echo "Graph filtering not working with stack tracer?" + exit -1 +fi + +count=`cat trace | grep 'schedule()' | wc -l` +if [ $count -eq 0 ]; then + echo "No schedule traces found?" + exit -1 +fi + +echo "Graph filtering works with stack tracer" + +echo "Now testing if filtering still works without stack tracer" + +echo 0 > /proc/sys/kernel/stack_tracer_enabled +echo > trace +sleep 1 + + +count=`cat trace | grep '()' | grep -v schedule | wc -l` + +if [ $count -ne 0 ]; then + echo "Graph filtering not working after stack tracer disabled?" + exit -1 +fi + +count=`cat trace | grep 'schedule()' | wc -l` +if [ $count -eq 0 ]; then + echo "No schedule traces found?" + exit -1 +fi + +do_reset + +echo "SUCCESS!" +exit 0
tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc new file +78 -0
78 lines not on this page · raw message
Signature · 1 line
1.8.1.4
Quoted 3 lines · Steven Rostedt
[
Masami, I took two of my test scripts and added some basic comments
Thanks!
Quoted 3 lines
to them and copied them pretty much unchanged into a ftrace directory under test.d. Is this fine, or is there more massaging I need to do to them?
Yeah, ftrace has a double meaning :), so test.d/ftrace is fine to me.
Quoted 3 lines
I know the echos don't show up, but I kept them anyway. What should happen with them?
I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail. Then the user can reconfigure that. Maybe we should keep the detailed log of such results. (you can do it with --keep option)
Quoted 26 lines
]
Added two test cases to get the feel of adding tests to ftracetest. The two cases are:
function profiling test, to make sure function profiling still works
with function tracing (was a regression)function graph filter test to make sure that the function graph filter
does filter and also continues to filter when another function tracer is running (like the stack tracer)Signed-off-by Steven Rostedt <[email protected]>--- .../ftrace/test.d/ftrace/fgraph-filter.tc | 111 +++++++++++++++++++++ .../ftrace/test.d/ftrace/func_profiler.tc | 78 +++++++++++++++ 2 files changed, 189 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc create mode 100644 tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tctools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +0
-0
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc line 9
Quoted 9 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +9
-0@@ -0,0 +1,111 @@ +#!/bin/sh +# description: ftrace - function graph filters + +# Make sure that function graph filtering works, and is not +# affected by other tracers enabled (like stack tracer) + +if ! grep -q function_graph available_tracers; then + echo "no function graph tracer configured" + exit 0;
This should call exit_unsupported, because the test is not passed.
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc line 14
Quoted 5 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +5
-0+fi + +if [ ! -f set_ftrace_filter ]; then + echo "set_ftrace_filter not found? Is dynamic ftrace not set?" + exit 0
Ditto.
Quoted 32 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +32
-0+fi + +do_reset() { + echo nop > current_tracer + echo 0 > /proc/sys/kernel/stack_tracer_enabled + echo 1 > tracing_on + echo > trace + echo > set_ftrace_filter +} + +echo 0 > tracing_on +# clear trace +echo > trace +# filter something, schedule is always good +if ! echo "schedule" > set_ftrace_filter; then + # test for powerpc 64 + if ! echo ".schedule" > set_ftrace_filter; then + echo "can not enable schedule filter" + exit -1 + fi +fi + +echo function_graph > current_tracer +echo 1 > tracing_on +sleep 1 +# search for functions (has "()" on the line), and make sure +# that only the schedule function was found +count=`cat trace | grep '()' | grep -v schedule | wc -l` +if [ $count -ne 0 ]; then + echo "Graph filtering not working by itself?" + exit -1; +fi
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc line 56
Quoted 10 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +10
-0+ +# Make sure we did find something +count=`cat trace | grep 'schedule()' | wc -l` +if [ $count -eq 0 ]; then + echo "No schedule traces found?" + exit -1 +fi + +echo "Graph filtering works by itself" +
I think the following part should be a separated test for stack trace.
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc line 61
Quoted 5 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +5
-0+if [ ! -f stack_trace ]; then + echo "Stack tracer not configured, can't continue test" + # stack tracer not configured in this kernel? pass test anyway + do_reset + exit 0;
Here should call exit_unsupported.
Quoted 75 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +50
-0+fi + +echo "Now testing with stack tracer" + +echo 1 > /proc/sys/kernel/stack_tracer_enabled + +echo 0 > tracing_on +echo > trace +echo 1 > tracing_on +sleep 1 + +count=`cat trace | grep '()' | grep -v schedule | wc -l` + +if [ $count -ne 0 ]; then + echo "Graph filtering not working with stack tracer?" + exit -1 +fi + +count=`cat trace | grep 'schedule()' | wc -l` +if [ $count -eq 0 ]; then + echo "No schedule traces found?" + exit -1 +fi + +echo "Graph filtering works with stack tracer" + +echo "Now testing if filtering still works without stack tracer" + +echo 0 > /proc/sys/kernel/stack_tracer_enabled +echo > trace +sleep 1 + + +count=`cat trace | grep '()' | grep -v schedule | wc -l` + +if [ $count -ne 0 ]; then + echo "Graph filtering not working after stack tracer disabled?" + exit -1 +fi + +count=`cat trace | grep 'schedule()' | wc -l` +if [ $count -eq 0 ]; then + echo "No schedule traces found?" + exit -1 +fi + +do_reset + +echo "SUCCESS!" +exit 0tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc new file +19
-0@@ -0,0 +1,78 @@ +#!/bin/sh +# description: ftrace - function profiler with function tracing + +# There was a bug after a rewrite of the ftrace infrastructure that +# caused the function_profiler not to be able to run with the function +# tracer, because the function_profiler used the function_graph tracer +# and it was assumed the two could not run simultaneously. +# +# There was another related bug where the solution to the first bug +# broke the way filtering of the function tracer worked. +# +# This test triggers those bugs on those kernels. +# +# We need function_graph and profiling to to run this test +if ! grep -q function_graph available_tracers; then + echo "no function graph tracer configured" + # allow it to pass, as the kernel doesn't support it + exit 0; +fi
tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc line 29
Quoted 10 lines
tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc new file +10
-0+ +if [ ! -f set_ftrace_filter ]; then + echo "set_ftrace_filter not found? Is dynamic ftrace not set?" + exit 0 +fi + +if [ ! -f function_profile_enabled ]; then + echo "function_profile_enabled not found, function profiling enabled?" + exit 0 +fi
These checks should call exit_unsupported if failed.
tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc line 35
Quoted 6 lines
tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc new file +6
-0+ +echo "Testing function tracer with profiler:" +echo "enable function tracer" +echo function > current_tracer || exit 1 +echo "enable profiler" +echo 1 > function_profile_enabled || exit 1
Now you don't need "|| exit 1", since if the command fails, the shell exits with error code.
tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc line 41
Quoted 6 lines
tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc new file +6
-0+ +sleep 1 + +echo "Now filter on just schedule" +echo '*schedule' > set_ftrace_filter +> trace
echo > trace?
Thank you,
Quoted 38 lines
tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc new file +37
-0+ +echo "Now disable function profiler" +echo 0 > function_profile_enabled || exit 1 + +sleep 1 + +# make sure only schedule functions exist + +echo "testing if only schedule is being traced" +if grep -v -e '^#' -e 'schedule' trace; then + echo "more than schedule was found" + exit 1 +fi + +echo "Make sure schedule was traced" +if ! grep -e 'schedule' trace > /dev/null; then + cat trace + echo "can not find schedule in trace" + exit 1 +fi + +echo "clear filter" +echo > set_ftrace_filter || exit 1 +> trace + +sleep 1 + +echo "make sure something other than scheduler is being traced" +if ! grep -v -e '^#' -e 'schedule' trace > /dev/null; then + cat trace + echo "no other functions besides schedule was found" + exit 1 +fi + +echo nop > current_tracer || exit 1 + +exit 0
Signature · 4 lines
Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: [email protected]
Steven Rostedt
maintainerQuoted 7 lines · On Wed, 24 Sep 2014 11:58:50 +0900 Masami Hiramatsu…
Quoted 3 lines · Steven Rostedt
to them and copied them pretty much unchanged into a ftrace directory under test.d. Is this fine, or is there more massaging I need to do to them?
Yeah, ftrace has a double meaning :), so test.d/ftrace is fine to me.
I was thinking that more complex tests can go into ftrace, and simple tests can go into basic.
Quoted 7 lines
Quoted 4 lines
I know the echos don't show up, but I kept them anyway. What should happen with them?
I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail.
OK, that answers the exit codes (as you also stated below), but what about the echos themselves?
Quoted 2 lines
Then the user can reconfigure that. Maybe we should keep the detailed log of such results. (you can do it with --keep option)
You mean keep the echos, as they are ignored anyway, but if we add --keep, then the echos will show? Maybe that option should be -v, like other tools use for "verbose".
Quoted 6 lines
Quoted 6 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +3
-0@@ -0,0 +1,111 @@ +#!/bin/sh +# description: ftrace - function graph filters +
Quoted 8 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc line 9
Quoted 6 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +6
-0+# Make sure that function graph filtering works, and is not +# affected by other tracers enabled (like stack tracer) + +if ! grep -q function_graph available_tracers; then + echo "no function graph tracer configured" + exit 0;This should call
exit_unsupported, because the test is not passed.
Will update this and others.
Quoted 4 lines
Quoted 4 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +4
-0+# Make sure we did find something +count=`cat trace | grep 'schedule()' | wc -l` +if [ $count -eq 0 ]; then + echo "No schedule traces found?"
Quoted 8 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc line 56
Quoted 5 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +5
-0+ exit -1 +fi + +echo "Graph filtering works by itself" +I think the following part should be a separated test for stack trace.
Good point. A lot of my test scripts do multiple tests. I think for putting them into tools/testing/selftests/ftrace, I'll break them up and make them separate tests. For example, the simple graph filtering above is an example of something that can go into the basic directory, but the test against stack tracer should be in ftrace. What do you think?
Quoted 6 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc line 22
Quoted 4 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +4
-0+ +echo "Now filter on just schedule" +echo '*schedule' > set_ftrace_filter +> traceecho > trace?
OK. Hmm, I wonder if we should make a bunch of functions that the tests can use. Like a "clear_trace" call that does this. Can we export functions that this shell will be able to use? When we see lots of duplicate code we may want to have something like that. Well, for this, it may not be that big of a deal, because "echo > trace" is rather simple. But I do have other operations that are a bit more intrusive.
-- Steve
Steven Rostedt
maintainerQuoted 3 lines · On Wed, 24 Sep 2014 11:42:35 -0400 Steven Rostedt <[email protected]> wrote:
Quoted 2 lines
I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail.
Are these functions defined anywhere? I don't seem them in your v5 patch set, or in selftests anywhere.
-- Steve
Quoted 12 lines · Steven Rostedt
Quoted 3 lines · On Wed, 24 Sep 2014 11:42:35 -0400 Steven Rostedt <[email protected]> wrote:
Quoted 2 lines
I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail.
Are these functions defined anywhere? I don't seem them in your v5 patch set, or in selftests anywhere.
-- Steve
Steve,
I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread:
https://lkml.org/lkml/2014/9/23/926
thanks,
-- Shuah
Signature · 4 lines
Shuah Khan Sr. Linux Kernel Developer Samsung Research America (Silicon Valley) [email protected] | (970) 217-8978
Steven Rostedt
maintainerQuoted 6 lines · On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan <[email protected]> wrote:
I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread:
Thanks,
Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version.
-- Steve
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan <[email protected]> wrote:
I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread:
Quoted 8 lines
Thanks,
Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version.
One simple approach is to have the user install the required tools for the test. Indicate what is needed and fail the test with appropriate message with xskip. This is what I have in mind for tests that can't be run because of a missing config. The same can be extended to external tools that the test depends on.
thanks,
-- Shuah
Signature · 4 lines
Shuah Khan Sr. Linux Kernel Developer Samsung Research America (Silicon Valley) [email protected] | (970) 217-8978
Steven Rostedt
maintainerQuoted 5 lines · On Wed, 24 Sep 2014 10:22:12 -0600 Shuah Khan <[email protected]> wrote:
One simple approach is to have the user install the required tools for the test. Indicate what is needed and fail the test with appropriate message with xskip. This is what I have in mind for tests that can't be run because of a missing config. The same can be extended to external tools that the test depends on.
Should I hold off adding my tests until we have all this in place. Or should I just push my tests with simply passing if the config isn't supported for now and add it later?
I wont add any of the tests that require other tools besides echo, grep and cat.
-- Steve
Quoted 10 lines · Steven Rostedt
Quoted 5 lines · On Wed, 24 Sep 2014 10:22:12 -0600 Shuah Khan <[email protected]> wrote:
One simple approach is to have the user install the required tools for the test. Indicate what is needed and fail the test with appropriate message with xskip. This is what I have in mind for tests that can't be run because of a missing config. The same can be extended to external tools that the test depends on.
Quoted 7 lines
Should I hold off adding my tests until we have all this in place. Or should I just push my tests with simply passing if the config isn't supported for now and add it later?
I wont add any of the tests that require other tools besides echo, grep and cat.
Please go ahead and get them in. The framework might have to wait until 3.19, so there is no need to have it gate test development. We can evolve them as we go. If tests require special, it means that they aren't suitable for certain envs such as qemu boot with busybox. There is a plan to add test targets to cover such cases.
I would recommend at least printing a message saying what you are missing if you find something missing.
thanks,
-- Shuah
Signature · 4 lines
Shuah Khan Sr. Linux Kernel Developer Samsung Research America (Silicon Valley) [email protected] | (970) 217-8978
Quoted 11 lines · Shuah Khan
Quoted 10 lines · Steven Rostedt
Quoted 5 lines · On Wed, 24 Sep 2014 10:22:12 -0600 Shuah Khan <[email protected]> wrote:
One simple approach is to have the user install the required tools for the test. Indicate what is needed and fail the test with appropriate message with xskip. This is what I have in mind for tests that can't be run because of a missing config. The same can be extended to external tools that the test depends on.
Quoted 6 lines
Quoted 6 lines
Should I hold off adding my tests until we have all this in place. Or should I just push my tests with simply passing if the config isn't supported for now and add it later?
I wont add any of the tests that require other tools besides echo, grep and cat.
They are supported by busybox :)
Quoted 5 lines
Please go ahead and get them in. The framework might have to wait until 3.19, so there is no need to have it gate test development. We can evolve them as we go. If tests require special, it means that they aren't suitable for certain envs such as qemu boot with busybox. There is a plan to add test targets to cover such cases.
Agreed, anyway, in some env those tools(commands) may not supported by busybox because of the memory size etc. In that case, we can just mark the test unsupported or xfail.
Thank you,
Quoted 7 lines
I would recommend at least printing a message saying what you are missing if you find something missing.
thanks,
-- Shuah
Signature · 4 lines
Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: [email protected]
Steven Rostedt
maintainerQuoted 10 lines · On Fri, 26 Sep 2014 21:04:26 +0900 Masami Hiramatsu…
Quoted 5 lines
Please go ahead and get them in. The framework might have to wait until 3.19, so there is no need to have it gate test development. We can evolve them as we go. If tests require special, it means that they aren't suitable for certain envs such as qemu boot with busybox. There is a plan to add test targets to cover such cases.
Agreed, anyway, in some env those tools(commands) may not supported by busybox because of the memory size etc. In that case, we can just mark the test unsupported or xfail.
Note, if memory size is such a big issue it that other tools may not be there, then most of the ftrace features will probably be disabled due to memory constraints on the kernel itself.
As we have a way to state "unsupported" we can just return that from tests that require tools not available, and say why it is the case.
-- Steve
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Fri, 26 Sep 2014 21:04:26 +0900 Masami Hiramatsu…
Quoted 5 lines
Please go ahead and get them in. The framework might have to wait until 3.19, so there is no need to have it gate test development. We can evolve them as we go. If tests require special, it means that they aren't suitable for certain envs such as qemu boot with busybox. There is a plan to add test targets to cover such cases.
Quoted 8 lines
Quoted 4 lines
Agreed, anyway, in some env those tools(commands) may not supported by busybox because of the memory size etc. In that case, we can just mark the test unsupported or xfail.
Note, if memory size is such a big issue it that other tools may not be there, then most of the ftrace features will probably be disabled due to memory constraints on the kernel itself.
Agreed :)
Quoted 3 lines
As we have a way to state "unsupported" we can just return that from tests that require tools not available, and say why it is the case.
OK, if we can count "unsupported" as a test failure (yes, it is unexpectedly failed), it is easy to fix. I'll update XFAIL and other error code support patch and send it.
Thank you,
Signature · 4 lines
Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: [email protected]
Hi Steve,
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan <[email protected]> wrote:
I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread:
Quoted 7 lines
Thanks,
Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version.
Yeah, and I think we can at least build perf (if not found) for test as we can always have the source. The trace-cmd is different and it'd be another reason I need to work on 'perf ftrace' command soon. ;-)
Thanks,
Namhyung
Quoted 21 lines · Namhyung Kim
Hi Steve,
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan <[email protected]> wrote:
I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread:
Quoted 7 lines
Thanks,
Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version.
Quoted 3 lines
Yeah, and I think we can at least build perf (if not found) for test as we can always have the source. The trace-cmd is different and it'd be another reason I need to work on 'perf ftrace' command soon. ;-)
+1 :)
For the kselftest, we can only expect to have tools in kernel tree (which should be static-linked) and busybox, so that we can use qemu emulator to run kselftest on other archs. I guess perf still need some shared objects to run. That should be solved too.
Anyway, at first we'd better add simpler tests which is not relay on other tools except busybox shell.
Thank you,
Signature · 4 lines
Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: [email protected]
Hi Masami,
2014-09-26 (금), 20:54 +0900, Masami Hiramatsu:
Quoted 26 lines
Quoted 21 lines · Namhyung Kim
Hi Steve,
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan <[email protected]> wrote:
I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread:
Quoted 7 lines
Thanks,
Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version.
Quoted 3 lines
Yeah, and I think we can at least build perf (if not found) for test as we can always have the source. The trace-cmd is different and it'd be another reason I need to work on 'perf ftrace' command soon. ;-)
Quoted 6 lines
+1 :)
For the kselftest, we can only expect to have tools in kernel tree (which should be static-linked) and busybox, so that we can use qemu emulator to run kselftest on other archs. I guess perf still need some shared objects to run. That should be solved too.
The perf should be built with no extra dependencies (other than libc). It won't provide rich features but basic functionality (record, report, stat) should be supported. And I'm also thinking about implementing a very simple elf parser for symbol loading in such a minimal build.
Thanks,
Namhyung
Quoted 7 lines
Anyway, at first we'd better add simpler tests which is not relay on other tools except busybox shell.
Thank you,
Quoted 36 lines · Namhyung Kim
Hi Masami,
2014-09-26 (금), 20:54 +0900, Masami Hiramatsu:
Quoted 26 lines
Quoted 21 lines · Namhyung Kim
Hi Steve,
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan <[email protected]> wrote:
I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread:
Quoted 7 lines
Thanks,
Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version.
Quoted 3 lines
Yeah, and I think we can at least build perf (if not found) for test as we can always have the source. The trace-cmd is different and it'd be another reason I need to work on 'perf ftrace' command soon. ;-)
Quoted 6 lines
+1 :)
For the kselftest, we can only expect to have tools in kernel tree (which should be static-linked) and busybox, so that we can use qemu emulator to run kselftest on other archs. I guess perf still need some shared objects to run. That should be solved too.
Quoted 4 lines
The perf should be built with no extra dependencies (other than libc). It won't provide rich features but basic functionality (record, report, stat) should be supported. And I'm also thinking about implementing a very simple elf parser for symbol loading in such a minimal build.
Maybe you can install some static libraries (e.g. elfutils-libelf-devel-static on fedora) for enabling it. :)
Some libraries may not work, but I could enable libelf by installing that.
Thank you,
Signature · 4 lines
Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: [email protected]
Hi Masami,
Quoted 9 lines · Masami Hiramatsu
Quoted 4 lines · Namhyung Kim
The perf should be built with no extra dependencies (other than libc). It won't provide rich features but basic functionality (record, report, stat) should be supported. And I'm also thinking about implementing a very simple elf parser for symbol loading in such a minimal build.
Maybe you can install some static libraries (e.g. elfutils-libelf-devel-static on fedora) for enabling it. :)
Some libraries may not work, but I could enable libelf by installing that.
Yep, installing static libs should also work. But I'd like to have minimal depencency especially for embedded systems. And AFAIK cross building elfutils/libelf always has been a PITA.
Thanks,
Namhyung
Steven Rostedt
maintainerQuoted 3 lines · On Fri, 26 Sep 2014 15:06:51 +0900 Namhyung Kim <[email protected]> wrote:
Yeah, and I think we can at least build perf (if not found) for test as we can always have the source. The trace-cmd is different and it'd be another reason I need to work on 'perf ftrace' command soon. ;-)
Well, if a tool is not found. I wouldn't recommend building it. Either case, I'll hold off test that is required.
Hmm, maybe I'll post all my tests on my website, for others to play with. That way I'm not limited in tests that take too much time or use tools not supported by the kernel selftest infrastructure.
-- Steve
Quoted 9 lines · Steven Rostedt
Quoted 3 lines · On Wed, 24 Sep 2014 11:42:35 -0400 Steven Rostedt <[email protected]> wrote:
Quoted 2 lines
I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail.
Are these functions defined anywhere? I don't seem them in your v5 patch set, or in selftests anywhere.
Hmm, could you check my 4/4 patch in the series??
http://www.mail-archive.com/[email protected]/msg730578.html
Thanks,
Signature · 4 lines
Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: [email protected]
Steven Rostedt
maintainerQuoted 8 lines · On Thu, 25 Sep 2014 09:35:05 +0900 Masami Hiramatsu…
Quoted 7 lines · Steven Rostedt
Quoted 3 lines · On Wed, 24 Sep 2014 11:42:35 -0400 Steven Rostedt <[email protected]> wrote:
Quoted 2 lines
I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail.
Quoted 5 lines
Quoted 2 lines
Are these functions defined anywhere? I don't seem them in your v5 patch set, or in selftests anywhere.
Hmm, could you check my 4/4 patch in the series??
http://www.mail-archive.com/[email protected]/msg730578.html
Hmm, I somehow missed this. Can you check to make sure my for-next patches are your latest version. I'll pull in 4/4 tomorrow.
Thanks,
-- Steve
Quoted 11 lines · Steven Rostedt
Quoted 8 lines · On Thu, 25 Sep 2014 09:35:05 +0900 Masami Hiramatsu…
Quoted 7 lines · Steven Rostedt
Quoted 3 lines · On Wed, 24 Sep 2014 11:42:35 -0400 Steven Rostedt <[email protected]> wrote:
Quoted 2 lines
I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Are these functions defined anywhere? I don't seem them in your v5 patch set, or in selftests anywhere.
Hmm, could you check my 4/4 patch in the series??
http://www.mail-archive.com/[email protected]/msg730578.htmlHmm, I somehow missed this. Can you check to make sure my for-next patches are your latest version. I'll pull in 4/4 tomorrow.
Yeah, I ensured that your for-next patches are the v5. You will be able to cleanly apply 4/4 on it :)
Thank you,
Quoted 5 lines
Thanks,
-- Steve
Signature · 4 lines
Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: [email protected]
Quoted 5 lines · Steven Rostedt
Quoted 1 line · On Wed, 24 Sep 2014 11:58:50 +0900 Masami Hiramatsu…
(2014/09/24 6:38), Steven Rostedt wrote:
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
to them and copied them pretty much unchanged into a ftrace directory under test.d. Is this fine, or is there more massaging I need to do to them?
Yeah, ftrace has a double meaning :), so test.d/ftrace is fine to me.
I was thinking that more complex tests can go into ftrace, and simple tests can go into basic.
Hmm, what kind of "complex" test will be there?
I think unit tests are usually simple, selftests/ftrace/test.d/ftrace/ looks redundant if the first ftrace and the second one has same meaning.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
I know the echos don't show up, but I kept them anyway. What should happen with them?
I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail.
OK, that answers the exit codes (as you also stated below), but what about the echos themselves?
Again, ftracetest is for unit tests, exit code will return one concrete result, but we can anything with echo (including typo)
Quoted 6 lines
Quoted 3 lines
Then the user can reconfigure that. Maybe we should keep the detailed log of such results. (you can do it with --keep option)
You mean keep the echos, as they are ignored anyway, but if we add --keep, then the echos will show?
No, --keep keeps all output logs of the test script. Since all scripts run under set -x, all commands and outputs are logged to a logfile.
Quoted 2 lines
Maybe that option should be -v, like
other tools use for "verbose".
OK, it will be easy, just turning "> $LOGFILE" into "| tee $LOGFILE" :)
Quoted 31 lines
Quoted 6 lines
Quoted 6 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +3
-0@@ -0,0 +1,111 @@ +#!/bin/sh +# description: ftrace - function graph filters +Quoted 8 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc line 9
Quoted 6 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +6
-0+# Make sure that function graph filtering works, and is not +# affected by other tracers enabled (like stack tracer) + +if ! grep -q function_graph available_tracers; then + echo "no function graph tracer configured" + exit 0;This should call
exit_unsupported, because the test is not passed.Will update this and others.
Quoted 4 lines
Quoted 4 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +4
-0+# Make sure we did find something +count=`cat trace | grep 'schedule()' | wc -l` +if [ $count -eq 0 ]; then + echo "No schedule traces found?"Quoted 8 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc line 56
Quoted 5 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +5
-0+ exit -1 +fi + +echo "Graph filtering works by itself" +I think the following part should be a separated test for stack trace.
Quoted 6 lines
Good point. A lot of my test scripts do multiple tests. I think for putting them into
tools/testing/selftests/ftrace, I'll break them up and make them separate tests. For example, the simple graph filtering above is an example of something that can go into the basic directory, but the test against stack tracer should be in ftrace. What do you think?
Hm, I think only really basic operation tests go into the basic directory, like reading and writing (no format checks). And test.d/ftrace may be for more advanced tests, like checking output, setting options, etc.
Quoted 10 lines
Quoted 6 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc line 22
Quoted 4 lines
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc new file +4
-0+ +echo "Now filter on just schedule" +echo '*schedule' > set_ftrace_filter +> traceecho > trace?
OK. Hmm, I wonder if we should make a bunch of functions that the tests can use. Like a "
clear_trace" call that does this. Can we export functions that this shell will be able to use?
Yeah, it's a good idea :) We can use "." to include it. I think test.d/functions will be good.
Quoted 4 lines
When we see lots of
duplicate code we may want to have something like that. Well, for this, it may not be that big of a deal, because "echo > trace" is rather simple. But I do have other operations that are a bit more intrusive.
Agreed.
Thank you!
Signature · 4 lines
Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: [email protected]