gcov kernel support
Subsystems (2)
cover gcov kernel support
This is version #3 of the gcov kernel support patch set (see further below for an in-depth explanation). Patch base is 2.6.26-rc4. Please replace the previous gcov patch set with this one.
Changes based on mailing list feedback:
* removed patch "kbuild: make source and include paths absolute". Replaced by new approach (patch 5) which creates symbolic links in the build directory instead.
* updated docs to match new approach
* removed patch "kbuild: delay object file renaming during module versioning". No longer required.
* gcov core code no longer creates symbolic links in debugfs
* updated patch "gcov: architecture specific compile flag adjustments". Contains changes for powerpc.
Patch overview:
[PATCH 1/6] kernel: call constructors
[PATCH 2/6] kernel: introduce GCC_VERSION_LOWER macro [PATCH 3/6] seq_file: add function to write binary data [PATCH 4/6] gcov: add gcov profiling infrastructure [PATCH 5/6] gcov: create links to .gcda files in build directory [PATCH 6/6] gcov: architecture specific compile flag adjustments
===
gcov profiling kernel support enables the use of GCC's coverage testing tool gcov [1] with the Linux kernel. Coverage data of a running kernel is exported in gcov-compatible format via the "gcov "debugfs directory.
Example:
To get coverage data for file base.c in directory drivers/base, simply change to the kernel build directory and run gcov with the -o option (assumptions: kernel was built in /tmp/linux and debugfs is mounted at /sys/kernel/debug):
# cd /tmp/linux # gcov -o drivers/base/ bus.c
This will create source code files annotated with execution counts in the current directory. In addition, graphical gcov front-ends such as lcov [2] can be used to automate the process of collecting data for the entire kernel and provide coverage overviews in HTML format.
Possible uses:
* debugging (has this line been executed at all?)
* test improvement (how do I change my test to cover these lines?)
* minimizing kernel configurations (do I need this option if the associated code is never executed?)
Known issues:
* some architecture specific problems: the patch has been tested successfully on s390 and i386. Known problems exist on x86_64 and arm (to be investigated)
* GCC's profiling mechanism together with optimization sometimes produces skewed data (see [1])
* GCC's profiling code assumes single-threaded execution
* gcov assumes that a program has finished when coverage data is analyzed
Despite these issues, the data which can be obtained has been proven to be sufficiently accurate for most practical uses.
History:
Hubertus Franke <[email protected]> wrote the first version of this patch around 2002. Since then it has been adapted to new versions of the kernel and GCC with contributions by several people (see file kernel/gcov/fs.c, write me if I missed anyone). Due to regular requests, I rewrote the gcov-kernel patch from scratch so that it would (hopefully) be fit for inclusion into the upstream kernel.
--
[1] http://gcc.gnu.org/onlinedocs/gcc/Gcov.html
[2] http://ltp.sourceforge.net/coverage/lcov.php
Quoted 1 line · On Mon, 02 Jun 2008 15:33:51 +0200 Peter Oberparleiter…
This is version #3 of the gcov kernel support patch set
My build tree is now filled with dead symlinks, like
lrwxrwxrwx 1 akpm akpm 64 Jun 9 00:06 security/selinux/nlmsgtab.gcda -> /sys/kernel/debug/gcov/usr/src/25/security/selinux/nlmsgtab.gcda
Which causes (at least)
ctags: Warning: cannot open source file "security/selinux/ss/conditional.gcda" : No such file or directory
ctags: Warning: cannot open source file "security/selinux/netlink.gcda" : No such file or directory
ctags: Warning: cannot open source file "security/selinux/netlabel.gcda" : No such file or directory
and probably other thing which I haven't discovered yet.
Quoted 7 lines · Andrew Morton
Quoted 1 line · On Mon, 02 Jun 2008 15:33:51 +0200 Peter Oberparleiter…
This is version #3 of the gcov kernel support patch set
My build tree is now filled with dead symlinks, like
lrwxrwxrwx 1 akpm akpm 64 Jun 9 00:06
security/selinux/nlmsgtab.gcda->/sys/kernel/debug/gcov/usr/src/25/security/selinux/nlmsgtab.gcda
Unfortunately a necessary evil of this approach: symlinks are created for all compiled source files while link targets are only available when the corresponding code is executed. In other words: those links will be dead for source files which don't compile to actual code and for modules as long as they are not loaded.
Quoted 7 lines
Which causes (at least)
ctags: Warning: cannot open source file "
security/selinux/ss/conditional.gcda" : No such file or directory
ctags: Warning: cannot open source file "security/selinux/netlink.gcda" : No such file or directory
ctags: Warning: cannot open source file "security/selinux/netlabel.gcda" : No such file or directoryand probably other thing which I haven't discovered yet.
I'm sure there's some kind of 'find' magic that can be used to work around dead links. I'll see how this can be applied to the ctags case - though neither 'make tags' nor 'make TAGS' seem to produce any warnings on my system..
Regards,
Peter
Quoted 7 lines · On Mon, 09 Jun 2008 15:49:16 +0200 Peter Oberparleiter…
Quoted 6 lines · Andrew Morton
Quoted 1 line · On Mon, 02 Jun 2008 15:33:51 +0200 Peter Oberparleiter…
This is version #3 of the gcov kernel support patch set
My build tree is now filled with dead symlinks, like
Quoted 7 lines
Quoted 1 line
lrwxrwxrwx 1 akpm akpm 64 Jun 9 00:06
security/selinux/nlmsgtab.gcda->/sys/kernel/debug/gcov/usr/src/25/security/selinux/nlmsgtab.gcdaUnfortunately a necessary evil of this approach: symlinks are created for all compiled source files while link targets are only available when the corresponding code is executed. In other words: those links will be dead for source files which don't compile to actual code and for modules as long as they are not loaded.
It doesn't seem awfully useful. I don't run kernels on my build machines and I'm sure many are in the same situation. So gcov is going to need a way of locating these files on the *target* machine. And once that is available, there is no need to add all these symlinks into the build directory.
Quoted 6 lines
Quoted 5 lines
Which causes (at least)
ctags: Warning: cannot open source file "
security/selinux/ss/conditional.gcda" : No such file or directory
ctags: Warning: cannot open source file "security/selinux/netlink.gcda" : No such file or directory
ctags: Warning: cannot open source file "security/selinux/netlabel.gcda" : No such file or directory
Quoted 6 lines
Quoted 1 line
and probably other thing which I haven't discovered yet.
I'm sure there's some kind of 'find' magic that can be used to work around dead links. I'll see how this can be applied to the ctags case - though neither 'make tags' nor 'make TAGS' seem to produce any warnings on my system..
I use
ctags -R --excmd=pattern --format=1
Quoted 17 lines · Andrew Morton
Quoted 7 lines · On Mon, 09 Jun 2008 15:49:16 +0200 Peter Oberparleiter…
Quoted 6 lines · Andrew Morton
Quoted 1 line · On Mon, 02 Jun 2008 15:33:51 +0200 Peter Oberparleiter…
This is version #3 of the gcov kernel support patch set
My build tree is now filled with dead symlinks, like
Quoted 7 lines
Quoted 1 line
lrwxrwxrwx 1 akpm akpm 64 Jun 9 00:06
security/selinux/nlmsgtab.gcda->/sys/kernel/debug/gcov/usr/src/25/security/selinux/nlmsgtab.gcdaUnfortunately a necessary evil of this approach: symlinks are created for all compiled source files while link targets are only available when the corresponding code is executed. In other words: those links will be dead for source files which don't compile to actual code and for modules as long as they are not loaded.
Quoted 5 lines
It doesn't seem awfully useful. I don't run kernels on my build machines and I'm sure many are in the same situation. So gcov is going to need a way of locating these files on the *target* machine. And once that is available, there is no need to add all these symlinks into the build directory.
I don't see any other feasibly way to do it if we want the kernel to work out-of-the-box with gcov. If the kernel was a user-space application, gcc/libgcov would create the .gcda files in exactly the same place where the symbolic links are now.
If we removed those symlinks, users would have to manually copy files from /sys on the test machine to the correct position in /objtree on the build machine before being able to get any kind of result. This would IMO reduce the usefulness of the gcov kernel infrastructure noticeably (though gcov-wrappers such as lcov could be modified to hide the additional effort).
How about a CONFIG_GCOV_PROFILE_SYMLINKS configuration option?
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Which causes (at least)
ctags: Warning: cannot open source file "
security/selinux/ss/conditional.gcda" : No such file or directory
ctags: Warning: cannot open source file "security/selinux/netlink.gcda" : No such file or directory
ctags: Warning: cannot open source file "security/selinux/netlabel.gcda" : No such file or directoryand probably other thing which I haven't discovered yet.
I would argue that any mechanism that tries to access all files in /objtree regardless of filename extension is brave at best, if not broken.
Quoted 8 lines
Quoted 4 lines
I'm sure there's some kind of 'find' magic that can be used to work around dead links. I'll see how this can be applied to the ctags case - though neither 'make tags' nor 'make TAGS' seem to produce any warnings on my system..
I use
ctags -R --excmd=pattern --format=1
ctags -R --excmd=pattern --format=1 --exclude=\*.gcda
:)
Regards,
Peter
Quoted 29 lines · On Wed, 11 Jun 2008 14:59:54 +0200 Peter Oberparleiter…
Quoted 17 lines · Andrew Morton
Quoted 7 lines · On Mon, 09 Jun 2008 15:49:16 +0200 Peter Oberparleiter…
Quoted 6 lines · Andrew Morton
Quoted 1 line · On Mon, 02 Jun 2008 15:33:51 +0200 Peter Oberparleiter…
This is version #3 of the gcov kernel support patch set
My build tree is now filled with dead symlinks, like
Quoted 7 lines
Quoted 1 line
lrwxrwxrwx 1 akpm akpm 64 Jun 9 00:06
security/selinux/nlmsgtab.gcda->/sys/kernel/debug/gcov/usr/src/25/security/selinux/nlmsgtab.gcdaUnfortunately a necessary evil of this approach: symlinks are created for all compiled source files while link targets are only available when the corresponding code is executed. In other words: those links will be dead for source files which don't compile to actual code and for modules as long as they are not loaded.
Quoted 5 lines
It doesn't seem awfully useful. I don't run kernels on my build machines and I'm sure many are in the same situation. So gcov is going to need a way of locating these files on the *target* machine. And once that is available, there is no need to add all these symlinks into the build directory.
I don't see any other feasibly way to do it if we want the kernel to work out-of-the-box with gcov. If the kernel was a user-space application, gcc/libgcov would create the .gcda files in exactly the same place where the symbolic links are now.
Quoted 6 lines
If we removed those symlinks, users would have to manually copy files from /sys on the test machine to the correct position in /objtree on the build machine before being able to get any kind of result. This would IMO reduce the usefulness of the gcov kernel infrastructure noticeably (though gcov-wrappers such as lcov could be modified to hide the additional effort).
gcov needs both the .gcda files and the source tree available to do its work, I assume.
So a sensible scenario would be to copy the entire build tree, including the .gcda symlinks over to the target system, yes? tar+scp+untar?
If so, it'd be good to get that tested and documented...
Quoted 1 line
How about a
CONFIG_GCOV_PROFILE_SYMLINKSconfiguration option?
What use would that be?
Quoted 11 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Which causes (at least)
ctags: Warning: cannot open source file "
security/selinux/ss/conditional.gcda" : No such file or directory
ctags: Warning: cannot open source file "security/selinux/netlink.gcda" : No such file or directory
ctags: Warning: cannot open source file "security/selinux/netlabel.gcda" : No such file or directoryand probably other thing which I haven't discovered yet.
I would argue that any mechanism that tries to access all files in /objtree regardless of filename extension is brave at best, if not broken.
Well, these things happen. You'll probably need to add these to .gitignore.
<does Samsummoning dance>
Quoted 39 lines · Andrew Morton
Quoted 29 lines · On Wed, 11 Jun 2008 14:59:54 +0200 Peter Oberparleiter…
Quoted 17 lines · Andrew Morton
Quoted 7 lines · On Mon, 09 Jun 2008 15:49:16 +0200 Peter Oberparleiter…
Quoted 6 lines · Andrew Morton
Quoted 1 line · On Mon, 02 Jun 2008 15:33:51 +0200 Peter Oberparleiter…
This is version #3 of the gcov kernel support patch set
My build tree is now filled with dead symlinks, like
Quoted 7 lines
Quoted 1 line
lrwxrwxrwx 1 akpm akpm 64 Jun 9 00:06
security/selinux/nlmsgtab.gcda->/sys/kernel/debug/gcov/usr/src/25/security/selinux/nlmsgtab.gcdaUnfortunately a necessary evil of this approach: symlinks are created for all compiled source files while link targets are only available when the corresponding code is executed. In other words: those links will be dead for source files which don't compile to actual code and for modules as long as they are not loaded.
Quoted 5 lines
It doesn't seem awfully useful. I don't run kernels on my build machines and I'm sure many are in the same situation. So gcov is going to need a way of locating these files on the *target* machine. And once that is available, there is no need to add all these symlinks into the build directory.
I don't see any other feasibly way to do it if we want the kernel to work out-of-the-box with gcov. If the kernel was a user-space application, gcc/libgcov would create the .gcda files in exactly the same place where the symbolic links are now.
Quoted 6 lines
If we removed those symlinks, users would have to manually copy files from /sys on the test machine to the correct position in /objtree on the build machine before being able to get any kind of result. This would IMO reduce the usefulness of the gcov kernel infrastructure noticeably (though gcov-wrappers such as lcov could be modified to hide the additional effort).
Quoted 8 lines
gcov needs both the .gcda files and the source tree available to do its work, I assume.
So a sensible scenario would be to copy the entire build tree, including the .gcda symlinks over to the target system, yes? tar+scp+untar?
If so, it'd be good to get that tested and documented...
I followed your advice and did some more testing and found that
a) dead links are *really* (really) bothersome
b) CONFIG_MODVERSIONS=y breaks the current approach
The latter is not directly related to this link business but due to the kbuild simplification patch being nacked by Sam Ravnborg (unlike my initial assumption, the patch was still required by gcov).
To get to the point: I reworked the gcov approach again (I'm starting to feel bad about the amount of changes, but - hey..). The result:
* no symbolic links needed in the build directory
* gcov plays nicely with CONFIG_MODVERSIONS=y
* a recommended approach for "gcov + separated test/build" is documented in Documentation/gcov.txt
All this comes as a patch against the code introduced by gcov-add-gcov-profiling-infrastructure.patch (-mm3). If needed, I can also merge the changes into the old patch and post that one (whatever suits you best).
For -mm, please:
* drop gcov-create-links-to-gcda-files-in-build-directory.patch
* add this patch anywhere after gcov-add-gcov-profiling-infrastructure.patch
--
[PATCH] gcov: revert link changes
From: Peter Oberparleiter <[email protected]>
Change the gcov mechanism back to using symbolic links in sysfs. This is necessary to fix problems when profiling with CONFIG_MODVERSIONS=y. The correct usage of the gcov tool now looks like this:
# cd $OBJTREE # gcov -o /sys/kernel/debug/gcov/$OBJTREE/init main.c
Also update documentation to include a description on how to use gcov profiling in a separated build/test machine environment.
--- Documentation/gcov.txt | 138 ++++++++++++++++++++++++++++++++++++----- kernel/gcov/Makefile | 2 kernel/gcov/fs.c | 123 +++++++++++++++++++++++++++++++++++- kernel/gcov/gcc_3_4.c | 6 + kernel/gcov/gcov.h | 9 ++ 5 files changed, 262 insertions(+), 16 deletions(-)
Documentation/gcov.txt +124 -15
@@ -6,7 +6,10 @@ Using gcov with the Linux kernel 3. Customization 4. Files 5. Modules -6. Troubleshooting +6. Separated build and test machines +7. Troubleshooting +Appendix A: sample script: gather_on_build.sh +Appendix B: sample script: gather_on_test.sh 1. Introduction
@@ -15,16 +18,11 @@ Using gcov with the Linux kernel gcov profiling kernel support enables the use of GCC's coverage testing tool gcov [1] with the Linux kernel. Coverage data of a running kernel is exported in gcov-compatible format via the "gcov "debugfs directory. +To get coverage data for a specific file, change to the kernel build +directory and use gcov with the -o option as follows (requires root): -Example: - -To get coverage data for file base.c in directory drivers/base, simply -change to the kernel build directory and run gcov with the -o option -(assumptions: kernel was built in /tmp/linux and debugfs is mounted at -/sys/kernel/debug): - -# cd /tmp/linux -# gcov -o drivers/base/ bus.c +# cd /tmp/linux-out +# gcov -o /sys/kernel/debug/gcov/tmp/linux/kernel spinlock.c This will create source code files annotated with execution counts in the current directory. In addition, graphical gcov front-ends such
@@ -38,6 +36,11 @@ Possible uses: * minimizing kernel configurations (do I need this option if the associated code is never executed?) +-- + +[1] http://gcc.gnu.org/onlinedocs/gcc/Gcov.html +[2] http://ltp.sourceforge.net/coverage/lcov.php + 2. Preparation ==============
@@ -99,6 +102,11 @@ The gcov kernel support creates the foll The actual gcov data file as understood by the gcov tool. Resets file coverage data to zero when written to. + /sys/kernel/debug/gcov/path/to/compile/dir/file.gcno + Symbolic link to a static data file required by the gcov + tool. This file is generated by gcc when compiling with + option -ftest-coverage. + 5. Modules ==========
@@ -119,7 +127,56 @@ At run-time, a user can also choose to d module by writing to its data file or the global reset file. -6. Troubleshooting +6. Separated build and test machines +==================================== + +The gcov kernel profiling infrastructure is designed to work out-of-the +box for setups where kernels are built and run on the same machine. In +cases where the kernel runs on a separate machine, special preparations +must be made, depending on where the gcov tool is used: + +a) gcov is run on the TEST machine + +The gcov tool version on the test machine must be compatible with the +gcc version used for kernel build. Also the following files need to be +copied from build to test machine: + +from the source tree: + - all C source files + headers + +from the build tree: + - all C source files + headers + - all .gcda and .gcno files + - all links to directories + +It is important to note that these files need to be placed into the +exact same file system location on the test machine as on the build +machine. If any of the path components is symbolic link, the actual +directory needs to be used instead (due to make's CURDIR handling). + +b) gcov is run on the BUILD machine + +The following files need to be copied after each test case from test +to build machine: + +from the gcov directory in sysfs: + - all .gcda files + - all links to .gcno files + +These files can be copied to any location on the build machine. gcov +must then be called with the -o option pointing to that directory. + +Example directory setup on the build machine: + + /tmp/linux: kernel source tree + /tmp/out: kernel build directory as specified by make O= + /tmp/coverage: location of the files copied from the test machine + + [user@build] cd /tmp/out + [user@build] gcov -o /tmp/coverage/tmp/out/init main.c + + +7. Troubleshooting ================== Problem: Compilation aborts during linker step.
@@ -127,10 +184,62 @@ Cause: Profiling flags are specified linked to the main kernel or which are linked by a custom linker procedure. Solution: Exclude affected source files from profiling by specifying - GCOV := n in the corresponding Makefile. + GCOV := n or GCOV_basename.o := n in the corresponding + Makefile. --- +Appendix A: gather_on_build.sh +============================== -[1] http://gcc.gnu.org/onlinedocs/gcc/Gcov.html -[2] http://ltp.sourceforge.net/coverage/lcov.php +Sample script to gather coverage meta files on the build machine +(see 6a): + +#!/bin/bash + +KSRC=$1 +KOBJ=$2 +DEST=$3 + +if [ -z "$KSRC" ] || [ -z "$KOBJ" ] || [ -z "$DEST" ]; then + echo "Usage: $0 <ksrc directory> <kobj directory> <output.tar.gz>" >&2 + exit 1 +fi + +KSRC=$(cd $KSRC; printf "all:\n\t@echo \${CURDIR}\n" | make -f -) +KOBJ=$(cd $KOBJ; printf "all:\n\t@echo \${CURDIR}\n" | make -f -) + +find $KSRC $KOBJ \( -name '*.gcno' -o -name '*.[ch]' -o -type l \) -a \ + -perm /u+r,g+r | tar cfz $DEST -P -T - + +if [ $? -eq 0 ] ; then + echo "$DEST successfully created, copy to test system and unpack with:" + echo " tar xfz $DEST -P" +else + echo "Could not create file $DEST" +fi + + +Appendix B: gather_on_test.sh +============================= + +Sample script to gather coverage data files on the test machine +(see 6b): + +#!/bin/bash + +DEST=$1 +GCDA=/sys/kernel/debug/gcov + +if [ -z "$DEST" ] ; then + echo "Usage: $0 <output.tar.gz>" >&2 + exit 1 +fi + +find $GCDA -name '*.gcno' -o -name '*.gcda' | tar cfz $DEST -T - + +if [ $? -eq 0 ] ; then + echo "$DEST successfully created, copy to build system and unpack with:" + echo " tar xfz $DEST" +else + echo "Could not create file $DEST" +fi
kernel/gcov/fs.c +121 -2
172 lines not on this page · raw message
kernel/gcov/gcc_3_4.c +6 -0
@@ -20,6 +20,12 @@#include <linux/seq_file.h> #include "gcov.h" +/* Symbolic links to be created for each profiling data file. */ +const struct gcov_link gcov_link[] = { + { obj_tree,"gcno" },/* Link to .gcno file in $(objtree). */ + {0 ,NULL }, +}; +/* Determine whether a counter is active. Based on gcc magic. Doesn't change * at run-time. */ static int counter_active (struct gcov_info *info,unsigned int type)
kernel/gcov/gcov.h +9 -0
@@ -116,4 +116,13 @@ void gcov_info_add(struct gcov_info *desstruct gcov_info *gcov_info_dup (struct gcov_info *info);void gcov_info_free (struct gcov_info *info); +struct gcov_link { +enum { + obj_tree, + src_tree, + } dir; +const char *ext; +}; +extern const struct gcov_link gcov_link[]; +#endif /* GCOV_H */
kernel/gcov/Makefile +2 -0
@@ -1 +1,3 @@ +EXTRA_CFLAGS := -DSRCTREE='"$(srctree)"' -DOBJTREE='"$(objtree)"' + obj-$(CONFIG_GCOV_PROFILE) := base.o fs.o gcc_3_4.o