coverity, static checking etc.
I gave a lightning talk on this last year. This year I have a bit more data so could probably fill a whole session.
Last year I had been doing the coverity scans on an almost daily basis for 2-3 months. Now that we're a year in, I'd like to share some results, and show some of the more common trends and bug patterns that seem to pop up.
[ spoiler: For the most part, it's all pretty positive, but we still suck ]
It would also be good to have some more discussion about other tools we could be making more use of. (Nomination: Dan Carpenter for smatch).
Dave
Quoted 8 lines · Dave Jones
I gave a lightning talk on this last year. This year I have a bit more data so could probably fill a whole session.
Last year I had been doing the coverity scans on an almost daily basis for 2-3 months. Now that we're a year in, I'd like to share some results, and show some of the more common trends and bug patterns that seem to pop up.
Quoted 4 lines
[ spoiler: For the most part, it's all pretty positive, but we still suck ]
It would also be good to have some more discussion about other tools we could be making more use of. (Nomination: Dan Carpenter for smatch).
I'd like to see this topic as well. I think we could do a lot better here than we do. And don't forget that GCC is one of our top static analysis tools, if only because it's the only one *everyone* runs; that includes both warnings and the possibility of shipping and building our own GCC plugin.
I'd also like to nominate Christopher Li, for Sparse.
- Josh Triplett
Quoted 8 lines · Dave Jones
I gave a lightning talk on this last year. This year I have a bit more data so could probably fill a whole session.
Last year I had been doing the coverity scans on an almost daily basis for 2-3 months. Now that we're a year in, I'd like to share some results, and show some of the more common trends and bug patterns that seem to pop up.
Quoted 4 lines
[ spoiler: For the most part, it's all pretty positive, but we still suck ]
It would also be good to have some more discussion about other tools we could be making more use of. (Nomination: Dan Carpenter for smatch).
I'm interested in hearing about this, too. I've spent a fair amount of time digging through coverity scans and it's been quite useful. But it's still a hassle and there's lots of room for improvement in terms of making coverity/smatch/autobuilder/etc. results more accessible and usable.
Bjorn
Quoted 7 lines · On Fri, 2014-05-09 at 10:19 -0700, [email protected] wrote:
I'd like to see this topic as well. I think we could do a lot better here than we do. And don't forget that GCC is one of our top static analysis tools, if only because it's the only one *everyone* runs; that includes both warnings and the possibility of shipping and building our own GCC plugin.
I'd also like to nominate Christopher Li, for Sparse.
Seconded, I'm also interested in general in whether people still think sparse is useful and we should give it attention, or should focus more on really getting everything into gcc - we have a number of sparse warnings in very low-level header files that get used everywhere, for example the one I just fixed in [1] or the one I tried to fix but that ended up being buggy ([2]), but there doesn't seem to be much attention to these during the patch submission etc.
johannes
[1] http://thread.gmane.org/gmane.linux.kernel/1699192
[2] http://thread.gmane.org/gmane.linux.kernel/1642101/focus=1642108
Quoted 10 lines · Johannes Berg
Quoted 7 lines · On Fri, 2014-05-09 at 10:19 -0700, [email protected] wrote:
I'd like to see this topic as well. I think we could do a lot better here than we do. And don't forget that GCC is one of our top static analysis tools, if only because it's the only one *everyone* runs; that includes both warnings and the possibility of shipping and building our own GCC plugin.
I'd also like to nominate Christopher Li, for Sparse.
Quoted 8 lines
Seconded, I'm also interested in general in whether people still think sparse is useful and we should give it attention, or should focus more on really getting everything into gcc - we have a number of sparse warnings in very low-level header files that get used everywhere, for example the one I just fixed in [1] or the one I tried to fix but that ended up being buggy ([2]), but there doesn't seem to be much attention to these during the patch submission etc.
I use it (together with smatch) as part of my automated sanity tests.
Guenter
Quoted 7 lines · Johannes Berg
Seconded, I'm also interested in general in whether people still think sparse is useful and we should give it attention, or should focus more on really getting everything into gcc - we have a number of sparse warnings in very low-level header files that get used everywhere, for example the one I just fixed in [1] or the one I tried to fix but that ended up being buggy ([2]), but there doesn't seem to be much attention to these during the patch submission etc.
I certainly do all my builds with sparse and routinely find it identifies useful issues, though obviously getting the same capabilities in the compiler directly would be just as useful.
Quoted 9 lines · Johannes Berg
Quoted 1 line
I'd also like to nominate Christopher Li, for Sparse.
Seconded, I'm also interested in general in whether people still think sparse is useful and we should give it attention, or should focus more on really getting everything into gcc - we have a number of sparse warnings in very low-level header files that get used everywhere, for example the one I just fixed in [1] or the one I tried to fix but that ended up being buggy ([2]), but there doesn't seem to be much attention to these during the patch submission etc.
Just for the sake of datapoint -- running 'make C=1' before every git push is part of my automated workflow. So yes, I am routinely using sparse.
Signature · 2 lines
Jiri Kosina SUSE Labs
Quoted 10 lines · Johannes Berg
Quoted 7 lines · On Fri, 2014-05-09 at 10:19 -0700, [email protected] wrote:
I'd like to see this topic as well. I think we could do a lot better here than we do. And don't forget that GCC is one of our top static analysis tools, if only because it's the only one *everyone* runs; that includes both warnings and the possibility of shipping and building our own GCC plugin.
I'd also like to nominate Christopher Li, for Sparse.
Quoted 7 lines
Seconded, I'm also interested in general in whether people still think sparse is useful and we should give it attention, or should focus more on really getting everything into gcc - we have a number of sparse warnings in very low-level header files that get used everywhere, for example the one I just fixed in [1] or the one I tried to fix but that ended up being buggy ([2]), but there doesn't seem to be much attention to these during the patch submission etc.
I definitely still find Sparse useful and regularly check my own kernel code with it. Fengguang Wu's automated build-and-test bot also uses Sparse.
I *would* like to see more of Sparse's functionality make it into GCC, so that it gets used as part of default builds. HPA and I have been working to document some of the major bits so that GCC can implement them [1], and Tom Tromey is working on GCC implementations [2].
[1]
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59850
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59851
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59852
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59855
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59856
[2] https://github.com/tromey/gcc/tree/add-sparse-attributes
- Josh Triplett
Quoted 12 lines · Jiri Kosina
Quoted 9 lines · Johannes Berg
Quoted 1 line
I'd also like to nominate Christopher Li, for Sparse.
Seconded, I'm also interested in general in whether people still think sparse is useful and we should give it attention, or should focus more on really getting everything into gcc - we have a number of sparse warnings in very low-level header files that get used everywhere, for example the one I just fixed in [1] or the one I tried to fix but that ended up being buggy ([2]), but there doesn't seem to be much attention to these during the patch submission etc.
Quoted 2 lines
Just for the sake of datapoint -- running 'make C=1' before every git push is part of my automated workflow. So yes, I am routinely using sparse.
Same here - I try to run sparse every time I apply a new patch.
Thanks.
Signature · 1 line
Dmitry
Quoted 8 lines · On Friday 09 May 2014 13:07:09 Dave Jones wrote:
I gave a lightning talk on this last year. This year I have a bit more data so could probably fill a whole session.
Last year I had been doing the coverity scans on an almost daily basis for 2-3 months. Now that we're a year in, I'd like to share some results, and show some of the more common trends and bug patterns that seem to pop up.
Quoted 4 lines
[ spoiler: For the most part, it's all pretty positive, but we still suck ]
It would also be good to have some more discussion about other tools we could be making more use of. (Nomination: Dan Carpenter for smatch).
I'd be interested in this. One thing I'd been meaning to ask you about for ages is what I can do to get scan results for ARM (or any other architecture for that matter) specific code. We have a lot of that these days, and as I understand it, the results on the public website are just for x86 builds.
Arnd
Quoted 11 lines · Arnd Bergmann
Quoted 10 lines · On Friday 09 May 2014 13:07:09 Dave Jones wrote:
I gave a lightning talk on this last year. This year I have a bit more data so could probably fill a whole session.
Last year I had been doing the coverity scans on an almost daily basis for 2-3 months. Now that we're a year in, I'd like to share some results, and show some of the more common trends and bug patterns that seem to pop up.
[ spoiler: For the most part, it's all pretty positive, but we still suck ]
Quoted 8 lines
Quoted 2 lines
It would also be good to have some more discussion about other tools we could be making more use of. (Nomination: Dan Carpenter for smatch).
I'd be interested in this. One thing I'd been meaning to ask you about for ages is what I can do to get scan results for ARM (or any other architecture for that matter) specific code. We have a lot of that these days, and as I understand it, the results on the public website are just for x86 builds.
yeah, right now their tool (definitely front-end, but back-end too iirc)
is x86 only. So unless it's something that CONFIG_COMPILE_TEST would pick up,
it's not going to be in the scans.
Dave
Quoted 4 lines · Dave Jones
Last year I had been doing the coverity scans on an almost daily basis for 2-3 months. Now that we're a year in, I'd like to share some results, and show some of the more common trends and bug patterns that seem to pop up.
This probably doesn't add too much to the discussion, but as a subsystem maintainer, I like having easy-to-run analysis tools (or easily available scan results like coverity). It seems to lead to interesting patches from people who aren't really interested in the subsystem but just trawl through scan results.
It's pretty cool getting fixes for subtle (but in retrospect obvious) bugs from people who say "compile tested only because I have no hardware."
- R.
Quoted 12 lines · Roland Dreier
Quoted 4 lines · Dave Jones
Last year I had been doing the coverity scans on an almost daily basis for 2-3 months. Now that we're a year in, I'd like to share some results, and show some of the more common trends and bug patterns that seem to pop up.
This probably doesn't add too much to the discussion, but as a subsystem maintainer, I like having easy-to-run analysis tools (or easily available scan results like coverity). It seems to lead to interesting patches from people who aren't really interested in the subsystem but just trawl through scan results.
Quoted 3 lines
It's pretty cool getting fixes for subtle (but in retrospect obvious) bugs from people who say "compile tested only because I have no hardware."
I've been apprehensive about approving some of the people signing up for coverity. There's been a large influx of people over the last few months whose only prior kernel commits have been things like checkpatch fixes. On one hand, it's great to see people wanting to progress beyond that, but I know some maintainers have had a less than positive reaction with getting crap patches based on coverity results. It's a tricky balance, but I think for the most part my judgment is erring on the right side of the approve/deny fence.
The one thing I can't wait for Coverity to implement is the ability to say "bugs for this subsystem go to this mailing list". At that point the mails it sends out should be a lot more useful, and maintainers themselves will be able to take action. (I know some people hate the web UI, so this will hopefully appease them).
Most people have got a lot better at jumping on things as they get introduced, which is great. The stuff that sucks is the old reports that have been in their database forever. I've (along with several other people) been periodically going through these trying to weed out the real bugs from the false positive/intentionals.
I'll need to come up with some way to periodically send a bunch of those to lists too.
Dave
Sounds interesting! This sounds like it would be more of a technical track talk as opposed to the core plenary session, so it can reacher a larger set of people. Would you agree?
- Ted
Quoted 21 lines · On Friday 09 May 2014 16:33:07 Dave Jones wrote:
Quoted 11 lines · Arnd Bergmann
Quoted 10 lines · On Friday 09 May 2014 13:07:09 Dave Jones wrote:
I gave a lightning talk on this last year. This year I have a bit more data so could probably fill a whole session.
Last year I had been doing the coverity scans on an almost daily basis for 2-3 months. Now that we're a year in, I'd like to share some results, and show some of the more common trends and bug patterns that seem to pop up.
[ spoiler: For the most part, it's all pretty positive, but we still suck ]
Quoted 8 lines
Quoted 2 lines
It would also be good to have some more discussion about other tools we could be making more use of. (Nomination: Dan Carpenter for smatch).
I'd be interested in this. One thing I'd been meaning to ask you about for ages is what I can do to get scan results for ARM (or any other architecture for that matter) specific code. We have a lot of that these days, and as I understand it, the results on the public website are just for x86 builds.
Quoted 3 lines
yeah, right now their tool (definitely front-end, but back-end too iirc)
is x86 only. So unless it's something thatCONFIG_COMPILE_TESTwould pick up,
it's not going to be in the scans.
I have in the past successfully built architecture ports with the wrong compiler by just removing all the inline assembly and slightly tweaking the command line, for the purpose of review.
I'd assume I could do the same here, but I haven't figured out if there is a way for me to actually get the tool so I can run it on my own machine. When I signed for an account up last year, I only tried joining the "Linux" project. Maybe I need to add another git tree that is configured for building an ARM kernel?
Arnd
Quoted 10 lines
Last year I had been doing the coverity scans on an almost daily basis for 2-3 months. Now that we're a year in, I'd like to share some results, and show some of the more common trends and bug patterns that seem to pop up.
[ spoiler: For the most part, it's all pretty positive, but we still suck ]
It would also be good to have some more discussion about other tools we could be making more use of. (Nomination: Dan Carpenter for smatch).
I'm definately interested.
In my workflow, I use sparse/smatch/coccicheck/cppcheck before applying my own work, or patches to the i2c branches. (Oh, and rats and flawfinder, too, but so far, they didn't point to something worthwhile.)
I am interested in workflows and experiences from other people, how usage of static analyzers could be spread (gcc inclusion sounds great), how to make them more robust, etc... And by doing that, get a better feeling when an issue left the scope of static code checking and needs some proper handling.
Thanks,
Wolfram
Quoted 6 lines · Wolfram Sang
Quoted 6 lines
Last year I had been doing the coverity scans on an almost daily basis for 2-3 months. Now that we're a year in, I'd like to share some results, and show some of the more common trends and bug patterns that seem to pop up.
Quoted 6 lines
Quoted 4 lines
[ spoiler: For the most part, it's all pretty positive, but we still suck ]
It would also be good to have some more discussion about other tools we could be making more use of. (Nomination: Dan Carpenter for smatch).
I'm definately interested.
I am also interested in this because using these tools is just one way how to clean the source code and it must be automated.
Quoted 9 lines
In my workflow, I use sparse/smatch/coccicheck/cppcheck before applying my own work, or patches to the i2c branches. (Oh, and rats and flawfinder, too, but so far, they didn't point to something worthwhile.)
I am interested in workflows and experiences from other people, how usage of static analyzers could be spread (gcc inclusion sounds great), how to make them more robust, etc... And by doing that, get a better feeling when an issue left the scope of static code checking and needs some proper handling.
I expect that this is already the part of aiaiai. The part of this discussion should be also kernel-doc format checking because a lot of patches are trying to use kernel-doc format but it is just broken - even checker is in the kernel.
Everybody is using own testing - isn't it better to just add all these checking the part of the linux kernel that everybody can enable it and test it? I hope that everybody is running checkpatch before patch is sent. Why not just tell them run this in-kernel tool with all that checking enabled before you send the patch?
Thanks,
Michal
Signature · 5 lines
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91 w: www.monstr.eu p: +42-0-721842854 Maintainer of Linux kernel - Microblaze cpu - http://www.monstr.eu/fdt/ Maintainer of Linux kernel - Xilinx Zynq ARM architecture Microblaze U-BOOT custodian and responsible for u-boot arm zynq platform
I think that Julia Lawall would be interested in this topic. I've added her in CC.
Quoted 33 lines · Wolfram Sang
Quoted 10 lines
Last year I had been doing the coverity scans on an almost daily basis for 2-3 months. Now that we're a year in, I'd like to share some results, and show some of the more common trends and bug patterns that seem to pop up.
[ spoiler: For the most part, it's all pretty positive, but we still suck ]
It would also be good to have some more discussion about other tools we could be making more use of. (Nomination: Dan Carpenter for smatch).
I'm definately interested.
In my workflow, I use sparse/smatch/coccicheck/cppcheck before applying my own work, or patches to the i2c branches. (Oh, and rats and flawfinder, too, but so far, they didn't point to something worthwhile.)
I am interested in workflows and experiences from other people, how usage of static analyzers could be spread (gcc inclusion sounds great), how to make them more robust, etc... And by doing that, get a better feeling when an issue left the scope of static code checking and needs some proper handling.
Thanks,
Wolfram
_______________________________________________
Ksummit-discuss mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss
Signature · 1 line
Peter
Quoted 3 lines
is sent. Why not just tell them run this in-kernel tool with all that checking enabled before you send the patch?
make C=1 ? That could be perhaps updated to call sparse *and* smatch if it finds them installed.
The other tools I run one needs to know. They produce many false positives so they are not suited for an occasional user.
Quoted 3 lines · On Fri, May 09, 2014 at 08:57:58PM +0000, [email protected] wrote:
Sounds interesting! This sounds like it would be more of a technical track talk as opposed to the core plenary session, so it can reacher a larger set of people. Would you agree?
Yes. I think so. I'd love to give a talk about Smatch.
regards,
dan carpenter
Quoted 6 lines · Wolfram Sang
Quoted 3 lines
is sent. Why not just tell them run this in-kernel tool with all that checking enabled before you send the patch?
make C=1 ? That could be perhaps updated to call sparse *and* smatch if it finds them installed.
Smatch doesn't install very well... :( I should try fix that before kernel summit.
regards,
dan carpenter
Quoted 5 lines · Wolfram Sang
Quoted 2 lines
is sent. Why not just tell them run this in-kernel tool with all that checking enabled before you send the patch?
make C=1 ? That could be perhaps updated to call sparse *and* smatch if it finds them installed.
I do that with a simple wrapper:
~/bin/sparse:
#!/bin/sh
/path/to/real/sparse "$@" || exit 100
/path/to/smatch -p=kernel "$@" || exit 200
but integrating it would be better :)
johannes
Quoted 7 lines · Johannes Berg
Quoted 5 lines · Wolfram Sang
Quoted 2 lines
is sent. Why not just tell them run this in-kernel tool with all that checking enabled before you send the patch?
make C=1 ? That could be perhaps updated to call sparse *and* smatch if it finds them installed.
Quoted 8 lines
I do that with a simple wrapper:
~/bin/sparse:
#!/bin/sh/path/to/real/sparse "$@" || exit 100
/path/to/smatch -p=kernel "$@" || exit 200
I use "make C=1 ..." for sparse, followed by
"make C=1 CHECK="${SMATCH_PATH}/smatch --project=kernel ..." for smatch.
Guenter
Ditto. I personally use 'C=1 CHECK="ninja-check"' which is a wrapper which calls all static analyzers I know of.
Still, to make it easier for other developers not so much interested in static analyzing, brushing up C=1 to automatically use sparse and smatch if installed might be worthwhile.
Great to hear that Dan wants to improve installing smatch. I'll add the C=1 extension to my todo-list, yet I won't be angry if somebody else gets around to do it sooner.
Quoted 2 lines
Ditto. I personally use 'C=1 CHECK="ninja-check"' which is a wrapper which calls all static analyzers I know of.
+1 on ninja-check
Care to share your wrapper?
Peter
Quoted 6 lines · Peter Huewe
Quoted 2 lines
Ditto. I personally use 'C=1 CHECK="ninja-check"' which is a wrapper which calls all static analyzers I know of.
+1 on ninja-check
Care to share your wrapper?
Here, be warned that there will be some output you need to get used to ;)
#!/bin/sh -u
# wrapper to call various static checkers for kernel builds. # Use: make C=1 CHECK='ninja-check' ... # done by Wolfram Sang in 2012-14, version 20140514 - WTFPLv2
check_for()
{
command -v $1 > /dev/null
ret=$?
[ $ret -eq 0 ] && echo " $1" | tr a-z A-Z
return $ret
}# Get filename (last argument)
eval file_to_check=\${$#}
check_for sparse && sparse -Wsparse-all "$@"
check_for smatch && smatch --project=kernel "$@" 1>&2
check_for cppcheck && cppcheck -f -q --template=gcc --enable=all --language=c "$file_to_check"
check_for spatch && MODE=report scripts/coccicheck "$file_to_check" 1>&2
check_for flawfinder && flawfinder --minlevel=0 --quiet --dataonly --singleline "$file_to_check" 1>&2
check_for rats && rats --resultsonly -w 3 "$file_to_check" 1>&2
Em Mon, 12 May 2014 08:48:11 +0200
Michal Simek <[email protected]> escreveu:
Quoted 4 lines
The part of this discussion
should be also kernel-doc format checking because a lot of patches are trying to use kernel-doc format but it is just broken - even checker is in the kernel.
There are several drivers still using other styles for documenting functions.
I can be partially blamed for accepting such patches, but, I don't feel it is fair to refuse merging a patchset just because they're following some other documentation style. We have some really big drivers at media ported from another OS using Doxygen style. Fixing them would likely require hundreds of hours that could otherwise be used implementing new stuff or fixing real bugs.
So, I would rather prefer to have a (semi-)automated tool capable of converting from Doxygen and other styles into kernel-doc style that would make such conversion an easy task.
Regards,
Mauro
Hi Mauro
Quoted 17 lines · Mauro Carvalho Chehab
Em Mon, 12 May 2014 08:48:11 +0200
Michal Simek <[email protected]> escreveu:Quoted 4 lines
The part of this discussion
should be also kernel-doc format checking because a lot of patches are trying to use kernel-doc format but it is just broken - even checker is in the kernel.There are several drivers still using other styles for documenting functions.
I can be partially blamed for accepting such patches, but, I don't feel it is fair to refuse merging a patchset just because they're following some other documentation style. We have some really big drivers at media ported from another OS using Doxygen style. Fixing them would likely require hundreds of hours that could otherwise be used implementing new stuff or fixing real bugs.
Quoted 3 lines
So, I would rather prefer to have a (semi-)automated tool capable of converting from Doxygen and other styles into kernel-doc style that would make such conversion an easy task.
I have no experience with Doxygen but I can't see any problem. If Doxygen is used in the kernel in any subsystem then it should be checked if possible.
I am not definitely saying - use kernel-doc everywhere. But I am saying when you use kernel-doc format it should be used and checked properly.
The same is for Doxygen and others.
Warning/error should be checked and if there is any problem patch should be rejected.
Thanks,
Michal
Signature · 5 lines
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91 w: www.monstr.eu p: +42-0-721842854 Maintainer of Linux kernel - Microblaze cpu - http://www.monstr.eu/fdt/ Maintainer of Linux kernel - Xilinx Zynq ARM architecture Microblaze U-BOOT custodian and responsible for u-boot arm zynq platform