Add a section for static analysis tools
Subsystems (3)
v3 cover Add a section for static analysis tools
Hi all,
This is the third patch version in the direction of complementing the testing guide documentation page with information about static analysis tools.
Thank you for your suggestions and comments so far. These docs wouldn't get so helpful without them.
Change log v2 -> v3:
- Changed the paragraph about Sparse to make it sound better (hopefully)
- Minor adjusts to make the considerations about Coccinelle sound better and be precise
Change log v1 -> v2:
- New patch adding considerations on when to use each tool
- Brought generic tool characteristics to the intro paragraph
- Made explicit that these tools run at compile time
- Added a note of caution about false positives
- Updated Coccinelle info to make it sound better and be more skimmable
Marcelo Schmitt (2):
Documentation: dev-tools: Add a section for static analysis tools
Documentation: dev-tools: Enhance static analysis section with
discussion
Documentation/dev-tools/testing-overview.rst | 63 ++++++++++++++++++++ 1 file changed, 63 insertions(+)
Signature · 1 line
2.35.1
PATCH v3 1/2 Documentation: dev-tools: Add a section for static analysis tools
Complement the Kernel Testing Guide documentation page by adding a section about static analysis tools.
---
Change log v2 -> v3:
- Added Julia's acknowledgment tag
Change log v1 -> v2:
- Brought generic tool characteristics to the intro paragraph
- Made explicit that these tools run at compile time
- Added a note of caution about false positives
- Updated Coccinelle info to make it sound better and be more skimmable
Documentation/dev-tools/testing-overview.rst | 31 ++++++++++++++++++++ 1 file changed, 31 insertions(+)
Documentation/dev-tools/testing-overview.rst +31 -0
@@ -115,3 +115,34 @@ that none of these errors are occurring during the test. Some of these tools integrate with KUnit or kselftest and will automatically fail tests if an issue is detected. +Static Analysis Tools +===================== + +In addition to testing a running kernel, one can also analyze kernel source code +directly (**at compile time**) using **static analysis** tools. The tools +commonly used in the kernel allow one to inspect the whole source tree or just +specific files within it. They make it easier to detect and fix problems during +the development process. + +Sparse can help test the kernel by performing type-checking, lock checking, +value range checking, in addition to reporting various errors and warnings while +examining the code. See the Documentation/dev-tools/sparse.rst documentation +page for details on how to use it. + +Smatch extends Sparse and provides additional checks for programming logic +mistakes such as missing breaks in switch statements, unused return values on +error checking, forgetting to set an error code in the return of an error path, +etc. Smatch also has tests against more serious issues such as integer +overflows, null pointer dereferences, and memory leaks. See the project page at +http://smatch.sourceforge.net/. + +Coccinelle is another static analyzer at our disposal. Coccinelle is often used +to aid refactoring and collateral evolution of source code, but it can also help +to avoid certain bugs that occur in common code patterns. The types of tests +available include API tests, tests for correct usage of kernel iterators, checks +for the soundness of free operations, analysis of locking behavior, and further +tests known to help keep consistent kernel usage. See the +Documentation/dev-tools/coccinelle.rst documentation page for details. + +Beware, though, that static analysis tools suffer from **false positives**. +Errors and warns need to be evaluated carefully before attempting to fix them.
Signature · 1 line
2.35.1
Re: [PATCH v3 1/2] Documentation: dev-tools: Add a section for static analysis tools
Quoted 52 lines · Marcelo Schmitt
Complement the Kernel Testing Guide documentation page by adding a section about static analysis tools.
Signed-off-by Marcelo Schmitt <[email protected]>Acked-by Daniel Latypov <[email protected]>Acked-by Dan Carpenter <[email protected]>Acked-by Julia Lawall <[email protected]>Reviewed-by David Gow <[email protected]>Reviewed-by Shuah Khan <[email protected]>---
Change log v2 -> v3:
- Added Julia's acknowledgment tagChange log v1 -> v2:
- Brought generic tool characteristics to the intro paragraph
- Made explicit that these tools run at compile time
- Added a note of caution about false positives
- Updated Coccinelle info to make it sound better and be more skimmableDocumentation/dev-tools/testing-overview.rst | 31 ++++++++++++++++++++ 1 file changed, 31 insertions(+)Documentation/dev-tools/testing-overview.rst +21
-0@@ -115,3 +115,34 @@ that none of these errors are occurring during the test. Some of these tools integrate with KUnit or kselftest and will automatically fail tests if an issue is detected. +Static Analysis Tools +===================== + +In addition to testing a running kernel, one can also analyze kernel source code +directly (**at compile time**) using **static analysis** tools. The tools +commonly used in the kernel allow one to inspect the whole source tree or just +specific files within it. They make it easier to detect and fix problems during +the development process. + +Sparse can help test the kernel by performing type-checking, lock checking, +value range checking, in addition to reporting various errors and warnings while +examining the code. See the Documentation/dev-tools/sparse.rst documentation +page for details on how to use it. + +Smatch extends Sparse and provides additional checks for programming logic +mistakes such as missing breaks in switch statements, unused return values on +error checking, forgetting to set an error code in the return of an error path, +etc. Smatch also has tests against more serious issues such as integer +overflows, null pointer dereferences, and memory leaks. See the project page at +http://smatch.sourceforge.net/. +
Documentation/dev-tools/testing-overview.rst that none of these errors are occurring during the test. · line 148
Quoted 10 lines
Documentation/dev-tools/testing-overview.rst +10
-0+Coccinelle is another static analyzer at our disposal. Coccinelle is often used +to aid refactoring and collateral evolution of source code, but it can also help +to avoid certain bugs that occur in common code patterns. The types of tests +available include API tests, tests for correct usage of kernel iterators, checks +for the soundness of free operations, analysis of locking behavior, and further +tests known to help keep consistent kernel usage. See the +Documentation/dev-tools/coccinelle.rst documentation page for details. + +Beware, though, that static analysis tools suffer from **false positives**. +Errors and warns need to be evaluated carefully before attempting to fix them.
Hi Marcelo,
Should we include static analysis tools based on LLVM? For example, Clang static analysis.
Quoted 3 lines
--
2.35.1
Re: [PATCH v3 1/2] Documentation: dev-tools: Add a section for static analysis tools
Hi Dongliang,
Quoted 67 lines · Dongliang Mu
Quoted 52 lines · Marcelo Schmitt
Complement the Kernel Testing Guide documentation page by adding a section about static analysis tools.
Signed-off-by Marcelo Schmitt <[email protected]>Acked-by Daniel Latypov <[email protected]>Acked-by Dan Carpenter <[email protected]>Acked-by Julia Lawall <[email protected]>Reviewed-by David Gow <[email protected]>Reviewed-by Shuah Khan <[email protected]>---
Change log v2 -> v3:
- Added Julia's acknowledgment tagChange log v1 -> v2:
- Brought generic tool characteristics to the intro paragraph
- Made explicit that these tools run at compile time
- Added a note of caution about false positives
- Updated Coccinelle info to make it sound better and be more skimmableDocumentation/dev-tools/testing-overview.rst | 31 ++++++++++++++++++++ 1 file changed, 31 insertions(+)Documentation/dev-tools/testing-overview.rst +21
-0@@ -115,3 +115,34 @@ that none of these errors are occurring during the test. Some of these tools integrate with KUnit or kselftest and will automatically fail tests if an issue is detected. +Static Analysis Tools +===================== + +In addition to testing a running kernel, one can also analyze kernel source code +directly (**at compile time**) using **static analysis** tools. The tools +commonly used in the kernel allow one to inspect the whole source tree or just +specific files within it. They make it easier to detect and fix problems during +the development process. + +Sparse can help test the kernel by performing type-checking, lock checking, +value range checking, in addition to reporting various errors and warnings while +examining the code. See the Documentation/dev-tools/sparse.rst documentation +page for details on how to use it. + +Smatch extends Sparse and provides additional checks for programming logic +mistakes such as missing breaks in switch statements, unused return values on +error checking, forgetting to set an error code in the return of an error path, +etc. Smatch also has tests against more serious issues such as integer +overflows, null pointer dereferences, and memory leaks. See the project page at +http://smatch.sourceforge.net/. +Documentation/dev-tools/testing-overview.rst that none of these errors are occurring during the test. · line 148
Quoted 10 lines
Documentation/dev-tools/testing-overview.rst +10
-0+Coccinelle is another static analyzer at our disposal. Coccinelle is often used +to aid refactoring and collateral evolution of source code, but it can also help +to avoid certain bugs that occur in common code patterns. The types of tests +available include API tests, tests for correct usage of kernel iterators, checks +for the soundness of free operations, analysis of locking behavior, and further +tests known to help keep consistent kernel usage. See the +Documentation/dev-tools/coccinelle.rst documentation page for details. + +Beware, though, that static analysis tools suffer from **false positives**. +Errors and warns need to be evaluated carefully before attempting to fix them.Hi Marcelo,
Quoted 2 lines
Should we include static analysis tools based on LLVM? For example, Clang static analysis.
I think that would be a good addition. I haven't checked out Clang tools though, so it would take me a bit more time to write something about that.
Quoted 4 lines
Quoted 4 lines
--
2.35.1
Jonathan Corbet
maintainerRe: [PATCH v3 1/2] Documentation: dev-tools: Add a section for static analysis tools
Quoted 7 lines · Marcelo Schmitt
Quoted 2 lines · Dongliang Mu
Should we include static analysis tools based on LLVM? For example, Clang static analysis.
I think that would be a good addition. I haven't checked out Clang tools though, so it would take me a bit more time to write something about that.
That seems like a good topic for a future patch. Meanwhile I've applied this series, thanks.
jon
PATCH v3 2/2 Documentation: dev-tools: Enhance static analysis section with discussion
In reply to Marcelo SchmittEnhance the static analysis tools section with a discussion on when to use each of them.
This was mainly taken from Dan Carpenter and Julia Lawall's comments on a previous documentation patch for static analysis tools.
---
Change log v2 -> v3:
- Changed the paragraph about Sparse to make it sound better (hopefully)
- Minor adjusts to make the considerations about Coccinelle sound better and be precise
Documentation/dev-tools/testing-overview.rst | 32 ++++++++++++++++++++ 1 file changed, 32 insertions(+)
Documentation/dev-tools/testing-overview.rst +32 -0
@@ -146,3 +146,35 @@ Documentation/dev-tools/coccinelle.rst documentation page for details. Beware, though, that static analysis tools suffer from **false positives**. Errors and warns need to be evaluated carefully before attempting to fix them. + +When to use Sparse and Smatch +----------------------------- + +Sparse does type checking, such as verifying that annotated variables do not +cause endianness bugs, detecting places that use ``__user`` pointers improperly, +and analyzing the compatibility of symbol initializers. + +Smatch does flow analysis and, if allowed to build the function database, it +also does cross function analysis. Smatch tries to answer questions like where +is this buffer allocated? How big is it? Can this index be controlled by the +user? Is this variable larger than that variable? + +It's generally easier to write checks in Smatch than it is to write checks in +Sparse. Nevertheless, there are some overlaps between Sparse and Smatch checks. + +Strong points of Smatch and Coccinelle +-------------------------------------- + +Coccinelle is probably the easiest for writing checks. It works before the +pre-processor so it's easier to check for bugs in macros using Coccinelle. +Coccinelle also creates patches for you, which no other tool does. + +For example, with Coccinelle you can do a mass conversion from +``kmalloc(x * size, GFP_KERNEL)`` to ``kmalloc_array(x, size, GFP_KERNEL)``, and +that's really useful. If you just created a Smatch warning and try to push the +work of converting on to the maintainers they would be annoyed. You'd have to +argue about each warning if can really overflow or not. + +Coccinelle does no analysis of variable values, which is the strong point of +Smatch. On the other hand, Coccinelle allows you to do simple things in a simple +way.
Signature · 1 line
2.35.1
Re: [PATCH v3 2/2] Documentation: dev-tools: Enhance static analysis section with discussion
Quoted 13 lines · Marcelo Schmitt
Enhance the static analysis tools section with a discussion on when to use each of them.
This was mainly taken from Dan Carpenter and Julia Lawall's comments on a previous documentation patch for static analysis tools.
Signed-off-by Marcelo Schmitt <[email protected]>Acked-by David Gow <[email protected]>Cc Dan Carpenter <[email protected]>Cc Julia Lawall <[email protected]>
Quoted 5 lines
---
Change log v2 -> v3:
- Changed the paragraph about Sparse to make it sound better (hopefully)
- Minor adjusts to make the considerations about Coccinelle sound better and be precise
These (plus the cut down note on Sparse/Smatch overlaps) are definitely an improvement.
Assuming no-one with more knowledge of these that me objects, I think this is good-to-go!
Thanks,
-- David
Quoted 23 lines · Marcelo Schmitt
Hi all,
This is the third patch version in the direction of complementing the testing guide documentation page with information about static analysis tools.
Thank you for your suggestions and comments so far. These docs wouldn't get so helpful without them.
Change log v2 -> v3:
- Changed the paragraph about Sparse to make it sound better (hopefully)
- Minor adjusts to make the considerations about Coccinelle sound better and be preciseChange log v1 -> v2:
- New patch adding considerations on when to use each tool
- Brought generic tool characteristics to the intro paragraph
- Made explicit that these tools run at compile time
- Added a note of caution about false positives
- Updated Coccinelle info to make it sound better and be more skimmable
Quoted 8 lines
Marcelo Schmitt (2):
Documentation: dev-tools: Add a section for static analysis tools
Documentation: dev-tools: Enhance static analysis section with
discussionDocumentation/dev-tools/testing-overview.rst | 63 ++++++++++++++++++++ 1 file changed, 63 insertions(+)
This is looking pretty good to me: thanks for helping to improve the documentation!
CCing Hu Haowen as an FYI for the zh_CN translation.
-- David