Skip to contentSkip to conversation

Add a section for static analysis tools

PATCH Superseded Marcelo Schmitt 5 participants 13 messages Mar 29, 2022 – Apr 1, 2022

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover Add a section for static analysis tools

Hi,

Thanks to everybody who commented on v1 for your kind and much helpful feedback.

I tried to add suggestions and ideas while keeping the text concise. Also, I took Dan and Julia's comments and included them into the documentation (patch 2) because I think they were very helpful in comparing the tools.
I didn't feel comfortable adding something comparing Sparse and Coccinelle directly as I'm not an expert with any of these tools either. Anyhow, that can be something to do in the future.

Thanks,
Marcelo

Marcelo Schmitt (2):
Documentation: dev-tools: Add a section for static analysis tools
Documentation: dev-tools: Enhance static analysis section with
discussion

 Documentation/dev-tools/testing-overview.rst | 64 ++++++++++++++++++++
 1 file changed, 64 insertions(+)
Signature · 1 line
2.35.1
#1 lore raw

PATCH v2 1/2 Documentation: dev-tools: Add a section for static analysis tools

Complement the Kernel Testing Guide documentation page by adding a section about static analysis tools.

---
Change log:
- Brought generic tool characteristics to the intro paragraph
- Made explicit that these tools run at compile time
- Added a note of caution about false positives
- Updated Coccinelle info to make it sound better and be more skimmable

 Documentation/dev-tools/testing-overview.rst | 31 ++++++++++++++++++++
 1 file changed, 31 insertions(+)
Documentation/dev-tools/testing-overview.rst +31 -0
index 65feb81edb14..b5e02dd3fd94 100644
@@ -115,3 +115,34 @@ that none of these errors are occurring during the test.
 Some of these tools integrate with KUnit or kselftest and will
 automatically fail tests if an issue is detected.
 
+Static Analysis Tools
+=====================
+
+In addition to testing a running kernel, one can also analyze kernel source code
+directly (**at compile time**) using **static analysis** tools. The tools
+commonly used in the kernel allow one to inspect the whole source tree or just
+specific files within it. They make it easier to detect and fix problems during
+the development process.
+
+Sparse can help test the kernel by performing type-checking, lock checking,
+value range checking, in addition to reporting various errors and warnings while
+examining the code. See the Documentation/dev-tools/sparse.rst documentation
+page for details on how to use it.
+
+Smatch extends Sparse and provides additional checks for programming logic
+mistakes such as missing breaks in switch statements, unused return values on
+error checking, forgetting to set an error code in the return of an error path,
+etc. Smatch also has tests against more serious issues such as integer
+overflows, null pointer dereferences, and memory leaks. See the project page at
+http://smatch.sourceforge.net/.
+
+Coccinelle is another static analyzer at our disposal. Coccinelle is often used
+to aid refactoring and collateral evolution of source code, but it can also help
+to avoid certain bugs that occur in common code patterns. The types of tests
+available include API tests, tests for correct usage of kernel iterators, checks
+for the soundness of free operations, analysis of locking behavior, and further
+tests known to help keep consistent kernel usage. See the
+Documentation/dev-tools/coccinelle.rst documentation page for details.
+
+Beware, though, that static analysis tools suffer from **false positives**.
+Errors and warns need to be evaluated carefully before attempting to fix them.
Signature · 1 line
2.35.1
#5 lore raw

PATCH v2 2/2 Documentation: dev-tools: Enhance static analysis section with discussion

In reply to Marcelo Schmitt

Enhance the static analysis tools section with a discussion on when to use each of them.

This was mainly taken from Dan Carpenter and Julia Lawall's comments on the previous documentation patch for static analysis tools.

Lore: https://lore.kernel.org/linux-doc/20220329090911.GX3293@kadam/T/#mb97770c8e938095aadc3ee08f4ac7fe32ae386e6

Cc Dan Carpenter <[email protected]>
Cc Julia Lawall <[email protected]>
---
 Documentation/dev-tools/testing-overview.rst | 33 ++++++++++++++++++++
 1 file changed, 33 insertions(+)
Documentation/dev-tools/testing-overview.rst +33 -0
index b5e02dd3fd94..91e479045d3a 100644
@@ -146,3 +146,36 @@ Documentation/dev-tools/coccinelle.rst documentation page for details.
 
 Beware, though, that static analysis tools suffer from **false positives**.
 Errors and warns need to be evaluated carefully before attempting to fix them.
+
+When to use Sparse and Smatch
+-----------------------------
+
+Sparse is useful for type checking, detecting places that use ``__user``
+pointers improperly, or finding endianness bugs. Sparse runs much faster than
+Smatch.
+
+Smatch does flow analysis and, if allowed to build the function database, it
+also does cross function analysis. Smatch tries to answer questions like where
+is this buffer allocated? How big is it? Can this index be controlled by the
+user? Is this variable larger than that variable?
+
+It's generally easier to write checks in Smatch than it is to write checks in
+Sparse. Nevertheless, there are some overlaps between Sparse and Smatch checks
+because there is no reason for re-implementing Sparse's check in Smatch.
+
+Strong points of Smatch and Coccinelle
+--------------------------------------
+
+Coccinelle is probably the easiest for writing checks. It works before the
+pre-compiler so it's easier to check for bugs in macros using Coccinelle.
+Coccinelle also writes patches fixes for you which no other tool does.
+
+With Coccinelle you can do a mass conversion from
+``kmalloc(x * size, GFP_KERNEL)`` to ``kmalloc_array(x, size, GFP_KERNEL)``, and
+that's really useful. If you just created a Smatch warning and try to push the
+work of converting on to the maintainers they would be annoyed. You'd have to
+argue about each warning if can really overflow or not.
+
+Coccinelle does no analysis of variable values, which is the strong point of
+Smatch. On the other hand, Coccinelle allows you to do simple things in a simple
+way.
Signature · 1 line
2.35.1
#2 lore raw

Re: [PATCH v2 1/2] Documentation: dev-tools: Add a section for static analysis tools

In reply to Marcelo Schmitt
Quoted 9 lines · Marcelo Schmitt

Complement the Kernel Testing Guide documentation page by adding a section about static analysis tools.

Quoted 6 lines

---
Change log:
- Brought generic tool characteristics to the intro paragraph
- Made explicit that these tools run at compile time
- Added a note of caution about false positives
- Updated Coccinelle info to make it sound better and be more skimmable

This looks a lot nicer to me!
Thanks for doing this.

Daniel

Reviewed-by on v2 1/2
#3 lore raw

Re: [PATCH v2 1/2] Documentation: dev-tools: Add a section for static analysis tools

In reply to Marcelo Schmitt
Quoted 9 lines · Marcelo Schmitt

Complement the Kernel Testing Guide documentation page by adding a section about static analysis tools.

Quoted 7 lines

---
Change log:
- Brought generic tool characteristics to the intro paragraph
- Made explicit that these tools run at compile time
- Added a note of caution about false positives
- Updated Coccinelle info to make it sound better and be more skimmable

Looks better to me: most of the things which I feel are still missing are added in the next patch.

I think it would be possible to combine the two if you wanted to, which might make the overall descriptions of the tools a bit stronger, but this works either way.

This is still

-- David

Acked-by on v2 2/2
#6 lore raw

Re: [PATCH v2 2/2] Documentation: dev-tools: Enhance static analysis section with discussion

In reply to Marcelo Schmitt
Quoted 7 lines · Marcelo Schmitt

Enhance the static analysis tools section with a discussion on when to use each of them.

This was mainly taken from Dan Carpenter and Julia Lawall's comments on the previous documentation patch for static analysis tools.

Quoted 6 lines

Lore: https://lore.kernel.org/linux-doc/20220329090911.GX3293@kadam/T/#mb97770c8e938095aadc3ee08f4ac7fe32ae386e6

Cc Dan Carpenter <[email protected]>
Cc Julia Lawall <[email protected]>

---

Thanks: this sort of "when to use which tool" information is really what the testing guide page needs.

I'm not familiar enough with these tools that I can really review the details properly, but nothing stands out as obviously wrong to me. I've made a few comments below regardless, but feel free to ignore them if they're not quite right.

Cheers,
-- David

Quoted 8 lines
 Documentation/dev-tools/testing-overview.rst | 33 ++++++++++++++++++++
 1 file changed, 33 insertions(+)
Documentation/dev-tools/testing-overview.rst +0 -0
index b5e02dd3fd94..91e479045d3a 100644

Documentation/dev-tools/testing-overview.rst Documentation/dev-tools/coccinelle.rst documentation page for details. · line 155

Quoted 10 lines
Documentation/dev-tools/testing-overview.rst +7 -0
@@ -146,3 +146,36 @@ Documentation/dev-tools/coccinelle.rst documentation page for details.
 
 Beware, though, that static analysis tools suffer from **false positives**.
 Errors and warns need to be evaluated carefully before attempting to fix them.
+
+When to use Sparse and Smatch
+-----------------------------
+
+Sparse is useful for type checking, detecting places that use ``__user``
+pointers improperly, or finding endianness bugs. Sparse runs much faster than
+Smatch.

Given that the __user pointer and endianness stuff is found as a result of Sparse's type checking support, would rewording this as "Sparse does type checking, such as [detecting places...]" or similar be more clear?

Documentation/dev-tools/testing-overview.rst Documentation/dev-tools/coccinelle.rst documentation page for details. · line 164

Quoted 9 lines
Documentation/dev-tools/testing-overview.rst +9 -0
+
+Smatch does flow analysis and, if allowed to build the function database, it
+also does cross function analysis. Smatch tries to answer questions like where
+is this buffer allocated? How big is it? Can this index be controlled by the
+user? Is this variable larger than that variable?
+
+It's generally easier to write checks in Smatch than it is to write checks in
+Sparse. Nevertheless, there are some overlaps between Sparse and Smatch checks
+because there is no reason for re-implementing Sparse's check in Smatch.

This last sentence isn't totally clear to me. Should this "because" be "so"?

Documentation/dev-tools/testing-overview.rst Documentation/dev-tools/coccinelle.rst documentation page for details. · line 173

Quoted 9 lines
Documentation/dev-tools/testing-overview.rst +9 -0
+
+Strong points of Smatch and Coccinelle
+--------------------------------------
+
+Coccinelle is probably the easiest for writing checks. It works before the
+pre-compiler so it's easier to check for bugs in macros using Coccinelle.
+Coccinelle also writes patches fixes for you which no other tool does.
+
+With Coccinelle you can do a mass conversion from

(Maybe start this with "For example," just to make it clear that this paragraph is mostly following on from how useful it is that Coccinelle produces fixes, not just warnings.)

Quoted 11 lines
Documentation/dev-tools/testing-overview.rst +8 -1
+``kmalloc(x * size, GFP_KERNEL)`` to ``kmalloc_array(x, size, GFP_KERNEL)``, and
+that's really useful. If you just created a Smatch warning and try to push the
+work of converting on to the maintainers they would be annoyed. You'd have to
+argue about each warning if can really overflow or not.
+
+Coccinelle does no analysis of variable values, which is the strong point of
+Smatch. On the other hand, Coccinelle allows you to do simple things in a simple
+way.
--

2.35.1

Acked-by on v2 1/2
#4 lore raw

Re: [PATCH v2 1/2] Documentation: dev-tools: Add a section for static analysis tools

In reply to Marcelo Schmitt
Quoted 8 lines · Marcelo Schmitt

Complement the Kernel Testing Guide documentation page by adding a section about static analysis tools.

Quoted 53 lines

---
Change log:
- Brought generic tool characteristics to the intro paragraph
- Made explicit that these tools run at compile time
- Added a note of caution about false positives
- Updated Coccinelle info to make it sound better and be more skimmable

 Documentation/dev-tools/testing-overview.rst | 31 ++++++++++++++++++++
 1 file changed, 31 insertions(+)
Documentation/dev-tools/testing-overview.rst +31 -1
index 65feb81edb14..b5e02dd3fd94 100644
@@ -115,3 +115,34 @@ that none of these errors are occurring during the test.
 Some of these tools integrate with KUnit or kselftest and will
 automatically fail tests if an issue is detected.
 
+Static Analysis Tools
+=====================
+
+In addition to testing a running kernel, one can also analyze kernel source code
+directly (**at compile time**) using **static analysis** tools. The tools
+commonly used in the kernel allow one to inspect the whole source tree or just
+specific files within it. They make it easier to detect and fix problems during
+the development process.
+
+Sparse can help test the kernel by performing type-checking, lock checking,
+value range checking, in addition to reporting various errors and warnings while
+examining the code. See the Documentation/dev-tools/sparse.rst documentation
+page for details on how to use it.
+
+Smatch extends Sparse and provides additional checks for programming logic
+mistakes such as missing breaks in switch statements, unused return values on
+error checking, forgetting to set an error code in the return of an error path,
+etc. Smatch also has tests against more serious issues such as integer
+overflows, null pointer dereferences, and memory leaks. See the project page at
+http://smatch.sourceforge.net/.
+
+Coccinelle is another static analyzer at our disposal. Coccinelle is often used
+to aid refactoring and collateral evolution of source code, but it can also help
+to avoid certain bugs that occur in common code patterns. The types of tests
+available include API tests, tests for correct usage of kernel iterators, checks
+for the soundness of free operations, analysis of locking behavior, and further
+tests known to help keep consistent kernel usage. See the
+Documentation/dev-tools/coccinelle.rst documentation page for details.
+
+Beware, though, that static analysis tools suffer from **false positives**.
+Errors and warns need to be evaluated carefully before attempting to fix them.
--

2.35.1

#12 lore raw

Re: [PATCH v2 2/2] Documentation: dev-tools: Enhance static analysis section with discussion

In reply to Marcelo Schmitt

Documentation/dev-tools/testing-overview.rst Documentation/dev-tools/coccinelle.rst documentation page for details. · line 170

Quoted 5 lines
Documentation/dev-tools/testing-overview.rst +5 -0
+Strong points of Smatch and Coccinelle
+--------------------------------------
+
+Coccinelle is probably the easiest for writing checks. It works before the
+pre-compiler so it's easier to check for bugs in macros using Coccinelle.

pre-processor

Quoted 1 line

+Coccinelle also writes patches fixes for you which no other tool does.

writes patches fixes -> creates patches

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+With Coccinelle you can do a mass conversion from

you can -> you can, for example,

julia

Quoted 12 lines

+``kmalloc(x * size, GFP_KERNEL)`` to ``kmalloc_array(x, size, GFP_KERNEL)``, and

Documentation/dev-tools/testing-overview.rst +7 -1
+that's really useful. If you just created a Smatch warning and try to push the
+work of converting on to the maintainers they would be annoyed. You'd have to
+argue about each warning if can really overflow or not.
+
+Coccinelle does no analysis of variable values, which is the strong point of
+Smatch. On the other hand, Coccinelle allows you to do simple things in a simple
+way.
--

2.35.1

#7 lore raw

Re: [PATCH v2 2/2] Documentation: dev-tools: Enhance static analysis section with discussion

In reply to David Gow
Quoted 4 lines
Quoted 4 lines
Documentation/dev-tools/testing-overview.rst +4 -0
+Strong points of Smatch and Coccinelle
+--------------------------------------
+
+Coccinelle is probably the easiest for writing checks. It works before the
Quoted 8 lines

Documentation/dev-tools/testing-overview.rst Documentation/dev-tools/coccinelle.rst documentation page for details. · line 173

Quoted 4 lines
Documentation/dev-tools/testing-overview.rst +4 -0
+pre-compiler so it's easier to check for bugs in macros using Coccinelle.
+Coccinelle also writes patches fixes for you which no other tool does.
+
+With Coccinelle you can do a mass conversion from

(Maybe start this with "For example," just to make it clear that this paragraph is mostly following on from how useful it is that Coccinelle produces fixes, not just warnings.)

I also suggested "for example", in a different place, but either is fine.

julia

Quoted 13 lines
Quoted 12 lines

+``kmalloc(x * size, GFP_KERNEL)`` to ``kmalloc_array(x, size, GFP_KERNEL)``, and

Documentation/dev-tools/testing-overview.rst +7 -1
+that's really useful. If you just created a Smatch warning and try to push the
+work of converting on to the maintainers they would be annoyed. You'd have to
+argue about each warning if can really overflow or not.
+
+Coccinelle does no analysis of variable values, which is the strong point of
+Smatch. On the other hand, Coccinelle allows you to do simple things in a simple
+way.
--

2.35.1

#8 lore raw

Re: [PATCH v2 2/2] Documentation: dev-tools: Enhance static analysis section with discussion

In reply to David Gow
Quoted 38 lines · David Gow
Quoted 7 lines · Marcelo Schmitt

Enhance the static analysis tools section with a discussion on when to use each of them.

This was mainly taken from Dan Carpenter and Julia Lawall's comments on the previous documentation patch for static analysis tools.

Quoted 6 lines

Lore: https://lore.kernel.org/linux-doc/20220329090911.GX3293@kadam/T/#mb97770c8e938095aadc3ee08f4ac7fe32ae386e6

Cc Dan Carpenter <[email protected]>
Cc Julia Lawall <[email protected]>

---

Thanks: this sort of "when to use which tool" information is really what the testing guide page needs.

I'm not familiar enough with these tools that I can really review the details properly, but nothing stands out as obviously wrong to me. I've made a few comments below regardless, but feel free to ignore them if they're not quite right.

Cheers,
-- David

Quoted 9 lines
 Documentation/dev-tools/testing-overview.rst | 33 ++++++++++++++++++++
 1 file changed, 33 insertions(+)
Documentation/dev-tools/testing-overview.rst +0 -0
index b5e02dd3fd94..91e479045d3a 100644
@@ -146,3 +146,36 @@ Documentation/dev-tools/coccinelle.rst documentation page for details.
 
Quoted 14 lines

Documentation/dev-tools/testing-overview.rst Documentation/dev-tools/coccinelle.rst documentation page for details. · line 155

Quoted 9 lines
Documentation/dev-tools/testing-overview.rst +7 -0
 Beware, though, that static analysis tools suffer from **false positives**.
 Errors and warns need to be evaluated carefully before attempting to fix them.
+
+When to use Sparse and Smatch
+-----------------------------
+
+Sparse is useful for type checking, detecting places that use ``__user``
+pointers improperly, or finding endianness bugs. Sparse runs much faster than
+Smatch.

Given that the __user pointer and endianness stuff is found as a result of Sparse's type checking support, would rewording this as "Sparse does type checking, such as [detecting places...]" or similar be more clear?

Myabe. I tried changing it a little while adding a bit of information from
https://lwn.net/Articles/689907/

"Sparse does type checking, such as verifying that annotated variables do not cause endianness bugs, detecting places that use ``__user`` pointers improperly, and analyzing the compatibility of symbol initializers."

Does it sound better?

Quoted 12 lines

Documentation/dev-tools/testing-overview.rst Documentation/dev-tools/coccinelle.rst documentation page for details. · line 164

Quoted 10 lines
Documentation/dev-tools/testing-overview.rst +9 -0
+
+Smatch does flow analysis and, if allowed to build the function database, it
+also does cross function analysis. Smatch tries to answer questions like where
+is this buffer allocated? How big is it? Can this index be controlled by the
+user? Is this variable larger than that variable?
+
+It's generally easier to write checks in Smatch than it is to write checks in
+Sparse. Nevertheless, there are some overlaps between Sparse and Smatch checks
+because there is no reason for re-implementing Sparse's check in Smatch.

This last sentence isn't totally clear to me. Should this "because" be "so"?

Smatch uses (is shipped with) a modified Sparse implementation which it uses as a C parser. Apparently, Sparse does some checkings while parsing the code for Smatch so that's why we have some overlapping between the checks made when we run Smatch and the ones made when we run Sparse alone.

I didn't dig into the code, but I guess further modifying Sparse to prevent it from doing some types of cheks wouldn't add much to Smatch. That last saying should've reflected this fact, but it seems to cause confusion without a proper context. Reading the sentence back again, I think we could just drop the last part:

"Nevertheless, there are some overlaps between Sparse and Smatch checks."

Quoted 14 lines

Documentation/dev-tools/testing-overview.rst Documentation/dev-tools/coccinelle.rst documentation page for details. · line 173

Quoted 10 lines
Documentation/dev-tools/testing-overview.rst +9 -0
+
+Strong points of Smatch and Coccinelle
+--------------------------------------
+
+Coccinelle is probably the easiest for writing checks. It works before the
+pre-compiler so it's easier to check for bugs in macros using Coccinelle.
+Coccinelle also writes patches fixes for you which no other tool does.
+
+With Coccinelle you can do a mass conversion from

(Maybe start this with "For example," just to make it clear that this paragraph is mostly following on from how useful it is that Coccinelle produces fixes, not just warnings.)

Will do

Quoted 12 lines
Quoted 12 lines
Documentation/dev-tools/testing-overview.rst +8 -1
+``kmalloc(x * size, GFP_KERNEL)`` to ``kmalloc_array(x, size, GFP_KERNEL)``, and
+that's really useful. If you just created a Smatch warning and try to push the
+work of converting on to the maintainers they would be annoyed. You'd have to
+argue about each warning if can really overflow or not.
+
+Coccinelle does no analysis of variable values, which is the strong point of
+Smatch. On the other hand, Coccinelle allows you to do simple things in a simple
+way.
--

2.35.1

#10 lore raw

Re: [PATCH v2 2/2] Documentation: dev-tools: Enhance static analysis section with discussion

In reply to David Gow
Quoted 12 lines · David Gow

Documentation/dev-tools/testing-overview.rst Documentation/dev-tools/coccinelle.rst documentation page for details. · line 164

Quoted 9 lines
Documentation/dev-tools/testing-overview.rst +9 -0
+
+Smatch does flow analysis and, if allowed to build the function database, it
+also does cross function analysis. Smatch tries to answer questions like where
+is this buffer allocated? How big is it? Can this index be controlled by the
+user? Is this variable larger than that variable?
+
+It's generally easier to write checks in Smatch than it is to write checks in
+Sparse. Nevertheless, there are some overlaps between Sparse and Smatch checks
+because there is no reason for re-implementing Sparse's check in Smatch.

This last sentence isn't totally clear to me. Should this "because" be "so"?

I stopped reading your email when you wrote "Cheers, David" but I should have scrolled down.

There is not very much overlap between Sparse and Smatch. Both have a warning for if (!x & y). That is a tiny thing. The big overlap is when it comes to the locking checks. The Smatch check for locking is honestly way better and more capable.

I always run both Sparse and Smatch on my patches. I should run Coccinelle as well, but I'm more familiar with Sparse and Smatch.

regards,
dan carpenter

#9 lore raw

Re: [PATCH v2 2/2] Documentation: dev-tools: Enhance static analysis section with discussion

In reply to Marcelo Schmitt
Quoted 55 lines · Marcelo Schmitt
Quoted 38 lines · David Gow
Quoted 7 lines · Marcelo Schmitt

Enhance the static analysis tools section with a discussion on when to use each of them.

This was mainly taken from Dan Carpenter and Julia Lawall's comments on the previous documentation patch for static analysis tools.

Quoted 6 lines

Lore: https://lore.kernel.org/linux-doc/20220329090911.GX3293@kadam/T/#mb97770c8e938095aadc3ee08f4ac7fe32ae386e6

Cc Dan Carpenter <[email protected]>
Cc Julia Lawall <[email protected]>

---

Thanks: this sort of "when to use which tool" information is really what the testing guide page needs.

I'm not familiar enough with these tools that I can really review the details properly, but nothing stands out as obviously wrong to me. I've made a few comments below regardless, but feel free to ignore them if they're not quite right.

Cheers,
-- David

Quoted 9 lines
 Documentation/dev-tools/testing-overview.rst | 33 ++++++++++++++++++++
 1 file changed, 33 insertions(+)
Documentation/dev-tools/testing-overview.rst +0 -0
index b5e02dd3fd94..91e479045d3a 100644
@@ -146,3 +146,36 @@ Documentation/dev-tools/coccinelle.rst documentation page for details.
 
Quoted 14 lines

Documentation/dev-tools/testing-overview.rst Documentation/dev-tools/coccinelle.rst documentation page for details. · line 155

Quoted 9 lines
Documentation/dev-tools/testing-overview.rst +7 -0
 Beware, though, that static analysis tools suffer from **false positives**.
 Errors and warns need to be evaluated carefully before attempting to fix them.
+
+When to use Sparse and Smatch
+-----------------------------
+
+Sparse is useful for type checking, detecting places that use ``__user``
+pointers improperly, or finding endianness bugs. Sparse runs much faster than
+Smatch.

Given that the __user pointer and endianness stuff is found as a result of Sparse's type checking support, would rewording this as "Sparse does type checking, such as [detecting places...]" or similar be more clear?

Quoted 9 lines

Myabe. I tried changing it a little while adding a bit of information from
https://lwn.net/Articles/689907/

"Sparse does type checking, such as verifying that annotated variables do not cause endianness bugs, detecting places that use ``__user`` pointers improperly, and analyzing the compatibility of symbol initializers."

Does it sound better?

Yeah: that sounds much better to me. Thanks!

Quoted 18 lines
Quoted 12 lines

Documentation/dev-tools/testing-overview.rst Documentation/dev-tools/coccinelle.rst documentation page for details. · line 164

Quoted 10 lines
Documentation/dev-tools/testing-overview.rst +9 -0
+
+Smatch does flow analysis and, if allowed to build the function database, it
+also does cross function analysis. Smatch tries to answer questions like where
+is this buffer allocated? How big is it? Can this index be controlled by the
+user? Is this variable larger than that variable?
+
+It's generally easier to write checks in Smatch than it is to write checks in
+Sparse. Nevertheless, there are some overlaps between Sparse and Smatch checks
+because there is no reason for re-implementing Sparse's check in Smatch.

This last sentence isn't totally clear to me. Should this "because" be "so"?

Smatch uses (is shipped with) a modified Sparse implementation which it uses as a C parser. Apparently, Sparse does some checkings while parsing the code for Smatch so that's why we have some overlapping between the checks made when we run Smatch and the ones made when we run Sparse alone.

Quoted 8 lines

I didn't dig into the code, but I guess further modifying Sparse to prevent it from doing some types of cheks wouldn't add much to Smatch. That last saying should've reflected this fact, but it seems to cause confusion without a proper context. Reading the sentence back again, I think we could just drop the last part:

"Nevertheless, there are some overlaps between Sparse and Smatch checks."

Yeah, I do think that makes more sense. I don't think the fact that some of the checks overlap causes any problems at all, to be honest, so you _could_ get rid of the whole sentence without losing too much, but I'm also happy with it as it is in v3.

Quoted 29 lines
Quoted 14 lines

Documentation/dev-tools/testing-overview.rst Documentation/dev-tools/coccinelle.rst documentation page for details. · line 173

Quoted 10 lines
Documentation/dev-tools/testing-overview.rst +9 -0
+
+Strong points of Smatch and Coccinelle
+--------------------------------------
+
+Coccinelle is probably the easiest for writing checks. It works before the
+pre-compiler so it's easier to check for bugs in macros using Coccinelle.
+Coccinelle also writes patches fixes for you which no other tool does.
+
+With Coccinelle you can do a mass conversion from

(Maybe start this with "For example," just to make it clear that this paragraph is mostly following on from how useful it is that Coccinelle produces fixes, not just warnings.)

Will do

Quoted 12 lines
Quoted 12 lines
Documentation/dev-tools/testing-overview.rst +8 -1
+``kmalloc(x * size, GFP_KERNEL)`` to ``kmalloc_array(x, size, GFP_KERNEL)``, and
+that's really useful. If you just created a Smatch warning and try to push the
+work of converting on to the maintainers they would be annoyed. You'd have to
+argue about each warning if can really overflow or not.
+
+Coccinelle does no analysis of variable values, which is the strong point of
+Smatch. On the other hand, Coccinelle allows you to do simple things in a simple
+way.
--

2.35.1

#11 lore raw

Re: [PATCH v2 2/2] Documentation: dev-tools: Enhance static analysis section with discussion

In reply to Dan Carpenter
Quoted 18 lines · Dan Carpenter
Quoted 12 lines · David Gow

Documentation/dev-tools/testing-overview.rst Documentation/dev-tools/coccinelle.rst documentation page for details. · line 164

Quoted 9 lines
Documentation/dev-tools/testing-overview.rst +9 -0
+
+Smatch does flow analysis and, if allowed to build the function database, it
+also does cross function analysis. Smatch tries to answer questions like where
+is this buffer allocated? How big is it? Can this index be controlled by the
+user? Is this variable larger than that variable?
+
+It's generally easier to write checks in Smatch than it is to write checks in
+Sparse. Nevertheless, there are some overlaps between Sparse and Smatch checks
+because there is no reason for re-implementing Sparse's check in Smatch.

This last sentence isn't totally clear to me. Should this "because" be "so"?

I stopped reading your email when you wrote "Cheers, David" but I should have scrolled down.

Quoted 7 lines

There is not very much overlap between Sparse and Smatch. Both have a warning for if (!x & y). That is a tiny thing. The big overlap is when it comes to the locking checks. The Smatch check for locking is honestly way better and more capable.

I always run both Sparse and Smatch on my patches. I should run Coccinelle as well, but I'm more familiar with Sparse and Smatch.

Makes sense. I agree that the overlap doesn't seem particularly
important: it's the differences which should be more evident.

v3[1] of the patch cuts this down to just "Nevertheless, there are some overlaps between Sparse and Smatch checks.", which I think is an improvement.

Thanks,
-- David

[1]: https://lore.kernel.org/all/62f461a20600b95e694016c4e5348ef2e260fa87.1648674305.git.marcelo.schmitt1@gmail.com/