travis-ci: added clang static analysis
PATCH 1/1 travis-ci: added clang static analysis
Added a job to run clang static code analysis on the master and maint branch
--- .travis.yml | 17 ++++++++++++++++- ci/run-static-analysis.sh | 9 ++++++++- 2 files changed, 24 insertions(+), 2 deletions(-)
.travis.yml +16 -1
@@ -48,7 +48,7 @@ matrix: before_install: before_script: script: ci/run-linux32-docker.sh - - env: jobname=StaticAnalysis + - env: jobname=CocciStaticAnalysis os: linux compiler: addons:
@@ -59,6 +59,21 @@ matrix: before_script: script: ci/run-static-analysis.sh after_failure: + - if: branch IN (master, maint) + env: jobname=ClangStaticAnalysis + os: linux + compiler: + add_ons: + apt: + sources: + - ubuntu-toolchain-r-test + - llvm-toolchain-trusty + packages: + - clang + before_install: + before_script: + script: ci/run-static-analysis.sh + after_failure: - env: jobname=Documentation os: linux compiler:
ci/run-static-analysis.sh +8 -1
@@ -5,6 +5,13 @@ . ${0%/*}/lib-travisci.sh -make coccicheck +case "$jobname" in +ClangStaticAnalysis) + scan-build -analyze-headers --status-bugs make + ;; +CocciStaticAnalysis) + make coccicheck + ;; +esac save_good_tree
Signature · 1 line
2.16.2.248.ge2408a6f7.dirty
Quoted 4 lines · SiddharthaMishra
Added a job to run clang static code analysis on the master and maint branch
Signed-off-by SiddharthaMishra <[email protected]>---
Why on 'master' and 'maint' and not others? Quite frankly, I find this choice of branches rather odd, as these two branches are not where the real development happens. If we do not want to increase the number of jobs and limit the test only to a single branch, I would probably pick 'next', and if we can afford two, probably 'pu' and 'next'.
Quoted 7 lines · Junio C Hamano
Quoted 4 lines · SiddharthaMishra
Added a job to run clang static code analysis on the master and maint branch
Signed-off-by SiddharthaMishra <[email protected]>---
Quoted 6 lines
Why on 'master' and 'maint' and not others? Quite frankly, I find this choice of branches rather odd, as these two branches are not where the real development happens. If we do not want to increase the number of jobs and limit the test only to a single branch, I would probably pick 'next', and if we can afford two, probably 'pu' and 'next'.
I might have been misinterpreting it, but I did so because that's what it said in the microproject description. Thinking about it, I guess it does makes more sense to work on the other two branches instead. Are there any other glaring issues you see in the code?
Hi,
Quoted 1 line · Siddhartha Mishra
Are there any other glaring issues you see in the code?
The rest looks good to me!
Ciao,
Johannes
Hi,
That looks interesting but I agree with Dscho that we should not limit this to master/maint.
I assume you did run this on TravisCI already? Can you share a link? I assume you did find errors? Can we fix them or are there too many? If there are existing errors, how do we define a "successful" build?
Thanks for working on this,
Lars
Quoted 67 lines · SiddharthaMishra
Added a job to run clang static code analysis on the master and maint branch
Signed-off-by SiddharthaMishra <[email protected]>---
.travis.yml | 17 ++++++++++++++++-ci/run-static-analysis.sh| 9 ++++++++-2 files changed, 24 insertions(+), 2 deletions(-).travis.yml +16
-1@@ -48,7 +48,7 @@ matrix: before_install: before_script: script: ci/run-linux32-docker.sh - - env: jobname=StaticAnalysis + - env: jobname=CocciStaticAnalysis os: linux compiler: addons:@@ -59,6 +59,21 @@ matrix: before_script: script: ci/run-static-analysis.sh after_failure: + - if: branch IN (master, maint) + env: jobname=ClangStaticAnalysis + os: linux + compiler: + add_ons: + apt: + sources: + - ubuntu-toolchain-r-test + - llvm-toolchain-trusty + packages: + - clang + before_install: + before_script: + script: ci/run-static-analysis.sh + after_failure: - env: jobname=Documentation os: linux compiler:ci/run-static-analysis.sh +0
-0@@ -5,6 +5,13 @@. ${0%/*}/lib-travisci.sh
ci/run-static-analysis.sh +8
-1-make coccicheck +case "$jobname" in +ClangStaticAnalysis) + scan-build -analyze-headers --status-bugs make + ;; +CocciStaticAnalysis) + make coccicheck + ;; +esac
save_good_treeSignature · 1 line
2.16.2.248.ge2408a6f7.dirty
Quoted 5 lines · Lars Schneider
Hi,
That looks interesting but I agree with Dscho that we should not limit this to master/maint.
Quoted 7 lines
I assume you did run this on TravisCI already? Can you share a link? I assume you did find errors? Can we fix them or are there too many? If there are existing errors, how do we define a "successful" build?
Thanks for working on this,
Lars
Thanks for the reply,
I assume there will be false positives in the code which we can't fix by making small modifications to the code as recommended in the FAQ
(https://clang-analyzer.llvm.org/faq.html). According to the FAQ, there is no solid mechanism for suppressing a specific warning, so are options are limited. Some of the things which might help reduce the noise are:
1) To add specific tags in our source code to tell the analyzer to ignore the code. This is probably a bad idea since it is intrusive and forces changes to the actual source code which only affect one task.
2) Count the number of bugs in the previous pushed build and fail the build if the number of bugs increases. It doesn't help remove the noise from the error log but it does tell you if you've added more bugs. However if you add a bug and remove one, it'll pass the job and might mislead you into thinking that the code is correct.
3) Write a script to check the diff of the error log from that of the previous pushed build(ignoring the line numbers). I haven't thought about how exactly it would be implemented so I'm not commenting on it.
Is there a better solution that I'm missing or should I try coming up with a script to come up the diff?
Thanks for the time,
Siddhartha
Quoted 80 lines · Lars Schneider
Hi,
That looks interesting but I agree with Dscho that we should not limit this to master/maint.
I assume you did run this on TravisCI already? Can you share a link? I assume you did find errors? Can we fix them or are there too many? If there are existing errors, how do we define a "successful" build?
Thanks for working on this,
LarsQuoted 67 lines · SiddharthaMishra
Added a job to run clang static code analysis on the master and maint branch
Signed-off-by SiddharthaMishra <[email protected]>---
.travis.yml | 17 ++++++++++++++++-ci/run-static-analysis.sh| 9 ++++++++-2 files changed, 24 insertions(+), 2 deletions(-).travis.yml +16
-1@@ -48,7 +48,7 @@ matrix: before_install: before_script: script: ci/run-linux32-docker.sh - - env: jobname=StaticAnalysis + - env: jobname=CocciStaticAnalysis os: linux compiler: addons:@@ -59,6 +59,21 @@ matrix: before_script: script: ci/run-static-analysis.sh after_failure: + - if: branch IN (master, maint) + env: jobname=ClangStaticAnalysis + os: linux + compiler: + add_ons: + apt: + sources: + - ubuntu-toolchain-r-test + - llvm-toolchain-trusty + packages: + - clang + before_install: + before_script: + script: ci/run-static-analysis.sh + after_failure: - env: jobname=Documentation os: linux compiler:ci/run-static-analysis.sh +0
-0@@ -5,6 +5,13 @@. ${0%/*}/lib-travisci.sh
ci/run-static-analysis.sh +8
-1-make coccicheck +case "$jobname" in +ClangStaticAnalysis) + scan-build -analyze-headers --status-bugs make + ;; +CocciStaticAnalysis) + make coccicheck + ;; +esac
save_good_tree
--
2.16.2.248.ge2408a6f7.dirty
I'm extremely sorry, I forgot to attach my travis-ci job build with my previous message. Here it is :
https://travis-ci.org/SiddharthaMishra/git/jobs/349478746#L1204.
I also apologize for not trimming the email properly in the previous message.
Sorry for the inconvenience,
Siddhartha
Quoted 37 lines · Siddhartha Mishra
Quoted 5 lines · Lars Schneider
Hi,
That looks interesting but I agree with Dscho that we should not limit this to master/maint.
Quoted 7 lines
I assume you did run this on TravisCI already? Can you share a link? I assume you did find errors? Can we fix them or are there too many? If there are existing errors, how do we define a "successful" build?
Thanks for working on this,
LarsThanks for the reply,
I assume there will be false positives in the code which we can't fix by making small modifications to the code as recommended in the FAQ
(https://clang-analyzer.llvm.org/faq.html). According to the FAQ, there is no solid mechanism for suppressing a specific warning, so are options are limited. Some of the things which might help reduce the noise are:1) To add specific tags in our source code to tell the analyzer to ignore the code. This is probably a bad idea since it is intrusive and forces changes to the actual source code which only affect one task.
2) Count the number of bugs in the previous pushed build and fail the build if the number of bugs increases. It doesn't help remove the noise from the error log but it does tell you if you've added more bugs. However if you add a bug and remove one, it'll pass the job and might mislead you into thinking that the code is correct.
Quoted 6 lines
3) Write a script to check the diff of the error log from that of the previous pushed build(ignoring the line numbers). I haven't thought about how exactly it would be implemented so I'm not commenting on it.
Is there a better solution that I'm missing or should I try coming up with a script to come up the diff?
That's a good summary and I don't see a better solution. While (3) sounds nice, I think (2) is the fastest/most pragmatic solution.
We already use the Travis cache [1]. You could use that mechanism to store a file with the latest number of bugs in the cache directory $HOME/travis-cache
If the "number of bugs" file does not exist, then create it and don't complain. If the file exists and the previous number of bugs is higher or equal, then don't complain either. If the file exists and the previous number of bugs is lower, then let the build fail.
Do you think that could work?
Cheers,
Lars
[1] https://docs.travis-ci.com/user/caching/
Quoted 86 lines
Thanks for the time,
SiddharthaQuoted 80 lines · Lars Schneider
Hi,
That looks interesting but I agree with Dscho that we should not limit this to master/maint.
I assume you did run this on TravisCI already? Can you share a link? I assume you did find errors? Can we fix them or are there too many? If there are existing errors, how do we define a "successful" build?
Thanks for working on this,
LarsQuoted 67 lines · SiddharthaMishra
Added a job to run clang static code analysis on the master and maint branch
Signed-off-by SiddharthaMishra <[email protected]>---
.travis.yml | 17 ++++++++++++++++-ci/run-static-analysis.sh| 9 ++++++++-2 files changed, 24 insertions(+), 2 deletions(-).travis.yml +16
-1@@ -48,7 +48,7 @@ matrix: before_install: before_script: script: ci/run-linux32-docker.sh - - env: jobname=StaticAnalysis + - env: jobname=CocciStaticAnalysis os: linux compiler: addons:@@ -59,6 +59,21 @@ matrix: before_script: script: ci/run-static-analysis.sh after_failure: + - if: branch IN (master, maint) + env: jobname=ClangStaticAnalysis + os: linux + compiler: + add_ons: + apt: + sources: + - ubuntu-toolchain-r-test + - llvm-toolchain-trusty + packages: + - clang + before_install: + before_script: + script: ci/run-static-analysis.sh + after_failure: - env: jobname=Documentation os: linux compiler:ci/run-static-analysis.sh +0
-0@@ -5,6 +5,13 @@. ${0%/*}/lib-travisci.sh
ci/run-static-analysis.sh +8
-1-make coccicheck +case "$jobname" in +ClangStaticAnalysis) + scan-build -analyze-headers --status-bugs make + ;; +CocciStaticAnalysis) + make coccicheck + ;; +esac
save_good_tree
--
2.16.2.248.ge2408a6f7.dirty
Quoted 8 lines · Lars Schneider
That's a good summary and I don't see a better solution. While (3) sounds nice, I think (2) is the fastest/most pragmatic solution.
We already use the Travis cache [1]. You could use that mechanism to store a file with the latest number of bugs in the cache directory $HOME/travis-cache
Quoted 6 lines
If the "number of bugs" file does not exist, then create it and don't complain. If the file exists and the previous number of bugs is higher or equal, then don't complain either. If the file exists and the previous number of bugs is lower, then let the build fail.
Do you think that could work?
Apologies for having sent this message twice, the last one wasn't in plain text and were therefore not properly forwarded to the mail group.
If we only consider the number of bugs in the last pushed commit, a situation might happen that a commit creates a lot of bugs. If a subsequent commit then fixes only a few of the previously created bugs the build would still pass since the number of bugs from the last commit has decreased.
Should this be the intended behaviour or should adding the number of allowed bugs be a more deliberate process?
Thanks,
Siddhartha