Coverity scan

dpdk-dev Thomas Monjalon 2 participants 4 messages Mar 9, 2020 – Mar 11, 2020

Share & tools

Thomas Monjalon

author

2020-03-09 15:53 UTC #0 lore raw

We have a public Coverity scan triggered by John for the community:
https://scan.coverity.com/projects/dpdk-data-plane-development-kit
Note there is a tool to help with this task:

	http://thyrsus.com/gitweb/?p=coverity-submit.git;a=shortlog;h=refs/tags/1.13

I see two issues with this scan:
- it is run manually
- not all code is scanned currently

Note that we should be able to run one scan per day for free:
https://scan.coverity.com/faq#frequency

With David, we looked at automating the Coverity scan,
with the help of Travis automation:
https://scan.coverity.com/travis_ci
Such automation cannot be configured on the existing Coverity project. I tried to open a new Coverity project connected to our GitHub. I have a very poor confidence in Coverity/Travis/GitHub integration. I will explain below why.

1/ The instructions were wrong. In this command, there are two mistakes:
openssl s_client -connect https://scan.coverity.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | sudo tee -a /etc/ssl/certs/ca-
For the record, a proper a simpler command is:
true | openssl s_client -connect scan.coverity.com:443 | openssl x509 |
sudo tee -a /etc/ssl/certs/ca-certificates.crt

2/ The coverity scan is triggered as a job addon. The rest of the job must be cancelled with this tricky patch:

-script: ./.ci/${TRAVIS_OS_NAME}-build.sh
+script: if [ "${COVERITY_SCAN_BRANCH}" != 1 ] ; then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi

3/ We need only to prepare the source code once per day. But our .travis.yml has many jobs which must be dropped or ignored.

4/ A big encrypted token must be added in the configuration:

	# encrypted COVERITY_SCAN_TOKEN
	- secure: "VgRYG9N5adKkM9/QpPgswn1c+VXS1mFVN0vgdjuC/bDv2x4u...etc..."

5/ The addon is triggered when pushing to a specific branch (adding config for the record):
coverity_scan:
project:
name: "DPDK/dpdk"
notification_email: [email protected]
build_command_prepend: "meson build -Dexamples=all"
build_command: "ninja -C build"
branch_pattern: coverity_scan

6/ This attempt failed with this log (no more information):

	$ export PROJECT_NAME=DPDK/dpdk
	Coverity Scan analysis selected for branch coverity_scan.
	Coverity Scan API access denied. Check $PROJECT_NAME and $COVERITY_SCAN_TOKEN.

So I am giving up with Travis+Coverity.
The only benefit of Travis is to have a central build configuration. So when a driver is enabled in Travis, it would be scanned in Coverity.
Note: Coverity does a build step to prepare the sources.

Now the question: how can we better configure the community Coverity scan? I propose to set it up in our community lab.
Comments? Suggestions?

Aaron Conole

2020-03-11 17:34 UTC #1 lore raw

Quoted 9 lines · Thomas Monjalon

We have a public Coverity scan triggered by John for the community:
https://scan.coverity.com/projects/dpdk-data-plane-development-kit
Note there is a tool to help with this task:
	http://thyrsus.com/gitweb/?p=coverity-submit.git;a=shortlog;h=refs/tags/1.13
I see two issues with this scan:
- it is run manually
- not all code is scanned currently

Quoted 7 lines

Note that we should be able to run one scan per day for free:
https://scan.coverity.com/faq#frequency
With David, we looked at automating the Coverity scan,
with the help of Travis automation:
https://scan.coverity.com/travis_ci
Such automation cannot be configured on the existing Coverity project.

Why not?

Quoted 1 line

I tried to open a new Coverity project connected to our GitHub.

I don't know that it will work. Either you'll need a separate GitHub, or you'll need to use a special branch.

Quoted 2 lines

I have a very poor confidence in Coverity/Travis/GitHub integration. I will explain below why.

Hrrm.. lots of projects use it. And they do just what you prescribe below (skipping jobs/builds when on the coverity branch).

Quoted 8 lines

1/ The instructions were wrong. In this command, there are two mistakes:
openssl s_client -connect https://scan.coverity.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | sudo tee -a /etc/ssl/certs/ca-
For the record, a proper a simpler command is:
true | openssl s_client -connect scan.coverity.com:443 | openssl x509 |
sudo tee -a /etc/ssl/certs/ca-certificates.crt

Okay, that's fixable.

Review

Quoted 5 lines

2/ The coverity scan is triggered as a job addon. The rest of the job must be cancelled with this tricky patch:
(quoted diff) +1 -1
-script: ./.ci/${TRAVIS_OS_NAME}-build.sh
+script: if [ "${COVERITY_SCAN_BRANCH}" != 1 ] ; then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi

More than that, because we probably also want:

if ([[ "${TRAVIS_JOB_NUMBER##*.}" == "1" ]] && [[ "${TRAVIS_BRANCH}" == "coverity_scan" ]]); then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi

That will only do one job (which solves 3/ below)

Quoted 6 lines

3/ We need only to prepare the source code once per day. But our .travis.yml has many jobs which must be dropped or ignored.
4/ A big encrypted token must be added in the configuration:
	# encrypted COVERITY_SCAN_TOKEN
	- secure: "VgRYG9N5adKkM9/QpPgswn1c+VXS1mFVN0vgdjuC/bDv2x4u...etc..."

Why it's a problem?

Quoted 10 lines

5/ The addon is triggered when pushing to a specific branch (adding config for the record):
coverity_scan:
project:
name: "DPDK/dpdk"
notification_email: [email protected]
build_command_prepend: "meson build -Dexamples=all"
build_command: "ninja -C build"
branch_pattern: coverity_scan

Quoted 4 lines

6/ This attempt failed with this log (no more information):

	$ export PROJECT_NAME=DPDK/dpdk
	Coverity Scan analysis selected for branch coverity_scan.
	Coverity Scan API access denied. Check $PROJECT_NAME and $COVERITY_SCAN_TOKEN.

Probably there is an issue with the token + PROJECT_NAME.

Quoted 5 lines

So I am giving up with Travis+Coverity.
The only benefit of Travis is to have a central build configuration. So when a driver is enabled in Travis, it would be scanned in Coverity.
Note: Coverity does a build step to prepare the sources.

I can try to assist with this if you've not completely abandoned the idea.

Quoted 4 lines

Now the question: how can we better configure the community Coverity scan? I propose to set it up in our community lab.
Comments? Suggestions?

Since we do have something working, but it's manual, is there a way to at least make it happen automatically? Maybe some cron job?

Thomas Monjalon

author

2020-03-11 18:02 UTC #2 lore raw

11/03/2020 18:34, Aaron Conole:

Quoted 14 lines

Quoted 12 lines · Thomas Monjalon
We have a public Coverity scan triggered by John for the community:
https://scan.coverity.com/projects/dpdk-data-plane-development-kit
Note there is a tool to help with this task:
	http://thyrsus.com/gitweb/?p=coverity-submit.git;a=shortlog;h=refs/tags/1.13
I see two issues with this scan:
- it is run manually
- not all code is scanned currently
Note that we should be able to run one scan per day for free:
https://scan.coverity.com/faq#frequency

Quoted 6 lines

Quoted 4 lines
With David, we looked at automating the Coverity scan,
with the help of Travis automation:
https://scan.coverity.com/travis_ci
Such automation cannot be configured on the existing Coverity project.
Why not?

Because Coverity does not allow it.
Travis integration is possible only if the project was created with GitHub credentials.

Quoted 10 lines

Quoted 1 line
I tried to open a new Coverity project connected to our GitHub.
I don't know that it will work. Either you'll need a separate GitHub, or you'll need to use a special branch.
Quoted 2 lines
I have a very poor confidence in Coverity/Travis/GitHub integration. I will explain below why.
Hrrm.. lots of projects use it. And they do just what you prescribe below (skipping jobs/builds when on the coverity branch).

Which project is using Travis integration of Coverity?
How do they automatically update the specific branch without conflict?

Quoted 14 lines

Quoted 8 lines
1/ The instructions were wrong. In this command, there are two mistakes:
openssl s_client -connect https://scan.coverity.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | sudo tee -a /etc/ssl/certs/ca-
For the record, a proper a simpler command is:
true | openssl s_client -connect scan.coverity.com:443 | openssl x509 |
sudo tee -a /etc/ssl/certs/ca-certificates.crt
Okay, that's fixable.
Quoted 3 lines
2/ The coverity scan is triggered as a job addon. The rest of the job must be cancelled with this tricky patch:

Quoted 8 lines

Review
Quoted 2 lines
(quoted diff) +1 -1
-script: ./.ci/${TRAVIS_OS_NAME}-build.sh
+script: if [ "${COVERITY_SCAN_BRANCH}" != 1 ] ; then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi
More than that, because we probably also want:
if ([[ "${TRAVIS_JOB_NUMBER##*.}" == "1" ]] && [[ "${TRAVIS_BRANCH}" == "coverity_scan" ]]); then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi
That will only do one job (which solves 3/ below)

OK good

Quoted 8 lines

Quoted 6 lines
3/ We need only to prepare the source code once per day. But our .travis.yml has many jobs which must be dropped or ignored.
4/ A big encrypted token must be added in the configuration:
	# encrypted COVERITY_SCAN_TOKEN
	- secure: "VgRYG9N5adKkM9/QpPgswn1c+VXS1mFVN0vgdjuC/bDv2x4u...etc..."
Why it's a problem?

It's not a problem. I explained all steps in this email, that's why it's here.

Quoted 10 lines

Quoted 10 lines
5/ The addon is triggered when pushing to a specific branch (adding config for the record):
coverity_scan:
project:
name: "DPDK/dpdk"
notification_email: [email protected]
build_command_prepend: "meson build -Dexamples=all"
build_command: "ninja -C build"
branch_pattern: coverity_scan

Quoted 6 lines

Quoted 4 lines
6/ This attempt failed with this log (no more information):
	$ export PROJECT_NAME=DPDK/dpdk
	Coverity Scan analysis selected for branch coverity_scan.
	Coverity Scan API access denied. Check $PROJECT_NAME and $COVERITY_SCAN_TOKEN.
Probably there is an issue with the token + PROJECT_NAME.

Probably. How can I debug it?

Quoted 6 lines

Quoted 4 lines
So I am giving up with Travis+Coverity.
The only benefit of Travis is to have a central build configuration. So when a driver is enabled in Travis, it would be scanned in Coverity.
Note: Coverity does a build step to prepare the sources.
I can try to assist with this if you've not completely abandoned the idea.

OK, feel free to ping me for troubleshooting.

Quoted 6 lines

Quoted 3 lines
Now the question: how can we better configure the community Coverity scan? I propose to set it up in our community lab.
Comments? Suggestions?
Since we do have something working, but it's manual, is there a way to at least make it happen automatically? Maybe some cron job?

Yes a cron job, but where? I proposed a server of the community lab.

Aaron Conole

2020-03-11 18:35 UTC #3 lore raw

Quoted 31 lines · Thomas Monjalon

11/03/2020 18:34, Aaron Conole:
Quoted 14 lines
Quoted 12 lines · Thomas Monjalon
We have a public Coverity scan triggered by John for the community:
https://scan.coverity.com/projects/dpdk-data-plane-development-kit
Note there is a tool to help with this task:
	http://thyrsus.com/gitweb/?p=coverity-submit.git;a=shortlog;h=refs/tags/1.13
I see two issues with this scan:
- it is run manually
- not all code is scanned currently
Note that we should be able to run one scan per day for free:
https://scan.coverity.com/faq#frequency
Quoted 6 lines
Quoted 4 lines
With David, we looked at automating the Coverity scan,
with the help of Travis automation:
https://scan.coverity.com/travis_ci
Such automation cannot be configured on the existing Coverity project.
Why not?
Because Coverity does not allow it.
Travis integration is possible only if the project was created with GitHub credentials.
Quoted 5 lines
Quoted 1 line
I tried to open a new Coverity project connected to our GitHub.
I don't know that it will work. Either you'll need a separate GitHub, or you'll need to use a special branch.

Quoted 8 lines

Quoted 5 lines
Quoted 2 lines
I have a very poor confidence in Coverity/Travis/GitHub integration. I will explain below why.
Hrrm.. lots of projects use it. And they do just what you prescribe below (skipping jobs/builds when on the coverity branch).
Which project is using Travis integration of Coverity?
How do they automatically update the specific branch without conflict?

RabbitMQ-c looked to be doing it:

https://github.com/rabbitmq/rabbitmq-c redirects to:
https://github.com/alanxz/rabbitmq-c/blob/master/.travis.yml

Looks like they haven't done a push in a while, though. :-/

That project also uses coveralls (something I want to integrate the DPDK project with).

Quoted 30 lines

Quoted 20 lines
Quoted 9 lines
1/ The instructions were wrong. In this command, there are two mistakes:
openssl s_client -connect https://scan.coverity.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | sudo tee -a /etc/ssl/certs/ca-
For the record, a proper a simpler command is:
true | openssl s_client -connect scan.coverity.com:443 | openssl x509 |
sudo tee -a /etc/ssl/certs/ca-certificates.crt
Okay, that's fixable.
Review
Quoted 5 lines
2/ The coverity scan is triggered as a job addon. The rest of the job must be cancelled with this tricky patch:
(quoted diff) +1 -1
-script: ./.ci/${TRAVIS_OS_NAME}-build.sh
+script: if [ "${COVERITY_SCAN_BRANCH}" != 1 ] ; then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi
More than that, because we probably also want:
Quoted 4 lines
if ([[ "${TRAVIS_JOB_NUMBER##*.}" == "1" ]] && [[ "${TRAVIS_BRANCH}" == "coverity_scan" ]]); then ./.ci/${TRAVIS_OS_NAME}-build.sh ; fi
That will only do one job (which solves 3/ below)
OK good
Quoted 3 lines
Quoted 3 lines
3/ We need only to prepare the source code once per day. But our .travis.yml has many jobs which must be dropped or ignored.

Quoted 7 lines

Quoted 5 lines
Quoted 3 lines
4/ A big encrypted token must be added in the configuration:
	# encrypted COVERITY_SCAN_TOKEN
	- secure: "VgRYG9N5adKkM9/QpPgswn1c+VXS1mFVN0vgdjuC/bDv2x4u...etc..."
Why it's a problem?
It's not a problem. I explained all steps in this email, that's why it's here.

Ahh, okay.

Quoted 11 lines

Quoted 11 lines
Quoted 11 lines
5/ The addon is triggered when pushing to a specific branch (adding config for the record):
coverity_scan:
project:
name: "DPDK/dpdk"
notification_email: [email protected]
build_command_prepend: "meson build -Dexamples=all"
build_command: "ninja -C build"
branch_pattern: coverity_scan

Quoted 8 lines

Quoted 6 lines
Quoted 4 lines
6/ This attempt failed with this log (no more information):
	$ export PROJECT_NAME=DPDK/dpdk
	Coverity Scan analysis selected for branch coverity_scan.
	Coverity Scan API access denied. Check $PROJECT_NAME and $COVERITY_SCAN_TOKEN.
Probably there is an issue with the token + PROJECT_NAME.
Probably. How can I debug it?

Need to go through the steps at (which you probably already did):

https://docs.travis-ci.com/user/coverity-scan/

And make sure the account that has access to the travis repo also has access to the coverity scan project.

The nice thing on that page is step 8 will have the blocks needed for the security token (that needs to go in env: global: - secure=...) and the project settings.

I will ping you offline, though.

Quoted 9 lines

Quoted 7 lines
Quoted 5 lines
So I am giving up with Travis+Coverity.
The only benefit of Travis is to have a central build configuration. So when a driver is enabled in Travis, it would be scanned in Coverity.
Note: Coverity does a build step to prepare the sources.
I can try to assist with this if you've not completely abandoned the idea.
OK, feel free to ping me for troubleshooting.

Will do.

Quoted 9 lines

Quoted 7 lines
Quoted 4 lines
Now the question: how can we better configure the community Coverity scan? I propose to set it up in our community lab.
Comments? Suggestions?
Since we do have something working, but it's manual, is there a way to at least make it happen automatically? Maybe some cron job?
Yes a cron job, but where? I proposed a server of the community lab.

If it's just a matter of pushing something (or even a few small steps), we can add it to the robot's sunday "master" rebuild. Just need whatever credentials.