Fix cve-check (for recipe sysroots)

Recipe sysroots broke cve-check in several places, this patch set should get it running again.

The CA cert fix is a workaround really: Native libcurl is broken and looks for CA cert bundle in the wrong place.

Note that the NVD CVE database is flaky: I have serious problems getting populate_cve_db to succeed during mornings in Europe as the xml files and their metadata does not match for hours. I've reported this to NVD.

I mentioned error output improvements in email but did not implement as that requires more upstream changes: I'll talk to the maintainer about them.

Jussi

The following changes since commit e758547db9048d4aa1c1415d6af8072f519fae24:

nss: Fix nss-native so the checksum doesn't change with BUILD_ARCH (2017-02-09 10:52:03 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib jku/cve-check
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=jku/cve-check

Jussi Kukkonen (3):
cve-check.bbclass: Fix dependencies
cve-check-tool: Fixes for recipe sysroots
cve-check-tool: Use CA cert bundle in correct sysroot

 meta/classes/cve-check.bbclass                     |   2 +-
 .../cve-check-tool/cve-check-tool_5.6.4.bb         |   7 +-
 ...ow-overriding-default-CA-certificate-file.patch | 215 +++++++++++++++++++++
 3 files changed, 221 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-curl-allow-overriding-default-CA-certificate-file.patch

2.1.4

With recipe-specific sysroots the cve_check task must depend on cve-check-tool-native:do_populate_sysroot to get the cve-check-tool binary into the recipe sysroot.

A normal DEPENDS isn't used to avoid cyclic dependencies.

---
 meta/classes/cve-check.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index aad0573..0e4294f 100644
@@ -62,7 +62,7 @@ python do_cve_check () {
 }
 
 addtask cve_check after do_unpack before do_build
-do_cve_check[depends] = "cve-check-tool-native:do_populate_cve_db"
+do_cve_check[depends] = "cve-check-tool-native:do_populate_sysroot cve-check-tool-native:do_populate_cve_db"
 do_cve_check[nostamp] = "1"
 
 python cve_check_cleanup () {

2.1.4

* Use --enable-relative-plugins so cve-check-tool looks for loadable modules relative to binary location instead of hard-coding a wrong sysroot location
* do_populate_cve_db() assumes that the binary cve-check-update is in the sysroot. Ensure that this is true by adding a task dependency

---
 meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

index 6e35421..c78af67 100644
@@ -22,7 +22,7 @@ RDEPENDS_${PN} = "ca-certificates"
 
 inherit pkgconfig autotools
 
-EXTRA_OECONF = "--disable-coverage"
+EXTRA_OECONF = "--disable-coverage --enable-relative-plugins"
 CFLAGS_append = " -Wno-error=pedantic"
 
 do_populate_cve_db() {

@@ -50,6 +50,7 @@ do_populate_cve_db() {
 }
 
 addtask populate_cve_db after do_populate_sysroot
+do_populate_cve_db[depends] = "cve-check-tool-native:do_populate_sysroot"
 do_populate_cve_db[nostamp] = "1"
 do_populate_cve_db[progress] = "percent"
 

2.1.4

Native libcurl looks for CA certs in the wrong place by default.
* Add patch that allows overriding the default CA certificate location. Patch is originally from meta-security-isafw.
* Use the new --cacert to set the correct CA bundle path

---
 .../cve-check-tool/cve-check-tool_5.6.4.bb         |   4 +-
 ...ow-overriding-default-CA-certificate-file.patch | 215 +++++++++++++++++++++
 2 files changed, 218 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-curl-allow-overriding-default-CA-certificate-file.patch

index c78af67..fcd3182 100644
@@ -9,6 +9,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=e8c1458438ead3c34974bc0be3a03ed6"
 SRC_URI = "https://github.com/ikeydoherty/${BPN}/releases/download/v${PV}/${BP}.tar.xz \
            file://check-for-malloc_trim-before-using-it.patch \
            file://0001-print-progress-in-percent-when-downloading-CVE-db.patch \
+           file://0001-curl-allow-overriding-default-CA-certificate-file.patch \
           "
 
 SRC_URI[md5sum] = "c5f4247140fc9be3bf41491d31a34155"

@@ -39,7 +40,8 @@ do_populate_cve_db() {
     [ -z "${cve_file}" ] && cve_file="${TMPDIR}/cve_check"
 
     bbdebug 2 "Updating cve-check-tool database located in $cve_dir"
-    if cve-check-update -d "$cve_dir" ; then
+    # --cacert works around curl-native not finding the CA bundle
+    if cve-check-update --cacert ${sysconfdir}/ssl/certs/ca-certificates.crt -d "$cve_dir" ; then
         printf "CVE database was updated on %s UTC\n\n" "$(LANG=C date --utc +'%F %T')" > "$cve_file"
     else
         bbwarn "Error in executing cve-check-update"

2.1.4

I went back to this patch to see how the problem was solved, because I am facing it again elsewhere. Now that I think about it again, I'm starting to wonder which SSL certificates the native tools really should trust. Tools like Python or wget are taken from the host, which means they use the host defaults for SSL. That native tools built with bitbake then try to use ca-certificates-native looks inconsistent to me.

There is https://bugzilla.yoctoproject.org/show_bug.cgi?id=9883 open about some aspect of this, but it doesn't actually address the underlying question about what the right behavior should be. It's based on the assumption that libcurl-native should always use ca- certificates-native.

Thoughts anyone?

Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

I agree it should use ca-certificates-native for all native; it allows for self-signed internal certificates to be added for internal development.

Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

But that's not what bitbake itself uses. Are you saying that bitbake fetchers etc. should also use whatever certificates are configured for ca-certificates-native? That leads to a chicken-and-egg problem.

A solution where custom certificates need to be configured in two different places (system for bitbake, ca-certificates-native for some other tools) sounds sub-optimal to me.

Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

I know and we could override it to a place.

We ended doing it ourselves:
https://github.com/updatehub/meta-updatehub/commit/e9c34faddb6a59d560147798b37ac6a0e3fa57bf

but this is not generic.

Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

== Series Details ==

Series: Fix cve-check (for recipe sysroots)
Revision: 1
URL : https://patchwork.openembedded.org/series/5241/
State : failure

== Summary ==

Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:

* Patch            [3/3] cve-check-tool: Use CA cert bundle in correct sysroot
 Issue             Missing or incorrectly formatted CVE tag in commit message [test_cve_presence_in_commit_message] 
  Suggested fix    Include a "CVE-xxxx-xxxx" tag in the commit message

If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).

---

Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

This is a false positive, your series is fine.

The presence of CVE tags on the commit message area must only be done if the CVE tag is present on the commit subject, not the whole payload.

Isn't it so that the upstream maintainer has abandoned this tool (and the Big Rewrite of it too)? Then we need to find an alternative approach, or roll our own.

Alex

Abandoned may be a strong word, but I'm not sure how much development is being done. I do plan on looking at this and dedicating some actual engineering time towards it soon.

Ross

https://github.com/ikeydoherty/cve-check-tool/commits/master
Last commit on Sep 1 2016.

https://github.com/ikeydoherty/cve-check-tool/commits/rewrite-snapshot
Last commit on Aug 8 2016.

Same story with open issues and pull requests: no activity in months.

Alex