Skip to contentSkip to conversation

Patch review

PATCH New Armin Kuster 1 participant 29 messages Jan 13, 2020

lore
Share & tools
lore
#0 lore raw

cover Patch review

Please have review comments back by Wednesday

The following changes since commit 7009d823a0799ce7132bd77329b273a476718c8c:

python: update to 2.7.17 (2019-11-23 21:01:39 -0800)

are available in the Git repository at:

git://git.openembedded.org/openembedded-core-contrib stable/warrior-nut
http://cgit.openembedded.org//log/?h=stable/warrior-nut

Alexander Kanavin (1):
sudo: correct SRC_URI

Anuj Mittal (1):
openssl: set CVE vendor to openssl

Armin Kuster (1):
stress: update SRC_URI

Chen Qi (2):
libxfont2: set CVE_PRODUCT
webkitgtk: set CVE_PRODUCT

Ferry Toth (1):
sudo: Fix fetching sources

Khem Raj (1):
sdk: Install nativesdk locales for all TCLIBC variants

Niko Mauno (1):
cve-check: Switch to NVD CVE JSON feed version 1.1

Oleksandr Kravchuk (1):
popt: fix SRC_URI

Ross Burton (18):
flex: set CVE_PRODUCT to include vendor
git: set CVE vendor to git-scm
subversion: set CVE vendor to Apache
ed: set CVE vendor to avoid false positives
boost: set CVE vendor to Boost
libpam: set CVE_PRODUCT
libsndfile1: whitelist CVE-2018-13419
procps: whitelist CVE-2018-1121
cve-check: ensure all known CVEs are in the report
cve-check: failure to parse versions should be more visible
cve-check: we don't actually need to unpack to check
cve-update-db-native: don't refresh more than once an hour
cve-update-db-native: don't hardcode the database name
cve-update-db-native: add an index on the CVE ID column
cve-update-db-native: clean up proxy handling
cve-check: rewrite look to fix false negatives
cve-check: neaten get_cve_info
cve-check: fetch CVE data once at a time instead of in a single call

Zang Ruochen (1):
libpcap: upgrade 1.9.0 -> 1.9.1

 meta/classes/cve-check.bbclass                | 105 +++++++++---------
 meta/lib/oe/sdk.py                            |   4 -
 ...-add-missing-limits.h-for-musl-syste.patch |  29 -----
 .../{libpcap_1.9.0.bb => libpcap_1.9.1.bb}    |   5 +-
 .../openssl/openssl_1.1.1b.bb                 |   2 +
 .../recipes-core/meta/cve-update-db-native.bb |  46 +++-----
 meta/recipes-devtools/flex/flex_2.6.0.bb      |   3 +
 meta/recipes-devtools/git/git.inc             |   2 +
 .../subversion/subversion_1.11.1.bb           |   2 +
 meta/recipes-extended/ed/ed_1.15.bb           |   2 +
 meta/recipes-extended/pam/libpam_1.3.0.bb     |   2 +
 meta/recipes-extended/procps/procps_3.3.15.bb |   3 +
 meta/recipes-extended/stress/stress_1.0.4.bb  |   2 +-
 meta/recipes-extended/sudo/sudo_1.8.27.bb     |   2 +-
 .../xorg-lib/libxfont2_2.0.3.bb               |   2 +
 .../libsndfile/libsndfile1_1.0.28.bb          |   3 +
 meta/recipes-sato/webkit/webkitgtk_2.22.7.bb  |   2 +
 meta/recipes-support/boost/boost.inc          |   2 +
 meta/recipes-support/popt/popt_1.16.bb        |   2 +-
 19 files changed, 101 insertions(+), 119 deletions(-)
 delete mode 100644 meta/recipes-connectivity/libpcap/libpcap/0001-pcap-usb-linux.c-add-missing-limits.h-for-musl-syste.patch
 rename meta/recipes-connectivity/libpcap/{libpcap_1.9.0.bb => libpcap_1.9.1.bb} (88%)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/28 openssl: set CVE vendor to openssl

From: Anuj Mittal <[email protected]>

Differentiate it from openssl gem for Ruby.

---
 meta/recipes-connectivity/openssl/openssl_1.1.1b.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-connectivity/openssl/openssl_1.1.1b.bb +2 -0
index 9e36df807c..13e6ad4db7 100644
@@ -204,3 +204,5 @@ RREPLACES_openssl-conf = "openssl10-conf"
 RCONFLICTS_openssl-conf = "openssl10-conf"
 
 BBCLASSEXTEND = "native nativesdk"
+
+CVE_PRODUCT = "openssl:openssl"
Signature · 1 line
2.17.1
#2 lore raw

PATCH 2/28 flex: set CVE_PRODUCT to include vendor

In reply to Armin Kuster

From: Ross Burton <[email protected]>

There are many projects called Flex and they have CVEs, so also set the vendor to remove these false positives.

---
 meta/recipes-devtools/flex/flex_2.6.0.bb | 3 +++
 1 file changed, 3 insertions(+)
meta/recipes-devtools/flex/flex_2.6.0.bb +3 -0
index b477cd8c7f..12ce0cb461 100644
@@ -68,3 +68,6 @@ do_install_ptest() {
 	    -e 's/^builddir = \(.*\)/builddir = ./' -e 's/^top_builddir = \(.*\)/top_builddir = ./' \
 	    -i ${D}${PTEST_PATH}/Makefile
 }
+
+# Not Apache Flex, or Adobe Flex, or IBM Flex.
+CVE_PRODUCT = "flex_project:flex"
Signature · 1 line
2.17.1
#3 lore raw

PATCH 3/28 git: set CVE vendor to git-scm

In reply to Armin Kuster

From: Ross Burton <[email protected]>

There's a Jenkins plugin for Git.

---
 meta/recipes-devtools/git/git.inc | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-devtools/git/git.inc +2 -0
index 26a22ac1e8..6e137432f0 100644
@@ -13,6 +13,8 @@ S = "${WORKDIR}/git-${PV}"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=7c0d7ef03a7eb04ce795b0f60e68e7e1"
 
+CVE_PRODUCT = "git-scm:git"
+
 PACKAGECONFIG ??= ""
 PACKAGECONFIG[cvsserver] = ""
 PACKAGECONFIG[svn] = ""
Signature · 1 line
2.17.1
#4 lore raw

PATCH 4/28 subversion: set CVE vendor to Apache

In reply to Armin Kuster

From: Ross Burton <[email protected]>

There's a Jenkins plugin for Subversion.

---
 meta/recipes-devtools/subversion/subversion_1.11.1.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-devtools/subversion/subversion_1.11.1.bb +2 -0
index 8abac7408e..9909461bf7 100644
@@ -18,6 +18,8 @@ SRC_URI[sha256sum] = "9efd2750ca4d72ec903431a24b9c732b6cbb84aad9b7563f59dd96dea5
 
 inherit autotools pkgconfig gettext
 
+CVE_PRODUCT = "apache:subversion"
+
 PACKAGECONFIG ?= ""
 
 PACKAGECONFIG[sasl] = "--with-sasl,--without-sasl,cyrus-sasl"
Signature · 1 line
2.17.1
#5 lore raw

PATCH 5/28 ed: set CVE vendor to avoid false positives

In reply to Armin Kuster

From: Ross Burton <[email protected]>

---
 meta/recipes-extended/ed/ed_1.15.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-extended/ed/ed_1.15.bb +2 -0
index c79310325f..886c3ddcab 100644
@@ -9,6 +9,8 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=0c7051aef9219dc7237f206c5c4179a7 \
 
 SECTION = "base"
 
+CVE_PRODUCT = "gnu:ed"
+
 # LSB states that ed should be in /bin/
 bindir = "${base_bindir}"
 
Signature · 1 line
2.17.1
#6 lore raw

PATCH 6/28 boost: set CVE vendor to Boost

In reply to Armin Kuster

From: Ross Burton <[email protected]>

There's a Boost module for Drupal.

---
 meta/recipes-support/boost/boost.inc | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-support/boost/boost.inc +2 -0
index c2e2cbb352..f385541653 100644
@@ -2,6 +2,8 @@ SUMMARY = "Free peer-reviewed portable C++ source libraries"
 SECTION = "libs"
 DEPENDS = "bjam-native zlib bzip2"
 
+CVE_PRODUCT = "boost:boost"
+
 ARM_INSTRUCTION_SET_armv4 = "arm"
 ARM_INSTRUCTION_SET_armv5 = "arm"
 
Signature · 1 line
2.17.1
#7 lore raw

PATCH 7/28 libpam: set CVE_PRODUCT

In reply to Armin Kuster

From: Ross Burton <[email protected]>

---
 meta/recipes-extended/pam/libpam_1.3.0.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-extended/pam/libpam_1.3.0.bb +2 -0
index c124e3bb02..ad6ac4b701 100644
@@ -164,3 +164,5 @@ CONFFILES_${PN}-runtime += "${sysconfdir}/pam.d/common-password"
 CONFFILES_${PN}-runtime += "${sysconfdir}/pam.d/common-session-noninteractive"
 CONFFILES_${PN}-runtime += "${sysconfdir}/pam.d/common-account"
 CONFFILES_${PN}-runtime += "${sysconfdir}/security/limits.conf"
+
+CVE_PRODUCT = "linux-pam"
Signature · 1 line
2.17.1
#8 lore raw

PATCH 8/28 libxfont2: set CVE_PRODUCT

In reply to Armin Kuster

From: Chen Qi <[email protected]>

---
 meta/recipes-graphics/xorg-lib/libxfont2_2.0.3.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-graphics/xorg-lib/libxfont2_2.0.3.bb +2 -0
index 2b72d6750c..6994d79e89 100644
@@ -20,3 +20,5 @@ SRC_URI[sha256sum] = "0e8ab7fd737ccdfe87e1f02b55f221f0bd4503a1c5f28be4ed6a54586b
 
 PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)}"
 PACKAGECONFIG[ipv6] = "--enable-ipv6,--disable-ipv6,"
+
+CVE_PRODUCT = "libxfont libxfont2"
Signature · 1 line
2.17.1
#9 lore raw

PATCH 9/28 libsndfile1: whitelist CVE-2018-13419

In reply to Armin Kuster

From: Ross Burton <[email protected]>

This is a memory leak that nobody else can replicate and has been rejected by upstream.

---
 meta/recipes-multimedia/libsndfile/libsndfile1_1.0.28.bb | 3 +++
 1 file changed, 3 insertions(+)
meta/recipes-multimedia/libsndfile/libsndfile1_1.0.28.bb +3 -0
index 77393db847..6044bf09c7 100644
@@ -38,3 +38,6 @@ do_configure_prepend_arm() {
 	export ac_cv_sys_file_offset_bits=64
 }
 
+# This can't be replicated and is just a memory leak.
+# https://github.com/erikd/libsndfile/issues/398
+CVE_CHECK_WHITELIST += "CVE-2018-13419"
Signature · 1 line
2.17.1
#10 lore raw

PATCH 10/28 webkitgtk: set CVE_PRODUCT

In reply to Armin Kuster

From: Chen Qi <[email protected]>

---
 meta/recipes-sato/webkit/webkitgtk_2.22.7.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-sato/webkit/webkitgtk_2.22.7.bb +2 -0
index 301bf10cea..26e673cbcb 100644
@@ -30,6 +30,8 @@ inherit cmake pkgconfig gobject-introspection perlnative distro_features_check u
 
 REQUIRED_DISTRO_FEATURES = "x11 opengl"
 
+CVE_PRODUCT = "webkitgtk webkitgtk\+"
+
 DEPENDS = "zlib libsoup-2.4 curl libxml2 cairo libxslt libxt libidn libgcrypt \
            gtk+3 gstreamer1.0 gstreamer1.0-plugins-base flex-native gperf-native sqlite3 \
 	   pango icu bison-native gawk intltool-native libwebp \
Signature · 1 line
2.17.1
#11 lore raw

PATCH 11/28 procps: whitelist CVE-2018-1121

In reply to Armin Kuster

From: Ross Burton <[email protected]>

This CVE is about race conditions in 'ps' which make it unsuitable for security audits. As these race conditions are unavoidable ps shouldn't be used for security auditing, so this isn't a valid CVE.

---
 meta/recipes-extended/procps/procps_3.3.15.bb | 3 +++
 1 file changed, 3 insertions(+)
meta/recipes-extended/procps/procps_3.3.15.bb +3 -0
index 9756db0e7b..a20917b223 100644
@@ -64,3 +64,6 @@ python __anonymous() {
         d.setVarFlag('ALTERNATIVE_LINK_NAME', prog, '%s/%s' % (d.getVar('base_sbindir'), prog))
 }
 
+# 'ps' isn't suitable for use as a security tool so whitelist this CVE.
+# https://bugzilla.redhat.com/show_bug.cgi?id=1575473#c3
+CVE_CHECK_WHITELIST += "CVE-2018-1121"
Signature · 1 line
2.17.1
#12 lore raw

PATCH 12/28 libpcap: upgrade 1.9.0 -> 1.9.1

In reply to Armin Kuster

From: Zang Ruochen <[email protected]>

-libpcap/0001-pcap-usb-linux.c-add-missing-limits.h-for-musl-syste.patch
Removed since this is included in 1.9.1.

[CVE-2018-16301 CVE-2019-15161 CVE-2019-15162 CVE-2019-15163 CVE-2019-15164 CVE-2019-15165]
---
 ...-add-missing-limits.h-for-musl-syste.patch | 29 -------------------
 .../{libpcap_1.9.0.bb => libpcap_1.9.1.bb}    |  5 ++--
 2 files changed, 2 insertions(+), 32 deletions(-)
 delete mode 100644 meta/recipes-connectivity/libpcap/libpcap/0001-pcap-usb-linux.c-add-missing-limits.h-for-musl-syste.patch
 rename meta/recipes-connectivity/libpcap/{libpcap_1.9.0.bb => libpcap_1.9.1.bb} (88%)
meta/recipes-connectivity/libpcap/libpcap/0001-pcap-usb-linux.c-add-missing-limits.h-for-musl-syste.patch deleted +0 -29
deleted file mode 100644
index 01773834c7..0000000000
@@ -1,29 +0,0 @@
-From aafa3512b7b742f5e66a5543e41974cc5e7eebfa Mon Sep 17 00:00:00 2001
-From: maxice8 <[email protected]>
-Date: Sun, 22 Jul 2018 18:54:17 -0300
-Subject: [PATCH] pcap-usb-linux.c: add missing limits.h for musl systems.
-
-fix compilation on musl libc systems like Void Linux and Alpine.
-
-Upstream-Status: Backport [https://github.com/the-tcpdump-group/libpcap/commit/d557c98a16dc254aaff03762b694fe624e180bea]
-
-Signed-off-by: Anuj Mittal <[email protected]>
----
- pcap-usb-linux.c | 1 +
- 1 file changed, 1 insertion(+)
-
-diff --git a/pcap-usb-linux.c b/pcap-usb-linux.c
-index 6f8adf65..b92c05ea 100644
---- a/pcap-usb-linux.c
-+++ b/pcap-usb-linux.c
-@@ -50,6 +50,7 @@
- #include <stdlib.h>
- #include <unistd.h>
- #include <fcntl.h>
-+#include <limits.h>
- #include <string.h>
- #include <dirent.h>
- #include <byteswap.h>
--- 
-2.17.1
-
meta/recipes-connectivity/libpcap/libpcap_1.9.1.bb renamed from meta/recipes-connectivity/libpcap/libpcap_1.9.0.bb +2 -3
similarity index 88%
rename from meta/recipes-connectivity/libpcap/libpcap_1.9.0.bb
rename to meta/recipes-connectivity/libpcap/libpcap_1.9.1.bb
index 78361561e6..a0f66ac0fa 100644
@@ -11,10 +11,9 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=5eb289217c160e2920d2e35bddc36453 \
 DEPENDS = "flex-native bison-native"
 
 SRC_URI = "https://www.tcpdump.org/release/${BP}.tar.gz \
-           file://0001-pcap-usb-linux.c-add-missing-limits.h-for-musl-syste.patch \
            "
-SRC_URI[md5sum] = "dffd65cb14406ab9841f421732eb0f33"
-SRC_URI[sha256sum] = "2edb88808e5913fdaa8e9c1fcaf272e19b2485338742b5074b9fe44d68f37019"
+SRC_URI[md5sum] = "21af603d9a591c7d96a6457021d84e6c"
+SRC_URI[sha256sum] = "635237637c5b619bcceba91900666b64d56ecb7be63f298f601ec786ce087094"
 
 inherit autotools binconfig-disabled pkgconfig bluetooth
 
Signature · 1 line
2.17.1
#13 lore raw

PATCH 13/28 sdk: Install nativesdk locales for all TCLIBC variants

In reply to Armin Kuster

From: Khem Raj <[email protected]>

install_locales() here is actually operating on nativesdk and only glibc is the default library for nativesdk, since thats what most of desktop/server distros use, therefore bailing out based on TCLIBC is not needed here, since nativesdk-glibc would be required for all non-glibc targetting SDKs as well.

Fixes SDK install time error

ERROR: OE-core's config sanity checker detected a potential misconfiguration. Either fix the cause of this error or at your own risk disable the checker (see sanity.conf). Following is the list of potential problems / advisories:
Your system needs to support the en_US.UTF-8 locale.
ERROR: SDK preparation failed

---
 meta/lib/oe/sdk.py | 4 ----
 1 file changed, 4 deletions(-)
meta/lib/oe/sdk.py +0 -4
index b4fbdb799e..d02a274812 100644
@@ -88,10 +88,6 @@ class Sdk(object, metaclass=ABCMeta):
             bb.warn("cannot remove SDK dir: %s" % path)
 
     def install_locales(self, pm):
-        # This is only relevant for glibc
-        if self.d.getVar("TCLIBC") != "glibc":
-            return
-
         linguas = self.d.getVar("SDKIMAGE_LINGUAS")
         if linguas:
             import fnmatch
Signature · 1 line
2.17.1
#14 lore raw

PATCH 14/28 cve-check: ensure all known CVEs are in the report

In reply to Armin Kuster

From: Ross Burton <[email protected]>

CVEs that are whitelisted or were not vulnerable when there are version comparisons were not included in the report, so alter the logic to ensure that all relevant CVEs are in the report for completeness.

(From OE-Core rev: 98256ff05fcfe9d5ccad360582c36eafb577c264)

---
 meta/classes/cve-check.bbclass | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
meta/classes/cve-check.bbclass +7 -2
index c00d2910be..f87bcc9dc6 100644
@@ -208,12 +208,14 @@ def check_cves(d, patched_cves):
 
             if cve in cve_whitelist:
                 bb.note("%s-%s has been whitelisted for %s" % (product, pv, cve))
+                # TODO: this should be in the report as 'whitelisted'
+                patched_cves.add(cve)
             elif cve in patched_cves:
                 bb.note("%s has been patched" % (cve))
             else:
                 to_append = False
                 if (operator_start == '=' and pv == version_start):
-                    cves_unpatched.append(cve)
+                    to_append = True
                 else:
                     if operator_start:
                         try:
@@ -243,8 +245,11 @@ def check_cves(d, patched_cves):
                         to_append = to_append_start or to_append_end
 
                 if to_append:
+                    bb.note("%s-%s is vulnerable to %s" % (product, pv, cve))
                     cves_unpatched.append(cve)
-                bb.debug(2, "%s-%s is not patched for %s" % (product, pv, cve))
+                else:
+                    bb.note("%s-%s is not vulnerable to %s" % (product, pv, cve))
+                    patched_cves.add(cve)
     conn.close()
 
     return (list(patched_cves), cves_unpatched)
Signature · 1 line
2.17.1
#15 lore raw

PATCH 15/28 cve-check: failure to parse versions should be more visible

In reply to Armin Kuster

From: Ross Burton <[email protected]>

---
 meta/classes/cve-check.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/classes/cve-check.bbclass +2 -2
index f87bcc9dc6..1c8b2223a2 100644
@@ -222,7 +222,7 @@ def check_cves(d, patched_cves):
                             to_append_start =  (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start))
                             to_append_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start))
                         except:
-                            bb.note("%s: Failed to compare %s %s %s for %s" %
+                            bb.warn("%s: Failed to compare %s %s %s for %s" %
                                     (product, pv, operator_start, version_start, cve))
                             to_append_start = False
                     else:
@@ -233,7 +233,7 @@ def check_cves(d, patched_cves):
                             to_append_end  = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end))
                             to_append_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end))
                         except:
-                            bb.note("%s: Failed to compare %s %s %s for %s" %
+                            bb.warn("%s: Failed to compare %s %s %s for %s" %
                                     (product, pv, operator_end, version_end, cve))
                             to_append_end = False
                     else:
Signature · 1 line
2.17.1
#16 lore raw

PATCH 16/28 cve-check: we don't actually need to unpack to check

In reply to Armin Kuster

From: Ross Burton <[email protected]>

The patch scanner works with patch files in the layer, not in the workdir, so it doesn't need to unpack.

(From OE-Core rev: 2cba6ada970deb5156e1ba0182f4f372851e3c17)

---
 meta/classes/cve-check.bbclass | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
meta/classes/cve-check.bbclass +1 -2
index 1c8b2223a2..3326944d79 100644
@@ -62,7 +62,7 @@ python do_cve_check () {
 
 }
 
-addtask cve_check after do_unpack before do_build
+addtask cve_check before do_build
 do_cve_check[depends] = "cve-update-db-native:do_populate_cve_db"
 do_cve_check[nostamp] = "1"
 
@@ -70,7 +70,6 @@ python cve_check_cleanup () {
     """
     Delete the file used to gather all the CVE information.
     """
-
     bb.utils.remove(e.data.getVar("CVE_CHECK_TMP_FILE"))
 }
 
Signature · 1 line
2.17.1
#17 lore raw

PATCH 17/28 cve-update-db-native: don't refresh more than once an hour

In reply to Armin Kuster

From: Ross Burton <[email protected]>

We already fetch the yearly CVE metadata and check that for updates before downloading the full data, but we can speed up CVE checking further by only checking the CVE metadata once an hour.

(From OE-Core rev: 50d898fd360c58fe85460517d965f62b7654771a)

---
 meta/recipes-core/meta/cve-update-db-native.bb | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
meta/recipes-core/meta/cve-update-db-native.bb +9 -1
index 2c427a5884..19875a49b1 100644
@@ -31,8 +31,16 @@ python do_populate_cve_db() {
     db_dir = os.path.join(d.getVar("DL_DIR"), 'CVE_CHECK')
     db_file = os.path.join(db_dir, 'nvdcve_1.0.db')
     json_tmpfile = os.path.join(db_dir, 'nvd.json.gz')
-    proxy = d.getVar("https_proxy")
 
+    # Don't refresh the database more than once an hour
+    try:
+        import time
+        if time.time() - os.path.getmtime(db_file) < (60*60):
+            return
+    except OSError:
+        pass
+
+    proxy = d.getVar("https_proxy")
     if proxy:
         # instantiate an opener but do not install it as the global
         # opener unless if we're really sure it's applicable for all
Signature · 1 line
2.17.1
#18 lore raw

PATCH 18/28 cve-update-db-native: don't hardcode the database name

In reply to Armin Kuster

From: Ross Burton <[email protected]>

Don't hardcode the database filename, there's a variable for this in cve-check.bbclass.

(From OE-Core rev: 0d188a9dc4ae64c64cd661e9d9c3841e86f226ab)

---
 meta/recipes-core/meta/cve-update-db-native.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-core/meta/cve-update-db-native.bb +2 -2
index 19875a49b1..c15534de08 100644
@@ -28,8 +28,8 @@ python do_populate_cve_db() {
     BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
     YEAR_START = 2002
 
-    db_dir = os.path.join(d.getVar("DL_DIR"), 'CVE_CHECK')
-    db_file = os.path.join(db_dir, 'nvdcve_1.0.db')
+    db_file = d.getVar("CVE_CHECK_DB_FILE")
+    db_dir = os.path.dirname(db_file)
     json_tmpfile = os.path.join(db_dir, 'nvd.json.gz')
 
     # Don't refresh the database more than once an hour
Signature · 1 line
2.17.1
#19 lore raw

PATCH 19/28 cve-update-db-native: add an index on the CVE ID column

In reply to Armin Kuster

From: Ross Burton <[email protected]>

Create an index on the PRODUCTS table which contains a row for each CPE, drastically increasing the performance of lookups for a specific CVE.

(From OE-Core rev: b4048b05b3a00d85c40d09961f846eadcebd812e)

---
 meta/recipes-core/meta/cve-update-db-native.bb | 3 +++
 1 file changed, 3 insertions(+)
meta/recipes-core/meta/cve-update-db-native.bb +3 -0
index c15534de08..08b18f064f 100644
@@ -120,11 +120,14 @@ python do_populate_cve_db() {
 
 def initialize_db(c):
     c.execute("CREATE TABLE IF NOT EXISTS META (YEAR INTEGER UNIQUE, DATE TEXT)")
+
     c.execute("CREATE TABLE IF NOT EXISTS NVD (ID TEXT UNIQUE, SUMMARY TEXT, \
         SCOREV2 TEXT, SCOREV3 TEXT, MODIFIED INTEGER, VECTOR TEXT)")
+
     c.execute("CREATE TABLE IF NOT EXISTS PRODUCTS (ID TEXT, \
         VENDOR TEXT, PRODUCT TEXT, VERSION_START TEXT, OPERATOR_START TEXT, \
         VERSION_END TEXT, OPERATOR_END TEXT)")
+    c.execute("CREATE INDEX IF NOT EXISTS PRODUCT_ID_IDX on PRODUCTS(ID);")
 
 def parse_node_and_insert(c, node, cveId):
     # Parse children node if needed
Signature · 1 line
2.17.1
#20 lore raw

PATCH 20/28 cve-update-db-native: clean up proxy handling

In reply to Armin Kuster

From: Ross Burton <[email protected]>

urllib handles adding proxy handlers if the proxies are set in the environment, so call bb.utils.export_proxies() to do that and remove the manual setup.

(From OE-Core rev: 6b73004668b3b71c9c38814b79fbb58c893ed434)

---
 .../recipes-core/meta/cve-update-db-native.bb | 31 +++----------------
 1 file changed, 5 insertions(+), 26 deletions(-)
meta/recipes-core/meta/cve-update-db-native.bb +5 -26
index 08b18f064f..db1d69a28e 100644
@@ -21,10 +21,12 @@ python do_populate_cve_db() {
     """
     Update NVD database with json data feed
     """
-
+    import bb.utils
     import sqlite3, urllib, urllib.parse, shutil, gzip
     from datetime import date
 
+    bb.utils.export_proxies(d)
+
     BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
     YEAR_START = 2002
 
@@ -40,16 +42,6 @@ python do_populate_cve_db() {
     except OSError:
         pass
 
-    proxy = d.getVar("https_proxy")
-    if proxy:
-        # instantiate an opener but do not install it as the global
-        # opener unless if we're really sure it's applicable for all
-        # urllib requests
-        proxy_handler = urllib.request.ProxyHandler({'https': proxy})
-        proxy_opener = urllib.request.build_opener(proxy_handler)
-    else:
-        proxy_opener = None
-
     cve_f = open(os.path.join(d.getVar("TMPDIR"), 'cve_check'), 'a')
 
     if not os.path.isdir(db_dir):
@@ -67,15 +59,7 @@ python do_populate_cve_db() {
         json_url = year_url + ".json.gz"
 
         # Retrieve meta last modified date
-
-        response = None
-
-        if proxy_opener:
-            response = proxy_opener.open(meta_url)
-        else:
-            req = urllib.request.Request(meta_url)
-            response = urllib.request.urlopen(req)
-
+        response = urllib.request.urlopen(meta_url)
         if response:
             for l in response.read().decode("utf-8").splitlines():
                 key, value = l.split(":", 1)
@@ -95,12 +79,7 @@ python do_populate_cve_db() {
 
             # Update db with current year json file
             try:
-                if proxy_opener:
-                    response = proxy_opener.open(json_url)
-                else:
-                    req = urllib.request.Request(json_url)
-                    response = urllib.request.urlopen(req)
-
+                response = urllib.request.urlopen(json_url)
                 if response:
                     update_db(c, gzip.decompress(response.read()).decode('utf-8'))
                 c.execute("insert or replace into META values (?, ?)", [year, last_modified])
Signature · 1 line
2.17.1
#21 lore raw

PATCH 21/28 cve-check: rewrite look to fix false negatives

In reply to Armin Kuster

From: Ross Burton <[email protected]>

A previous optimisation was premature and resulted in false-negatives in the report.

Rewrite the checking algorithm to first get the list of potential CVEs by vendor:product, then iterate through every matching CPE for that CVE to determine if the bounds match or not. By doing this in two stages we can know if we've checked every CPE, instead of accidentally breaking out of the scan too early.

(From OE-Core rev: d61aff9e22704ad69df1f7ab0f8784f4e7cc0c69)

---
 meta/classes/cve-check.bbclass | 63 ++++++++++++++++++----------------
 1 file changed, 34 insertions(+), 29 deletions(-)
meta/classes/cve-check.bbclass +34 -29
index 3326944d79..c1cbdbde7b 100644
@@ -165,7 +165,6 @@ def check_cves(d, patched_cves):
     """
     Connect to the NVD database and find unpatched cves.
     """
-    import ast, csv, tempfile, subprocess, io
     from distutils.version import LooseVersion
 
     cves_unpatched = []
@@ -187,68 +186,74 @@ def check_cves(d, patched_cves):
     cve_whitelist = d.getVar("CVE_CHECK_WHITELIST").split()
 
     import sqlite3
-    db_file = d.getVar("CVE_CHECK_DB_FILE")
-    conn = sqlite3.connect(db_file)
+    db_file = d.expand("file:${CVE_CHECK_DB_FILE}?mode=ro")
+    conn = sqlite3.connect(db_file, uri=True)
 
+    # For each of the known product names (e.g. curl has CPEs using curl and libcurl)...
     for product in products:
-        c = conn.cursor()
         if ":" in product:
             vendor, product = product.split(":", 1)
-            c.execute("SELECT * FROM PRODUCTS WHERE PRODUCT IS ? AND VENDOR IS ?", (product, vendor))
         else:
-            c.execute("SELECT * FROM PRODUCTS WHERE PRODUCT IS ?", (product,))
+            vendor = "%"
 
-        for row in c:
-            cve = row[0]
-            version_start = row[3]
-            operator_start = row[4]
-            version_end = row[5]
-            operator_end = row[6]
+        # Find all relevant CVE IDs.
+        for cverow in conn.execute("SELECT DISTINCT ID FROM PRODUCTS WHERE PRODUCT IS ? AND VENDOR LIKE ?", (product, vendor)):
+            cve = cverow[0]
 
             if cve in cve_whitelist:
                 bb.note("%s-%s has been whitelisted for %s" % (product, pv, cve))
                 # TODO: this should be in the report as 'whitelisted'
                 patched_cves.add(cve)
+                continue
             elif cve in patched_cves:
                 bb.note("%s has been patched" % (cve))
-            else:
-                to_append = False
+                continue
+
+            vulnerable = False
+            for row in conn.execute("SELECT * FROM PRODUCTS WHERE ID IS ? AND PRODUCT IS ? AND VENDOR LIKE ?", (cve, product, vendor)):
+                (_, _, _, version_start, operator_start, version_end, operator_end) = row
+                #bb.debug(2, "Evaluating row " + str(row))
+
                 if (operator_start == '=' and pv == version_start):
-                    to_append = True
+                    vulnerable = True
                 else:
                     if operator_start:
                         try:
-                            to_append_start =  (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start))
-                            to_append_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start))
+                            vulnerable_start =  (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start))
+                            vulnerable_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start))
                         except:
                             bb.warn("%s: Failed to compare %s %s %s for %s" %
                                     (product, pv, operator_start, version_start, cve))
-                            to_append_start = False
+                            vulnerable_start = False
                     else:
-                        to_append_start = False
+                        vulnerable_start = False
 
                     if operator_end:
                         try:
-                            to_append_end  = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end))
-                            to_append_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end))
+                            vulnerable_end  = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end))
+                            vulnerable_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end))
                         except:
                             bb.warn("%s: Failed to compare %s %s %s for %s" %
                                     (product, pv, operator_end, version_end, cve))
-                            to_append_end = False
+                            vulnerable_end = False
                     else:
-                        to_append_end = False
+                        vulnerable_end = False
 
                     if operator_start and operator_end:
-                        to_append = to_append_start and to_append_end
+                        vulnerable = vulnerable_start and vulnerable_end
                     else:
-                        to_append = to_append_start or to_append_end
+                        vulnerable = vulnerable_start or vulnerable_end
 
-                if to_append:
+                if vulnerable:
                     bb.note("%s-%s is vulnerable to %s" % (product, pv, cve))
                     cves_unpatched.append(cve)
-                else:
-                    bb.note("%s-%s is not vulnerable to %s" % (product, pv, cve))
-                    patched_cves.add(cve)
+                    break
+
+            if not vulnerable:
+                bb.note("%s-%s is not vulnerable to %s" % (product, pv, cve))
+                # TODO: not patched but not vulnerable
+                patched_cves.add(cve)
+
     conn.close()
 
     return (list(patched_cves), cves_unpatched)
Signature · 1 line
2.17.1
#22 lore raw

PATCH 22/28 cve-check: neaten get_cve_info

In reply to Armin Kuster

From: Ross Burton <[email protected]>

Remove obsolete Python 2 code, and use convenience methods for neatness.

(From OE-Core rev: f19253cc9e70c974a8e21a142086c13d7cde04ff)

---
 meta/classes/cve-check.bbclass | 18 +++++-------------
 1 file changed, 5 insertions(+), 13 deletions(-)
meta/classes/cve-check.bbclass +5 -13
index c1cbdbde7b..e95716d9de 100644
@@ -261,23 +261,15 @@ def check_cves(d, patched_cves):
 def get_cve_info(d, cves):
     """
     Get CVE information from the database.
-
-    Unfortunately the only way to get CVE info is set the output to
-    html (hard to parse) or query directly the database.
     """
 
-    try:
-        import sqlite3
-    except ImportError:
-        from pysqlite2 import dbapi2 as sqlite3
+    import sqlite3
 
     cve_data = {}
-    db_file = d.getVar("CVE_CHECK_DB_FILE")
-    placeholder = ",".join("?" * len(cves))
-    query = "SELECT * FROM NVD WHERE id IN (%s)" % placeholder
-    conn = sqlite3.connect(db_file)
-    cur = conn.cursor()
-    for row in cur.execute(query, tuple(cves)):
+    conn = sqlite3.connect(d.getVar("CVE_CHECK_DB_FILE"))
+    placeholders = ",".join("?" * len(cves))
+    query = "SELECT * FROM NVD WHERE id IN (%s)" % placeholders
+    for row in conn.execute(query, tuple(cves)):
         cve_data[row[0]] = {}
         cve_data[row[0]]["summary"] = row[1]
         cve_data[row[0]]["scorev2"] = row[2]
Signature · 1 line
2.17.1
#23 lore raw

PATCH 23/28 cve-check: fetch CVE data once at a time instead of in a single call

In reply to Armin Kuster

From: Ross Burton <[email protected]>

This code used to construct a single SQL statement that fetched the NVD data for every CVE requested. For recipes such as the kernel where there are over 2000 CVEs to report this can hit the variable count limit and the query fails with "sqlite3.OperationalError: too many SQL variables". The default limit is 999 variables, but some distributions such as Debian set the default to 250000.

As the NVD table has an index on the ID column, whilst requesting the data CVE-by-CVE is five times slower when working with 2000 CVEs the absolute time different is insignificant: 0.05s verses 0.01s on my machine.

(From OE-Core rev: 53d0cc1e9b7190fa66d7ff1c59518f91b0128d99)

---
 meta/classes/cve-check.bbclass | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
meta/classes/cve-check.bbclass +10 -10
index e95716d9de..19ed5548b3 100644
@@ -267,17 +267,17 @@ def get_cve_info(d, cves):
 
     cve_data = {}
     conn = sqlite3.connect(d.getVar("CVE_CHECK_DB_FILE"))
-    placeholders = ",".join("?" * len(cves))
-    query = "SELECT * FROM NVD WHERE id IN (%s)" % placeholders
-    for row in conn.execute(query, tuple(cves)):
-        cve_data[row[0]] = {}
-        cve_data[row[0]]["summary"] = row[1]
-        cve_data[row[0]]["scorev2"] = row[2]
-        cve_data[row[0]]["scorev3"] = row[3]
-        cve_data[row[0]]["modified"] = row[4]
-        cve_data[row[0]]["vector"] = row[5]
-    conn.close()
 
+    for cve in cves:
+        for row in conn.execute("SELECT * FROM NVD WHERE ID IS ?", (cve,)):
+            cve_data[row[0]] = {}
+            cve_data[row[0]]["summary"] = row[1]
+            cve_data[row[0]]["scorev2"] = row[2]
+            cve_data[row[0]]["scorev3"] = row[3]
+            cve_data[row[0]]["modified"] = row[4]
+            cve_data[row[0]]["vector"] = row[5]
+
+    conn.close()
     return cve_data
 
 def cve_write_data(d, patched, unpatched, cve_data):
Signature · 1 line
2.17.1
#24 lore raw

PATCH 24/28 cve-check: Switch to NVD CVE JSON feed version 1.1

In reply to Armin Kuster

From: Niko Mauno <[email protected]>

Switch to recently released version 1.1 of NVD CVE JSON feed, as in
https://nvd.nist.gov/General/News/JSON-1-1-Vulnerability-Feed-Release
it is mentioned that

Due to changes required to support CVSS v3.1 scoring, the JSON vulnerability feeds must be modified. This will require the consumers of this data to update their internal processes. We will be providing the JSON 1.1 schema on the data feeds page and the information below to prepare for this transition.
...
The JSON 1.1 data feeds will be available on September 9th, 2019. At that time the current JSON 1.0 data feeds will no longer available.

This change was tested briefly by issuing 'bitbake core-image-minimal' with 'cve-check.bbclass' inherited via local.conf, and then comparing the content between the resulting two 'DEPLOY_DIR_IMAGE/core-image-minimal-qemux86.cve' files, which did not seem to contain any other change, except total of 167 entries like

CVSS v3 BASE SCORE: 0.0

were replaced with similar 'CVSS v3 BASE SCORE:' entries which had scores that were greater than '0.0' (up to '9.8').

(From OE-Core rev: cc20e4d8ff2f3aa52a2658404af9a0ff358cc323)

(cherry picked from commit c92b8804d6e59b2707332859957f0e6a46db0a73)
---
 meta/classes/cve-check.bbclass                 | 2 +-
 meta/recipes-core/meta/cve-update-db-native.bb | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
meta/classes/cve-check.bbclass +1 -1
index 19ed5548b3..01b3637469 100644
@@ -26,7 +26,7 @@ CVE_PRODUCT ??= "${BPN}"
 CVE_VERSION ??= "${PV}"
 
 CVE_CHECK_DB_DIR ?= "${DL_DIR}/CVE_CHECK"
-CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve_1.0.db"
+CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve_1.1.db"
 
 CVE_CHECK_LOG ?= "${T}/cve.log"
 CVE_CHECK_TMP_FILE ?= "${TMPDIR}/cve_check"
meta/recipes-core/meta/cve-update-db-native.bb +1 -1
index db1d69a28e..575254af40 100644
@@ -27,7 +27,7 @@ python do_populate_cve_db() {
 
     bb.utils.export_proxies(d)
 
-    BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
+    BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-"
     YEAR_START = 2002
 
     db_file = d.getVar("CVE_CHECK_DB_FILE")
Signature · 1 line
2.17.1
#25 lore raw

PATCH 25/28 popt: fix SRC_URI

In reply to Armin Kuster

From: Oleksandr Kravchuk <[email protected]>

rpm5.org has been down for about a year now. Use linuxfromscratch.org as an alternative reliable source instead.

(cherry picked from commit d9224014da9a512b1b8837e4e7a736d465c97be3)
---
 meta/recipes-support/popt/popt_1.16.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-support/popt/popt_1.16.bb +1 -1
index 478288f9bf..27e49c2ca2 100644
@@ -8,7 +8,7 @@ PR = "r3"
 
 DEPENDS = "virtual/libiconv"
 
-SRC_URI = "http://rpm5.org/files/popt/popt-${PV}.tar.gz \
+SRC_URI = "http://anduin.linuxfromscratch.org/BLFS/popt/popt-${PV}.tar.gz \
            file://pkgconfig_fix.patch \
            file://popt_fix_for_automake-1.12.patch \
            file://disable_tests.patch \
Signature · 1 line
2.17.1
#26 lore raw

PATCH 26/28 sudo: correct SRC_URI

In reply to Armin Kuster

From: Alexander Kanavin <[email protected]>

The old URI returns 404, and has an invalid TLS certificate.

(cherry picked from commit 73ff6aba0a53ffc3ee0a5859a3ad4c8021be4de0)
---
 meta/recipes-extended/sudo/sudo_1.8.27.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-extended/sudo/sudo_1.8.27.bb +1 -1
index 7460a5bfed..bbca8395b9 100644
@@ -1,6 +1,6 @@
 require sudo.inc
 
-SRC_URI = "http://ftp.sudo.ws/sudo/dist/sudo-${PV}.tar.gz \
+SRC_URI = "http://www.sudo.ws/sudo/dist/sudo-${PV}.tar.gz \
            ${@bb.utils.contains('DISTRO_FEATURES', 'pam', '${PAM_SRC_URI}', '', d)} \
            file://0001-Include-sys-types.h-for-id_t-definition.patch \
            file://CVE-2019-14287-1.patch \
Signature · 1 line
2.17.1
#27 lore raw

PATCH 27/28 sudo: Fix fetching sources

In reply to Armin Kuster

From: Ferry Toth <[email protected]>

It looks like https://www.sudo.ws/download.html changed certificate and directory structure. This breaks fetching sources.

(cherry picked from commit f02e9f46ce54fed3c7ddfad7d1003a2fb7ba3a67)
---
 meta/recipes-extended/sudo/sudo_1.8.27.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-extended/sudo/sudo_1.8.27.bb +1 -1
index bbca8395b9..43c27bf37f 100644
@@ -1,6 +1,6 @@
 require sudo.inc
 
-SRC_URI = "http://www.sudo.ws/sudo/dist/sudo-${PV}.tar.gz \
+SRC_URI = "https://www.sudo.ws/dist/sudo-${PV}.tar.gz \
            ${@bb.utils.contains('DISTRO_FEATURES', 'pam', '${PAM_SRC_URI}', '', d)} \
            file://0001-Include-sys-types.h-for-id_t-definition.patch \
            file://CVE-2019-14287-1.patch \
Signature · 1 line
2.17.1
#28 lore raw

PATCH 28/28 stress: update SRC_URI

In reply to Armin Kuster

Fixes:
WARNING: stress-1.0.4-r0 do_fetch: Failed to fetch URL http://people.seas.harvard.edu/~apw/stress/stress-1.0.4.tar.gz, attempting MIRRORS if available

---
 meta/recipes-extended/stress/stress_1.0.4.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-extended/stress/stress_1.0.4.bb +1 -1
index e9179d3e19..42046d5bf4 100644
@@ -5,7 +5,7 @@ HOMEPAGE = "http://people.seas.harvard.edu/~apw/stress/"
 LICENSE = "GPLv2"
 LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
 
-SRC_URI = "http://people.seas.harvard.edu/~apw/${BPN}/${BP}.tar.gz \
+SRC_URI = "https://fossies.org/linux/privat/${BP}.tar.gz \
            file://texinfo.patch \
            "
 
Signature · 1 line
2.17.1