Patch review
cover Patch review
Please have review comments back by Wednesday
The following changes since commit 7009d823a0799ce7132bd77329b273a476718c8c:
python: update to 2.7.17 (2019-11-23 21:01:39 -0800)
are available in the Git repository at:
git://git.openembedded.org/openembedded-core-contrib stable/warrior-nut
http://cgit.openembedded.org//log/?h=stable/warrior-nut
Alexander Kanavin (1):
sudo: correct SRC_URI
Anuj Mittal (1):
openssl: set CVE vendor to openssl
Armin Kuster (1):
stress: update SRC_URI
Chen Qi (2):
libxfont2: set CVE_PRODUCT
webkitgtk: set CVE_PRODUCT
Ferry Toth (1):
sudo: Fix fetching sources
Khem Raj (1):
sdk: Install nativesdk locales for all TCLIBC variants
Niko Mauno (1):
cve-check: Switch to NVD CVE JSON feed version 1.1
Oleksandr Kravchuk (1):
popt: fix SRC_URI
Ross Burton (18):
flex: set CVE_PRODUCT to include vendor
git: set CVE vendor to git-scm
subversion: set CVE vendor to Apache
ed: set CVE vendor to avoid false positives
boost: set CVE vendor to Boost
libpam: set CVE_PRODUCT
libsndfile1: whitelist CVE-2018-13419
procps: whitelist CVE-2018-1121
cve-check: ensure all known CVEs are in the report
cve-check: failure to parse versions should be more visible
cve-check: we don't actually need to unpack to check
cve-update-db-native: don't refresh more than once an hour
cve-update-db-native: don't hardcode the database name
cve-update-db-native: add an index on the CVE ID column
cve-update-db-native: clean up proxy handling
cve-check: rewrite look to fix false negatives
cve-check: neaten get_cve_info
cve-check: fetch CVE data once at a time instead of in a single call
Zang Ruochen (1):
libpcap: upgrade 1.9.0 -> 1.9.1
meta/classes/cve-check.bbclass | 105 +++++++++---------
meta/lib/oe/sdk.py | 4 -
...-add-missing-limits.h-for-musl-syste.patch | 29 -----
.../{libpcap_1.9.0.bb => libpcap_1.9.1.bb} | 5 +-
.../openssl/openssl_1.1.1b.bb | 2 +
.../recipes-core/meta/cve-update-db-native.bb | 46 +++-----
meta/recipes-devtools/flex/flex_2.6.0.bb | 3 +
meta/recipes-devtools/git/git.inc | 2 +
.../subversion/subversion_1.11.1.bb | 2 +
meta/recipes-extended/ed/ed_1.15.bb | 2 +
meta/recipes-extended/pam/libpam_1.3.0.bb | 2 +
meta/recipes-extended/procps/procps_3.3.15.bb | 3 +
meta/recipes-extended/stress/stress_1.0.4.bb | 2 +-
meta/recipes-extended/sudo/sudo_1.8.27.bb | 2 +-
.../xorg-lib/libxfont2_2.0.3.bb | 2 +
.../libsndfile/libsndfile1_1.0.28.bb | 3 +
meta/recipes-sato/webkit/webkitgtk_2.22.7.bb | 2 +
meta/recipes-support/boost/boost.inc | 2 +
meta/recipes-support/popt/popt_1.16.bb | 2 +-
19 files changed, 101 insertions(+), 119 deletions(-)
delete mode 100644 meta/recipes-connectivity/libpcap/libpcap/0001-pcap-usb-linux.c-add-missing-limits.h-for-musl-syste.patch
rename meta/recipes-connectivity/libpcap/{libpcap_1.9.0.bb => libpcap_1.9.1.bb} (88%)Signature · 1 line
2.17.1
PATCH 1/28 openssl: set CVE vendor to openssl
From: Anuj Mittal <[email protected]>
Differentiate it from openssl gem for Ruby.
--- meta/recipes-connectivity/openssl/openssl_1.1.1b.bb | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-connectivity/openssl/openssl_1.1.1b.bb +2 -0
@@ -204,3 +204,5 @@ RREPLACES_openssl-conf = "openssl10-conf" RCONFLICTS_openssl-conf = "openssl10-conf" BBCLASSEXTEND = "native nativesdk" + +CVE_PRODUCT = "openssl:openssl"
Signature · 1 line
2.17.1
PATCH 2/28 flex: set CVE_PRODUCT to include vendor
In reply to Armin KusterFrom: Ross Burton <[email protected]>
There are many projects called Flex and they have CVEs, so also set the vendor to remove these false positives.
--- meta/recipes-devtools/flex/flex_2.6.0.bb | 3 +++ 1 file changed, 3 insertions(+)
meta/recipes-devtools/flex/flex_2.6.0.bb +3 -0
@@ -68,3 +68,6 @@ do_install_ptest() { -e 's/^builddir = \(.*\)/builddir = ./' -e 's/^top_builddir = \(.*\)/top_builddir = ./' \ -i ${D}${PTEST_PATH}/Makefile } + +# Not Apache Flex, or Adobe Flex, or IBM Flex. +CVE_PRODUCT = "flex_project:flex"
Signature · 1 line
2.17.1
PATCH 3/28 git: set CVE vendor to git-scm
In reply to Armin KusterFrom: Ross Burton <[email protected]>
There's a Jenkins plugin for Git.
--- meta/recipes-devtools/git/git.inc | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-devtools/git/git.inc +2 -0
@@ -13,6 +13,8 @@ S = "${WORKDIR}/git-${PV}" LIC_FILES_CHKSUM = "file://COPYING;md5=7c0d7ef03a7eb04ce795b0f60e68e7e1" +CVE_PRODUCT = "git-scm:git" + PACKAGECONFIG ??= "" PACKAGECONFIG[cvsserver] = "" PACKAGECONFIG[svn] = ""
Signature · 1 line
2.17.1
PATCH 4/28 subversion: set CVE vendor to Apache
In reply to Armin KusterFrom: Ross Burton <[email protected]>
There's a Jenkins plugin for Subversion.
--- meta/recipes-devtools/subversion/subversion_1.11.1.bb | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-devtools/subversion/subversion_1.11.1.bb +2 -0
@@ -18,6 +18,8 @@ SRC_URI[sha256sum] = "9efd2750ca4d72ec903431a24b9c732b6cbb84aad9b7563f59dd96dea5 inherit autotools pkgconfig gettext +CVE_PRODUCT = "apache:subversion" + PACKAGECONFIG ?= "" PACKAGECONFIG[sasl] = "--with-sasl,--without-sasl,cyrus-sasl"
Signature · 1 line
2.17.1
PATCH 5/28 ed: set CVE vendor to avoid false positives
In reply to Armin KusterFrom: Ross Burton <[email protected]>
--- meta/recipes-extended/ed/ed_1.15.bb | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-extended/ed/ed_1.15.bb +2 -0
@@ -9,6 +9,8 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=0c7051aef9219dc7237f206c5c4179a7 \ SECTION = "base" +CVE_PRODUCT = "gnu:ed" + # LSB states that ed should be in /bin/ bindir = "${base_bindir}"
Signature · 1 line
2.17.1
PATCH 6/28 boost: set CVE vendor to Boost
In reply to Armin KusterFrom: Ross Burton <[email protected]>
There's a Boost module for Drupal.
--- meta/recipes-support/boost/boost.inc | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-support/boost/boost.inc +2 -0
@@ -2,6 +2,8 @@ SUMMARY = "Free peer-reviewed portable C++ source libraries" SECTION = "libs" DEPENDS = "bjam-native zlib bzip2" +CVE_PRODUCT = "boost:boost" + ARM_INSTRUCTION_SET_armv4 = "arm" ARM_INSTRUCTION_SET_armv5 = "arm"
Signature · 1 line
2.17.1
PATCH 7/28 libpam: set CVE_PRODUCT
In reply to Armin KusterFrom: Ross Burton <[email protected]>
--- meta/recipes-extended/pam/libpam_1.3.0.bb | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-extended/pam/libpam_1.3.0.bb +2 -0
@@ -164,3 +164,5 @@ CONFFILES_${PN}-runtime += "${sysconfdir}/pam.d/common-password" CONFFILES_${PN}-runtime += "${sysconfdir}/pam.d/common-session-noninteractive" CONFFILES_${PN}-runtime += "${sysconfdir}/pam.d/common-account" CONFFILES_${PN}-runtime += "${sysconfdir}/security/limits.conf" + +CVE_PRODUCT = "linux-pam"
Signature · 1 line
2.17.1
PATCH 8/28 libxfont2: set CVE_PRODUCT
In reply to Armin KusterFrom: Chen Qi <[email protected]>
--- meta/recipes-graphics/xorg-lib/libxfont2_2.0.3.bb | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-graphics/xorg-lib/libxfont2_2.0.3.bb +2 -0
@@ -20,3 +20,5 @@ SRC_URI[sha256sum] = "0e8ab7fd737ccdfe87e1f02b55f221f0bd4503a1c5f28be4ed6a54586b PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)}" PACKAGECONFIG[ipv6] = "--enable-ipv6,--disable-ipv6," + +CVE_PRODUCT = "libxfont libxfont2"
Signature · 1 line
2.17.1
PATCH 9/28 libsndfile1: whitelist CVE-2018-13419
In reply to Armin KusterFrom: Ross Burton <[email protected]>
This is a memory leak that nobody else can replicate and has been rejected by upstream.
--- meta/recipes-multimedia/libsndfile/libsndfile1_1.0.28.bb | 3 +++ 1 file changed, 3 insertions(+)
meta/recipes-multimedia/libsndfile/libsndfile1_1.0.28.bb +3 -0
@@ -38,3 +38,6 @@ do_configure_prepend_arm() { export ac_cv_sys_file_offset_bits=64 } +# This can't be replicated and is just a memory leak. +# https://github.com/erikd/libsndfile/issues/398 +CVE_CHECK_WHITELIST += "CVE-2018-13419"
Signature · 1 line
2.17.1
PATCH 10/28 webkitgtk: set CVE_PRODUCT
In reply to Armin KusterFrom: Chen Qi <[email protected]>
--- meta/recipes-sato/webkit/webkitgtk_2.22.7.bb | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-sato/webkit/webkitgtk_2.22.7.bb +2 -0
@@ -30,6 +30,8 @@ inherit cmake pkgconfig gobject-introspection perlnative distro_features_check u REQUIRED_DISTRO_FEATURES = "x11 opengl" +CVE_PRODUCT = "webkitgtk webkitgtk\+" + DEPENDS = "zlib libsoup-2.4 curl libxml2 cairo libxslt libxt libidn libgcrypt \ gtk+3 gstreamer1.0 gstreamer1.0-plugins-base flex-native gperf-native sqlite3 \ pango icu bison-native gawk intltool-native libwebp \
Signature · 1 line
2.17.1
PATCH 11/28 procps: whitelist CVE-2018-1121
In reply to Armin KusterFrom: Ross Burton <[email protected]>
This CVE is about race conditions in 'ps' which make it unsuitable for security audits. As these race conditions are unavoidable ps shouldn't be used for security auditing, so this isn't a valid CVE.
--- meta/recipes-extended/procps/procps_3.3.15.bb | 3 +++ 1 file changed, 3 insertions(+)
meta/recipes-extended/procps/procps_3.3.15.bb +3 -0
@@ -64,3 +64,6 @@ python __anonymous() { d.setVarFlag('ALTERNATIVE_LINK_NAME', prog, '%s/%s' % (d.getVar('base_sbindir'), prog)) } +# 'ps' isn't suitable for use as a security tool so whitelist this CVE. +# https://bugzilla.redhat.com/show_bug.cgi?id=1575473#c3 +CVE_CHECK_WHITELIST += "CVE-2018-1121"
Signature · 1 line
2.17.1
PATCH 12/28 libpcap: upgrade 1.9.0 -> 1.9.1
In reply to Armin KusterFrom: Zang Ruochen <[email protected]>
-libpcap/0001-pcap-usb-linux.c-add-missing-limits.h-for-musl-syste.patch
Removed since this is included in 1.9.1.
---
...-add-missing-limits.h-for-musl-syste.patch | 29 -------------------
.../{libpcap_1.9.0.bb => libpcap_1.9.1.bb} | 5 ++--
2 files changed, 2 insertions(+), 32 deletions(-)
delete mode 100644 meta/recipes-connectivity/libpcap/libpcap/0001-pcap-usb-linux.c-add-missing-limits.h-for-musl-syste.patch
rename meta/recipes-connectivity/libpcap/{libpcap_1.9.0.bb => libpcap_1.9.1.bb} (88%)meta/recipes-connectivity/libpcap/libpcap/0001-pcap-usb-linux.c-add-missing-limits.h-for-musl-syste.patch deleted +0 -29
@@ -1,29 +0,0 @@ -From aafa3512b7b742f5e66a5543e41974cc5e7eebfa Mon Sep 17 00:00:00 2001 -From: maxice8 <[email protected]> -Date: Sun, 22 Jul 2018 18:54:17 -0300 -Subject: [PATCH] pcap-usb-linux.c: add missing limits.h for musl systems. - -fix compilation on musl libc systems like Void Linux and Alpine. - -Upstream-Status: Backport [https://github.com/the-tcpdump-group/libpcap/commit/d557c98a16dc254aaff03762b694fe624e180bea] - -Signed-off-by: Anuj Mittal <[email protected]> ---- - pcap-usb-linux.c | 1 + - 1 file changed, 1 insertion(+) - -diff --git a/pcap-usb-linux.c b/pcap-usb-linux.c -index 6f8adf65..b92c05ea 100644 ---- a/pcap-usb-linux.c -+++ b/pcap-usb-linux.c -@@ -50,6 +50,7 @@ - #include <stdlib.h> - #include <unistd.h> - #include <fcntl.h> -+#include <limits.h> - #include <string.h> - #include <dirent.h> - #include <byteswap.h> --- -2.17.1 -
meta/recipes-connectivity/libpcap/libpcap_1.9.1.bb renamed from meta/recipes-connectivity/libpcap/libpcap_1.9.0.bb +2 -3
@@ -11,10 +11,9 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=5eb289217c160e2920d2e35bddc36453 \ DEPENDS = "flex-native bison-native" SRC_URI = "https://www.tcpdump.org/release/${BP}.tar.gz \ - file://0001-pcap-usb-linux.c-add-missing-limits.h-for-musl-syste.patch \ " -SRC_URI[md5sum] = "dffd65cb14406ab9841f421732eb0f33" -SRC_URI[sha256sum] = "2edb88808e5913fdaa8e9c1fcaf272e19b2485338742b5074b9fe44d68f37019" +SRC_URI[md5sum] = "21af603d9a591c7d96a6457021d84e6c" +SRC_URI[sha256sum] = "635237637c5b619bcceba91900666b64d56ecb7be63f298f601ec786ce087094" inherit autotools binconfig-disabled pkgconfig bluetooth
Signature · 1 line
2.17.1
PATCH 13/28 sdk: Install nativesdk locales for all TCLIBC variants
In reply to Armin KusterFrom: Khem Raj <[email protected]>
install_locales() here is actually operating on nativesdk and only glibc is the default library for nativesdk, since thats what most of desktop/server distros use, therefore bailing out based on TCLIBC is not needed here, since nativesdk-glibc would be required for all non-glibc targetting SDKs as well.
Fixes SDK install time error
ERROR: OE-core's config sanity checker detected a potential misconfiguration. Either fix the cause of this error or at your own risk disable the checker (see sanity.conf). Following is the list of potential problems / advisories:
Your system needs to support the en_US.UTF-8 locale.
ERROR: SDK preparation failed
--- meta/lib/oe/sdk.py | 4 ---- 1 file changed, 4 deletions(-)
meta/lib/oe/sdk.py +0 -4
@@ -88,10 +88,6 @@ class Sdk(object, metaclass=ABCMeta): bb.warn("cannot remove SDK dir: %s" % path) def install_locales(self, pm): - # This is only relevant for glibc - if self.d.getVar("TCLIBC") != "glibc": - return - linguas = self.d.getVar("SDKIMAGE_LINGUAS") if linguas: import fnmatch
Signature · 1 line
2.17.1
PATCH 14/28 cve-check: ensure all known CVEs are in the report
In reply to Armin KusterFrom: Ross Burton <[email protected]>
CVEs that are whitelisted or were not vulnerable when there are version comparisons were not included in the report, so alter the logic to ensure that all relevant CVEs are in the report for completeness.
(From OE-Core rev: 98256ff05fcfe9d5ccad360582c36eafb577c264)
--- meta/classes/cve-check.bbclass | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
meta/classes/cve-check.bbclass +7 -2
@@ -208,12 +208,14 @@ def check_cves(d, patched_cves): if cve in cve_whitelist: bb.note("%s-%s has been whitelisted for %s" % (product, pv, cve)) + # TODO: this should be in the report as 'whitelisted' + patched_cves.add(cve) elif cve in patched_cves: bb.note("%s has been patched" % (cve)) else: to_append = False if (operator_start == '=' and pv == version_start): - cves_unpatched.append(cve) + to_append = True else: if operator_start: try:
@@ -243,8 +245,11 @@ def check_cves(d, patched_cves): to_append = to_append_start or to_append_end if to_append: + bb.note("%s-%s is vulnerable to %s" % (product, pv, cve)) cves_unpatched.append(cve) - bb.debug(2, "%s-%s is not patched for %s" % (product, pv, cve)) + else: + bb.note("%s-%s is not vulnerable to %s" % (product, pv, cve)) + patched_cves.add(cve) conn.close() return (list(patched_cves), cves_unpatched)
Signature · 1 line
2.17.1
PATCH 15/28 cve-check: failure to parse versions should be more visible
In reply to Armin KusterFrom: Ross Burton <[email protected]>
--- meta/classes/cve-check.bbclass | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/classes/cve-check.bbclass +2 -2
@@ -222,7 +222,7 @@ def check_cves(d, patched_cves): to_append_start = (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start)) to_append_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start)) except: - bb.note("%s: Failed to compare %s %s %s for %s" % + bb.warn("%s: Failed to compare %s %s %s for %s" % (product, pv, operator_start, version_start, cve)) to_append_start = False else:
@@ -233,7 +233,7 @@ def check_cves(d, patched_cves): to_append_end = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end)) to_append_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end)) except: - bb.note("%s: Failed to compare %s %s %s for %s" % + bb.warn("%s: Failed to compare %s %s %s for %s" % (product, pv, operator_end, version_end, cve)) to_append_end = False else:
Signature · 1 line
2.17.1
PATCH 16/28 cve-check: we don't actually need to unpack to check
In reply to Armin KusterFrom: Ross Burton <[email protected]>
The patch scanner works with patch files in the layer, not in the workdir, so it doesn't need to unpack.
(From OE-Core rev: 2cba6ada970deb5156e1ba0182f4f372851e3c17)
--- meta/classes/cve-check.bbclass | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
meta/classes/cve-check.bbclass +1 -2
@@ -62,7 +62,7 @@ python do_cve_check () { } -addtask cve_check after do_unpack before do_build +addtask cve_check before do_build do_cve_check[depends] = "cve-update-db-native:do_populate_cve_db" do_cve_check[nostamp] = "1"
@@ -70,7 +70,6 @@ python cve_check_cleanup () { """ Delete the file used to gather all the CVE information. """ - bb.utils.remove(e.data.getVar("CVE_CHECK_TMP_FILE")) }
Signature · 1 line
2.17.1
PATCH 17/28 cve-update-db-native: don't refresh more than once an hour
In reply to Armin KusterFrom: Ross Burton <[email protected]>
We already fetch the yearly CVE metadata and check that for updates before downloading the full data, but we can speed up CVE checking further by only checking the CVE metadata once an hour.
(From OE-Core rev: 50d898fd360c58fe85460517d965f62b7654771a)
--- meta/recipes-core/meta/cve-update-db-native.bb | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
meta/recipes-core/meta/cve-update-db-native.bb +9 -1
@@ -31,8 +31,16 @@ python do_populate_cve_db() { db_dir = os.path.join(d.getVar("DL_DIR"), 'CVE_CHECK') db_file = os.path.join(db_dir, 'nvdcve_1.0.db') json_tmpfile = os.path.join(db_dir, 'nvd.json.gz') - proxy = d.getVar("https_proxy") + # Don't refresh the database more than once an hour + try: + import time + if time.time() - os.path.getmtime(db_file) < (60*60): + return + except OSError: + pass + + proxy = d.getVar("https_proxy") if proxy: # instantiate an opener but do not install it as the global # opener unless if we're really sure it's applicable for all
Signature · 1 line
2.17.1
PATCH 18/28 cve-update-db-native: don't hardcode the database name
In reply to Armin KusterFrom: Ross Burton <[email protected]>
Don't hardcode the database filename, there's a variable for this in cve-check.bbclass.
(From OE-Core rev: 0d188a9dc4ae64c64cd661e9d9c3841e86f226ab)
--- meta/recipes-core/meta/cve-update-db-native.bb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-core/meta/cve-update-db-native.bb +2 -2
@@ -28,8 +28,8 @@ python do_populate_cve_db() { BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-" YEAR_START = 2002 - db_dir = os.path.join(d.getVar("DL_DIR"), 'CVE_CHECK') - db_file = os.path.join(db_dir, 'nvdcve_1.0.db') + db_file = d.getVar("CVE_CHECK_DB_FILE") + db_dir = os.path.dirname(db_file) json_tmpfile = os.path.join(db_dir, 'nvd.json.gz') # Don't refresh the database more than once an hour
Signature · 1 line
2.17.1
PATCH 19/28 cve-update-db-native: add an index on the CVE ID column
In reply to Armin KusterFrom: Ross Burton <[email protected]>
Create an index on the PRODUCTS table which contains a row for each CPE, drastically increasing the performance of lookups for a specific CVE.
(From OE-Core rev: b4048b05b3a00d85c40d09961f846eadcebd812e)
--- meta/recipes-core/meta/cve-update-db-native.bb | 3 +++ 1 file changed, 3 insertions(+)
meta/recipes-core/meta/cve-update-db-native.bb +3 -0
@@ -120,11 +120,14 @@ python do_populate_cve_db() { def initialize_db(c): c.execute("CREATE TABLE IF NOT EXISTS META (YEAR INTEGER UNIQUE, DATE TEXT)") + c.execute("CREATE TABLE IF NOT EXISTS NVD (ID TEXT UNIQUE, SUMMARY TEXT, \ SCOREV2 TEXT, SCOREV3 TEXT, MODIFIED INTEGER, VECTOR TEXT)") + c.execute("CREATE TABLE IF NOT EXISTS PRODUCTS (ID TEXT, \ VENDOR TEXT, PRODUCT TEXT, VERSION_START TEXT, OPERATOR_START TEXT, \ VERSION_END TEXT, OPERATOR_END TEXT)") + c.execute("CREATE INDEX IF NOT EXISTS PRODUCT_ID_IDX on PRODUCTS(ID);") def parse_node_and_insert(c, node, cveId): # Parse children node if needed
Signature · 1 line
2.17.1
PATCH 20/28 cve-update-db-native: clean up proxy handling
In reply to Armin KusterFrom: Ross Burton <[email protected]>
urllib handles adding proxy handlers if the proxies are set in the environment, so call bb.utils.export_proxies() to do that and remove the manual setup.
(From OE-Core rev: 6b73004668b3b71c9c38814b79fbb58c893ed434)
--- .../recipes-core/meta/cve-update-db-native.bb | 31 +++---------------- 1 file changed, 5 insertions(+), 26 deletions(-)
meta/recipes-core/meta/cve-update-db-native.bb +5 -26
@@ -21,10 +21,12 @@ python do_populate_cve_db() { """ Update NVD database with json data feed """ - + import bb.utils import sqlite3, urllib, urllib.parse, shutil, gzip from datetime import date + bb.utils.export_proxies(d) + BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-" YEAR_START = 2002
@@ -40,16 +42,6 @@ python do_populate_cve_db() { except OSError: pass - proxy = d.getVar("https_proxy") - if proxy: - # instantiate an opener but do not install it as the global - # opener unless if we're really sure it's applicable for all - # urllib requests - proxy_handler = urllib.request.ProxyHandler({'https': proxy}) - proxy_opener = urllib.request.build_opener(proxy_handler) - else: - proxy_opener = None - cve_f = open(os.path.join(d.getVar("TMPDIR"), 'cve_check'), 'a') if not os.path.isdir(db_dir):
@@ -67,15 +59,7 @@ python do_populate_cve_db() { json_url = year_url + ".json.gz" # Retrieve meta last modified date - - response = None - - if proxy_opener: - response = proxy_opener.open(meta_url) - else: - req = urllib.request.Request(meta_url) - response = urllib.request.urlopen(req) - + response = urllib.request.urlopen(meta_url) if response: for l in response.read().decode("utf-8").splitlines(): key, value = l.split(":", 1)
@@ -95,12 +79,7 @@ python do_populate_cve_db() { # Update db with current year json file try: - if proxy_opener: - response = proxy_opener.open(json_url) - else: - req = urllib.request.Request(json_url) - response = urllib.request.urlopen(req) - + response = urllib.request.urlopen(json_url) if response: update_db(c, gzip.decompress(response.read()).decode('utf-8')) c.execute("insert or replace into META values (?, ?)", [year, last_modified])
Signature · 1 line
2.17.1
PATCH 21/28 cve-check: rewrite look to fix false negatives
In reply to Armin KusterFrom: Ross Burton <[email protected]>
A previous optimisation was premature and resulted in false-negatives in the report.
Rewrite the checking algorithm to first get the list of potential CVEs by vendor:product, then iterate through every matching CPE for that CVE to determine if the bounds match or not. By doing this in two stages we can know if we've checked every CPE, instead of accidentally breaking out of the scan too early.
(From OE-Core rev: d61aff9e22704ad69df1f7ab0f8784f4e7cc0c69)
--- meta/classes/cve-check.bbclass | 63 ++++++++++++++++++---------------- 1 file changed, 34 insertions(+), 29 deletions(-)
meta/classes/cve-check.bbclass +34 -29
@@ -165,7 +165,6 @@ def check_cves(d, patched_cves): """ Connect to the NVD database and find unpatched cves. """ - import ast, csv, tempfile, subprocess, io from distutils.version import LooseVersion cves_unpatched = []
@@ -187,68 +186,74 @@ def check_cves(d, patched_cves): cve_whitelist = d.getVar("CVE_CHECK_WHITELIST").split() import sqlite3 - db_file = d.getVar("CVE_CHECK_DB_FILE") - conn = sqlite3.connect(db_file) + db_file = d.expand("file:${CVE_CHECK_DB_FILE}?mode=ro") + conn = sqlite3.connect(db_file, uri=True) + # For each of the known product names (e.g. curl has CPEs using curl and libcurl)... for product in products: - c = conn.cursor() if ":" in product: vendor, product = product.split(":", 1) - c.execute("SELECT * FROM PRODUCTS WHERE PRODUCT IS ? AND VENDOR IS ?", (product, vendor)) else: - c.execute("SELECT * FROM PRODUCTS WHERE PRODUCT IS ?", (product,)) + vendor = "%" - for row in c: - cve = row[0] - version_start = row[3] - operator_start = row[4] - version_end = row[5] - operator_end = row[6] + # Find all relevant CVE IDs. + for cverow in conn.execute("SELECT DISTINCT ID FROM PRODUCTS WHERE PRODUCT IS ? AND VENDOR LIKE ?", (product, vendor)): + cve = cverow[0] if cve in cve_whitelist: bb.note("%s-%s has been whitelisted for %s" % (product, pv, cve)) # TODO: this should be in the report as 'whitelisted' patched_cves.add(cve) + continue elif cve in patched_cves: bb.note("%s has been patched" % (cve)) - else: - to_append = False + continue + + vulnerable = False + for row in conn.execute("SELECT * FROM PRODUCTS WHERE ID IS ? AND PRODUCT IS ? AND VENDOR LIKE ?", (cve, product, vendor)): + (_, _, _, version_start, operator_start, version_end, operator_end) = row + #bb.debug(2, "Evaluating row " + str(row)) + if (operator_start == '=' and pv == version_start): - to_append = True + vulnerable = True else: if operator_start: try: - to_append_start = (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start)) - to_append_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start)) + vulnerable_start = (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start)) + vulnerable_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start)) except: bb.warn("%s: Failed to compare %s %s %s for %s" % (product, pv, operator_start, version_start, cve)) - to_append_start = False + vulnerable_start = False else: - to_append_start = False + vulnerable_start = False if operator_end: try: - to_append_end = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end)) - to_append_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end)) + vulnerable_end = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end)) + vulnerable_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end)) except: bb.warn("%s: Failed to compare %s %s %s for %s" % (product, pv, operator_end, version_end, cve)) - to_append_end = False + vulnerable_end = False else: - to_append_end = False + vulnerable_end = False if operator_start and operator_end: - to_append = to_append_start and to_append_end + vulnerable = vulnerable_start and vulnerable_end else: - to_append = to_append_start or to_append_end + vulnerable = vulnerable_start or vulnerable_end - if to_append: + if vulnerable: bb.note("%s-%s is vulnerable to %s" % (product, pv, cve)) cves_unpatched.append(cve) - else: - bb.note("%s-%s is not vulnerable to %s" % (product, pv, cve)) - patched_cves.add(cve) + break + + if not vulnerable: + bb.note("%s-%s is not vulnerable to %s" % (product, pv, cve)) + # TODO: not patched but not vulnerable + patched_cves.add(cve) + conn.close() return (list(patched_cves), cves_unpatched)
Signature · 1 line
2.17.1
PATCH 22/28 cve-check: neaten get_cve_info
In reply to Armin KusterFrom: Ross Burton <[email protected]>
Remove obsolete Python 2 code, and use convenience methods for neatness.
(From OE-Core rev: f19253cc9e70c974a8e21a142086c13d7cde04ff)
--- meta/classes/cve-check.bbclass | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-)
meta/classes/cve-check.bbclass +5 -13
@@ -261,23 +261,15 @@ def check_cves(d, patched_cves): def get_cve_info(d, cves): """ Get CVE information from the database. - - Unfortunately the only way to get CVE info is set the output to - html (hard to parse) or query directly the database. """ - try: - import sqlite3 - except ImportError: - from pysqlite2 import dbapi2 as sqlite3 + import sqlite3 cve_data = {} - db_file = d.getVar("CVE_CHECK_DB_FILE") - placeholder = ",".join("?" * len(cves)) - query = "SELECT * FROM NVD WHERE id IN (%s)" % placeholder - conn = sqlite3.connect(db_file) - cur = conn.cursor() - for row in cur.execute(query, tuple(cves)): + conn = sqlite3.connect(d.getVar("CVE_CHECK_DB_FILE")) + placeholders = ",".join("?" * len(cves)) + query = "SELECT * FROM NVD WHERE id IN (%s)" % placeholders + for row in conn.execute(query, tuple(cves)): cve_data[row[0]] = {} cve_data[row[0]]["summary"] = row[1] cve_data[row[0]]["scorev2"] = row[2]
Signature · 1 line
2.17.1
PATCH 23/28 cve-check: fetch CVE data once at a time instead of in a single call
In reply to Armin KusterFrom: Ross Burton <[email protected]>
This code used to construct a single SQL statement that fetched the NVD data for every CVE requested. For recipes such as the kernel where there are over 2000 CVEs to report this can hit the variable count limit and the query fails with "sqlite3.OperationalError: too many SQL variables". The default limit is 999 variables, but some distributions such as Debian set the default to 250000.
As the NVD table has an index on the ID column, whilst requesting the data CVE-by-CVE is five times slower when working with 2000 CVEs the absolute time different is insignificant: 0.05s verses 0.01s on my machine.
(From OE-Core rev: 53d0cc1e9b7190fa66d7ff1c59518f91b0128d99)
--- meta/classes/cve-check.bbclass | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
meta/classes/cve-check.bbclass +10 -10
@@ -267,17 +267,17 @@ def get_cve_info(d, cves): cve_data = {} conn = sqlite3.connect(d.getVar("CVE_CHECK_DB_FILE")) - placeholders = ",".join("?" * len(cves)) - query = "SELECT * FROM NVD WHERE id IN (%s)" % placeholders - for row in conn.execute(query, tuple(cves)): - cve_data[row[0]] = {} - cve_data[row[0]]["summary"] = row[1] - cve_data[row[0]]["scorev2"] = row[2] - cve_data[row[0]]["scorev3"] = row[3] - cve_data[row[0]]["modified"] = row[4] - cve_data[row[0]]["vector"] = row[5] - conn.close() + for cve in cves: + for row in conn.execute("SELECT * FROM NVD WHERE ID IS ?", (cve,)): + cve_data[row[0]] = {} + cve_data[row[0]]["summary"] = row[1] + cve_data[row[0]]["scorev2"] = row[2] + cve_data[row[0]]["scorev3"] = row[3] + cve_data[row[0]]["modified"] = row[4] + cve_data[row[0]]["vector"] = row[5] + + conn.close() return cve_data def cve_write_data(d, patched, unpatched, cve_data):
Signature · 1 line
2.17.1
PATCH 24/28 cve-check: Switch to NVD CVE JSON feed version 1.1
In reply to Armin KusterFrom: Niko Mauno <[email protected]>
Switch to recently released version 1.1 of NVD CVE JSON feed, as in
https://nvd.nist.gov/General/News/JSON-1-1-Vulnerability-Feed-Release
it is mentioned that
Due to changes required to support CVSS v3.1 scoring, the JSON vulnerability feeds must be modified. This will require the consumers of this data to update their internal processes. We will be providing the JSON 1.1 schema on the data feeds page and the information below to prepare for this transition.
...
The JSON 1.1 data feeds will be available on September 9th, 2019. At that time the current JSON 1.0 data feeds will no longer available.
This change was tested briefly by issuing 'bitbake core-image-minimal' with 'cve-check.bbclass' inherited via local.conf, and then comparing the content between the resulting two 'DEPLOY_DIR_IMAGE/core-image-minimal-qemux86.cve' files, which did not seem to contain any other change, except total of 167 entries like
CVSS v3 BASE SCORE: 0.0
were replaced with similar 'CVSS v3 BASE SCORE:' entries which had scores that were greater than '0.0' (up to '9.8').
(From OE-Core rev: cc20e4d8ff2f3aa52a2658404af9a0ff358cc323)
--- meta/classes/cve-check.bbclass | 2 +- meta/recipes-core/meta/cve-update-db-native.bb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
meta/classes/cve-check.bbclass +1 -1
@@ -26,7 +26,7 @@ CVE_PRODUCT ??= "${BPN}" CVE_VERSION ??= "${PV}" CVE_CHECK_DB_DIR ?= "${DL_DIR}/CVE_CHECK" -CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve_1.0.db" +CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve_1.1.db" CVE_CHECK_LOG ?= "${T}/cve.log" CVE_CHECK_TMP_FILE ?= "${TMPDIR}/cve_check"
meta/recipes-core/meta/cve-update-db-native.bb +1 -1
@@ -27,7 +27,7 @@ python do_populate_cve_db() { bb.utils.export_proxies(d) - BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-" + BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-" YEAR_START = 2002 db_file = d.getVar("CVE_CHECK_DB_FILE")
Signature · 1 line
2.17.1
PATCH 25/28 popt: fix SRC_URI
In reply to Armin KusterFrom: Oleksandr Kravchuk <[email protected]>
rpm5.org has been down for about a year now. Use linuxfromscratch.org as an alternative reliable source instead.
--- meta/recipes-support/popt/popt_1.16.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-support/popt/popt_1.16.bb +1 -1
@@ -8,7 +8,7 @@ PR = "r3" DEPENDS = "virtual/libiconv" -SRC_URI = "http://rpm5.org/files/popt/popt-${PV}.tar.gz \ +SRC_URI = "http://anduin.linuxfromscratch.org/BLFS/popt/popt-${PV}.tar.gz \ file://pkgconfig_fix.patch \ file://popt_fix_for_automake-1.12.patch \ file://disable_tests.patch \
Signature · 1 line
2.17.1
PATCH 26/28 sudo: correct SRC_URI
In reply to Armin KusterFrom: Alexander Kanavin <[email protected]>
The old URI returns 404, and has an invalid TLS certificate.
--- meta/recipes-extended/sudo/sudo_1.8.27.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-extended/sudo/sudo_1.8.27.bb +1 -1
@@ -1,6 +1,6 @@ require sudo.inc -SRC_URI = "http://ftp.sudo.ws/sudo/dist/sudo-${PV}.tar.gz \ +SRC_URI = "http://www.sudo.ws/sudo/dist/sudo-${PV}.tar.gz \ ${@bb.utils.contains('DISTRO_FEATURES', 'pam', '${PAM_SRC_URI}', '', d)} \ file://0001-Include-sys-types.h-for-id_t-definition.patch \ file://CVE-2019-14287-1.patch \
Signature · 1 line
2.17.1
PATCH 27/28 sudo: Fix fetching sources
In reply to Armin KusterFrom: Ferry Toth <[email protected]>
It looks like https://www.sudo.ws/download.html changed certificate and directory structure. This breaks fetching sources.
--- meta/recipes-extended/sudo/sudo_1.8.27.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-extended/sudo/sudo_1.8.27.bb +1 -1
@@ -1,6 +1,6 @@ require sudo.inc -SRC_URI = "http://www.sudo.ws/sudo/dist/sudo-${PV}.tar.gz \ +SRC_URI = "https://www.sudo.ws/dist/sudo-${PV}.tar.gz \ ${@bb.utils.contains('DISTRO_FEATURES', 'pam', '${PAM_SRC_URI}', '', d)} \ file://0001-Include-sys-types.h-for-id_t-definition.patch \ file://CVE-2019-14287-1.patch \
Signature · 1 line
2.17.1
PATCH 28/28 stress: update SRC_URI
In reply to Armin KusterFixes:
WARNING: stress-1.0.4-r0 do_fetch: Failed to fetch URL http://people.seas.harvard.edu/~apw/stress/stress-1.0.4.tar.gz, attempting MIRRORS if available
--- meta/recipes-extended/stress/stress_1.0.4.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-extended/stress/stress_1.0.4.bb +1 -1
@@ -5,7 +5,7 @@ HOMEPAGE = "http://people.seas.harvard.edu/~apw/stress/" LICENSE = "GPLv2" LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263" -SRC_URI = "http://people.seas.harvard.edu/~apw/${BPN}/${BP}.tar.gz \ +SRC_URI = "https://fossies.org/linux/privat/${BP}.tar.gz \ file://texinfo.patch \ "
Signature · 1 line
2.17.1