Skip to contentSkip to conversation

cve-check: Depends on cve-update-db-native

PATCH In review Pierre Le Magourou 2 participants 4 messages Jul 5, 2019

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/3 cve-check: Depends on cve-update-db-native

From: Pierre Le Magourou <[email protected]>

do_populate_cve_db is a native task.

---
 meta/classes/cve-check.bbclass                                       | 2 +-
 meta/recipes-core/meta/{cve-update-db.bb => cve-update-db-native.bb} | 0
 2 files changed, 1 insertion(+), 1 deletion(-)
 rename meta/recipes-core/meta/{cve-update-db.bb => cve-update-db-native.bb} (100%)
meta/classes/cve-check.bbclass +1 -1
index 81071e3f19..6ffa0c4688 100644
@@ -63,7 +63,7 @@ python do_cve_check () {
 }
 
 addtask cve_check after do_unpack before do_build
-do_cve_check[depends] = "cve-update-db:do_populate_cve_db"
+do_cve_check[depends] = "cve-update-db-native:do_populate_cve_db"
 do_cve_check[nostamp] = "1"
 
 python cve_check_cleanup () {
meta/recipes-core/meta/cve-update-db-native.bb renamed from meta/recipes-core/meta/cve-update-db.bb +0 -0
similarity index 100%
rename from meta/recipes-core/meta/cve-update-db.bb
rename to meta/recipes-core/meta/cve-update-db-native.bb
Signature · 1 line
2.11.0
#1 lore raw

PATCH v2 2/3 cve-update-db: Use NVD CPE data to populate PRODUCTS table

From: Pierre Le Magourou <[email protected]>

Instead of using expanded list of affected versions that is not reliable, use the 'cpe_match' node in the 'configurations' json node.

For cve-check to correctly match affected CVE, the sqlite database need to contain operator_start, operator_end and the corresponding versions fields.

---
 meta/recipes-core/meta/cve-update-db-native.bb | 88 ++++++++++++++++++++++----
 1 file changed, 74 insertions(+), 14 deletions(-)
meta/recipes-core/meta/cve-update-db-native.bb +74 -14
index ae8f1a958b..d658c7bfea 100644
@@ -25,7 +25,7 @@ python do_populate_cve_db() {
     YEAR_START = 2002
 
     db_dir = d.getVar("DL_DIR") + '/CVE_CHECK'
-    db_file = db_dir + '/nvd-json.db'
+    db_file = db_dir + '/nvdcve.db'
     json_tmpfile = db_dir + '/nvd.json.gz'
     proxy = d.getVar("https_proxy")
     cve_f = open(d.getVar("TMPDIR") + '/cve_check', 'a')
@@ -99,9 +99,76 @@ def initialize_db(c):
     c.execute("CREATE TABLE IF NOT EXISTS NVD (ID TEXT UNIQUE, SUMMARY TEXT, \
         SCOREV2 TEXT, SCOREV3 TEXT, MODIFIED INTEGER, VECTOR TEXT)")
     c.execute("CREATE TABLE IF NOT EXISTS PRODUCTS (HASH INTEGER UNIQUE, ID TEXT, \
-        VENDOR TEXT, PRODUCT TEXT, VERSION TEXT, OPERATOR TEXT)")
-    c.execute("CREATE INDEX IF NOT EXISTS PRODUCT_IDX ON PRODUCTS \
-        (PRODUCT, VERSION)")
+        VENDOR TEXT, PRODUCT TEXT, VERSION_START TEXT, OPERATOR_START TEXT, \
+        VERSION_END TEXT, OPERATOR_END TEXT)")
+
+def insert_elt(c, db_values):
+    product_str = db_values[0] + db_values[1] + db_values[2] + db_values[3]
+    hashstr = hash_djb2(product_str)
+    db_values.insert(0, hashstr)
+    query = "insert or replace into PRODUCTS values (?, ?, ?, ?, ?, ?, ?, ?)"
+    c.execute(query, db_values)
+
+def parse_node_and_insert(c, node, cveId):
+    # Parse children node if needed
+    try:
+        for child in node['children']:
+            parse_node_and_insert(c, child, cveId)
+    except:
+        pass
+
+    # Exit if the cpe_match node does not exists
+    try:
+        cpe_match = node['cpe_match']
+    except:
+        return
+
+    for cpe in cpe_match:
+        if not cpe['vulnerable']:
+            return
+        cpe23 = cpe['cpe23Uri'].split(':')
+        vendor = cpe23[3]
+        product = cpe23[4]
+        version = cpe23[5]
+
+        if version != '*':
+            # Version is defined, this is a '=' match
+            db_values = [cveId, vendor, product, version, '=', '', '']
+            insert_elt(c, db_values)
+        else:
+            # Parse start version, end version and operators
+            op_start = ''
+            op_end = ''
+            v_start = ''
+            v_end = ''
+
+            try:
+                if cpe['versionStartIncluding']:
+                    op_start = '>='
+                    v_start = cpe['versionStartIncluding']
+            except:
+                pass
+            try:
+                if cpe['versionStartExcluding']:
+                    op_start = '>'
+                    v_start = cpe['versionStartExcluding']
+            except:
+                pass
+            try:
+                if cpe['versionEndIncluding']:
+                    op_end = '<='
+                    v_end = cpe['versionEndIncluding']
+            except:
+                pass
+            try:
+                if cpe['versionEndExcluding']:
+                    op_end = '<'
+                    v_end = cpe['versionEndExcluding']
+            except:
+                pass
+
+            db_values = [cveId, vendor, product, v_start, op_start, v_end, op_end]
+            insert_elt(c, db_values)
 
 def update_db(c, json_filename):
     import json
@@ -125,16 +192,9 @@ def update_db(c, json_filename):
         c.execute("insert or replace into NVD values (?, ?, ?, ?, ?, ?)",
                 [cveId, cveDesc, cvssv2, cvssv3, date, accessVector])
 
-        for vendor in elt['cve']['affects']['vendor']['vendor_data']:
-            for product in vendor['product']['product_data']:
-                for version in product['version']['version_data']:
-                    product_str = cveId+vendor['vendor_name']+product['product_name']+version['version_value']
-                    hashstr = hash_djb2(product_str)
-                    c.execute("insert or replace into PRODUCTS values (?, ?, ?, ?, ?, ?)",
-                            [ hashstr, cveId, vendor['vendor_name'],
-                                product['product_name'], version['version_value'],
-                                version['version_affected']])
-
+        configurations = elt['configurations']['nodes']
+        for config in configurations:
+            parse_node_and_insert(c, config, cveId)
 
 
 addtask do_populate_cve_db before do_fetch
Signature · 1 line
2.11.0
#2 lore raw

PATCH v2 3/3 cve-check: Update unpatched CVE matching

In reply to Pierre Le Magourou

From: Pierre Le Magourou <[email protected]>

Now that cve-update-db added CPE information to NVD database. We can check for unpatched versions with operators '<', '<=', '>', and '>='.

---
 meta/classes/cve-check.bbclass | 54 +++++++++++++++++++++++++++++++-----------
 1 file changed, 40 insertions(+), 14 deletions(-)
meta/classes/cve-check.bbclass +40 -14
index 6ffa0c4688..ffd624333f 100644
@@ -26,7 +26,7 @@ CVE_PRODUCT ??= "${BPN}"
 CVE_VERSION ??= "${PV}"
 
 CVE_CHECK_DB_DIR ?= "${DL_DIR}/CVE_CHECK"
-CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvd-json.db"
+CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve.db"
 
 CVE_CHECK_LOG ?= "${T}/cve.log"
 CVE_CHECK_TMP_FILE ?= "${TMPDIR}/cve_check"
@@ -189,27 +189,53 @@ def check_cves(d, patched_cves):
     conn = sqlite3.connect(db_file)
     c = conn.cursor()
 
-    query = """SELECT * FROM PRODUCTS WHERE
-               (PRODUCT IS '{0}' AND VERSION = '{1}' AND OPERATOR IS '=') OR
-               (PRODUCT IS '{0}' AND OPERATOR IS '<=');"""
+    query = "SELECT * FROM PRODUCTS WHERE PRODUCT IS '{0}';"
+
     for product in products:
         for row in c.execute(query.format(product, pv)):
             cve = row[1]
-            version = row[4]
-
-            try:
-                discardVersion = LooseVersion(version) < LooseVersion(pv)
-            except:
-                discardVersion = True
+            version_start = row[4]
+            operator_start = row[5]
+            version_end = row[6]
+            operator_end = row[7]
 
             if pv in cve_whitelist.get(cve, []):
                 bb.note("%s-%s has been whitelisted for %s" % (product, pv, cve))
             elif cve in patched_cves:
                 bb.note("%s has been patched" % (cve))
-            elif discardVersion:
-                bb.debug(2, "Do not consider version %s " % (version))
             else:
-                cves_unpatched.append(cve)
+                if (operator_start == '=' and pv == version_start):
+                    cves_unpatched.append(cve)
+                else:
+                    if operator_start:
+                        try:
+                            to_append_start =  (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start))
+                            to_append_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start))
+                        except:
+                            bb.note("%s: Failed to compare %s %s %s for %s" %
+                                    (product, pv, operator_start, version_start, cve))
+                            to_append_start = False
+                    else:
+                        to_append_start = False
+
+                    if operator_end:
+                        try:
+                            to_append_end  = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end))
+                            to_append_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end))
+                        except:
+                            bb.note("%s: Failed to compare %s %s %s for %s" %
+                                    (product, pv, operator_end, version_end, cve))
+                            to_append_end = False
+                    else:
+                        to_append_end = False
+
+                    if operator_start and operator_end:
+                        to_append = to_append_start and to_append_end
+                    else:
+                        to_append = to_append_start or to_append_end
+
+                if to_append:
+                    cves_unpatched.append(cve)
                 bb.debug(2, "%s-%s is not patched for %s" % (product, pv, cve))
     conn.close()
 
@@ -217,7 +243,7 @@ def check_cves(d, patched_cves):
 
 def get_cve_info(d, cves):
     """
-    Get CVE information from the database used by cve-check-tool.
+    Get CVE information from the database.
 
     Unfortunately the only way to get CVE info is set the output to
     html (hard to parse) or query directly the database.
Signature · 1 line
2.11.0
#3 lore raw

✗ patchtest: failure for "[v2] cve-check: Depends on cve..." and 2 more

In reply to Pierre Le Magourou

== Series Details ==

Series: "[v2] cve-check: Depends on cve..." and 2 more
Revision: 1
URL : https://patchwork.openembedded.org/series/18539/
State : failure

== Summary ==

Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:

* Issue Series does not apply on top of target branch [test_series_merge_on_head]
Suggested fix Rebase your series on top of targeted branch
Targeted branch master (currently at a0a96db236)

If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).

---

Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe