cve-check: Fix errors in log lines
PATCH 1/2 cve-check: Fix errors in log lines
Two warning lines in cve_check.py reference a variable that doesn't exist. These would cause a runtime error if the conditions they are hidden in were to be entered. The log lines have been updated to no longer reference an undefined variable.
--- meta/lib/oe/cve_check.py | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
meta/lib/oe/cve_check.py +10 -3
@@ -253,7 +253,10 @@ def decode_cve_status(d, cve): description = status_split[4].strip() elif len(status_split) >= 2 and status_split[1].strip() == "cpe": # Malformed CPE - bb.warn('Invalid CPE information for CVE_STATUS[%s] = "%s", not setting CPE' % (detail, cve, status)) + bb.warn( + 'Invalid CPE information for CVE_STATUS[%s] = "%s", not setting CPE' + % (cve, status) + ) else: # Other case: no CPE, the syntax is then: # detail: description
@@ -263,9 +266,13 @@ def decode_cve_status(d, cve): status_out["product"] = product status_out["description"] = description - status_mapping = d.getVarFlag("CVE_CHECK_STATUSMAP", status_out['detail']) + detail = status_out["detail"] + status_mapping = d.getVarFlag("CVE_CHECK_STATUSMAP", detail) if status_mapping is None: - bb.warn('Invalid detail "%s" for CVE_STATUS[%s] = "%s", fallback to Unpatched' % (detail, cve, status)) + bb.warn( + 'Invalid detail "%s" for CVE_STATUS[%s] = "%s", fallback to Unpatched' + % (detail, cve, status) + ) status_mapping = "Unpatched" status_out["mapping"] = status_mapping
Signature · 1 line
2.34.1
PATCH v2 1/1 cve-check: Rework patch parsing
The cve_check functionality to parse CVE IDs from the patch filename and patch contents have been reworked to improve parsing and also utilize tests. This ensures that the parsing works as intended.
Additionally, the new patched_cves dict has a few issues I tried to fix as well. If multiple patch files exist for a single CVE ID, only the last one will show up with the "resource" key. The value for the "resource" key has been updated to hold a list and return all patch files associated with a given CVE ID. Also, at the end of get_patch_cves, CVE_STATUS can overwrite an existing entry in the dict. This could cause an issue, for example, if a CVE has been addressed via a patch, but a CVE_STATUS line also exists that ignores the given CVE ID. A warning has been added if this ever happens.
---
I noticed that there are some patches, especially in older verisons of Yocto, where the "CVE: " tag was used with multiple CVE IDs in different formats, like "CVE-YYYY-XXXX & CVE-YYYY-XXXX" or "CVE-YYYY-XXXX, CVE-YYYY-XXXX". Currently, only space-delimited CVE IDs will be parsed, but documentation doesn't indicate that is the only supported format. I figured it'd be nice to update the code to be able to support multiple formats, that way this patch could be backported to fix those patches. I also wanted to add unit tests to ensure the patch parsing behavior is preserved.
I'd also like to update the patch filename parsing to parse multiple CVE IDs from the filename, but based on the comments, it seems like there was a reason why only the last CVE ID is extracted from the filename. I'd be happy to submit a V2 patch or an additional patch to update the function if that sounds good for the maintainers.
V2 Changes:
I mistakenly created this patch without fb3f440 applied. I updated get_patched_cves to return a dict instead of a set.
I also improved the docstrings for these new functions to be more descriptive and also specify the return types.
I also found a few issues with the dictionary object created by get_patched_cves that have now been addressed in this commit.
meta/lib/oe/cve_check.py | 166 ++++++++++++------ meta/lib/oeqa/selftest/cases/cve_check.py | 205 ++++++++++++++++++++++ 2 files changed, 317 insertions(+), 54 deletions(-)
meta/lib/oe/cve_check.py +112 -54
@@ -5,9 +5,11 @@ # import collections -import re -import itertools import functools +import itertools +import os.path +import re +import oe.patch _Version = collections.namedtuple( "_Version", ["release", "patch_l", "pre_l", "pre_v"]
@@ -71,76 +73,132 @@ def _cmpkey(release, patch_l, pre_l, pre_v): return _release, _patch, _pre -def get_patched_cves(d): +def parse_cve_from_filename(patch_filename): """ - Get patches that solve CVEs using the "CVE: " tag. + Parses CVE ID from the filename + + Matches the last "CVE-YYYY-ID" in the file name, also if written + in lowercase. Possible to have multiple CVE IDs in a single + file name, but only the last one will be detected from the file name. + + Returns the last CVE ID foudn in the filename. If no CVE ID is found + an empty string is returned. """ + cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d{4,})", re.IGNORECASE) - import re - import oe.patch + # Check patch file name for CVE ID + fname_match = cve_file_name_match.search(patch_filename) + return fname_match.group(1).upper() if fname_match else "" - cve_match = re.compile(r"CVE:( CVE-\d{4}-\d+)+") - # Matches the last "CVE-YYYY-ID" in the file name, also if written - # in lowercase. Possible to have multiple CVE IDs in a single - # file name, but only the last one will be detected from the file name. - # However, patch files contents addressing multiple CVE IDs are supported - # (cve_match regular expression) - cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d+)", re.IGNORECASE) +def parse_cves_from_patch_contents(patch_contents): + """ + Parses CVE IDs from patch contents + Matches all CVE IDs contained on a line that starts with "CVE: ". Any + delimiter (',', '&', "and", etc.) can be used without any issues. Multiple + "CVE:" lines can also exist. + + Returns a set of all CVE IDs found in the patch contents. + """ + cve_ids = set() + cve_match = re.compile(r"CVE-\d{4}-\d{4,}") + # Search for one or more "CVE: " lines + for line in patch_contents.split("\n"): + if not line.startswith("CVE:"): + continue + cve_ids.update(cve_match.findall(line)) + return cve_ids + + +def parse_cves_from_patch_file(patch_file): + """ + Parses CVE IDs associated with a particular patch file, using both the filename + and patch contents. + + Returns a set of all CVE IDs found in the patch filename and contents. + """ + cve_ids = set() + filename_cve = parse_cve_from_filename(patch_file) + if filename_cve: + bb.debug(2, "Found %s from patch file name %s" % (filename_cve, patch_file)) + cve_ids.add(parse_cve_from_filename(patch_file)) + + # Remote patches won't be present and compressed patches won't be + # unpacked, so say we're not scanning them + if not os.path.isfile(patch_file): + bb.note("%s is remote or compressed, not scanning content" % patch_file) + return cve_ids + + with open(patch_file, "r", encoding="utf-8") as f: + try: + patch_text = f.read() + except UnicodeDecodeError: + bb.debug( + 1, + "Failed to read patch %s using UTF-8 encoding" + " trying with iso8859-1" % patch_file, + ) + f.close() + with open(patch_file, "r", encoding="iso8859-1") as f: + patch_text = f.read() + + cve_ids.update(parse_cves_from_patch_contents(patch_text)) + + if not cve_ids: + bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + else: + bb.debug(2, "Patch %s solves %s" % (patch_file, ", ".join(sorted(cve_ids)))) + + return cve_ids + + +def get_patched_cves(d): + """ + Determines the CVE IDs that have been solved by either patches incuded within + SRC_URI or by setting CVE_STATUS. + + Returns a dictionary with the CVE IDs as keys and an associated dictonary of + relevant metadata as the value. + """ patched_cves = {} patches = oe.patch.src_patches(d) bb.debug(2, "Scanning %d patches for CVEs" % len(patches)) + + # Check each patch file for url in patches: patch_file = bb.fetch.decodeurl(url)[2] - - # Check patch file name for CVE ID - fname_match = cve_file_name_match.search(patch_file) - if fname_match: - cve = fname_match.group(1).upper() - patched_cves[cve] = {"abbrev-status": "Patched", "status": "fix-file-included", "resource": patch_file} - bb.debug(2, "Found %s from patch file name %s" % (cve, patch_file)) - - # Remote patches won't be present and compressed patches won't be - # unpacked, so say we're not scanning them - if not os.path.isfile(patch_file): - bb.note("%s is remote or compressed, not scanning content" % patch_file) - continue - - with open(patch_file, "r", encoding="utf-8") as f: - try: - patch_text = f.read() - except UnicodeDecodeError: - bb.debug(1, "Failed to read patch %s using UTF-8 encoding" - " trying with iso8859-1" % patch_file) - f.close() - with open(patch_file, "r", encoding="iso8859-1") as f: - patch_text = f.read() - - # Search for one or more "CVE: " lines - text_match = False - for match in cve_match.finditer(patch_text): - # Get only the CVEs without the "CVE: " tag - cves = patch_text[match.start()+5:match.end()] - for cve in cves.split(): - bb.debug(2, "Patch %s solves %s" % (patch_file, cve)) - patched_cves[cve] = {"abbrev-status": "Patched", "status": "fix-file-included", "resource": patch_file} - text_match = True - - if not fname_match and not text_match: - bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + for cve_id in parse_cves_from_patch_file(patch_file): + if cve_id not in patched_cves: + { + "abbrev-status": "Patched", + "status": "fix-file-included", + "resource": [patch_file], + } + else: + patched_cves[cve_id]["resource"].append(patch_file) # Search for additional patched CVEs - for cve in (d.getVarFlags("CVE_STATUS") or {}): - decoded_status = decode_cve_status(d, cve) + for cve_id in d.getVarFlags("CVE_STATUS") or {}: + decoded_status = decode_cve_status(d, cve_id) products = d.getVar("CVE_PRODUCT") - if has_cve_product_match(decoded_status, products) == True: - patched_cves[cve] = { + if has_cve_product_match(decoded_status, products): + if cve_id in patched_cves: + bb.warn( + 'CVE_STATUS[%s] = "%s" is overwriting previous status of "%s: %s"' + % ( + cve_id, + d.getVarFlag("CVE_STATUS", cve_id), + patched_cves[cve_id]["abbrev-status"], + patched_cves[cve_id]["status"], + ) + ) + patched_cves[cve_id] = { "abbrev-status": decoded_status["mapping"], "status": decoded_status["detail"], "justification": decoded_status["description"], "affected-vendor": decoded_status["vendor"], - "affected-product": decoded_status["product"] + "affected-product": decoded_status["product"], } return patched_cves
meta/lib/oeqa/selftest/cases/cve_check.py +205 -0
211 lines not on this page · raw message
Signature · 1 line
2.34.1
RE: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
Hello,
This patch has caused a regression in detecting cve patches. Master jumped by 32 CVEs after merging this commit.
These are for sure patched but it's not detected anymore. See https://valkyrie.yocto.io/pub/non-release/patchmetrics/
(master should be identical to scarthgap/styhead except for kernel CVEs)
Reports cannot be used anymore for picking CVEs for TODO list. So this patch should be reverted or fixed.
Peter
Quoted 498 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Colin McAllister via lists.openembedded.org
Sent: Monday, December 30, 2024 20:22
To: [email protected]
Cc: Colin McAllister <[email protected]>
Subject: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsingThe
cve_checkfunctionality to parse CVE IDs from the patch filename and patch contents have been reworked to improve parsing and also utilize tests. This ensures that the parsing works as intended.Additionally, the new
patched_cvesdict has a few issues I tried to fix as well. If multiple patch files exist for a single CVE ID, only the last one will show up with the "resource" key. The value for the "resource" key has been updated to hold a list and return all patch files associated with a given CVE ID. Also, at the end ofget_patch_cves,CVE_STATUScan overwrite an existing entry in the dict. This could cause an issue, for example, if a CVE has been addressed via a patch, but aCVE_STATUSline also exists that ignores the given CVE ID. A warning has been added if this ever happens.Signed-off-by Colin McAllister <[email protected]>---
I noticed that there are some patches, especially in older verisons of Yocto, where the "CVE: " tag was used with multiple CVE IDs in different formats, like "CVE-YYYY-XXXX & CVE-YYYY-XXXX" or "CVE-YYYY-XXXX, CVE-YYYY-XXXX". Currently, only space-delimited CVE IDs will be parsed, but documentation doesn't indicate that is the only supported format. I figured it'd be nice to update the code to be able to support multiple formats, that way this patch could be backported to fix those patches. I also wanted to add unit tests to ensure the patch parsing behavior is preserved.
I'd also like to update the patch filename parsing to parse multiple CVE IDs from the filename, but based on the comments, it seems like there was a reason why only the last CVE ID is extracted from the filename. I'd be happy to submit a V2 patch or an additional patch to update the function if that sounds good for the maintainers.
V2 Changes:
I mistakenly created this patch without fb3f440 applied. I updatedget_patched_cvesto return a dict instead of a set.I also improved the docstrings for these new functions to be more descriptive and also specify the return types.
I also found a few issues with the dictionary object created by
get_patched_cvesthat have now been addressed in this commit.meta/lib/oe/cve_check.py | 166 ++++++++++++------ meta/lib/oeqa/selftest/cases/cve_check.py | 205 ++++++++++++++++++++++ 2 files changed, 317 insertions(+), 54 deletions(-)meta/lib/oe/cve_check.py +39
-13@@ -5,9 +5,11 @@ # import collections -import re -import itertools import functools +import itertools +import os.path +import re +import oe.patch _Version = collections.namedtuple( "_Version", ["release", "patch_l", "pre_l", "pre_v"]@@ -71,76 +73,132 @@ def _cmpkey(release, patch_l, pre_l, pre_v): return _release, _patch, _pre -def get_patched_cves(d): +def parse_cve_from_filename(patch_filename): """ - Get patches that solve CVEs using the "CVE: " tag. + Parses CVE ID from the filename + + Matches the last "CVE-YYYY-ID" in the file name, also if written + in lowercase. Possible to have multiple CVE IDs in a single + file name, but only the last one will be detected from the file name. + + Returns the last CVE ID foudn in the filename. If no CVE ID is found + an empty string is returned. """ + cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d{4,})", re.IGNORECASE) - import re - import oe.patch + # Check patch file name for CVE ID + fname_match = cve_file_name_match.search(patch_filename) + return fname_match.group(1).upper() if fname_match else "" - cve_match = re.compile(r"CVE:( CVE-\d{4}-\d+)+") - # Matches the last "CVE-YYYY-ID" in the file name, also if written - # in lowercase. Possible to have multiple CVE IDs in a single - # file name, but only the last one will be detected from the file name. - # However, patch files contents addressing multiple CVE IDs are supported - # (cve_match regular expression) - cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d+)", re.IGNORECASE) +def parse_cves_from_patch_contents(patch_contents): + """ + Parses CVE IDs from patch contents + Matches all CVE IDs contained on a line that starts with "CVE: ". Any + delimiter (',', '&', "and", etc.) can be used without any issues. Multiple + "CVE:" lines can also exist. + + Returns a set of all CVE IDs found in the patch contents. + """ + cve_ids = set() + cve_match = re.compile(r"CVE-\d{4}-\d{4,}") + # Search for one or more "CVE: " lines + for line in patch_contents.split("\n"): + if not line.startswith("CVE:"): + continue + cve_ids.update(cve_match.findall(line)) + return cve_ids + + +def parse_cves_from_patch_file(patch_file): + """ + Parses CVE IDs associated with a particular patch file, using both thefilename
meta/lib/oe/cve_check.py +8
-0+ and patch contents. + + Returns a set of all CVE IDs found in the patch filename and contents. + """ + cve_ids = set() + filename_cve = parse_cve_from_filename(patch_file) + if filename_cve: + bb.debug(2, "Found %s from patch file name %s" % (filename_cve,
patch_file))meta/lib/oe/cve_check.py +6
-0+ cve_ids.add(parse_cve_from_filename(patch_file)) + + # Remote patches won't be present and compressed patches won't be + # unpacked, so say we're not scanning them + if not os.path.isfile(patch_file): + bb.note("%s is remote or compressed, not scanning content" %
patch_file)meta/lib/oe/cve_check.py +21
-0+ return cve_ids + + with open(patch_file, "r", encoding="utf-8") as f: + try: + patch_text = f.read() + except UnicodeDecodeError: + bb.debug( + 1, + "Failed to read patch %s using UTF-8 encoding" + " trying with iso8859-1" % patch_file, + ) + f.close() + with open(patch_file, "r", encoding="iso8859-1") as f: + patch_text = f.read() + + cve_ids.update(parse_cves_from_patch_contents(patch_text)) + + if not cve_ids: + bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + else: + bb.debug(2, "Patch %s solves %s" % (patch_file, ",".join(sorted(cve_ids))))
meta/lib/oe/cve_check.py +7
-0+ + return cve_ids + + +def get_patched_cves(d): + """ + Determines the CVE IDs that have been solved by either patches incudedwithin
meta/lib/oe/cve_check.py +3
-0+ SRC_URI or by setting CVE_STATUS. + + Returns a dictionary with the CVE IDs as keys and an associated dictonaryof
meta/lib/oe/cve_check.py +4
-6+ relevant metadata as the value. + """ patched_cves = {} patches = oe.patch.src_patches(d) bb.debug(2, "Scanning %d patches for CVEs" % len(patches)) + + # Check each patch file for url in patches: patch_file = bb.fetch.decodeurl(url)[2] - - # Check patch file name for CVE ID - fname_match = cve_file_name_match.search(patch_file) - if fname_match: - cve = fname_match.group(1).upper() - patched_cves[cve] = {"abbrev-status": "Patched", "status": "fix-file-included", "resource":
patch_file}meta/lib/oe/cve_check.py +0
-6- bb.debug(2, "Found %s from patch file name %s" % (cve, patch_file)) - - # Remote patches won't be present and compressed patches won't be - # unpacked, so say we're not scanning them - if not os.path.isfile(patch_file): - bb.note("%s is remote or compressed, not scanning content" %
patch_file)meta/lib/oe/cve_check.py +0
-20- continue - - with open(patch_file, "r", encoding="utf-8") as f: - try: - patch_text = f.read() - except UnicodeDecodeError: - bb.debug(1, "Failed to read patch %s using UTF-8 encoding" - " trying with iso8859-1" % patch_file) - f.close() - with open(patch_file, "r", encoding="iso8859-1") as f: - patch_text = f.read() - - # Search for one or more "CVE: " lines - text_match = False - for match in cve_match.finditer(patch_text): - # Get only the CVEs without the "CVE: " tag - cves = patch_text[match.start()+5:match.end()] - for cve in cves.split(): - bb.debug(2, "Patch %s solves %s" % (patch_file, cve)) - patched_cves[cve] = {"abbrev-status": "Patched", "status": "fix-file-included", "resource":
patch_file}meta/lib/oe/cve_check.py +24
-9- text_match = True - - if not fname_match and not text_match: - bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + for cve_id in parse_cves_from_patch_file(patch_file): + if cve_id not in patched_cves: + { + "abbrev-status": "Patched", + "status": "fix-file-included", + "resource": [patch_file], + } + else: + patched_cves[cve_id]["resource"].append(patch_file) # Search for additional patched CVEs - for cve in (d.getVarFlags("CVE_STATUS") or {}): - decoded_status = decode_cve_status(d, cve) + for cve_id in d.getVarFlags("CVE_STATUS") or {}: + decoded_status = decode_cve_status(d, cve_id) products = d.getVar("CVE_PRODUCT") - if has_cve_product_match(decoded_status, products) == True: - patched_cves[cve] = { + if has_cve_product_match(decoded_status, products): + if cve_id in patched_cves: + bb.warn( + 'CVE_STATUS[%s] = "%s" is overwriting previous status of "%s: %s"' + % ( + cve_id, + d.getVarFlag("CVE_STATUS", cve_id), + patched_cves[cve_id]["abbrev-status"], + patched_cves[cve_id]["status"], + ) + ) + patched_cves[cve_id] = { "abbrev-status": decoded_status["mapping"], "status": decoded_status["detail"], "justification": decoded_status["description"], "affected-vendor": decoded_status["vendor"], - "affected-product": decoded_status["product"] + "affected-product": decoded_status["product"], } return patched_cvesmeta/lib/oeqa/selftest/cases/cve_check.py +9
-0@@ -120,6 +120,211 @@ class CVECheck(OESelftestTestCase): self.assertEqual(has_cve_product_match(status, "test glibca:glibc"), True) self.assertEqual(has_cve_product_match(status, "glibca:glibc test"), True) + def test_parse_cve_from_patch_filename(self): + from oe.cve_check import parse_cve_from_filename + + # Patch filename without CVE ID + self.assertEqual(parse_cve_from_filename("0001-test.patch"), "") + + # Patch with single CVE ID + self.assertEqual( + parse_cve_from_filename("CVE-2022-12345.patch"), "CVE-2022-12345"
meta/lib/oeqa/selftest/cases/cve_check.py +5
-0+ ) + + # Patch with multiple CVE IDs + self.assertEqual( + parse_cve_from_filename("CVE-2022-41741-CVE-2022-41742.patch"),
meta/lib/oeqa/selftest/cases/cve_check.py +6
-0+ "CVE-2022-41742", + ) + + # Patches with CVE ID and appended text + self.assertEqual( + parse_cve_from_filename("CVE-2023-3019-0001.patch"), "CVE-2023-3019"
meta/lib/oeqa/selftest/cases/cve_check.py +3
-0+ ) + self.assertEqual( + parse_cve_from_filename("CVE-2024-21886-1.patch"), "CVE-2024-21886"
meta/lib/oeqa/selftest/cases/cve_check.py +5
-0+ ) + + # Patch with CVE ID and prepended text + self.assertEqual( + parse_cve_from_filename("grep-CVE-2012-5667.patch"), "CVE-2012-5667"
meta/lib/oeqa/selftest/cases/cve_check.py +3
-0+ ) + self.assertEqual( + parse_cve_from_filename("0001-CVE-2012-5667.patch"), "CVE-2012-5667"
meta/lib/oeqa/selftest/cases/cve_check.py +25
-0+ ) + + # Patch with CVE ID and both prepended and appended text + self.assertEqual( + parse_cve_from_filename( + "0001-tpm2_import-fix-fixed-AES-key-CVE-2021-3565-0001.patch" + ), + "CVE-2021-3565", + ) + + # Only grab the last CVE ID in the filename + self.assertEqual( + parse_cve_from_filename("CVE-2012-5667-CVE-2012-5668.patch"), + "CVE-2012-5668", + ) + + # Test invalid CVE ID with incorrect length (must be at least 4 digits) + self.assertEqual( + parse_cve_from_filename("CVE-2024-001.patch"), + "", + ) + + # Test valid CVE ID with very long length + self.assertEqual( + parse_cve_from_filename("CVE-2024-0000000000000000000000001.patch"),
meta/lib/oeqa/selftest/cases/cve_check.py +60
-0+ "CVE-2024-0000000000000000000000001", + ) + + def test_parse_cve_from_patch_contents(self): + import textwrap + from oe.cve_check import parse_cves_from_patch_contents + + # Standard patch file excerpt without any patches + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + remove "*" for root since we don't have a /etc/shadow so far. + + Upstream-Status: Inappropriate [configuration] + + Signed-off-by: Scott Garman <[email protected]> + + --- base-passwd/passwd.master~nobash + +++ base-passwd/passwd.master + @@ -1,4 +1,4 @@ + -root:*:0:0:root:/root:/bin/sh + +root::0:0:root:/root:/bin/sh + daemon:*:1:1:daemon:/usr/sbin:/bin/sh + bin:*:2:2:bin:/bin:/bin/sh + sys:*:3:3:sys:/dev:/bin/sh + """) + ), + set(), + ) + + # Patch file with multiple CVE IDs (space-separated) + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461 CVE-2019-6462 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + """), + ), + {"CVE-2019-6461", "CVE-2019-6462"}, + ) + + # Patch file with multiple CVE IDs (comma-separated w/ both space andno space)
meta/lib/oeqa/selftest/cases/cve_check.py +88
-0+ self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461,CVE-2019-6462, CVE-2019-6463 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + + """), + ), + {"CVE-2019-6461", "CVE-2019-6462", "CVE-2019-6463"}, + ) + + # Patch file with multiple CVE IDs (&-separated) + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461 & CVE-2019-6462 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + """), + ), + {"CVE-2019-6461", "CVE-2019-6462"}, + ) + + # Patch file with multiple lines with CVE IDs + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461 & CVE-2019-6462 + + CVE: CVE-2019-6463 & CVE-2019-6464 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + + """), + ), + {"CVE-2019-6461", "CVE-2019-6462", "CVE-2019-6463", "CVE-2019-6464"},
+ )def test_recipe_report_json(self): config = """--
2.34.1
A message that is not in the archive (sent off-list or not yet received): [email protected]
RE: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
Or should we rather rename all those patches which violate current code? What is then the meaning of CVE: tag within the patch if it's not used but it's enforced? We should probably remove cve_payload_tag handling from test_cve_tag_format...
Peter
Quoted 527 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Peter Marko via lists.openembedded.org
Sent: Friday, January 10, 2025 9:34
To: [email protected]; [email protected]
Subject: Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsingHello,
This patch has caused a regression in detecting cve patches. Master jumped by 32 CVEs after merging this commit.
These are for sure patched but it's not detected anymore. See https://valkyrie.yocto.io/pub/non-release/patchmetrics/
(master should be identical to scarthgap/styhead except for kernel CVEs)Reports cannot be used anymore for picking CVEs for TODO list. So this patch should be reverted or fixed.
Peter
Quoted 97 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Colin McAllister via lists.openembedded.org
Sent: Monday, December 30, 2024 20:22
To: [email protected]
Cc: Colin McAllister <[email protected]>
Subject: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsingThe
cve_checkfunctionality to parse CVE IDs from the patch filename and patch contents have been reworked to improve parsing and also utilize tests. This ensures that the parsing works as intended.Additionally, the new
patched_cvesdict has a few issues I tried to fix as well. If multiple patch files exist for a single CVE ID, only the last one will show up with the "resource" key. The value for the "resource" key has been updated to hold a list and return all patch files associated with a given CVE ID. Also, at the end ofget_patch_cves,CVE_STATUScan overwrite an existing entry in the dict. This could cause an issue, for example, if a CVE has been addressed via a patch, but aCVE_STATUSline also exists that ignores the given CVE ID. A warning has been added if this ever happens.Signed-off-by Colin McAllister <[email protected]>---
I noticed that there are some patches, especially in older verisons of Yocto, where the "CVE: " tag was used with multiple CVE IDs in different formats, like "CVE-YYYY-XXXX & CVE-YYYY-XXXX" or "CVE-YYYY-XXXX, CVE-YYYY-XXXX". Currently, only space-delimited CVE IDs will be parsed, but documentation doesn't indicate that is the only supported format. I figured it'd be nice to update the code to be able to support multiple formats, that way this patch could be backported to fix those patches. I also wanted to add unit tests to ensure the patch parsing behavior is preserved.
I'd also like to update the patch filename parsing to parse multiple CVE IDs from the filename, but based on the comments, it seems like there was a reason why only the last CVE ID is extracted from the filename. I'd be happy to submit a V2 patch or an additional patch to update the function if that sounds good for the maintainers.
V2 Changes:
I mistakenly created this patch without fb3f440 applied. I updatedget_patched_cvesto return a dict instead of a set.I also improved the docstrings for these new functions to be more descriptive and also specify the return types.
I also found a few issues with the dictionary object created by
get_patched_cvesthat have now been addressed in this commit.meta/lib/oe/cve_check.py | 166 ++++++++++++------ meta/lib/oeqa/selftest/cases/cve_check.py | 205 ++++++++++++++++++++++ 2 files changed, 317 insertions(+), 54 deletions(-)meta/lib/oe/cve_check.py +14
-6@@ -5,9 +5,11 @@ # import collections -import re -import itertools import functools +import itertools +import os.path +import re +import oe.patch _Version = collections.namedtuple( "_Version", ["release", "patch_l", "pre_l", "pre_v"]@@ -71,76 +73,132 @@ def _cmpkey(release, patch_l, pre_l, pre_v): return _release, _patch, _pre -def get_patched_cves(d): +def parse_cve_from_filename(patch_filename): """ - Get patches that solve CVEs using the "CVE: " tag. + Parses CVE ID from the filename + + Matches the last "CVE-YYYY-ID" in the file name, also if written + in lowercase. Possible to have multiple CVE IDs in a single + file name, but only the last one will be detected from the file name. + + Returns the last CVE ID foudn in the filename. If no CVE ID is found + an empty string is returned. """ + cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d{4,})", re.IGNORECASE) - import re - import oe.patchmeta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 88
Quoted 10 lines
meta/lib/oe/cve_check.py +3
-5+ # Check patch file name for CVE ID + fname_match = cve_file_name_match.search(patch_filename) + return fname_match.group(1).upper() if fname_match else "" - cve_match = re.compile(r"CVE:( CVE-\d{4}-\d+)+") - # Matches the last "CVE-YYYY-ID" in the file name, also if written - # in lowercase. Possible to have multiple CVE IDs in a single - # file name, but only the last one will be detected from the file name. - # However, patch files contents addressing multiple CVE IDs aresupported
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 90
Quoted 2 lines
meta/lib/oe/cve_check.py +0
-2- # (cve_match regular expression) - cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d+)",re.IGNORECASE)
Quoted 134 lines
meta/lib/oe/cve_check.py +22
-0+def parse_cves_from_patch_contents(patch_contents): + """ + Parses CVE IDs from patch contents + Matches all CVE IDs contained on a line that starts with "CVE: ". Any + delimiter (',', '&', "and", etc.) can be used without any issues. Multiple + "CVE:" lines can also exist. + + Returns a set of all CVE IDs found in the patch contents. + """ + cve_ids = set() + cve_match = re.compile(r"CVE-\d{4}-\d{4,}") + # Search for one or more "CVE: " lines + for line in patch_contents.split("\n"): + if not line.startswith("CVE:"): + continue + cve_ids.update(cve_match.findall(line)) + return cve_ids + + +def parse_cves_from_patch_file(patch_file): + """ + Parses CVE IDs associated with a particular patch file, using both thefilename
meta/lib/oe/cve_check.py +8
-0+ and patch contents. + + Returns a set of all CVE IDs found in the patch filename and contents. + """ + cve_ids = set() + filename_cve = parse_cve_from_filename(patch_file) + if filename_cve: + bb.debug(2, "Found %s from patch file name %s" % (filename_cve,
patch_file))meta/lib/oe/cve_check.py +6
-0+ cve_ids.add(parse_cve_from_filename(patch_file)) + + # Remote patches won't be present and compressed patches won't be + # unpacked, so say we're not scanning them + if not os.path.isfile(patch_file): + bb.note("%s is remote or compressed, not scanning content" %
patch_file)meta/lib/oe/cve_check.py +21
-0+ return cve_ids + + with open(patch_file, "r", encoding="utf-8") as f: + try: + patch_text = f.read() + except UnicodeDecodeError: + bb.debug( + 1, + "Failed to read patch %s using UTF-8 encoding" + " trying with iso8859-1" % patch_file, + ) + f.close() + with open(patch_file, "r", encoding="iso8859-1") as f: + patch_text = f.read() + + cve_ids.update(parse_cves_from_patch_contents(patch_text)) + + if not cve_ids: + bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + else: + bb.debug(2, "Patch %s solves %s" % (patch_file, ",".join(sorted(cve_ids))))
meta/lib/oe/cve_check.py +7
-0+ + return cve_ids + + +def get_patched_cves(d): + """ + Determines the CVE IDs that have been solved by either patches incudedwithin
meta/lib/oe/cve_check.py +3
-0+ SRC_URI or by setting CVE_STATUS. + + Returns a dictionary with the CVE IDs as keys and an associated dictonaryof
meta/lib/oe/cve_check.py +4
-6+ relevant metadata as the value. + """ patched_cves = {} patches = oe.patch.src_patches(d) bb.debug(2, "Scanning %d patches for CVEs" % len(patches)) + + # Check each patch file for url in patches: patch_file = bb.fetch.decodeurl(url)[2] - - # Check patch file name for CVE ID - fname_match = cve_file_name_match.search(patch_file) - if fname_match: - cve = fname_match.group(1).upper() - patched_cves[cve] = {"abbrev-status": "Patched", "status": "fix-file-included", "resource":
patch_file}meta/lib/oe/cve_check.py +0
-6- bb.debug(2, "Found %s from patch file name %s" % (cve, patch_file)) - - # Remote patches won't be present and compressed patches won't be - # unpacked, so say we're not scanning them - if not os.path.isfile(patch_file): - bb.note("%s is remote or compressed, not scanning content" %
patch_file)meta/lib/oe/cve_check.py +0
-20- continue - - with open(patch_file, "r", encoding="utf-8") as f: - try: - patch_text = f.read() - except UnicodeDecodeError: - bb.debug(1, "Failed to read patch %s using UTF-8 encoding" - " trying with iso8859-1" % patch_file) - f.close() - with open(patch_file, "r", encoding="iso8859-1") as f: - patch_text = f.read() - - # Search for one or more "CVE: " lines - text_match = False - for match in cve_match.finditer(patch_text): - # Get only the CVEs without the "CVE: " tag - cves = patch_text[match.start()+5:match.end()] - for cve in cves.split(): - bb.debug(2, "Patch %s solves %s" % (patch_file, cve)) - patched_cves[cve] = {"abbrev-status": "Patched", "status": "fix-file-included", "resource":
patch_file}meta/lib/oe/cve_check.py +9
-5- text_match = True - - if not fname_match and not text_match: - bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + for cve_id in parse_cves_from_patch_file(patch_file): + if cve_id not in patched_cves: + { + "abbrev-status": "Patched", + "status": "fix-file-included", + "resource": [patch_file], + } + else: + patched_cves[cve_id]["resource"].append(patch_file) # Search for additional patched CVEs - for cve in (d.getVarFlags("CVE_STATUS") or {}):meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 10 lines
meta/lib/oe/cve_check.py +6
-3- decoded_status = decode_cve_status(d, cve) + for cve_id in d.getVarFlags("CVE_STATUS") or {}: + decoded_status = decode_cve_status(d, cve_id) products = d.getVar("CVE_PRODUCT") - if has_cve_product_match(decoded_status, products) == True: - patched_cves[cve] = { + if has_cve_product_match(decoded_status, products): + if cve_id in patched_cves: + bb.warn( + 'CVE_STATUS[%s] = "%s" is overwriting previous status of "%s:%s"'
Quoted 23 lines
meta/lib/oe/cve_check.py +9
-1+ % ( + cve_id, + d.getVarFlag("CVE_STATUS", cve_id), + patched_cves[cve_id]["abbrev-status"], + patched_cves[cve_id]["status"], + ) + ) + patched_cves[cve_id] = { "abbrev-status": decoded_status["mapping"], "status": decoded_status["detail"], "justification": decoded_status["description"], "affected-vendor": decoded_status["vendor"], - "affected-product": decoded_status["product"] + "affected-product": decoded_status["product"], } return patched_cvesmeta/lib/oeqa/selftest/cases/cve_check.py +0
-0meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase): · line 120
Quoted 1 line
meta/lib/oeqa/selftest/cases/cve_check.py +0
-0@@ -120,6 +120,211 @@ class CVECheck(OESelftestTestCase): self.assertEqual(has_cve_product_match(status, "test glibca:glibc"),True)
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase): · line 121
Quoted 1 line
meta/lib/oeqa/selftest/cases/cve_check.py +0
-0self.assertEqual(has_cve_product_match(status, "glibca:glibc test"),True)
Quoted 28 lines
meta/lib/oeqa/selftest/cases/cve_check.py +9
-0+ def test_parse_cve_from_patch_filename(self): + from oe.cve_check import parse_cve_from_filename + + # Patch filename without CVE ID + self.assertEqual(parse_cve_from_filename("0001-test.patch"), "") + + # Patch with single CVE ID + self.assertEqual( + parse_cve_from_filename("CVE-2022-12345.patch"), "CVE-2022-12345"
meta/lib/oeqa/selftest/cases/cve_check.py +5
-0+ ) + + # Patch with multiple CVE IDs + self.assertEqual( + parse_cve_from_filename("CVE-2022-41741-CVE-2022-41742.patch"),
meta/lib/oeqa/selftest/cases/cve_check.py +6
-0+ "CVE-2022-41742", + ) + + # Patches with CVE ID and appended text + self.assertEqual( + parse_cve_from_filename("CVE-2023-3019-0001.patch"), "CVE-2023-3019"
meta/lib/oeqa/selftest/cases/cve_check.py +3
-0+ ) + self.assertEqual( + parse_cve_from_filename("CVE-2024-21886-1.patch"), "CVE-2024-21886"
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 5 lines
meta/lib/oeqa/selftest/cases/cve_check.py +5
-0+ ) + + # Patch with CVE ID and prepended text + self.assertEqual( + parse_cve_from_filename("grep-CVE-2012-5667.patch"), "CVE-2012-
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 11 lines
5667"
meta/lib/oeqa/selftest/cases/cve_check.py +3
-0+ ) + self.assertEqual( + parse_cve_from_filename("0001-CVE-2012-5667.patch"), "CVE-2012-5667"
meta/lib/oeqa/selftest/cases/cve_check.py +6
-0+ ) + + # Patch with CVE ID and both prepended and appended text + self.assertEqual( + parse_cve_from_filename( + "0001-tpm2_import-fix-fixed-AES-key-CVE-2021-3565-0001.patch"
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 7 lines
meta/lib/oeqa/selftest/cases/cve_check.py +7
-0+ ), + "CVE-2021-3565", + ) + + # Only grab the last CVE ID in the filename + self.assertEqual( + parse_cve_from_filename("CVE-2012-5667-CVE-2012-5668.patch"),
Quoted 169 lines
meta/lib/oeqa/selftest/cases/cve_check.py +12
-0+ "CVE-2012-5668", + ) + + # Test invalid CVE ID with incorrect length (must be at least 4 digits) + self.assertEqual( + parse_cve_from_filename("CVE-2024-001.patch"), + "", + ) + + # Test valid CVE ID with very long length + self.assertEqual( + parse_cve_from_filename("CVE-2024-0000000000000000000000001.patch"),
meta/lib/oeqa/selftest/cases/cve_check.py +60
-0+ "CVE-2024-0000000000000000000000001", + ) + + def test_parse_cve_from_patch_contents(self): + import textwrap + from oe.cve_check import parse_cves_from_patch_contents + + # Standard patch file excerpt without any patches + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + remove "*" for root since we don't have a /etc/shadow so far. + + Upstream-Status: Inappropriate [configuration] + + Signed-off-by: Scott Garman <[email protected]> + + --- base-passwd/passwd.master~nobash + +++ base-passwd/passwd.master + @@ -1,4 +1,4 @@ + -root:*:0:0:root:/root:/bin/sh + +root::0:0:root:/root:/bin/sh + daemon:*:1:1:daemon:/usr/sbin:/bin/sh + bin:*:2:2:bin:/bin:/bin/sh + sys:*:3:3:sys:/dev:/bin/sh + """) + ), + set(), + ) + + # Patch file with multiple CVE IDs (space-separated) + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461 CVE-2019-6462 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + """), + ), + {"CVE-2019-6461", "CVE-2019-6462"}, + ) + + # Patch file with multiple CVE IDs (comma-separated w/ both space andno space)
meta/lib/oeqa/selftest/cases/cve_check.py +88
-0+ self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461,CVE-2019-6462, CVE-2019-6463 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + + """), + ), + {"CVE-2019-6461", "CVE-2019-6462", "CVE-2019-6463"}, + ) + + # Patch file with multiple CVE IDs (&-separated) + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461 & CVE-2019-6462 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + """), + ), + {"CVE-2019-6461", "CVE-2019-6462"}, + ) + + # Patch file with multiple lines with CVE IDs + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461 & CVE-2019-6462 + + CVE: CVE-2019-6463 & CVE-2019-6464 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + + """), + ), + {"CVE-2019-6461", "CVE-2019-6462", "CVE-2019-6463", "CVE-2019-6464"},
+ )def test_recipe_report_json(self): config = """--
2.34.1
A message that is not in the archive (sent off-list or not yet received): [email protected]
RE: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
Looking at the reopened CVEs in meta-oe, I can't understand why are some reported.
$ grep -rI CVE-2023-33460 ./ ./meta-oe/recipes-devtools/yajl/yajl/CVE-2023-33460.patch:CVE: CVE-2023-33460 ./meta-oe/recipes-devtools/yajl/yajl_2.1.0.bb: file://CVE-2023-33460.patch \ $ grep -rI CVE-2020-29074 ./ ./meta-oe/recipes-graphics/x11vnc/files/CVE-2020-29074.patch:CVE: CVE-2020-29074 ./meta-oe/recipes-graphics/x11vnc/x11vnc_0.9.16.bb: file://CVE-2020-29074.patch \
These look good to me, but they are not matched anymore and show as open.
Peter
Quoted 550 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Peter Marko via lists.openembedded.org
Sent: Friday, January 10, 2025 10:21
To: [email protected]; [email protected]
Subject: Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsingOr should we rather rename all those patches which violate current code? What is then the meaning of CVE: tag within the patch if it's not used but it's enforced?
We should probably removecve_payload_taghandling fromtest_cve_tag_format...Peter
Quoted 6 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Peter Marko via lists.openembedded.org
Sent: Friday, January 10, 2025 9:34
To: [email protected]; openembedded-Quoted 187 lines
Subject: Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
Hello,
This patch has caused a regression in detecting cve patches. Master jumped by 32 CVEs after merging this commit.
These are for sure patched but it's not detected anymore. See https://valkyrie.yocto.io/pub/non-release/patchmetrics/
(master should be identical to scarthgap/styhead except for kernel CVEs)Reports cannot be used anymore for picking CVEs for TODO list. So this patch should be reverted or fixed.
Peter
Quoted 97 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Colin McAllister via lists.openembedded.org
Sent: Monday, December 30, 2024 20:22
To: [email protected]
Cc: Colin McAllister <[email protected]>
Subject: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsingThe
cve_checkfunctionality to parse CVE IDs from the patch filename and patch contents have been reworked to improve parsing and also utilize tests. This ensures that the parsing works as intended.Additionally, the new
patched_cvesdict has a few issues I tried to fix as well. If multiple patch files exist for a single CVE ID, only the last one will show up with the "resource" key. The value for the "resource" key has been updated to hold a list and return all patch files associated with a given CVE ID. Also, at the end ofget_patch_cves,CVE_STATUScan overwrite an existing entry in the dict. This could cause an issue, for example, if a CVE has been addressed via a patch, but aCVE_STATUSline also exists that ignores the given CVE ID. A warning has been added if this ever happens.Signed-off-by Colin McAllister <[email protected]>---
I noticed that there are some patches, especially in older verisons of Yocto, where the "CVE: " tag was used with multiple CVE IDs in different formats, like "CVE-YYYY-XXXX & CVE-YYYY-XXXX" or "CVE-YYYY-XXXX, CVE-YYYY-XXXX". Currently, only space-delimited CVE IDs will be parsed, but documentation doesn't indicate that is the only supported format. I figured it'd be nice to update the code to be able to support multiple formats, that way this patch could be backported to fix those patches. I also wanted to add unit tests to ensure the patch parsing behavior is preserved.
I'd also like to update the patch filename parsing to parse multiple CVE IDs from the filename, but based on the comments, it seems like there was a reason why only the last CVE ID is extracted from the filename. I'd be happy to submit a V2 patch or an additional patch to update the function if that sounds good for the maintainers.
V2 Changes:
I mistakenly created this patch without fb3f440 applied. I updatedget_patched_cvesto return a dict instead of a set.I also improved the docstrings for these new functions to be more descriptive and also specify the return types.
I also found a few issues with the dictionary object created by
get_patched_cvesthat have now been addressed in this commit.meta/lib/oe/cve_check.py | 166 ++++++++++++------ meta/lib/oeqa/selftest/cases/cve_check.py | 205 ++++++++++++++++++++++ 2 files changed, 317 insertions(+), 54 deletions(-)meta/lib/oe/cve_check.py +14
-6@@ -5,9 +5,11 @@ # import collections -import re -import itertools import functools +import itertools +import os.path +import re +import oe.patch _Version = collections.namedtuple( "_Version", ["release", "patch_l", "pre_l", "pre_v"]@@ -71,76 +73,132 @@ def _cmpkey(release, patch_l, pre_l, pre_v): return _release, _patch, _pre -def get_patched_cves(d): +def parse_cve_from_filename(patch_filename): """ - Get patches that solve CVEs using the "CVE: " tag. + Parses CVE ID from the filename + + Matches the last "CVE-YYYY-ID" in the file name, also if written + in lowercase. Possible to have multiple CVE IDs in a single + file name, but only the last one will be detected from the file name. + + Returns the last CVE ID foudn in the filename. If no CVE ID is found + an empty string is returned. """ + cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d{4,})", re.IGNORECASE) - import re - import oe.patchmeta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 88
Quoted 10 lines
meta/lib/oe/cve_check.py +3
-5+ # Check patch file name for CVE ID + fname_match = cve_file_name_match.search(patch_filename) + return fname_match.group(1).upper() if fname_match else "" - cve_match = re.compile(r"CVE:( CVE-\d{4}-\d+)+") - # Matches the last "CVE-YYYY-ID" in the file name, also if written - # in lowercase. Possible to have multiple CVE IDs in a single - # file name, but only the last one will be detected from the file name. - # However, patch files contents addressing multiple CVE IDs aresupported
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 90
Quoted 2 lines
meta/lib/oe/cve_check.py +0
-2- # (cve_match regular expression) - cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d+)",re.IGNORECASE)
Quoted 61 lines
meta/lib/oe/cve_check.py +22
-0+def parse_cves_from_patch_contents(patch_contents): + """ + Parses CVE IDs from patch contents + Matches all CVE IDs contained on a line that starts with "CVE: ". Any + delimiter (',', '&', "and", etc.) can be used without any issues. Multiple + "CVE:" lines can also exist. + + Returns a set of all CVE IDs found in the patch contents. + """ + cve_ids = set() + cve_match = re.compile(r"CVE-\d{4}-\d{4,}") + # Search for one or more "CVE: " lines + for line in patch_contents.split("\n"): + if not line.startswith("CVE:"): + continue + cve_ids.update(cve_match.findall(line)) + return cve_ids + + +def parse_cves_from_patch_file(patch_file): + """ + Parses CVE IDs associated with a particular patch file, using both thefilename
meta/lib/oe/cve_check.py +8
-0+ and patch contents. + + Returns a set of all CVE IDs found in the patch filename and contents. + """ + cve_ids = set() + filename_cve = parse_cve_from_filename(patch_file) + if filename_cve: + bb.debug(2, "Found %s from patch file name %s" % (filename_cve,
patch_file))meta/lib/oe/cve_check.py +6
-0+ cve_ids.add(parse_cve_from_filename(patch_file)) + + # Remote patches won't be present and compressed patches won't be + # unpacked, so say we're not scanning them + if not os.path.isfile(patch_file): + bb.note("%s is remote or compressed, not scanning content" %
patch_file)meta/lib/oe/cve_check.py +21
-0+ return cve_ids + + with open(patch_file, "r", encoding="utf-8") as f: + try: + patch_text = f.read() + except UnicodeDecodeError: + bb.debug( + 1, + "Failed to read patch %s using UTF-8 encoding" + " trying with iso8859-1" % patch_file, + ) + f.close() + with open(patch_file, "r", encoding="iso8859-1") as f: + patch_text = f.read() + + cve_ids.update(parse_cves_from_patch_contents(patch_text)) + + if not cve_ids: + bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + else: + bb.debug(2, "Patch %s solves %s" % (patch_file, ",Quoted 8 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 8 lines
".join(sorted(cve_ids))))
meta/lib/oe/cve_check.py +7
-0+ + return cve_ids + + +def get_patched_cves(d): + """ + Determines the CVE IDs that have been solved by either patchesincuded
Quoted 4 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 4 lines
within
meta/lib/oe/cve_check.py +3
-0+ SRC_URI or by setting CVE_STATUS. + + Returns a dictionary with the CVE IDs as keys and an associateddictonary
Quoted 10 lines
Quoted 10 lines
of
meta/lib/oe/cve_check.py +4
-0+ relevant metadata as the value. + """ patched_cves = {} patches = oe.patch.src_patches(d) bb.debug(2, "Scanning %d patches for CVEs" % len(patches)) + + # Check each patch file for url in patches: patch_file = bb.fetch.decodeurl(url)[2]Quoted 8 lines
Quoted 8 lines
meta/lib/oe/cve_check.py +0
-6- - # Check patch file name for CVE ID - fname_match = cve_file_name_match.search(patch_file) - if fname_match: - cve = fname_match.group(1).upper() - patched_cves[cve] = {"abbrev-status": "Patched", "status": "fix-file-included", "resource":
patch_file}
- bb.debug(2, "Found %s from patch file name %s" % (cve,
patch_file))Quoted 18 lines
Quoted 18 lines
meta/lib/oe/cve_check.py +0
-5- - # Remote patches won't be present and compressed patches won't be - # unpacked, so say we're not scanning them - if not os.path.isfile(patch_file): - bb.note("%s is remote or compressed, not scanning content" %
patch_file)meta/lib/oe/cve_check.py +0
-12- continue - - with open(patch_file, "r", encoding="utf-8") as f: - try: - patch_text = f.read() - except UnicodeDecodeError: - bb.debug(1, "Failed to read patch %s using UTF-8 encoding" - " trying with iso8859-1" % patch_file) - f.close() - with open(patch_file, "r", encoding="iso8859-1") as f: - patch_text = f.read() -Quoted 8 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 8 lines
meta/lib/oe/cve_check.py +0
-8- # Search for one or more "CVE: " lines - text_match = False - for match in cve_match.finditer(patch_text): - # Get only the CVEs without the "CVE: " tag - cves = patch_text[match.start()+5:match.end()] - for cve in cves.split(): - bb.debug(2, "Patch %s solves %s" % (patch_file, cve)) - patched_cves[cve] = {"abbrev-status": "Patched", "status": "fix-file-
Quoted 74 lines
Quoted 17 lines
included", "resource":
patch_file}meta/lib/oe/cve_check.py +9
-5- text_match = True - - if not fname_match and not text_match: - bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + for cve_id in parse_cves_from_patch_file(patch_file): + if cve_id not in patched_cves: + { + "abbrev-status": "Patched", + "status": "fix-file-included", + "resource": [patch_file], + } + else: + patched_cves[cve_id]["resource"].append(patch_file) # Search for additional patched CVEs - for cve in (d.getVarFlags("CVE_STATUS") or {}):meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 10 lines
meta/lib/oe/cve_check.py +6
-3- decoded_status = decode_cve_status(d, cve) + for cve_id in d.getVarFlags("CVE_STATUS") or {}: + decoded_status = decode_cve_status(d, cve_id) products = d.getVar("CVE_PRODUCT") - if has_cve_product_match(decoded_status, products) == True: - patched_cves[cve] = { + if has_cve_product_match(decoded_status, products): + if cve_id in patched_cves: + bb.warn( + 'CVE_STATUS[%s] = "%s" is overwriting previous status of "%s:%s"'
Quoted 23 lines
meta/lib/oe/cve_check.py +9
-1+ % ( + cve_id, + d.getVarFlag("CVE_STATUS", cve_id), + patched_cves[cve_id]["abbrev-status"], + patched_cves[cve_id]["status"], + ) + ) + patched_cves[cve_id] = { "abbrev-status": decoded_status["mapping"], "status": decoded_status["detail"], "justification": decoded_status["description"], "affected-vendor": decoded_status["vendor"], - "affected-product": decoded_status["product"] + "affected-product": decoded_status["product"], } return patched_cvesmeta/lib/oeqa/selftest/cases/cve_check.py +0
-0meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase): · line 120
Quoted 1 line
meta/lib/oeqa/selftest/cases/cve_check.py +0
-0@@ -120,6 +120,211 @@ class CVECheck(OESelftestTestCase): self.assertEqual(has_cve_product_match(status, "test glibca:glibc"),True)
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase): · line 121
Quoted 1 line
meta/lib/oeqa/selftest/cases/cve_check.py +0
-0self.assertEqual(has_cve_product_match(status, "glibca:glibc test"),True)
Quoted 19 lines
meta/lib/oeqa/selftest/cases/cve_check.py +9
-0+ def test_parse_cve_from_patch_filename(self): + from oe.cve_check import parse_cve_from_filename + + # Patch filename without CVE ID + self.assertEqual(parse_cve_from_filename("0001-test.patch"), "") + + # Patch with single CVE ID + self.assertEqual( + parse_cve_from_filename("CVE-2022-12345.patch"), "CVE-2022-12345"
meta/lib/oeqa/selftest/cases/cve_check.py +5
-0+ ) + + # Patch with multiple CVE IDs + self.assertEqual( + parse_cve_from_filename("CVE-2022-41741-CVE-2022-41742.patch"),
meta/lib/oeqa/selftest/cases/cve_check.py +2
-0+ "CVE-2022-41742", + )Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ + # Patches with CVE ID and appended text + self.assertEqual( + parse_cve_from_filename("CVE-2023-3019-0001.patch"), "CVE-2023-3019"
meta/lib/oeqa/selftest/cases/cve_check.py +3
-0+ ) + self.assertEqual( + parse_cve_from_filename("CVE-2024-21886-1.patch"), "CVE-2024-
Quoted 92 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 6 lines
21886"
meta/lib/oeqa/selftest/cases/cve_check.py +5
-0+ ) + + # Patch with CVE ID and prepended text + self.assertEqual( + parse_cve_from_filename("grep-CVE-2012-5667.patch"), "CVE-2012-
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 11 lines
5667"
meta/lib/oeqa/selftest/cases/cve_check.py +3
-0+ ) + self.assertEqual( + parse_cve_from_filename("0001-CVE-2012-5667.patch"), "CVE-2012-5667"
meta/lib/oeqa/selftest/cases/cve_check.py +6
-0+ ) + + # Patch with CVE ID and both prepended and appended text + self.assertEqual( + parse_cve_from_filename( + "0001-tpm2_import-fix-fixed-AES-key-CVE-2021-3565-0001.patch"
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 7 lines
meta/lib/oeqa/selftest/cases/cve_check.py +7
-0+ ), + "CVE-2021-3565", + ) + + # Only grab the last CVE ID in the filename + self.assertEqual( + parse_cve_from_filename("CVE-2012-5667-CVE-2012-5668.patch"),
Quoted 65 lines
meta/lib/oeqa/selftest/cases/cve_check.py +12
-0+ "CVE-2012-5668", + ) + + # Test invalid CVE ID with incorrect length (must be at least 4 digits) + self.assertEqual( + parse_cve_from_filename("CVE-2024-001.patch"), + "", + ) + + # Test valid CVE ID with very long length + self.assertEqual( + parse_cve_from_filename("CVE-2024-0000000000000000000000001.patch"),
meta/lib/oeqa/selftest/cases/cve_check.py +52
-0+ "CVE-2024-0000000000000000000000001", + ) + + def test_parse_cve_from_patch_contents(self): + import textwrap + from oe.cve_check import parse_cves_from_patch_contents + + # Standard patch file excerpt without any patches + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + remove "*" for root since we don't have a /etc/shadow so far. + + Upstream-Status: Inappropriate [configuration] + + Signed-off-by: Scott Garman <[email protected]> + + --- base-passwd/passwd.master~nobash + +++ base-passwd/passwd.master + @@ -1,4 +1,4 @@ + -root:*:0:0:root:/root:/bin/sh + +root::0:0:root:/root:/bin/sh + daemon:*:1:1:daemon:/usr/sbin:/bin/sh + bin:*:2:2:bin:/bin:/bin/sh + sys:*:3:3:sys:/dev:/bin/sh + """) + ), + set(), + ) + + # Patch file with multiple CVE IDs (space-separated) + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461 CVE-2019-6462 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; +Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + """), + ), + {"CVE-2019-6461", "CVE-2019-6462"}, + ) + + # Patch file with multiple CVE IDs (comma-separated w/ both spaceand
Quoted 96 lines
Quoted 96 lines
no space)
meta/lib/oeqa/selftest/cases/cve_check.py +88
-0+ self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461,CVE-2019-6462, CVE-2019-6463 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + + """), + ), + {"CVE-2019-6461", "CVE-2019-6462", "CVE-2019-6463"}, + ) + + # Patch file with multiple CVE IDs (&-separated) + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461 & CVE-2019-6462 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + """), + ), + {"CVE-2019-6461", "CVE-2019-6462"}, + ) + + # Patch file with multiple lines with CVE IDs + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461 & CVE-2019-6462 + + CVE: CVE-2019-6463 & CVE-2019-6464 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + + """), + ), + {"CVE-2019-6461", "CVE-2019-6462", "CVE-2019-6463", "CVE-2019-6464"},
+ )def test_recipe_report_json(self): config = """--
2.34.1
Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
Hi Peter,
I'm currently looking at this. I added CVE-2023-33460.patch to test_parse_cve_from_patch_contents and ran oe-selftest. I see the issue why these patches are no longer being marked as solved.
2025-01-11 15:17:33,396 - oe-selftest - INFO - ====================================================================== 2025-01-11 15:17:33,396 - oe-selftest - INFO - FAIL: test_parse_cve_from_patch_contents (cve_check.CVECheck) 2025-01-11 15:17:33,397 - oe-selftest - INFO - ---------------------------------------------------------------------- 2025-01-11 15:17:33,397 - oe-selftest - INFO - Traceback (most recent call
last):
File
"/home/colin/development/yocto/poky/meta/lib/oeqa/selftest/cases/cve_check.py", line 209, in test_parse_cve_from_patch_contents
self.assertEqual(
AssertionError: Items in the first set but not the second:
'CVE-2023-33460'
Items in the second set but not the first:
'E'
'2'
'V'
'C'
'-'
'0'
'4'
'3'
'6'
2025-01-11 15:17:33,397 - oe-selftest - INFO - ---------------------------------------------------------------------- 2025-01-11 15:17:33,397 - oe-selftest - INFO - Ran 1 test in 0.247s 2025-01-11 15:17:33,398 - oe-selftest - INFO - FAILED 2025-01-11 15:17:33,398 - oe-selftest - INFO - (failures=1) 2025-01-11 15:17:46,988 - oe-selftest - INFO - RESULTS: 2025-01-11 15:17:46,988 - oe-selftest - INFO - RESULTS - cve_check.CVECheck.test_parse_cve_from_patch_contents: FAILED (0.02s) 2025-01-11 15:17:46,996 - oe-selftest - INFO - SUMMARY: 2025-01-11 15:17:46,996 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 0.249s 2025-01-11 15:17:46,996 - oe-selftest - INFO - oe-selftest - FAIL -
Required tests failed (successes=0, skipped=0, failures=1, errors=0)
For some reason the CVE string itself is getting put into a set, instead of the full string being placed into a set. I'll track down where this is happening and push up a fix, including adding this update to the test.
Not sure why I didn't add a test case with a single CVE ID in the patch contents...This was my mistake and I apologize. Sorry for any inconveniences I caused.
Best,
Colin
Quoted 629 lines · Peter
Looking at the reopened CVEs in meta-oe, I can't understand why are some reported.
$ grep -rI CVE-2023-33460 ./ ./meta-oe/recipes-devtools/yajl/yajl/CVE-2023-33460.patch:CVE: CVE-2023-33460 ./meta-oe/recipes-devtools/yajl/yajl_2.1.0.bb: file://CVE-2023-33460.patch \ $ grep -rI CVE-2020-29074 ./ ./meta-oe/recipes-graphics/x11vnc/files/CVE-2020-29074.patch:CVE: CVE-2020-29074 ./meta-oe/recipes-graphics/x11vnc/x11vnc_0.9.16.bb: file://CVE-2020-29074.patch \These look good to me, but they are not matched anymore and show as open.
Peter
Quoted 10 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Peter Marko via lists.openembedded.org
Sent: Friday, January 10, 2025 10:21
To: [email protected]; [email protected]
Subject: Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsingOr should we rather rename all those patches which violate current code? What is then the meaning of CVE: tag within the patch if it's not used
but it's
Quoted 6 lines
enforced?
We should probably removecve_payload_taghandling fromtest_cve_tag_format...Peter
Quoted 16 lines
Quoted 6 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Peter Marko via lists.openembedded.org
Sent: Friday, January 10, 2025 9:34
To: [email protected]; openembedded-Quoted 9 lines
Subject: Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
Hello,
This patch has caused a regression in detecting cve patches. Master jumped by 32 CVEs after merging this commit. These are for sure patched but it's not detected anymore. See https://valkyrie.yocto.io/pub/non-release/patchmetrics/
(master should be identical to scarthgap/styhead except for kernelCVEs)
Quoted 15 lines
Quoted 15 lines
Reports cannot be used anymore for picking CVEs for TODO list. So this patch should be reverted or fixed.
Peter
Quoted 9 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Colin McAllister via lists.openembedded.org
Sent: Monday, December 30, 2024 20:22
To: [email protected]
Cc: Colin McAllister <[email protected]>
Subject: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsingQuoted 1 line
Quoted 1 line
Quoted 1 line
The
cve_checkfunctionality to parse CVE IDs from the patch filenameand
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
patch contents have been reworked to improve parsing and also utilize tests. This ensures that the parsing works as intended.
Additionally, the new
patched_cvesdict has a few issues I tried tofix
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
as well. If multiple patch files exist for a single CVE ID, only the last one will show up with the "resource" key. The value for the "resource" key has been updated to hold a list and return all patch files associated with a given CVE ID. Also, at the end of
get_patch_cves,CVE_STATUScan overwrite an existing entry in thedict.
Quoted 1 line
Quoted 1 line
Quoted 1 line
This could cause an issue, for example, if a CVE has been addressed
via
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
a patch, but a
CVE_STATUSline also exists that ignores the given CVE ID. A warning has been added if this ever happens.Signed-off-by Colin McAllister <[email protected]>---
I noticed that there are some patches, especially in older verisons
of
Quoted 1 line
Quoted 1 line
Quoted 1 line
Yocto, where the "CVE: " tag was used with multiple CVE IDs in
different
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
formats, like "CVE-YYYY-XXXX & CVE-YYYY-XXXX" or "CVE-YYYY-XXXX, CVE-YYYY-XXXX". Currently, only space-delimited CVE IDs will be parsed, but documentation doesn't indicate that is the
only
Quoted 1 line
Quoted 1 line
Quoted 1 line
supported format. I figured it'd be nice to update the code to be
able
Quoted 1 line
Quoted 1 line
Quoted 1 line
to support multiple formats, that way this patch could be backported
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
fix those patches. I also wanted to add unit tests to ensure the
patch
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
parsing behavior is preserved.
I'd also like to update the patch filename parsing to parse multiple
CVE
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
IDs from the filename, but based on the comments, it seems like there was a reason why only the last CVE ID is extracted from the filename. I'd be happy to submit a V2 patch or an additional patch to update
the
Quoted 39 lines
Quoted 39 lines
Quoted 39 lines
function if that sounds good for the maintainers.
V2 Changes:
I mistakenly created this patch without fb3f440 applied. I updatedget_patched_cvesto return a dict instead of a set.I also improved the docstrings for these new functions to be more descriptive and also specify the return types.
I also found a few issues with the dictionary object created by
get_patched_cvesthat have now been addressed in this commit.meta/lib/oe/cve_check.py | 166 ++++++++++++------ meta/lib/oeqa/selftest/cases/cve_check.py | 205 ++++++++++++++++++++++ 2 files changed, 317 insertions(+), 54 deletions(-)meta/lib/oe/cve_check.py +4
-2@@ -5,9 +5,11 @@ # import collections -import re -import itertools import functools +import itertools +import os.path +import re +import oe.patch _Version = collections.namedtuple( "_Version", ["release", "patch_l", "pre_l", "pre_v"]@@ -71,76 +73,132 @@ def _cmpkey(release, patch_l, pre_l, pre_v): return _release, _patch, _preQuoted 9 lines
Quoted 9 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 9 lines
meta/lib/oe/cve_check.py +6
-2-def get_patched_cves(d): +def parse_cve_from_filename(patch_filename): """ - Get patches that solve CVEs using the "CVE: " tag. + Parses CVE ID from the filename + + Matches the last "CVE-YYYY-ID" in the file name, also if written + in lowercase. Possible to have multiple CVE IDs in a single + file name, but only the last one will be detected from the filename.
Quoted 2 lines
Quoted 2 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 84
Quoted 2 lines
meta/lib/oe/cve_check.py +2
-0+ + Returns the last CVE ID foudn in the filename. If no CVE ID isfound
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
meta/lib/oe/cve_check.py +5
-2+ an empty string is returned. """ + cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d{4,})", re.IGNORECASE) - import re - import oe.patch + # Check patch file name for CVE ID + fname_match = cve_file_name_match.search(patch_filename) + return fname_match.group(1).upper() if fname_match else ""Quoted 3 lines
Quoted 3 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 3 lines
meta/lib/oe/cve_check.py +0
-2- cve_match = re.compile(r"CVE:( CVE-\d{4}-\d+)+") - # Matches the last "CVE-YYYY-ID" in the file name, also ifwritten
Quoted 2 lines
Quoted 2 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 87
Quoted 2 lines
meta/lib/oe/cve_check.py +0
-2- # in lowercase. Possible to have multiple CVE IDs in a single - # file name, but only the last one will be detected from thefile name.
Quoted 10 lines
Quoted 10 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 88
Quoted 1 line
meta/lib/oe/cve_check.py +0
-1- # However, patch files contents addressing multiple CVE IDs aresupported
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 90
Quoted 2 lines
meta/lib/oe/cve_check.py +0
-2- # (cve_match regular expression) - cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d+)",re.IGNORECASE)
Quoted 5 lines
meta/lib/oe/cve_check.py +4
-0+def parse_cves_from_patch_contents(patch_contents): + """ + Parses CVE IDs from patch contents + Matches all CVE IDs contained on a line that starts with "CVE:". Any
Quoted 1 line
Quoted 1 line
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 100
Quoted 1 line
meta/lib/oe/cve_check.py +1
-0+ delimiter (',', '&', "and", etc.) can be used without anyissues. Multiple
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
meta/lib/oe/cve_check.py +9
-0+ "CVE:" lines can also exist. + + Returns a set of all CVE IDs found in the patch contents. + """ + cve_ids = set() + cve_match = re.compile(r"CVE-\d{4}-\d{4,}") + # Search for one or more "CVE: " lines + for line in patch_contents.split("\n"): + if not line.startswith("CVE:"):Quoted 8 lines
Quoted 8 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 8 lines
meta/lib/oe/cve_check.py +8
-0+ continue + cve_ids.update(cve_match.findall(line)) + return cve_ids + + +def parse_cves_from_patch_file(patch_file): + """ + Parses CVE IDs associated with a particular patch file, usingboth the
Quoted 4 lines
Quoted 4 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 4 lines
filename
meta/lib/oe/cve_check.py +3
-0+ and patch contents. + + Returns a set of all CVE IDs found in the patch filename andcontents.
Quoted 5 lines
Quoted 5 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 5 lines
meta/lib/oe/cve_check.py +5
-0+ """ + cve_ids = set() + filename_cve = parse_cve_from_filename(patch_file) + if filename_cve: + bb.debug(2, "Found %s from patch file name %s" %(
filename_cve,Quoted 4 lines
Quoted 4 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 4 lines
patch_file))meta/lib/oe/cve_check.py +3
-0+ cve_ids.add(parse_cve_from_filename(patch_file)) + + # Remote patches won't be present and compressed patches won'tbe
Quoted 39 lines
Quoted 25 lines
Quoted 25 lines
meta/lib/oe/cve_check.py +3
-0+ # unpacked, so say we're not scanning them + if not os.path.isfile(patch_file): + bb.note("%s is remote or compressed, not scanning content" %
patch_file)meta/lib/oe/cve_check.py +21
-0+ return cve_ids + + with open(patch_file, "r", encoding="utf-8") as f: + try: + patch_text = f.read() + except UnicodeDecodeError: + bb.debug( + 1, + "Failed to read patch %s using UTF-8 encoding" + " trying with iso8859-1" % patch_file, + ) + f.close() + with open(patch_file, "r", encoding="iso8859-1") as f: + patch_text = f.read() + + cve_ids.update(parse_cves_from_patch_contents(patch_text)) + + if not cve_ids: + bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + else: + bb.debug(2, "Patch %s solves %s" % (patch_file, ",Quoted 8 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 8 lines
".join(sorted(cve_ids))))
meta/lib/oe/cve_check.py +7
-0+ + return cve_ids + + +def get_patched_cves(d): + """ + Determines the CVE IDs that have been solved by either patchesincuded
Quoted 4 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 4 lines
within
meta/lib/oe/cve_check.py +3
-0+ SRC_URI or by setting CVE_STATUS. + + Returns a dictionary with the CVE IDs as keys and an associateddictonary
Quoted 16 lines
Quoted 8 lines
Quoted 8 lines
of
meta/lib/oe/cve_check.py +4
-0+ relevant metadata as the value. + """ patched_cves = {} patches = oe.patch.src_patches(d) bb.debug(2, "Scanning %d patches for CVEs" % len(patches)) + + # Check each patch fileQuoted 8 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 8 lines
for url in patches:
patch_file= bb.fetch.decodeurl(url)[2]meta/lib/oe/cve_check.py +0
-6- - # Check patch file name for CVE ID - fname_match = cve_file_name_match.search(patch_file) - if fname_match: - cve = fname_match.group(1).upper() - patched_cves[cve] = {"abbrev-status": "Patched","status": "fix-file-
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
included", "resource":
patch_file}
- bb.debug(2, "Found %s from patch file name %s" % (cve,
patch_file))Quoted 2 lines
Quoted 2 lines
meta/lib/oe/cve_check.py +0
-2- - # Remote patches won't be present and compressed patcheswon't be
Quoted 3 lines
Quoted 3 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 3 lines
meta/lib/oe/cve_check.py +0
-3- # unpacked, so say we're not scanning them - if not os.path.isfile(patch_file): - bb.note("%s is remote or compressed, not scanningcontent" %
Quoted 8 lines
Quoted 8 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 8 lines
patch_file)meta/lib/oe/cve_check.py +0
-7- continue - - with open(patch_file, "r", encoding="utf-8") as f: - try: - patch_text = f.read() - except UnicodeDecodeError: - bb.debug(1, "Failed to read patch %s using UTF-8encoding"
Quoted 3 lines
Quoted 3 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 3 lines
meta/lib/oe/cve_check.py +0
-3- " trying with iso8859-1" % patch_file) - f.close() - with open(patch_file, "r", encoding="iso8859-1") asf:
Quoted 9 lines
Quoted 9 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 9 lines
meta/lib/oe/cve_check.py +0
-9- patch_text = f.read() - - # Search for one or more "CVE: " lines - text_match = False - for match in cve_match.finditer(patch_text): - # Get only the CVEs without the "CVE: " tag - cves = patch_text[match.start()+5:match.end()] - for cve in cves.split(): - bb.debug(2, "Patch %s solves %s" % (patch_file,cve))
Quoted 1 line
Quoted 1 line
Quoted 1 line
- patched_cves[cve] = {"abbrev-status": "Patched",
"status": "fix-
Quoted 28 lines
file-
Quoted 27 lines
Quoted 27 lines
included", "resource":
patch_file}meta/lib/oe/cve_check.py +15
-8- text_match = True - - if not fname_match and not text_match: - bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + for cve_id in parse_cves_from_patch_file(patch_file): + if cve_id not in patched_cves: + { + "abbrev-status": "Patched", + "status": "fix-file-included", + "resource": [patch_file], + } + else: + patched_cves[cve_id]["resource"].append(patch_file) # Search for additional patched CVEs - for cve in (d.getVarFlags("CVE_STATUS") or {}): - decoded_status = decode_cve_status(d, cve) + for cve_id in d.getVarFlags("CVE_STATUS") or {}: + decoded_status = decode_cve_status(d, cve_id) products = d.getVar("CVE_PRODUCT") - if has_cve_product_match(decoded_status, products) == True: - patched_cves[cve] = { + if has_cve_product_match(decoded_status, products): + if cve_id in patched_cves: + bb.warn( + 'CVE_STATUS[%s] = "%s" is overwriting previousstatus of "%s:
Quoted 17 lines
Quoted 17 lines
%s"'
Quoted 16 lines
meta/lib/oe/cve_check.py +9
-1+ % ( + cve_id, + d.getVarFlag("CVE_STATUS", cve_id), + patched_cves[cve_id]["abbrev-status"], + patched_cves[cve_id]["status"], + ) + ) + patched_cves[cve_id] = { "abbrev-status": decoded_status["mapping"], "status": decoded_status["detail"], "justification": decoded_status["description"], "affected-vendor": decoded_status["vendor"], - "affected-product": decoded_status["product"] + "affected-product": decoded_status["product"], }Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase): · line 120
Quoted 8 lines
return
patched_cvesmeta/lib/oeqa/selftest/cases/cve_check.py +0
-0@@ -120,6 +120,211 @@ class CVECheck(OESelftestTestCase): self.assertEqual(has_cve_product_match(status, "testglibca:glibc"),
Quoted 2 lines
Quoted 2 lines
True)
Quoted 1 line
meta/lib/oeqa/selftest/cases/cve_check.py +0
-0self.assertEqual(has_cve_product_match(status,"glibca:glibc test"),
Quoted 7 lines
Quoted 7 lines
True)
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase): · line 128
Quoted 6 lines
meta/lib/oeqa/selftest/cases/cve_check.py +5
-0+ def test_parse_cve_from_patch_filename(self): + from oe.cve_check import parse_cve_from_filename + + # Patch filename without CVE ID +
self.assertEqual(parse_cve_from_filename("0001-test.patch"), "")Quoted 4 lines
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ + # Patch with single CVE ID + self.assertEqual( + parse_cve_from_filename("CVE-2022-12345.patch"),"CVE-2022-
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
12345"
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ ) + + # Patch with multiple CVE IDs + self.assertEqual(Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
+ parse_cve_from_filename("CVE-2022-41741-CVE-2022-
41742.patch"),meta/lib/oeqa/selftest/cases/cve_check.py +6
-0+ "CVE-2022-41742", + ) + + # Patches with CVE ID and appended text + self.assertEqual( + parse_cve_from_filename("CVE-2023-3019-0001.patch"),"CVE-
Quoted 11 lines
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 4 lines
2023-3019"
meta/lib/oeqa/selftest/cases/cve_check.py +3
-0+ ) + self.assertEqual( + parse_cve_from_filename("CVE-2024-21886-1.patch"), "CVE-2024-
Quoted 6 lines
Quoted 6 lines
21886"
meta/lib/oeqa/selftest/cases/cve_check.py +5
-0+ ) + + # Patch with CVE ID and prepended text + self.assertEqual( + parse_cve_from_filename("grep-CVE-2012-5667.patch"),"CVE-
Quoted 5 lines
Quoted 5 lines
2012-
Quoted 4 lines
5667"
meta/lib/oeqa/selftest/cases/cve_check.py +3
-0+ ) + self.assertEqual( + parse_cve_from_filename("0001-CVE-2012-5667.patch"),"CVE-
Quoted 15 lines
Quoted 15 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 7 lines
2012-5667"
meta/lib/oeqa/selftest/cases/cve_check.py +6
-0+ ) + + # Patch with CVE ID and both prepended and appended text + self.assertEqual( + parse_cve_from_filename( + "0001-tpm2_import-fix-fixed-AES-key-CVE-2021-3565-0001.patch"
Quoted 7 lines
meta/lib/oeqa/selftest/cases/cve_check.py +7
-0+ ), + "CVE-2021-3565", + ) + + # Only grab the last CVE ID in the filename + self.assertEqual( + parse_cve_from_filename("CVE-2012-5667-CVE-2012-Quoted 5 lines
Quoted 5 lines
5668.patch"),
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ "CVE-2012-5668", + ) + + # Test invalid CVE ID with incorrect length (must be atleast 4 digits)
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ self.assertEqual( + parse_cve_from_filename("CVE-2024-001.patch"), + "", + ) + + # Test valid CVE ID with very long length + self.assertEqual( + parse_cve_from_filename("CVE-2024-0000000000000000000000001.patch"),
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ "CVE-2024-0000000000000000000000001", + ) + + def test_parse_cve_from_patch_contents(self):Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ import textwrap + from oe.cve_check import parse_cves_from_patch_contents + + # Standard patch file excerpt without any patches + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + remove "*" for root since we don't have a /etc/shadowso far.
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
meta/lib/oeqa/selftest/cases/cve_check.py +15
-0+ + Upstream-Status: Inappropriate [configuration] + + Signed-off-by: Scott Garman <[email protected]> + + --- base-passwd/passwd.master~nobash + +++ base-passwd/passwd.master + @@ -1,4 +1,4 @@ + -root:*:0:0:root:/root:/bin/sh + +root::0:0:root:/root:/bin/sh + daemon:*:1:1:daemon:/usr/sbin:/bin/sh + bin:*:2:2:bin:/bin:/bin/sh + sys:*:3:3:sys:/dev:/bin/sh + """) + ),Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ set(), + ) + + # Patch file with multiple CVE IDs (space-separated) + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function
_cairo_arc_in_direction().Quoted 10 lines
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py +10
-0+ + CVE: CVE-2019-6461 CVE-2019-6462 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction(
cairo_t*cr,Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI *
MAX_FULL_CIRCLES) {Quoted 1 line
Quoted 1 line
Quoted 1 line
+
angle_max= fmod (angle_max-angle_min, 2 *
M_PI);Quoted 6 lines
Quoted 6 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 6 lines
meta/lib/oeqa/selftest/cases/cve_check.py +6
-0+ """), + ), + {"CVE-2019-6461", "CVE-2019-6462"}, + ) + + # Patch file with multiple CVE IDs (comma-separated w/ bothspace
Quoted 6 lines
and
Quoted 5 lines
Quoted 5 lines
no space)
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function
_cairo_arc_in_direction().Quoted 10 lines
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py +10
-0+ + CVE: CVE-2019-6461,CVE-2019-6462, CVE-2019-6463 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction(
cairo_t*cr,Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI *
MAX_FULL_CIRCLES) {Quoted 1 line
Quoted 1 line
Quoted 1 line
+
angle_max= fmod (angle_max-angle_min, 2 *
M_PI);Quoted 11 lines
Quoted 11 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 11 lines
meta/lib/oeqa/selftest/cases/cve_check.py +11
-0+ + """), + ), + {"CVE-2019-6461", "CVE-2019-6462", "CVE-2019-6463"}, + ) + + # Patch file with multiple CVE IDs (&-separated) + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function
_cairo_arc_in_direction().Quoted 10 lines
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py +10
-0+ + CVE: CVE-2019-6461 & CVE-2019-6462 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction(
cairo_t*cr,Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI *
MAX_FULL_CIRCLES) {Quoted 1 line
Quoted 1 line
Quoted 1 line
+
angle_max= fmod (angle_max-angle_min, 2 *
M_PI);Quoted 10 lines
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py +10
-0+ """), + ), + {"CVE-2019-6461", "CVE-2019-6462"}, + ) + + # Patch file with multiple lines with CVE IDs + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function
_cairo_arc_in_direction().Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ + CVE: CVE-2019-6461 & CVE-2019-6462 + + CVE: CVE-2019-6463 & CVE-2019-6464Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction(
cairo_t*cr,Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI *
MAX_FULL_CIRCLES) {Quoted 1 line
Quoted 1 line
Quoted 1 line
+
angle_max= fmod (angle_max-angle_min, 2 *
M_PI);Quoted 4 lines
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ + """), + ), + {"CVE-2019-6461", "CVE-2019-6462", "CVE-2019-6463","CVE-
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
2019-6464"},
+ )def test_recipe_report_json(self): config = """--
2.34.1
Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
Actually, the test I added was wrong, I was asserting equal to set("CVE-2023-33460") instead of {"CVE-2023-33460"} in my test case.
Back to square one...
Colin
Quoted 695 lines · Colin
Hi Peter,
I'm currently looking at this. I added CVE-2023-33460.patch to
test_parse_cve_from_patch_contentsand ran oe-selftest. I see the issue why these patches are no longer being marked as solved.2025-01-11 15:17:33,396 - oe-selftest - INFO - ====================================================================== 2025-01-11 15:17:33,396 - oe-selftest - INFO - FAIL: test_parse_cve_from_patch_contents (cve_check.CVECheck) 2025-01-11 15:17:33,397 - oe-selftest - INFO - ---------------------------------------------------------------------- 2025-01-11 15:17:33,397 - oe-selftest - INFO - Traceback (most recent calllast):
File
"/home/colin/development/yocto/poky/meta/lib/oeqa/selftest/cases/cve_check.py", line 209, intest_parse_cve_from_patch_contentsself.assertEqual(AssertionError: Items in the first set but not the second:
'CVE-2023-33460'
Items in the second set but not the first:
'E'
'2'
'V'
'C'
'-'
'0'
'4'
'3'
'6'2025-01-11 15:17:33,397 - oe-selftest - INFO - ---------------------------------------------------------------------- 2025-01-11 15:17:33,397 - oe-selftest - INFO - Ran 1 test in 0.247s 2025-01-11 15:17:33,398 - oe-selftest - INFO - FAILED 2025-01-11 15:17:33,398 - oe-selftest - INFO - (failures=1) 2025-01-11 15:17:46,988 - oe-selftest - INFO - RESULTS: 2025-01-11 15:17:46,988 - oe-selftest - INFO - RESULTS - cve_check.CVECheck.test_parse_cve_from_patch_contents: FAILED (0.02s) 2025-01-11 15:17:46,996 - oe-selftest - INFO - SUMMARY: 2025-01-11 15:17:46,996 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 0.249s 2025-01-11 15:17:46,996 - oe-selftest - INFO - oe-selftest - FAIL -Required tests failed (successes=0, skipped=0, failures=1, errors=0)
For some reason the CVE string itself is getting put into a set, instead of the full string being placed into a set. I'll track down where this is happening and push up a fix, including adding this update to the test.
Not sure why I didn't add a test case with a single CVE ID in the patch contents...This was my mistake and I apologize. Sorry for any inconveniences I caused.
Best,
ColinQuoted 636 lines · Peter
Looking at the reopened CVEs in meta-oe, I can't understand why are some reported.
$ grep -rI CVE-2023-33460 ./ ./meta-oe/recipes-devtools/yajl/yajl/CVE-2023-33460.patch:CVE: CVE-2023-33460 ./meta-oe/recipes-devtools/yajl/yajl_2.1.0.bb: file://CVE-2023-33460.patch \ $ grep -rI CVE-2020-29074 ./ ./meta-oe/recipes-graphics/x11vnc/files/CVE-2020-29074.patch:CVE: CVE-2020-29074 ./meta-oe/recipes-graphics/x11vnc/x11vnc_0.9.16.bb: file://CVE-2020-29074.patch \These look good to me, but they are not matched anymore and show as open.
Peter
Quoted 10 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Peter Marko via lists.openembedded.org
Sent: Friday, January 10, 2025 10:21
To: [email protected]; [email protected]
Subject: Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsingOr should we rather rename all those patches which violate current code? What is then the meaning of CVE: tag within the patch if it's not used
but it's
Quoted 6 lines
enforced?
We should probably removecve_payload_taghandling fromtest_cve_tag_format...Peter
Quoted 16 lines
Quoted 6 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Peter Marko via lists.openembedded.org
Sent: Friday, January 10, 2025 9:34
To: [email protected]; openembedded-Quoted 9 lines
Subject: Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
Hello,
This patch has caused a regression in detecting cve patches. Master jumped by 32 CVEs after merging this commit. These are for sure patched but it's not detected anymore. See https://valkyrie.yocto.io/pub/non-release/patchmetrics/
(master should be identical to scarthgap/styhead except for kernelCVEs)
Quoted 15 lines
Quoted 15 lines
Reports cannot be used anymore for picking CVEs for TODO list. So this patch should be reverted or fixed.
Peter
Quoted 9 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Colin McAllister via lists.openembedded.org
Sent: Monday, December 30, 2024 20:22
To: [email protected]
Cc: Colin McAllister <[email protected]>
Subject: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsingQuoted 1 line
Quoted 1 line
Quoted 1 line
The
cve_checkfunctionality to parse CVE IDs from the patchfilename and
Quoted 1 line
Quoted 1 line
Quoted 1 line
patch contents have been reworked to improve parsing and also
utilize
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
tests. This ensures that the parsing works as intended.
Additionally, the new
patched_cvesdict has a few issues I tried tofix
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
as well. If multiple patch files exist for a single CVE ID, only the last one will show up with the "resource" key. The value for the "resource" key has been updated to hold a list and return all patch files associated with a given CVE ID. Also, at the end of
get_patch_cves,CVE_STATUScan overwrite an existing entry in thedict.
Quoted 1 line
Quoted 1 line
Quoted 1 line
This could cause an issue, for example, if a CVE has been addressed
via
Quoted 1 line
Quoted 1 line
Quoted 1 line
a patch, but a
CVE_STATUSline also exists that ignores the givenCVE
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
ID. A warning has been added if this ever happens.
Signed-off-by Colin McAllister <[email protected]>---
I noticed that there are some patches, especially in older verisons
of
Quoted 1 line
Quoted 1 line
Quoted 1 line
Yocto, where the "CVE: " tag was used with multiple CVE IDs in
different
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
formats, like "CVE-YYYY-XXXX & CVE-YYYY-XXXX" or "CVE-YYYY-XXXX, CVE-YYYY-XXXX". Currently, only space-delimited CVE IDs will be parsed, but documentation doesn't indicate that is the
only
Quoted 1 line
Quoted 1 line
Quoted 1 line
supported format. I figured it'd be nice to update the code to be
able
Quoted 1 line
Quoted 1 line
Quoted 1 line
to support multiple formats, that way this patch could be
backported to
Quoted 1 line
Quoted 1 line
Quoted 1 line
fix those patches. I also wanted to add unit tests to ensure the
patch
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
parsing behavior is preserved.
I'd also like to update the patch filename parsing to parse
multiple CVE
Quoted 1 line
Quoted 1 line
Quoted 1 line
IDs from the filename, but based on the comments, it seems like
there
Quoted 1 line
Quoted 1 line
Quoted 1 line
was a reason why only the last CVE ID is extracted from the
filename.
Quoted 1 line
Quoted 1 line
Quoted 1 line
I'd be happy to submit a V2 patch or an additional patch to update
the
Quoted 39 lines
Quoted 39 lines
Quoted 39 lines
function if that sounds good for the maintainers.
V2 Changes:
I mistakenly created this patch without fb3f440 applied. I updatedget_patched_cvesto return a dict instead of a set.I also improved the docstrings for these new functions to be more descriptive and also specify the return types.
I also found a few issues with the dictionary object created by
get_patched_cvesthat have now been addressed in this commit.meta/lib/oe/cve_check.py | 166 ++++++++++++------ meta/lib/oeqa/selftest/cases/cve_check.py | 205 ++++++++++++++++++++++ 2 files changed, 317 insertions(+), 54 deletions(-)meta/lib/oe/cve_check.py +4
-2@@ -5,9 +5,11 @@ # import collections -import re -import itertools import functools +import itertools +import os.path +import re +import oe.patch _Version = collections.namedtuple( "_Version", ["release", "patch_l", "pre_l", "pre_v"]@@ -71,76 +73,132 @@ def _cmpkey(release, patch_l, pre_l, pre_v): return _release, _patch, _preQuoted 7 lines
Quoted 7 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 7 lines
meta/lib/oe/cve_check.py +4
-2-def get_patched_cves(d): +def parse_cve_from_filename(patch_filename): """ - Get patches that solve CVEs using the "CVE: " tag. + Parses CVE ID from the filename + + Matches the last "CVE-YYYY-ID" in the file name, also ifwritten
Quoted 2 lines
Quoted 2 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 82
Quoted 2 lines
meta/lib/oe/cve_check.py +2
-0+ in lowercase. Possible to have multiple CVE IDs in a single + file name, but only the last one will be detected from thefile name.
Quoted 2 lines
Quoted 2 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 84
Quoted 2 lines
meta/lib/oe/cve_check.py +2
-0+ + Returns the last CVE ID foudn in the filename. If no CVE ID isfound
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
meta/lib/oe/cve_check.py +5
-2+ an empty string is returned. """ + cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d{4,})", re.IGNORECASE) - import re - import oe.patch + # Check patch file name for CVE ID + fname_match = cve_file_name_match.search(patch_filename) + return fname_match.group(1).upper() if fname_match else ""Quoted 3 lines
Quoted 3 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 3 lines
meta/lib/oe/cve_check.py +0
-2- cve_match = re.compile(r"CVE:( CVE-\d{4}-\d+)+") - # Matches the last "CVE-YYYY-ID" in the file name, also ifwritten
Quoted 2 lines
Quoted 2 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 87
Quoted 2 lines
meta/lib/oe/cve_check.py +0
-2- # in lowercase. Possible to have multiple CVE IDs in a single - # file name, but only the last one will be detected from thefile name.
Quoted 10 lines
Quoted 10 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 88
Quoted 1 line
meta/lib/oe/cve_check.py +0
-1- # However, patch files contents addressing multiple CVE IDs aresupported
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 90
Quoted 2 lines
meta/lib/oe/cve_check.py +0
-2- # (cve_match regular expression) - cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d+)",re.IGNORECASE)
Quoted 5 lines
meta/lib/oe/cve_check.py +4
-0+def parse_cves_from_patch_contents(patch_contents): + """ + Parses CVE IDs from patch contents + Matches all CVE IDs contained on a line that starts with "CVE:". Any
Quoted 1 line
Quoted 1 line
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 100
Quoted 1 line
meta/lib/oe/cve_check.py +1
-0+ delimiter (',', '&', "and", etc.) can be used without anyissues. Multiple
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
meta/lib/oe/cve_check.py +9
-0+ "CVE:" lines can also exist. + + Returns a set of all CVE IDs found in the patch contents. + """ + cve_ids = set() + cve_match = re.compile(r"CVE-\d{4}-\d{4,}") + # Search for one or more "CVE: " lines + for line in patch_contents.split("\n"): + if not line.startswith("CVE:"):Quoted 8 lines
Quoted 8 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 8 lines
meta/lib/oe/cve_check.py +8
-0+ continue + cve_ids.update(cve_match.findall(line)) + return cve_ids + + +def parse_cves_from_patch_file(patch_file): + """ + Parses CVE IDs associated with a particular patch file, usingboth the
Quoted 4 lines
Quoted 4 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 4 lines
filename
meta/lib/oe/cve_check.py +3
-0+ and patch contents. + + Returns a set of all CVE IDs found in the patch filename andcontents.
Quoted 5 lines
Quoted 5 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 5 lines
meta/lib/oe/cve_check.py +5
-0+ """ + cve_ids = set() + filename_cve = parse_cve_from_filename(patch_file) + if filename_cve: + bb.debug(2, "Found %s from patch file name %s" %(
filename_cve,Quoted 4 lines
Quoted 4 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 4 lines
patch_file))meta/lib/oe/cve_check.py +3
-0+ cve_ids.add(parse_cve_from_filename(patch_file)) + + # Remote patches won't be present and compressed patches won'tbe
Quoted 3 lines
Quoted 3 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 3 lines
meta/lib/oe/cve_check.py +3
-0+ # unpacked, so say we're not scanning them + if not os.path.isfile(patch_file): + bb.note("%s is remote or compressed, not scanning content"%
Quoted 36 lines
Quoted 22 lines
Quoted 22 lines
patch_file)meta/lib/oe/cve_check.py +21
-0+ return cve_ids + + with open(patch_file, "r", encoding="utf-8") as f: + try: + patch_text = f.read() + except UnicodeDecodeError: + bb.debug( + 1, + "Failed to read patch %s using UTF-8 encoding" + " trying with iso8859-1" % patch_file, + ) + f.close() + with open(patch_file, "r", encoding="iso8859-1") as f: + patch_text = f.read() + + cve_ids.update(parse_cves_from_patch_contents(patch_text)) + + if not cve_ids: + bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + else: + bb.debug(2, "Patch %s solves %s" % (patch_file, ",Quoted 8 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 8 lines
".join(sorted(cve_ids))))
meta/lib/oe/cve_check.py +7
-0+ + return cve_ids + + +def get_patched_cves(d): + """ + Determines the CVE IDs that have been solved by either patchesincuded
Quoted 4 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 4 lines
within
meta/lib/oe/cve_check.py +3
-0+ SRC_URI or by setting CVE_STATUS. + + Returns a dictionary with the CVE IDs as keys and an associateddictonary
Quoted 16 lines
Quoted 8 lines
Quoted 8 lines
of
meta/lib/oe/cve_check.py +4
-0+ relevant metadata as the value. + """ patched_cves = {} patches = oe.patch.src_patches(d) bb.debug(2, "Scanning %d patches for CVEs" % len(patches)) + + # Check each patch fileQuoted 8 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 8 lines
for url in patches:
patch_file= bb.fetch.decodeurl(url)[2]meta/lib/oe/cve_check.py +0
-6- - # Check patch file name for CVE ID - fname_match = cve_file_name_match.search(patch_file) - if fname_match: - cve = fname_match.group(1).upper() - patched_cves[cve] = {"abbrev-status": "Patched","status": "fix-file-
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
included", "resource":
patch_file}
- bb.debug(2, "Found %s from patch file name %s" % (cve,
patch_file))Quoted 2 lines
Quoted 2 lines
meta/lib/oe/cve_check.py +0
-2- - # Remote patches won't be present and compressed patcheswon't be
Quoted 3 lines
Quoted 3 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 3 lines
meta/lib/oe/cve_check.py +0
-3- # unpacked, so say we're not scanning them - if not os.path.isfile(patch_file): - bb.note("%s is remote or compressed, not scanningcontent" %
Quoted 8 lines
Quoted 8 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 8 lines
patch_file)meta/lib/oe/cve_check.py +0
-7- continue - - with open(patch_file, "r", encoding="utf-8") as f: - try: - patch_text = f.read() - except UnicodeDecodeError: - bb.debug(1, "Failed to read patch %s using UTF-8encoding"
Quoted 3 lines
Quoted 3 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 3 lines
meta/lib/oe/cve_check.py +0
-3- " trying with iso8859-1" % patch_file) - f.close() - with open(patch_file, "r", encoding="iso8859-1")as f:
Quoted 9 lines
Quoted 9 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 9 lines
meta/lib/oe/cve_check.py +0
-9- patch_text = f.read() - - # Search for one or more "CVE: " lines - text_match = False - for match in cve_match.finditer(patch_text): - # Get only the CVEs without the "CVE: " tag - cves = patch_text[match.start()+5:match.end()] - for cve in cves.split(): - bb.debug(2, "Patch %s solves %s" % (patch_file,cve))
Quoted 1 line
Quoted 1 line
Quoted 1 line
- patched_cves[cve] = {"abbrev-status": "Patched",
"status": "fix-
Quoted 28 lines
file-
Quoted 27 lines
Quoted 27 lines
included", "resource":
patch_file}meta/lib/oe/cve_check.py +15
-8- text_match = True - - if not fname_match and not text_match: - bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + for cve_id in parse_cves_from_patch_file(patch_file): + if cve_id not in patched_cves: + { + "abbrev-status": "Patched", + "status": "fix-file-included", + "resource": [patch_file], + } + else: + patched_cves[cve_id]["resource"].append(patch_file) # Search for additional patched CVEs - for cve in (d.getVarFlags("CVE_STATUS") or {}): - decoded_status = decode_cve_status(d, cve) + for cve_id in d.getVarFlags("CVE_STATUS") or {}: + decoded_status = decode_cve_status(d, cve_id) products = d.getVar("CVE_PRODUCT") - if has_cve_product_match(decoded_status, products) == True: - patched_cves[cve] = { + if has_cve_product_match(decoded_status, products): + if cve_id in patched_cves: + bb.warn( + 'CVE_STATUS[%s] = "%s" is overwriting previousstatus of "%s:
Quoted 17 lines
Quoted 17 lines
%s"'
Quoted 16 lines
meta/lib/oe/cve_check.py +9
-1+ % ( + cve_id, + d.getVarFlag("CVE_STATUS", cve_id), + patched_cves[cve_id]["abbrev-status"], + patched_cves[cve_id]["status"], + ) + ) + patched_cves[cve_id] = { "abbrev-status": decoded_status["mapping"], "status": decoded_status["detail"], "justification": decoded_status["description"], "affected-vendor": decoded_status["vendor"], - "affected-product": decoded_status["product"] + "affected-product": decoded_status["product"], }Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase): · line 120
Quoted 8 lines
return
patched_cvesmeta/lib/oeqa/selftest/cases/cve_check.py +0
-0@@ -120,6 +120,211 @@ class CVECheck(OESelftestTestCase): self.assertEqual(has_cve_product_match(status, "testglibca:glibc"),
Quoted 2 lines
Quoted 2 lines
True)
Quoted 1 line
meta/lib/oeqa/selftest/cases/cve_check.py +0
-0self.assertEqual(has_cve_product_match(status,"glibca:glibc test"),
Quoted 7 lines
Quoted 7 lines
True)
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase): · line 128
Quoted 6 lines
meta/lib/oeqa/selftest/cases/cve_check.py +5
-0+ def test_parse_cve_from_patch_filename(self): + from oe.cve_check import parse_cve_from_filename + + # Patch filename without CVE ID +
self.assertEqual(parse_cve_from_filename("0001-test.patch"), "")Quoted 4 lines
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ + # Patch with single CVE ID + self.assertEqual( + parse_cve_from_filename("CVE-2022-12345.patch"),"CVE-2022-
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
12345"
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ ) + + # Patch with multiple CVE IDs + self.assertEqual(Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
+ parse_cve_from_filename("CVE-2022-41741-CVE-2022-
41742.patch"),meta/lib/oeqa/selftest/cases/cve_check.py +6
-0+ "CVE-2022-41742", + ) + + # Patches with CVE ID and appended text + self.assertEqual( + parse_cve_from_filename("CVE-2023-3019-0001.patch"),"CVE-
Quoted 4 lines
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 4 lines
2023-3019"
meta/lib/oeqa/selftest/cases/cve_check.py +3
-0+ ) + self.assertEqual( + parse_cve_from_filename("CVE-2024-21886-1.patch"),"CVE-
Quoted 7 lines
2024-
Quoted 6 lines
Quoted 6 lines
21886"
meta/lib/oeqa/selftest/cases/cve_check.py +5
-0+ ) + + # Patch with CVE ID and prepended text + self.assertEqual( + parse_cve_from_filename("grep-CVE-2012-5667.patch"),"CVE-
Quoted 5 lines
Quoted 5 lines
2012-
Quoted 4 lines
5667"
meta/lib/oeqa/selftest/cases/cve_check.py +3
-0+ ) + self.assertEqual( + parse_cve_from_filename("0001-CVE-2012-5667.patch"),"CVE-
Quoted 15 lines
Quoted 15 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 7 lines
2012-5667"
meta/lib/oeqa/selftest/cases/cve_check.py +6
-0+ ) + + # Patch with CVE ID and both prepended and appended text + self.assertEqual( + parse_cve_from_filename( + "0001-tpm2_import-fix-fixed-AES-key-CVE-2021-3565-0001.patch"
Quoted 7 lines
meta/lib/oeqa/selftest/cases/cve_check.py +7
-0+ ), + "CVE-2021-3565", + ) + + # Only grab the last CVE ID in the filename + self.assertEqual( + parse_cve_from_filename("CVE-2012-5667-CVE-2012-Quoted 5 lines
Quoted 5 lines
5668.patch"),
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ "CVE-2012-5668", + ) + + # Test invalid CVE ID with incorrect length (must be atleast 4 digits)
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ self.assertEqual( + parse_cve_from_filename("CVE-2024-001.patch"), + "", + ) + + # Test valid CVE ID with very long length + self.assertEqual( + parse_cve_from_filename("CVE-2024-0000000000000000000000001.patch"),
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ "CVE-2024-0000000000000000000000001", + ) + + def test_parse_cve_from_patch_contents(self):Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ import textwrap + from oe.cve_check import parse_cves_from_patch_contents + + # Standard patch file excerpt without any patches + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + remove "*" for root since we don't have a /etc/shadowso far.
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
meta/lib/oeqa/selftest/cases/cve_check.py +15
-0+ + Upstream-Status: Inappropriate [configuration] + + Signed-off-by: Scott Garman <[email protected]> + + --- base-passwd/passwd.master~nobash + +++ base-passwd/passwd.master + @@ -1,4 +1,4 @@ + -root:*:0:0:root:/root:/bin/sh + +root::0:0:root:/root:/bin/sh + daemon:*:1:1:daemon:/usr/sbin:/bin/sh + bin:*:2:2:bin:/bin:/bin/sh + sys:*:3:3:sys:/dev:/bin/sh + """) + ),Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ set(), + ) + + # Patch file with multiple CVE IDs (space-separated) + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function
_cairo_arc_in_direction().Quoted 10 lines
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py +10
-0+ + CVE: CVE-2019-6461 CVE-2019-6462 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction(
cairo_t*cr,Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI *
MAX_FULL_CIRCLES) {Quoted 1 line
Quoted 1 line
Quoted 1 line
+
angle_max= fmod (angle_max-angle_min, 2 *
M_PI);Quoted 6 lines
Quoted 6 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 6 lines
meta/lib/oeqa/selftest/cases/cve_check.py +6
-0+ """), + ), + {"CVE-2019-6461", "CVE-2019-6462"}, + ) + + # Patch file with multiple CVE IDs (comma-separated w/both space
Quoted 6 lines
and
Quoted 5 lines
Quoted 5 lines
no space)
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function
_cairo_arc_in_direction().Quoted 10 lines
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py +10
-0+ + CVE: CVE-2019-6461,CVE-2019-6462, CVE-2019-6463 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction(
cairo_t*cr,Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI *
MAX_FULL_CIRCLES) {Quoted 1 line
Quoted 1 line
Quoted 1 line
+
angle_max= fmod (angle_max-angle_min, 2 *
M_PI);Quoted 11 lines
Quoted 11 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 11 lines
meta/lib/oeqa/selftest/cases/cve_check.py +11
-0+ + """), + ), + {"CVE-2019-6461", "CVE-2019-6462", "CVE-2019-6463"}, + ) + + # Patch file with multiple CVE IDs (&-separated) + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function
_cairo_arc_in_direction().Quoted 10 lines
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py +10
-0+ + CVE: CVE-2019-6461 & CVE-2019-6462 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction(
cairo_t*cr,Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI *
MAX_FULL_CIRCLES) {Quoted 1 line
Quoted 1 line
Quoted 1 line
+
angle_max= fmod (angle_max-angle_min, 2 *
M_PI);Quoted 10 lines
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 10 lines
meta/lib/oeqa/selftest/cases/cve_check.py +10
-0+ """), + ), + {"CVE-2019-6461", "CVE-2019-6462"}, + ) + + # Patch file with multiple lines with CVE IDs + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function
_cairo_arc_in_direction().Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ + CVE: CVE-2019-6461 & CVE-2019-6462 + + CVE: CVE-2019-6463 & CVE-2019-6464Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction(
cairo_t*cr,Quoted 8 lines
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI *
MAX_FULL_CIRCLES) {Quoted 1 line
Quoted 1 line
Quoted 1 line
+
angle_max= fmod (angle_max-angle_min, 2 *
M_PI);Quoted 4 lines
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 4 lines
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ + """), + ), + {"CVE-2019-6461", "CVE-2019-6462", "CVE-2019-6463","CVE-
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
2019-6464"},
+ )def test_recipe_report_json(self): config = """--
2.34.1
RE: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
This patch fixes this problem.
Sorry for my additional spam.
https://lists.openembedded.org/g/openembedded-core/message/209573
Peter
From: Colin <[email protected]>
Sent: Saturday, January 11, 2025 16:40
To: Marko, Peter (FT D EU SK BFS1) <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
Actually, the test I added was wrong, I was asserting equal to set("CVE-2023-33460") instead of {"CVE-2023-33460"} in my test case.
Back to square one...
Colin
On Sat, Jan 11, 2025 at 9:25 AM Colin <[email protected]<mailto:[email protected]>> wrote:
Hi Peter,
I'm currently looking at this. I added CVE-2023-33460.patch to test_parse_cve_from_patch_contents and ran oe-selftest. I see the issue why these patches are no longer being marked as solved.
2025-01-11 15:17:33,396 - oe-selftest - INFO - ======================================================================
2025-01-11 15:17:33,396 - oe-selftest - INFO - FAIL: test_parse_cve_from_patch_contents (cve_check.CVECheck)
2025-01-11 15:17:33,397 - oe-selftest - INFO - ----------------------------------------------------------------------
2025-01-11 15:17:33,397 - oe-selftest - INFO - Traceback (most recent call last):
File "/home/colin/development/yocto/poky/meta/lib/oeqa/selftest/cases/cve_check.py", line 209, in test_parse_cve_from_patch_contents
self.assertEqual(AssertionError: Items in the first set but not the second:
'CVE-2023-33460'
Items in the second set but not the first:
'E'
'2'
'V'
'C'
'-'
'0'
'4'
'3'
'6'
2025-01-11 15:17:33,397 - oe-selftest - INFO - ---------------------------------------------------------------------- 2025-01-11 15:17:33,397 - oe-selftest - INFO - Ran 1 test in 0.247s 2025-01-11 15:17:33,398 - oe-selftest - INFO - FAILED 2025-01-11 15:17:33,398 - oe-selftest - INFO - (failures=1) 2025-01-11 15:17:46,988 - oe-selftest - INFO - RESULTS: 2025-01-11 15:17:46,988 - oe-selftest - INFO - RESULTS - cve_check.CVECheck.test_parse_cve_from_patch_contents: FAILED (0.02s) 2025-01-11 15:17:46,996 - oe-selftest - INFO - SUMMARY: 2025-01-11 15:17:46,996 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 0.249s 2025-01-11 15:17:46,996 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (successes=0, skipped=0, failures=1, errors=0)
For some reason the CVE string itself is getting put into a set, instead of the full string being placed into a set. I'll track down where this is happening and push up a fix, including adding this update to the test.
Not sure why I didn't add a test case with a single CVE ID in the patch contents...This was my mistake and I apologize. Sorry for any inconveniences I caused.
Best,
Colin
On Fri, Jan 10, 2025 at 3:26 AM Marko, Peter <[email protected]<mailto:[email protected]>> wrote:
Looking at the reopened CVEs in meta-oe, I can't understand why are some reported.
$ grep -rI CVE-2023-33460 ./ ./meta-oe/recipes-devtools/yajl/yajl/CVE-2023-33460.patch:CVE: CVE-2023-33460 ./meta-oe/recipes-devtools/yajl/yajl_2.1.0.bb<http://yajl_2.1.0.bb>: file://CVE-2023-33460.patch \ $ grep -rI CVE-2020-29074 ./ ./meta-oe/recipes-graphics/x11vnc/files/CVE-2020-29074.patch:CVE: CVE-2020-29074 ./meta-oe/recipes-graphics/x11vnc/x11vnc_0.9.16.bb<http://x11vnc_0.9.16.bb>: file://CVE-2020-29074.patch \
These look good to me, but they are not matched anymore and show as open.
Peter
Quoted 550 lines
-----Original Message-----
From: [email protected]<mailto:[email protected]> <openembedded- [email protected]<mailto:[email protected]>> On Behalf Of Peter Marko via lists.openembedded.org<http://lists.openembedded.org>
Sent: Friday, January 10, 2025 10:21
To: [email protected]<mailto:[email protected]>; [email protected]<mailto:[email protected]>
Subject: Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsingOr should we rather rename all those patches which violate current code? What is then the meaning of CVE: tag within the patch if it's not used but it's enforced?
We should probably removecve_payload_taghandling fromtest_cve_tag_format...Peter
Quoted 6 lines
-----Original Message-----
From: [email protected]<mailto:[email protected]> <openembedded- [email protected]<mailto:[email protected]>> On Behalf Of Peter Marko via lists.openembedded.org<http://lists.openembedded.org>
Sent: Friday, January 10, 2025 9:34
To: [email protected]<mailto:[email protected]>; openembedded-[email protected]<mailto:[email protected]>
Quoted 187 lines
Subject: Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
Hello,
This patch has caused a regression in detecting cve patches. Master jumped by 32 CVEs after merging this commit.
These are for sure patched but it's not detected anymore. See https://valkyrie.yocto.io/pub/non-release/patchmetrics/
(master should be identical to scarthgap/styhead except for kernel CVEs)Reports cannot be used anymore for picking CVEs for TODO list. So this patch should be reverted or fixed.
Peter
Quoted 97 lines
-----Original Message-----
From: [email protected]<mailto:[email protected]> <openembedded- [email protected]<mailto:[email protected]>> On Behalf Of Colin McAllister via lists.openembedded.org<http://lists.openembedded.org>
Sent: Monday, December 30, 2024 20:22
To: [email protected]<mailto:[email protected]>
Cc: Colin McAllister <[email protected]<mailto:[email protected]>>
Subject: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsingThe
cve_checkfunctionality to parse CVE IDs from the patch filename and patch contents have been reworked to improve parsing and also utilize tests. This ensures that the parsing works as intended.Additionally, the new
patched_cvesdict has a few issues I tried to fix as well. If multiple patch files exist for a single CVE ID, only the last one will show up with the "resource" key. The value for the "resource" key has been updated to hold a list and return all patch files associated with a given CVE ID. Also, at the end ofget_patch_cves,CVE_STATUScan overwrite an existing entry in the dict. This could cause an issue, for example, if a CVE has been addressed via a patch, but aCVE_STATUSline also exists that ignores the given CVE ID. A warning has been added if this ever happens.---
I noticed that there are some patches, especially in older verisons of Yocto, where the "CVE: " tag was used with multiple CVE IDs in different formats, like "CVE-YYYY-XXXX & CVE-YYYY-XXXX" or "CVE-YYYY-XXXX, CVE-YYYY-XXXX". Currently, only space-delimited CVE IDs will be parsed, but documentation doesn't indicate that is the only supported format. I figured it'd be nice to update the code to be able to support multiple formats, that way this patch could be backported to fix those patches. I also wanted to add unit tests to ensure the patch parsing behavior is preserved.
I'd also like to update the patch filename parsing to parse multiple CVE IDs from the filename, but based on the comments, it seems like there was a reason why only the last CVE ID is extracted from the filename. I'd be happy to submit a V2 patch or an additional patch to update the function if that sounds good for the maintainers.
V2 Changes:
I mistakenly created this patch without fb3f440 applied. I updatedget_patched_cvesto return a dict instead of a set.I also improved the docstrings for these new functions to be more descriptive and also specify the return types.
I also found a few issues with the dictionary object created by
get_patched_cvesthat have now been addressed in this commit.meta/lib/oe/cve_check.py | 166 ++++++++++++------ meta/lib/oeqa/selftest/cases/cve_check.py | 205 ++++++++++++++++++++++ 2 files changed, 317 insertions(+), 54 deletions(-)meta/lib/oe/cve_check.py +14
-6@@ -5,9 +5,11 @@ # import collections -import re -import itertools import functools +import itertools +import os.path +import re +import oe.patch _Version = collections.namedtuple( "_Version", ["release", "patch_l", "pre_l", "pre_v"]@@ -71,76 +73,132 @@ def _cmpkey(release, patch_l, pre_l, pre_v): return _release, _patch, _pre -def get_patched_cves(d): +def parse_cve_from_filename(patch_filename): """ - Get patches that solve CVEs using the "CVE: " tag. + Parses CVE ID from the filename + + Matches the last "CVE-YYYY-ID" in the file name, also if written + in lowercase. Possible to have multiple CVE IDs in a single + file name, but only the last one will be detected from the file name. + + Returns the last CVE ID foudn in the filename. If no CVE ID is found + an empty string is returned. """ + cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d{4,})", re.IGNORECASE) - import re - import oe.patchmeta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 88
Quoted 10 lines
meta/lib/oe/cve_check.py +3
-5+ # Check patch file name for CVE ID + fname_match = cve_file_name_match.search(patch_filename) + return fname_match.group(1).upper() if fname_match else "" - cve_match = re.compile(r"CVE:( CVE-\d{4}-\d+)+") - # Matches the last "CVE-YYYY-ID" in the file name, also if written - # in lowercase. Possible to have multiple CVE IDs in a single - # file name, but only the last one will be detected from the file name. - # However, patch files contents addressing multiple CVE IDs aresupported
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v): · line 90
Quoted 2 lines
meta/lib/oe/cve_check.py +0
-2- # (cve_match regular expression) - cve_file_name_match = re.compile(r".*(CVE-\d{4}-\d+)",re.IGNORECASE)
Quoted 61 lines
meta/lib/oe/cve_check.py +22
-0+def parse_cves_from_patch_contents(patch_contents): + """ + Parses CVE IDs from patch contents + Matches all CVE IDs contained on a line that starts with "CVE: ". Any + delimiter (',', '&', "and", etc.) can be used without any issues. Multiple + "CVE:" lines can also exist. + + Returns a set of all CVE IDs found in the patch contents. + """ + cve_ids = set() + cve_match = re.compile(r"CVE-\d{4}-\d{4,}") + # Search for one or more "CVE: " lines + for line in patch_contents.split("\n"): + if not line.startswith("CVE:"): + continue + cve_ids.update(cve_match.findall(line)) + return cve_ids + + +def parse_cves_from_patch_file(patch_file): + """ + Parses CVE IDs associated with a particular patch file, using both thefilename
meta/lib/oe/cve_check.py +8
-0+ and patch contents. + + Returns a set of all CVE IDs found in the patch filename and contents. + """ + cve_ids = set() + filename_cve = parse_cve_from_filename(patch_file) + if filename_cve: + bb.debug(2, "Found %s from patch file name %s" % (filename_cve,
patch_file))meta/lib/oe/cve_check.py +6
-0+ cve_ids.add(parse_cve_from_filename(patch_file)) + + # Remote patches won't be present and compressed patches won't be + # unpacked, so say we're not scanning them + if not os.path.isfile(patch_file): + bb.note("%s is remote or compressed, not scanning content" %
patch_file)meta/lib/oe/cve_check.py +21
-0+ return cve_ids + + with open(patch_file, "r", encoding="utf-8") as f: + try: + patch_text = f.read() + except UnicodeDecodeError: + bb.debug( + 1, + "Failed to read patch %s using UTF-8 encoding" + " trying with iso8859-1" % patch_file, + ) + f.close() + with open(patch_file, "r", encoding="iso8859-1") as f: + patch_text = f.read() + + cve_ids.update(parse_cves_from_patch_contents(patch_text)) + + if not cve_ids: + bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + else: + bb.debug(2, "Patch %s solves %s" % (patch_file, ",Quoted 8 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 8 lines
".join(sorted(cve_ids))))
meta/lib/oe/cve_check.py +7
-0+ + return cve_ids + + +def get_patched_cves(d): + """ + Determines the CVE IDs that have been solved by either patchesincuded
Quoted 4 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 4 lines
within
meta/lib/oe/cve_check.py +3
-0+ SRC_URI or by setting CVE_STATUS. + + Returns a dictionary with the CVE IDs as keys and an associateddictonary
Quoted 10 lines
Quoted 10 lines
of
meta/lib/oe/cve_check.py +4
-0+ relevant metadata as the value. + """ patched_cves = {} patches = oe.patch.src_patches(d) bb.debug(2, "Scanning %d patches for CVEs" % len(patches)) + + # Check each patch file for url in patches: patch_file = bb.fetch.decodeurl(url)[2]Quoted 8 lines
Quoted 8 lines
meta/lib/oe/cve_check.py +0
-6- - # Check patch file name for CVE ID - fname_match = cve_file_name_match.search(patch_file) - if fname_match: - cve = fname_match.group(1).upper() - patched_cves[cve] = {"abbrev-status": "Patched", "status": "fix-file-included", "resource":
patch_file}
- bb.debug(2, "Found %s from patch file name %s" % (cve,
patch_file))Quoted 18 lines
Quoted 18 lines
meta/lib/oe/cve_check.py +0
-5- - # Remote patches won't be present and compressed patches won't be - # unpacked, so say we're not scanning them - if not os.path.isfile(patch_file): - bb.note("%s is remote or compressed, not scanning content" %
patch_file)meta/lib/oe/cve_check.py +0
-12- continue - - with open(patch_file, "r", encoding="utf-8") as f: - try: - patch_text = f.read() - except UnicodeDecodeError: - bb.debug(1, "Failed to read patch %s using UTF-8 encoding" - " trying with iso8859-1" % patch_file) - f.close() - with open(patch_file, "r", encoding="iso8859-1") as f: - patch_text = f.read() -Quoted 8 lines
meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 8 lines
meta/lib/oe/cve_check.py +0
-8- # Search for one or more "CVE: " lines - text_match = False - for match in cve_match.finditer(patch_text): - # Get only the CVEs without the "CVE: " tag - cves = patch_text[match.start()+5:match.end()] - for cve in cves.split(): - bb.debug(2, "Patch %s solves %s" % (patch_file, cve)) - patched_cves[cve] = {"abbrev-status": "Patched", "status": "fix-file-
Quoted 74 lines
Quoted 17 lines
included", "resource":
patch_file}meta/lib/oe/cve_check.py +9
-5- text_match = True - - if not fname_match and not text_match: - bb.debug(2, "Patch %s doesn't solve CVEs" % patch_file) + for cve_id in parse_cves_from_patch_file(patch_file): + if cve_id not in patched_cves: + { + "abbrev-status": "Patched", + "status": "fix-file-included", + "resource": [patch_file], + } + else: + patched_cves[cve_id]["resource"].append(patch_file) # Search for additional patched CVEs - for cve in (d.getVarFlags("CVE_STATUS") or {}):meta/lib/oe/cve_check.py def _cmpkey(release, patch_l, pre_l, pre_v):
Quoted 10 lines
meta/lib/oe/cve_check.py +6
-3- decoded_status = decode_cve_status(d, cve) + for cve_id in d.getVarFlags("CVE_STATUS") or {}: + decoded_status = decode_cve_status(d, cve_id) products = d.getVar("CVE_PRODUCT") - if has_cve_product_match(decoded_status, products) == True: - patched_cves[cve] = { + if has_cve_product_match(decoded_status, products): + if cve_id in patched_cves: + bb.warn( + 'CVE_STATUS[%s] = "%s" is overwriting previous status of "%s:%s"'
Quoted 23 lines
meta/lib/oe/cve_check.py +9
-1+ % ( + cve_id, + d.getVarFlag("CVE_STATUS", cve_id), + patched_cves[cve_id]["abbrev-status"], + patched_cves[cve_id]["status"], + ) + ) + patched_cves[cve_id] = { "abbrev-status": decoded_status["mapping"], "status": decoded_status["detail"], "justification": decoded_status["description"], "affected-vendor": decoded_status["vendor"], - "affected-product": decoded_status["product"] + "affected-product": decoded_status["product"], } return patched_cvesmeta/lib/oeqa/selftest/cases/cve_check.py +0
-0meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase): · line 120
Quoted 1 line
meta/lib/oeqa/selftest/cases/cve_check.py +0
-0@@ -120,6 +120,211 @@ class CVECheck(OESelftestTestCase): self.assertEqual(has_cve_product_match(status, "test glibca:glibc"),True)
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase): · line 121
Quoted 1 line
meta/lib/oeqa/selftest/cases/cve_check.py +0
-0self.assertEqual(has_cve_product_match(status, "glibca:glibc test"),True)
Quoted 19 lines
meta/lib/oeqa/selftest/cases/cve_check.py +9
-0+ def test_parse_cve_from_patch_filename(self): + from oe.cve_check import parse_cve_from_filename + + # Patch filename without CVE ID + self.assertEqual(parse_cve_from_filename("0001-test.patch"), "") + + # Patch with single CVE ID + self.assertEqual( + parse_cve_from_filename("CVE-2022-12345.patch"), "CVE-2022-12345"
meta/lib/oeqa/selftest/cases/cve_check.py +5
-0+ ) + + # Patch with multiple CVE IDs + self.assertEqual( + parse_cve_from_filename("CVE-2022-41741-CVE-2022-41742.patch"),
meta/lib/oeqa/selftest/cases/cve_check.py +2
-0+ "CVE-2022-41742", + )Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +4
-0+ + # Patches with CVE ID and appended text + self.assertEqual( + parse_cve_from_filename("CVE-2023-3019-0001.patch"), "CVE-2023-3019"
meta/lib/oeqa/selftest/cases/cve_check.py +3
-0+ ) + self.assertEqual( + parse_cve_from_filename("CVE-2024-21886-1.patch"), "CVE-2024-
Quoted 92 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 6 lines
21886"
meta/lib/oeqa/selftest/cases/cve_check.py +5
-0+ ) + + # Patch with CVE ID and prepended text + self.assertEqual( + parse_cve_from_filename("grep-CVE-2012-5667.patch"), "CVE-2012-
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 11 lines
5667"
meta/lib/oeqa/selftest/cases/cve_check.py +3
-0+ ) + self.assertEqual( + parse_cve_from_filename("0001-CVE-2012-5667.patch"), "CVE-2012-5667"
meta/lib/oeqa/selftest/cases/cve_check.py +6
-0+ ) + + # Patch with CVE ID and both prepended and appended text + self.assertEqual( + parse_cve_from_filename( + "0001-tpm2_import-fix-fixed-AES-key-CVE-2021-3565-0001.patch"
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 7 lines
meta/lib/oeqa/selftest/cases/cve_check.py +7
-0+ ), + "CVE-2021-3565", + ) + + # Only grab the last CVE ID in the filename + self.assertEqual( + parse_cve_from_filename("CVE-2012-5667-CVE-2012-5668.patch"),
Quoted 65 lines
meta/lib/oeqa/selftest/cases/cve_check.py +12
-0+ "CVE-2012-5668", + ) + + # Test invalid CVE ID with incorrect length (must be at least 4 digits) + self.assertEqual( + parse_cve_from_filename("CVE-2024-001.patch"), + "", + ) + + # Test valid CVE ID with very long length + self.assertEqual( + parse_cve_from_filename("CVE-2024-0000000000000000000000001.patch"),
meta/lib/oeqa/selftest/cases/cve_check.py +52
-0+ "CVE-2024-0000000000000000000000001", + ) + + def test_parse_cve_from_patch_contents(self): + import textwrap + from oe.cve_check import parse_cves_from_patch_contents + + # Standard patch file excerpt without any patches + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + remove "*" for root since we don't have a /etc/shadow so far. + + Upstream-Status: Inappropriate [configuration] + + Signed-off-by: Scott Garman <[email protected]<mailto:[email protected]>> + + --- base-passwd/passwd.master~nobash + +++ base-passwd/passwd.master + @@ -1,4 +1,4 @@ + -root:*:0:0:root:/root:/bin/sh + +root::0:0:root:/root:/bin/sh + daemon:*:1:1:daemon:/usr/sbin:/bin/sh + bin:*:2:2:bin:/bin:/bin/sh + sys:*:3:3:sys:/dev:/bin/sh + """) + ), + set(), + ) + + # Patch file with multiple CVE IDs (space-separated) + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461 CVE-2019-6462 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]<mailto:[email protected]>> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; +Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py class CVECheck(OESelftestTestCase):
Quoted 8 lines
meta/lib/oeqa/selftest/cases/cve_check.py +8
-0+ if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + """), + ), + {"CVE-2019-6461", "CVE-2019-6462"}, + ) + + # Patch file with multiple CVE IDs (comma-separated w/ both spaceand
Quoted 96 lines
Quoted 96 lines
no space)
meta/lib/oeqa/selftest/cases/cve_check.py +88
-0+ self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461,CVE-2019-6462, CVE-2019-6463 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]<mailto:[email protected]>> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + + """), + ), + {"CVE-2019-6461", "CVE-2019-6462", "CVE-2019-6463"}, + ) + + # Patch file with multiple CVE IDs (&-separated) + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461 & CVE-2019-6462 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]<mailto:[email protected]>> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + """), + ), + {"CVE-2019-6461", "CVE-2019-6462"}, + ) + + # Patch file with multiple lines with CVE IDs + self.assertEqual( + parse_cves_from_patch_contents( + textwrap.dedent("""\ + There is an assertion in function _cairo_arc_in_direction(). + + CVE: CVE-2019-6461 & CVE-2019-6462 + + CVE: CVE-2019-6463 & CVE-2019-6464 + Upstream-Status: Pending + Signed-off-by: Ross Burton <[email protected]<mailto:[email protected]>> + + diff --git a/src/cairo-arc.c b/src/cairo-arc.c + index 390397bae..1bde774a4 100644 + --- a/src/cairo-arc.c + +++ b/src/cairo-arc.c + @@ -186,7 +186,8 @@ _cairo_arc_in_direction (cairo_t *cr, + if (cairo_status (cr)) + return; + + - assert (angle_max >= angle_min); + + if (angle_max < angle_min) + + return; + + if (angle_max - angle_min > 2 * M_PI * MAX_FULL_CIRCLES) { + angle_max = fmod (angle_max - angle_min, 2 * M_PI); + + """), + ), + {"CVE-2019-6461", "CVE-2019-6462", "CVE-2019-6463", "CVE-2019-6464"},
+ )def test_recipe_report_json(self): config = """--
2.34.1
Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
In reply to Colin McAllisterQuoted 27 lines · Colin McAllister via lists.openembedded.org
The
cve_checkfunctionality to parse CVE IDs from the patch filename and patch contents have been reworked to improve parsing and also utilize tests. This ensures that the parsing works as intended.Additionally, the new
patched_cvesdict has a few issues I tried to fix as well. If multiple patch files exist for a single CVE ID, only the last one will show up with the "resource" key. The value for the "resource" key has been updated to hold a list and return all patch files associated with a given CVE ID. Also, at the end ofget_patch_cves,CVE_STATUScan overwrite an existing entry in the dict. This could cause an issue, for example, if a CVE has been addressed via a patch, but aCVE_STATUSline also exists that ignores the given CVE ID. A warning has been added if this ever happens.Signed-off-by Colin McAllister <[email protected]>---
I noticed that there are some patches, especially in older verisons of Yocto, where the "CVE: " tag was used with multiple CVE IDs in different formats, like "CVE-YYYY-XXXX & CVE-YYYY-XXXX" or "CVE-YYYY-XXXX, CVE-YYYY-XXXX". Currently, only space-delimited CVE IDs will be parsed, but documentation doesn't indicate that is the only supported format. I figured it'd be nice to update the code to be able to support multiple formats, that way this patch could be backported to fix those patches. I also wanted to add unit tests to ensure the patch parsing behavior is preserved.
Quoted 5 lines
I'd also like to update the patch filename parsing to parse multiple CVE IDs from the filename, but based on the comments, it seems like there was a reason why only the last CVE ID is extracted from the filename. I'd be happy to submit a V2 patch or an additional patch to update the function if that sounds good for the maintainers.
I think this resulted in a few issues. The weekly CVE report gained 32 new entries this week and many of them are clearly patched.
New this week: 32 CVEs
CVE-2014-8139 (CVSS3: 7.8 HIGH): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-8139 * CVE-2014-8140 (CVSS3: 7.8 HIGH): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-8140 * CVE-2014-8141 (CVSS3: 7.8 HIGH): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-8141 * CVE-2014-9636 (CVSS3: N/A): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-9636 * CVE-2014-9913 (CVSS3: 4.0 MEDIUM): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-9913 * CVE-2015-7696 (CVSS3: N/A): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7696 * CVE-2015-7697 (CVSS3: N/A): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7697 * CVE-2016-9844 (CVSS3: 4.0 MEDIUM): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2016-9844 * CVE-2018-1000035 (CVSS3: 7.8 HIGH): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-1000035 * CVE-2018-1000156 (CVSS3: 7.8 HIGH): patch:patch-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-1000156 * CVE-2018-10195 (CVSS3: 7.1 HIGH): lrzsz https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-10195 * CVE-2018-18384 (CVSS3: 5.5 MEDIUM): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-18384 * CVE-2018-20969 (CVSS3: 7.8 HIGH): patch:patch-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-20969 * CVE-2018-6951 (CVSS3: 7.5 HIGH): patch:patch-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-6951 * CVE-2018-6952 (CVSS3: 7.5 HIGH): patch:patch-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-6952 * CVE-2019-13232 (CVSS3: 3.3 LOW): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2019-13232 * CVE-2019-13636 (CVSS3: 5.9 MEDIUM): patch:patch-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2019-13636 * CVE-2019-13638 (CVSS3: 7.8 HIGH): patch:patch-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2019-13638 * CVE-2019-20633 (CVSS3: 5.5 MEDIUM): patch:patch-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2019-20633 * CVE-2020-27748 (CVSS3: 6.5 MEDIUM): xdg-utils https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2020-27748 * CVE-2021-3468 (CVSS3: 5.5 MEDIUM): avahi https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2021-3468 * CVE-2021-4217 (CVSS3: 3.3 LOW): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2021-4217 * CVE-2022-0529 (CVSS3: 5.5 MEDIUM): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2022-0529 * CVE-2022-0530 (CVSS3: 5.5 MEDIUM): unzip:unzip-native https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2022-0530 * CVE-2022-33065 (CVSS3: 7.8 HIGH): libsndfile1 https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2022-33065 * CVE-2022-4055 (CVSS3: 7.4 HIGH): xdg-utils https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2022-4055 * CVE-2023-38469 (CVSS3: 5.5 MEDIUM): avahi https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2023-38469 * CVE-2023-38470 (CVSS3: 5.5 MEDIUM): avahi https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2023-38470 * CVE-2023-38471 (CVSS3: 5.5 MEDIUM): avahi https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2023-38471 * CVE-2023-38472 (CVSS3: 5.5 MEDIUM): avahi https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2023-38472 * CVE-2023-38473 (CVSS3: 5.5 MEDIUM): avahi https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2023-38473 * CVE-2024-50612 (CVSS3: 5.5 MEDIUM): libsndfile1 https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2024-50612 *
meta/recipes-extended/unzip/unzip has:
0001-unzip-fix-CVE-2018-1000035.patch
09-cve-2014-8139-crc-overflow.patch
10-cve-2014-8140-test-compr-eb.patch
11-cve-2014-8141-getzip64data.patch
18-cve-2014-9913-unzip-buffer-overflow.patch
19-cve-2016-9844-zipinfo-buffer-overflow.patch
cve-2014-9636.patch
CVE-2015-7696.patch
CVE-2015-7697.patch
CVE-2018-18384.patch
CVE-2019-13232_p1.patch
CVE-2019-13232_p2.patch
CVE-2019-13232_p3.patch
CVE-2021-4217.patch
CVE-2022-0529.patch
CVE-2022-0530.patch
which cover some of the above.
Cheers,
Richard
RE: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
Richard,
This regression is fixed by https://lists.openembedded.org/g/openembedded-core/message/209573
Peter
Quoted 136 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Richard Purdie via lists.openembedded.org
Sent: Sunday, January 12, 2025 13:18
To: [email protected]; [email protected]
Subject: Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsingQuoted 27 lines · Colin McAllister via lists.openembedded.org
The
cve_checkfunctionality to parse CVE IDs from the patch filename and patch contents have been reworked to improve parsing and also utilize tests. This ensures that the parsing works as intended.Additionally, the new
patched_cvesdict has a few issues I tried to fix as well. If multiple patch files exist for a single CVE ID, only the last one will show up with the "resource" key. The value for the "resource" key has been updated to hold a list and return all patch files associated with a given CVE ID. Also, at the end ofget_patch_cves,CVE_STATUScan overwrite an existing entry in the dict. This could cause an issue, for example, if a CVE has been addressed via a patch, but aCVE_STATUSline also exists that ignores the given CVE ID. A warning has been added if this ever happens.Signed-off-by Colin McAllister <[email protected]>---
I noticed that there are some patches, especially in older verisons of Yocto, where the "CVE: " tag was used with multiple CVE IDs in different formats, like "CVE-YYYY-XXXX & CVE-YYYY-XXXX" or "CVE-YYYY-XXXX, CVE-YYYY-XXXX". Currently, only space-delimited CVE IDs will be parsed, but documentation doesn't indicate that is the only supported format. I figured it'd be nice to update the code to be able to support multiple formats, that way this patch could be backported to fix those patches. I also wanted to add unit tests to ensure the patch parsing behavior is preserved.
Quoted 5 lines
I'd also like to update the patch filename parsing to parse multiple CVE IDs from the filename, but based on the comments, it seems like there was a reason why only the last CVE ID is extracted from the filename. I'd be happy to submit a V2 patch or an additional patch to update the function if that sounds good for the maintainers.
I think this resulted in a few issues. The weekly CVE report gained 32 new entries this week and many of them are clearly patched.
New this week: 32 CVEs
CVE-2014-8139 (CVSS3: 7.8 HIGH): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-8139 * CVE-2014-8140 (CVSS3: 7.8 HIGH): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-8140 * CVE-2014-8141 (CVSS3: 7.8 HIGH): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-8141 * CVE-2014-9636 (CVSS3: N/A): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-9636 * CVE-2014-9913 (CVSS3: 4.0 MEDIUM): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-9913 * CVE-2015-7696 (CVSS3: N/A): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7696 * CVE-2015-7697 (CVSS3: N/A): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-7697 * CVE-2016-9844 (CVSS3: 4.0 MEDIUM): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2016-9844 * CVE-2018-1000035 (CVSS3: 7.8 HIGH): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-1000035 * CVE-2018-1000156 (CVSS3: 7.8 HIGH): patch:patch-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-1000156 * CVE-2018-10195 (CVSS3: 7.1 HIGH): lrzsz
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-10195 * CVE-2018-18384 (CVSS3: 5.5 MEDIUM): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-18384 * CVE-2018-20969 (CVSS3: 7.8 HIGH): patch:patch-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-20969 * CVE-2018-6951 (CVSS3: 7.5 HIGH): patch:patch-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-6951 * CVE-2018-6952 (CVSS3: 7.5 HIGH): patch:patch-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2018-6952 * CVE-2019-13232 (CVSS3: 3.3 LOW): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2019-13232 * CVE-2019-13636 (CVSS3: 5.9 MEDIUM): patch:patch-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2019-13636 * CVE-2019-13638 (CVSS3: 7.8 HIGH): patch:patch-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2019-13638 * CVE-2019-20633 (CVSS3: 5.5 MEDIUM): patch:patch-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2019-20633 * CVE-2020-27748 (CVSS3: 6.5 MEDIUM): xdg-utils
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2020-27748 * CVE-2021-3468 (CVSS3: 5.5 MEDIUM): avahi
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2021-3468 * CVE-2021-4217 (CVSS3: 3.3 LOW): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2021-4217 * CVE-2022-0529 (CVSS3: 5.5 MEDIUM): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2022-0529 * CVE-2022-0530 (CVSS3: 5.5 MEDIUM): unzip:unzip-native
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2022-0530 * CVE-2022-33065 (CVSS3: 7.8 HIGH): libsndfile1
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2022-33065 * CVE-2022-4055 (CVSS3: 7.4 HIGH): xdg-utils
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2022-4055 * CVE-2023-38469 (CVSS3: 5.5 MEDIUM): avahi
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2023-38469 * CVE-2023-38470 (CVSS3: 5.5 MEDIUM): avahi
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2023-38470 * CVE-2023-38471 (CVSS3: 5.5 MEDIUM): avahi
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2023-38471 * CVE-2023-38472 (CVSS3: 5.5 MEDIUM): avahi
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2023-38472 * CVE-2023-38473 (CVSS3: 5.5 MEDIUM): avahi
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2023-38473 * CVE-2024-50612 (CVSS3: 5.5 MEDIUM): libsndfile1
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2024-50612 *meta/recipes-extended/unzip/unzip has:
0001-unzip-fix-CVE-2018-1000035.patch
09-cve-2014-8139-crc-overflow.patch
10-cve-2014-8140-test-compr-eb.patch
11-cve-2014-8141-getzip64data.patch
18-cve-2014-9913-unzip-buffer-overflow.patch
19-cve-2016-9844-zipinfo-buffer-overflow.patch
cve-2014-9636.patch
CVE-2015-7696.patch
CVE-2015-7697.patch
CVE-2018-18384.patch
CVE-2019-13232_p1.patch
CVE-2019-13232_p2.patch
CVE-2019-13232_p3.patch
CVE-2021-4217.patch
CVE-2022-0529.patch
CVE-2022-0530.patchwhich cover some of the above.
Cheers,
Richard
Re: [OE-core] [PATCH v2 1/1] cve-check: Rework patch parsing
Quoted 3 lines · Peter
This regression is fixed by
https://lists.openembedded.org/g/openembedded-core/message/209573
Thanks for the tip, I've merged that.
Cheers,
Richard