Check compiled files to filter kernel CVEs
v5 cover Check compiled files to filter kernel CVEs
From: Daniel Turull <[email protected]>
Since kernel.org became a CNA, more information is available in the published CVEs, including details about which files are affected by a given CVE.
I have rewritten the original patch after the feedback received, including only the basic functionality in the build (extracting the sources as a text file and inside spdx) and created a postprocessing script that enrich the cve-summary.
To filter out CVEs that are not applicable, we extract the files used during the kernel compilation and compare it with the metadata in the CVE.
To enabled, add in your local.conf SPDX_INCLUDE_COMPILED_SOURCES. It will generated spdx files with only the used sources. If only the kernel is wanted use in your local.conf: SPDX_INCLUDE_COMPILED_SOURCES:pn-linux-yocto = "1".
This could use as a base to run the vulnerability check independently and run an external tool to filter the CVEs.
v1: initial proposal
v2:
- rewrite kernel_vulns to fetch similarly as cve-update-db-native
- add functionality into cve_check.py, for the classes that uses oe.get_patched_cves function
- add linux-vulns into the cve-check results
- add only compiled files in the spdx, so the check can be done outside the build
- include compiled files into spdx when CVE_CHECK_KERNEL_CONFIG and SPDX_INCLUDE_SOURCES is enabled
v3:
- make inclusion of compiled files generic for SPDX, so other systems that has knowledge of used files can also make more accurate sboms
- have the functions to extract files in kernel.bbclass. For other recipes that in the future want to use this feature can add the function in their recipe or in a build bbclass.
- move order of patches
- explicitly have the save_compiled_files added only when having the CVE_CHECK_KERNEL_CONFIG
- add first kernel cves in cve_check, so manual CVE_STATUS is preserved
- add CVE_STATUS for false positives
v4:
- Refactor and reduce series to 3 patches, one for spdx, one for the kernel, and one standalone script
v5:
- Use debug information from do_package to extract compiled files
- Read spdx file from the kernel in the script to process files
Daniel Turull
Daniel Turull (3):
package: change location of debugsources to PKGDESTWORK
spdx: add option to include only compiled sources
improve_kernel_cve_report: add script for postprocesing of kernel CVE
data
meta/classes/create-spdx-2.2.bbclass | 9 + meta/classes/spdx-common.bbclass | 3 + meta/lib/oe/package.py | 5 +- meta/lib/oe/spdx30_tasks.py | 10 + meta/lib/oe/spdx_common.py | 49 ++ scripts/contrib/improve_kernel_cve_report.py | 467 +++++++++++++++++++ 6 files changed, 542 insertions(+), 1 deletion(-) create mode 100755 scripts/contrib/improve_kernel_cve_report.py
PATCH v5 1/3 package: change location of debugsources to PKGDESTWORK
From: Daniel Turull <[email protected]>
Storing file generated doing packaging in WORKDIR, doesn't get cached in sstate and task depending on them fail to find them.
Store it instead in pkgdata/debugsources.
--- meta/lib/oe/package.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
meta/lib/oe/package.py +4 -1
@@ -972,7 +972,10 @@ def copydebugsources(debugsrcdir, sources, d): cpath = oe.cachedpath.CachedPath() if debugsrcdir and sources: - sourcefile = d.expand("${WORKDIR}/debugsources.list") + sourcefile = d.expand("${PKGDESTWORK}/debugsources/${PN}-debugsources.list") + debugdir = os.path.dirname(sourcefile) + if not os.path.isdir(debugdir): + bb.utils.mkdirhier(debugdir) bb.utils.remove(sourcefile) # filenames are null-separated - this is an artefact of the previous use
Re: [OE-core] [PATCH v5 1/3] package: change location of debugsources to PKGDESTWORK
Quoted 28 lines · Daniel Turull via lists.openembedded.org
From: Daniel Turull <[email protected]>
Storing file generated doing packaging in WORKDIR, doesn't get cached in sstate and task depending on them fail to find them.
Store it instead in pkgdata/debugsources.
Signed-off-by Daniel Turull <[email protected]>---
meta/lib/oe/package.py | 5 ++++-1 file changed, 4 insertions(+), 1 deletion(-)meta/lib/oe/package.py +0
-0@@ -972,7 +972,10 @@ def copydebugsources(debugsrcdir, sources, d): cpath = oe.cachedpath.CachedPath()
if debugsrcdir and sources:
- sourcefile =d.expand("${WORKDIR}/debugsources.list")
+ sourcefile =d.expand("${PKGDESTWORK}/debugsources/${PN}-debugsources.list")+ debugdir =os.path.dirname(sourcefile)
+ if notos.path.isdir(debugdir):
+bb.utils.mkdirhier(debugdir)
bb.utils.remove(sourcefile)
Quoted 1 line
# filenames are null-separated - this is an artefact of the previous use
This needs a little bit more explanation/justification in the commit message as this change is a fairly big deal. I can guess why you're doing that based on the other patches but the commit should stand alone with information.
Effectively by moving the file, you're taking it from being internal API to an external API which can be used outside of do_package. When we do that, we need to be really clear about it and also be 100% sure that what we're exposing is the right information in the right form.
There are other implications such as increasing the size of the sstate objects too.
This file is a null separated list, is that a good format to export it in for wider use? It is certainly different to any other file in pkgdata.
Also, the file just contains a list of all sources referenced by all files in the output. Do we need to include more information, such as which files have which dependencies?
I'm asking these questions as I'm worried about future implications for this data. Changing the format later is hard.
Cheers,
Richard
RE: [OE-core] [PATCH v5 1/3] package: change location of debugsources to PKGDESTWORK
Thanks Richard,
Yes, I'm exposing the file to be able to extract the source code used in the other patches in the series. Since the file was there, I though it will be minimal changes on the current code and less overhead creating new tasks to extract the information that is already available.
Will it make more sense to create a new file under "${PKGDESTWORK}/debugsources/ with the correct format and the first filtering done, more aligned with existing files, since the current file is use a temporal placeholder? Or something in the line of:binary_file: src1 src2 src3 ...
I can also create a better explanation in thee commit message, in the lines of:
The information provided in debugsource can be use to have more detailed information on the files used during the compilation and improve quality on the SPDX.
I'll try to measure the increase on the sstate.
Cheers,
Daniel
Quoted 65 lines
-----Original Message-----
From: Richard Purdie <[email protected]>
Sent: Tuesday, 27 May 2025 15:11
To: Daniel Turull <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [PATCH v5 1/3] package: change location of debugsources to PKGDESTWORKQuoted 28 lines · Daniel Turull via lists.openembedded.org
From: Daniel Turull <[email protected]>
Storing file generated doing packaging in WORKDIR, doesn't get cached in sstate and task depending on them fail to find them.
Store it instead in pkgdata/debugsources.
Signed-off-by Daniel Turull <[email protected]>---
meta/lib/oe/package.py | 5 ++++-1 file changed, 4 insertions(+), 1 deletion(-)meta/lib/oe/package.py index renamed from meta/lib/oe/package.py +0
-0
0bcc04ea54..6fcd001db1100644meta/lib/oe/package.py +0
-0@@ -972,7 +972,10 @@ def copydebugsources(debugsrcdir, sources, d): cpath = oe.cachedpath.CachedPath()if debugsrcdir and sources:
- sourcefile =d.expand("${WORKDIR}/debugsources.list")+ sourcefile = +d.expand("${PKGDESTWORK}/debugsources/${PN}-debugsources.list") + debugdir =os.path.dirname(sourcefile)
+ if notos.path.isdir(debugdir):
+bb.utils.mkdirhier(debugdir)bb.utils.remove(sourcefile)Quoted 2 lines
# filenames are null-separated - this is an artefact of the previous useThis needs a little bit more explanation/justification in the commit message as this change is a fairly big deal. I can guess why you're doing that based on the other patches but the commit should stand alone with information.
Effectively by moving the file, you're taking it from being internal API to an external API which can be used outside of
do_package. When we do that, we need to be really clear about it and also be 100% sure that what we're exposing is the right information in the right form.There are other implications such as increasing the size of the sstate objects too.
This file is a null separated list, is that a good format to export it in for wider use? It is certainly different to any other file in pkgdata.
Also, the file just contains a list of all sources referenced by all files in the output. Do we need to include more information, such as which files have which dependencies?
I'm asking these questions as I'm worried about future implications for this data. Changing the format later is hard.
Cheers,
Richard
Re: [OE-core] [PATCH v5 1/3] package: change location of debugsources to PKGDESTWORK
Quoted 6 lines · Daniel Turull via lists.openembedded.org
Thanks Richard,
Yes, I'm exposing the file to be able to extract the source code used in the other patches in the series. Since the file was there, I though it will be minimal changes on the current code and less overhead creating new tasks to extract the information that is already available.
Right, I totally understand why you've done this and that is fine. In order to properly review and get this merged, we just need to consider the other issues and properly document it. It does need to be a conscious decision to put the data into pkgdata (and make sure we get the format right).
Quoted 5 lines
Will it make more sense to create a new file under "${PKGDESTWORK}/debugsources/ with the correct format and the first filtering done, more aligned with existing files, since the current file is use a temporal placeholder? Or something in the line of:
binary_file: src1 src2 src3 ...
It is null delimited as some filenames can have spaces in. I know there has been talk of changing our existing file format to be something less problematic like json, the extended data is already in json. I'd probably suggest using json for this given it is natively supported by python.
The above format (per binary file), in json would probably make sense though.
Quoted 6 lines
I can also create a better explanation in thee commit message, in the lines of:
The information provided in debugsource can be use to have more detailed information on the files used during the compilation and improve quality on the SPDX.I'll try to measure the increase on the sstate.
I don't expect this to be a huge issue but some kind of number would be useful. I'd note we did compress the extended data using zstd, just to improve the IO impact of the files. We could consider that here too and probably reuse that code.
Cheers,
Richard
PATCH v5 2/3 spdx: add option to include only compiled sources
In reply to [email protected]From: Daniel Turull <[email protected]>
When SPDX_INCLUDE_COMPILED_SOURCES is enabled, only include the source code files that are used during compilation.
It uses debugsource information generated during do_package.
This enables an external tool to use the SPDX information to disregard vulnerabilities that are not compiled.
As example, when used with the default config with linux-yocto, the spdx size is reduced from 156MB to 61MB.
--- meta/classes/create-spdx-2.2.bbclass | 9 +++++ meta/classes/spdx-common.bbclass | 3 ++ meta/lib/oe/spdx30_tasks.py | 10 ++++++ meta/lib/oe/spdx_common.py | 49 ++++++++++++++++++++++++++++ 4 files changed, 71 insertions(+)
meta/classes/create-spdx-2.2.bbclass +9 -0
@@ -137,6 +137,11 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv spdx_files = [] file_counter = 1 + + check_compiled_sources = d.getVar("SPDX_INCLUDE_COMPILED_SOURCES") == "1" + if check_compiled_sources: + compiled_sources, types = oe.spdx_common.get_compiled_sources(d) + bb.debug(1, f"Total compiled files: {len(compiled_sources)}") for subdir, dirs, files in os.walk(topdir): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):
@@ -147,6 +152,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv filename = str(filepath.relative_to(topdir)) if not filepath.is_symlink() and filepath.is_file(): + # Check if file is compiled + if check_compiled_sources: + if not oe.spdx_common.is_compiled_source(filename, compiled_sources, types): + continue spdx_file = oe.spdx.SPDXFile() spdx_file.SPDXID = get_spdxid(file_counter) for t in get_types(filepath):
meta/classes/spdx-common.bbclass +3 -0
@@ -26,6 +26,7 @@ SPDX_TOOL_VERSION ??= "1.0" SPDXRUNTIMEDEPLOY = "${SPDXDIR}/runtime-deploy" SPDX_INCLUDE_SOURCES ??= "0" +SPDX_INCLUDE_COMPILED_SOURCES ??= "0" SPDX_UUID_NAMESPACE ??= "sbom.openembedded.org" SPDX_NAMESPACE_PREFIX ??= "http://spdx.org/spdxdocs"
@@ -40,6 +41,8 @@ SPDX_MULTILIB_SSTATE_ARCHS ??= "${SSTATE_ARCHS}" python () { from oe.cve_check import extend_cve_status extend_cve_status(d) + if d.getVar("SPDX_INCLUDE_COMPILED_SOURCES") == "1": + d.setVar("SPDX_INCLUDE_SOURCES", "1") } def create_spdx_source_deps(d):
meta/lib/oe/spdx30_tasks.py +10 -0
@@ -156,6 +156,11 @@ def add_package_files( bb.note(f"Skip {topdir}") return spdx_files + check_compiled_sources = d.getVar("SPDX_INCLUDE_COMPILED_SOURCES") == "1" + if check_compiled_sources: + compiled_sources, types = oe.spdx_common.get_compiled_sources(d) + bb.debug(1, f"Total compiled files: {len(compiled_sources)}") + for subdir, dirs, files in os.walk(topdir, onerror=walk_error): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):
@@ -171,6 +176,11 @@ def add_package_files( filename = str(filepath.relative_to(topdir)) file_purposes = get_purposes(filepath) + # Check if file is compiled + if check_compiled_sources: + if not oe.spdx_common.is_compiled_source(filename, compiled_sources, types): + continue + spdx_file = objset.new_file( get_spdxid(file_counter), filename,
meta/lib/oe/spdx_common.py +49 -0
52 lines not on this page · raw message
PATCH v5 3/3 improve_kernel_cve_report: add script for postprocesing of kernel CVE data
In reply to [email protected]From: Daniel Turull <[email protected]>
Adding postprocessing script to process data from linux CNA that includes more accurate metadata and it is updated directly by the source.
Example of enhanced CVE from a report from cve-check:
{
"id": "CVE-2024-26710",
"status": "Ignored",
"link": "https://nvd.nist.gov/vuln/detail/CVE-2024-26710",
"summary": "In the Linux kernel, the following vulnerability [...]",
"scorev2": "0.0",
"scorev3": "5.5",
"scorev4": "0.0",
"modified": "2025-03-17T15:36:11.620",
"vector": "LOCAL",
"vectorString": "CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H",
"detail": "not-applicable-config",
"description": "Source code not compiled by config. ['arch/powerpc/include/asm/thread_info.h']"
},
And same from a report generated with vex:
{
"id": "CVE-2024-26710",
"status": "Ignored",
"link": "https://nvd.nist.gov/vuln/detail/CVE-2024-26710",
"detail": "not-applicable-config",
"description": "Source code not compiled by config. ['arch/powerpc/include/asm/thread_info.h']"
},
For unpatched CVEs, provide more context in the description:
Tested with 6.12.22 kernel
{
"id": "CVE-2025-39728",
"status": "Unpatched",
"link": "https://nvd.nist.gov/vuln/detail/CVE-2025-39728", "summary": "In the Linux kernel, the following vulnerability has been [...], "scorev2": "0.0",
"scorev3": "0.0",
"scorev4": "0.0",
"modified": "2025-04-21T14:23:45.950",
"vector": "UNKNOWN",
"vectorString": "UNKNOWN",
"detail": "version-in-range",
"description": "Needs backporting (fixed from 6.12.23)"
},
--- scripts/contrib/improve_kernel_cve_report.py | 467 +++++++++++++++++++ 1 file changed, 467 insertions(+) create mode 100755 scripts/contrib/improve_kernel_cve_report.py
scripts/contrib/improve_kernel_cve_report.py new file +467 -0
@@ -0,0 +1,467 @@ +#! /usr/bin/env python3 +# +# Copyright OpenEmbedded Contributors +# +# The script uses another source of CVE information from linux-vulns +# to enrich the cve-summary from cve-check or vex. +# It can also use the list of compiled files from the kernel spdx to ignore CVEs +# that are not affected since the files are not compiled. +# +# It creates a new json file with updated CVE information +# +# Compiled files can be extracted adding the following in local.conf +# SPDX_INCLUDE_COMPILED_SOURCES:pn-linux-yocto = "1" +# +# Tested with the following CVE sources: +# - https://git.kernel.org/pub/scm/linux/security/vulns.git +# - https://github.com/CVEProject/cvelistV5 +# +# Example: +# python3 ./openembedded-core/scripts/contrib/improve_kernel_cve_report.py --spdx tmp/deploy/spdx/3.0.1/qemux86_64/recipes/recipe-linux-yocto.spdx.json --kernel-version 6.12.27 --datadir ./vulns +# python3 ./openembedded-core/scripts/contrib/improve_kernel_cve_report.py --spdx tmp/deploy/spdx/3.0.1/qemux86_64/recipes/recipe-linux-yocto.spdx.json --datadir ./vulns --old-cve-report build/tmp/log/cve/cve-summary.json +# +# SPDX-License-Identifier: GPLv2 + +import argparse +import json +import sys +import logging +import glob +import os +import pathlib +from packaging.version import Version + +def is_linux_cve(cve_info): + '''Return true is the CVE belongs to Linux''' + if not "affected" in cve_info["containers"]["cna"]: + return False + for affected in cve_info["containers"]["cna"]["affected"]: + if not "product" in affected: + return False + if affected["product"] == "Linux" and affected["vendor"] == "Linux": + return True + return False + +def get_kernel_cves(datadir, compiled_files, version): + """ + Get CVEs for the kernel + """ + cves = {} + + check_config = len(compiled_files) > 0 + + base_version = Version(f"{version.major}.{version.minor}") + + # Check all CVES from kernel vulns + pattern = os.path.join(datadir, '**', "CVE-*.json") + cve_files = glob.glob(pattern, recursive=True) + not_applicable_config = 0 + fixed_as_later_backport = 0 + vulnerable = 0 + not_vulnerable = 0 + for cve_file in sorted(cve_files): + cve_info = {} + with open(cve_file, "r", encoding='ISO-8859-1') as f: + cve_info = json.load(f) + + if len(cve_info) == 0: + logging.error("Not valid data in %s. Aborting", cve_file) + break + + if not is_linux_cve(cve_info): + continue + cve_id = os.path.basename(cve_file)[:-5] + description = cve_info["containers"]["cna"]["descriptions"][0]["value"] + if cve_file.find("rejected") >= 0: + logging.debug("%s is rejected by the CNA", cve_id) + cves[cve_id] = { + "id": cve_id, + "status": "Ignored", + "detail": "rejected", + "summary": description, + "description": f"Rejected by CNA" + } + continue + if any(elem in cve_file for elem in ["review", "reverved", "testing"]): + continue + + is_vulnerable, first_affected, last_affected, better_match_first, better_match_last, affected_versions = get_cpe_applicability(cve_info, version) + + logging.debug("%s: %s (%s - %s) (%s - %s)", cve_id, is_vulnerable, better_match_first, better_match_last, first_affected, last_affected) + + if is_vulnerable is None: + logging.warning("%s doesn't have good metadata", cve_id) + if is_vulnerable: + is_affected = True + affected_files = [] + if check_config: + is_affected, affected_files = check_kernel_compiled_files(compiled_files, cve_info) + + if not is_affected and len(affected_files) > 0: + logging.debug( + "%s - not applicable configuration since affected files not compiled: %s", + cve_id, affected_files) + cves[cve_id] = { + "id": cve_id, + "status": "Ignored", + "detail": "not-applicable-config", + "summary": description, + "description": f"Source code not compiled by config. {affected_files}" + } + not_applicable_config +=1 + # Check if we have backport + else: + if not better_match_last: + fixed_in = last_affected + else: + fixed_in = better_match_last + logging.debug("%s needs backporting (fixed from %s)", cve_id, fixed_in) + cves[cve_id] = { + "id": cve_id, + "status": "Unpatched", + "detail": "version-in-range", + "summary": description, + "description": f"Needs backporting (fixed from {fixed_in})" + } + vulnerable += 1 + if (better_match_last and + Version(f"{better_match_last.major}.{better_match_last.minor}") == base_version): + fixed_as_later_backport += 1 + # Not vulnerable + else: + if not first_affected: + logging.debug("%s - not known affected %s", + cve_id, + better_match_last) + cves[cve_id] = { + "id": cve_id, + "status": "Patched", + "detail": "version-not-in-range", + "summary": description, + "description": "No CPE match" + } + not_vulnerable += 1 + continue + backport_base = Version(f"{better_match_last.major}.{better_match_last.minor}") + if version < first_affected: + logging.debug('%s - fixed-version: only affects %s onwards', + cve_id, + first_affected) + cves[cve_id] = { + "id": cve_id, + "status": "Patched", + "detail": "fixed-version", + "summary": description, + "description": f"only affects {first_affected} onwards" + } + not_vulnerable += 1 + elif last_affected <= version: + logging.debug("%s - fixed-version: Fixed from version %s", + cve_id, + last_affected) + cves[cve_id] = { + "id": cve_id, + "status": "Patched", + "detail": "fixed-version", + "summary": description, + "description": f"fixed-version: Fixed from version {last_affected}" + } + not_vulnerable += 1 + elif backport_base == base_version: + logging.debug("%s - cpe-stable-backport: Backported in %s", + cve_id, + better_match_last) + cves[cve_id] = { + "id": cve_id, + "status": "Patched", + "detail": "cpe-stable-backport", + "summary": description, + "description": f"Backported in {better_match_last}" + } + not_vulnerable += 1 + else: + logging.debug("%s - version not affected %s", cve_id, str(affected_versions)) + cves[cve_id] = { + "id": cve_id, + "status": "Patched", + "detail": "version-not-in-range", + "summary": description, + "description": f"Range {affected_versions}" + } + not_vulnerable += 1 + + logging.info("Total CVEs ignored due to not applicable config: %d", not_applicable_config) + logging.info("Total CVEs not vulnerable due version-not-in-range: %d", not_vulnerable) + logging.info("Total vulnerable CVEs: %d", vulnerable) + + logging.info("Total CVEs already backported in %s: %s", base_version, + fixed_as_later_backport) + return cves + +def read_spdx(spdx_file): + '''Open SPDX file and extract compiled files''' + with open(spdx_file, 'r', encoding='ISO-8859-1') as f: + spdx = json.load(f) + if "spdxVersion" in spdx: + if spdx["spdxVersion"] == "SPDX-2.2": + return read_spdx2(spdx) + if "@graph" in spdx: + return read_spdx3(spdx) + return [] + +def read_spdx2(spdx): + ''' + Read spdx2 compiled files from spdx + ''' + cfiles = [] + if 'files' not in spdx: + return cfiles + for item in spdx['files']: + for ftype in item['fileTypes']: + if ftype == "SOURCE": + filename = item["fileName"][item["fileName"].find("/")+1:] + cfiles.append(filename) + return cfiles + +def read_spdx3(spdx): + ''' + Read spdx3 compiled files from spdx + ''' + cfiles = [] + for item in spdx["@graph"]: + if "software_primaryPurpose" not in item: + continue + if item["software_primaryPurpose"] == "source": + filename = item['name'][item['name'].find("/")+1:] + cfiles.append(filename) + return cfiles + +def check_kernel_compiled_files(compiled_files, cve_info): + """ + Return if a CVE affected us depending on compiled files + """ + files_affected = [] + is_affected = False + + for item in cve_info['containers']['cna']['affected']: + if "programFiles" in item: + for f in item['programFiles']: + if f not in files_affected: + files_affected.append(f) + + if len(files_affected) > 0: + for f in files_affected: + if f in compiled_files: + logging.debug("File match: %s", f) + is_affected = True + return is_affected, files_affected + +def get_cpe_applicability(cve_info, v): + ''' + Check if version is affected and return affected versions + ''' + base_branch = Version(f"{v.major}.{v.minor}") + affected = [] + if not 'cpeApplicability' in cve_info["containers"]["cna"]: + return None, None, None, None, None, None + + for nodes in cve_info["containers"]["cna"]["cpeApplicability"]: + for node in nodes.values(): + vulnerable = False + matched_branch = False + first_affected = Version("5000") + last_affected = Version("0") + better_match_first = Version("0") + better_match_last = Version("5000") + + if len(node[0]['cpeMatch']) == 0: + first_affected = None + last_affected = None + better_match_first = None + better_match_last = None + + for cpe_match in node[0]['cpeMatch']: + version_start_including = Version("0") + version_end_excluding = Version("0") + if 'versionStartIncluding' in cpe_match: + version_start_including = Version(cpe_match['versionStartIncluding']) + else: + version_start_including = Version("0") + # if versionEndExcluding is missing we are in a branch, which is not fixed. + if "versionEndExcluding" in cpe_match: + version_end_excluding = Version(cpe_match["versionEndExcluding"]) + else: + # if versionEndExcluding is missing we are in a branch, which is not fixed. + version_end_excluding = Version( + f"{version_start_including.major}.{version_start_including.minor}.5000" + ) + affected.append(f" {version_start_including}-{version_end_excluding}") + # Detect if versionEnd is in fixed in base branch. It has precedence over the rest + branch_end = Version(f"{version_end_excluding.major}.{version_end_excluding.minor}") + if branch_end == base_branch: + if version_start_including <= v < version_end_excluding: + vulnerable = cpe_match['vulnerable'] + # If we don't match in our branch, we are not vulnerable, + # since we have a backport + matched_branch = True + better_match_first = version_start_including + better_match_last = version_end_excluding + if version_start_including <= v < version_end_excluding and not matched_branch: + if version_end_excluding < better_match_last: + better_match_first = max(version_start_including, better_match_first) + better_match_last = min(better_match_last, version_end_excluding) + vulnerable = cpe_match['vulnerable'] + matched_branch = True + + first_affected = min(version_start_including, first_affected) + last_affected = max(version_end_excluding, last_affected) + # Not a better match, we use the first and last affected instead of the fake .5000 + if vulnerable and better_match_last == Version(f"{base_branch}.5000"): + better_match_last = last_affected + better_match_first = first_affected + return vulnerable, first_affected, last_affected, better_match_first, better_match_last, affected + +def copy_data(old, new): + '''Update dictionary with new entries, while keeping the old ones''' + for k in new.keys(): + old[k] = new[k] + return old + +# Function taken from cve_check.bbclass. Adapted to cve fields +def cve_update(cve_data, cve, entry): + # If no entry, just add it + if cve not in cve_data: + cve_data[cve] = entry + return + # If we are updating, there might be change in the status + if cve_data[cve]['status'] == "Unknown": + cve_data[cve] = copy_data(cve_data[cve], entry) + return + if cve_data[cve]['status'] == entry['status']: + return + if entry['status'] == "Unpatched" and cve_data[cve]['status'] == "Patched": + logging.warning("CVE entry %s update from Patched to Unpatched from the scan result", cve) + cve_data[cve] = copy_data(cve_data[cve], entry) + return + if entry['status'] == "Patched" and cve_data[cve]['status'] == "Unpatched": + logging.warning("CVE entry %s update from Unpatched to Patched from the scan result", cve) + cve_data[cve] = copy_data(cve_data[cve], entry) + return + # If we have an "Ignored", it has a priority + if cve_data[cve]['status'] == "Ignored": + logging.debug("CVE %s not updating because Ignored", cve) + return + # If we have an "Ignored", it has a priority + if entry['status'] == "Ignored": + cve_data[cve] = copy_data(cve_data[cve], entry) + logging.debug("CVE entry %s updated from Unpatched to Ignored", cve) + return + logging.warning("Unhandled CVE entry update for %s %s from %s %s to %s", + cve, cve_data[cve]['status'], cve_data[cve]['detail'], entry['status'], entry['detail']) + +def main(): + parser = argparse.ArgumentParser( + description="Update cve-summary with kernel compiled files and kernel CVE information" + ) + parser.add_argument( + "-s", + "--spdx", + help="SPDX2/3 for the kernel. Needs to include compiled sources", + ) + parser.add_argument( + "--datadir", + type=pathlib.Path, + help="Directory where CVE data is", + required=True + ) + parser.add_argument( + "--old-cve-report", + help="CVE report to update. (Optional)", + ) + parser.add_argument( + "--kernel-version", + help="Kernel version. Needed if old cve_report is not provided (Optional)", + type=Version + ) + parser.add_argument( + "--new-cve-report", + help="Output file", + default="cve-summary-enhance.json" + ) + parser.add_argument( + "-D", + "--debug", + help='Enable debug ', + action="store_true") + + args = parser.parse_args() + + if args.debug: + log_level=logging.DEBUG + else: + log_level=logging.INFO + logging.basicConfig(format='[%(filename)s:%(lineno)d] %(message)s', level=log_level) + + if not args.kernel_version and not args.old_cve_report: + parser.error("either --kernel-version or --old-cve-report are needed") + return -1 + + # by default we don't check the compiled files, unless provided + compiled_files = [] + if args.spdx: + compiled_files = read_spdx(args.spdx) + logging.info("Total compiled files %d", len(compiled_files)) + + if args.old_cve_report: + with open(args.old_cve_report, encoding='ISO-8859-1') as f: + cve_report = json.load(f) + else: + #If summary not provided, we create one + cve_report = { + "version": "1", + "package": [ + { + "name": "linux-yocto", + "version": str(args.kernel_version), + "products": [ + { + "product": "linux_kernel", + "cvesInRecord": "Yes" + } + ], + "issue": [] + } + ] + } + + for pkg in cve_report['package']: + is_kernel = False + for product in pkg['products']: + if product['product'] == "linux_kernel": + is_kernel=True + if not is_kernel: + continue + + kernel_cves = get_kernel_cves(args.datadir, + compiled_files, + Version(pkg["version"])) + logging.info("Total kernel cves from kernel CNA: %s", len(kernel_cves)) + cves = {issue["id"]: issue for issue in pkg["issue"]} + logging.info("Total kernel before processing cves: %s", len(cves)) + + for cve in kernel_cves: + cve_update(cves, cve, kernel_cves[cve]) + + pkg["issue"] = [] + for cve in sorted(cves): + pkg["issue"].extend([cves[cve]]) + logging.info("Total kernel cves after processing: %s", len(pkg['issue'])) + + with open(args.new_cve_report, "w", encoding='ISO-8859-1') as f: + json.dump(cve_report, f, indent=2) + + return 0 + +if __name__ == "__main__": + sys.exit(main()) +