Check compiled files to filter kernel CVEs
v4 cover Check compiled files to filter kernel CVEs
From: Daniel Turull <[email protected]>
Since kernel.org became a CNA, more information is available in the published CVEs, including details about which files are affected by a given CVE.
I have rewritten the original patch after the feedback received, including only the basic functionality in the build (extracting the sources as a text file and inside spdx) and created a postprocessing script that enrich the cve-summary.
To filter out CVEs that are not applicable, we extract the files used during the kernel compilation and compare it with the metadata in the CVE.
To enabled, add in your local.conf SPDX_INCLUDE_COMPILED_SOURCES.
This could use as a base to run the vulnerability check independently and run an external tool to filter the CVEs.
This patch can reduce the noise of kernel CVEs by around 70% with the default qemu kernel config and even more with smaller kernels.
Kernel | Total CVEs | Fix backported | Vulnerable | Filter with x86-64 qemu yocto compiled files 6.12.28 | 1201 | 1138 | 63 | 25 6.6.90 | 3121 | 2877 | 244 | 80 6.1.138 | 2979 | 2522 | 457 | 142 5.15.182 | 3767 | 3005 | 762 | 230 5.10.237 | 3586 | 2622 | 964 | 288 5.4.293 | 3030 | 1806 | 1224 | 394
When looking at the spdx source files included in the linux-yocto with a qemu-x86-64 build, it goes from 86989 to 38312 files included, since only c files are removed.
v1: initial proposal
v2:
- rewrite kernel_vulns to fetch similarly as cve-update-db-native
- add functionality into cve_check.py, for the classes that uses oe.get_patched_cves function
- add linux-vulns into the cve-check results
- add only compiled files in the spdx, so the check can be done outside the build
- include compiled files into spdx when CVE_CHECK_KERNEL_CONFIG and SPDX_INCLUDE_SOURCES is enabled
v3:
- make inclusion of compiled files generic for SPDX, so other systems that has knowledge of used files can also make more accurate sboms
- have the functions to extract files in kernel.bbclass. For other recipes that in the future want to use this feature can add the function in their recipe or in a build bbclass.
- move order of patches
- explicitly have the save_compiled_files added only when having the CVE_CHECK_KERNEL_CONFIG
- add first kernel cves in cve_check, so manual CVE_STATUS is preserved
- add CVE_STATUS for false positives
v4:
- Refactor and reduce series to 3 patches, one for spdx, one for the kernel, and one standalone script
Daniel Turull
Daniel Turull (3):
spdx: add option to include only compiled sources
kernel: add support to extract compiled files
improve_kernel_cve_report: add script for postprocesing of kernel CVE
data
meta/classes-recipe/kernel.bbclass | 19 + meta/classes/create-spdx-2.2.bbclass | 9 + meta/classes/spdx-common.bbclass | 3 + meta/lib/oe/spdx30_tasks.py | 9 + meta/lib/oe/spdx_common.py | 33 ++ scripts/contrib/improve_kernel_cve_report.py | 437 +++++++++++++++++++ 6 files changed, 510 insertions(+) create mode 100755 scripts/contrib/improve_kernel_cve_report.py
PATCH v4 1/3 spdx: add option to include only compiled sources
From: Daniel Turull <[email protected]>
When SPDX_INCLUDE_COMPILED_SOURCES is enabled, only include the source code (.c) files that are used during compilation.
This enables an external tool to use the SPDX information to disregard vulnerabilities that are not compiled.
This commit adds the basics, so recipes can implement it own methods.
--- meta/classes/create-spdx-2.2.bbclass | 9 ++++++++ meta/classes/spdx-common.bbclass | 3 +++ meta/lib/oe/spdx30_tasks.py | 9 ++++++++ meta/lib/oe/spdx_common.py | 33 ++++++++++++++++++++++++++++ 4 files changed, 54 insertions(+)
meta/classes/create-spdx-2.2.bbclass +9 -0
@@ -137,6 +137,11 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv spdx_files = [] file_counter = 1 + + check_compiled_sources = d.getVar("SPDX_INCLUDE_COMPILED_SOURCES") == "1" + if check_compiled_sources: + compiled_sources = oe.spdx_common.get_compiled_sources(d) + bb.debug(1, f"Total compiled files: {len(compiled_sources)}") for subdir, dirs, files in os.walk(topdir): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):
@@ -147,6 +152,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv filename = str(filepath.relative_to(topdir)) if not filepath.is_symlink() and filepath.is_file(): + # Check if file is compiled + if check_compiled_sources: + if not oe.spdx_common.is_compiled_source(file, compiled_sources): + break spdx_file = oe.spdx.SPDXFile() spdx_file.SPDXID = get_spdxid(file_counter) for t in get_types(filepath):
meta/classes/spdx-common.bbclass +3 -0
@@ -26,6 +26,9 @@ SPDX_TOOL_VERSION ??= "1.0" SPDXRUNTIMEDEPLOY = "${SPDXDIR}/runtime-deploy" SPDX_INCLUDE_SOURCES ??= "0" +SPDX_INCLUDE_COMPILED_SOURCES ??= "0" +SPDX_COMPILED_SOURCES_DIR ??= "${LOG_DIR}/spdx-compiled/${PN}" +SPDX_COMPILED_SOURCES ??= "${SPDX_FILES_DIR}/compiled_src-${BP}.txt" SPDX_UUID_NAMESPACE ??= "sbom.openembedded.org" SPDX_NAMESPACE_PREFIX ??= "http://spdx.org/spdxdocs"
meta/lib/oe/spdx30_tasks.py +9 -0
@@ -156,6 +156,11 @@ def add_package_files( bb.note(f"Skip {topdir}") return spdx_files + check_compiled_sources = d.getVar("SPDX_INCLUDE_COMPILED_SOURCES") == "1" + if check_compiled_sources: + compiled_sources = oe.spdx_common.get_compiled_sources(d) + bb.debug(1, f"Total compiled files: {len(compiled_sources)}") + for subdir, dirs, files in os.walk(topdir, onerror=walk_error): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):
@@ -167,6 +172,10 @@ def add_package_files( filepath = Path(subdir) / file if filepath.is_symlink() or not filepath.is_file(): continue + # Check if file is compiled + if check_compiled_sources: + if not oe.spdx_common.is_compiled_source(file, compiled_sources): + break filename = str(filepath.relative_to(topdir)) file_purposes = get_purposes(filepath)
meta/lib/oe/spdx_common.py +33 -0
@@ -242,3 +242,36 @@ def fetch_data_to_uri(fd, name): uri = uri + "@" + fd.revision return uri + + +def is_compiled_source (filename, compiled_sources): + """ + Check if the file, is a compiled file + """ + import os + # If we don't have compiled source, we asume all are compiled. + if len(compiled_sources) == 0: + return True + _, extension = os.path.splitext(filename) + # Special case, that we need to ignore, since this is not a source file + # We filter .c files + if filename.rfind(".mod.c") > 0 or extension != ".c": + return True + # Check that the c file is in the list + if filename in compiled_sources: + return True + return False + +def get_compiled_sources(d): + """ + Return compiled files from the SPDX_COMPILED_FILES file + """ + cfiles = [] + sources = d.getVar('SPDX_COMPILED_SOURCES') + if not sources: + return cfiles + if not os.path.isfile(sources): + return cfiles + with open(sources, 'r') as f: + cfiles = [line.strip() for line in f] + return cfiles
PATCH v4 2/3 kernel: add support to extract compiled files
In reply to [email protected]From: Daniel Turull <[email protected]>
Use gen_compile_commands.py to extract files used during compilation for the used kernel configuration.
To enable set SPDX_INCLUDED_COMPILED_SOURCES="1"
The location can be controlled with KERNEL_FILES_DIR
--- meta/classes-recipe/kernel.bbclass | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
meta/classes-recipe/kernel.bbclass +19 -0
@@ -159,6 +159,9 @@ set -e image_task = d.getVar('INITRAMFS_TASK') if image_task: d.appendVarFlag('do_configure', 'depends', ' ${INITRAMFS_TASK}') + if d.getVar("SPDX_INCLUDE_COMPILED_SOURCES") == "1": + bb.build.addtask('do_save_compiled_files', 'do_build', 'do_compile do_compile_kernelmodules', d) + d.appendVarFlag('do_create_spdx', 'depends', f'{pn}:do_save_compiled_files') } # Here we pull in all various kernel image types which we support.
@@ -867,3 +870,19 @@ EXPORT_FUNCTIONS do_deploy # Add using Device Tree support inherit kernel-devicetree + +# in case we don't use spdx bbclass +SPDX_COMPILED_SOURCES_DIR ?= "${LOG_DIR}/spdx-compiled/${PN}" +SPDX_COMPILED_SOURCES ?= "${SPDX_COMPILED_SOURCES_DIR}/compiled_src-${MACHINE}-${BP}.txt" + +KERNEL_COMPILED_FILES ?= "${SPDX_COMPILED_SOURCES_DIR}/compiled_commands-${MACHINE}-${BP}.json" + +do_save_compiled_files() { + bbdebug 1 "Saving compiled files in ${SPDX_COMPILED_SOURCES}" + mkdir -p ${SPDX_COMPILED_SOURCES_DIR} + ${S}/scripts/clang-tools/gen_compile_commands.py -o ${KERNEL_COMPILED_FILES} -d ${B} + # Make paths releative to the kernel source + sed -i 's|${B}/||g' ${KERNEL_COMPILED_FILES} + sed -i 's|${S}/||g' ${KERNEL_COMPILED_FILES} + grep '"file": ' ${KERNEL_COMPILED_FILES} | awk '{print $2}' | tr -d '"' > ${SPDX_COMPILED_SOURCES} +}