Skip to contentSkip to conversation

Check compiled files to filter kernel CVEs

PATCH Superseded [email protected] 1 participant 3 messages May 14, 2025 – May 15, 2025

v4 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v4 cover Check compiled files to filter kernel CVEs

From: Daniel Turull <[email protected]>

Since kernel.org became a CNA, more information is available in the published CVEs, including details about which files are affected by a given CVE.

I have rewritten the original patch after the feedback received, including only the basic functionality in the build (extracting the sources as a text file and inside spdx) and created a postprocessing script that enrich the cve-summary.

To filter out CVEs that are not applicable, we extract the files used during the kernel compilation and compare it with the metadata in the CVE.

To enabled, add in your local.conf SPDX_INCLUDE_COMPILED_SOURCES.

This could use as a base to run the vulnerability check independently and run an external tool to filter the CVEs.

This patch can reduce the noise of kernel CVEs by around 70% with the default qemu kernel config and even more with smaller kernels.

Kernel     | Total CVEs | Fix backported | Vulnerable | Filter with x86-64 qemu yocto compiled files
6.12.28    |   1201     |  1138          | 63         |  25
6.6.90     |   3121     |  2877          | 244        |  80
6.1.138    |   2979     |  2522          | 457        |  142
5.15.182   |   3767     |  3005          | 762        |  230
5.10.237   |   3586     |  2622          | 964        |  288
5.4.293    |   3030     |  1806          | 1224       |  394

When looking at the spdx source files included in the linux-yocto with a qemu-x86-64 build, it goes from 86989 to 38312 files included, since only c files are removed.

v1: initial proposal
v2:
- rewrite kernel_vulns to fetch similarly as cve-update-db-native
- add functionality into cve_check.py, for the classes that uses oe.get_patched_cves function
- add linux-vulns into the cve-check results
- add only compiled files in the spdx, so the check can be done outside the build
- include compiled files into spdx when CVE_CHECK_KERNEL_CONFIG and SPDX_INCLUDE_SOURCES is enabled
v3:
- make inclusion of compiled files generic for SPDX, so other systems that has knowledge of used files can also make more accurate sboms
- have the functions to extract files in kernel.bbclass. For other recipes that in the future want to use this feature can add the function in their recipe or in a build bbclass.
- move order of patches
- explicitly have the save_compiled_files added only when having the CVE_CHECK_KERNEL_CONFIG
- add first kernel cves in cve_check, so manual CVE_STATUS is preserved
- add CVE_STATUS for false positives
v4:
- Refactor and reduce series to 3 patches, one for spdx, one for the kernel, and one standalone script

CC Peter Marko <[email protected]>
CC Marta Rybczynska <[email protected]>
CC Joshua Watt <[email protected]>

Daniel Turull

Daniel Turull (3):
spdx: add option to include only compiled sources
kernel: add support to extract compiled files
improve_kernel_cve_report: add script for postprocesing of kernel CVE
data

 meta/classes-recipe/kernel.bbclass           |  19 +
 meta/classes/create-spdx-2.2.bbclass         |   9 +
 meta/classes/spdx-common.bbclass             |   3 +
 meta/lib/oe/spdx30_tasks.py                  |   9 +
 meta/lib/oe/spdx_common.py                   |  33 ++
 scripts/contrib/improve_kernel_cve_report.py | 437 +++++++++++++++++++
 6 files changed, 510 insertions(+)
 create mode 100755 scripts/contrib/improve_kernel_cve_report.py
#1 lore raw

PATCH v4 1/3 spdx: add option to include only compiled sources

From: Daniel Turull <[email protected]>

When SPDX_INCLUDE_COMPILED_SOURCES is enabled, only include the source code (.c) files that are used during compilation.

This enables an external tool to use the SPDX information to disregard vulnerabilities that are not compiled.

This commit adds the basics, so recipes can implement it own methods.

CC Joshua Watt <[email protected]>
CC Peter Marko <[email protected]>
---
 meta/classes/create-spdx-2.2.bbclass |  9 ++++++++
 meta/classes/spdx-common.bbclass     |  3 +++
 meta/lib/oe/spdx30_tasks.py          |  9 ++++++++
 meta/lib/oe/spdx_common.py           | 33 ++++++++++++++++++++++++++++
 4 files changed, 54 insertions(+)
meta/classes/create-spdx-2.2.bbclass +9 -0
index 7e8f8b9ff5..dd8ee6ecbe 100644
@@ -137,6 +137,11 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
     spdx_files = []
 
     file_counter = 1
+
+    check_compiled_sources = d.getVar("SPDX_INCLUDE_COMPILED_SOURCES") == "1"
+    if check_compiled_sources:
+        compiled_sources = oe.spdx_common.get_compiled_sources(d)
+        bb.debug(1, f"Total compiled files: {len(compiled_sources)}")
     for subdir, dirs, files in os.walk(topdir):
         dirs[:] = [d for d in dirs if d not in ignore_dirs]
         if subdir == str(topdir):
@@ -147,6 +152,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
             filename = str(filepath.relative_to(topdir))
 
             if not filepath.is_symlink() and filepath.is_file():
+                # Check if file is compiled
+                if check_compiled_sources:
+                     if not oe.spdx_common.is_compiled_source(file, compiled_sources):
+                          break
                 spdx_file = oe.spdx.SPDXFile()
                 spdx_file.SPDXID = get_spdxid(file_counter)
                 for t in get_types(filepath):
meta/classes/spdx-common.bbclass +3 -0
index 713a7fc651..e9dde34513 100644
@@ -26,6 +26,9 @@ SPDX_TOOL_VERSION ??= "1.0"
 SPDXRUNTIMEDEPLOY = "${SPDXDIR}/runtime-deploy"
 
 SPDX_INCLUDE_SOURCES ??= "0"
+SPDX_INCLUDE_COMPILED_SOURCES ??= "0"
+SPDX_COMPILED_SOURCES_DIR ??= "${LOG_DIR}/spdx-compiled/${PN}"
+SPDX_COMPILED_SOURCES ??= "${SPDX_FILES_DIR}/compiled_src-${BP}.txt"
 
 SPDX_UUID_NAMESPACE ??= "sbom.openembedded.org"
 SPDX_NAMESPACE_PREFIX ??= "http://spdx.org/spdxdocs"
meta/lib/oe/spdx30_tasks.py +9 -0
index 61d7ba45e3..083e004330 100644
@@ -156,6 +156,11 @@ def add_package_files(
         bb.note(f"Skip {topdir}")
         return spdx_files
 
+    check_compiled_sources = d.getVar("SPDX_INCLUDE_COMPILED_SOURCES") == "1"
+    if check_compiled_sources:
+        compiled_sources = oe.spdx_common.get_compiled_sources(d)
+        bb.debug(1, f"Total compiled files: {len(compiled_sources)}")
+
     for subdir, dirs, files in os.walk(topdir, onerror=walk_error):
         dirs[:] = [d for d in dirs if d not in ignore_dirs]
         if subdir == str(topdir):
@@ -167,6 +172,10 @@ def add_package_files(
             filepath = Path(subdir) / file
             if filepath.is_symlink() or not filepath.is_file():
                 continue
+            # Check if file is compiled
+            if check_compiled_sources:
+                 if not oe.spdx_common.is_compiled_source(file, compiled_sources):
+                      break
 
             filename = str(filepath.relative_to(topdir))
             file_purposes = get_purposes(filepath)
meta/lib/oe/spdx_common.py +33 -0
index 4caefc7673..e1b7f576dd 100644
@@ -242,3 +242,36 @@ def fetch_data_to_uri(fd, name):
         uri = uri + "@" + fd.revision
 
     return uri
+
+
+def is_compiled_source (filename, compiled_sources):
+    """
+    Check if the file, is a compiled file
+    """
+    import os
+    # If we don't have compiled source, we asume all are compiled.
+    if len(compiled_sources) == 0:
+        return True
+    _, extension = os.path.splitext(filename)
+    # Special case, that we need to ignore, since this is not a source file
+    # We filter .c files
+    if filename.rfind(".mod.c") > 0 or extension != ".c":
+        return True
+    # Check that the c file is in the list
+    if filename in compiled_sources:
+        return True
+    return False
+
+def get_compiled_sources(d):
+    """
+    Return compiled files from the SPDX_COMPILED_FILES file
+    """
+    cfiles = []
+    sources = d.getVar('SPDX_COMPILED_SOURCES')
+    if not sources:
+        return cfiles
+    if not os.path.isfile(sources):
+        return cfiles
+    with open(sources, 'r') as f:
+        cfiles = [line.strip() for line in f]
+    return cfiles
#2 lore raw

PATCH v4 2/3 kernel: add support to extract compiled files

In reply to [email protected]

From: Daniel Turull <[email protected]>

Use gen_compile_commands.py to extract files used during compilation for the used kernel configuration.

To enable set SPDX_INCLUDED_COMPILED_SOURCES="1"

The location can be controlled with KERNEL_FILES_DIR

CC Marta Rybczynska <[email protected]>
CC Bruce Ashfield <[email protected]>
CC Peter Marko <[email protected]>
---
 meta/classes-recipe/kernel.bbclass | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
meta/classes-recipe/kernel.bbclass +19 -0
index 36ce659762..acef8b86ce 100644
@@ -159,6 +159,9 @@ set -e
     image_task = d.getVar('INITRAMFS_TASK')
     if image_task:
         d.appendVarFlag('do_configure', 'depends', ' ${INITRAMFS_TASK}')
+    if d.getVar("SPDX_INCLUDE_COMPILED_SOURCES") == "1":
+        bb.build.addtask('do_save_compiled_files', 'do_build', 'do_compile do_compile_kernelmodules', d)
+        d.appendVarFlag('do_create_spdx', 'depends', f'{pn}:do_save_compiled_files')
 }
 
 # Here we pull in all various kernel image types which we support.
@@ -867,3 +870,19 @@ EXPORT_FUNCTIONS do_deploy
 
 # Add using Device Tree support
 inherit kernel-devicetree
+
+# in case we don't use spdx bbclass
+SPDX_COMPILED_SOURCES_DIR ?= "${LOG_DIR}/spdx-compiled/${PN}"
+SPDX_COMPILED_SOURCES ?= "${SPDX_COMPILED_SOURCES_DIR}/compiled_src-${MACHINE}-${BP}.txt"
+
+KERNEL_COMPILED_FILES ?= "${SPDX_COMPILED_SOURCES_DIR}/compiled_commands-${MACHINE}-${BP}.json"
+
+do_save_compiled_files() {
+    bbdebug 1 "Saving compiled files in ${SPDX_COMPILED_SOURCES}"
+    mkdir -p ${SPDX_COMPILED_SOURCES_DIR}
+    ${S}/scripts/clang-tools/gen_compile_commands.py -o ${KERNEL_COMPILED_FILES} -d ${B}
+    # Make paths releative to the kernel source
+    sed -i 's|${B}/||g' ${KERNEL_COMPILED_FILES}
+    sed -i 's|${S}/||g' ${KERNEL_COMPILED_FILES}
+    grep '"file": ' ${KERNEL_COMPILED_FILES} | awk '{print $2}' | tr -d '"' > ${SPDX_COMPILED_SOURCES}
+}