Check compiled files to filter kernel CVEs
v2 cover Check compiled files to filter kernel CVEs
From: Daniel Turull <[email protected]>
Since kernel.org became a CNA, more information is available in the published CVEs, including details about which files are affected by a given CVE.
This series adds functionality to fetch the database from kernel.org CNA information:
git.kernel.org/pub/scm/linux/security/vulns.git
I have rewritten the original patch and integrated it with cve_check.py, cve_check, vex and create-spdx.
To filter out CVEs that are not applicable, we extract the files used during the kernel compilation and compare it with the metadata in the CVE.
The CVE_CHECK_KERNEL is enabled by default, but not the CVE_CHECK_KERNEL_CONFIG, since it needs to build the kernel and it is dependent on kernel configuration.
To include the files into SPDX, SPDX_INCLUDE_SOURCES needs to be enabled.
This could use as a base to run the vulnerability check independently and run an external tool to filter the CVEs.
In addition, we also integrate the data with the output of cve_check, vex, and create-spdx.
As a side effect of using better data from directly the kernel CNA, the number of vulnerabilities reported increases.
Numbers from 2025-04-28
Standing CVEs:
- Before: 18 CVEs, mostly old that needs to be checked and clean up in a new commit. There is only one from 2025
- Checking with kernel vulns database: 82 CVEs
- Checking with only compiled files: 46 CVEs
v1: initial proposal
v2:
- rewrite kernel_vulns to fetch similarly as cve-update-db-native
- add functionality into cve_check.py, for the classes that uses oe.get_patched_cves function
- add linux-vulns into the cve-check results
- add only compiled files in the spdx, so the check can be done outside the build
- include compiled files into spdx when CVE_CHECK_KERNEL_CONFIG and SPDX_INCLUDE_SOURCES is enabled
Daniel Turull
Daniel Turull (6):
linux-vulns: fetch kernel.org CNA info
cve-check: fix debug message
kernel: add support to extract compiled files
cve-check: move message outsite check_cves and sort cve-check, vex, spdx: use metadata from linux-vulns to enhance CVE
reporting
spdx: add option to include only compiled kernel files
meta/classes-recipe/kernel.bbclass | 11 ++ meta/classes/create-spdx-2.2.bbclass | 8 + meta/classes/cve-check.bbclass | 35 +++- meta/classes/spdx-common.bbclass | 7 + meta/classes/vex.bbclass | 10 ++ meta/conf/distro/include/maintainers.inc | 1 + meta/lib/oe/cve_check.py | 210 +++++++++++++++++++++- meta/lib/oe/spdx30_tasks.py | 8 + meta/lib/oe/spdx_common.py | 34 ++++ meta/recipes-core/meta/linux-vulns_git.bb | 76 ++++++++ 10 files changed, 391 insertions(+), 9 deletions(-) create mode 100644 meta/recipes-core/meta/linux-vulns_git.bb
PATCH v2 1/6 linux-vulns: fetch kernel.org CNA info
From: Daniel Turull <[email protected]>
Add CVE data source for kernel.org.
It includes more information than the one provided by NVD. Use similar mechanism and same variables as cve-check to define when to update.
To use without internet access, change variable VULNS_URL to a local copy or mirror.
--- meta/conf/distro/include/maintainers.inc | 1 + meta/recipes-core/meta/linux-vulns_git.bb | 76 +++++++++++++++++++++++ 2 files changed, 77 insertions(+) create mode 100644 meta/recipes-core/meta/linux-vulns_git.bb
meta/conf/distro/include/maintainers.inc +1 -0
@@ -468,6 +468,7 @@ RECIPE_MAINTAINER:pn-lighttpd = "Unassigned <[email protected]>" RECIPE_MAINTAINER:pn-linux-dummy = "Unassigned <[email protected]>" RECIPE_MAINTAINER:pn-linux-firmware = "Otavio Salvador <[email protected]>" RECIPE_MAINTAINER:pn-linux-libc-headers = "Bruce Ashfield <[email protected]>" +RECIPE_MAINTAINER:pn-linux-vulns = "Unassigned <[email protected]>" RECIPE_MAINTAINER:pn-linux-yocto = "Bruce Ashfield <[email protected]>" RECIPE_MAINTAINER:pn-linux-yocto-dev = "Bruce Ashfield <[email protected]>" RECIPE_MAINTAINER:pn-linux-yocto-rt = "Bruce Ashfield <[email protected]>"
meta/recipes-core/meta/linux-vulns_git.bb new file +76 -0
76 lines not on this page · raw message
PATCH v2 2/6 cve-check: fix debug message
In reply to [email protected]From: Daniel Turull <[email protected]>
Debug level was not added as a parameter, causing a warning.
--- meta/classes/cve-check.bbclass | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
meta/classes/cve-check.bbclass +3 -3
@@ -288,16 +288,16 @@ def cve_update(d, cve_data, cve, entry): if entry['status'] == "version-in-range" and cve_data[cve]['status'] == "version-not-in-range": # New result from the scan, vulnerable cve_data[cve] = entry - bb.debug("CVE entry %s update from Patched to Unpatched from the scan result" % cve) + bb.debug(1, "CVE entry %s update from Patched to Unpatched from the scan result" % cve) return if entry['abbrev-status'] == "Patched" and cve_data[cve]['abbrev-status'] == "Unpatched": if entry['status'] == "version-not-in-range" and cve_data[cve]['status'] == "version-in-range": # Range does not match the scan, but we already have a vulnerable match, ignore - bb.debug("CVE entry %s update from Patched to Unpatched from the scan result - not applying" % cve) + bb.debug(1, "CVE entry %s update from Patched to Unpatched from the scan result - not applying" % cve) return # If we have an "Ignored", it has a priority if cve_data[cve]['abbrev-status'] == "Ignored": - bb.debug("CVE %s not updating because Ignored" % cve) + bb.debug(1, "CVE %s not updating because Ignored" % cve) return bb.warn("Unhandled CVE entry update for %s from %s to %s" % (cve, cve_data[cve], entry))
PATCH v2 3/6 kernel: add support to extract compiled files
In reply to [email protected]From: Daniel Turull <[email protected]>
Use gen_compile_commands.py to extract files used during compilation for the used kernel configuration.
--- meta/classes-recipe/kernel.bbclass | 11 +++++++++++ 1 file changed, 11 insertions(+)
meta/classes-recipe/kernel.bbclass +11 -0
@@ -867,3 +867,14 @@ EXPORT_FUNCTIONS do_deploy # Add using Device Tree support inherit kernel-devicetree + +KERNEL_FILES_DIR ?= "${LOG_DIR}/cve/kernel_files" +KERNEL_SRC_FILES ?= "${KERNEL_FILES_DIR}/compile_commands.json" + +do_save_compiled_files() { + bbdebug 1 "Saving compiled files in ${KERNEL_SRC_FILES}" + mkdir -p ${KERNEL_FILES_DIR} + ${S}/scripts/clang-tools/gen_compile_commands.py -o ${KERNEL_SRC_FILES} -d ${B} +} + +addtask do_save_compiled_files after do_compile do_compile_kernelmodules
Re: [OE-core] [PATCH v2 3/6] kernel: add support to extract compiled files
Quoted 30 lines · Daniel Turull via lists.openembedded.org
From: Daniel Turull <[email protected]>
Use
gen_compile_commands.pyto extract files used during compilation for the used kernel configuration.CC Peter Marko <[email protected]>Signed-off-by Daniel Turull <[email protected]>--- meta/classes-recipe/kernel.bbclass | 11 +++++++++++ 1 file changed, 11 insertions(+)meta/classes-recipe/kernel.bbclass +8
-0@@ -867,3 +867,14 @@ EXPORT_FUNCTIONS do_deploy # Add using Device Tree support inherit kernel-devicetree + +KERNEL_FILES_DIR ?= "${LOG_DIR}/cve/kernel_files" +KERNEL_SRC_FILES ?= "${KERNEL_FILES_DIR}/compile_commands.json" + +do_save_compiled_files() { + bbdebug 1 "Saving compiled files in ${KERNEL_SRC_FILES}" + mkdir -p ${KERNEL_FILES_DIR} + ${S}/scripts/clang-tools/gen_compile_commands.py -o${KERNEL_SRC_FILES} -d ${B}
meta/classes-recipe/kernel.bbclass EXPORT_FUNCTIONS do_deploy · line 880
Quoted 3 lines
meta/classes-recipe/kernel.bbclass +3
-0+} + +addtask do_save_compiled_files after do_compile do_compile_kernelmodules
This is something that I wouldn't want running in my development environment.
I can't tell fully. Is this opt in ? Is there a way to opt-out ?
Bruce
Quoted 12 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#215604):
https://lists.openembedded.org/g/openembedded-core/message/215604
Mute This Topic: https://lists.openembedded.org/mt/112499916/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
RE: [OE-core] [PATCH v2 3/6] kernel: add support to extract compiled files
Hi,
I have put it as default, but you have a good point. I’ll add the task conditionally if CVE_CHECK_KERNEL_CONFIG is set to 1 in the next patch version series.
Best regards,
Daniel
From: Bruce Ashfield <[email protected]>
Sent: Monday, 28 April 2025 16:25
To: Daniel Turull <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [OE-core] [PATCH v2 3/6] kernel: add support to extract compiled files
On Mon, Apr 28, 2025 at 9:42 AM Daniel Turull via lists.openembedded.org<http://lists.openembedded.org/> <[email protected]<mailto:[email protected]>> wrote:
From: Daniel Turull <[email protected]<mailto:[email protected]>>
Use gen_compile_commands.py to extract files used during compilation for the used kernel configuration.
--- meta/classes-recipe/kernel.bbclass | 11 +++++++++++ 1 file changed, 11 insertions(+)
meta/classes-recipe/kernel.bbclass +11 -0
@@ -867,3 +867,14 @@ EXPORT_FUNCTIONS do_deploy # Add using Device Tree support inherit kernel-devicetree + +KERNEL_FILES_DIR ?= "${LOG_DIR}/cve/kernel_files" +KERNEL_SRC_FILES ?= "${KERNEL_FILES_DIR}/compile_commands.json" + +do_save_compiled_files() { + bbdebug 1 "Saving compiled files in ${KERNEL_SRC_FILES}" + mkdir -p ${KERNEL_FILES_DIR} + ${S}/scripts/clang-tools/gen_compile_commands.py -o ${KERNEL_SRC_FILES} -d ${B} +} + +addtask do_save_compiled_files after do_compile do_compile_kernelmodules
This is something that I wouldn't want running in my development environment.
I can't tell fully. Is this opt in ? Is there a way to opt-out ?
Bruce
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#215604): https://lists.openembedded.org/g/openembedded-core/message/215604
Mute This Topic: https://lists.openembedded.org/mt/112499916/1050810
Group Owner: [email protected]<mailto:openembedded-core%[email protected]>
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]<mailto:[email protected]>]
-=-=-=-=-=-=-=-=-=-=-=-
--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
PATCH v2 4/6 cve-check: move message outsite check_cves and sort
In reply to [email protected]From: Daniel Turull <[email protected]>
When adding corrections from multiple sources of CVEs, the message showing standing CVES should be at the end. Also sort them.
--- meta/classes/cve-check.bbclass | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
meta/classes/cve-check.bbclass +6 -4
@@ -173,6 +173,12 @@ python do_cve_check () { if len(cve_data) or (d.getVar("CVE_CHECK_COVERAGE") == "1" and status): get_cve_info(d, cve_data) cve_write_data(d, cve_data, status) + + if d.getVar("CVE_CHECK_SHOW_WARNINGS") == "1": + unpatched_cves = [cve for cve in cve_data if cve_data[cve]["abbrev-status"] == "Unpatched"] + if unpatched_cves: + bb.warn("Found unpatched CVE (%s)" % " ".join(sorted(unpatched_cves))) + else: bb.note("No CVE database found, skipping CVE check")
@@ -422,10 +428,6 @@ def check_cves(d, cve_data): if not cves_in_recipe: bb.note("No CVE records for products in recipe %s" % (pn)) - if d.getVar("CVE_CHECK_SHOW_WARNINGS") == "1": - unpatched_cves = [cve for cve in cve_data if cve_data[cve]["abbrev-status"] == "Unpatched"] - if unpatched_cves: - bb.warn("Found unpatched CVE (%s)" % " ".join(unpatched_cves)) return (cve_data, cves_status)
PATCH v2 5/6 cve-check, vex, spdx: use metadata from linux-vulns to enhance CVE reporting
In reply to [email protected]From: Daniel Turull <[email protected]>
Introducing two new options to check kernel CVEs using linux-vulns.
- CVE_CHECK_KERNEL = "1"
To add linux-vulns metadata on cve-report for cve_check, vex and spdx. This will check for kernel CVEs, and use the metadata to resolve them. Enabled by default
- CVE_CHECK_KERNEL_CONFIG = "1"
CVE Check using kernel compiled files, disabled by default since it requires a compiled kernel and it will increase cve-check times.
Metadata in the CVE information includes the affected files, and using the compiled files by the kernel we can ignore some of the cves.
The above variables are defined in cve_check.bbclass, vex.bbclass and spdx.bbclass in case not all classes are used at the same time. The ones in cve_check has priority.
Example of output with CVE_CHECK_KERNEL_CONFIG when using cve-check:
{
"id": "CVE-2024-26710",
"status": "Ignored",
"link": "https://nvd.nist.gov/vuln/detail/CVE-2024-26710",
"summary": "In the Linux kernel, the following vulnerability [...]",
"scorev2": "0.0",
"scorev3": "5.5",
"scorev4": "0.0",
"modified": "2025-03-17T15:36:11.620",
"vector": "LOCAL",
"vectorString": "CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H",
"detail": "not-applicable-config",
"description": "Source code not compiled by config. ['arch/powerpc/include/asm/thread_info.h']"
},
And same with vex:
{
"id": "CVE-2024-26710",
"status": "Ignored",
"link": "https://nvd.nist.gov/vuln/detail/CVE-2024-26710",
"detail": "not-applicable-config",
"description": "Source code not compiled by config. ['arch/powerpc/include/asm/thread_info.h']"
},
And new information for the Unpatched showing where the fix (if any) is available:
Tested with 6.12.22 kernel
{
"id": "CVE-2025-39728",
"status": "Unpatched",
"link": "https://nvd.nist.gov/vuln/detail/CVE-2025-39728", "summary": "In the Linux kernel, the following vulnerability has been [...], "scorev2": "0.0",
"scorev3": "0.0",
"scorev4": "0.0",
"modified": "2025-04-21T14:23:45.950",
"vector": "UNKNOWN",
"vectorString": "UNKNOWN",
"detail": "version-in-range",
"description": "Needs backporting (fixed from 6.12.23)"
},
Tested with cve-check, create-spdx2.2, create-spdx3.0, vex
--- meta/classes/cve-check.bbclass | 19 ++- meta/classes/spdx-common.bbclass | 7 ++ meta/classes/vex.bbclass | 10 ++ meta/lib/oe/cve_check.py | 210 ++++++++++++++++++++++++++++++- 4 files changed, 244 insertions(+), 2 deletions(-)
meta/classes/cve-check.bbclass +18 -1
@@ -34,6 +34,13 @@ CVE_VERSION ??= "${PV}" # Possible database sources: NVD1, NVD2, FKIE NVD_DB_VERSION ?= "FKIE" +# CVE Check using kernel CNA and compiled files +CVE_CHECK_KERNEL ?= "1" +# CVE Check using kernel compiled files +CVE_CHECK_KERNEL_CONFIG ?= "0" +# Location of the linux-vulns data +CVE_CHECK_KERNEL_DB_DIR ?= "${DL_DIR}/CVE_CHECK/vulns" + # Use different file names for each database source, as they synchronize at different moments, so may be slightly different CVE_CHECK_DB_FILENAME ?= "${@'nvdcve_2-2.db' if d.getVar('NVD_DB_VERSION') == 'NVD2' else 'nvdcve_1-3.db' if d.getVar('NVD_DB_VERSION') == 'NVD1' else 'nvdfkie_1-1.db'}" CVE_CHECK_DB_FETCHER ?= "${@'cve-update-nvd2-native' if d.getVar('NVD_DB_VERSION') == 'NVD2' else 'cve-update-db-native'}"
@@ -111,6 +118,9 @@ python () { if nvd_database_type not in ("NVD1", "NVD2", "FKIE"): bb.erroronce("Malformed NVD_DB_VERSION, must be one of: NVD1, NVD2, FKIE. Defaulting to NVD2") d.setVar("NVD_DB_VERSION", "NVD2") + + from oe.cve_check import extend_cve_kernel_config + extend_cve_kernel_config(d, "do_cve_check") } def generate_json_report(d, out_path, link_path):
@@ -161,7 +171,7 @@ python do_cve_check () { """ Check recipe for patched and unpatched CVEs """ - from oe.cve_check import get_patched_cves + from oe.cve_check import get_patched_cves, get_kernel_cves with bb.utils.fileslocked([d.getVar("CVE_CHECK_DB_FILE_LOCK")], shared=True): if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")):
@@ -170,6 +180,13 @@ python do_cve_check () { except FileNotFoundError: bb.fatal("Failure in searching patches") cve_data, status = check_cves(d, patched_cves) + + cve_check_kernel = d.getVar("CVE_CHECK_KERNEL") + if "linux_kernel" in d.getVar("CVE_PRODUCT") and cve_check_kernel == "1": + kernel_unpatched_cves, _kernel_patched_cves = get_kernel_cves(d) + # Patched CVEs are already returned in the get_patched_cves(). Adding the unpatched + cve_data.update(kernel_unpatched_cves) + if len(cve_data) or (d.getVar("CVE_CHECK_COVERAGE") == "1" and status): get_cve_info(d, cve_data) cve_write_data(d, cve_data, status)
meta/classes/spdx-common.bbclass +7 -0
@@ -37,9 +37,16 @@ SPDX_CUSTOM_ANNOTATION_VARS ??= "" SPDX_MULTILIB_SSTATE_ARCHS ??= "${SSTATE_ARCHS}" +CVE_CHECK_KERNEL ??= "0" +CVE_CHECK_KERNEL_CONFIG ??= "0" +CVE_CHECK_KERNEL_DB_DIR ?= "${DL_DIR}/CVE_CHECK/vulns" + python () { from oe.cve_check import extend_cve_status extend_cve_status(d) + + from oe.cve_check import extend_cve_kernel_config + extend_cve_kernel_config(d, "do_create_spdx") } def create_spdx_source_deps(d):
meta/classes/vex.bbclass +10 -0
@@ -26,6 +26,13 @@ CVE_PRODUCT ??= "${BPN}" CVE_VERSION ??= "${PV}" +# CVE Check using kernel CNA and compiled files +CVE_CHECK_KERNEL ?= "0" +# CVE Check using kernel compiled files +CVE_CHECK_KERNEL_CONFIG ?= "0" + +CVE_CHECK_KERNEL_DB_DIR ?= "${DL_DIR}/CVE_CHECK/vulns" + CVE_CHECK_SUMMARY_DIR ?= "${LOG_DIR}/cve" CVE_CHECK_SUMMARY_FILE_NAME_JSON = "cve-summary.json"
@@ -78,6 +85,9 @@ python () { from oe.cve_check import extend_cve_status extend_cve_status(d) + + from oe.cve_check import extend_cve_kernel_config + extend_cve_kernel_config(d, "do_generate_vex") } def generate_json_report(d, out_path, link_path):
meta/lib/oe/cve_check.py +209 -1
232 lines not on this page · raw message
PATCH v2 6/6 spdx: add option to include only compiled kernel files
In reply to [email protected]From: Daniel Turull <[email protected]>
When CVE_CHECK_KERNEL_CONFIG is enabled, only include the source code (.c, .h) files that are used during compilation.
This enables an external tool to use the SPDX information to disregard vulnerabilities that are not compiled.
--- meta/classes/create-spdx-2.2.bbclass | 8 +++++++ meta/lib/oe/spdx30_tasks.py | 8 +++++++ meta/lib/oe/spdx_common.py | 34 ++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+)
meta/classes/create-spdx-2.2.bbclass +8 -0
@@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv spdx_files = [] file_counter = 1 + + check_kernel_compiled = bb.data.inherits_class("kernel", d) and d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" + if check_kernel_compiled: + kernel_sources = oe.spdx_common.get_kernel_compiled_files(d) for subdir, dirs, files in os.walk(topdir): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):
@@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv filename = str(filepath.relative_to(topdir)) if not filepath.is_symlink() and filepath.is_file(): + # When creating spdx for the kernel, we only include compiled files. + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): + break spdx_file = oe.spdx.SPDXFile() spdx_file.SPDXID = get_spdxid(file_counter) for t in get_types(filepath):
meta/lib/oe/spdx30_tasks.py +8 -0
@@ -156,6 +156,10 @@ def add_package_files( bb.note(f"Skip {topdir}") return spdx_files + check_kernel_compiled = bb.data.inherits_class("kernel", d) and d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" + if check_kernel_compiled: + kernel_sources = oe.spdx_common.get_kernel_compiled_files(d) + for subdir, dirs, files in os.walk(topdir, onerror=walk_error): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):
@@ -167,6 +171,10 @@ def add_package_files( filepath = Path(subdir) / file if filepath.is_symlink() or not filepath.is_file(): continue + # When creating spdx for the kernel, we only include compiled files + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): + break filename = str(filepath.relative_to(topdir)) file_purposes = get_purposes(filepath)
meta/lib/oe/spdx_common.py +34 -0
@@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name): uri = uri + "@" + fd.revision return uri + +def is_kernel_compiled(filename, kernel_sources, d): + """ + Check if the file, is a kernel compiled file + """ + import os + + _, extension = os.path.splitext(filename) + # Special case, that we need to ignore, since this is not a source file + if filename.rfind(".mod.c") > 0: + return True + # We filter .c files and header files + if extension not in [".c", ".h"]: + return True + # Check that the c file is in the list + if filename in kernel_sources: + return True + return False + +def get_kernel_compiled_files(d): + """ + Get results from the save_compiled files and include also header files + """ + import json + import os + kfiles = [] + with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f: + for item in json.load(f): + kfile = os.path.basename(item['file']) + # Return also the correspondig header file + hfile = ".h".join(kfile.rsplit(".c", 1)) + kfiles.append(kfile) + kfiles.append(hfile) + return kfiles
Re: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
I see the appeal here; however I've been trying to avoid too much kernel specific stuff in SPDX. Is there a way to make this more generic so it could be used by any recipe?
Quoted 99 lines · On Mon, Apr 28, 2025 at 7:42 AM <[email protected]> wrote:
From: Daniel Turull <[email protected]>
When
CVE_CHECK_KERNEL_CONFIGis enabled, only include the source code (.c, .h) files that are used during compilation.This enables an external tool to use the SPDX information to disregard vulnerabilities that are not compiled.
CC Joshua Watt <[email protected]>CC Peter Marko <[email protected]>Signed-off-by Daniel Turull <[email protected]>--- meta/classes/create-spdx-2.2.bbclass | 8 +++++++ meta/lib/oe/spdx30_tasks.py | 8 +++++++ meta/lib/oe/spdx_common.py | 34 ++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+)meta/classes/create-spdx-2.2.bbclass +8
-0@@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv spdx_files = [] file_counter = 1 + + check_kernel_compiled = bb.data.inherits_class("kernel", d) and d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" + if check_kernel_compiled: + kernel_sources = oe.spdx_common.get_kernel_compiled_files(d) for subdir, dirs, files in os.walk(topdir): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):@@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv filename = str(filepath.relative_to(topdir)) if not filepath.is_symlink() and filepath.is_file(): + # When creating spdx for the kernel, we only include compiled files. + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): + break spdx_file = oe.spdx.SPDXFile() spdx_file.SPDXID = get_spdxid(file_counter) for t in get_types(filepath):meta/lib/oe/spdx30_tasks.py +8
-0@@ -156,6 +156,10 @@ def add_package_files( bb.note(f"Skip {topdir}") return spdx_files + check_kernel_compiled = bb.data.inherits_class("kernel", d) and d.getVar("CVE_CHECK_KERNEL_CONFIG") == "1" + if check_kernel_compiled: + kernel_sources = oe.spdx_common.get_kernel_compiled_files(d) + for subdir, dirs, files in os.walk(topdir, onerror=walk_error): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):@@ -167,6 +171,10 @@ def add_package_files( filepath = Path(subdir) / file if filepath.is_symlink() or not filepath.is_file(): continue + # When creating spdx for the kernel, we only include compiled files + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): + break filename = str(filepath.relative_to(topdir)) file_purposes = get_purposes(filepath)meta/lib/oe/spdx_common.py +21
-0@@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name): uri = uri + "@" + fd.revision return uri + +def is_kernel_compiled(filename, kernel_sources, d): + """ + Check if the file, is a kernel compiled file + """ + import os + + _, extension = os.path.splitext(filename) + # Special case, that we need to ignore, since this is not a source file + if filename.rfind(".mod.c") > 0: + return True + # We filter .c files and header files + if extension not in [".c", ".h"]: + return True + # Check that the c file is in the list + if filename in kernel_sources: + return True + return False + +def get_kernel_compiled_files(d): + """
meta/lib/oe/spdx_common.py def fetch_data_to_uri(fd, name): · line 275
Quoted 10 lines
meta/lib/oe/spdx_common.py +10
-0+ Get results from the save_compiled files and include also header files + """ + import json + import os + kfiles = [] + with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f: + for item in json.load(f): + kfile = os.path.basename(item['file']) + # Return also the correspondig header file + hfile = ".h".join(kfile.rsplit(".c", 1))
This feels a little bit wrong to me. I think a better option would be to include all headers, since we don't have a good way of knowing which are actually used.
Quoted 3 lines
meta/lib/oe/spdx_common.py +3
-0+ kfiles.append(kfile) + kfiles.append(hfile) + return kfiles
RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
I think that only kernel provides file-based CVE information right now.
Peter
Quoted 137 lines
-----Original Message-----
From: Joshua Watt <[email protected]>
Sent: Monday, April 28, 2025 15:58
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; Marko, Peter (FT D EU SK BFS1) <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH v2 6/6] spdx: add option to include only compiled kernel filesI see the appeal here; however I've been trying to avoid too much kernel specific stuff in SPDX. Is there a way to make this more generic so it could be used by any recipe?
meta/classes/create-spdx-
Quoted 19 lines · On Mon, Apr 28, 2025 at 7:42 AM <[email protected]> wrote:
From: Daniel Turull <[email protected]>
When
CVE_CHECK_KERNEL_CONFIGis enabled, only include the source code (.c, .h) files that are used during compilation.This enables an external tool to use the SPDX information to disregard vulnerabilities that are not compiled.
CC Joshua Watt <[email protected]>CC Peter Marko <[email protected]>Signed-off-by Daniel Turull <[email protected]>--- meta/classes/create-spdx-2.2.bbclass | 8 +++++++ meta/lib/oe/spdx30_tasks.py | 8 +++++++ meta/lib/oe/spdx_common.py | 34 ++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+)meta/classes/create-spdx- renamed from meta/classes/create-spdx-2.2.bbclass +0
-02.2.bbclass
meta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir,
Quoted 4 lines
index
7e8f8b9ff5..6bf0c70bd4100644meta/classes/create-spdx-2.2.bbclass +0
-0@@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
get_spdxid,get_types, *, archivmeta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
Quoted 5 lines
meta/classes/create-spdx-2.2.bbclass +2
-0spdx_files = [] file_counter = 1 + + check_kernel_compiled = bb.data.inherits_class("kernel", d) and
d.getVar("CVE_CHECK_KERNEL_CONFIG")== "1"meta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir,
Quoted 6 lines
meta/classes/create-spdx-2.2.bbclass +2
-0+ if check_kernel_compiled: + kernel_sources = oe.spdx_common.get_kernel_compiled_files(d) for subdir, dirs, files in os.walk(topdir): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):@@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
get_spdxid,get_types, *, archivmeta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
Quoted 6 lines
meta/classes/create-spdx-2.2.bbclass +3
-0filename = str(filepath.relative_to(topdir)) if not filepath.is_symlink() and filepath.is_file(): + # When creating spdx for the kernel, we only include compiled files. + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources,d):
meta/lib/oe/spdx30_tasks.py def add_package_files( · line 159
Quoted 13 lines
meta/classes/create-spdx-2.2.bbclass +1
-0+ break spdx_file = oe.spdx.SPDXFile() spdx_file.SPDXID = get_spdxid(file_counter) for t in get_types(filepath):meta/lib/oe/spdx30_tasks.py +1
-0@@ -156,6 +156,10 @@ def add_package_files( bb.note(f"Skip {topdir}") return spdx_files + check_kernel_compiled = bb.data.inherits_class("kernel", d) and
d.getVar("CVE_CHECK_KERNEL_CONFIG")== "1"Quoted 46 lines
meta/lib/oe/spdx30_tasks.py +7
-0+ if check_kernel_compiled: + kernel_sources = oe.spdx_common.get_kernel_compiled_files(d) + for subdir, dirs, files in os.walk(topdir, onerror=walk_error): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):@@ -167,6 +171,10 @@ def add_package_files( filepath = Path(subdir) / file if filepath.is_symlink() or not filepath.is_file(): continue + # When creating spdx for the kernel, we only include compiled files + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): + break filename = str(filepath.relative_to(topdir)) file_purposes = get_purposes(filepath)meta/lib/oe/spdx_common.py +21
-0@@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name): uri = uri + "@" + fd.revision return uri + +def is_kernel_compiled(filename, kernel_sources, d): + """ + Check if the file, is a kernel compiled file + """ + import os + + _, extension = os.path.splitext(filename) + # Special case, that we need to ignore, since this is not a source file + if filename.rfind(".mod.c") > 0: + return True + # We filter .c files and header files + if extension not in [".c", ".h"]: + return True + # Check that the c file is in the list + if filename in kernel_sources: + return True + return False + +def get_kernel_compiled_files(d): + """meta/lib/oe/spdx_common.py def fetch_data_to_uri(fd, name): · line 275
Quoted 10 lines
meta/lib/oe/spdx_common.py +10
-0+ Get results from the save_compiled files and include also header files + """ + import json + import os + kfiles = [] + with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f: + for item in json.load(f): + kfile = os.path.basename(item['file']) + # Return also the correspondig header file + hfile = ".h".join(kfile.rsplit(".c", 1))This feels a little bit wrong to me. I think a better option would be to include all headers, since we don't have a good way of knowing which are actually used.
Quoted 3 lines
meta/lib/oe/spdx_common.py +3
-0+ kfiles.append(kfile) + kfiles.append(hfile) + return kfiles
RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
Hi,
Yes, from what I saw and Peter says only the kernel has file information that can be machine readable, and we can also extract the files that we compile.
I could try to normalize how the files are input into the spdx generation and then have a generic variable that any recipe can set to enable the inclusion of a subset of files. In this case, if/when more recipes provides this,
spdx doesn't need to change.
Best regards
Daniel
-----Original Message-----
From: Marko, Peter <[email protected]>
Sent: Monday, 28 April 2025 16:00
To: Joshua Watt <[email protected]>; Daniel Turull <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
I think that only kernel provides file-based CVE information right now.
Peter
Quoted 142 lines
-----Original Message-----
From: Joshua Watt <[email protected]>
Sent: Monday, April 28, 2025 15:58
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; Marko, Peter (FT D EU SK BFS1) <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH v2 6/6] spdx: add option to include only compiled kernel filesI see the appeal here; however I've been trying to avoid too much kernel specific stuff in SPDX. Is there a way to make this more generic so it could be used by any recipe?
meta/classes/create-spdx-
Quoted 20 lines · On Mon, Apr 28, 2025 at 7:42 AM <[email protected]> wrote:
From: Daniel Turull <[email protected]>
When
CVE_CHECK_KERNEL_CONFIGis enabled, only include the source code (.c, .h) files that are used during compilation.This enables an external tool to use the SPDX information to disregard vulnerabilities that are not compiled.
CC Joshua Watt <[email protected]>CC Peter Marko <[email protected]>Signed-off-by Daniel Turull <[email protected]>--- meta/classes/create-spdx-2.2.bbclass | 8 +++++++ meta/lib/oe/spdx30_tasks.py | 8 +++++++ meta/lib/oe/spdx_common.py | 34 ++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+)meta/classes/create-spdx- renamed from meta/classes/create-spdx-2.2.bbclass +0
-02.2.bbclass
meta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir,
Quoted 4 lines
index
7e8f8b9ff5..6bf0c70bd4100644meta/classes/create-spdx-2.2.bbclass +0
-0@@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
get_spdxid,get_types, *, archivmeta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
Quoted 5 lines
meta/classes/create-spdx-2.2.bbclass +2
-0spdx_files = [] file_counter = 1 + + check_kernel_compiled = bb.data.inherits_class("kernel", d) and
d.getVar("CVE_CHECK_KERNEL_CONFIG")== "1"meta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir,
Quoted 7 lines
meta/classes/create-spdx-2.2.bbclass +3
-0+ if check_kernel_compiled: + kernel_sources = + oe.spdx_common.get_kernel_compiled_files(d) for subdir, dirs, files in os.walk(topdir): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):@@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
get_spdxid,get_types, *, archivmeta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
Quoted 7 lines
meta/classes/create-spdx-2.2.bbclass +4
-0filename = str(filepath.relative_to(topdir)) if not filepath.is_symlink() and filepath.is_file(): + # When creating spdx for the kernel, we only include compiled files. + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, + kernel_sources,d):
meta/lib/oe/spdx30_tasks.py def add_package_files( · line 159
Quoted 13 lines
meta/classes/create-spdx-2.2.bbclass +1
-0+ break spdx_file = oe.spdx.SPDXFile() spdx_file.SPDXID = get_spdxid(file_counter) for t in get_types(filepath):(quoted diff) +0
-0
b/meta/lib/oe/spdx30_tasks.pyindexba965821f8..14f26773c5100644meta/lib/oe/spdx30_tasks.py +1
-0@@ -156,6 +156,10 @@ def add_package_files( bb.note(f"Skip {topdir}") return spdx_files + check_kernel_compiled = bb.data.inherits_class("kernel", d) and
d.getVar("CVE_CHECK_KERNEL_CONFIG")== "1"Quoted 47 lines
meta/lib/oe/spdx30_tasks.py +8
-0+ if check_kernel_compiled: + kernel_sources = + oe.spdx_common.get_kernel_compiled_files(d) + for subdir, dirs, files in os.walk(topdir, onerror=walk_error): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):@@ -167,6 +171,10 @@ def add_package_files( filepath = Path(subdir) / file if filepath.is_symlink() or not filepath.is_file(): continue + # When creating spdx for the kernel, we only include compiled files + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): + break filename = str(filepath.relative_to(topdir)) file_purposes = get_purposes(filepath) diff --git
a/meta/lib/oe/spdx_common.pyb/meta/lib/oe/spdx_common.pyindex4caefc7673..c87e3875c7100644meta/lib/oe/spdx_common.py +21
-0@@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name): uri = uri + "@" + fd.revision return uri + +def is_kernel_compiled(filename, kernel_sources, d): + """ + Check if the file, is a kernel compiled file + """ + import os + + _, extension = os.path.splitext(filename) + # Special case, that we need to ignore, since this is not a source file + if filename.rfind(".mod.c") > 0: + return True + # We filter .c files and header files + if extension not in [".c", ".h"]: + return True + # Check that the c file is in the list + if filename in kernel_sources: + return True + return False + +def get_kernel_compiled_files(d): + """meta/lib/oe/spdx_common.py def fetch_data_to_uri(fd, name): · line 275
Quoted 10 lines
meta/lib/oe/spdx_common.py +10
-0+ Get results from the save_compiled files and include also header files + """ + import json + import os + kfiles = [] + with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f: + for item in json.load(f): + kfile = os.path.basename(item['file']) + # Return also the correspondig header file + hfile = ".h".join(kfile.rsplit(".c", 1))This feels a little bit wrong to me. I think a better option would be to include all headers, since we don't have a good way of knowing which are actually used.
Quoted 3 lines
meta/lib/oe/spdx_common.py +3
-0+ kfiles.append(kfile) + kfiles.append(hfile) + return kfiles
Re: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
Quoted 9 lines · Daniel Turull
Hi,
Yes, from what I saw and Peter says only the kernel has file information that can be machine readable, and we can also extract the files that we compile.
I could try to normalize how the files are input into the spdx generation and then have a generic variable that any recipe can set to enable the inclusion of a subset of files. In this case, if/when more recipes provides this,
spdx doesn't need to change.
I was thinking that maybe the way to do this is to have this somehow call a function when enabled, and then have the recipe (or even better the build system bbclass) implement that function; that way each recipe or build system can implement the correct method for collecting the data. I'm not quite sure if that's the best way to do it though
Quoted 157 lines
Best regards
Daniel-----Original Message-----
From: Marko, Peter <[email protected]>
Sent: Monday, 28 April 2025 16:00
To: Joshua Watt <[email protected]>; Daniel Turull <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel filesI think that only kernel provides file-based CVE information right now.
Peter
Quoted 142 lines
-----Original Message-----
From: Joshua Watt <[email protected]>
Sent: Monday, April 28, 2025 15:58
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; Marko, Peter (FT D EU SK BFS1) <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH v2 6/6] spdx: add option to include only compiled kernel filesI see the appeal here; however I've been trying to avoid too much kernel specific stuff in SPDX. Is there a way to make this more generic so it could be used by any recipe?
meta/classes/create-spdx-
Quoted 20 lines · On Mon, Apr 28, 2025 at 7:42 AM <[email protected]> wrote:
From: Daniel Turull <[email protected]>
When
CVE_CHECK_KERNEL_CONFIGis enabled, only include the source code (.c, .h) files that are used during compilation.This enables an external tool to use the SPDX information to disregard vulnerabilities that are not compiled.
CC Joshua Watt <[email protected]>CC Peter Marko <[email protected]>Signed-off-by Daniel Turull <[email protected]>--- meta/classes/create-spdx-2.2.bbclass | 8 +++++++ meta/lib/oe/spdx30_tasks.py | 8 +++++++ meta/lib/oe/spdx_common.py | 34 ++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+)meta/classes/create-spdx- renamed from meta/classes/create-spdx-2.2.bbclass +0
-02.2.bbclass
meta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir,
Quoted 4 lines
index
7e8f8b9ff5..6bf0c70bd4100644meta/classes/create-spdx-2.2.bbclass +0
-0@@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
get_spdxid,get_types, *, archivmeta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
Quoted 5 lines
meta/classes/create-spdx-2.2.bbclass +2
-0spdx_files = [] file_counter = 1 + + check_kernel_compiled = bb.data.inherits_class("kernel", d) and
d.getVar("CVE_CHECK_KERNEL_CONFIG")== "1"meta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir,
Quoted 7 lines
meta/classes/create-spdx-2.2.bbclass +3
-0+ if check_kernel_compiled: + kernel_sources = + oe.spdx_common.get_kernel_compiled_files(d) for subdir, dirs, files in os.walk(topdir): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):@@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
get_spdxid,get_types, *, archivmeta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
Quoted 7 lines
meta/classes/create-spdx-2.2.bbclass +4
-0filename = str(filepath.relative_to(topdir)) if not filepath.is_symlink() and filepath.is_file(): + # When creating spdx for the kernel, we only include compiled files. + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, + kernel_sources,d):
meta/lib/oe/spdx30_tasks.py def add_package_files( · line 159
Quoted 13 lines
meta/classes/create-spdx-2.2.bbclass +1
-0+ break spdx_file = oe.spdx.SPDXFile() spdx_file.SPDXID = get_spdxid(file_counter) for t in get_types(filepath):(quoted diff) +0
-0
b/meta/lib/oe/spdx30_tasks.pyindexba965821f8..14f26773c5100644meta/lib/oe/spdx30_tasks.py +1
-0@@ -156,6 +156,10 @@ def add_package_files( bb.note(f"Skip {topdir}") return spdx_files + check_kernel_compiled = bb.data.inherits_class("kernel", d) and
d.getVar("CVE_CHECK_KERNEL_CONFIG")== "1"Quoted 47 lines
meta/lib/oe/spdx30_tasks.py +8
-0+ if check_kernel_compiled: + kernel_sources = + oe.spdx_common.get_kernel_compiled_files(d) + for subdir, dirs, files in os.walk(topdir, onerror=walk_error): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):@@ -167,6 +171,10 @@ def add_package_files( filepath = Path(subdir) / file if filepath.is_symlink() or not filepath.is_file(): continue + # When creating spdx for the kernel, we only include compiled files + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): + break filename = str(filepath.relative_to(topdir)) file_purposes = get_purposes(filepath) diff --git
a/meta/lib/oe/spdx_common.pyb/meta/lib/oe/spdx_common.pyindex4caefc7673..c87e3875c7100644meta/lib/oe/spdx_common.py +21
-0@@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name): uri = uri + "@" + fd.revision return uri + +def is_kernel_compiled(filename, kernel_sources, d): + """ + Check if the file, is a kernel compiled file + """ + import os + + _, extension = os.path.splitext(filename) + # Special case, that we need to ignore, since this is not a source file + if filename.rfind(".mod.c") > 0: + return True + # We filter .c files and header files + if extension not in [".c", ".h"]: + return True + # Check that the c file is in the list + if filename in kernel_sources: + return True + return False + +def get_kernel_compiled_files(d): + """meta/lib/oe/spdx_common.py def fetch_data_to_uri(fd, name): · line 275
Quoted 10 lines
meta/lib/oe/spdx_common.py +10
-0+ Get results from the save_compiled files and include also header files + """ + import json + import os + kfiles = [] + with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f: + for item in json.load(f): + kfile = os.path.basename(item['file']) + # Return also the correspondig header file + hfile = ".h".join(kfile.rsplit(".c", 1))This feels a little bit wrong to me. I think a better option would be to include all headers, since we don't have a good way of knowing which are actually used.
Quoted 3 lines
meta/lib/oe/spdx_common.py +3
-0+ kfiles.append(kfile) + kfiles.append(hfile) + return kfiles
RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
In reply to Marko, PeterHi,
I missed to answer your comment.
Daniel
-----Original Message-----
From: Marko, Peter <[email protected]>
Sent: Monday, 28 April 2025 16:00
To: Joshua Watt <[email protected]>; Daniel Turull <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH v2 6/6] spdx: add option to include only compiled kernel files
I think that only kernel provides file-based CVE information right now.
Peter
Quoted 130 lines
-----Original Message-----
From: Joshua Watt <[email protected]>
Sent: Monday, April 28, 2025 15:58
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; Marko, Peter (FT D EU SK BFS1) <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH v2 6/6] spdx: add option to include only compiled kernel filesI see the appeal here; however I've been trying to avoid too much kernel specific stuff in SPDX. Is there a way to make this more generic so it could be used by any recipe?
meta/classes/create-spdx-
Quoted 20 lines · On Mon, Apr 28, 2025 at 7:42 AM <[email protected]> wrote:
From: Daniel Turull <[email protected]>
When
CVE_CHECK_KERNEL_CONFIGis enabled, only include the source code (.c, .h) files that are used during compilation.This enables an external tool to use the SPDX information to disregard vulnerabilities that are not compiled.
CC Joshua Watt <[email protected]>CC Peter Marko <[email protected]>Signed-off-by Daniel Turull <[email protected]>--- meta/classes/create-spdx-2.2.bbclass | 8 +++++++ meta/lib/oe/spdx30_tasks.py | 8 +++++++ meta/lib/oe/spdx_common.py | 34 ++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+)meta/classes/create-spdx- renamed from meta/classes/create-spdx-2.2.bbclass +0
-02.2.bbclass
meta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir,
Quoted 4 lines
index
7e8f8b9ff5..6bf0c70bd4100644meta/classes/create-spdx-2.2.bbclass +0
-0@@ -137,6 +137,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
get_spdxid,get_types, *, archivmeta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
Quoted 5 lines
meta/classes/create-spdx-2.2.bbclass +2
-0spdx_files = [] file_counter = 1 + + check_kernel_compiled = bb.data.inherits_class("kernel", d) and
d.getVar("CVE_CHECK_KERNEL_CONFIG")== "1"meta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir,
Quoted 7 lines
meta/classes/create-spdx-2.2.bbclass +3
-0+ if check_kernel_compiled: + kernel_sources = + oe.spdx_common.get_kernel_compiled_files(d) for subdir, dirs, files in os.walk(topdir): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):@@ -147,6 +151,10 @@ def add_package_files(d, doc, spdx_pkg, topdir,
get_spdxid,get_types, *, archivmeta/classes/create-spdx-2.2.bbclass def add_package_files(d, doc, spdx_pkg, topdir, get_spdxid, get_types, *, archiv
Quoted 7 lines
meta/classes/create-spdx-2.2.bbclass +4
-0filename = str(filepath.relative_to(topdir)) if not filepath.is_symlink() and filepath.is_file(): + # When creating spdx for the kernel, we only include compiled files. + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, + kernel_sources,d):
meta/lib/oe/spdx30_tasks.py def add_package_files( · line 159
Quoted 13 lines
meta/classes/create-spdx-2.2.bbclass +1
-0+ break spdx_file = oe.spdx.SPDXFile() spdx_file.SPDXID = get_spdxid(file_counter) for t in get_types(filepath):(quoted diff) +0
-0
b/meta/lib/oe/spdx30_tasks.pyindexba965821f8..14f26773c5100644meta/lib/oe/spdx30_tasks.py +1
-0@@ -156,6 +156,10 @@ def add_package_files( bb.note(f"Skip {topdir}") return spdx_files + check_kernel_compiled = bb.data.inherits_class("kernel", d) and
d.getVar("CVE_CHECK_KERNEL_CONFIG")== "1"Quoted 53 lines
meta/lib/oe/spdx30_tasks.py +8
-0+ if check_kernel_compiled: + kernel_sources = + oe.spdx_common.get_kernel_compiled_files(d) + for subdir, dirs, files in os.walk(topdir, onerror=walk_error): dirs[:] = [d for d in dirs if d not in ignore_dirs] if subdir == str(topdir):@@ -167,6 +171,10 @@ def add_package_files( filepath = Path(subdir) / file if filepath.is_symlink() or not filepath.is_file(): continue + # When creating spdx for the kernel, we only include compiled files + if check_kernel_compiled: + if not oe.spdx_common.is_kernel_compiled(file, kernel_sources, d): + break filename = str(filepath.relative_to(topdir)) file_purposes = get_purposes(filepath) diff --git
a/meta/lib/oe/spdx_common.pyb/meta/lib/oe/spdx_common.pyindex4caefc7673..c87e3875c7100644meta/lib/oe/spdx_common.py +27
-0@@ -242,3 +242,37 @@ def fetch_data_to_uri(fd, name): uri = uri + "@" + fd.revision return uri + +def is_kernel_compiled(filename, kernel_sources, d): + """ + Check if the file, is a kernel compiled file + """ + import os + + _, extension = os.path.splitext(filename) + # Special case, that we need to ignore, since this is not a source file + if filename.rfind(".mod.c") > 0: + return True + # We filter .c files and header files + if extension not in [".c", ".h"]: + return True + # Check that the c file is in the list + if filename in kernel_sources: + return True + return False + +def get_kernel_compiled_files(d): + """ + Get results from the save_compiled files and include also header files + """ + import json + import os + kfiles = [] + with open(d.getVar('KERNEL_SRC_FILES'), 'r') as f:
Quoted 8 lines
meta/lib/oe/spdx_common.py def fetch_data_to_uri(fd, name): · line 275
Quoted 4 lines
meta/lib/oe/spdx_common.py +4
-0+ for item in json.load(f): + kfile = os.path.basename(item['file']) + # Return also the correspondig header file + hfile = ".h".join(kfile.rsplit(".c", 1))This feels a little bit wrong to me. I think a better option would be to include all headers, since we don't have a good way of knowing which are actually used.
Yes, I had my doubt regarding this. It probably doesn't have an impact in the rest of the functionality for the CVEs I'll keep only the "c" files and try to make it more generic.
Quoted 4 lines
Quoted 4 lines
meta/lib/oe/spdx_common.py +3
-0+ kfiles.append(kfile) + kfiles.append(hfile) + return kfiles