SDKs w/ multilibs and spdx30 'isfile'
cover SDKs w/ multilibs and spdx30 'isfile'
From: Mark Hatle <[email protected]>
This resolves problems when building an SDK that includes multilibs. It also adds the isfile change that was previou put in for spdx 2.2.
Introduce a new multilib SSTATE_ARCHs, as an optimization this just defaults to SSTATE_ARCHs in most cases. If building an SDK we do expand it with the multilibs.
Additionally fixes the issue where were a debug component may refer to something that is not a file. This change was already put into the spdx 2.2 class. See:
commit a798d00d54a1424f9972de43ff05a0ca8950d7de
Author: Mark Hatle <[email protected]> Date: Mon Jul 15 14:56:06 2024 -0500
create-sdpx-2.2.bbclass: Switch from exists to isfile checking debugsrc
While debugsrc is almost always a file (or link), there are apparently cases where a directory could be returned from the dwarfsrcfiles processing. When this happens, the hashing fails and an error results when building the SPDX documents.
(From OE-Core rev: 02e262c291c0b2066132b4cb2ca5fda8145284a9)
Mark Hatle (2):
spdx30_tasks.py: switch from exists to isfile checking debugsrc create-spdx-*: Support multilibs via SPDX_MULTILIB_SSTATE_ARCHS
meta/classes-recipe/populate_sdk_base.bbclass | 4 ++++ meta/classes/create-spdx-2.2.bbclass | 12 ++++++------ meta/classes/create-spdx-3.0.bbclass | 4 ++-- meta/classes/spdx-common.bbclass | 2 ++ meta/lib/oe/sbom30.py | 2 +- meta/lib/oe/spdx30_tasks.py | 3 ++- 6 files changed, 17 insertions(+), 10 deletions(-)
Signature · 1 line
2.34.1
PATCH 1/2 spdx30_tasks.py: switch from exists to isfile checking debugsrc
From: Mark Hatle <[email protected]>
Same change as previously made to the create-spdx-2.2.bbclass, while debugsrc is almost always a file (or link), there are apparently cases where a directory could be returned from the dwarfsrcfiles processing. When this happens, the hashing fails and an error results when building the SPDX documents.
--- meta/lib/oe/spdx30_tasks.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
meta/lib/oe/spdx30_tasks.py +2 -1
@@ -238,7 +238,8 @@ def get_package_sources_from_debug( if file_sha256 is None: continue else: - if not debugsrc_path.exists(): + # We can only hash files below, skip directories, links, etc. + if not debugsrc_path.isfile(): source_hash_cache[debugsrc_path] = None continue
Signature · 1 line
2.34.1
PATCH 2/2 create-spdx-*: Support multilibs via SPDX_MULTILIB_SSTATE_ARCHS
In reply to Mark HatleFrom: Mark Hatle <[email protected]>
When a create-spdx-* classes is processing documents, it needs to find the document in a path that is related to the SSTATE_ARCH when a packge is generated. The SSTATE_ARCH can be affected by multilib configurations, resulting is something like armv8a-mlib.
When the image (or SDK) is being generated and the components are collected, the system has no knowledge of the multilib arch and will fail to find it, such as:
ERROR: meta-toolchain-1.0-r0 do_populate_sdk: No SPDX file found
for package libilp32-libgcc-dbg,
False sstate:libilp32-libgcc:armv8a-ilp32-mllibilp32-elf:14.1.0:r0:armv8a-ilp32:12:
sstate:libilp32-libgcc::14.1.0:r0::12:
Adding in the new SPDX_MULTILIB_SSTATE_ARCHS will provide a full set of SSTATE_ARCHS including ones that contain the multilib extension which will allow create-spdx-* to correctly find the document it is looking for. This would also be valuable to any other function doing a similar search through SSTATE_ARCH that may have been extended with multilib configurations.
--- meta/classes-recipe/populate_sdk_base.bbclass | 4 ++++ meta/classes/create-spdx-2.2.bbclass | 12 ++++++------ meta/classes/create-spdx-3.0.bbclass | 4 ++-- meta/classes/spdx-common.bbclass | 2 ++ meta/lib/oe/sbom30.py | 2 +- 5 files changed, 15 insertions(+), 9 deletions(-)
meta/classes-recipe/populate_sdk_base.bbclass +4 -0
@@ -6,6 +6,10 @@ PACKAGES = "" +# This exists as an optimization for SPDX processing to only run in image and +# SDK processing context. This class happens to be common to these usages. +SPDX_MULTILIB_SSTATE_ARCHS = "${@all_multilib_tune_values(d, 'SSTATE_ARCHS')}" + inherit image-postinst-intercepts image-artifact-names # Wildcards specifying complementary packages to install for every package that has been explicitly
meta/classes/create-spdx-2.2.bbclass +6 -6
@@ -267,7 +267,7 @@ def collect_dep_recipes(d, doc, spdx_recipe): import oe.spdx_common deploy_dir_spdx = Path(d.getVar("DEPLOY_DIR_SPDX")) - package_archs = d.getVar("SSTATE_ARCHS").split() + package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() dep_recipes = []
@@ -312,7 +312,7 @@ def collect_dep_recipes(d, doc, spdx_recipe): return dep_recipes -collect_dep_recipes[vardepsexclude] = "SSTATE_ARCHS" +collect_dep_recipes[vardepsexclude] = "SPDX_MULTILIB_SSTATE_ARCHS" def collect_dep_sources(d, dep_recipes): import oe.sbom
@@ -610,7 +610,7 @@ python do_create_runtime_spdx() { providers = oe.spdx_common.collect_package_providers(d) pkg_arch = d.getVar("SSTATE_PKGARCH") - package_archs = d.getVar("SSTATE_ARCHS").split() + package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() if not is_native:
@@ -716,7 +716,7 @@ python do_create_runtime_spdx() { oe.sbom.write_doc(d, runtime_doc, pkg_arch, "runtime", spdx_deploy, indent=get_json_indent(d)) } -do_create_runtime_spdx[vardepsexclude] += "OVERRIDES SSTATE_ARCHS" +do_create_runtime_spdx[vardepsexclude] += "OVERRIDES SPDX_MULTILIB_SSTATE_ARCHS" addtask do_create_runtime_spdx after do_create_spdx before do_build do_rm_work SSTATETASKS += "do_create_runtime_spdx"
@@ -798,7 +798,7 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx import bb.compress.zstd providers = oe.spdx_common.collect_package_providers(d) - package_archs = d.getVar("SSTATE_ARCHS").split() + package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() creation_time = datetime.now(tz=timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
@@ -950,4 +950,4 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx tar.addfile(info, fileobj=index_str) -combine_spdx[vardepsexclude] += "BB_NUMBER_THREADS SSTATE_ARCHS" +combine_spdx[vardepsexclude] += "BB_NUMBER_THREADS SPDX_MULTILIB_SSTATE_ARCHS"
meta/classes/create-spdx-3.0.bbclass +2 -2
@@ -118,7 +118,7 @@ IMAGE_CLASSES:append = " create-spdx-image-3.0" oe.spdx30_tasks.set_timestamp_now[vardepsexclude] = "SPDX_INCLUDE_TIMESTAMPS" oe.spdx30_tasks.get_package_sources_from_debug[vardepsexclude] += "STAGING_KERNEL_DIR" -oe.spdx30_tasks.collect_dep_objsets[vardepsexclude] = "SSTATE_ARCHS" +oe.spdx30_tasks.collect_dep_objsets[vardepsexclude] = "SPDX_MULTILIB_SSTATE_ARCHS"
@@ -164,7 +164,7 @@ python do_create_package_spdx() { import oe.spdx30_tasks oe.spdx30_tasks.create_package_spdx(d) } -do_create_package_spdx[vardepsexclude] += "OVERRIDES SSTATE_ARCHS" +do_create_package_spdx[vardepsexclude] += "OVERRIDES SPDX_MULTILIB_SSTATE_ARCHS" addtask do_create_package_spdx after do_create_spdx before do_build do_rm_work SSTATETASKS += "do_create_package_spdx"
meta/classes/spdx-common.bbclass +2 -0
@@ -37,6 +37,8 @@ SPDX_LICENSES ??= "${COREBASE}/meta/files/spdx-licenses.json" SPDX_CUSTOM_ANNOTATION_VARS ??= "" +SPDX_MULTILIB_SSTATE_ARCHS ??= "${SSTATE_ARCHS}" + python() { import oe.spdx_common oe.spdx_common.load_spdx_license_data(d)
meta/lib/oe/sbom30.py +1 -1
@@ -921,7 +921,7 @@ def load_jsonld_by_arch(d, arch, subdir, name, *, required=False): def find_jsonld(d, subdir, name, *, required=False): - package_archs = d.getVar("SSTATE_ARCHS").split() + package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() for arch in package_archs:
Signature · 1 line
2.34.1