Skip to contentSkip to conversation

Various SPDX Fixes

PATCH New Joshua Watt 1 participant 6 messages Dec 3, 2024

lore
Share & tools
lore
#0 lore raw

cover Various SPDX Fixes

In the course of making training material for SPDX, I discovered that several of SPDX options to add additional information about the build were broken. Fix these, and add a selftest to make sure that they continue working in the future.

Joshua Watt (5):
lib: sbom30: Fix import handling
lib: sbom30: Fix agent reference
classes: create-spdx: Fix variable dependencies
lib: spdx: Fix SPDX_BUILD_HOST
lib: oeqa: spdx: Add tests for extra options

 .../create-spdx-image-3.0.bbclass             |   3 +
 meta/classes/create-spdx-3.0.bbclass          |  10 +-
 meta/lib/oe/sbom30.py                         |  17 +--
 meta/lib/oe/spdx30_tasks.py                   |   2 +-
 meta/lib/oeqa/selftest/cases/spdx.py          | 118 +++++++++++++++---
 5 files changed, 123 insertions(+), 27 deletions(-)
Signature · 1 line
2.47.1
#1 lore raw

PATCH 1/5 lib: sbom30: Fix import handling

Fixes a bug in the way that imports where being handled, due to a misspelled variable

---
 meta/lib/oe/sbom30.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oe/sbom30.py +1 -1
index 0a7b4c05fbe..ea70f6fb69d 100644
@@ -364,7 +364,7 @@ class ObjectSet(oe.spdx30.SHACLObjectSet):
         if not spdxid:
             bb.fatal(f"{key} is not a valid SPDX_IMPORTS key")
 
-        for i in self.docs.import_:
+        for i in self.doc.import_:
             if i.externalSpdxId == spdxid:
                 # Already imported
                 return spdxid
Signature · 1 line
2.47.1
#2 lore raw

PATCH 2/5 lib: sbom30: Fix agent reference

In reply to Joshua Watt

When a agent reference was being used, the code was not using the correct base variable to look up the agent

---
 meta/lib/oe/sbom30.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oe/sbom30.py +1 -1
index ea70f6fb69d..7d01715ce24 100644
@@ -393,7 +393,7 @@ class ObjectSet(oe.spdx30.SHACLObjectSet):
         if ref_varname:
             if ref_varname == varname:
                 bb.fatal(f"{varname} cannot reference itself")
-            return new_agent(varname, creation_info=creation_info)
+            return self.new_agent(ref_varname, creation_info=creation_info)
 
         import_key = self.d.getVar(f"{varname}_import")
         if import_key:
Signature · 1 line
2.47.1
#3 lore raw

PATCH 3/5 classes: create-spdx: Fix variable dependencies

In reply to Joshua Watt

A few variable dependencies were incorrect from the SPDX tasks, which prevented rebuilds from occurring, or caused them to occur when they should not.

---
 meta/classes-recipe/create-spdx-image-3.0.bbclass |  3 +++
 meta/classes/create-spdx-3.0.bbclass              | 10 +++++++++-
 2 files changed, 12 insertions(+), 1 deletion(-)
meta/classes-recipe/create-spdx-image-3.0.bbclass +3 -0
index 51446162857..e0f1766bb76 100644
@@ -55,6 +55,9 @@ do_create_image_spdx[sstate-outputdirs] = "${DEPLOY_DIR_SPDX}"
 do_create_image_spdx[cleandirs] = "${SPDXIMAGEWORK}"
 do_create_image_spdx[dirs] = "${SPDXIMAGEWORK}"
 do_create_image_spdx[file-checksums] += "${SPDX3_LIB_DEP_FILES}"
+do_create_image_spdx[vardeps] += "\
+    SPDX_IMAGE_PURPOSE \
+    "
 
 python do_create_image_spdx_setscene() {
     sstate_setscene(d)
meta/classes/create-spdx-3.0.bbclass +9 -1
index 640f5490bda..25f3aa5f433 100644
@@ -134,7 +134,15 @@ python do_create_spdx() {
     import oe.spdx30_tasks
     oe.spdx30_tasks.create_spdx(d)
 }
-do_create_spdx[vardepsexclude] += "BB_NUMBER_THREADS"
+do_create_spdx[vardepsexclude] += "BB_NUMBER_THREADS SPDX_BUILD_HOST"
+do_create_spdx[vardeps] += "\
+    SPDX_INCLUDE_BITBAKE_PARENT_BUILD \
+    SPDX_PACKAGE_ADDITIONAL_PURPOSE \
+    SPDX_PROFILES \
+    SPDX_NAMESPACE_PREFIX \
+    SPDX_UUID_NAMESPACE \
+    "
+
 addtask do_create_spdx after \
     do_collect_spdx_deps \
     do_deploy_source_date_epoch \
Signature · 1 line
2.47.1
#4 lore raw

PATCH 4/5 lib: spdx: Fix SPDX_BUILD_HOST

In reply to Joshua Watt

The call to make a new import for the SPDX_BUILD_HOST was incorrect, since it was asking for the key named "SPDX_BUILD_HOST", instead of the key name contained in that variable

---
 meta/lib/oe/spdx30_tasks.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oe/spdx30_tasks.py +1 -1
index 30f1a3a1661..3d7035909f0 100644
@@ -908,7 +908,7 @@ def write_bitbake_spdx(d):
                 [build],
                 oe.spdx30.RelationshipType.hasHost,
                 oe.spdx30.LifecycleScopeType.build,
-                [objset.new_import("SPDX_BUILD_HOST")],
+                [objset.new_import(host_import_key)],
             )
 
         if invoked_by:
Signature · 1 line
2.47.1
#5 lore raw

PATCH 5/5 lib: oeqa: spdx: Add tests for extra options

In reply to Joshua Watt

Adds a test for several of the extra options provided by the SPDX classes. In particular, these are the options that can produce non-reproducible results, so are not enabled by default in OE core. This test takes care to configure the build so that the tests do run in a reproducible manner so that pre-built test objects can be pulled from sstate

---
 meta/lib/oe/sbom30.py                |  13 +--
 meta/lib/oeqa/selftest/cases/spdx.py | 118 +++++++++++++++++++++++----
 2 files changed, 108 insertions(+), 23 deletions(-)
meta/lib/oe/sbom30.py +4 -9
index 7d01715ce24..9a3b188dbbb 100644
@@ -875,16 +875,11 @@ class ObjectSet(oe.spdx30.SHACLObjectSet):
             else:
                 missing_spdxids.add(spdxid)
 
-        bb.debug(1, "Linking...")
-        missing = self.link()
-        if missing != missing_spdxids:
-            bb.fatal(
-                f"Linked document doesn't match missing SPDX ID list. Got: {missing}\nExpected: {missing_spdxids}"
-            )
-
         self.doc.import_ = sorted(imports.values(), key=lambda e: e.externalSpdxId)
-
-        return missing_spdxids
+        bb.debug(1, "Linking...")
+        self.link()
+        self.missing_ids -= set(imports.keys())
+        return self.missing_ids
 
 
 def load_jsonld(d, path, required=False):
meta/lib/oeqa/selftest/cases/spdx.py +104 -14

157 lines not on this page · raw message

Signature · 1 line
2.47.1