SPDX 3.0.1 documentation and bugfix
cover SPDX 3.0.1 documentation and bugfix
From: Stefano Tondo <[email protected]>
This series adds documentation for undocumented SPDX variables and fixes a critical bug in the custom annotations feature.
Patch 1/2: Fixes NameError in SPDX_CUSTOM_ANNOTATION_VARS feature by
correcting method call syntax. Adds regression test to prevent future occurrences.
Patch 2/2: Adds [doc] strings for 8 SPDX variables in spdx-common.bbclass.
Makes variables discoverable via bitbake-getvar and IDE completion.
Note: A companion patch for yocto-docs will add three of these variables to the reference manual (SPDX_UUID_NAMESPACE, SPDX_LICENSES, and SPDX_MULTILIB_SSTATE_ARCHS).
Both patches have been tested with core-image-minimal builds.
Stefano Tondo (2):
spdx30_tasks: Fix SPDX_CUSTOM_ANNOTATION_VARS implementation
spdx-common: Add documentation for undocumented SPDX variables
meta/classes/spdx-common.bbclass | 26 ++++++++++ meta/lib/oe/spdx30_tasks.py | 4 +- meta/lib/oeqa/selftest/cases/spdx.py | 74 ++++++++++++++++++++++++++++ 3 files changed, 101 insertions(+), 3 deletions(-)
Signature · 1 line
2.51.1
PATCH 1/2 spdx30_tasks: Fix SPDX_CUSTOM_ANNOTATION_VARS implementation
From: Stefano Tondo <[email protected]>
Fix incorrect function call when processing SPDX_CUSTOM_ANNOTATION_VARS. The code was calling new_annotation() as a standalone function, but it should be called as a method on the build_objset object.
Error:
new_annotation(d, build_objset, build, ...)
Corrected to:
build_objset.new_annotation(d, build_objset, build, ...)
This bug would cause a NameError at runtime if SPDX_CUSTOM_ANNOTATION_VARS was set to a non-empty value, preventing SPDX document generation.
The fix aligns with how new_annotation() is called elsewhere in the codebase and matches the SBOMObjset class method signature.
--- meta/lib/oe/spdx30_tasks.py | 4 +- meta/lib/oeqa/selftest/cases/spdx.py | 74 ++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+), 3 deletions(-)
meta/lib/oe/spdx30_tasks.py +1 -3
@@ -498,9 +498,7 @@ def create_spdx(d): build_objset.set_is_native(is_native) for var in (d.getVar("SPDX_CUSTOM_ANNOTATION_VARS") or "").split(): - new_annotation( - d, - build_objset, + build_objset.new_annotation( build, "%s=%s" % (var, d.getVar(var)), oe.spdx30.AnnotationType.other,
meta/lib/oeqa/selftest/cases/spdx.py +74 -0
77 lines not on this page · raw message
Signature · 1 line
2.51.1
Re: [OE-core] [OE-core 1/2] spdx30_tasks: Fix SPDX_CUSTOM_ANNOTATION_VARS implementation
Quoted 10 lines · Stefano Tondo via lists.openembedded.org
From: Stefano Tondo <[email protected]>
Fix incorrect function call when processing
SPDX_CUSTOM_ANNOTATION_VARS. The code was callingnew_annotation()as a standalone function, but it should be called as a method on thebuild_objsetobject.Error:
new_annotation(d, build_objset, build, ...)
Quoted 8 lines
Corrected to:
build_objset.new_annotation(d, build_objset, build, ...)This bug would cause a NameError at runtime if
SPDX_CUSTOM_ANNOTATION_VARSwas set to a non-empty value, preventing SPDX document generation.The fix aligns with how
new_annotation()is called elsewhere in the codebase and matches the SBOMObjset class method signature.
LGTM, thanks
Quoted 116 lines
Signed-off-by Stefano Tondo <[email protected]>--- meta/lib/oe/spdx30_tasks.py | 4 +- meta/lib/oeqa/selftest/cases/spdx.py | 74 ++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+), 3 deletions(-)meta/lib/oe/spdx30_tasks.py +1
-3@@ -498,9 +498,7 @@ def create_spdx(d): build_objset.set_is_native(is_native) for var in (d.getVar("SPDX_CUSTOM_ANNOTATION_VARS") or "").split(): - new_annotation( - d, - build_objset, + build_objset.new_annotation( build, "%s=%s" % (var, d.getVar(var)), oe.spdx30.AnnotationType.other,meta/lib/oeqa/selftest/cases/spdx.py +74
-1@@ -286,3 +286,77 @@ class SPDX30Check(SPDX3CheckBase, OESelftestTestCase): break else: self.assertTrue(False, "Unable to find imported Host SpdxID") + + def test_custom_annotation_vars(self): + """ + Test that SPDX_CUSTOM_ANNOTATION_VARS properly creates annotations + without runtime errors. This is a regression test for the bug where + new_annotation() was called as a standalone function instead of as + a method on build_objset, causing a NameError. + + The test verifies: + 1. The build completes successfully (no NameError) + 2. Each configured annotation variable appears exactly once + 3. The annotation values match the configured variables + + We check for exact equality (not >=) to prevent regressions where + one annotation might appear multiple times while another is missing. + """ + ANNOTATION_VAR1 = "TestAnnotation1" + ANNOTATION_VAR2 = "TestAnnotation2" + + # This will fail with NameError if new_annotation() is called incorrectly + objset = self.check_recipe_spdx( + "base-files", + "{DEPLOY_DIR_SPDX}/{MACHINE_ARCH}/packages/package-base-files.spdx.json", + extraconf=textwrap.dedent( + f"""\ + ANNOTATION1 = "{ANNOTATION_VAR1}" + ANNOTATION2 = "{ANNOTATION_VAR2}" + SPDX_CUSTOM_ANNOTATION_VARS = "ANNOTATION1 ANNOTATION2" + """ + ), + ) + + # If we got here, the build succeeded (no NameError) + # Now verify the annotations were actually created + + # Find the build element + build = None + for o in objset.foreach_type(oe.spdx30.build_Build): + build = o + break + + self.assertIsNotNone(build, "Unable to find Build element") + + # Find annotation objects that reference our build + found_annotations = [] + for obj in objset.objects(): + if isinstance(obj, oe.spdx30.Annotation): + if hasattr(obj, "subject") and build._id == obj.subject._id: + found_annotations.append(obj) + + # Check each annotation separately to ensure exactly one occurrence of each + annotation1_count = 0 + annotation2_count = 0 + + for annotation in found_annotations: + if hasattr(annotation, "statement"): + if f"ANNOTATION1={ANNOTATION_VAR1}" in annotation.statement: + annotation1_count += 1 + self.logger.info(f"Found ANNOTATION1: {annotation.statement}") + if f"ANNOTATION2={ANNOTATION_VAR2}" in annotation.statement: + annotation2_count += 1 + self.logger.info(f"Found ANNOTATION2: {annotation.statement}") + + # Each annotation should appear exactly once + self.assertEqual( + annotation1_count, + 1, + f"Expected exactly 1 occurrence of ANNOTATION1, found {annotation1_count}", + ) + self.assertEqual( + annotation2_count, + 1, + f"Expected exactly 1 occurrence of ANNOTATION2, found {annotation2_count}", + ) --2.51.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#226259): https://lists.openembedded.org/g/openembedded-core/message/226259
Mute This Topic: https://lists.openembedded.org/mt/116279440/3616693
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [OE-core 1/2] spdx30_tasks: Fix SPDX_CUSTOM_ANNOTATION_VARS implementation
In reply to Stefano TondoQuoted 12 lines · Stefano Tondo via lists.openembedded.org
From: Stefano Tondo <[email protected]>
Fix incorrect function call when processing
SPDX_CUSTOM_ANNOTATION_VARS. The code was callingnew_annotation()as a standalone function, but it should be called as a method on thebuild_objsetobject.Error:
new_annotation(d, build_objset, build, ...)Corrected to:
build_objset.new_annotation(d, build_objset, build, ...)
Quoted 8 lines
This bug would cause a NameError at runtime if
SPDX_CUSTOM_ANNOTATION_VARSwas set to a non-empty value, preventing SPDX document generation.The fix aligns with how
new_annotation()is called elsewhere in the codebase and matches the SBOMObjset class method signature.Signed-off-by Stefano Tondo <[email protected]>---
Hi Stefano,
Thanks for your patch.
It looks like the new test is failing on the autobuilder:
2025-11-15 11:49:34,729 - oe-selftest - INFO - spdx.SPDX30Check.test_custom_annotation_vars (subunit.RemotedTestCase)
2025-11-15 11:49:34,730 - oe-selftest - INFO - ... FAIL
...
2025-11-15 11:49:34,730 - oe-selftest - INFO - 3: 37/41 297/643 (40.52s) (0 failed) (spdx.SPDX30Check.test_custom_annotation_vars)
2025-11-15 11:49:34,730 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/spdx.py", line 330, in test_custom_annotation_vars
self.assertIsNotNone(build, "Unable to find Build element")
File "/usr/lib/python3.12/unittest/case.py", line 1287, in assertIsNotNone
self.fail(self._formatMessage(msg, standardMsg))
File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
raise self.failureException(msg)AssertionError: unexpectedly None : Unable to find Build element
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2835
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2699
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/2593
Can you have a loot at the issue?
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 2/2 spdx-common: Add documentation for undocumented SPDX variables
In reply to Stefano TondoFrom: Stefano Tondo <[email protected]>
Add missing [doc] strings for seven SPDX-related BitBake variables that were previously undocumented in the spdx-common bbclass.
Variables documented:
- SPDX_INCLUDE_SOURCES: Control source file inclusion in SBOM
- SPDX_INCLUDE_COMPILED_SOURCES: Control compiled source inclusion
- SPDX_UUID_NAMESPACE: Namespace for UUID generation
- SPDX_NAMESPACE_PREFIX: URI prefix for SPDX documents
- SPDX_PRETTY: JSON output formatting control
- SPDX_LICENSES: Path to SPDX license mapping file
- SPDX_CUSTOM_ANNOTATION_VARS: Custom annotation variables
- SPDX_MULTILIB_SSTATE_ARCHS: Multilib sstate architecture list
This improves discoverability of these configuration options and helps users understand how to customize SPDX/SBOM generation.
--- meta/classes/spdx-common.bbclass | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
meta/classes/spdx-common.bbclass +26 -0
@@ -26,17 +26,43 @@ SPDX_TOOL_VERSION ??= "1.0" SPDXRUNTIMEDEPLOY = "${SPDXDIR}/runtime-deploy" SPDX_INCLUDE_SOURCES ??= "0" +SPDX_INCLUDE_SOURCES[doc] = "If set to '1', include source code files in the \ + SPDX output. This will create File objects for all source files used during \ + the build. Note: This significantly increases SBOM size and generation time." + SPDX_INCLUDE_COMPILED_SOURCES ??= "0" +SPDX_INCLUDE_COMPILED_SOURCES[doc] = "If set to '1', include compiled source \ + files (object files, etc.) in the SPDX output. This automatically enables \ + SPDX_INCLUDE_SOURCES. Note: This significantly increases SBOM size." SPDX_UUID_NAMESPACE ??= "sbom.openembedded.org" +SPDX_UUID_NAMESPACE[doc] = "The namespace used for generating UUIDs in SPDX \ + documents. This should be a domain name or unique identifier for your \ + organization to ensure globally unique SPDX IDs." + SPDX_NAMESPACE_PREFIX ??= "http://spdx.org/spdxdocs" +SPDX_NAMESPACE_PREFIX[doc] = "The URI prefix used for SPDX document namespaces. \ + Combined with other identifiers to create unique document URIs." + SPDX_PRETTY ??= "0" +SPDX_PRETTY[doc] = "If set to '1', generate human-readable formatted JSON output \ + with indentation and line breaks. If '0', generate compact JSON output. \ + Pretty formatting makes files larger but easier to read." SPDX_LICENSES ??= "${COREBASE}/meta/files/spdx-licenses.json" +SPDX_LICENSES[doc] = "Path to the JSON file containing SPDX license identifier \ + mappings. This file maps common license names to official SPDX license \ + identifiers." SPDX_CUSTOM_ANNOTATION_VARS ??= "" +SPDX_CUSTOM_ANNOTATION_VARS[doc] = "Space-separated list of variable names whose \ + values will be added as custom annotations to SPDX documents. Each variable's \ + name and value will be recorded as an annotation for traceability." SPDX_MULTILIB_SSTATE_ARCHS ??= "${SSTATE_ARCHS}" +SPDX_MULTILIB_SSTATE_ARCHS[doc] = "The list of sstate architectures to consider \ + when collecting SPDX dependencies. This includes multilib architectures when \ + multilib is enabled. Defaults to SSTATE_ARCHS." python () { from oe.cve_check import extend_cve_status
Signature · 1 line
2.51.1
Re: [OE-core] [OE-core 2/2] spdx-common: Add documentation for undocumented SPDX variables
LGTM, Thanks
Quoted 84 lines · Stefano Tondo via lists.openembedded.org
From: Stefano Tondo <[email protected]>
Add missing [doc] strings for seven SPDX-related BitBake variables that were previously undocumented in the spdx-common bbclass.
Variables documented:
-SPDX_INCLUDE_SOURCES: Control source file inclusion in SBOM
-SPDX_INCLUDE_COMPILED_SOURCES: Control compiled source inclusion
-SPDX_UUID_NAMESPACE: Namespace for UUID generation
-SPDX_NAMESPACE_PREFIX: URI prefix for SPDX documents
-SPDX_PRETTY: JSON output formatting control
-SPDX_LICENSES: Path to SPDX license mapping file
-SPDX_CUSTOM_ANNOTATION_VARS: Custom annotation variables
-SPDX_MULTILIB_SSTATE_ARCHS: Multilib sstate architecture listThis improves discoverability of these configuration options and helps users understand how to customize SPDX/SBOM generation.
Signed-off-by Stefano Tondo <[email protected]>--- meta/classes/spdx-common.bbclass | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)meta/classes/spdx-common.bbclass +26
-1@@ -26,17 +26,43 @@ SPDX_TOOL_VERSION ??= "1.0" SPDXRUNTIMEDEPLOY = "${SPDXDIR}/runtime-deploy" SPDX_INCLUDE_SOURCES ??= "0" +SPDX_INCLUDE_SOURCES[doc] = "If set to '1', include source code files in the \ + SPDX output. This will create File objects for all source files used during \ + the build. Note: This significantly increases SBOM size and generation time." + SPDX_INCLUDE_COMPILED_SOURCES ??= "0" +SPDX_INCLUDE_COMPILED_SOURCES[doc] = "If set to '1', include compiled source \ + files (object files, etc.) in the SPDX output. This automatically enables \ + SPDX_INCLUDE_SOURCES. Note: This significantly increases SBOM size." SPDX_UUID_NAMESPACE ??= "sbom.openembedded.org" +SPDX_UUID_NAMESPACE[doc] = "The namespace used for generating UUIDs in SPDX \ + documents. This should be a domain name or unique identifier for your \ + organization to ensure globally unique SPDX IDs." + SPDX_NAMESPACE_PREFIX ??= "http://spdx.org/spdxdocs" +SPDX_NAMESPACE_PREFIX[doc] = "The URI prefix used for SPDX document namespaces. \ + Combined with other identifiers to create unique document URIs." + SPDX_PRETTY ??= "0" +SPDX_PRETTY[doc] = "If set to '1', generate human-readable formatted JSON output \ + with indentation and line breaks. If '0', generate compact JSON output. \ + Pretty formatting makes files larger but easier to read." SPDX_LICENSES ??= "${COREBASE}/meta/files/spdx-licenses.json" +SPDX_LICENSES[doc] = "Path to the JSON file containing SPDX license identifier \ + mappings. This file maps common license names to official SPDX license \ + identifiers." SPDX_CUSTOM_ANNOTATION_VARS ??= "" +SPDX_CUSTOM_ANNOTATION_VARS[doc] = "Space-separated list of variable names whose \ + values will be added as custom annotations to SPDX documents. Each variable's \ + name and value will be recorded as an annotation for traceability." SPDX_MULTILIB_SSTATE_ARCHS ??= "${SSTATE_ARCHS}" +SPDX_MULTILIB_SSTATE_ARCHS[doc] = "The list of sstate architectures to consider \ + when collecting SPDX dependencies. This includes multilib architectures when \ + multilib is enabled. Defaults to SSTATE_ARCHS." python () { from oe.cve_check import extend_cve_status --2.51.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#226260): https://lists.openembedded.org/g/openembedded-core/message/226260
Mute This Topic: https://lists.openembedded.org/mt/116279441/3616693
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-