lib: Fix dependencies on SPDX code
PATCH 1/1 lib: Fix dependencies on SPDX code
The SPDX library code was being ignored from taskhash calculations due to accidentally being omitted from BBIMPORTS. This meant that changes in the code or dependent variables would not cause the task to rebuild correctly.
In order to add spdx_common, convert the Dep object from a named tuple to a frozen dataclass. These function more or less equivalently, but the bitbake code parser cannot handle named tuples.
--- meta/lib/oe/__init__.py | 2 +- meta/lib/oe/spdx_common.py | 8 ++++++-- 2 files changed, 7 insertions(+), 3 deletions(-)
meta/lib/oe/__init__.py +1 -1
@@ -11,4 +11,4 @@ __path__ = extend_path(__path__, __name__) # processed correctly (e.g. qa) BBIMPORTS = ["qa", "data", "path", "utils", "types", "package", "packagedata", \ "packagegroup", "sstatesig", "lsb", "cachedpath", "license", \ - "reproducible", "rust", "buildcfg", "go"] + "reproducible", "rust", "buildcfg", "go", "spdx30_tasks", "spdx_common"]
meta/lib/oe/spdx_common.py +6 -2
@@ -12,7 +12,7 @@ import re import shutil from pathlib import Path - +from dataclasses import dataclass LIC_REGEX = re.compile( rb"^\W*SPDX-License-Identifier:\s*([ \w\d.()+-]+?)(?:\s+\W*)?$",
@@ -77,7 +77,11 @@ def process_sources(d): return True -Dep = collections.namedtuple("Dep", ["pn", "hashfn", "in_taskhash"]) +@dataclass(frozen=True) +class Dep(object): + pn: str + hashfn: str + in_taskhash: bool def collect_direct_deps(d, dep_task):
Signature · 1 line
2.47.1
Quoted 11 lines · Joshua Watt via lists.openembedded.org
The SPDX library code was being ignored from taskhash calculations due to accidentally being omitted from BBIMPORTS. This meant that changes in the code or dependent variables would not cause the task to rebuild correctly.
In order to add
spdx_common, convert theDepobject from a named tuple to a frozen dataclass. These function more or less equivalently, but the bitbake code parser cannot handle named tuples.Signed-off-by Joshua Watt <[email protected]>---
Hi Joshua,
I believe this is triggering some issue on the autobuilder:
2025-03-10 09:45:19,401 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/selftest/cases/sstatetests.py", line 482, in test_sstate_allarch_samesigs
self.sstate_common_samesigs(configA, configB, allarch=True)
File "/srv/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/selftest/cases/sstatetests.py", line 213, in sstate_common_samesigs
self.assertEqual(files1, files2)
File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
assertion_func(first, second, msg=msg)
File "/usr/lib/python3.12/unittest/case.py", line 1184, in assertDictEqual
self.fail(self._formatMessage(msg, standardMsg))
File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
raise self.failureException(msg)AssertionError: {'nat[215727 chars]x': 'd8ed538990588f9d37804e867be98c9b4809de8f2[88942 chars]d1a'} != {'nat[215727 chars]x': '481bab8ce0c89a635d3ec3f3c4a28c0b5a67b2c23[88942 chars]d1a'}
https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/1715394/raw_inline
Can you have a look at this error please?
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v2 1/1 lib: Fix dependencies on SPDX code
In reply to Joshua WattThe SPDX library code was being ignored from taskhash calculations due to accidentally being omitted from BBIMPORTS. This meant that changes in the code or dependent variables would not cause the task to rebuild correctly.
In order to add spdx_common, convert the Dep object from a named tuple to a frozen dataclass. These function more or less equivalently, but the bitbake code parser cannot handle named tuples.
Finally, the vardepsexclude that used to be present on the recipe tasks needs to be moved to the python code in order for the variables to be correctly ignored. Several unused exclusions were removed
--- meta/classes/create-spdx-3.0.bbclass | 3 +-- meta/lib/oe/__init__.py | 2 +- meta/lib/oe/spdx_common.py | 8 ++++++-- 3 files changed, 8 insertions(+), 5 deletions(-)
meta/classes/create-spdx-3.0.bbclass +1 -2
@@ -134,7 +134,6 @@ python do_create_spdx() { import oe.spdx30_tasks oe.spdx30_tasks.create_spdx(d) } -do_create_spdx[vardepsexclude] += "BB_NUMBER_THREADS SPDX_BUILD_HOST" do_create_spdx[vardeps] += "\ SPDX_INCLUDE_BITBAKE_PARENT_BUILD \ SPDX_PACKAGE_ADDITIONAL_PURPOSE \
@@ -170,7 +169,7 @@ python do_create_package_spdx() { import oe.spdx30_tasks oe.spdx30_tasks.create_package_spdx(d) } -do_create_package_spdx[vardepsexclude] += "OVERRIDES SPDX_MULTILIB_SSTATE_ARCHS" +oe.spdx30_tasks.create_package_spdx[vardepsexclude] = "OVERRIDES" addtask do_create_package_spdx after do_create_spdx before do_build do_rm_work SSTATETASKS += "do_create_package_spdx"
meta/lib/oe/__init__.py +1 -1
@@ -11,4 +11,4 @@ __path__ = extend_path(__path__, __name__) # processed correctly (e.g. qa) BBIMPORTS = ["qa", "data", "path", "utils", "types", "package", "packagedata", \ "packagegroup", "sstatesig", "lsb", "cachedpath", "license", \ - "reproducible", "rust", "buildcfg", "go"] + "reproducible", "rust", "buildcfg", "go", "spdx30_tasks", "spdx_common"]
meta/lib/oe/spdx_common.py +6 -2
@@ -12,7 +12,7 @@ import re import shutil from pathlib import Path - +from dataclasses import dataclass LIC_REGEX = re.compile( rb"^\W*SPDX-License-Identifier:\s*([ \w\d.()+-]+?)(?:\s+\W*)?$",
@@ -77,7 +77,11 @@ def process_sources(d): return True -Dep = collections.namedtuple("Dep", ["pn", "hashfn", "in_taskhash"]) +@dataclass(frozen=True) +class Dep(object): + pn: str + hashfn: str + in_taskhash: bool def collect_direct_deps(d, dep_task):
Signature · 1 line
2.47.1