Slight code cleanup - remove dead code, fix typos, unify patterns
v2 cover Slight code cleanup - remove dead code, fix typos, unify patterns
---
Changes in v2:
- add commit message bodies following patchtest failures
- Link to v1: https://lore.kernel.org/r/[email protected]
---
Adam Blank (4):
lib/packagedata.py: slight improvement to code readability
package_pkgdata: fix typo to stop calling undefined function
sstate: remove dead code and unify path operations
package: update the comment block explaining 'emit_pkgdata'
meta/classes-global/package.bbclass | 3 +-- meta/classes-global/package_pkgdata.bbclass | 4 +--- meta/classes-global/sstate.bbclass | 5 ++--- meta/lib/oe/packagedata.py | 3 +-- 4 files changed, 5 insertions(+), 10 deletions(-)
---
base-commit: c56990178b31b893fbf695eaf6b67de501e9d2e9
change-id: 20260402-dead_code_and_unification-d2b31d998515
Best regards,
Signature · 1 line
Adam Blank <[email protected]>
PATCH v2 1/4 lib/packagedata.py: slight improvement to code readability
Make use of an existing variable rather than creating a new one when collecting files of a package.
--- meta/lib/oe/packagedata.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
meta/lib/oe/packagedata.py +1 -2
@@ -340,8 +340,7 @@ fi allow_empty = d.getVar('ALLOW_EMPTY:%s' % pkg) if not allow_empty: allow_empty = d.getVar('ALLOW_EMPTY') - root = "%s/%s" % (pkgdest, pkg) - os.chdir(root) + os.chdir(pkgdestpkg) g = glob('*') if g or allow_empty == "1": # Symlinks needed for reverse lookups (from the final package name)
Signature · 1 line
2.43.0
PATCH v2 2/4 package_pkgdata: fix typo to stop calling undefined function
In reply to Adam BlankThe function is named 'package_populate_pkgdata_dir' but the call was to 'staging_package_populate_pkgdata_dir'.
--- meta/classes-global/package_pkgdata.bbclass | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
meta/classes-global/package_pkgdata.bbclass +1 -3
@@ -47,7 +47,7 @@ python package_prepare_pkgdata() { # Detect bitbake -b usage nodeps = d.getVar("BB_LIMITEDDEPS") or False if nodeps: - staging_package_populate_pkgdata_dir(pkgdatadir, d) + package_populate_pkgdata_dir(pkgdatadir, d) return start = None
@@ -169,5 +169,3 @@ python package_prepare_pkgdata() { } package_prepare_pkgdata[cleandirs] = "${WORKDIR_PKGDATA}" package_prepare_pkgdata[vardepsexclude] += "MACHINE_ARCH PACKAGE_EXTRA_ARCHS SDK_ARCH BUILD_ARCH SDK_OS BB_TASKDEPDATA SSTATETASKS" - -
Signature · 1 line
2.43.0
PATCH v2 3/4 sstate: remove dead code and unify path operations
In reply to Adam BlankMost substring replacement operations performed on 'dirs' and 'plaindirs' are implemented in the same pattern, except two. Unify the implementation.
--- meta/classes-global/sstate.bbclass | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
meta/classes-global/sstate.bbclass +2 -3
@@ -333,9 +333,9 @@ def sstate_install(ss, d): for plain in ss['plaindirs']: workdir = d.getVar('WORKDIR') sharedworkdir = os.path.join(d.getVar('TMPDIR'), "work-shared") - src = sstateinst + "/" + plain.replace(workdir, '') + src = plain.replace(workdir, sstateinst) if sharedworkdir in plain: - src = sstateinst + "/" + plain.replace(sharedworkdir, '') + src = plain.replace(sharedworkdir, sstateinst) dest = plain bb.utils.mkdirhier(src) prepdir(dest)
@@ -639,7 +639,6 @@ def sstate_package(ss, d): for state in ss['dirs']: if not os.path.exists(state[1]): continue - srcbase = state[0].rstrip("/").rsplit('/', 1)[0] # Find and error for absolute symlinks. We could attempt to relocate but its not # clear where the symlink is relative to in this context. We could add that markup # to sstate tasks but there aren't many of these so better just avoid them entirely.
Signature · 1 line
2.43.0
PATCH v2 4/4 package: update the comment block explaining 'emit_pkgdata'
In reply to Adam BlankThe comment block mentioned PKGDATA_DIR which is now used differently.
--- meta/classes-global/package.bbclass | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
meta/classes-global/package.bbclass +1 -2
@@ -41,7 +41,7 @@ # # k) package_depchains - Adds automatic dependencies to -dbg and -dev packages # -# l) emit_pkgdata - saves the packaging data into PKGDATA_DIR for use in later +# l) emit_pkgdata - saves the packaging data into PKGDESTWORK for use in later # packaging steps inherit packagedata
@@ -610,4 +610,3 @@ python do_packagedata_setscene () { sstate_setscene(d) } addtask do_packagedata_setscene -
Signature · 1 line
2.43.0
Re: [OE-core] [PATCH v2 2/4] package_pkgdata: fix typo to stop calling undefined function
In reply to Adam BlankQuoted 5 lines · Adam Blank via lists.openembedded.org
The function is named '
package_populate_pkgdata_dir' but the call was to 'staging_package_populate_pkgdata_dir'.Signed-off-by Adam Blank <[email protected]>---
Hi Adam,
Thanks for your patch.
I can't say if this is another issue or something wrong with the tests, but two of these are failing because of this patch:
2026-04-02 19:56:53,924 - oe-selftest - INFO - sstatetests.SStateHashSameSigs2.test_sstate_allarch_samesigs (subunit.RemotedTestCase)
2026-04-02 19:56:53,926 - oe-selftest - INFO - ... FAIL
...
2026-04-02 19:56:53,926 - oe-selftest - INFO - 13: 17/52 271/679 (95.16s) (0 failed) (sstatetests.SStateHashSameSigs2.test_sstate_allarch_samesigs)
2026-04-02 19:56:53,926 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/sstatetests.py", line 469, in test_sstate_allarch_samesigs
self.sstate_common_samesigs(configA, configB, allarch=True)
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/sstatetests.py", line 206, in sstate_common_samesigs
self.assertEqual(files1, files2)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/unittest/case.py", line 907, in assertEqual
assertion_func(first, second, msg=msg)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/unittest/case.py", line 1206, in assertDictEqual
self.fail(self._formatMessage(msg, standardMsg))
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/unittest/case.py", line 732, in fail
raise self.failureException(msg)AssertionError: {'nat[31 chars]ckage_qa': '1cf298a771718a59519250687e84c92de5[302150 chars]adb'} != {'nat[31 chars]ckagedata': 'e9b33afd6d363736c82af96ef0a991046[302150 chars]bb4'}
{'adwaita-icon-theme/do_compile': '18aab6f6fe8018875194a0636a0f373a55f73fb4b8129d77a2e1d086d2ec578e',
'adwaita-icon-theme/do_configure': '7380b23df0d0ef5ef3abc408bf06c4b25284a707bbb39391092412c9e5fdd0f0',
- 'adwaita-icon-theme/do_create_package_spdx': '71291be90a9aae925880a5fb43a06ac5e77b516d30297dc2a512939386160379',
+ 'adwaita-icon-theme/do_create_package_spdx': 'e79bbe627c2567a418bd672bef357e0e2561f24100be568749bd5e0618503328', 'adwaita-icon-theme/do_create_recipe_spdx': '4a4714266c3fb489eb4e563917ccccb38cc0b9f66594a6107f3a502036a3c5b3',
- 'adwaita-icon-theme/do_create_spdx': 'd4a4708d298a5dd62e647fe7e886d5c673e9c3aac4365c743360dbcafb18be38',
+ 'adwaita-icon-theme/do_create_spdx': '98cd801f20f0ffb0e9bdffd12ee44f748e4d292e828f1cd37b7fdb1d24f195d6', 'adwaita-icon-theme/do_deploy_source_date_epoch': '9f88202fbc0193ad762daea7e7b614cbed0c61ff6a60a2dfba4bbd770e21f91f', 'adwaita-icon-theme/do_fetch': '2ddf4b31dca442e88c40cbadb087f7f14aa5e7903ddf25a9eaa010a784df31e0',
'adwaita-icon-theme/do_install': '997dd989d8d5379abaeaea11cb76db90603e4fd8e5b15eaa7d3bed5efc6c2f28',
- 'adwaita-icon-theme/do_package': 'ae407bcb5d420c505b3de2dd5d67eb6c9d81b31da8008393c72bda7d1f7d4e3e', - 'adwaita-icon-theme/do_package_qa': '0f93d17760ba7d350147722bf7a49a88580f3d2d511acace784be8737ab28624', - 'adwaita-icon-theme/do_package_write_rpm': 'd672adc3f3a53d3392340974f4a7edc92bd455a6b5782a527a0f22f76ffc18d3', - 'adwaita-icon-theme/do_packagedata': '252ddb5c78401938a7e0cc11803932ec5d5c55083a7bdcdee4e23c1f992c0f2f', + 'adwaita-icon-theme/do_package': 'fcae5f450961cf5319679099fa4948312f289150a73b75f4e6271aa41eaf35f4', + 'adwaita-icon-theme/do_package_qa': '4d363ce538fd446486e157c8c4e7c304e8bb39fcddc70effc8779dee824984e8', + 'adwaita-icon-theme/do_package_write_rpm': 'b9b134d7912f00b50297f2d18d64ff3589054c48574655b5f6749d9517451651', + 'adwaita-icon-theme/do_packagedata': '25015261e21fd33d2c29b66af7158a8818612793eb676574355245c359bfbb99', 'adwaita-icon-theme/do_patch': 'fbb075a802b4418e22df82386efbb58dd9bcb6902a5a2599d7b57921ba142e29', ... 2026-04-02 19:59:11,956 - oe-selftest - INFO - sstatetests.SStateHashSameSigs2.test_sstate_nativesdk_samesigs_multilib (subunit.RemotedTestCase) 2026-04-02 19:59:11,957 - oe-selftest - INFO - ... FAIL
...
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3580
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/3678
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/3468
Can you have a look at the issue?
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Re: [OE-core] [PATCH v2 2/4] package_pkgdata: fix typo to stop calling undefined function
Hmmm, can't see any connection between those tests and this particular patch...
The thing is, that since the patch fixes a call to an undefined function, there could not have been a test covering this case in the first place ;-) In addition, the function call takes place only in the 'bitbake -b' case...
Where can it be found, which other changes took place in those builds?
Quoted 107 lines · Mathieu Dubois-Briand
Quoted 5 lines · Adam Blank via lists.openembedded.org
The function is named '
package_populate_pkgdata_dir' but the call was to 'staging_package_populate_pkgdata_dir'.Signed-off-by Adam Blank <[email protected]>---
Hi Adam,
Thanks for your patch.
I can't say if this is another issue or something wrong with the tests, but two of these are failing because of this patch:
2026-04-02 19:56:53,924 - oe-selftest - INFO - sstatetests.SStateHashSameSigs2.test_sstate_allarch_samesigs (subunit.RemotedTestCase) 2026-04-02 19:56:53,926 - oe-selftest - INFO - ... FAIL ... 2026-04-02 19:56:53,926 - oe-selftest - INFO - 13: 17/52 271/679 (95.16s) (0 failed) (sstatetests.SStateHashSameSigs2.test_sstate_allarch_samesigs) 2026-04-02 19:56:53,926 - oe-selftest - INFO -testtools.testresult.real._StringException: Traceback (most recent call
last):
File
"/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/sstatetests.py", line 469, intest_sstate_allarch_samesigs
self.sstate_common_samesigs(configA, configB, allarch=True)~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/sstatetests.py", line 206, insstate_common_samesigsself.assertEqual(files1, files2) ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^File "
/usr/lib/python3.13/unittest/case.py", line 907, in assertEqual
assertion_func(first, second, msg=msg)~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/unittest/case.py", line 1206, in
assertDictEqualself.fail(self._formatMessage(msg, standardMsg)) ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "
/usr/lib/python3.13/unittest/case.py", line 732, in fail
raiseself.failureException(msg)
AssertionError: {'nat[31 chars]ckage_qa':
'1cf298a771718a59519250687e84c92de5[302150 chars]adb'} != {'nat[31 chars]ckagedata': 'e9b33afd6d363736c82af96ef0a991046[302150 chars]bb4'}
{'adwaita-icon-theme/do_compile':
'18aab6f6fe8018875194a0636a0f373a55f73fb4b8129d77a2e1d086d2ec578e',
'adwaita-icon-theme/do_configure':
'7380b23df0d0ef5ef3abc408bf06c4b25284a707bbb39391092412c9e5fdd0f0',
- 'adwaita-icon-theme/do_create_package_spdx':
'71291be90a9aae925880a5fb43a06ac5e77b516d30297dc2a512939386160379',
+ 'adwaita-icon-theme/do_create_package_spdx':
'e79bbe627c2567a418bd672bef357e0e2561f24100be568749bd5e0618503328',
'adwaita-icon-theme/do_create_recipe_spdx':
'4a4714266c3fb489eb4e563917ccccb38cc0b9f66594a6107f3a502036a3c5b3',
- 'adwaita-icon-theme/do_create_spdx':
'd4a4708d298a5dd62e647fe7e886d5c673e9c3aac4365c743360dbcafb18be38',
+ 'adwaita-icon-theme/do_create_spdx':
'98cd801f20f0ffb0e9bdffd12ee44f748e4d292e828f1cd37b7fdb1d24f195d6',
'adwaita-icon-theme/do_deploy_source_date_epoch':
'9f88202fbc0193ad762daea7e7b614cbed0c61ff6a60a2dfba4bbd770e21f91f',
'adwaita-icon-theme/do_fetch':
'2ddf4b31dca442e88c40cbadb087f7f14aa5e7903ddf25a9eaa010a784df31e0',
'adwaita-icon-theme/do_install':
'997dd989d8d5379abaeaea11cb76db90603e4fd8e5b15eaa7d3bed5efc6c2f28',
- 'adwaita-icon-theme/do_package':
'ae407bcb5d420c505b3de2dd5d67eb6c9d81b31da8008393c72bda7d1f7d4e3e',
- 'adwaita-icon-theme/do_package_qa':
'0f93d17760ba7d350147722bf7a49a88580f3d2d511acace784be8737ab28624',
- 'adwaita-icon-theme/do_package_write_rpm':
'd672adc3f3a53d3392340974f4a7edc92bd455a6b5782a527a0f22f76ffc18d3',
- 'adwaita-icon-theme/do_packagedata':
'252ddb5c78401938a7e0cc11803932ec5d5c55083a7bdcdee4e23c1f992c0f2f',
+ 'adwaita-icon-theme/do_package':
'fcae5f450961cf5319679099fa4948312f289150a73b75f4e6271aa41eaf35f4',
+ 'adwaita-icon-theme/do_package_qa':
'4d363ce538fd446486e157c8c4e7c304e8bb39fcddc70effc8779dee824984e8',
+ 'adwaita-icon-theme/do_package_write_rpm':
'b9b134d7912f00b50297f2d18d64ff3589054c48574655b5f6749d9517451651',
+ 'adwaita-icon-theme/do_packagedata':
'25015261e21fd33d2c29b66af7158a8818612793eb676574355245c359bfbb99',
'adwaita-icon-theme/do_patch':
'fbb075a802b4418e22df82386efbb58dd9bcb6902a5a2599d7b57921ba142e29',
...2026-04-02 19:59:11,956 - oe-selftest - INFO - sstatetests.SStateHashSameSigs2.test_sstate_nativesdk_samesigs_multilib (subunit.RemotedTestCase) 2026-04-02 19:59:11,957 - oe-selftest - INFO - ... FAIL...
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/3580
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/3678
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/3468Can you have a look at the issue?
Thanks,
Mathieu--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
Re: [OE-core] [PATCH v2 2/4] package_pkgdata: fix typo to stop calling undefined function
Quoted 11 lines · Adam Blank
Hmmm, can't see any connection between those tests and this particular patch...
The thing is, that since the patch fixes a call to an undefined function, there could not have been a test covering this case in the first place ;-) In addition, the function call takes place only in the 'bitbake -b' case...Where can it be found, which other changes took place in those builds?
On Fri, 3 Apr 2026 at 10:00, Mathieu Dubois-Briand < [email protected]> wrote:
The tested branch can be found here:
https://git.yoctoproject.org/poky-ci-archive/log/?h=oecore/autobuilder.yoctoproject.org/valkyrie/a-full-3579
But I confirm git bisect points to this commit, and the error is gone after a revert. You can easily reproduce the issue on your side:
oe-selftest -r sstatetests.SStateHashSameSigs2.test_sstate_allarch_samesigs
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Re: [OE-core] [PATCH v2 2/4] package_pkgdata: fix typo to stop calling undefined function
Quoted 15 lines · Mathieu Dubois-Briand
Quoted 4 lines · Adam Blank
Hmmm, can't see any connection between those tests and this particular patch...
The thing is, that since the patch fixes a call to an undefined function, there could not have been a test covering this case in the first place;-)
Quoted 1 line
In addition, the function call takes place only in the 'bitbake -b'
case...
Quoted 6 lines
Where can it be found, which other changes took place in those builds?
On Fri, 3 Apr 2026 at 10:00, Mathieu Dubois-Briand < [email protected]> wrote:
Quoted 9 lines
The tested branch can be found here:
But I confirm git bisect points to this commit, and the error is gone after a revert. You can easily reproduce the issue on your side:
oe-selftest -r sstatetests.SStateHashSameSigs2.test_sstate_allarch_samesigs
I've run this test with my change cherry-picked onto the master, and have got some inconclusive, partially random results... Here's what I did:
- the test started FAILING (as in the autobuilder)
- I replaced the corrected function invocation with an exception throw and nothing changed (this proves, that this branch does not take part in this test)
- I limited the packages built during the test to only some of the ones failing - the test started PASSING
- I started restoring the original package list, and finally restored its original state - the test kept PASSING
- I restored the original state of the workspace - master + my commit (the exact same configuration that I started with) and the test kept PASSING
- I rerun the test a few times, each time in a new, fresh build, and after about 3 FAILING runs it suddenly PASSED 2 times, before it started FAILING again - all the time in the same state of the workspace
I did confirm, that the signatures differed indeed in the failing runs. I chased it for a while, looked here and there (e.g. found it suspicious that native, nativesdk, cross-canadian, etc. clear
out do_packagedata[stamp-extra-info] and allarch does not) but I cannot even dream of the type of free time I'd need to familiarise myself sufficiently with BitBake's hashing machinery.
I think the way to go now is to file a bug - what do you think?
Thanks,
Adam
Quoted 9 lines
Thanks,
Mathieu--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
Re: [OE-core] [PATCH v2 2/4] package_pkgdata: fix typo to stop calling undefined function
Quoted 50 lines · Adam Blank via lists.openembedded.org
Quoted 15 lines · Mathieu Dubois-Briand
Quoted 4 lines · Adam Blank
Hmmm, can't see any connection between those tests and this particular patch...
The thing is, that since the patch fixes a call to an undefined function, there could not have been a test covering this case in the first place;-)
Quoted 1 line
In addition, the function call takes place only in the 'bitbake -b'
case...
Quoted 6 lines
Where can it be found, which other changes took place in those builds?
On Fri, 3 Apr 2026 at 10:00, Mathieu Dubois-Briand < [email protected]> wrote:
Quoted 9 lines
The tested branch can be found here:
But I confirm git bisect points to this commit, and the error is gone after a revert. You can easily reproduce the issue on your side:
oe-selftest -r sstatetests.SStateHashSameSigs2.test_sstate_allarch_samesigsI've run this test with my change cherry-picked onto the master, and have got some inconclusive, partially random results... Here's what I did:
- the test started FAILING (as in the autobuilder)
- I replaced the corrected function invocation with an exception throw and nothing changed (this proves, that this branch does not take part in this test)
- I limited the packages built during the test to only some of the ones failing - the test started PASSING
- I started restoring the original package list, and finally restored its original state - the test kept PASSING
- I restored the original state of the workspace - master + my commit (the exact same configuration that I started with) and the test kept PASSING
- I rerun the test a few times, each time in a new, fresh build, and after about 3 FAILING runs it suddenly PASSED 2 times, before it started FAILING again - all the time in the same state of the workspaceI did confirm, that the signatures differed indeed in the failing runs. I chased it for a while, looked here and there (e.g. found it suspicious that native, nativesdk, cross-canadian, etc. clear
out do_packagedata[stamp-extra-info] and allarch does not) but I cannot even dream of the type of free time I'd need to familiarise myself sufficiently with BitBake's hashing machinery.
Quoted 5 lines
I think the way to go now is to file a bug - what do you think?
Thanks,
Adam
Thanks for taking the time to make these tests.
I am not familiar either with these bitbake parts, so yes, except if someone else comes with an explanation, opening a bug entry is probably the best move here.
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Re: [OE-core] [PATCH v2 2/4] package_pkgdata: fix typo to stop calling undefined function
I've dug some more, as I couldn't come to terms with it ;-)
I've got a fix and a reasoning behind it: when the typo is fixed, a new callable function is identified and treated as a dependency of 'do_package'. This in turn pulls in its dependency on 'PACKAGE_EXTRA_ARCHS' and the test must fail.
The fix is simply about making sure, that 'package_populate_pkgdata_dir' has 'PACKAGE_EXTRA_ARCHS' and a few other variables in its 'vardepsexclude'.
What bothered me was something else though. I could clearly see, how this part of code in 'package.bbclass' was based on 'staging.bbclass'. There is an identical chain of dependencies:
- do_prepare_recipe_sysroot -> extend_recipe_sysroot
-> staging_populate_sysroot_dir (referencing 'PACKAGE_EXTRA_ARCHS')
- do_package -> package_prepare_pkgdata -> package_populate_pkgdata_dir (referencing 'PACKAGE_EXTRA_ARCHS')
I couldn't believe that somehow 'do_package' had this issue, but 'do_prepare_recipe_sysroot' didn't (for no apparent reason). After an extremely long chase, I found 'extend_recipe_sysroot' in 'BB_BASEHASH_IGNORE_VARS'... Since this was more unexpected to me than the Spanish Inquisition, and has been there since 2017, I'd like to ask:
- Is it truly justified for 'extend_recipe_sysroot' to have this sort of globally unique, special treatment?
- Wouldn't it be better for it to be handled using the same mechanism that are applied to other functions?
Thanks,
Adam
Quoted 81 lines · Mathieu Dubois-Briand
Quoted 7 lines · Adam Blank via lists.openembedded.org
Quoted 4 lines · Mathieu Dubois-Briand
Quoted 3 lines · Adam Blank
Hmmm, can't see any connection between those tests and this particular patch...
The thing is, that since the patch fixes a call to an undefinedfunction,
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
there could not have been a test covering this case in the first place
;-)
Quoted 1 line
In addition, the function call takes place only in the 'bitbake -b'
case...
Quoted 3 lines
Where can it be found, which other changes took place in those builds?
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
On Fri, 3 Apr 2026 at 10:00, Mathieu Dubois-Briand < [email protected]> wrote:
The tested branch can be found here:
Quoted 4 lines
Quoted 4 lines
But I confirm git bisect points to this commit, and the error is gone after a revert. You can easily reproduce the issue on your side:
oe-selftest -rsstatetests.SStateHashSameSigs2.test_sstate_allarch_samesigs
Quoted 6 lines
Quoted 2 lines
I've run this test with my change cherry-picked onto the master, and have got some inconclusive, partially random results... Here's what I did:
- the test started FAILING (as in the autobuilder)
- I replaced the corrected function invocation with an exception throwand
Quoted 7 lines
nothing changed (this proves, that this branch does not take part in this test)
- I limited the packages built during the test to only some of the ones failing - the test started PASSING
- I started restoring the original package list, and finally restored its original state - the test kept PASSING
- I restored the original state of the workspace - master + my commit(the
Quoted 2 lines
exact same configuration that I started with) and the test kept PASSING
- I rerun the test a few times, each time in a new, fresh build, andafter
Quoted 1 line
about 3 FAILING runs it suddenly PASSED 2 times, before it started
FAILING
Quoted 4 lines
again - all the time in the same state of the workspace
I did confirm, that the signatures differed indeed in the failing runs. I chased it for a while, looked here and there (e.g. found it suspicious
that
Quoted 10 lines
native, nativesdk, cross-canadian, etc. clear
out do_packagedata[stamp-extra-info] and allarch does not) but I cannot even dream of the type of free time I'd need to familiarise myself sufficiently with BitBake's hashing machinery.I think the way to go now is to file a bug - what do you think?
Thanks,
AdamThanks for taking the time to make these tests.
I am not familiar either with these bitbake parts, so yes, except if someone else comes with an explanation, opening a bug entry is probably the best move here.
Thanks,
Mathieu--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
Re: [OE-core] [PATCH v2 2/4] package_pkgdata: fix typo to stop calling undefined function
Quoted 1 line · Adam Blank
I've dug some more, as I couldn't come to terms with it ;-)
I'm certainly a bit concerned with what you found and have been wanting to try and find time to understand what is going on. It would seem we simply must not be calling that code path?
Quoted 7 lines
I've got a fix and a reasoning behind it: when the typo is fixed, a new callable function is identified and treated as a dependency of '
do_package'. This in turn pulls in its dependency on 'PACKAGE_EXTRA_ARCHS' and the test must fail.
The fix is simply about making sure, that 'package_populate_pkgdata_dir' has 'PACKAGE_EXTRA_ARCHS' and a few other variables in its 'vardepsexclude'.
That is a good find and would certainly explain the error.
Quoted 7 lines
What bothered me was something else though. I could clearly see, how this part of code in 'package.bbclass' was based on 'staging.bbclass'. There is an identical chain of dependencies:
- do_prepare_recipe_sysroot -> extend_recipe_sysroot -Quoted 1 line
staging_populate_sysroot_dir (referencing '
PACKAGE_EXTRA_ARCHS')- do_package -> package_prepare_pkgdata -
Quoted 1 line
package_populate_pkgdata_dir (referencing '
PACKAGE_EXTRA_ARCHS')
That would bother me too and is a very good question.
Quoted 11 lines
I couldn't believe that somehow '
do_package' had this issue, but 'do_prepare_recipe_sysroot' didn't (for no apparent reason). After an extremely long chase, I found 'extend_recipe_sysroot' in 'BB_BASEHASH_IGNORE_VARS'... Since this was more unexpected to me than the Spanish Inquisition, and has been there since 2017, I'd like to ask:- Is it truly justified for '
extend_recipe_sysroot' to have this sort of globally unique, special treatment?
- Wouldn't it be better for it to be handled using the same mechanism that are applied to other functions?
Those are good questions. I suspect this:
* was done to make things a little easier on rebuilds if/as/when that code changes
* was by 'design' in that it was anticipated that extend_recipe_sysroot would be called from more places than it may have ended up being called from
* may have predated the exclusion mechanisms being reliable
I'd not be against using more exclusions and moving it out of BB_BASEHASH_IGNORE_VARS but we'd have to be very careful about the potential new dependency chains and carefully audit them.
Cheers,
Richard