insane.bbclass: in file-rdeps do not look into RDEPENDS recursively
PATCH 1/7 insane.bbclass: in file-rdeps do not look into RDEPENDS recursively
Recursive RDEPENDS resolution requires that all of the dependent recipes' packaging has completed. There is no mechanism to ensure that and therefore races were observed.
This change effectively requires recipes to list their runtime file dependencies explicitly rather than have them pulled indirectly. This may require a bit of fixing in layers, but should result in a better definition of runtime file dependencies.
--- meta/classes/insane.bbclass | 18 ------------------ 1 file changed, 18 deletions(-)
meta/classes/insane.bbclass +0 -18
@@ -722,25 +722,7 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d): filerdepends[subkey] = key[13:] if filerdepends: - next = rdepends done = rdepends[:] - # Find all the rdepends on the dependency chain - while next: - new = [] - for rdep in next: - rdep_data = oe.packagedata.read_subpkgdata(rdep, d) - sub_rdeps = rdep_data.get("RDEPENDS_" + rdep) - if not sub_rdeps: - continue - for sub_rdep in bb.utils.explode_deps(sub_rdeps): - if sub_rdep in done: - continue - if oe.packagedata.has_subpkgdata(sub_rdep, d): - # It's a new rdep - done.append(sub_rdep) - new.append(sub_rdep) - next = new - # Add the rprovides of itself if pkg not in done: done.insert(0, pkg)
Signature · 1 line
2.17.1
PATCH 2/7 stress-ng: add a patch to remove unneeded bash dependency
--- ...tion-remove-the-shebang-at-the-start.patch | 23 +++++++++++++++++++ .../stress-ng/stress-ng_0.10.00.bb | 1 + 2 files changed, 24 insertions(+) create mode 100644 meta/recipes-extended/stress-ng/stress-ng/0001-bash-completion-remove-the-shebang-at-the-start.patch
meta/recipes-extended/stress-ng/stress-ng/0001-bash-completion-remove-the-shebang-at-the-start.patch new file +23 -0
@@ -0,0 +1,23 @@ +From 042147675c7c2ea7dd65b2597f2e350376a710aa Mon Sep 17 00:00:00 2001 +From: Alexander Kanavin <[email protected]> +Date: Tue, 6 Aug 2019 17:28:56 +0200 +Subject: [PATCH] bash-completion: remove the shebang at the start + +bash completion files do not need to specify that. + +Upstream-Status: Pending +Signed-off-by: Alexander Kanavin <[email protected]> +--- + bash-completion/stress-ng | 2 -- + 1 file changed, 2 deletions(-) + +diff --git a/bash-completion/stress-ng b/bash-completion/stress-ng +index 8b1421c..7f195be 100755 +--- a/bash-completion/stress-ng ++++ b/bash-completion/stress-ng +@@ -1,5 +1,3 @@ +-#!/bin/bash +-# + # stress-ng tab completion for bash. + # + # Copyright (C) 2019 Canonical
meta/recipes-extended/stress-ng/stress-ng_0.10.00.bb +1 -0
@@ -7,6 +7,7 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263" SRC_URI = "https://kernel.ubuntu.com/~cking/tarballs/${BPN}/${BP}.tar.xz \ file://0001-Do-not-preserve-ownership-when-installing-example-jo.patch \ + file://0001-bash-completion-remove-the-shebang-at-the-start.patch \ " SRC_URI[md5sum] = "46aa41d37690324ceab4febfcc549018" SRC_URI[sha256sum] = "d09dd2a1aea549e478995bf9be90b38906a4cdf33ea7b245ef9d46aa5213c074"
Signature · 1 line
2.17.1
PATCH 3/7 elfutils: use PRIVATE_LIBS for the ptest package
In reply to Alexander KanavinEXCLUDE_PACKAGES_FROM_SHLIBS is too broad: it suppresses both generation of required and provided shlibs. We need to suppress only the provided shlibs (to avoid clashes with the main package providing the same shlibs), and run the required shlib dependencies generator as usual.
--- meta/recipes-devtools/elfutils/elfutils_0.176.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/elfutils/elfutils_0.176.bb +1 -1
@@ -142,4 +142,4 @@ INHIBIT_PACKAGE_STRIP_FILES = "\ ${PKGD}${PTEST_PATH}/backends/libebl_x86_64.so \ " -EXCLUDE_PACKAGES_FROM_SHLIBS = "${PN}-ptest" +PRIVATE_LIBS_${PN}-ptest = "libdw.so.1 libelf.so.1"
Signature · 1 line
2.17.1
PATCH 4/7 apt: add a missing perl runtime dependency
In reply to Alexander Kanavin/usr/lib/dpkg/methods/apt/setup is a perl script.
--- meta/recipes-devtools/apt/apt-package.inc | 1 + 1 file changed, 1 insertion(+)
meta/recipes-devtools/apt/apt-package.inc +1 -0
@@ -35,6 +35,7 @@ FILES_${PN} = "${bindir}/apt-cdrom ${bindir}/apt-get \ ${libdir}/dpkg \ ${systemd_unitdir}/system \ " +RDEPENDS_${PN} += "perl" FILES_${PN}-utils = "${bindir}/apt-sortpkgs ${bindir}/apt-extracttemplates" FILES_${PN}-doc = "${@get_files_apt_doc(d, bb, d.getVar('apt-manpages'))} \ ${docdir}/apt"
Signature · 1 line
2.17.1
PATCH 5/7 attr: add a missing perl runtime dependency
In reply to Alexander Kanavin/usr/lib/attr/ptest/test/sort-getfattr-output is a perl script.
--- meta/recipes-support/attr/attr.inc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-support/attr/attr.inc +1 -1
@@ -41,6 +41,6 @@ do_install_ptest() { sed -e 's|; @echo|; echo|' -i ${D}${PTEST_PATH}/test/Makefile } -RDEPENDS_${PN}-ptest = "attr coreutils perl-module-filehandle perl-module-getopt-std perl-module-posix make" +RDEPENDS_${PN}-ptest = "attr coreutils perl-module-filehandle perl-module-getopt-std perl-module-posix make perl" BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.17.1
PATCH 6/7 ofono: correct the python3 runtime dependency
In reply to Alexander Kanavin--- meta/recipes-connectivity/ofono/ofono.inc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-connectivity/ofono/ofono.inc +1 -1
@@ -35,5 +35,5 @@ RRECOMMENDS_${PN} += "kernel-module-tun mobile-broadband-provider-info" FILES_${PN} += "${systemd_unitdir}" FILES_${PN}-tests = "${libdir}/${BPN}/test" -RDEPENDS_${PN}-tests = "python3 python3-dbus" +RDEPENDS_${PN}-tests = "python3-core python3-dbus" RDEPENDS_${PN}-tests += "${@bb.utils.contains('GI_DATA_ENABLED', 'True', 'python3-pygobject', '', d)}"
Signature · 1 line
2.17.1
PATCH 7/7 bluez5: correct the python3 runtime dependency
In reply to Alexander Kanavin--- meta/recipes-connectivity/bluez5/bluez5.inc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-connectivity/bluez5/bluez5.inc +1 -1
@@ -139,7 +139,7 @@ def get_noinst_tools_paths (d, bb, tools): FILES_${PN}-noinst-tools = "${@get_noinst_tools_paths(d, bb, d.getVar('NOINST_TOOLS'))}" -RDEPENDS_${PN}-testtools += "python3 python3-dbus" +RDEPENDS_${PN}-testtools += "python3-core python3-dbus" RDEPENDS_${PN}-testtools += "${@bb.utils.contains('GI_DATA_ENABLED', 'True', 'python3-pygobject', '', d)}" SYSTEMD_SERVICE_${PN} = "${@bb.utils.contains('PACKAGECONFIG', 'systemd', 'bluetooth.service', '', d)}"
Signature · 1 line
2.17.1
Quoted 10 lines · Alexander Kanavin
Recursive RDEPENDS resolution requires that all of the dependent recipes' packaging has completed. There is no mechanism to ensure that
and therefore races were observed.This change effectively requires recipes to list their runtime file dependencies explicitly rather than have them pulled indirectly. This may require a bit of fixing in layers, but should result in a better definition of runtime file dependencies.
Quoted 4 lines
Signed-off-by Alexander Kanavin <[email protected]>--- meta/classes/insane.bbclass | 18 ------------------ 1 file changed, 18 deletions(-)
You might want to have a look at
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/621
and maybe other pieces of that build yet to finish, not sure...
Cheers,
Richard
Quoted 4 lines · Richard Purdie
You might want to have a look at
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/621
and maybe other pieces of that build yet to finish, not sure...
I ran bitbake world to check for newly exposed missing dependencies, but gcc-cross-canadian wasn't included in that. Will fix that too now.
Alex
Quoted 11 lines · Alexander Kanavin
Quoted 4 lines · Richard Purdie
You might want to have a look at
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/621
and maybe other pieces of that build yet to finish, not sure...I ran bitbake world to check for newly exposed missing dependencies, but gcc-cross-canadian wasn't included in that. Will fix that too now.
There are several packages in meta-openembedded which are showing these errors one that I see is net kit-rusers seems to be failing for wrong reason
I will send full list once I am close to computer
Quoted 8 lines
Alex
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Quoted 12 lines · Khem Raj
Quoted 8 lines · Alexander Kanavin
Quoted 4 lines · Richard Purdie
You might want to have a look at
https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/621
and maybe other pieces of that build yet to finish, not sure...
Quoted 6 lines
Quoted 1 line
I ran bitbake world to check for newly exposed missing dependencies, but gcc-cross-canadian wasn't included in that. Will fix that too now.
There are several packages in meta-openembedded which are showing these errors one that I see is net kit-rusers seems to be failing for wrong reason
I will send full list once I am close to computer
here is full list
https://errors.yoctoproject.org/Errors/Build/86513/
all do_package_qa are due to this change.
Quoted 7 lines
Quoted 7 lines
Alex
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Quoted 10 lines · Khem Raj
Quoted 1 line
There are several packages in meta-openembedded which are showing these
errors one that I see is net kit-rusers seems to be failing for wrong reason
Quoted 2 lines
I will send full list once I am close to computer
here is full list
https://errors.yoctoproject.org/Errors/Build/86513/
alldo_package_qaare due to this change.
There are plenty of recipes in that list that come from oe-core (not meta-oe), and do not show errors on the AB, e.g. glib-2.0:
ERROR: QA Issue: /usr/lib/libgio-2.0.so.0.6000.6 contained in package glib-2.0 requires libz.so.1(ZLIB_1.2.2)(64bit), but no providers found in RDEPENDS_glib-2.0? [file-rdeps]
ERROR: QA Issue: /usr/lib/libgio-2.0.so.0.6000.6 contained in package glib-2.0 requires libz.so.1()(64bit), but no providers found in RDEPENDS_glib-2.0? [file-rdeps]
ERROR: QA Issue: /usr/lib/libglib-2.0.so.0.6000.6 contained in package glib-2.0 requires libpcre.so.1()(64bit), but no providers found in RDEPENDS_glib-2.0? [file-rdeps]
ERROR: QA Issue: /usr/lib/libgobject-2.0.so.0.6000.6 contained in package glib-2.0 requires libffi.so.6()(64bit), but no providers found in RDEPENDS_glib-2.0? [file-rdeps]
Can I see the configuration where this happened? Might be some other setting that breaks shlib resolution (that was previously unnoticed due to recursive qa RDEPENDS).
Alex
Quoted 13 lines · Alexander Kanavin
Quoted 9 lines · Khem Raj
Quoted 4 lines
There are several packages in meta-openembedded which are showing these errors one that I see is net kit-rusers seems to be failing for wrong reason
I will send full list once I am close to computer
here is full list
https://errors.yoctoproject.org/Errors/Build/86513/
alldo_package_qaare due to this change.
Quoted 9 lines
There are plenty of recipes in that list that come from oe-core (not meta-oe), and do not show errors on the AB, e.g. glib-2.0:
ERROR: QA Issue:
/usr/lib/libgio-2.0.so.0.6000.6contained in package glib-2.0 requires libz.so.1(ZLIB_1.2.2)(64bit), but no providers found in RDEPENDS_glib-2.0? [file-rdeps]
ERROR: QA Issue:/usr/lib/libgio-2.0.so.0.6000.6contained in package glib-2.0 requires libz.so.1()(64bit), but no providers found in RDEPENDS_glib-2.0? [file-rdeps]
ERROR: QA Issue:/usr/lib/libglib-2.0.so.0.6000.6contained in package glib-2.0 requires libpcre.so.1()(64bit), but no providers found in RDEPENDS_glib-2.0? [file-rdeps]
ERROR: QA Issue:/usr/lib/libgobject-2.0.so.0.6000.6contained in package glib-2.0 requires libffi.so.6()(64bit), but no providers found in RDEPENDS_glib-2.0? [file-rdeps]Can I see the configuration where this happened? Might be some other setting that breaks shlib resolution (that was previously unnoticed due to recursive qa RDEPENDS).
a running musl world build did not show this issue but that was rebased tree without this fix. I will have to do a new build for glibc and see if it was my build system injected behavior. For this patch, I would suggest it's tested across multiple layers before it's installed to oe-core.
Quoted 2 lines
Alex
Quoted 6 lines · Khem Raj
a running musl world build did not show this issue but that was rebased tree without this fix. I will have to do a new build for glibc and see if it was my build system injected behavior. For this patch, I would suggest it's tested across multiple layers before it's installed to oe-core.
We should certainly at least fix the meta-oe issues, if you can produce a list of affected recipes I would appreciate :)
(don't really want to start a build of entire meta-oe set here)
Alex
Quoted 13 lines · Alexander Kanavin
Quoted 9 lines · Khem Raj
Quoted 4 lines
There are several packages in meta-openembedded which are showing these errors one that I see is net kit-rusers seems to be failing for wrong reason
I will send full list once I am close to computer
here is full list
https://errors.yoctoproject.org/Errors/Build/86513/
alldo_package_qaare due to this change.
Quoted 8 lines
There are plenty of recipes in that list that come from oe-core (not meta-oe), and do not show errors on the AB, e.g. glib-2.0:
ERROR: QA Issue:
/usr/lib/libgio-2.0.so.0.6000.6contained in package glib-2.0 requires libz.so.1(ZLIB_1.2.2)(64bit), but no providers found in RDEPENDS_glib-2.0? [file-rdeps]
ERROR: QA Issue:/usr/lib/libgio-2.0.so.0.6000.6contained in package glib-2.0 requires libz.so.1()(64bit), but no providers found in RDEPENDS_glib-2.0? [file-rdeps]
ERROR: QA Issue:/usr/lib/libglib-2.0.so.0.6000.6contained in package glib-2.0 requires libpcre.so.1()(64bit), but no providers found in RDEPENDS_glib-2.0? [file-rdeps]
ERROR: QA Issue:/usr/lib/libgobject-2.0.so.0.6000.6contained in package glib-2.0 requires libffi.so.6()(64bit), but no providers found in RDEPENDS_glib-2.0? [file-rdeps]Can I see the configuration where this happened? Might be some other setting that breaks shlib resolution (that was previously unnoticed due to recursive qa RDEPENDS).
well, this seems to go away once I disable hash equivalency
#INHERIT += "reproducible_build"
#BB_HASHSERVE = "localhost:0"
#BB_SIGNATURE_HANDLER = "OEEquivHash"
Richard, another issue I saw was slowness in world builds when I had this enabled
I was on master-next for both core and bitbake. It was spending a lot of time in ensuring
the build is not needed so much that the build took almost same time than the build with
out it where the build without it will build everything.
Quoted 9 lines · Khem Raj
Richard, another issue I saw was slowness in world builds when I had this enabled
I was on master-next for both core and bitbake. It was spending a lot of time in ensuring
the build is not needed so much that the build took almost same time than the build with
out it where the build without it will build everything.
world builds seem to have regressed (preparation time wise) even without having hash equivalency. Just oe-core is bearable, but adding meta-oe layers makes it spent many minutes after initializing tasks but before any tasks actually begin.
Alex
Quoted 6 lines · Alexander Kanavin
Quoted 5 lines · Khem Raj
Richard, another issue I saw was slowness in world builds when I had this enabled I was on master-next for both core and bitbake. It was spending a lot of time in ensuring the build is not needed so much that the build took almost same time than the build with out it where the build without it will build everything.
Were you using the patch in meta-yocto master-next or your own?
There is definitely a server connection problem with the hashserver right now but if you used the setting in meta-yocto master-next you'd probably only ever see timeouts which would be unbearably slow.
Quoted 5 lines
world builds seem to have regressed (preparation time wise) even without having hash equivalency. Just oe-core is bearable, but adding meta-oe layers makes it spent many minutes after initializing tasks but before any tasks actually begin.
Is that with master or current master-next?
I'm curious if master-next is better or worse...
To be honest I'm worrying more about correctness right now.
Cheers,
Richard
Quoted 6 lines · On Mon, 12 Aug 2019 at 22:50, <[email protected]> wrote:
Quoted 5 lines
world builds seem to have regressed (preparation time wise) even without having hash equivalency. Just oe-core is bearable, but adding meta-oe layers makes it spent many minutes after initializing tasks but before any tasks actually begin.
Quoted 6 lines
Is that with master or current master-next?
I'm curious if master-next is better or worse...
To be honest I'm worrying more about correctness right now.
Certainly. I didn't want to raise this now as you have plenty on your plate, but as Khem did, I had to chime in.
This was with master@9983b07fffd19082abded7c3f15cc77d306dd69c.
Alex