packagegroup.bbclass: Drop build-time dependencies
RFC PATCH 1/1 packagegroup.bbclass: Drop build-time dependencies
* RFC because:
packagegroup.bbclass inherits allarch and by default sets PACKAGE_ARCH = "all", but then it has runtime-dependency on TUNE_PKGARCH packages and different signatures, so instead of setting INHIBIT_DEFAULT_DEPS in addition to inherit allarch I would like to drop default allarch completely.
* drop virtual/kernel dependency from packagegroup-core-boot because kernel isn't needed to build packagegroup and building it doesn't influence if it's included in image or not.
--- meta/classes/packagegroup.bbclass | 4 ++++ meta/recipes-core/packagegroups/packagegroup-core-boot.bb | 1 - 2 files changed, 4 insertions(+), 1 deletion(-)
meta/classes/packagegroup.bbclass +4 -0
@@ -11,6 +11,10 @@ PACKAGES = "${PN}" # need to be set to MACHINE_ARCH after inheriting packagegroup.bbclass inherit allarch +# allarch.bbclass does that only when PACKAGE_ARCH = "all", +# but we don't want any build time dependencies even for MACHINE_ARCH packagegroups +INHIBIT_DEFAULT_DEPS = "1" + # This automatically adds -dbg and -dev flavours of all PACKAGES # to the list. Their dependencies (RRECOMMENDS) are handled as usual # by package_depchains in a following step.
meta/recipes-core/packagegroups/packagegroup-core-boot.bb +0 -1
@@ -5,7 +5,6 @@ SUMMARY = "Minimal boot requirements" DESCRIPTION = "The minimal set of packages required to boot the system" LICENSE = "MIT" -DEPENDS = "virtual/kernel" PR = "r11" inherit packagegroup
Signature · 1 line
1.8.4.2
Hi Martin,
Quoted 6 lines · On Friday 01 November 2013 13:12:12 Martin Jansa wrote:
* RFC because:
packagegroup.bbclass inherits allarch and by default setsPACKAGE_ARCH= "all", but then it has runtime-dependency onTUNE_PKGARCHpackages and different signatures, so instead of settingINHIBIT_DEFAULT_DEPSin addition to inherit allarch I would like to drop default allarch completely.
The fix looks OK, but I don't think we should be going further and removing the inherit. To my mind, the question of dependencies comes down to whether or not the packagegroup output changes depending on TUNE_PKGARCH, which in the general case it should not do.
If we need to do some optimisation to avoid the packagegroup recipe rebuilding when it shouldn't due to packages it "contains" changing, we can certainly look at that.
Quoted 3 lines
* drop virtual/kernel dependency from packagegroup-core-boot because kernel isn't needed to build packagegroup and building it doesn't influence if it's included in image or not.
Yes, I agree this seems erroneous.
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Quoted 10 lines · Paul Eggleton
Hi Martin,
Quoted 6 lines · On Friday 01 November 2013 13:12:12 Martin Jansa wrote:
* RFC because:
packagegroup.bbclass inherits allarch and by default setsPACKAGE_ARCH= "all", but then it has runtime-dependency onTUNE_PKGARCHpackages and different signatures, so instead of settingINHIBIT_DEFAULT_DEPSin addition to inherit allarch I would like to drop default allarch completely.
Quoted 8 lines
The fix looks OK, but I don't think we should be going further and removing the inherit. To my mind, the question of dependencies comes down to whether or not the packagegroup output changes depending on
TUNE_PKGARCH, which in the general case it should not do.If we need to do some optimisation to avoid the packagegroup recipe rebuilding when it shouldn't due to packages it "contains" changing, we can certainly look at that.
A better idea might be something like:
meta/lib/oe/sstatesig.py +6 -0
@@ -11,6 +11,8 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache): def isKernel(fn): inherits = " ".join(dataCache.inherits[fn]) return inherits.find("module-base.bbclass") != -1 or inherits.find("linux-kernel-base.bbclass") != -1 + def isPackageGroup(x): + return "packagegroup-" in x # Always include our own inter-task dependencies if recipename == depname:
@@ -27,6 +29,10 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache): # Only target packages beyond here + # packagegroups are assumed to have well behaved names which don't change between architecures/tunes + if isPackageGroup(recipename): + return False + # Drop native/cross/nativesdk dependencies from target recipes if isNative(depname) or isCross(depname) or isNativeSDK(depname): return False
although this is untested and needs more careful thought...
Cheers,
Richard
Quoted 29 lines · On Friday 01 November 2013 18:14:03 Richard Purdie wrote:
Quoted 8 lines · Paul Eggleton
Quoted 6 lines · On Friday 01 November 2013 13:12:12 Martin Jansa wrote:
* RFC because:
packagegroup.bbclass inherits allarch and by default setsPACKAGE_ARCH= "all", but then it has runtime-dependency onTUNE_PKGARCHpackages and different signatures, so instead of settingINHIBIT_DEFAULT_DEPSin addition to inherit allarch I would like to drop default allarch completely.Quoted 8 lines
The fix looks OK, but I don't think we should be going further and removing the inherit. To my mind, the question of dependencies comes down to whether or not the packagegroup output changes depending on
TUNE_PKGARCH, which in the general case it should not do.If we need to do some optimisation to avoid the packagegroup recipe rebuilding when it shouldn't due to packages it "contains" changing, we can certainly look at that.
A better idea might be something like:
meta/lib/oe/sstatesig.py +0
-0@@ -11,6 +11,8 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep,depname, dataCache): def isKernel(fn):
inherits = " ".join(dataCache.inherits[fn]) returninherits.find("module-base.bbclass")!= -1 orinherits.find("linux-kernel-base.bbclass")!= -1
meta/lib/oe/sstatesig.py def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache): · line 15
Quoted 2 lines
meta/lib/oe/sstatesig.py +2
-0+ def isPackageGroup(x): + return "packagegroup-" in x
OK, but I think we can rely on checking inheritance for this. It didn't used to be the case but after my task -> packagegroup cleanup some time ago all of these recipes inherit the packagegroup class.
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
PATCH 1/7 allarch: Always inhibit default dependencies and set empty TARGET_PREFIX
In reply to Martin Jansa* typical case where we inherit allarch and override PACKAGE_ARCH are packagegroup recipes, but those need default dependencies inhibited even when they are MACHINE_ARCH or TUNE_PKGARCH. I don't know about any recipe which inherits allarch and needs default dependencies.
* set empty TARGET_PREFIX
This has a bit weird reason caused by unsupported setup where external-toolchain is used in some DISTRO only for some MACHINEs and internal is used for other MACHINEs.
Because external-toolchain usually comes with different TARGET_PREFIX it was causing allarch recipes to have different signatures even when they don't use toolchain at all.
Empty TARGET_PREFIX also helps to find allarch recipes which still have default dependency on e.g. virtual/${TARGET_PREFIX}gcc.
--- meta/classes/allarch.bbclass | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
meta/classes/allarch.bbclass +5 -3
@@ -9,12 +9,13 @@ STAGING_DIR_HOST := "${STAGING_DIR_HOST}" PACKAGE_ARCH = "all" python () { + # No need for virtual/libc or a cross compiler even for recipes which + # change PACKAGE_ARCH e.g. to MACHINE_ARCH + d.setVar("INHIBIT_DEFAULT_DEPS","1") + # Allow this class to be included but overridden - only set # the values if we're still "all" package arch. if d.getVar("PACKAGE_ARCH") == "all": - # No need for virtual/libc or a cross compiler - d.setVar("INHIBIT_DEFAULT_DEPS","1") - # Set these to a common set of values, we shouldn't be using them other that for WORKDIR directory # naming anyway d.setVar("TARGET_ARCH", "allarch")
@@ -23,6 +24,7 @@ python () { d.setVar("TARGET_LD_ARCH", "none") d.setVar("TARGET_AS_ARCH", "none") d.setVar("PACKAGE_EXTRA_ARCHS", "") + d.setVar("TARGET_PREFIX", "") # No need to do shared library processing or debug symbol handling d.setVar("EXCLUDE_FROM_SHLIBS", "1")
Signature · 1 line
1.8.4.3
PATCH 2/7 packagegroup-core-boot: Drop build-time dependency on virtual/kernel
* because kernel isn't needed to build packagegroup and building it doesn't influence if it's included in image or not.
--- meta/recipes-core/packagegroups/packagegroup-core-boot.bb | 1 - 1 file changed, 1 deletion(-)
meta/recipes-core/packagegroups/packagegroup-core-boot.bb +0 -1
@@ -5,7 +5,6 @@ SUMMARY = "Minimal boot requirements" DESCRIPTION = "The minimal set of packages required to boot the system" LICENSE = "MIT" -DEPENDS = "virtual/kernel" PR = "r11" inherit packagegroup
Signature · 1 line
1.8.4.3
Re: [PATCH 2/7] packagegroup-core-boot: Drop build-time dependency on virtual/kernel
Quoted 13 lines · Martin Jansa
* because kernel isn't needed to build packagegroup and building it doesn't influence if it's included in image or not.
Signed-off-by Martin Jansa <[email protected]>--- meta/recipes-core/packagegroups/packagegroup-core-boot.bb | 1 - 1 file changed, 1 deletion(-)meta/recipes-core/packagegroups/packagegroup-core-boot.bb +0
-0
meta/recipes-core/packagegroups/packagegroup-core-boot.bb line 10
Quoted 7 lines
meta/recipes-core/packagegroups/packagegroup-core-boot.bb +0
-1@@ -5,7 +5,6 @@ SUMMARY = "Minimal boot requirements" DESCRIPTION = "The minimal set of packages required to boot the system" LICENSE = "MIT" -DEPENDS = "virtual/kernel" PR = "r11" inherit packagegroup
What about adding this dependency in image.bbclass? Otherwise, the kernel will not get built if it's not pulled in by other packages like v86d. For example, if we build core-image-minimal for qemumips, the kernel is not built by default and we have to bitbake linux-yocto' before we can use runqemu' to start target.
Best Regards,
Chen Qi
PATCH 3/7 xuser-account: Drop allarch inherit
In reply to Martin Jansa* has build time dependency on TUNE_PKGARCH recipes:
bitbake-diffsigs sstate-diff/1383303628/*/all*/xuser-account/*do_configure*
Hash for dependent task base-passwd_3.5.26.bb.do_populate_sysroot changed from 09b3168865b6212269d4e5512a963b9e to ffa22dc51e42a164a7392ec719151629 Hash for dependent task shadow_4.1.4.3.bb.do_populate_sysroot changed from f426f0c93b2d341050b2547804645314 to 89bdbb151f8c200553c9a74e89504457 Hash for dependent task shadow-sysroot_4.1.4.3.bb.do_populate_sysroot changed from c42103fe54ac5109eab8de4fb0eb9a75 to f83f628e766fdbd49eb2a44cf6134339
--- meta/recipes-support/user-creation/xuser-account_0.1.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-support/user-creation/xuser-account_0.1.bb +1 -1
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384 SRC_URI = "" -inherit allarch useradd +inherit useradd do_configure() { :
Signature · 1 line
1.8.4.3
PATCH 4/7 linux-firmware: Drop allarch
In reply to Martin Jansa* the recipe itself is allarch, but update-alternatives adds dependency on
virtual/${MLPREFIX}update-alternatives which can be TUNE_PKGARCH opkg
causing linux-firmware signature to be different for different TUNE_PKGARCHs
Hash for dependent task opkg_svn.bb.do_populate_sysroot changed from 8d215059308161841a84542dd1dcbf3a to 80b280c741c392cf5b30dd67b61dd56e
--- meta/recipes-kernel/linux-firmware/linux-firmware_git.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-kernel/linux-firmware/linux-firmware_git.bb +1 -1
@@ -32,7 +32,7 @@ SRC_URI = "git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware S = "${WORKDIR}/git" -inherit allarch update-alternatives +inherit update-alternatives do_compile() { :
Signature · 1 line
1.8.4.3
Re: [PATCH 4/7] linux-firmware: Drop allarch
Quoted 15 lines · Martin Jansa
* the recipe itself is allarch, but update-alternatives adds dependency on
virtual/${MLPREFIX}update-alternatives which can beTUNE_PKGARCHopkg
causing linux-firmware signature to be different for differentTUNE_PKGARCHs
Hash for dependent task opkg_svn.bb.do_populate_sysroot changed from8d215059308161841a84542dd1dcbf3ato80b280c741c392cf5b30dd67b61dd56eSigned-off-by Martin Jansa <[email protected]>--- meta/recipes-kernel/linux-firmware/linux-firmware_git.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/recipes-kernel/linux-firmware/linux-firmware_git.bb +0
-0
meta/recipes-kernel/linux-firmware/linux-firmware_git.bb SRC_URI = "git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware · line 35
Quoted 5 lines
meta/recipes-kernel/linux-firmware/linux-firmware_git.bb +1
-1@@ -32,7 +32,7 @@ SRC_URI = "git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware S = "${WORKDIR}/git" -inherit allarch update-alternatives +inherit update-alternatives
I think this is one case where we should whitelist update-alternatives as being ABI/API safe instead. There is no point in building the firmware multiple times.
Cheers,
Richard
PATCH 5/7 ppp-dialin: Drop allarch
In reply to Martin Jansa* depends on some TUNE_PKGARCH recipes:
base-passwd, ppp_, shadow-sysroot, shadow
Hash for dependent task base-passwd_3.5.26.bb.do_populate_sysroot changed from ff08f0198f5424a21ccb61957325ffdf to d9d962e2d8c0d591b916ed68aa826efe Hash for dependent task ppp_2.4.5.bb.do_populate_sysroot changed from 515251fb9d5196733f9b6ca0f6cc041b to 5cfb0b7b71f4440c3147da52c22234cc Hash for dependent task shadow-sysroot_4.1.4.3.bb.do_populate_sysroot changed from 3e5bddc578887636c1809e4cdf28c1c2 to 8f03231a1d4bb1aea6c4adce1febc1b4 Hash for dependent task shadow_4.1.4.3.bb.do_populate_sysroot changed from 518f7edcba3545e5c60c3a13cef23169 to 7dbf1646de02a117bf0d573276b878c0
--- meta/recipes-connectivity/ppp-dialin/ppp-dialin_0.1.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-connectivity/ppp-dialin/ppp-dialin_0.1.bb +1 -1
@@ -10,7 +10,7 @@ LIC_FILES_CHKSUM = "file://${COREBASE}/LICENSE;md5=3f40d7994397109285ec7b81fdeb3 SRC_URI = "file://host-peer \ file://ppp-dialin" -inherit allarch useradd +inherit useradd do_install() { install -d ${D}${sysconfdir}/ppp/peers
Signature · 1 line
1.8.4.3
PATCH 6/7 initramfs-framework: Drop allarch
In reply to Martin Jansa* depends on TUNE_PKGARCH busybox and udev
Hash for dependent task busybox_1.20.2.bb.do_packagedata changed from b814437371de0c3253bdd3610c051f30 to 7dd9c851d5f7c675dd65cc7872c30743 Hash for dependent task udev_182.bb.do_packagedata changed from 1034300ce0e27d444e0e7d9e2a239605 to c13031b26b2a86dcf9bcce33b7823477
--- meta/recipes-core/initrdscripts/initramfs-framework_1.0.bb | 2 -- 1 file changed, 2 deletions(-)
meta/recipes-core/initrdscripts/initramfs-framework_1.0.bb +0 -2
@@ -5,8 +5,6 @@ RDEPENDS_${PN} += "busybox" PR = "r2" -inherit allarch - SRC_URI = "file://init \ file://finish \ file://mdev \
Signature · 1 line
1.8.4.3
PATCH 7/7 resolvconf: Drop allarch
In reply to Martin Jansa* has runtime dependency on TUNE_PKGARCH bash
Hash for dependent task bash_4.2.bb.do_packagedata changed from d6eafb155c37bc2da7327492afa449fb to 0ae080aaa92025af690cf16ef0cd2b7e
--- meta/recipes-connectivity/resolvconf/resolvconf_1.74.bb | 2 -- 1 file changed, 2 deletions(-)
meta/recipes-connectivity/resolvconf/resolvconf_1.74.bb +0 -2
@@ -16,8 +16,6 @@ SRC_URI = "${DEBIAN_MIRROR}/main/r/resolvconf/resolvconf_${PV}.tar.gz" SRC_URI[md5sum] = "2f190d3bb8960b69157f63590c262e93" SRC_URI[sha256sum] = "2e72e6884e9105cbf57101ab0f11e768717b6f76b7f5100c6a2a0cc69bb3d4a0" -inherit allarch - do_compile () { : }
Signature · 1 line
1.8.4.3
Re: [PATCH 1/7] allarch: Always inhibit default dependencies and set empty TARGET_PREFIX
In reply to Martin JansaQuoted 5 lines · Martin Jansa
* typical case where we inherit allarch and override
PACKAGE_ARCHare packagegroup recipes, but those need default dependencies inhibited even when they areMACHINE_ARCHorTUNE_PKGARCH. I don't know about any recipe which inherits allarch and needs default dependencies.
The code there was added to allow the allarch class to be enabled or disabled. I don't remember exactly why we needed to do that however it was added for a reason and making part of it unconditional again will probably break whyever we made it optional :(.
I can understand how you came to this conclusion though. Which cases is this causing problems for?
Quoted 9 lines
* set empty
TARGET_PREFIX
This has a bit weird reason caused by unsupported setup where external-toolchain is used in some DISTRO only for some MACHINEs and internal is used for other MACHINEs.
Because external-toolchain usually comes with differentTARGET_PREFIXit was causing allarch recipes to have different signatures even when they don't use toolchain at all.
EmptyTARGET_PREFIXalso helps to find allarch recipes which still have default dependency on e.g. virtual/${TARGET_PREFIX}gcc.
This seems ok, I'd have taken it if it was a separate patch.
Cheers,
Richard
Re: [PATCH 1/7] allarch: Always inhibit default dependencies and set empty TARGET_PREFIX
Quoted 11 lines · Richard Purdie
Quoted 5 lines · Martin Jansa
* typical case where we inherit allarch and override
PACKAGE_ARCHare packagegroup recipes, but those need default dependencies inhibited even when they areMACHINE_ARCHorTUNE_PKGARCH. I don't know about any recipe which inherits allarch and needs default dependencies.The code there was added to allow the allarch class to be enabled or disabled. I don't remember exactly why we needed to do that however it was added for a reason and making part of it unconditional again will probably break whyever we made it optional :(.
I think that use case was MACHINE_ARCH packagegroups loosing -gnueabi suffix, so that everything except packagegroups (or other MACHINE_ARCH allarch recipes) was built in workdir:
MACHINE-oe-linux-gnueabi
and packagegroups in
MACHINE-oe-linux
I don't think we had the use case where we needed to conditionally keep default deps (but of course my memory isn't perfect and I can be wrong).
Quoted 13 lines
I can understand how you came to this conclusion though. Which cases is this causing problems for?
Quoted 9 lines
* set empty
TARGET_PREFIX
This has a bit weird reason caused by unsupported setup where external-toolchain is used in some DISTRO only for some MACHINEs and internal is used for other MACHINEs.
Because external-toolchain usually comes with differentTARGET_PREFIXit was causing allarch recipes to have different signatures even when they don't use toolchain at all.
EmptyTARGET_PREFIXalso helps to find allarch recipes which still have default dependency on e.g. virtual/${TARGET_PREFIX}gcc.
Quoted 1 line
This seems ok, I'd have taken it if it was a separate patch.
It's related to above, because "thanks" to empty TARGET_PREFIX I've found few MACHINE_ARCH+allarch recipes in meta-webos which weren't using toolchain, but default deps had nonexistent "virtual/gcc".
So we cannot merge TARGET_PREFIX without the above part.
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]
Re: [PATCH 1/7] allarch: Always inhibit default dependencies and set empty TARGET_PREFIX
Quoted 36 lines · Martin Jansa
Quoted 11 lines · Richard Purdie
Quoted 5 lines · Martin Jansa
* typical case where we inherit allarch and override
PACKAGE_ARCHare packagegroup recipes, but those need default dependencies inhibited even when they areMACHINE_ARCHorTUNE_PKGARCH. I don't know about any recipe which inherits allarch and needs default dependencies.The code there was added to allow the allarch class to be enabled or disabled. I don't remember exactly why we needed to do that however it was added for a reason and making part of it unconditional again will probably break whyever we made it optional :(.
I think that use case was
MACHINE_ARCHpackagegroups loosing -gnueabi suffix, so that everything except packagegroups (or otherMACHINE_ARCHallarch recipes) was built in workdir:
MACHINE-oe-linux-gnueabi
and packagegroups in
MACHINE-oe-linuxI don't think we had the use case where we needed to conditionally keep default deps (but of course my memory isn't perfect and I can be wrong).
Quoted 13 lines
I can understand how you came to this conclusion though. Which cases is this causing problems for?
Quoted 9 lines
* set empty
TARGET_PREFIX
This has a bit weird reason caused by unsupported setup where external-toolchain is used in some DISTRO only for some MACHINEs and internal is used for other MACHINEs.
Because external-toolchain usually comes with differentTARGET_PREFIXit was causing allarch recipes to have different signatures even when they don't use toolchain at all.
EmptyTARGET_PREFIXalso helps to find allarch recipes which still have default dependency on e.g. virtual/${TARGET_PREFIX}gcc.
Quoted 5 lines
Quoted 1 line
This seems ok, I'd have taken it if it was a separate patch.
It's related to above, because "thanks" to empty
TARGET_PREFIXI've found few MACHINE_ARCH+allarch recipes in meta-webos which weren't using toolchain, but default deps had nonexistent "virtual/gcc".
Ah sorry this wasn't very accurate, I've found it with earlier version of this patch which was unconditionally inhibiting default deps only for packagegroup.bbclass not allarch.bbclass where it should be.
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]