Problem with RDEPENDS in multimachine builds for allarch packages
Hi,
I recently ran into a problem that I described here:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8578 and while debugging this issue an other problem came to light.
I have a multimachine configuration (armv5/imx6) and the same distro for both of them (poky 2.0 RC based).
I have some allarch packages which have no DEPENDS, however which have an RDEPENDS_${PN} setting.
I can see that even though there were no changes to the recipe itself and also no changes to the recipes listed in RDEPENDS, my recipe still gets rebuilt each time.
What we tried:
<bluelightning> so cleansstate the recipe, change machine, cleansstate again,
then build it, then change machine, build it again and note the first task that executes
The logs for these steps, one with RDEPENDS and one without are at the end of this mail. I can see that with present RDEPENDS "real" tasks get executed even though no changes occured in the recipe; this is not the case when I drop RDEPENDS.
Is this a bug or is this just something that needs to be documented? What is the "desired" behaviour?
Kind regards,
Sergey
What I get with RDEPENDS is:
***********
NOTE: Running setscene task 347 of 374 (/oe/dss-oe/yocto/dS/meta-digitalstrom-devel/recipes-digitalstrom/dss11-help/dss11-installation-manual-tr-tr_0.9.bb, do_populate_sysroot_setscene)
NOTE: Running setscene task 350 of 374 (/oe/dss-oe/yocto/dS/meta-digitalstrom-devel/recipes-digitalstrom/dss11-help/dss11-installation-manual-tr-tr_0.9.bb, do_packagedata_setscene)
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_populate_sysroot_setscene: Started
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_packagedata_setscene: Started
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_populate_sysroot_setscene: Succeeded
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_packagedata_setscene: Succeeded
NOTE: Running setscene task 352 of 374 (/oe/dss-oe/yocto/dS/meta-digitalstrom-devel/recipes-digitalstrom/dss11-help/dss11-installation-manual-tr-tr_0.9.bb, do_package_setscene)
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_package_setscene: Started
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_package_setscene: Succeeded
NOTE: Running task 1175 of 1181 (ID: 14, /oe/dss-oe/yocto/dS/meta-digitalstrom-devel/recipes-digitalstrom/dss11-help/dss11-installation-manual-tr-tr_0.9.bb, do_package_write_ipk)
NOTE: Running task 1176 of 1181 (ID: 9, /oe/dss-oe/yocto/dS/meta-digitalstrom-devel/recipes-digitalstrom/dss11-help/dss11-installation-manual-tr-tr_0.9.bb, do_package_qa)
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_package_write_ipk: Started
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_package_qa: Started
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_package_qa: Succeeded
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_package_write_ipk: Succeeded
NOTE: Running noexec task 1179 of 1181 (ID: 10, /oe/dss-oe/yocto/dS/meta-digitalstrom-devel/recipes-digitalstrom/dss11-help/dss11-installation-manual-tr-tr_0.9.bb, do_build)
NOTE: Running task 1180 of 1181 (ID: 11, /oe/dss-oe/yocto/dS/meta-digitalstrom-devel/recipes-digitalstrom/dss11-help/dss11-installation-manual-tr-tr_0.9.bb, do_rm_work)
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_rm_work: Started
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_rm_work: Succeeded
NOTE: Running task 1181 of 1181 (ID: 12, /oe/dss-oe/yocto/dS/meta-digitalstrom-devel/recipes-digitalstrom/dss11-help/dss11-installation-manual-tr-tr_0.9.bb, do_rm_work_all)
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_rm_work_all: Started
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_rm_work_all: Succeeded
***********
We can see that some "real" tasks are getting executed (do_package_write_ipk).
Now, when I remove the RDEPENDS from my recipe the log gets much shorter:
***********
NOTE: Running setscene task 73 of 76 (/oe/dss-oe/yocto/dS/meta-digitalstrom-devel/recipes-digitalstrom/dss11-help/dss11-installation-manual-tr-tr_0.9.bb, do_populate_sysroot_setscene)
NOTE: Running setscene task 74 of 76 (/oe/dss-oe/yocto/dS/meta-digitalstrom-devel/recipes-digitalstrom/dss11-help/dss11-installation-manual-tr-tr_0.9.bb, do_package_write_ipk_setscene)
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_populate_sysroot_setscene: Started
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_package_write_ipk_setscene: Started
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_populate_sysroot_setscene: Succeeded
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_package_write_ipk_setscene: Succeeded
NOTE: Running setscene task 75 of 76 (/oe/dss-oe/yocto/dS/meta-digitalstrom-devel/recipes-digitalstrom/dss11-help/dss11-installation-manual-tr-tr_0.9.bb, do_packagedata_setscene)
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_packagedata_setscene: Started
NOTE: recipe dss11-installation-manual-tr-tr-0.9-r2: task do_packagedata_setscene: Succeeded
***********
Here no "real" tasks get executed.
On Mon, Oct 26, 2015 at 8:13 AM, Sergey 'Jin' Bostandzhyan <[email protected]
Quoted 24 lines
wrote:
I recently ran into a problem that I described here:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8578 and while debugging this issue an other problem came to light.I have a multimachine configuration (armv5/imx6) and the same distro for both
of them (poky 2.0 RC based).I have some allarch packages which have no DEPENDS, however which have an RDEPENDS_${PN} setting.
I can see that even though there were no changes to the recipe itself and also no changes to the recipes listed in RDEPENDS, my recipe still gets rebuilt each time.
What we tried:
<bluelightning> so cleansstate the recipe, change machine, cleansstate again,
then build it, then change machine, build it again and note the first task that executes
Quoted 9 lines
The logs for these steps, one with RDEPENDS and one without are at the end of
this mail. I can see that with present RDEPENDS "real" tasks get executed even though no changes occured in the recipe; this is not the case when I drop RDEPENDS.Is this a bug or is this just something that needs to be documented? What is the "desired" behaviour?
Afaik the most likely cause is your rdepends isn't allarch, so is being rebuilt for the new machine, which causes yours to be rebuilt since it depends on it.
Signature · 5 lines
Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics
Quoted 2 lines · Christopher Larson
Afaik the most likely cause is your rdepends isn't allarch, so is being rebuilt for the new machine, which causes yours to be rebuilt since it depends on it.
So, that means that anything arch specific listed in RDEPENDS will always cause the package to rebuild even if there were no changes in the package itself nor in the RDEPENDS packages? Is this really the desired behaviour?
I'd expect that if nothing has changed, nothing should get rebuild.. I often saw a huge number of packages being rebuilt and could never explain why this was happening, now it's becoming clear that probably some of them had some RDEPENDS set.
If you say it's not a bug - ok, now I at least know what's happening. Still, imho it would be nice and it would speed up things if only stuff that has really changed or stuff that depends on other things that have really changed gets triggered.
Kind regards,
Sergey
Quoted 14 lines · Sergey 'Jin' Bostandzhyan
Quoted 2 lines · Christopher Larson
Afaik the most likely cause is your rdepends isn't allarch, so is being rebuilt for the new machine, which causes yours to be rebuilt since it depends on it.
So, that means that anything arch specific listed in RDEPENDS will always cause the package to rebuild even if there were no changes in the package itself nor in the RDEPENDS packages? Is this really the desired behaviour?
I'd expect that if nothing has changed, nothing should get rebuild.. I often saw a huge number of packages being rebuilt and could never explain why this was happening, now it's becoming clear that probably some of them had some RDEPENDS set.
Quoted 4 lines
If you say it's not a bug - ok, now I at least know what's happening. Still, imho it would be nice and it would speed up things if only stuff that has really changed or stuff that depends on other things that have really changed gets triggered.
You can use openembedded-core/scripts/sstate-diff-machines.sh
to compare the signatures. It's also useful to store the output directories in case you see some unexpected rebuilds in some future build.
This script is used to generate reports like this one:
http://lists.openembedded.org/pipermail/openembedded-core/2015-October/111554.html
Regards,
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]
Quoted 22 lines · Sergey 'Jin' Bostandzhyan
Hi,
I recently ran into a problem that I described here:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8578 and while debugging this issue an other problem came to light.I have a multimachine configuration (armv5/imx6) and the same distro for both of them (poky 2.0 RC based).
I have some allarch packages which have no DEPENDS, however which have an RDEPENDS_${PN} setting.
I can see that even though there were no changes to the recipe itself and also no changes to the recipes listed in RDEPENDS, my recipe still gets rebuilt each time.
What we tried:
<bluelightning> so cleansstate the recipe, change machine, cleansstate again,
then build it, then change machine, build it again and note the first task that executes
Quoted 6 lines
The logs for these steps, one with RDEPENDS and one without are at the end of this mail. I can see that with present RDEPENDS "real" tasks get executed even though no changes occured in the recipe; this is not the case when I drop RDEPENDS.
Is this a bug or is this just something that needs to be documented?
I think it's the same problem as described here:
http://thread.gmane.org/gmane.comp.handhelds.openembedded.core/70070/focus=70072
Quoted 1 line
What is the "desired" behaviour?
Apparently, this needless rebuilding is "desired".
Kind regards,
Mike Looijmans
System Expert
TOPIC Embedded Products Eindhovenseweg 32-C, NL-5683 KH Best Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
Telefax: +31 (0) 499 33 69 70
E-mail: [email protected]
Website: www.topicproducts.com
Please consider the environment before printing this e-mail
Visit us at : Aerospace Electrical Systems Expo Europe which will be held from 17.11.2015 till 19.11.2015, Findorffstrasse 101 Bremen, Germany, Hall 5, stand number C65
http://www.aesexpo.eu
Quoted 28 lines · Mike Looijmans
Quoted 27 lines · Sergey 'Jin' Bostandzhyan
Hi,
I recently ran into a problem that I described here:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8578 and while debugging this issue an other problem came to light.I have a multimachine configuration (armv5/imx6) and the same distro for both of them (poky 2.0 RC based).
I have some allarch packages which have no DEPENDS, however which have an RDEPENDS_${PN} setting.
I can see that even though there were no changes to the recipe itself and also no changes to the recipes listed in RDEPENDS, my recipe still gets rebuilt each time.
What we tried:
<bluelightning> so cleansstate the recipe, change machine, cleansstate again,
then build it, then change machine, build it again and note the first task that executesThe logs for these steps, one with RDEPENDS and one without are at the end of this mail. I can see that with present RDEPENDS "real" tasks get executed even though no changes occured in the recipe; this is not the case when I drop RDEPENDS.
Quoted 8 lines
Quoted 1 line
Is this a bug or is this just something that needs to be documented?
I think it's the same problem as described here:
http://thread.gmane.org/gmane.comp.handhelds.openembedded.core/70070/focus=70072Quoted 1 line
What is the "desired" behaviour?
Apparently, this needless rebuilding is "desired".
Its more that we can't tell the difference between a case where it is needless and a case where it is no. The system therefore rebuilds as if it didn't, there would be cases where it should have rebuilt and did not.
Cheers,
Richard
Quoted 34 lines · Richard Purdie
Quoted 33 lines · Mike Looijmans
Quoted 22 lines · Sergey 'Jin' Bostandzhyan
Hi,
I recently ran into a problem that I described here:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8578 and while debugging this issue an other problem came to light.I have a multimachine configuration (armv5/imx6) and the same distro for both of them (poky 2.0 RC based).
I have some allarch packages which have no DEPENDS, however which have an RDEPENDS_${PN} setting.
I can see that even though there were no changes to the recipe itself and also no changes to the recipes listed in RDEPENDS, my recipe still gets rebuilt each time.
What we tried:
<bluelightning> so cleansstate the recipe, change machine, cleansstate again,
then build it, then change machine, build it again and note the first task that executesQuoted 6 lines
The logs for these steps, one with RDEPENDS and one without are at the end of this mail. I can see that with present RDEPENDS "real" tasks get executed even though no changes occured in the recipe; this is not the case when I drop RDEPENDS.
Is this a bug or is this just something that needs to be documented?
I think it's the same problem as described here:
http://thread.gmane.org/gmane.comp.handhelds.openembedded.core/70070/focus=70072
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
What is the "desired" behaviour?
Apparently, this needless rebuilding is "desired".
Its more that we can't tell the difference between a case where it is needless and a case where it is no. The system therefore rebuilds as if it didn't, there would be cases where it should have rebuilt and did not.
But that would not solve any issue at all.
Whatever package got built last will end up in the feed, under the "all" architecture. So if machine X and Y have different versions of some package P, and the build for Y ran last, machine X will still install Y's version of P.
Kind regards,
Mike Looijmans
System Expert
TOPIC Embedded Products Eindhovenseweg 32-C, NL-5683 KH Best Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
Telefax: +31 (0) 499 33 69 70
E-mail: [email protected]
Website: www.topicproducts.com
Please consider the environment before printing this e-mail
Visit us at : Aerospace Electrical Systems Expo Europe which will be held from 17.11.2015 till 19.11.2015, Findorffstrasse 101 Bremen, Germany, Hall 5, stand number C65
http://www.aesexpo.eu