What keeps sstate from being used?
I just had a [nearly] complete build failed (building webkitgtk after some 6000 tasks). I decided to try the 'rebuild from sstate' by removing my 'tmp' directory and rebuild. Bitbake proceeded to run 2200 setscene tasks and then [it seems] started over on the build, rebuilding the target gcc, the kernel, who knows what else... These are all things that I thought had been completed before during the initial ~6000 steps.
What could be going on and why couldn't it just pick up using sstate from where it left off?
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------
Hi,
Check:
http://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#shared-state-cache
"... the build system detects changes in the "inputs" to a given task by creating a checksum (or signature) of the task's inputs. If the checksum changes, the system assumes the inputs have changed and the task needs to be rerun. For the second question, the shared state (sstate) code tracks which tasks add which output to the build process.... "
Quoted 22 lines · Gary Thomas
I just had a [nearly] complete build failed (building webkitgtk after some 6000 tasks). I decided to try the 'rebuild from sstate' by removing my 'tmp' directory and rebuild. Bitbake proceeded to run 2200 setscene tasks and then [it seems] started over on the build, rebuilding the target gcc, the kernel, who knows what else... These are all things that I thought had been completed before during the initial ~6000 steps.
What could be going on and why couldn't it just pick up using sstate from where it left off?
--
------------------------------------------------------------
Gary Thomas | Consulting for the
MLB Associates | Embedded world
------------------------------------------------------------
--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto
Doesn't the sysroots live on tmp/sysroots? Doesn't the crosscompiler and toolchains live in sysroots? Doesn't you need crosscompilers and toolchains to compile everything else?
2016-07-06 8:51 GMT-03:00 Chris Z. <[email protected]>:
Quoted 44 lines
Hi,
Check:
http://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#shared-state-cache"... the build system detects changes in the "inputs" to a given task by creating a checksum (or signature) of the task's inputs. If the checksum changes, the system assumes the inputs have changed and the task needs to be rerun. For the second question, the shared state (sstate) code tracks which tasks add which output to the build process.... "
Quoted 12 lines · Gary Thomas
I just had a [nearly] complete build failed (building webkitgtk after some 6000 tasks). I decided to try the 'rebuild from sstate' by removing my 'tmp' directory and rebuild. Bitbake proceeded to run 2200 setscene tasks and then [it seems] started over on the build, rebuilding the target gcc, the kernel, who knows what else... These are all things that I thought had been completed before during the initial ~6000 steps.
What could be going on and why couldn't it just pick up using sstate from where it left off?
Quoted 10 lines
--
------------------------------------------------------------
Gary Thomas | Consulting for the
MLB Associates | Embedded world
------------------------------------------------------------
--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto
Signature · 2 lines
"Do or do not. There is no try" Yoda Master
Quoted 9 lines · Chris Z.
Hi,
Check:
http://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#shared-state-cache"... the build system detects changes in the "inputs" to a given task by creating a checksum (or signature) of the task's inputs. If the checksum changes, the system assumes the inputs have changed and the task needs to be rerun. For the second question, the shared state (sstate) code tracks which tasks add which output to the build process.... "
I don't see how anything could have changed between the two runs. Literally, I ran:
% bitbake target
... failed with error building webkitgtk after some 6000 tasks
% mv tmp old
% bitbake target
I traced this down to [at least] these tasks:
gthomas@europa:p0382_2016-01-13$ bitbake-diffsigs sstate-cache/66/sstate:binutils-cross-arm::2.26:r0::3:66ee1a83fd248d206fd4fd9fb33ffe26_fetch.tgz.siginfo sstate-cache/db/sstate:binutils-cross-arm::2.26:r0::3:dbdcf77671cb88d7220531471eae70a2_fetch.tgz.siginfo
basehash changed from 366d439ed05d276413ffabf7423ebe44 to acc4c0150665538a4bdd394b6b6bda13
Variable SRC_URI value changed from ' git://sourceware.org/git/binutils-gdb.git;branch=binutils-${BINUPV}-branch;protocol=git file://0002-configure-widen-the-regexp-for-SH-architectures.patch file://0003-Point-scripts-location-to-libdir.patch file://0004-Only-generate-an-RPATH-entry-if-LD_RUN_PATH-is-not-e.patch file://0005-Explicitly-link-with-libm-on-uclibc.patch file://0006-Use-libtool-2.4.patch file://0007-Add-the-armv5e-architecture-to-binutils.patch file://0008-don-t-let-the-distro-compiler-point-to-the-wrong-ins.patch file://0009-warn-for-uses-of-system-directories-when-cross-linki.patch file://0010-Fix-rpath-in-libtool-when-sysroot-is-enabled.patch file://0011-Change-default-emulation-for-mips64-linux.patch file://0012-Add-support-for-Netlogic-XLP.patch file://0013-Fix-GOT-address-computations-in-initial-PLT-entries-.patch file://0014-Correct-nios2-_gp-address-computation.patch file://0015-fix-the-incorrect-assembling-for-ppc-wait-mnemonic.patch file://MIPS-GAS-Fix-an-ISA-override-not-lifting-ABI-restrictions.patch ' to ' git://sourceware.org/git/binutils-gdb.git;branch=binutils-${BINUPV}-branch;protocol=git file://0002-configure-widen-the-regexp-for-SH-architectures.patch file://0003-Point-scripts-location-to-libdir.patch file://0004-Only-generate-an-RPATH-entry-if-LD_RUN_PATH-is-not-e.patch file://0005-Explicitly-link-with-libm-on-uclibc.patch file://0006-Use-libtool-2.4.patch file://0007-Add-the-armv5e-architecture-to-binutils.patch file://0008-don-t-let-the-distro-compiler-point-to-the-wrong-ins.patch file://0009-warn-for-uses-of-system-directories-when-cross-linki.patch file://0010-Fix-rpath-in-libtool-when-sysroot-is-enabled.patch file://0011-Change-default-emulation-for-mips64-linux.patch file://0012-Add-support-for-Netlogic-XLP.patch file://0013-Fix-GOT-address-computations-in-initial-PLT-entries-.patch file://0014-Correct-nios2-_gp-address-computation.patch file://0015-fix-the-incorrect-assembling-for-ppc-wait-mnemonic.patch '
Dependency on checksum of file MIPS-GAS-Fix-an-ISA-override-not-lifting-ABI-restrictions.patch was removedgthomas@europa:p0382_2016-01-13$ ls -l sstate-cache/66/sstate:binutils-cross-arm::2.26:r0::3:66ee1a83fd248d206fd4fd9fb33ffe26_fetch.tgz.siginfo sstate-cache/db/sstate:binutils-cross-arm::2.26:r0::3:dbdcf77671cb88d7220531471eae70a2_fetch.tgz.siginfo -rw-rw-r-- 1 gthomas gthomas 3966 Jul 6 09:16 sstate-cache/66/sstate:binutils-cross-arm::2.26:r0::3:66ee1a83fd248d206fd4fd9fb33ffe26_fetch.tgz.siginfo -rw-rw-r-- 1 gthomas gthomas 3787 Jun 28 05:55 sstate-cache/db/sstate:binutils-cross-arm::2.26:r0::3:dbdcf77671cb88d7220531471eae70a2_fetch.tgz.siginfo
Notice that the timestamp from today was 09:16. I had updated my Poky/Yocto repo at 08:04 today and then immediately made the first run (seen above). When I started the second run it was 11:57, and binutils-cross-arm was created (I think from setscene) at 12:00. So why could these hashes have caused all those other ripples (which took until 14:15 to finish).
I also saw some very weird behaviour in all of this. I noticed that my virtual/kernel was totally rebuilt, but the image that's in the deploy directory came from the setscene task, i.e. the first run :-(
None of this seems quite right to me and I'd really like to understand it and, if needed, help figure out how to make it correct (and robust).
I'll save the total state of everything in case I can provide more data...
Quoted 12 lines · On Wed, Jul 6, 2016 at 12:10 PM, Gary Thomas <[email protected]…
I just had a [nearly] complete build failed (building webkitgtk after some 6000 tasks). I decided to try the 'rebuild from sstate' by removing my 'tmp' directory and rebuild. Bitbake proceeded to run 2200 setscene tasks and then [it seems] started over on the build, rebuilding the target gcc, the kernel, who knows what else... These are all things that I thought had been completed before during the initial ~6000 steps.
What could be going on and why couldn't it just pick up using sstate from where it left off?
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------
Quoted 3 lines · Gary Thomas
Dependency on checksum of file MIPS-GAS-Fix-an-ISA-override-not-lifting-ABI-restrictions.patch was removed
As this show binutils changing its SRC_URI this suggests that you did something in between the two runs, or those hashes are not the relevant pair.
Ross
Quoted 8 lines · Ross
On 6 July 2016 at 15:23, Gary Thomas <[email protected] <mailto:[email protected]>> wrote:
Dependency on checksum of file MIPS-GAS-Fix-an-ISA-override-not-lifting-ABI-restrictions.patch was removed
As this show binutils changing its
SRC_URIthis suggests that you did something in between the two runs, or those hashes are not the relevant pair.
I could send the whole trace of how I got there, but basically I noticed that 'patch' was being rebuilt for the target, so I tracked that back to gcc-cross-arm which led me to binutils-cross-arm
Maybe I've done something wrong or don't totally understand the output of the tools, but it's clear that something is a little off here as my original build was 99% complete (only a dozen or so tasks remaining) and when I reran it from sstate, it more or less started over.
If you have pointers on how I can diagnose this, or at least retrace it, I'm all ears as I really want to see this work the way it's advertised "on the tin"
Ideas/suggestions more than welcome
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------
Hi,
Gary what poky version you are using ?
It seems that you probably updated somehow local workspace recipe for binutils from poky master upstream. 5 days ago binutils recipe was updated with new patch.
Do you have some scripts or cron to update recipes and check that they are building with newer poky ?
Quoted 37 lines · Gary Thomas
Quoted 4 lines · Ross
On 6 July 2016 at 15:23, Gary Thomas <[email protected] <mailto:
[email protected]>> wrote:Quoted 8 lines
Dependency on checksum of file
MIPS-GAS-Fix-an-ISA-override-not-lifting-ABI-restrictions.patch was removedAs this show binutils changing its
SRC_URIthis suggests that you did something in between the two runs, or those hashes
are not the relevant pair.I could send the whole trace of how I got there, but basically I noticed that 'patch' was being rebuilt for the target, so I tracked that back to gcc-cross-arm which led me to binutils-cross-arm
Maybe I've done something wrong or don't totally understand the output of the tools, but it's clear that something is a little off here as my original build was 99% complete (only a dozen or so tasks remaining) and when I reran it from sstate, it more or less started over.
If you have pointers on how I can diagnose this, or at least retrace it, I'm all ears as I really want to see this work the way it's advertised "on the tin"
Ideas/suggestions more than welcome
--
------------------------------------------------------------
Gary Thomas | Consulting for the
MLB Associates | Embedded world
------------------------------------------------------------