Figuring out why sstate-cache is no longer used for a package
I've been struggling with this for a few days.
We have a build server that build various images overnight. One of the packages in that image is "fpga-image", which takes more than an hour to build.
We have been sharing the the build server's sstate-cache via HTTP and this has worked excellently up until yesterday.
The current situation is that a client will grab everything from the buildserver's HTTP sstate-cache, potentially finishing a build from scratch in about five minutes. However, for some reason, the fpga-image does not fall into this category, and eache machine insists on re-building it from scratch. I've been trying to debug this, but the sstate-cache is on another machine. I tried copying part of the build server's sstate-cache onto my machine, but that only results in "bitbake-diffsigs -t fpga-image .." yielding "ERROR: No sigdata files found matching fpga-image .." so that apparently is a dead end.
How can I determine what is causing the system to think that it needs to rebuild this package?
And/Or how do tell the system that this package only depends on the SRCREV and MACHINE and nothing else? (I've already set INHIBIT_DEFAULT_DEPS = "1", is there more that I can do?)
--
Mike Looijmans - TOPIC Automation
Quoted 9 lines · Mike Looijmans
The current situation is that a client will grab everything from the buildserver's HTTP sstate-cache, potentially finishing a build from scratch in about five minutes. However, for some reason, the fpga-image does not fall into this category, and eache machine insists on re-building it from scratch. I've been trying to debug this, but the sstate-cache is on another machine. I tried copying part of the build server's sstate-cache onto my machine, but that only results in "bitbake-diffsigs -t fpga-image .." yielding "ERROR: No sigdata files found matching fpga-image .." so that apparently is a dead end.
The trick is to use bitbake -S to write stamp data for a particular image and copy that around. The script sstate-diff-machines can help here.
Note that there's been some sstate fallout from the recent
optimisations: it was discovered yesterday that sstate hashes using the "contains()" optimisation wasn't reliable so could produce different hashes on different machines. Try oe-core master from today and see if that's any better: it certainly is for me.
Ross
Quoted 17 lines · Mike Looijmans
I've been struggling with this for a few days.
We have a build server that build various images overnight. One of the packages in that image is "fpga-image", which takes more than an hour to build.
We have been sharing the the build server's sstate-cache via HTTP and this has worked excellently up until yesterday.
The current situation is that a client will grab everything from the buildserver's HTTP sstate-cache, potentially finishing a build from scratch in about five minutes. However, for some reason, the fpga-image does not fall into this category, and eache machine insists on re-building it from scratch. I've been trying to debug this, but the sstate-cache is on another machine. I tried copying part of the build server's sstate-cache onto my machine, but that only results in "bitbake-diffsigs -t fpga-image .." yielding "ERROR: No sigdata files found matching fpga-image .." so that apparently is a dead end.
Quoted 2 lines
How can I determine what is causing the system to think that it needs to rebuild this package?
I use this script:openembedded-core/scripts/sstate-diff-machines.sh
to create "backup" of sstate signatures and when some next build unexpectedly doesn't reuse some sstate packages I use the script again and compare new and old signatures to see why.
As bonus each build which populates out SSTATE_MIRROR also creates tarball with these signatures, so I can easily download one tarball and compare what and why won't be reused from sstate.
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]
Quoted 33 lines · Martin Jansa
Quoted 4 lines · Mike Looijmans
I've been struggling with this for a few days.
We have a build server that build various images overnight. One of the packages in that image is "fpga-image", which takes more than an hour to
build.
Quoted 2 lines
We have been sharing the the build server's sstate-cache via HTTP and
this has
Quoted 4 lines
worked excellently up until yesterday.
The current situation is that a client will grab everything from the buildserver's HTTP sstate-cache, potentially finishing a build from
scratch in
Quoted 1 line
about five minutes. However, for some reason, the fpga-image does not
fall
Quoted 1 line
into this category, and eache machine insists on re-building it from
scratch.
Quoted 1 line
I've been trying to debug this, but the sstate-cache is on another
machine. I
Quoted 1 line
tried copying part of the build server's sstate-cache onto my machine,
but
Quoted 1 line
that only results in "bitbake-diffsigs -t fpga-image .." yielding
"ERROR: No
Quoted 1 line
sigdata files found matching fpga-image .." so that apparently is a dead
end.
Quoted 3 lines
How can I determine what is causing the system to think that it needs to rebuild this package?
I use this script:
openembedded-core/scripts/sstate-diff-machines.sh
Quoted 7 lines
to create "backup" of sstate signatures and when some next build unexpectedly doesn't reuse some sstate packages I use the script again and compare new and old signatures to see why.
As bonus each build which populates out
SSTATE_MIRRORalso creates tarball with these signatures, so I can easily download one tarball and compare what and why won't be reused from sstate.
You can also directly use bitbake-whatchanged -v <target>, with STAMPS_DIR set to the automated build’s stamps directory w/ sstates included.
Signature · 5 lines
Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics