Little sstate reuse when using uninative and yocto sstate cache
Hi,
I'm on latest dunfell and using uninative and have put the yocto servers in my SSTATE_MIRRORS:
my-distro.conf:
...
Quoted 3 lines
SSTATE_MIRRORS?= "file://.* http://sstate.yoctoproject.org/3.1.11/PATH;downloadfilename=PATH"
require conf/distro/include/yocto-uninative.inc
INHERIT += "uninative"
...
but I thought I'd be getting more sstate reuse than I am:
Quoted 1 line
bitbake m4-native
...
Quoted 1 line
Sstate summary: Wanted 12 Found 1 Missed 11 Current 0 (8% match, 0% complete)
Is the above reasonable, or should I be able to pull m4-native directly from yocto servers sstate ? If so, is there a way to debug what differs between my local siginfos and the ones on the sstate server ?
Regards Jacob
Quoted 12 lines · Jacob Kroon
I'm on latest dunfell and using uninative and have put the yocto servers in my
SSTATE_MIRRORS:my-distro.conf:
...Quoted 3 lines
SSTATE_MIRRORS?= "file://.* http://sstate.yoctoproject.org/3.1.11/PATH;downloadfilename=PATH"
require conf/distro/include/yocto-uninative.inc
INHERIT += "uninative"...
but I thought I'd be getting more sstate reuse than I am:
Quoted 7 lines
Quoted 1 line
bitbake m4-native
...
Quoted 1 line
Sstate summary: Wanted 12 Found 1 Missed 11 Current 0 (8% match, 0% complete)
Is the above reasonable, or should I be able to pull m4-native directly from yocto servers sstate ? If so, is there a way to debug what differs between my local siginfos and the ones on the sstate server ?
In dunfell hash equivalence is enabled and this unfortunately means the sstate doesn't work very well unless you also have the matching hash equivalence data.
We do now have a way to have a readonly hash equivalence server and we have made one available from the autobuilder so this data is shared. Unfortunately there isn't a way to configure dunfell to use one though :(. That patch is a feature but could be something we consider backporting.
That said, I did a lot of testing/work with master recently as sstate reuse with hash equivalence was not what we'd expected. We ended up making a lot of changes, including a bit of a rewrite of the server side of the hash equivalence. I mention this just to illustrate it isn't a simple problem and that fixing this in dunfell may be tricky and would need backporting of some invasive changes. I'm trying to at least get master/honister working right in the first instance.
Cheers,
Richard
Quoted 38 lines · Richard Purdie
Quoted 12 lines · Jacob Kroon
I'm on latest dunfell and using uninative and have put the yocto servers in my
SSTATE_MIRRORS:my-distro.conf:
...Quoted 3 lines
SSTATE_MIRRORS?= "file://.* http://sstate.yoctoproject.org/3.1.11/PATH;downloadfilename=PATH"
require conf/distro/include/yocto-uninative.inc
INHERIT += "uninative"...
but I thought I'd be getting more sstate reuse than I am:
Quoted 7 lines
Quoted 1 line
bitbake m4-native
...
Quoted 1 line
Sstate summary: Wanted 12 Found 1 Missed 11 Current 0 (8% match, 0% complete)
Is the above reasonable, or should I be able to pull m4-native directly from yocto servers sstate ? If so, is there a way to debug what differs between my local siginfos and the ones on the sstate server ?
In dunfell hash equivalence is enabled and this unfortunately means the sstate doesn't work very well unless you also have the matching hash equivalence data.
We do now have a way to have a readonly hash equivalence server and we have made one available from the autobuilder so this data is shared. Unfortunately there isn't a way to configure dunfell to use one though :(. That patch is a feature but could be something we consider backporting.
That said, I did a lot of testing/work with master recently as sstate reuse with hash equivalence was not what we'd expected. We ended up making a lot of changes, including a bit of a rewrite of the server side of the hash equivalence. I mention this just to illustrate it isn't a simple problem and that fixing this in dunfell may be tricky and would need backporting of some invasive changes. I'm trying to at least get master/honister working right in the first instance.
Quoted 5 lines
Cheers,
Richard
Thanks for the detailed answer. I will focus on getting this to perform well with my master build then.
Regards
Jacob