sstate improvements
cover sstate improvements
Include native/cross/nativesdk deps in target signatures to make sstate reuse a bit safer.
Make allarch a bit more independent on TARGET_* variables.
The following changes since commit 8163854adf87ac42a8f08ee25685d0ce1efb4724:
oe-selftest: separated the SStateBase and SStateTests in different modules (2014-01-16 12:18:44 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib jansa/sstate
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=jansa/sstate
Martin Jansa (2):
sstatesig: include native/cross/nativesdk deps in target signatures
allarch: Set empty TARGET_PREFIX and TARGET_FPU
meta/classes/allarch.bbclass | 2 ++ meta/lib/oe/sstatesig.py | 4 ---- 2 files changed, 2 insertions(+), 4 deletions(-)
Signature · 1 line
1.8.5.2
PATCH 1/2 sstatesig: include native/cross/nativesdk deps in target signatures
* I don't have any real evidence or good statistics for this, but when comparing signature dumps from my big bitbake world builds I usually see a lot of rebuilds caused by changes in .bbclasses and only very rare would be the case where oe-core upgrade brings changes in -native recipes and no change in .bbclasses used from target recipes
* changing the default to include them shouldn't cause significant increase in rebuilds and sstate reuse a bit safer
* people working on toolchain (e.g. using gcc from AUTOREV) can easily extend sstate_rundepfilter to ignore them again (it's easier than removing existing filter), example how add own signature handler in your layer is here:
https://github.com/openwebos/meta-webos/commit/9ac3a7c803e7793b3274e4998f167b6278db8042
--- meta/lib/oe/sstatesig.py | 4 ---- 1 file changed, 4 deletions(-)
meta/lib/oe/sstatesig.py +0 -4
@@ -36,10 +36,6 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache): if isPackageGroup(fn): return False - # Drop native/cross/nativesdk dependencies from target recipes - if isNative(depname) or isCross(depname) or isNativeSDK(depname): - return False - # Exclude well defined machine specific configurations which don't change ABI if depname in siggen.abisaferecipes and not isImage(fn): return False
Signature · 1 line
1.8.5.2
Re: [PATCH 1/2] sstatesig: include native/cross/nativesdk deps in target signatures
Quoted 13 lines · Martin Jansa
* I don't have any real evidence or good statistics for this, but when comparing signature dumps from my big bitbake world builds I usually see a lot of rebuilds caused by changes in .bbclasses and only very rare would be the case where oe-core upgrade brings changes in -native recipes and no change in .bbclasses used from target recipes
* changing the default to include them shouldn't cause significant increase in rebuilds and sstate reuse a bit safer
* people working on toolchain (e.g. using gcc from AUTOREV) can easily extendsstate_rundepfilterto ignore them again (it's easier than removing existing filter), example how add own signature handler in your layer is here:
https://github.com/openwebos/meta-webos/commit/9ac3a7c803e7793b3274e4998f167b6278db8042
Quoted 4 lines
Signed-off-by Martin Jansa <[email protected]>--- meta/lib/oe/sstatesig.py | 4 ---- 1 file changed, 4 deletions(-)
Whilst I merged this, we do have a problem here. This change means that the sstate is locked to the build system architecture.
For example, if you have a 32 bit machine, it can't use any of the sstate built on a 64 bit machine and vice versa.
I'm not sure what we do about this but its a problem...
Cheers,
Richard
Quoted 15 lines
meta/lib/oe/sstatesig.py +0
-4@@ -36,10 +36,6 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache): if isPackageGroup(fn): return False - # Drop native/cross/nativesdk dependencies from target recipes - if isNative(depname) or isCross(depname) or isNativeSDK(depname): - return False - # Exclude well defined machine specific configurations which don't change ABI if depname in siggen.abisaferecipes and not isImage(fn): return False
Re: [PATCH 1/2] sstatesig: include native/cross/nativesdk deps in target signatures
Quoted 19 lines · Richard Purdie
Quoted 13 lines · Martin Jansa
* I don't have any real evidence or good statistics for this, but when comparing signature dumps from my big bitbake world builds I usually see a lot of rebuilds caused by changes in .bbclasses and only very rare would be the case where oe-core upgrade brings changes in -native recipes and no change in .bbclasses used from target recipes
* changing the default to include them shouldn't cause significant increase in rebuilds and sstate reuse a bit safer
* people working on toolchain (e.g. using gcc from AUTOREV) can easily extendsstate_rundepfilterto ignore them again (it's easier than removing existing filter), example how add own signature handler in your layer is here:
https://github.com/openwebos/meta-webos/commit/9ac3a7c803e7793b3274e4998f167b6278db8042Quoted 4 lines
Signed-off-by Martin Jansa <[email protected]>--- meta/lib/oe/sstatesig.py | 4 ---- 1 file changed, 4 deletions(-)
Quoted 7 lines
Whilst I merged this, we do have a problem here. This change means that the sstate is locked to the build system architecture.
For example, if you have a 32 bit machine, it can't use any of the sstate built on a 64 bit machine and vice versa.
I'm not sure what we do about this but its a problem...
Agreed, that's bad, I was testing it in setup where we publish the sstate archive from the builder with the same architecture and have similar setup building just -native recipes on 32bit for 32bit devs.
So this problem should have hit us as well, I'll recheck.
Quoted 21 lines
Cheers,
Richard
Quoted 15 lines
meta/lib/oe/sstatesig.py +0
-4@@ -36,10 +36,6 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache): if isPackageGroup(fn): return False - # Drop native/cross/nativesdk dependencies from target recipes - if isNative(depname) or isCross(depname) or isNativeSDK(depname): - return False - # Exclude well defined machine specific configurations which don't change ABI if depname in siggen.abisaferecipes and not isImage(fn): return False
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]
PATCH 2/2 allarch: Set empty TARGET_PREFIX and TARGET_FPU
In reply to Martin Jansa* set empty TARGET_PREFIX
This has a bit weird reason caused by unsupported setup where external-toolchain is used in some DISTRO only for some MACHINEs and internal is used for other MACHINEs.
Because external-toolchain usually comes with different TARGET_PREFIX it was causing allarch recipes to have different signatures even when they don't use toolchain at all.
Empty TARGET_PREFIX also helps to find allarch recipes which still have default dependency on e.g. virtual/${TARGET_PREFIX}gcc.
* add TARGET_FPU just for completeness (it was used in icecc.bbclass but now it's vardepexcluded there as well)
--- meta/classes/allarch.bbclass | 2 ++ 1 file changed, 2 insertions(+)
meta/classes/allarch.bbclass +2 -0
@@ -22,6 +22,8 @@ python () { d.setVar("TARGET_CC_ARCH", "none") d.setVar("TARGET_LD_ARCH", "none") d.setVar("TARGET_AS_ARCH", "none") + d.setVar("TARGET_FPU", "") + d.setVar("TARGET_PREFIX", "") d.setVar("PACKAGE_EXTRA_ARCHS", "") d.setVar("SDK_ARCH", "none") d.setVar("SDK_CC_ARCH", "none")
Signature · 1 line
1.8.5.2