Skip to content

Add support for locked down sstate

Hongxu Jia openembedded-core 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest5 patches 4 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/5Richard Purdie : sstatesig/sstate: Add support for locked down sstate cache usagemeta/classes/sstate.bbclass meta/lib/oe/sstatesig.py scripts/gen-lockedsig-cache+117 -00 replies
2/5Richard Purdie : sstatesig: Improve to handle locking of multiple machinesmeta/lib/oe/sstatesig.py+28 -130 replies
3/5Richard Purdie : siggen/runqueue/bitbake-worker: Improve siggen data transfer interfacebitbake/bin/bitbake-worker bitbake/lib/bb/runqueue.py bitbake/lib/bb/siggen.py+7 -80 replies
4/5sstate/sstatesig: optimize the support for locked down sstate cache usagemeta/classes/sstate.bbclass meta/lib/oe/sstatesig.py+41 -162 replies
5/5sstatesig: incremental dump lockedsigsmeta/lib/oe/sstatesig.py+45 -222 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 0/5] Add support for locked down sstate

Patches 4/5, 3/5 are mine, based on Richard's 1/5, 2/5, 3/5.

Test Steps:

1. Prepare a locked sstat-cache by dumping lockedsigs file
1) vim local.conf
...
SIGGEN_DUMP_LOCKEDSIGS = "1" SIGGEN_LOCKEDSIGS_CONFIG = "/path/to/locked-sigs.inc" SSTATE_DIR ?= "/path/to/sstate-cache"
...

2) Build db-native from a clean environment

$ bitbake db-native

3) Back up lockedsigs file

$ mv /path/to/locked-sigs.inc  /path/to/locked-sigs.inc-back

4) Incrementally build ldconfig-native

$ vim local.conf
...
require ${SIGGEN_LOCKEDSIGS_CONFIG}
...
$ bitbake ldconfig-native

5) Check incrementally dumping lockedsiged file also

$ vimdiff /path/to/locked-sigs.inc-back /path/to/locked-sigs.inc

2. Create a new build which uses previous locked sstat-cache
1) vim local.conf
...
SSTATE_DIR ?= "/path/to/sstate-cache" SIGGEN_LOCKEDSIGS_CONFIG = "/path/to/locked-sigs.inc" require ${SIGGEN_LOCKEDSIGS_CONFIG} SIGGEN_LOCKEDSIGS_CHECK_LEVEL = "error"
...

2) Build db-native

$ bitbake db-native

3) Tweak db's do_configure task by adding comments

meta/recipes-support/db/db_6.0.30.bb +1 -0
@@ -82,6 +82,7 @@ do_configure() {
        gnu-configize --force ${S}
        export STRIP="true"
        oe_runconf
+       echo "hello"
 }
 

4) Build db-native and abort

$ bitbake db-native
...
ERROR: The db-native:do_configure sig (00546ab0e73b3afd50efc03f26e5f040) changed, use locked sig 9d8f2210506446e76d012d1ebd49a021 to instead
The db-native:do_compile sig (4442f979d97bdbed71a8fecbbbe41ed3) changed, use locked sig d29a7c568d10859176d733773433fe92 to instead
The db-native:do_install sig (562cb98d8eb25a7c79ded6a5359c56c1) changed, use locked sig 1e917d8d619c58987a1b2fe286e9288c to instead
The db-native:do_populate_sysroot sig (91627e36da9ac8fe489f36b67a87ed2e) changed, use locked sig f780d45dbf680e40a871820327c4c56c to instead
The db-native:do_build sig (ba289cba4a3bfda0c07aed9868fdcd5d) changed, use locked sig d9b84e5fb30011fd31bc0e7d7652fead to instead
...

5) Remove db-native sstate-cache

$ bitbake db-native -ccleansstate

6) Build db-native and abort

$ bitbake db-native
...
Locked sig is set for db-native:do_populate_sysroot (f780d45dbf680e40a871820327c4c56c) yet not in sstate cache?
...

7) Do not break the build
vim local.conf
...
SIGGEN_LOCKEDSIGS_CHECK_LEVEL = "warn"
...

8) Build db-native

$ bitbake db-native
...
WARNING Locked sig is set for db-native:do_populate_sysroot (f780d45dbf680e40a871820327c4c56c) yet not in sstate cache?
...

//Hongxu

The following changes since commit bd1a3ab8a63d6b2c109d595c67fcca99817e1daa:

nspr: Upgrade to 4.10.7 (2014-09-10 15:38:57 +0100)

are available in the git repository at:

git://git.pokylinux.org/poky-contrib hongxu/locked-sstate
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=hongxu/locked-sstate

Hongxu Jia (2):
sstate/sstatesig: optimize the support for locked down sstate cache
usage
sstatesig: incremental dump lockedsigs

Richard Purdie (1):
siggen/runqueue/bitbake-worker: Improve siggen data transfer interface

[email protected] (2):
Richard Purdie : sstatesig/sstate: Add support for locked down sstate
cache usage
Richard Purdie : sstatesig: Improve to handle locking of multiple
machines

 bitbake/bin/bitbake-worker  |   2 +-
 bitbake/lib/bb/runqueue.py  |   4 +-
 bitbake/lib/bb/siggen.py    |   9 ++--
 meta/classes/sstate.bbclass |  17 ++++++
 meta/lib/oe/sstatesig.py    | 123 ++++++++++++++++++++++++++++++++++++++++++++
 scripts/gen-lockedsig-cache |  40 ++++++++++++++
 6 files changed, 187 insertions(+), 8 deletions(-)
 create mode 100755 scripts/gen-lockedsig-cache
Signature · 1 line
1.9.1