Richard Purdie : bitbake.conf/sanity: Separate versions and PN stamp components into separate directories for WORKDIR and STAMP

oe-commits Martin Jansa 2 participants 2 messages Nov 20, 2012 – Nov 23, 2012 1 message not in the archive

lore

Share & tools

lore

A message that is not in the archive (sent off-list or not yet received): 20121120152957.729DC1034D@opal

Martin Jansa

author

2012-11-20 19:40 UTC #1 lore raw

Quoted 31 lines · On Tue, Nov 20, 2012 at 03:29:56PM +0000, [email protected] wrote:

Module: openembedded-core.git
Branch: master
Commit: 05075cf3138d1c61f5cf4fe0e1a4587acc00c692
URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=05075cf3138d1c61f5cf4fe0e1a4587acc00c692
Author: Richard Purdie <[email protected]>
Date:   Fri Nov 16 15:35:53 2012 +0000
bitbake.conf/sanity: Separate versions and PN stamp components into separate directories for WORKDIR and STAMP
This means some of the hacks we have to tell where the package name ends and the version starts in the directory layout becomes obsolete, simplifying the work of some of the cleanup scripts. It also makes the layout slightly more intuitive to the user.
It does force a rebuild onto the user but it will reuse sstate successfully.
Signed-off-by Richard Purdie <[email protected]>
---
 meta/classes/sanity.bbclass |    2 ++
 meta/conf/abi_version.conf  |    2 +-
 meta/conf/bitbake.conf      |    6 +++---
 3 files changed, 6 insertions(+), 4 deletions(-)
meta/classes/sanity.bbclass +0 -0
index 3806bc5..5b4f070 100644

meta/classes/sanity.bbclass def check_sanity(sanity_data): · line 609

Quoted 5 lines

meta/classes/sanity.bbclass +2 -0

@@ -605,6 +605,8 @@ def check_sanity(sanity_data):
             f.write(current_abi)
         elif abi == "7" and current_abi == "8":
             messages = messages + "Your configuration is using stamp files including the sstate hash but your build directory was built with stamp files that do not include this.\nTo continue, either rebuild or switch back to the OEBasic signature handler with BB_SIGNATURE_HANDLER = 'OEBasic'.\n"
+        elif (abi != current_abi and current_abi == "9"):
+            messages = messages + "The layout of the TMPDIR STAMPS directory has changed. Please clean out TMPDIR and rebuild (sstate will be still be valid and reused)\n"

The part about valid and reused sstate doesn't seem to be true, if I compare sstate checksums with and without this patch (current master and master with this patch reverted), they are different:

$ bitbake-diffsigs stamps.1353438701/om-gta02/armv4t-oe-linux-gnueabi/linux-libc-headers/3.4.3-r0.do_package.sigdata.9bf64da2bd20a51a9f34d0e909ea94de stamps.1353439128/om-gta02/armv4t-oe-linux-gnueabi/linux-libc-headers-3.4.3-r0.do_package.sigdata.e2d8b3b59ff70b134b71f02b92f9d952
basehash changed from fd8635729e529ea26a963c47c1af647d to 0fd2c572da433536e317e1c7c12c4f60
List of dependencies for variable WORKDIR changed from set(['PR', 'PV', 'MULTIMACH_TARGET_SYS', 'PN', 'EXTENDPE', 'TMPDIR']) to set(['PF', 'TMPDIR', 'MULTIMACH_TARGET_SYS'])
changed items: set(['PR', 'PV', 'PF', 'PN', 'EXTENDPE'])
Variable WORKDIR value changed from ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR} to ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PF}
Hash for dependent task linux-libc-headers_3.4.3.bb.do_install changed from ac25fb45e330ee0f9848210a1ad71924 to fe785052d57bae397d0ab38fa69af65d

$ bitbake-diffsigs stamps.1353438701/om-gta02/armv4t-oe-linux-gnueabi/linux-libc-headers/3.4.3-r0.do_patch.sigdata.ab675d07d2d05b3621d3d68752deb1d5 stamps.1353439128/om-gta02/armv4t-oe-linux-gnueabi/linux-libc-headers-3.4.3-r0.do_patch.sigdata.1da915f84f81b15ca5f4f76176add79e
Task dependencies changed from:
['ABIEXTENSION', 'ARMPKGARCH', 'ARMPKGSFX_DSP', 'ARMPKGSFX_EABI', 'ARMPKGSFX_ENDIAN', 'ARMPKGSFX_FPU', 'ARMPKGSFX_THUMB', 'EXTENDPE', 'LIBCEXTENSION', 'MULTIMACH_TARGET_SYS', 'PACKAGE_ARCH', 'PATCHTOOL', 'PE', 'PN', 'PR', 'PV', 'S', 'SRCREV', 'TARGET_OS', 'TARGET_VENDOR', 'TUNE_PKGARCH', 'WORKDIR', 'base_do_patch', 'patch_do_patch', 'patch_path', 'should_apply', 'src_patches']
to:
['ABIEXTENSION', 'ARMPKGARCH', 'ARMPKGSFX_DSP', 'ARMPKGSFX_EABI', 'ARMPKGSFX_ENDIAN', 'ARMPKGSFX_FPU', 'ARMPKGSFX_THUMB', 'EXTENDPE', 'LIBCEXTENSION', 'MULTIMACH_TARGET_SYS', 'PACKAGE_ARCH', 'PATCHTOOL', 'PE', 'PF', 'PN', 'PR', 'PV', 'S', 'SRCREV', 'TARGET_OS', 'TARGET_VENDOR', 'TUNE_PKGARCH', 'WORKDIR', 'base_do_patch', 'patch_do_patch', 'patch_path', 'should_apply', 'src_patches']
basehash changed from 944e176cca5d006ba0306f57e85084bc to 5fb73074cd1a6bf4b5ba41f893232325
List of dependencies for variable WORKDIR changed from set(['PR', 'PV', 'MULTIMACH_TARGET_SYS', 'PN', 'EXTENDPE', 'TMPDIR']) to set(['PF', 'TMPDIR', 'MULTIMACH_TARGET_SYS'])
changed items: set(['PR', 'PV', 'PF', 'PN', 'EXTENDPE'])
Dependency on Variable PF was removed
Variable WORKDIR value changed from ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR} to ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PF}

Quoted 5 lines

meta/conf/bitbake.conf +0 -0
index cf779a4..450a0ee 100644

meta/conf/bitbake.conf GITDIR = "${CO_DIR}/git2" · line 342

Quoted 9 lines

meta/conf/bitbake.conf +3 -3

@@ -337,9 +337,9 @@ GITDIR = "${CO_DIR}/git2"
 BZRDIR = "${CO_DIR}/bzr"
 HGDIR = "${CO_DIR}/hg"
 
-STAMP = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}-${EXTENDPE}${PV}-${PR}"
-STAMPCLEAN = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}-[0-9]*-*"
-WORKDIR = "${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PF}"
+STAMP = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}"
+STAMPCLEAN = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}/*-*"
+WORKDIR = "${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}"

This breaks paths in all PN-dbg, as reported on previous patch doing this with WORKDIR:
http://lists.linuxtogo.org/pipermail/openembedded-core/2012-September/030031.html

Cheers,

Signature · 1 line

Martin 'JaMa' Jansa     jabber: [email protected]

Richard Purdie

2012-11-23 21:53 UTC #2 lore raw

Quoted 27 lines · Martin Jansa

Quoted 26 lines · On Tue, Nov 20, 2012 at 03:29:56PM +0000, [email protected] wrote:
Module: openembedded-core.git
Branch: master
Commit: 05075cf3138d1c61f5cf4fe0e1a4587acc00c692
URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=05075cf3138d1c61f5cf4fe0e1a4587acc00c692
Author: Richard Purdie <[email protected]>
Date:   Fri Nov 16 15:35:53 2012 +0000
bitbake.conf/sanity: Separate versions and PN stamp components into separate directories for WORKDIR and STAMP
This means some of the hacks we have to tell where the package name ends and the version starts in the directory layout becomes obsolete, simplifying the work of some of the cleanup scripts. It also makes the layout slightly more intuitive to the user.
It does force a rebuild onto the user but it will reuse sstate successfully.
Signed-off-by Richard Purdie <[email protected]>
---
 meta/classes/sanity.bbclass |    2 ++
 meta/conf/abi_version.conf  |    2 +-
 meta/conf/bitbake.conf      |    6 +++---
 3 files changed, 6 insertions(+), 4 deletions(-)

Quoted 14 lines

meta/classes/sanity.bbclass def check_sanity(sanity_data): · line 609

Quoted 10 lines

meta/classes/sanity.bbclass +2 -0

index 3806bc5..5b4f070 100644
@@ -605,6 +605,8 @@ def check_sanity(sanity_data):
             f.write(current_abi)
         elif abi == "7" and current_abi == "8":
             messages = messages + "Your configuration is using stamp files including the sstate hash but your build directory was built with stamp files that do not include this.\nTo continue, either rebuild or switch back to the OEBasic signature handler with BB_SIGNATURE_HANDLER = 'OEBasic'.\n"
+        elif (abi != current_abi and current_abi == "9"):
+            messages = messages + "The layout of the TMPDIR STAMPS directory has changed. Please clean out TMPDIR and rebuild (sstate will be still be valid and reused)\n"

The part about valid and reused sstate doesn't seem to be true, if I compare sstate checksums with and without this patch (current master and master with this patch reverted), they are different:

They are indeed, sorry about that :/.

Quoted 10 lines

meta/classes/sanity.bbclass def check_sanity(sanity_data): · line 612
Quoted 6 lines
meta/classes/sanity.bbclass +3 -3
-STAMP = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}-${EXTENDPE}${PV}-${PR}"
-STAMPCLEAN = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}-[0-9]*-*"
-WORKDIR = "${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PF}"
+STAMP = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}"
+STAMPCLEAN = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}/*-*"
+WORKDIR = "${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}"
This breaks paths in all PN-dbg, as reported on previous patch doing this with WORKDIR:
http://lists.linuxtogo.org/pipermail/openembedded-core/2012-September/030031.html

I pushed a fix for this earlier in the week, thanks for reminding me about that problem, I had forgotten about it :(.

Cheers,

Richard