several machine specific sysroots fixes
From: Dongxiao Xu <[email protected]>
Hi Richard,
Here are several fixes for machine specific sysroots, please help to review and pull.
The first patch is to add ${MACHINE} tag on stamps for do_populate_sysroot_setscene and do_package_setscene. The second patch is to handle the rm_work issues.
The third patch is to remove ${MACHINE} in compile and package functions in perl bb files.
Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: dxu4/distro
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dxu4/distro
Thanks,
Dongxiao Xu <[email protected]>
---
Dongxiao Xu (3):
bitbake: add ${MACHINE} for do_populate_sysroot_setscene and
do_package_setscene
rm_work.bbclass: handle stamp files while doing rm_work
perl: Remove ${MACHINE} dependency in compile and package.
meta/classes/cross-canadian.bbclass | 2 + meta/classes/crosssdk.bbclass | 2 + meta/classes/native.bbclass | 2 + meta/classes/nativesdk.bbclass | 2 + meta/classes/package.bbclass | 1 + meta/classes/rm_work.bbclass | 17 +++++++++++ meta/classes/staging.bbclass | 1 + meta/recipes-devtools/perl/perl_5.12.2.bb | 42 +++++++++++++---------------- 8 files changed, 46 insertions(+), 23 deletions(-)
PATCH 1/3 bitbake: add ${MACHINE} for do_populate_sysroot_setscene and do_package_setscene
From: Dongxiao Xu <[email protected]>
${MACHINE} information is also needed to be tagged on stamps for do_populate_sysroot_setscene and do_package_setscene, otherwise if build two machines both from sstate, the second machine's sysroot will not be populated correctly.
--- meta/classes/cross-canadian.bbclass | 2 ++ meta/classes/crosssdk.bbclass | 2 ++ meta/classes/native.bbclass | 2 ++ meta/classes/nativesdk.bbclass | 2 ++ meta/classes/package.bbclass | 1 + meta/classes/staging.bbclass | 1 + 6 files changed, 10 insertions(+), 0 deletions(-)
meta/classes/cross-canadian.bbclass +2 -0
@@ -82,4 +82,6 @@ export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR_HOST}" SHLIBSDIR = "${STAGING_DIR}/${SDK_ARCH}-nativesdk${SDK_VENDOR}-${BUILD_OS}/shlibs" do_populate_sysroot[stamp-extra-info] = "" +do_populate_sysroot_setscene[stamp-extra-info] = "" do_package[stamp-extra-info] = "" +do_package_setscene[stamp-extra-info] = ""
meta/classes/crosssdk.bbclass +2 -0
@@ -17,4 +17,6 @@ target_prefix = "${SDKPATHNATIVE}${prefix_nativesdk}" target_exec_prefix = "${SDKPATHNATIVE}${exec_prefix_nativesdk}" do_populate_sysroot[stamp-extra-info] = "" +do_populate_sysroot_setscene[stamp-extra-info] = "" do_package[stamp-extra-info] = "" +do_package_setscene[stamp-extra-info] = ""
meta/classes/native.bbclass +2 -0
@@ -121,4 +121,6 @@ do_package_write_deb[noexec] = "1" do_package_write_rpm[noexec] = "1" do_populate_sysroot[stamp-extra-info] = "" +do_populate_sysroot_setscene[stamp-extra-info] = "" do_package[stamp-extra-info] = "" +do_package_setscene[stamp-extra-info] = ""
meta/classes/nativesdk.bbclass +2 -0
@@ -82,4 +82,6 @@ python __anonymous () { } do_populate_sysroot[stamp-extra-info] = "" +do_populate_sysroot_setscene[stamp-extra-info] = "" do_package[stamp-extra-info] = "" +do_package_setscene[stamp-extra-info] = ""
meta/classes/package.bbclass +1 -0
@@ -1095,6 +1095,7 @@ do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}" do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}" do_package[sstate-lockfile] = "${PACKAGELOCK}" do_package[stamp-extra-info] = "${MACHINE}" +do_package_setscene[stamp-extra-info] = "${MACHINE}" do_package_setscene[dirs] = "${STAGING_DIR}" python do_package_setscene () {
meta/classes/staging.bbclass +1 -0
@@ -86,6 +86,7 @@ do_populate_sysroot[sstate-name] = "populate-sysroot" do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}" do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/" do_populate_sysroot[stamp-extra-info] = "${MACHINE}" +do_populate_sysroot_setscene[stamp-extra-info] = "${MACHINE}" python do_populate_sysroot_setscene () { sstate_setscene(d)
Signature · 1 line
1.6.3.3
PATCH 2/3 rm_work.bbclass: handle stamp files while doing rm_work
In reply to Dongxiao XuFrom: Dongxiao Xu <[email protected]>
If rm_work is scheduled, we need to remove the stamps of fetch, unpack, patch, etc, while just leaving the setscene varients.
For example, we'd task stamps like:
xxx-1.4.7-r3.do_compile
xxx-1.4.7-r3.do_configure
xxx-1.4.7-r3.do_fetch
xxx-1.4.7-r3.do_generate_toolchain_file
xxx-1.4.7-r3.do_install
xxx-1.4.7-r3.do_package.emenlow
xxx-1.4.7-r3.do_package_write
xxx-1.4.7-r3.do_package_write_ipk
xxx-1.4.7-r3.do_package_write_rpm
xxx-1.4.7-r3.do_patch
xxx-1.4.7-r3.do_populate_sysroot.emenlow
xxx-1.4.7-r3.do_setscene
xxx-1.4.7-r3.do_unpack
and after rm_work, we have stamps of:
xxx-1.4.7-r3.do_package_setscene.emenlow
xxx-1.4.7-r3.do_package_write_ipk_setscene
xxx-1.4.7-r3.do_package_write_rpm_setscene
xxx-1.4.7-r3.do_populate_sysroot_setscene.emenlow
This can solve the issue we meet if we force to schedule a task after contents in ${WORKDIR} are removed.
Also this can solve the rm_work issue we met after machine specific sysroot is implemented.
Thanks for Richard's suggestion to tracking these issues down.
--- meta/classes/rm_work.bbclass | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-)
meta/classes/rm_work.bbclass +17 -0
@@ -26,6 +26,23 @@ do_rm_work () { # Need to add pseudo back or subsqeuent work in this workdir # might fail since setscene may not rerun to recreate it mkdir ${WORKDIR}/pseudo/ + + rm -rf `ls ${STAMP}* | grep -v sigdata | grep -v do_populate_sysroot | grep -v do_package` ${STAMP}.do_package_write + + for stamp in `ls ${STAMP}* | grep do_package_write_* | grep -v setscene` + do + mv $stamp $stamp\_setscene + done + + for task in do_populate_sysroot do_package + do + if [ -e ${STAMP}.$task ]; then + mv ${STAMP}.$task ${STAMP}.$task\_setscene + elif [ -e ${STAMP}.$task.${MACHINE} ]; then + mv ${STAMP}.$task.${MACHINE} ${STAMP}.$task\_setscene.${MACHINE} + fi + done + } addtask rm_work after do_${RMWORK_ORIG_TASK}
Signature · 1 line
1.6.3.3
PATCH 3/3 perl: Remove ${MACHINE} dependency in compile and package.
In reply to Dongxiao XuFrom: Dongxiao Xu <[email protected]>
The judgement on ${MACHINE} is meaningless, remove it in order to avoid being rebuild on second machine.
--- meta/recipes-devtools/perl/perl_5.12.2.bb | 42 +++++++++++++---------------- 1 files changed, 19 insertions(+), 23 deletions(-)
meta/recipes-devtools/perl/perl_5.12.2.bb +19 -23
@@ -151,12 +151,10 @@ do_configure() { } do_compile() { - if test "${MACHINE}" != "native"; then - sed -i -e 's|/usr/include|${STAGING_INCDIR}|g' ext/Errno/Errno_pm.PL - sed -i -e 's|/usr/include|${STAGING_INCDIR}|g' cpan/Compress-Raw-Zlib/config.in - sed -i -e 's|/usr/lib|${STAGING_LIBDIR}|g' cpan/Compress-Raw-Zlib/config.in + sed -i -e 's|/usr/include|${STAGING_INCDIR}|g' ext/Errno/Errno_pm.PL + sed -i -e 's|/usr/include|${STAGING_INCDIR}|g' cpan/Compress-Raw-Zlib/config.in + sed -i -e 's|/usr/lib|${STAGING_LIBDIR}|g' cpan/Compress-Raw-Zlib/config.in - fi cd Cross oe_runmake perl LD="${CCLD}" }
@@ -186,24 +184,22 @@ PACKAGE_PREPROCESS_FUNCS += "perl_package_preprocess" perl_package_preprocess () { # Fix up installed configuration - if test "${MACHINE}" != "native"; then - sed -i -e "s,${D},,g" \ - -e "s,-isystem${STAGING_INCDIR} ,,g" \ - -e "s,${STAGING_LIBDIR},${libdir},g" \ - -e "s,${STAGING_BINDIR},${bindir},g" \ - -e "s,${STAGING_INCDIR},${includedir},g" \ - -e "s,${STAGING_BINDIR_NATIVE}/,,g" \ - ${PKGD}${bindir}/h2xs \ - ${PKGD}${bindir}/h2ph \ - ${PKGD}${libdir}/perl/${PV}/pod/*.pod \ - ${PKGD}${libdir}/perl/${PV}/cacheout.pl \ - ${PKGD}${libdir}/perl/${PV}/FileCache.pm \ - ${PKGD}${libdir}/perl/config.sh \ - ${PKGD}${libdir}/perl/${PV}/Config.pm \ - ${PKGD}${libdir}/perl/${PV}/Config_heavy.pl \ - ${PKGD}${libdir}/perl/${PV}/CORE/perl.h \ - ${PKGD}${libdir}/perl/${PV}/CORE/pp.h - fi + sed -i -e "s,${D},,g" \ + -e "s,-isystem${STAGING_INCDIR} ,,g" \ + -e "s,${STAGING_LIBDIR},${libdir},g" \ + -e "s,${STAGING_BINDIR},${bindir},g" \ + -e "s,${STAGING_INCDIR},${includedir},g" \ + -e "s,${STAGING_BINDIR_NATIVE}/,,g" \ + ${PKGD}${bindir}/h2xs \ + ${PKGD}${bindir}/h2ph \ + ${PKGD}${libdir}/perl/${PV}/pod/*.pod \ + ${PKGD}${libdir}/perl/${PV}/cacheout.pl \ + ${PKGD}${libdir}/perl/${PV}/FileCache.pm \ + ${PKGD}${libdir}/perl/config.sh \ + ${PKGD}${libdir}/perl/${PV}/Config.pm \ + ${PKGD}${libdir}/perl/${PV}/Config_heavy.pl \ + ${PKGD}${libdir}/perl/${PV}/CORE/perl.h \ + ${PKGD}${libdir}/perl/${PV}/CORE/pp.h } PACKAGES = "perl-dbg perl perl-misc perl-dev perl-pod perl-doc perl-lib \
Signature · 1 line
1.6.3.3
Re: [PATCH 2/3] rm_work.bbclass: handle stamp files while doing rm_work
In reply to Dongxiao XuQuoted 17 lines · Dongxiao Xu
meta/classes/rm_work.bbclass +9
-0@@ -26,6 +26,23 @@ do_rm_work () { # Need to add pseudo back or subsqeuent work in this workdir # might fail since setscene may not rerun to recreate it mkdir ${WORKDIR}/pseudo/ + + rm -rf `ls ${STAMP}* | grep -v sigdata | grep -v do_populate_sysroot | grep -v do_package` ${STAMP}.do_package_write + + for stamp in `ls ${STAMP}* | grep do_package_write_* | grep -v setscene` + do + mv $stamp $stamp\_setscene + done + + for task in do_populate_sysroot do_package
meta/classes/rm_work.bbclass do_rm_work () { · line 47
Quoted 10 lines
meta/classes/rm_work.bbclass +8
-0+ do + if [ -e ${STAMP}.$task ]; then + mv ${STAMP}.$task ${STAMP}.$task\_setscene + elif [ -e ${STAMP}.$task.${MACHINE} ]; then + mv ${STAMP}.$task.${MACHINE} ${STAMP}.$task\_setscene.${MACHINE} + fi + done + } addtask rm_work after do_${RMWORK_ORIG_TASK}
This is the right approach, however, if I set BB_SIGNATURE_HANDLER = "basichash" it will break since the stamps then look like:
ncurses-5.7-r0.do_populate_sysroot.59a5b1c1ce0a1bab23be44c27ba308a5.qemux86
and also, if we add new tasks to sstate, this code would then need to be updated with the new tasknames.
I ended up writing http://git.pokylinux.org/cgit.cgi/poky/commit/?id=e8e9f56057f3732d2550375a23573ab7231eac22
which I merged into master to address the problem since its breaking the
autobuilder and people's builds out there. I'd have thought there would
be a neater way to do this but I can't find one at the moment. Any
improvements anyone can see are welcome.
Cheers,
Richard
Re: [PATCH 1/3] bitbake: add ${MACHINE} for do_populate_sysroot_setscene and do_package_setscene
In reply to Dongxiao XuQuoted 8 lines · Dongxiao Xu
From: Dongxiao Xu <[email protected]>
${MACHINE} information is also needed to be tagged on stamps for
do_populate_sysroot_setsceneanddo_package_setscene, otherwise if build two machines both from sstate, the second machine's sysroot will not be populated correctly.Signed-off-by Dongxiao Xu <[email protected]>
The _setscene stuff is really an internal implementation issue within bitbake. I don't like the idea that the metadata has to get this right and I'd prefer it if bitbake always did the right thing. In this case that means looking up the extra stamp info against the parent task, not the setscene one.
I therefore merged:
http://git.pokylinux.org/cgit.cgi/poky/commit/?id=c34913ea5a4f20ce4bca3c68a65e9f64f1e3910c
which should obsolete this patch?
Good catch on finding the problem though!
Cheers,
Richard
Richard Purdie
AppliedRe: [PATCH 3/3] perl: Remove ${MACHINE} dependency in compile and package.
In reply to Dongxiao XuQuoted 6 lines · Dongxiao Xu
From: Dongxiao Xu <[email protected]>
The judgement on ${MACHINE} is meaningless, remove it in order to avoid being rebuild on second machine.
Signed-off-by Dongxiao Xu <[email protected]>
I've merged this into master, thanks!
Richard
Re: [PATCH 2/3] rm_work.bbclass: handle stamp files while doing rm_work
In reply to Richard PurdieQuoted 34 lines · Richard Purdie
Quoted 20 lines · Dongxiao Xu
(quoted diff) +0
-0b/meta/classes/rm_work.bbclass index 260ecb0..a151f4c 100644
meta/classes/rm_work.bbclass +0
-0@@ -26,6 +26,23 @@ do_rm_work () { # Need to add pseudo back or subsqeuent work in this workdir # might fail since setscene may not rerun to recreate itmkdir ${WORKDIR}/pseudo/ +
+ rm -rf `ls ${STAMP}* | grep -v sigdata | grep -v
+do_populate_sysroot| grep -v do_package`
${STAMP}.do_package_write + + for stamp in `ls ${STAMP}* | grep
do_package_write_* | grep -v setscene` + do + mv $stamp
$stamp\_setscene + donemeta/classes/rm_work.bbclass +7
-0+ + for task in do_populate_sysroot do_package + do + if [ -e ${STAMP}.$task ]; then + mv ${STAMP}.$task ${STAMP}.$task\_setscene + elif [ -e ${STAMP}.$task.${MACHINE} ]; then + mv ${STAMP}.$task.${MACHINE}Quoted 4 lines
${STAMP}.$task\_setscene.${MACHINE} + fi + done
+
}
addtaskrm_workafter do_${RMWORK_ORIG_TASK}This is the right approach, however, if I set
BB_SIGNATURE_HANDLER= "basichash" it will break since the stamps then look like:ncurses-5.7-r0.do_populate_sysroot.59a5b1c1ce0a1bab23be44c27ba308a5.qemux86
and also, if we add new tasks to sstate, this code would then need to be updated with the new tasknames.
Quoted 6 lines
I ended up writing
http://git.pokylinux.org/cgit.cgi/poky/commit/?id=e8e9f56057f3732d2550375a23573ab7231eac22
which I merged into master to address the problem since its breaking the autobuilder and people's builds out there. I'd have thought there would be a neater way to do this but I can't find one at the moment. Any improvements anyone can see are welcome.
Thanks for your fix, I am not aware of such stamp format, thus it's not covered in my previous fix.
Thanks,
Dongxiao
Quoted 4 lines
Cheers,
Richard
Re: [PATCH 1/3] bitbake: add ${MACHINE} for do_populate_sysroot_setscene and do_package_setscene
In reply to Richard PurdieQuoted 16 lines · Richard Purdie
Quoted 8 lines · Dongxiao Xu
From: Dongxiao Xu <[email protected]>
${MACHINE} information is also needed to be tagged on stamps for
do_populate_sysroot_setsceneanddo_package_setscene, otherwise if build two machines both from sstate, the second machine's sysroot will not be populated correctly.Signed-off-by Dongxiao Xu <[email protected]>The
_setscenestuff is really an internal implementation issue within bitbake. I don't like the idea that the metadata has to get this right and I'd prefer it if bitbake always did the right thing. In this case that means looking up the extra stamp info against the parent task, not the setscene one.
Quoted 3 lines
I therefore merged:
http://git.pokylinux.org/cgit.cgi/poky/commit/?id=c34913ea5a4f20ce4bca3c68a65e9f64f1e3910c
Yes, this approach is better then specifically handling *_setscene stamp-extra-info in meta files. Thanks!
-- Dongxiao
Quoted 8 lines
which should obsolete this patch?
Good catch on finding the problem though!
Cheers,
Richard