Machine specific sysroot implementation
v4 cover Machine specific sysroot implementation
Hi Richard,
This RFC is the 4th version of machine specific sysroot implementation. Please help to review.
I tested the build of "poky-image-sdk" and "meta-toolchain-sdk" for atom-pc and emenlow in the same build directory if change emenlow's architecture to core2, and the test could be passed.
Changes from v3:
1) Make bb.build.stampfile() to be the main entry point so most users will be converted to use that, thus the datacache vs d handling code can be only contained in stamp_internal() which calls into siggen. Besides, using True/False to replace 1/0 in the patch.
2) When doing populate_sysroot, some la files need to be relocated. 0002 patch changes the relocation handling in destination dirs ("sysroot-destdirs" dirs) instead of source ("image" dirs). This ensures that, contents in "image" directory are not changed and when populating the second machine, the "sed" command can still find those key words.
3) Change emenlow's BASE_PACKAGE_ARCH and PACKAGE_EXTRA_ARCHS to be core2.
Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: dxu4/mach_sysroot_v4
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dxu4/mach_sysroot_v4
Thanks,
Dongxiao Xu <[email protected]>
---
Dongxiao Xu (5):
staging: Use relative path in sysroot-destdir for target recipes
staging: relocate *.la paths in destination dirs
bitbake: Introduce stamp-extra-info into build stamp file
bitbake: machine specific sysroots implementation
emenlow: Change PACKAGE_EXTRA_ARCHS and BASE_PACKAGE_ARCH to core2
bitbake/lib/bb/build.py | 14 +++----- bitbake/lib/bb/cache.py | 19 +++++++++- bitbake/lib/bb/runqueue.py | 12 +++--- bitbake/lib/bb/siggen.py | 20 +++++++++- meta-emenlow/conf/machine/emenlow.conf | 6 ++-- meta/classes/base.bbclass | 2 +- meta/classes/binconfig.bbclass | 6 ++-- meta/classes/cross-canadian.bbclass | 5 ++- meta/classes/cross.bbclass | 5 +++ meta/classes/crosssdk.bbclass | 2 + meta/classes/kernel.bbclass | 2 +- meta/classes/native.bbclass | 4 ++ meta/classes/nativesdk.bbclass | 7 +++- meta/classes/package.bbclass | 1 + meta/classes/siteconfig.bbclass | 11 +++--- meta/classes/sstate.bbclass | 6 ++- meta/classes/staging.bbclass | 37 +++++++++++-------- meta/classes/toolchain-scripts.bbclass | 4 +- meta/conf/bitbake.conf | 11 +++--- meta/recipes-connectivity/gupnp/gupnp_0.14.0.bb | 4 +- .../gcc/gcc-cross-intermediate.inc | 3 ++ meta/recipes-devtools/libtool/libtool-cross_2.4.bb | 4 +- .../libtool/libtool-nativesdk_2.4.bb | 4 +- meta/recipes-devtools/libtool/libtool_2.4.bb | 6 ++-- meta/recipes-gnome/gtk+/gtk+.inc | 4 +- meta/recipes-support/apr/apr_1.3.3.bb | 2 +- 26 files changed, 131 insertions(+), 70 deletions(-)
PATCH v4 1/5 staging: Use relative path in sysroot-destdir for target recipes
Original we used absolute path in sysroot-destdir for both native and target recipes. This commit changes target recipes to use relative path which is same as the image directory.
--- meta/classes/binconfig.bbclass | 6 ++-- meta/classes/cross-canadian.bbclass | 2 + meta/classes/kernel.bbclass | 2 +- meta/classes/native.bbclass | 2 + meta/classes/nativesdk.bbclass | 4 +- meta/classes/siteconfig.bbclass | 5 ++- meta/classes/staging.bbclass | 27 ++++++++++--------- meta/recipes-connectivity/gupnp/gupnp_0.14.0.bb | 4 +- .../gcc/gcc-cross-intermediate.inc | 3 ++ meta/recipes-devtools/libtool/libtool-cross_2.4.bb | 4 +- .../libtool/libtool-nativesdk_2.4.bb | 4 +- meta/recipes-devtools/libtool/libtool_2.4.bb | 6 ++-- meta/recipes-gnome/gtk+/gtk+.inc | 4 +- meta/recipes-support/apr/apr_1.3.3.bb | 2 +- 14 files changed, 42 insertions(+), 33 deletions(-)
meta/classes/binconfig.bbclass +3 -3
@@ -47,8 +47,8 @@ SYSROOT_PREPROCESS_FUNCS += "binconfig_sysroot_preprocess" binconfig_sysroot_preprocess () { for config in `find ${S} -name '${BINCONFIG_GLOB}'`; do configname=`basename $config` - install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS} - cat $config | sed ${@get_binconfig_mangle(d)} > ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/$configname - chmod u+x ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/$configname + install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts + cat $config | sed ${@get_binconfig_mangle(d)} > ${SYSROOT_DESTDIR}${bindir}/crossscripts/$configname + chmod u+x ${SYSROOT_DESTDIR}${bindir}/crossscripts/$configname done }
meta/classes/cross-canadian.bbclass +2 -0
@@ -71,6 +71,8 @@ base_sbindir = "${bindir}" libdir = "${exec_prefix}/lib/${OLD_MULTIMACH_ARCH}${TARGET_VENDOR}-${TARGET_OS}" libexecdir = "${exec_prefix}/libexec/${OLD_MULTIMACH_ARCH}${TARGET_VENDOR}-${TARGET_OS}" +do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/${base_prefix}" + FILES_${PN} = "${prefix}" FILES_${PN}-dbg += "${prefix}/.debug \ ${prefix}/bin/.debug \
meta/classes/kernel.bbclass +1 -1
@@ -185,7 +185,7 @@ kernel_do_install() { } sysroot_stage_all_append() { - sysroot_stage_dir ${D}/kernel ${SYSROOT_DESTDIR}${STAGING_KERNEL_DIR} + sysroot_stage_dir ${D}/kernel ${SYSROOT_DESTDIR}/kernel }
meta/classes/native.bbclass +2 -0
@@ -63,6 +63,8 @@ exec_prefix = "${STAGING_DIR_NATIVE}${prefix_native}" # Since we actually install these into situ there is no staging prefix STAGING_DIR_HOST = "" STAGING_DIR_TARGET = "" +do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_NATIVE}" + SHLIBSDIR = "${STAGING_DIR_NATIVE}/shlibs" PKG_CONFIG_DIR = "${libdir}/pkgconfig"
meta/classes/nativesdk.bbclass +2 -2
@@ -43,6 +43,8 @@ base_prefix = "${SDKPATHNATIVE}" prefix = "${SDKPATHNATIVE}${prefix_nativesdk}" exec_prefix = "${SDKPATHNATIVE}${prefix_nativesdk}" +do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/${base_prefix}" + FILES_${PN} += "${prefix}" FILES_${PN}-dbg += "${prefix}/.debug \ ${prefix}/bin/.debug \
@@ -78,5 +80,3 @@ python __anonymous () { bb.data.setVar("PROVIDES", provides, d) bb.data.setVar("OVERRIDES", bb.data.getVar("OVERRIDES", d, False) + ":virtclass-nativesdk", d) } - -
meta/classes/siteconfig.bbclass +3 -2
@@ -20,8 +20,9 @@ siteconfig_do_siteconfig_gencache () { sed -n -e "/ac_cv_c_bigendian/p" -e "/ac_cv_sizeof_/p" \ -e "/ac_cv_type_/p" -e "/ac_cv_header_/p" -e "/ac_cv_func_/p" \ < ${PN}_cache > ${PN}_config - mkdir -p ${SYSROOT_DESTDIR}${SITECONFIG_SYSROOTCACHE} - cp ${PN}_config ${SYSROOT_DESTDIR}${SITECONFIG_SYSROOTCACHE} + mkdir -p ${SYSROOT_DESTDIR}${datadir}/${TARGET_SYS}_config_site.d + cp ${PN}_config ${SYSROOT_DESTDIR}${datadir}/${TARGET_SYS}_config_site.d + } do_populate_sysroot[sstate-interceptfuncs] += "do_siteconfig "
meta/classes/staging.bbclass +14 -13
@@ -45,25 +45,25 @@ sysroot_stage_dirs() { from="$1" to="$2" - sysroot_stage_dir $from${includedir} $to${STAGING_INCDIR} + sysroot_stage_dir $from${includedir} $to${includedir} if [ "${BUILD_SYS}" = "${HOST_SYS}" ]; then - sysroot_stage_dir $from${bindir} $to${STAGING_DIR_HOST}${bindir} - sysroot_stage_dir $from${sbindir} $to${STAGING_DIR_HOST}${sbindir} - sysroot_stage_dir $from${base_bindir} $to${STAGING_DIR_HOST}${base_bindir} - sysroot_stage_dir $from${base_sbindir} $to${STAGING_DIR_HOST}${base_sbindir} - sysroot_stage_dir $from${libexecdir} $to${STAGING_DIR_HOST}${libexecdir} - sysroot_stage_dir $from${sysconfdir} $to${STAGING_DIR_HOST}${sysconfdir} - sysroot_stage_dir $from${localstatedir} $to${STAGING_DIR_HOST}${localstatedir} + sysroot_stage_dir $from${bindir} $to${bindir} + sysroot_stage_dir $from${sbindir} $to${sbindir} + sysroot_stage_dir $from${base_bindir} $to${base_bindir} + sysroot_stage_dir $from${base_sbindir} $to${base_sbindir} + sysroot_stage_dir $from${libexecdir} $to${libexecdir} + sysroot_stage_dir $from${sysconfdir} $to${sysconfdir} + sysroot_stage_dir $from${localstatedir} $to${localstatedir} fi if [ -d $from${libdir} ] then - sysroot_stage_libdir $from/${libdir} $to${STAGING_LIBDIR} + sysroot_stage_libdir $from/${libdir} $to${libdir} fi if [ -d $from${base_libdir} ] then - sysroot_stage_libdir $from${base_libdir} $to${STAGING_DIR_HOST}${base_libdir} + sysroot_stage_libdir $from${base_libdir} $to${base_libdir} fi - sysroot_stage_dir $from${datadir} $to${STAGING_DATADIR} + sysroot_stage_dir $from${datadir} $to${datadir} } sysroot_stage_all() {
@@ -76,6 +76,7 @@ do_populate_sysroot[dirs] = "${STAGING_DIR_TARGET}/${bindir} ${STAGING_DIR_TARGE ${STAGING_INCDIR_NATIVE} \ ${STAGING_DATADIR} \ ${SYSROOT_DESTDIR}${STAGING_DIR_TARGET} \ + ${SYSROOT_DESTDIR}${base_prefix} \ ${S} ${B}" # Could be compile but populate_sysroot and do_install shouldn't run at the same time
@@ -103,8 +104,8 @@ python do_populate_sysroot () { SSTATETASKS += "do_populate_sysroot" do_populate_sysroot[sstate-name] = "populate-sysroot" -do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${STAGING_DIR}" -do_populate_sysroot[sstate-outputdirs] = "${TMPDIR}/sysroots" +do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${base_prefix}" +do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/" python do_populate_sysroot_setscene () { sstate_setscene(d)
meta/recipes-connectivity/gupnp/gupnp_0.14.0.bb +2 -2
@@ -23,7 +23,7 @@ FILES_${PN}-dev += "${bindir}/gupnp-binding-tool" SYSROOT_PREPROCESS_FUNCS += "gupnp_sysroot_preprocess" gupnp_sysroot_preprocess () { - install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/ - install -m 755 ${D}${bindir}/gupnp-binding-tool ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/ + install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts/ + install -m 755 ${D}${bindir}/gupnp-binding-tool ${SYSROOT_DESTDIR}${bindir}/crossscripts/ }
meta/recipes-devtools/gcc/gcc-cross-intermediate.inc +3 -0
@@ -17,6 +17,9 @@ EXTRA_OECONF = "--with-local-prefix=${STAGING_DIR_TARGET}${target_prefix} \ ${EXTRA_OECONF_INTERMEDIATE} \ ${@get_gcc_fpu_setting(bb, d)}" +do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${STAGING_DIR_HOST} ${SYSROOT_DESTDIR}/${STAGING_DIR_TARGET}" +do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST} ${STAGING_DIR_TARGET}" + do_compile () { oe_runmake }
meta/recipes-devtools/libtool/libtool-cross_2.4.bb +2 -2
@@ -29,6 +29,6 @@ do_install () { SYSROOT_PREPROCESS_FUNCS += "libtoolcross_sysroot_preprocess" libtoolcross_sysroot_preprocess () { - install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/ - install -m 755 ${D}${bindir}/${HOST_SYS}-libtool ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/${HOST_SYS}-libtool + install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts + install -m 755 ${D}${bindir}/${HOST_SYS}-libtool ${SYSROOT_DESTDIR}${bindir}/crossscripts/${HOST_SYS}-libtool }
meta/recipes-devtools/libtool/libtool-nativesdk_2.4.bb +2 -2
@@ -22,6 +22,6 @@ do_install () { SYSROOT_PREPROCESS_FUNCS += "libtoolnativesdk_sysroot_preprocess" libtoolnativesdk_sysroot_preprocess () { - install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/ - install -m 755 ${D}${bindir}/${HOST_SYS}-libtool ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/${HOST_SYS}-libtool + install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts/ + install -m 755 ${D}${bindir}/${HOST_SYS}-libtool ${SYSROOT_DESTDIR}${bindir}/crossscripts/${HOST_SYS}-libtool }
meta/recipes-devtools/libtool/libtool_2.4.bb +3 -3
@@ -26,9 +26,9 @@ SYSROOT_PREPROCESS_FUNCS += "libtool_sysroot_preprocess" libtool_sysroot_preprocess () { if [ "${PN}" == "libtool" ]; then - rm -rf ${SYSROOT_DESTDIR}${STAGING_DIR_TARGET}${bindir}/* - rm -rf ${SYSROOT_DESTDIR}${STAGING_DIR_TARGET}${datadir}/aclocal/* - rm -rf ${SYSROOT_DESTDIR}${STAGING_DIR_TARGET}${datadir}/libtool/config/* + rm -rf ${SYSROOT_DESTDIR}${bindir}/* + rm -rf ${SYSROOT_DESTDIR}${datadir}/aclocal/* + rm -rf ${SYSROOT_DESTDIR}${datadir}/libtool/config/* fi }
meta/recipes-gnome/gtk+/gtk+.inc +2 -2
@@ -64,8 +64,8 @@ SYSROOT_PREPROCESS_FUNCS += "gtk_sysroot_preprocess" gtk_sysroot_preprocess () { if [ -e ${D}${bindir}/gtk-builder-convert ]; then - install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/ - install -m 755 ${D}${bindir}/gtk-builder-convert ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/ + install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts/ + install -m 755 ${D}${bindir}/gtk-builder-convert ${SYSROOT_DESTDIR}${bindir}/crossscripts/ fi }
meta/recipes-support/apr/apr_1.3.3.bb +1 -1
@@ -24,7 +24,7 @@ do_configure_prepend() { SYSROOT_PREPROCESS_FUNCS += "apr_sysroot_preprocess" apr_sysroot_preprocess () { - d=${SYSROOT_DESTDIR}${STAGING_DIR_TARGET}${datadir}/apr + d=${SYSROOT_DESTDIR}${datadir}/apr install -d $d/ cp ${S}/build/apr_rules.mk $d/ sed -i s,apr_builddir=.*,apr_builddir=,g $d/apr_rules.mk
Signature · 1 line
1.6.3.3
Re: [PATCH 1/5] staging: Use relative path in sysroot-destdir for target recipes
Hi Dongxaio,
I agree with this patch as you know and I want to merge it but I'm not entirely sure its 100% right yet.
Quoted 50 lines · Dongxiao Xu
meta/classes/staging.bbclass +12
-11@@ -45,25 +45,25 @@ sysroot_stage_dirs() { from="$1" to="$2" - sysroot_stage_dir $from${includedir} $to${STAGING_INCDIR} + sysroot_stage_dir $from${includedir} $to${includedir} if [ "${BUILD_SYS}" = "${HOST_SYS}" ]; then - sysroot_stage_dir $from${bindir} $to${STAGING_DIR_HOST}${bindir} - sysroot_stage_dir $from${sbindir} $to${STAGING_DIR_HOST}${sbindir} - sysroot_stage_dir $from${base_bindir} $to${STAGING_DIR_HOST}${base_bindir} - sysroot_stage_dir $from${base_sbindir} $to${STAGING_DIR_HOST}${base_sbindir} - sysroot_stage_dir $from${libexecdir} $to${STAGING_DIR_HOST}${libexecdir} - sysroot_stage_dir $from${sysconfdir} $to${STAGING_DIR_HOST}${sysconfdir} - sysroot_stage_dir $from${localstatedir} $to${STAGING_DIR_HOST}${localstatedir} + sysroot_stage_dir $from${bindir} $to${bindir} + sysroot_stage_dir $from${sbindir} $to${sbindir} + sysroot_stage_dir $from${base_bindir} $to${base_bindir} + sysroot_stage_dir $from${base_sbindir} $to${base_sbindir} + sysroot_stage_dir $from${libexecdir} $to${libexecdir} + sysroot_stage_dir $from${sysconfdir} $to${sysconfdir} + sysroot_stage_dir $from${localstatedir} $to${localstatedir} fi if [ -d $from${libdir} ] then - sysroot_stage_libdir $from/${libdir} $to${STAGING_LIBDIR} + sysroot_stage_libdir $from/${libdir} $to${libdir} fi if [ -d $from${base_libdir} ] then - sysroot_stage_libdir $from${base_libdir} $to${STAGING_DIR_HOST}${base_libdir} + sysroot_stage_libdir $from${base_libdir} $to${base_libdir} fi - sysroot_stage_dir $from${datadir} $to${STAGING_DATADIR} + sysroot_stage_dir $from${datadir} $to${datadir} } sysroot_stage_all() {@@ -76,6 +76,7 @@ do_populate_sysroot[dirs] = "${STAGING_DIR_TARGET}/${bindir} ${STAGING_DIR_TARGE ${STAGING_INCDIR_NATIVE} \ ${STAGING_DATADIR} \ ${SYSROOT_DESTDIR}${STAGING_DIR_TARGET} \ + ${SYSROOT_DESTDIR}${base_prefix} \ ${S} ${B}" # Could be compile but populate_sysroot and do_install shouldn't run at the same time
meta/classes/staging.bbclass python do_populate_sysroot () { · line 108
Quoted 7 lines
meta/classes/staging.bbclass +2
-2@@ -103,8 +104,8 @@ python do_populate_sysroot () { SSTATETASKS += "do_populate_sysroot" do_populate_sysroot[sstate-name] = "populate-sysroot" -do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${STAGING_DIR}" -do_populate_sysroot[sstate-outputdirs] = "${TMPDIR}/sysroots" +do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${base_prefix}" +do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/"
The reason why is I'm still not 100% comfortable with the two lines above. They look unbalanced and should really be paired to match such that they'd either be:
do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${base_prefix}" do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/${base_prefix}"
or
do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}" do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}"
but just looking at code I'm having trouble working out whether either, both or neither of these would work and in which cases.
The cross/crosssdk recipes are a mess as they install into two different sysroots. Dexuan is working on some changes to improve that so I'm least worried about those two classes and the patch I just merged for libgcc is going to help. I'm therefore aiming for this to work for the target/native/nativesdk/cross-canadian cases and the cross/crosssdk one is secondary and can be messy for now if needed.
I feel most comfortable aiming for the latter of my two options above, does that work assuming we hack cross/crosssdk to do whatever is needed to make them work with it? Can the others then remain unaltered?
The other minor thing I'm thinking about is adding a definition of:
bindir_crossscripts = "${bindir}/crossscripts"
to bitbake.conf (near the top next to bindir_cross) and using that in the sysroot functions.
Cheers,
Richard
Re: [PATCH 1/5] staging: Use relative path in sysroot-destdir for target recipes
Hi Richard,
Quoted 92 lines · Richard Purdie
Hi Dongxaio,
I agree with this patch as you know and I want to merge it but I'm not entirely sure its 100% right yet.
Quoted 58 lines · Dongxiao Xu
(quoted diff) +0
-0b/meta/classes/staging.bbclass index 8432565..4d2991b 100644
meta/classes/staging.bbclass +1
-8@@ -45,25 +45,25 @@ sysroot_stage_dirs() { from="$1" to="$2" - sysroot_stage_dir $from${includedir} $to${STAGING_INCDIR} + sysroot_stage_dir $from${includedir} $to${includedir} if [ "${BUILD_SYS}" = "${HOST_SYS}" ]; then - sysroot_stage_dir $from${bindir} $to${STAGING_DIR_HOST}${bindir} - sysroot_stage_dir $from${sbindir} $to${STAGING_DIR_HOST}${sbindir} - sysroot_stage_dir $from${base_bindir} $to${STAGING_DIR_HOST}${base_bindir} - sysroot_stage_dir $from${base_sbindir} $to${STAGING_DIR_HOST}${base_sbindir} - sysroot_stage_dir $from${libexecdir} $to${STAGING_DIR_HOST}${libexecdir} - sysroot_stage_dir $from${sysconfdir} $to${STAGING_DIR_HOST}${sysconfdir} - sysroot_stage_dir $from${localstatedir}$to${STAGING_DIR_HOST}${localstatedir} +
sysroot_stage_dir
$from${bindir} $to${bindir} +sysroot_stage_dir$from${sbindir}
$to${sbindir} +sysroot_stage_dir$from${base_bindir}
$to${base_bindir} +sysroot_stage_dir$from${base_sbindir}
$to${base_sbindir} +sysroot_stage_dir$from${libexecdir}
$to${libexecdir} +sysroot_stage_dir$from${sysconfdir}
$to${sysconfdir} +sysroot_stage_dir$from${localstatedir}meta/classes/staging.bbclass +1
-2$to${localstatedir} fi if [ -d $from${libdir} ] then - sysroot_stage_libdir $from/${libdir} $to${STAGING_LIBDIR} + sysroot_stage_libdir $from/${libdir} $to${libdir} fi if [ -d $from${base_libdir} ] then - sysroot_stage_libdir $from${base_libdir}$to${STAGING_DIR_HOST}${base_libdir} +
sysroot_stage_libdir
$from${base_libdir} $to${base_libdir} fi -sysroot_stage_dir
$from${datadir} $to${STAGING_DATADIR} +sysroot_stage_dir
$from${datadir} $to${datadir} }
sysroot_stage_all(){meta/classes/staging.bbclass +0
-0@@ -76,6 +76,7 @@ do_populate_sysroot[dirs] = "${STAGING_DIR_TARGET}/${bindir} ${STAGING_DIR_TARGE ${STAGING_INCDIR_NATIVE} \ ${STAGING_DATADIR} \${SYSROOT_DESTDIR}${STAGING_DIR_TARGET} \ +
${SYSROOT_DESTDIR}${base_prefix} \ ${S} ${B}"# Could be compile but
populate_sysrootanddo_installshouldn't run
at the same time @@ -103,8 +104,8 @@ pythondo_populate_sysroot() {SSTATETASKS += "
do_populate_sysroot" do_populate_sysroot[sstate-name] = "populate-sysroot"
-do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${STAGING_DIR}"meta/classes/staging.bbclass +1
-1-do_populate_sysroot[sstate-outputdirs] = "${TMPDIR}/sysroots" +do_populate_sysroot[sstate-inputdirs] =Quoted 2 lines
"${SYSROOT_DESTDIR}/${base_prefix}" +do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/"
The reason why is I'm still not 100% comfortable with the two lines above. They look unbalanced and should really be paired to match such that they'd either be:
do_populate_sysroot[sstate-inputdirs] =
"${SYSROOT_DESTDIR}/${base_prefix}"
do_populate_sysroot[sstate-outputdirs] =
"${STAGING_DIR_HOST}/${base_prefix}"or
do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}" do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}"
but just looking at code I'm having trouble working out whether either, both or neither of these would work and in which cases.
The cross/crosssdk recipes are a mess as they install into two different sysroots. Dexuan is working on some changes to improve that so I'm least worried about those two classes and the patch I just merged for libgcc is going to help. I'm therefore aiming for this to work for the target/native/nativesdk/cross-canadian cases and the cross/crosssdk one is secondary and can be messy for now if needed.
Quoted 4 lines
I feel most comfortable aiming for the latter of my two options above, does that work assuming we hack cross/crosssdk to do whatever is needed to make them work with it? Can the others then remain unaltered?
${base_prefix} has some kinds of values considerring different recipe type. (we do not discuss cross and crosssdk)
Normal target recipe: base_prefix = ""
Native recipes: base_prefix = "${STAGING_DIR_NATIVE}" Nativesdk recipes: base_prefix = "${SDKPATHNATIVE}" Cross-canadian recipes: base_prefix = "${SDKPATHNATIVE}"
For target recipes, the input and output dirs should be:
Input: ${SYSROOT_DESTDIR}
Output: ${STAGING_DIR_HOST}
For native recipes, the input and output dirs should be:
Input: ${SYSROOT_DESTDIR}/${STAGING_DIR_NATIVE}
Output: "${STAGING_DIR_NATIVE}"
For nativesdk recipes, the input and output dirs should be:
Input: ${SYSROOT_DESTDIR}/${SDKPATHNATIVE}
Output: "${STAGING_DIR_HOST}/${SDKPATHNATIVE}
For cross-canadian recipes, the input and output dirs should be:
Input: ${SYSROOT_DESTDIR}/${SDKPATHNATIVE}
Output: "${STAGING_DIR_HOST}/${SDKPATHNATIVE}
It is workable if change to your latter option. do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}" do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}"
In this case we need to specially handle native cases.
Quoted 7 lines
The other minor thing I'm thinking about is adding a definition of:
bindir_crossscripts= "${bindir}/crossscripts"to bitbake.conf (near the top next to
bindir_cross) and using that in the sysroot functions.
OK, I will change accordingly.
Thanks,
Dongxiao
Quoted 4 lines
Cheers,
Richard
Re: [PATCH 1/5] staging: Use relative path in sysroot-destdir for target recipes
In reply to Richard PurdieQuoted 126 lines · Dongxiao
Hi Richard,
Quoted 93 lines · Richard Purdie
Hi Dongxaio,
I agree with this patch as you know and I want to merge it but I'm not entirely sure its 100% right yet.
Quoted 59 lines · Dongxiao Xu
(quoted diff) +0
-0b/meta/classes/staging.bbclass index 8432565..4d2991b 100644
meta/classes/staging.bbclass +1
-8@@ -45,25 +45,25 @@ sysroot_stage_dirs() { from="$1" to="$2" - sysroot_stage_dir $from${includedir} $to${STAGING_INCDIR} + sysroot_stage_dir $from${includedir} $to${includedir} if [ "${BUILD_SYS}" = "${HOST_SYS}" ]; then - sysroot_stage_dir $from${bindir} $to${STAGING_DIR_HOST}${bindir} - sysroot_stage_dir $from${sbindir} $to${STAGING_DIR_HOST}${sbindir} - sysroot_stage_dir $from${base_bindir} $to${STAGING_DIR_HOST}${base_bindir} - sysroot_stage_dir $from${base_sbindir} $to${STAGING_DIR_HOST}${base_sbindir} - sysroot_stage_dir $from${libexecdir} $to${STAGING_DIR_HOST}${libexecdir} - sysroot_stage_dir $from${sysconfdir} $to${STAGING_DIR_HOST}${sysconfdir} - sysroot_stage_dir $from${localstatedir}$to${STAGING_DIR_HOST}${localstatedir} +
sysroot_stage_dir
$from${bindir} $to${bindir} +sysroot_stage_dir$from${sbindir}
$to${sbindir} +sysroot_stage_dir$from${base_bindir}
$to${base_bindir} +sysroot_stage_dir$from${base_sbindir}
$to${base_sbindir} +sysroot_stage_dir$from${libexecdir}
$to${libexecdir} +sysroot_stage_dir$from${sysconfdir}
$to${sysconfdir} +sysroot_stage_dir$from${localstatedir}meta/classes/staging.bbclass +1
-2$to${localstatedir} fi if [ -d $from${libdir} ] then - sysroot_stage_libdir $from/${libdir} $to${STAGING_LIBDIR} + sysroot_stage_libdir $from/${libdir} $to${libdir} fi if [ -d $from${base_libdir} ] then - sysroot_stage_libdir $from${base_libdir}$to${STAGING_DIR_HOST}${base_libdir} +
sysroot_stage_libdir
$from${base_libdir} $to${base_libdir} fi -sysroot_stage_dir
$from${datadir} $to${STAGING_DATADIR} +sysroot_stage_dir
$from${datadir} $to${datadir} }
sysroot_stage_all(){meta/classes/staging.bbclass +0
-0@@ -76,6 +76,7 @@ do_populate_sysroot[dirs] = "${STAGING_DIR_TARGET}/${bindir} ${STAGING_DIR_TARGE ${STAGING_INCDIR_NATIVE} \ ${STAGING_DATADIR} \${SYSROOT_DESTDIR}${STAGING_DIR_TARGET} \ +
${SYSROOT_DESTDIR}${base_prefix} \ ${S} ${B}"# Could be compile but
populate_sysrootanddo_installshouldn't
run at the same time @@ -103,8 +104,8 @@ pythondo_populate_sysroot() {SSTATETASKS += "
do_populate_sysroot" do_populate_sysroot[sstate-name] = "populate-sysroot"
-do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${STAGING_DIR}"meta/classes/staging.bbclass +1
-1-do_populate_sysroot[sstate-outputdirs] = "${TMPDIR}/sysroots" +do_populate_sysroot[sstate-inputdirs] =Quoted 2 lines
"${SYSROOT_DESTDIR}/${base_prefix}" +do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/"
The reason why is I'm still not 100% comfortable with the two lines above. They look unbalanced and should really be paired to match such that they'd either be:
do_populate_sysroot[sstate-inputdirs] =
"${SYSROOT_DESTDIR}/${base_prefix}"
do_populate_sysroot[sstate-outputdirs] =
"${STAGING_DIR_HOST}/${base_prefix}"or
do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}" do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}"
but just looking at code I'm having trouble working out whether either, both or neither of these would work and in which cases.
The cross/crosssdk recipes are a mess as they install into two different sysroots. Dexuan is working on some changes to improve that so I'm least worried about those two classes and the patch I just merged for libgcc is going to help. I'm therefore aiming for this to work for the target/native/nativesdk/cross-canadian cases and the cross/crosssdk one is secondary and can be messy for now if needed.
Quoted 4 lines
I feel most comfortable aiming for the latter of my two options above, does that work assuming we hack cross/crosssdk to do whatever is needed to make them work with it? Can the others then remain unaltered?
${base_prefix} has some kinds of values considerring different recipe type. (we do not discuss cross and crosssdk)
Normal target recipe:
base_prefix= ""
Native recipes:base_prefix= "${STAGING_DIR_NATIVE}" Nativesdk recipes:base_prefix= "${SDKPATHNATIVE}" Cross-canadian recipes:base_prefix= "${SDKPATHNATIVE}"For target recipes, the input and output dirs should be:
Input: ${SYSROOT_DESTDIR}
Output: ${STAGING_DIR_HOST}For native recipes, the input and output dirs should be:
Input: ${SYSROOT_DESTDIR}/${STAGING_DIR_NATIVE}
Output: "${STAGING_DIR_NATIVE}"For nativesdk recipes, the input and output dirs should be:
Input: ${SYSROOT_DESTDIR}/${SDKPATHNATIVE}
Output: "${STAGING_DIR_HOST}/${SDKPATHNATIVE}For cross-canadian recipes, the input and output dirs should be:
Input: ${SYSROOT_DESTDIR}/${SDKPATHNATIVE}
Output: "${STAGING_DIR_HOST}/${SDKPATHNATIVE}
Quoted 5 lines
It is workable if change to your latter option. do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}" do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}"
In this case we need to specially handle native cases.
Native case could also be handled by that pair, since for native, STAGING_DIR_HOST is "".
Thanks much for your hint, this simplifies the patch.
-- Dongxiao
Quoted 18 lines
Quoted 8 lines
The other minor thing I'm thinking about is adding a definition of:
bindir_crossscripts= "${bindir}/crossscripts"to bitbake.conf (near the top next to
bindir_cross) and using that in the sysroot functions.OK, I will change accordingly.
Thanks,
DongxiaoQuoted 4 lines
Cheers,
Richard
PATCH v4 2/5 staging: relocate *.la paths in destination dirs
In reply to Dongxiao XuKeep the la files in source directories unchanged, and do relocation in destination directories (sysroot-destdir).
--- meta/classes/staging.bbclass | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-)
meta/classes/staging.bbclass +4 -3
@@ -25,8 +25,10 @@ sysroot_stage_libdir() { src="$1" dest="$2" + sysroot_stage_dir $src $dest + olddir=`pwd` - cd $src + cd $dest las=$(find . -name \*.la -type f) cd $olddir echo "Found la files: $las"
@@ -36,9 +38,8 @@ sysroot_stage_libdir() { -e '/^dependency_libs=/s,${WORKDIR}[[:alnum:]/\._+-]*/\([[:alnum:]\._+-]*\),${STAGING_LIBDIR}/\1,g' \ -e "/^dependency_libs=/s,\([[:space:]']\)${libdir},\1${STAGING_LIBDIR},g" \ -e "/^dependency_libs=/s,\([[:space:]']\)${base_libdir},\1${STAGING_DIR_HOST}${base_libdir},g" \ - -i $src/$i + -i $dest/$i done - sysroot_stage_dir $src $dest } sysroot_stage_dirs() {
Signature · 1 line
1.6.3.3
Re: [PATCH 2/5] staging: relocate *.la paths in destination dirs
Quoted 4 lines · Dongxiao Xu
Keep the la files in source directories unchanged, and do relocation in destination directories (sysroot-destdir).
Signed-off-by Dongxiao Xu <[email protected]>
This is a standalone change and makes sense so I've merged this one into master.
Cheers,
Richard
PATCH v4 3/5 bitbake: Introduce stamp-extra-info into build stamp file
In reply to Dongxiao XuFor certain tasks, we need additional information in build stamp file except the task name and file name. stamp-extra-info is introduced as an flag adding to the end of stamp file name.
Besides, if we need to add common flags for tasks, we can set value for macro BB_STAMP_EXTRA.
--- bitbake/lib/bb/build.py | 14 +++++--------- bitbake/lib/bb/cache.py | 19 ++++++++++++++++++- bitbake/lib/bb/runqueue.py | 12 ++++++------ bitbake/lib/bb/siggen.py | 20 ++++++++++++++++++-- meta/classes/base.bbclass | 2 +- 5 files changed, 48 insertions(+), 19 deletions(-)
bitbake/lib/bb/build.py +5 -9
@@ -383,15 +383,11 @@ def stamp_internal(taskname, d, file_name): When called in task context, d will be a data store, file_name will not be set """ if file_name: - stamp = d.stamp[file_name] + is_dict = False else: - stamp = d.getVar('STAMP', True) - file_name = d.getVar('BB_FILENAME', True) + is_dict = True - if not stamp: - return - - stamp = bb.parse.siggen.stampfile(stamp, file_name, taskname) + stamp = bb.parse.siggen.stampfile(d, file_name, taskname, is_dict) bb.utils.mkdirhier(os.path.dirname(stamp))
@@ -420,8 +416,8 @@ def del_stamp(task, d, file_name = None): if os.access(stamp, os.F_OK): os.remove(stamp) -def stampfile(taskname, d): - return stamp_internal(taskname, d, None) +def stampfile(taskname, d, file_name): + return stamp_internal(taskname, d, file_name) def add_tasks(tasklist, d): task_deps = data.getVar('_task_deps', d)
bitbake/lib/bb/cache.py +18 -1
@@ -55,6 +55,8 @@ recipe_fields = ( 'provides', 'task_deps', 'stamp', + 'stamp_extra', + 'stamp_extra_info', 'broken', 'not_world', 'skipped',
@@ -102,6 +104,11 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)): for task in tasks) @classmethod + def stampvar(cls, var, tasks, stamp, metadata): + return dict((stamp + '.' + task, metadata.getVarFlag(task, var, True)) + for task in tasks) + + @classmethod def getvar(cls, var, metadata): return metadata.getVar(var, True) or ''
@@ -126,6 +133,7 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)): if not pn in packages: packages.append(pn) + stamp = cls.getvar('STAMP', metadata) return RecipeInfo( tasks = tasks, basetaskhashes = cls.taskvar('BB_BASEHASH', tasks, metadata),
@@ -147,7 +155,9 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)): defaultpref = cls.intvar('DEFAULT_PREFERENCE', metadata), broken = cls.getvar('BROKEN', metadata), not_world = cls.getvar('EXCLUDE_FROM_WORLD', metadata), - stamp = cls.getvar('STAMP', metadata), + stamp = stamp, + stamp_extra = cls.getvar('BB_STAMP_EXTRA', metadata), + stamp_extra_info = cls.stampvar('stamp-extra-info', tasks, stamp, metadata), packages_dynamic = cls.listvar('PACKAGES_DYNAMIC', metadata), depends = cls.depvar('DEPENDS', metadata), provides = cls.depvar('PROVIDES', metadata),
@@ -562,6 +572,8 @@ class CacheData(object): self.task_queues = {} self.task_deps = {} self.stamp = {} + self.stamp_extra = {} + self.stamp_extra_info = {} self.preferred = {} self.tasks = {} self.basetaskhash = {}
@@ -577,12 +589,17 @@ class CacheData(object): self.bbfile_config_priorities = [] def add_from_recipeinfo(self, fn, info): + tasks = info.tasks self.task_deps[fn] = info.task_deps self.pkg_fn[fn] = info.pn self.pkg_pn[info.pn].append(fn) self.pkg_pepvpr[fn] = (info.pe, info.pv, info.pr) self.pkg_dp[fn] = info.defaultpref self.stamp[fn] = info.stamp + self.stamp_extra[fn] = info.stamp_extra + + for task in tasks: + self.stamp_extra_info[self.stamp[fn] + '.' + task] = info.stamp_extra_info[self.stamp[fn] + '.' + task] provides = [info.pn] for provide in info.provides:
bitbake/lib/bb/runqueue.py +6 -6
@@ -794,7 +794,7 @@ class RunQueue: continue fn = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[task]] taskname = self.rqdata.runq_task[task] - stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname) + stampfile = bb.build.stampfile(taskname, self.rqdata.dataCache, fn) # If the stamp is missing its not current if not os.access(stampfile, os.F_OK): del unchecked[task]
@@ -815,7 +815,7 @@ class RunQueue: if task in unchecked: fn = self.taskData.fn_index[self.rqdata.runq_fnid[task]] taskname = self.rqdata.runq_task[task] - stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname) + stampfile = bb.build.stampfile(taskname, self.rqdata.dataCache, fn) iscurrent = True t1 = os.stat(stampfile)[stat.ST_MTIME]
@@ -823,7 +823,7 @@ class RunQueue: if iscurrent: fn2 = self.taskData.fn_index[self.rqdata.runq_fnid[dep]] taskname2 = self.rqdata.runq_task[dep] - stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2) + stampfile2 = bb.build.stampfile(taskname2, self.rqdata.dataCache, fn2) if fn == fn2 or (fulldeptree and fn2 not in stampwhitelist): if dep in notcurrent: iscurrent = False
@@ -875,7 +875,7 @@ class RunQueue: if taskname is None: taskname = self.rqdata.runq_task[task] - stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname) + stampfile = bb.build.stampfile(taskname, self.rqdata.dataCache, fn) # If the stamp is missing its not current if not os.access(stampfile, os.F_OK):
@@ -896,8 +896,8 @@ class RunQueue: if iscurrent: fn2 = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[dep]] taskname2 = self.rqdata.runq_task[dep] - stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2) - stampfile3 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2 + "_setscene") + stampfile2 = bb.build.stampfile(taskname2, self.rqdata.dataCache, fn2) + stampfile3 = bb.build.stampfile(taskname2 + "_setscene", self.rqdata.dataCache, fn2) t2 = get_timestamp(stampfile2) t3 = get_timestamp(stampfile3) if t3 and t3 > t2:
bitbake/lib/bb/siggen.py +18 -2
@@ -42,8 +42,24 @@ class SignatureGenerator(object): def set_taskdata(self, hashes, deps): return - def stampfile(self, stampbase, file_name, taskname): - return "%s.%s" % (stampbase, taskname) + def stampfile(self, d, file_name, taskname, is_dict): + if is_dict: + stamp = d.getVar('STAMP', True) + file_name = d.getVar('BB_FILENAME', True) + extra_info = taskname + '.' + (bb.data.getVarFlag(taskname, 'stamp-extra-info', d) or bb.data.getVar('BB_STAMP_EXTRA', d, True) or "") + extra_info = bb.data.expand(extra_info, d) + else: + stamp = d.stamp[file_name] + if stamp + "." + taskname in d.stamp_extra_info.keys(): + extra_info = taskname + '.' + (d.stamp_extra_info[stamp + "." + taskname] or d.stamp_extra[file_name] or "") + else: + extra_info = taskname + '.' + (d.stamp_extra[file_name] or "") + + if not stamp: + return + + return "%s.%s" % (stamp, extra_info.rstrip('.')) + class SignatureGeneratorBasic(SignatureGenerator): """
meta/classes/base.bbclass +1 -1
@@ -107,7 +107,7 @@ python base_scenefunction () { python base_do_setscene () { for f in (bb.data.getVar('SCENEFUNCS', d, 1) or '').split(): bb.build.exec_func(f, d) - if not os.path.exists(bb.build.stampfile("do_setscene", d)): + if not os.path.exists(bb.build.stampfile("do_setscene", d, None)): bb.build.make_stamp("do_setscene", d) } do_setscene[selfstamp] = "1"
Signature · 1 line
1.6.3.3
Re: [PATCH 3/5] bitbake: Introduce stamp-extra-info into build stamp file
Quoted 7 lines · Dongxiao Xu
For certain tasks, we need additional information in build stamp file except the task name and file name. stamp-extra-info is introduced as an flag adding to the end of stamp file name.
Besides, if we need to add common flags for tasks, we can set value for macro
BB_STAMP_EXTRA.
Quoted 8 lines
Signed-off-by Dongxiao Xu <[email protected]>--- bitbake/lib/bb/build.py | 14 +++++--------- bitbake/lib/bb/cache.py | 19 ++++++++++++++++++- bitbake/lib/bb/runqueue.py | 12 ++++++------ bitbake/lib/bb/siggen.py | 20 ++++++++++++++++++-- meta/classes/base.bbclass | 2 +- 5 files changed, 48 insertions(+), 19 deletions(-)
I like what this patch does but I've ended up changing the code to be a little different. I'll comment below on what those changes are and why I've made them. I've merged the changed code into master and it effectively behaves the same as your version for the needs of what we're doing.
Quoted 10 lines
bitbake/lib/bb/build.py +1
-1@@ -383,15 +383,11 @@ def stamp_internal(taskname, d, file_name): When called in task context, d will be a data store, file_name will not be set """ if file_name: - stamp = d.stamp[file_name] + is_dict = False
bitbake/lib/bb/build.py def stamp_internal(taskname, d, file_name): · line 390
Quoted 10 lines
bitbake/lib/bb/build.py +2
-6else: - stamp = d.getVar('STAMP', True) - file_name = d.getVar('BB_FILENAME', True) + is_dict = True - if not stamp: - return - - stamp = bb.parse.siggen.stampfile(stamp, file_name, taskname) + stamp = bb.parse.siggen.stampfile(d, file_name, taskname, is_dict)
I placed the dataCache vs dataDict code here rather than in siggen since those differences aren't something that should concern the siggen code and this is the correct abstraction point.
bitbake/lib/bb/build.py def del_stamp(task, d, file_name = None): · line 420
Quoted 10 lines
bitbake/lib/bb/build.py +2
-2bb.utils.mkdirhier(os.path.dirname(stamp))@@ -420,8 +416,8 @@ def del_stamp(task, d, file_name = None): if os.access(stamp, os.F_OK): os.remove(stamp) -def stampfile(taskname, d): - return stamp_internal(taskname, d, None) +def stampfile(taskname, d, file_name): + return stamp_internal(taskname, d, file_name)
Here, we can do stampfile(taskname, d, file_name = None) and then remove the need for the base.bbclass change in a similar way to the other stamp functions.
Quoted 7 lines
bitbake/lib/bb/build.py +0
-0def add_tasks(tasklist, d): task_deps = data.getVar('_task_deps', d)bitbake/lib/bb/cache.py +0
-0
bitbake/lib/bb/cache.py class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)): · line 111
Quoted 8 lines
bitbake/lib/bb/cache.py +5
-0@@ -102,6 +104,11 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)): for task in tasks) @classmethod + def stampvar(cls, var, tasks, stamp, metadata): + return dict((stamp + '.' + task, metadata.getVarFlag(task, var, True)) + for task in tasks) + + @classmethod
I found this very confusing. Why do we need to do all these string operations joining task to stamp and so on? Why can't we just store a list keyed by task?
The point of the class abstractions is to share useful functionality. I've changed this to a flaglist method which pulls a set of flags from a given list of variables.
Quoted 12 lines
bitbake/lib/bb/cache.py +1
-0def getvar(cls, var, metadata): return metadata.getVar(var, True) or ''@@ -126,6 +133,7 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)): if not pn in packages: packages.append(pn) + stamp = cls.getvar('STAMP', metadata) return RecipeInfo( tasks = tasks, basetaskhashes = cls.taskvar('BB_BASEHASH', tasks, metadata),
bitbake/lib/bb/cache.py class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)): · line 163
Quoted 10 lines
bitbake/lib/bb/cache.py +3
-1@@ -147,7 +155,9 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)): defaultpref = cls.intvar('DEFAULT_PREFERENCE', metadata), broken = cls.getvar('BROKEN', metadata), not_world = cls.getvar('EXCLUDE_FROM_WORLD', metadata), - stamp = cls.getvar('STAMP', metadata), + stamp = stamp, + stamp_extra = cls.getvar('BB_STAMP_EXTRA', metadata), + stamp_extra_info = cls.stampvar('stamp-extra-info', tasks, stamp, metadata), packages_dynamic = cls.listvar('PACKAGES_DYNAMIC', metadata), depends = cls.depvar('DEPENDS', metadata), provides = cls.depvar('PROVIDES', metadata),
I've removed BB_STAMP_EXTRA since we don't have a use for it at the moment and it was just complicating the code without good reason. If we do come up with a usecase we can add it easily enough.
Quoted 5 lines
bitbake/lib/bb/cache.py +1
-0@@ -577,12 +589,17 @@ class CacheData(object): self.bbfile_config_priorities = [] def add_from_recipeinfo(self, fn, info): + tasks = info.tasks
bitbake/lib/bb/cache.py class CacheData(object): · line 602
Quoted 10 lines
bitbake/lib/bb/cache.py +4
-0self.task_deps[fn] = info.task_deps self.pkg_fn[fn] = info.pn self.pkg_pn[info.pn].append(fn) self.pkg_pepvpr[fn] = (info.pe, info.pv, info.pr) self.pkg_dp[fn] = info.defaultpref self.stamp[fn] = info.stamp + self.stamp_extra[fn] = info.stamp_extra + + for task in tasks: + self.stamp_extra_info[self.stamp[fn] + '.' + task] = info.stamp_extra_info[self.stamp[fn] + '.' + task]
This is needlessly complicated.
Quoted 18 lines
bitbake/lib/bb/siggen.py +8
-2@@ -42,8 +42,24 @@ class SignatureGenerator(object): def set_taskdata(self, hashes, deps): return - def stampfile(self, stampbase, file_name, taskname): - return "%s.%s" % (stampbase, taskname) + def stampfile(self, d, file_name, taskname, is_dict): + if is_dict: + stamp = d.getVar('STAMP', True) + file_name = d.getVar('BB_FILENAME', True) + extra_info = taskname + '.' + (bb.data.getVarFlag(taskname, 'stamp-extra-info', d) or bb.data.getVar('BB_STAMP_EXTRA', d, True) or "") + extra_info = bb.data.expand(extra_info, d) + else: + stamp = d.stamp[file_name]
bitbake/lib/bb/siggen.py class SignatureGenerator(object): · line 62
Quoted 10 lines
bitbake/lib/bb/siggen.py +10
-0+ if stamp + "." + taskname in d.stamp_extra_info.keys(): + extra_info = taskname + '.' + (d.stamp_extra_info[stamp + "." + taskname] or d.stamp_extra[file_name] or "") + else: + extra_info = taskname + '.' + (d.stamp_extra[file_name] or "") + + if not stamp: + return + + return "%s.%s" % (stamp, extra_info.rstrip('.')) +
Again, I think this is overcomplicated. Also, you changed this version of the function but not the one in SignatureGeneratorBasicHash().
I'm hoping I didn't manage to break anything with my changes, let me know if I have missed anything though.
Cheers,
Richard
PATCH v4 4/5 bitbake: machine specific sysroots implementation
In reply to Dongxiao XuThis commit changes the sysroots path to be machine specific.
Changes includes:
1) STAGING_DIR_TARGET and STRAGING_DIR_HOST points to machine specific paths.
2) task stamp files. Adding ${MACHINE} info into stamp files for do_populate_sysroots and do_package tasks. Add a BB_STAMPTASK_BLACKLIST to keep native, nativesdk, crosssdk, and cross-canadian stamp unchanged.
3) siteconfig path. Separate the site config path for different machines to avoid one machine adopting the cache file of another machine.
4) sstate stamp file. Add machine name to sstate stamp file since some prebuild result could not be shared after sysroots are implemented as machine spedific. Keep native, nativesdk, crosssdk, and cross-canadian unchanged.
5) toolchain scripts. Change the environment path to point to machine specific sysroots in toolchain scripts bbclass.
6) Relocate la files when populating to a different machine of the same architecture.
--- meta/classes/cross-canadian.bbclass | 3 ++- meta/classes/cross.bbclass | 5 +++++ meta/classes/crosssdk.bbclass | 2 ++ meta/classes/native.bbclass | 2 ++ meta/classes/nativesdk.bbclass | 5 +++++ meta/classes/package.bbclass | 1 + meta/classes/siteconfig.bbclass | 6 +++--- meta/classes/sstate.bbclass | 6 ++++-- meta/classes/staging.bbclass | 3 +++ meta/classes/toolchain-scripts.bbclass | 4 ++-- meta/conf/bitbake.conf | 11 ++++++----- 11 files changed, 35 insertions(+), 13 deletions(-)
meta/classes/cross-canadian.bbclass +2 -1
@@ -30,7 +30,6 @@ MULTIMACH_TARGET_SYS = "${MULTIMACH_ARCH}${HOST_VENDOR}-${HOST_OS}" INHIBIT_DEFAULT_DEPS = "1" STAGING_DIR_HOST = "${STAGING_DIR}/${HOST_SYS}-nativesdk" -STAGING_DIR_TARGET = "${STAGING_DIR}/${OLD_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}" TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR}/${HOST_ARCH}-nativesdk${HOST_VENDOR}-${HOST_OS}"
@@ -84,3 +83,5 @@ export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR_HOST}" # Cross-canadian packages need to pull in nativesdk dynamic libs SHLIBSDIR = "${STAGING_DIR}/${SDK_ARCH}-nativesdk${SDK_VENDOR}-${BUILD_OS}/shlibs" +do_populate_sysroot[stamp-extra-info] = "" +do_package[stamp-extra-info] = ""
meta/classes/cross.bbclass +5 -0
@@ -20,6 +20,11 @@ HOST_OS = "${BUILD_OS}" HOST_PREFIX = "${BUILD_PREFIX}" HOST_CC_ARCH = "${BUILD_CC_ARCH}" +STAGING_DIR_HOST = "${STAGING_DIR}/${BASEPKG_HOST_SYS}" + +export PKG_CONFIG_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}${libdir}/pkgconfig" +export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}" + CPPFLAGS = "${BUILD_CPPFLAGS}" CFLAGS = "${BUILD_CFLAGS}" CXXFLAGS = "${BUILD_CFLAGS}"
meta/classes/crosssdk.bbclass +2 -0
@@ -18,3 +18,5 @@ target_base_libdir = "${SDKPATHNATIVE}${base_libdir_nativesdk}" target_prefix = "${SDKPATHNATIVE}${prefix_nativesdk}" target_exec_prefix = "${SDKPATHNATIVE}${exec_prefix_nativesdk}" +do_populate_sysroot[stamp-extra-info] = "" +do_package[stamp-extra-info] = ""
meta/classes/native.bbclass +2 -0
@@ -119,3 +119,5 @@ do_package_write_ipk[noexec] = "1" do_package_write_deb[noexec] = "1" do_package_write_rpm[noexec] = "1" +do_populate_sysroot[stamp-extra-info] = "" +do_package[stamp-extra-info] = ""
meta/classes/nativesdk.bbclass +5 -0
@@ -19,6 +19,8 @@ python () { #STAGING_DIR_HOST = "${STAGING_DIR}/${HOST_SYS}-nativesdk" #STAGING_DIR_TARGET = "${STAGING_DIR}/${BASEPKG_TARGET_SYS}-nativesdk" +STAGING_DIR_HOST = "${STAGING_DIR}/${BASEPKG_HOST_SYS}" +STAGING_DIR_TARGET = "${STAGING_DIR}/${BASEPKG_TARGET_SYS}" HOST_ARCH = "${SDK_ARCH}" HOST_VENDOR = "${SDK_VENDOR}"
@@ -80,3 +82,6 @@ python __anonymous () { bb.data.setVar("PROVIDES", provides, d) bb.data.setVar("OVERRIDES", bb.data.getVar("OVERRIDES", d, False) + ":virtclass-nativesdk", d) } + +do_populate_sysroot[stamp-extra-info] = "" +do_package[stamp-extra-info] = ""
meta/classes/package.bbclass +1 -0
@@ -1091,6 +1091,7 @@ do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}" do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}" do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}" do_package[sstate-lockfile] = "${PACKAGELOCK}" +do_package[stamp-extra-info] = "${MACHINE}" do_package_setscene[dirs] = "${STAGING_DIR}" python do_package_setscene () {
meta/classes/siteconfig.bbclass +3 -3
@@ -11,10 +11,10 @@ python siteconfig_do_siteconfig () { } siteconfig_do_siteconfig_gencache () { - mkdir -p ${WORKDIR}/site_config + mkdir -p ${WORKDIR}/site_config_${MACHINE} gen-site-config ${FILE_DIRNAME}/site_config \ - >${WORKDIR}/site_config/configure.ac - cd ${WORKDIR}/site_config + >${WORKDIR}/site_config_${MACHINE}/configure.ac + cd ${WORKDIR}/site_config_${MACHINE} autoconf CONFIG_SITE="" ./configure ${CONFIGUREOPTS} --cache-file ${PN}_cache sed -n -e "/ac_cv_c_bigendian/p" -e "/ac_cv_sizeof_/p" \
meta/classes/sstate.bbclass +4 -2
@@ -5,7 +5,7 @@ SSTATE_MANFILEBASE = "${SSTATE_MANIFESTS}/manifest-${SSTATE_PKGARCH}-" SSTATE_MANFILEPREFIX = "${SSTATE_MANFILEBASE}${PN}" -SSTATE_PKGARCH = "${MULTIMACH_ARCH}" +SSTATE_PKGARCH = "${MULTIMACH_ARCH}_${MACHINE}" SSTATE_PKGSPEC = "sstate-${PN}-${MULTIMACH_ARCH}${TARGET_VENDOR}-${TARGET_OS}-${PV}-${PR}-${SSTATE_PKGARCH}-${SSTATE_VERSION}-" SSTATE_PKGNAME = "${SSTATE_PKGSPEC}${BB_TASKHASH}" SSTATE_PKG = "${SSTATE_DIR}/${SSTATE_PKGNAME}"
@@ -17,7 +17,9 @@ BB_HASHFILENAME = "${SSTATE_PKGNAME}" python () { if bb.data.inherits_class('native', d): bb.data.setVar('SSTATE_PKGARCH', bb.data.getVar('BUILD_ARCH', d), d) - elif bb.data.inherits_class('cross', d) or bb.data.inherits_class('crosssdk', d): + elif bb.data.inherits_class('cross', d): + bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${BUILD_ARCH}_${BASE_PACKAGE_ARCH}_${MACHINE}", d), d) + elif bb.data.inherits_class('crosssdk', d): bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${BUILD_ARCH}_${BASE_PACKAGE_ARCH}", d), d) elif bb.data.inherits_class('nativesdk', d): bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${SDK_ARCH}", d), d)
meta/classes/staging.bbclass +3 -0
@@ -36,6 +36,8 @@ sysroot_stage_libdir() { do sed -e 's/^installed=yes$/installed=no/' \ -e '/^dependency_libs=/s,${WORKDIR}[[:alnum:]/\._+-]*/\([[:alnum:]\._+-]*\),${STAGING_LIBDIR}/\1,g' \ + -e "/^dependency_libs=/s,\(\-[LR]\)${STAGING_DIR}/[[:alnum:]\._+-]*${libdir},\1${STAGING_LIBDIR},g" \ + -e "/^dependency_libs=/s,\(\-[LR]\)${STAGING_DIR}/[[:alnum:]\._+-]*${base_libdir},\1${STAGING_DIR_HOST},g" \ -e "/^dependency_libs=/s,\([[:space:]']\)${libdir},\1${STAGING_LIBDIR},g" \ -e "/^dependency_libs=/s,\([[:space:]']\)${base_libdir},\1${STAGING_DIR_HOST}${base_libdir},g" \ -i $dest/$i
@@ -107,6 +109,7 @@ SSTATETASKS += "do_populate_sysroot" do_populate_sysroot[sstate-name] = "populate-sysroot" do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${base_prefix}" do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/" +do_populate_sysroot[stamp-extra-info] = "${MACHINE}" python do_populate_sysroot_setscene () { sstate_setscene(d)
meta/classes/toolchain-scripts.bbclass +2 -2
@@ -49,8 +49,8 @@ toolchain_create_tree_env_script () { echo 'export TARGET_PREFIX=${TARGET_PREFIX}' >> $script echo 'export CONFIGURE_FLAGS="--target=${TARGET_SYS} --host=${TARGET_SYS} --build=${BUILD_SYS}"' >> $script if [ "${TARGET_OS}" = "darwin8" ]; then - echo 'export TARGET_CFLAGS="-I${STAGING_DIR}${TARGET_SYS}${includedir}"' >> $script - echo 'export TARGET_LDFLAGS="-L${STAGING_DIR}${TARGET_SYS}${libdir}"' >> $script + echo 'export TARGET_CFLAGS="-I${STAGING_DIR}${MACHINE}${includedir}"' >> $script + echo 'export TARGET_LDFLAGS="-L${STAGING_DIR}${MACHINE}${libdir}"' >> $script # Workaround darwin toolchain sysroot path problems cd ${SDK_OUTPUT}${SDKTARGETSYSROOT}/usr ln -s /usr/local local
meta/conf/bitbake.conf +6 -5
@@ -276,7 +276,7 @@ STAGING_DATADIR_NATIVE = "${STAGING_DIR_NATIVE}${datadir_native}" # This should really be MULTIMACH_HOST_SYS but that breaks "all" and machine # specific packages - hack around it for now. -STAGING_DIR_HOST = "${STAGING_DIR}/${BASEPKG_HOST_SYS}" +STAGING_DIR_HOST = "${STAGING_DIR}/${MACHINE}" STAGING_BINDIR = "${STAGING_DIR_HOST}${bindir}" STAGING_LIBDIR = "${STAGING_DIR_HOST}${libdir}" STAGING_INCDIR = "${STAGING_DIR_HOST}${includedir}"
@@ -288,7 +288,7 @@ STAGING_PYDIR = "${STAGING_DIR}/lib/python2.4" # This should really be MULTIMACH_TARGET_SYS but that breaks "all" and machine # specific packages - hack around it for now. -STAGING_DIR_TARGET = "${STAGING_DIR}/${BASEPKG_TARGET_SYS}" +STAGING_DIR_TARGET = "${STAGING_DIR}/${MACHINE}" # Setting DEPLOY_DIR outside of TMPDIR is helpful, when you are using # packaged staging and/or multimachine.
@@ -311,7 +311,7 @@ SDKPATHNATIVE = "${SDKPATH}/sysroots/${SDK_SYS}" ################################################################## OLDEST_KERNEL = "2.4.0" -STAGING_KERNEL_DIR = "${STAGING_DIR}/${MULTIMACH_TARGET_SYS}/kernel" +STAGING_KERNEL_DIR = "${STAGING_DIR}/${MACHINE}/kernel" ################################################################## # Specific image creation and rootfs population info.
@@ -596,10 +596,10 @@ SLOT = "0" # Other -export PKG_CONFIG_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}${libdir}/pkgconfig" +export PKG_CONFIG_DIR = "${STAGING_DIR}/${MACHINE}/${libdir}/pkgconfig" export PKG_CONFIG_PATH = "${PKG_CONFIG_DIR}:${STAGING_DATADIR}/pkgconfig" export PKG_CONFIG_LIBDIR = "${PKG_CONFIG_DIR}" -export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}" +export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR}/${MACHINE}" export PKG_CONFIG_DISABLE_UNINSTALLED = "yes" export QMAKE_MKSPEC_PATH = "${STAGING_DATADIR_NATIVE}/qmake"
@@ -732,3 +732,4 @@ BB_SIGNATURE_HANDLER ?= "basic" BB_HASHTASK_WHITELIST ?= "(.*-cross$|.*-native$|.*-cross-initial$|.*-cross-intermediate$|^virtual:native:.*|^virtual:nativesdk:.*)" BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM USER" +BB_STAMPTASK_BLACKLIST ?= "(.*-native_*|.*-nativesdk_*|^virtual:native:*|^virtual:nativesdk:*|.*-crosssdk_*|.*-cross-canadian-*)"
Signature · 1 line
1.6.3.3
PATCH v4 5/5 emenlow: Change PACKAGE_EXTRA_ARCHS and BASE_PACKAGE_ARCH to core2
In reply to Dongxiao XuAfter machine specific sysroot is implemented, emenlow and atom-pc could build together as one architecture. Thus change emenlow architecture back to core2.
--- meta-emenlow/conf/machine/emenlow.conf | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-)
meta-emenlow/conf/machine/emenlow.conf +3 -3
@@ -5,11 +5,11 @@ # Webs-2120 box. TARGET_ARCH = "i586" -PACKAGE_EXTRA_ARCHS = "x86 emen" +PACKAGE_EXTRA_ARCHS = "x86 core2" include conf/machine/include/tune-atom.inc -BASE_PACKAGE_ARCH="emen" +BASE_PACKAGE_ARCH="core2" MACHINE_FEATURES = "kernel26 screen keyboard pci usbhost ext2 ext3 x86 \ acpi serial usbgadget"
@@ -45,4 +45,4 @@ GLIBC_ADDONS = "nptl" GLIBC_EXTRA_OECONF = "--with-tls" SRCREV_machine_pn-linux-yocto-stable_emenlow ?= "aae69fdf104b0a9d7b3710f808aac6ab303490f7" -SRCREV_meta_pn-linux-yocto-stable_emenlow ?= "50ccd2b3213b6a1bacb3f898c035119802dac420" +SRCREV_meta_pn-linux-yocto-stable_emenlow ?= "50ccd2b3213b6a1bacb3f898c035119802dac420"
Signature · 1 line
1.6.3.3