Skip to contentSkip to conversation

Machine specific sysroot implementation

RFC PATCH In review Dongxiao Xu 3 participants 6 messages Jan 11, 2011 – Jan 12, 2011

lore
Share & tools
lore
#0 lore raw

cover Machine specific sysroot implementation

Hi Richard,

This RFC is the third version of machine specific sysroot implementation. Please help to review.

Testings could pass for qemuppc and mpc8315d-rdb.

However if modify emenlow architecture to be "core2" and test builds for atom-pc and emenlow, it failed due to an libtool sysroot issue, which is stated in another email. But I think we can have a review of the patchset first.

After the libtool sysroot support is enabled, I will test again for atom-pc and emenlow.

Changes from v2:
1) Use the v1 approach to add ${MACHINE} into stamp file, and rebase the patch against latest bitbake which did a merge with upstream. This also can avoid adding unnecessary ${MACHINE} information for native/nativesdk/crosssdk/cross-canadian task stamps.

RFC URL: git://git.pokylinux.org/poky-contrib.git
Branch: dxu4/mach_sysroot_v3
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dxu4/mach_sysroot_v3

Thanks,
Dongxiao Xu <[email protected]>
---

Dongxiao Xu (3):
staging: Use relative path in sysroot-destdir for target recipes
bitbake: Introduce stamp-extra-info into build stamp file
bitbake: machine specific sysroots implementation

 bitbake/lib/bb/build.py                            |   10 ++-----
 bitbake/lib/bb/cache.py                            |   19 ++++++++++++-
 bitbake/lib/bb/runqueue.py                         |   12 ++++----
 bitbake/lib/bb/siggen.py                           |   20 ++++++++++++-
 meta/classes/binconfig.bbclass                     |    6 ++--
 meta/classes/cross-canadian.bbclass                |    5 +++-
 meta/classes/cross.bbclass                         |    5 +++
 meta/classes/crosssdk.bbclass                      |    2 +
 meta/classes/kernel.bbclass                        |    2 +-
 meta/classes/native.bbclass                        |    4 +++
 meta/classes/nativesdk.bbclass                     |    7 ++++-
 meta/classes/package.bbclass                       |    1 +
 meta/classes/siteconfig.bbclass                    |   11 ++++---
 meta/classes/sstate.bbclass                        |    6 +++-
 meta/classes/staging.bbclass                       |   28 ++++++++++---------
 meta/classes/toolchain-scripts.bbclass             |    4 +-
 meta/conf/bitbake.conf                             |   11 ++++---
 meta/recipes-connectivity/gupnp/gupnp_0.14.0.bb    |    4 +-
 .../gcc/gcc-cross-intermediate.inc                 |    3 ++
 meta/recipes-devtools/libtool/libtool-cross_2.4.bb |    4 +-
 .../libtool/libtool-nativesdk_2.4.bb               |    4 +-
 meta/recipes-devtools/libtool/libtool_2.4.bb       |    6 ++--
 meta/recipes-gnome/gtk+/gtk+.inc                   |    4 +-
 meta/recipes-support/apr/apr_1.3.3.bb              |    2 +-
 24 files changed, 119 insertions(+), 61 deletions(-)
#1 lore raw

PATCH 1/3 staging: Use relative path in sysroot-destdir for target recipes

Original we used absolute path in sysroot-destdir for both native and target recipes. This commit changes target recipes to use relative path which is same as the image directory.

---
 meta/classes/binconfig.bbclass                     |    6 ++--
 meta/classes/cross-canadian.bbclass                |    2 +
 meta/classes/kernel.bbclass                        |    2 +-
 meta/classes/native.bbclass                        |    2 +
 meta/classes/nativesdk.bbclass                     |    4 +-
 meta/classes/siteconfig.bbclass                    |    5 ++-
 meta/classes/staging.bbclass                       |   27 ++++++++++---------
 meta/recipes-connectivity/gupnp/gupnp_0.14.0.bb    |    4 +-
 .../gcc/gcc-cross-intermediate.inc                 |    3 ++
 meta/recipes-devtools/libtool/libtool-cross_2.4.bb |    4 +-
 .../libtool/libtool-nativesdk_2.4.bb               |    4 +-
 meta/recipes-devtools/libtool/libtool_2.4.bb       |    6 ++--
 meta/recipes-gnome/gtk+/gtk+.inc                   |    4 +-
 meta/recipes-support/apr/apr_1.3.3.bb              |    2 +-
 14 files changed, 42 insertions(+), 33 deletions(-)
meta/classes/binconfig.bbclass +3 -3
index 73ca4d6..7c12bd3 100644
@@ -47,8 +47,8 @@ SYSROOT_PREPROCESS_FUNCS += "binconfig_sysroot_preprocess"
 binconfig_sysroot_preprocess () {
 	for config in `find ${S} -name '${BINCONFIG_GLOB}'`; do
 		configname=`basename $config`
-		install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}
-		cat $config | sed ${@get_binconfig_mangle(d)} > ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/$configname
-		chmod u+x ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/$configname
+		install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts
+		cat $config | sed ${@get_binconfig_mangle(d)} > ${SYSROOT_DESTDIR}${bindir}/crossscripts/$configname
+		chmod u+x ${SYSROOT_DESTDIR}${bindir}/crossscripts/$configname
 	done
 }
meta/classes/cross-canadian.bbclass +2 -0
index 3f3a24d..9897105 100644
@@ -71,6 +71,8 @@ base_sbindir = "${bindir}"
 libdir = "${exec_prefix}/lib/${OLD_MULTIMACH_ARCH}${TARGET_VENDOR}-${TARGET_OS}"
 libexecdir = "${exec_prefix}/libexec/${OLD_MULTIMACH_ARCH}${TARGET_VENDOR}-${TARGET_OS}"
 
+do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/${base_prefix}"
+
 FILES_${PN} = "${prefix}"
 FILES_${PN}-dbg += "${prefix}/.debug \
                     ${prefix}/bin/.debug \
meta/classes/kernel.bbclass +1 -1
index 86a1923..c8303ae 100644
@@ -185,7 +185,7 @@ kernel_do_install() {
 }
 
 sysroot_stage_all_append() {
-	sysroot_stage_dir ${D}/kernel ${SYSROOT_DESTDIR}${STAGING_KERNEL_DIR}
+	sysroot_stage_dir ${D}/kernel ${SYSROOT_DESTDIR}/kernel
 }
 
 
meta/classes/native.bbclass +2 -0
index 3ca9d62..4355db2 100644
@@ -63,6 +63,8 @@ exec_prefix = "${STAGING_DIR_NATIVE}${prefix_native}"
 # Since we actually install these into situ there is no staging prefix
 STAGING_DIR_HOST = ""
 STAGING_DIR_TARGET = ""
+do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_NATIVE}"
+
 SHLIBSDIR = "${STAGING_DIR_NATIVE}/shlibs"
 PKG_CONFIG_DIR = "${libdir}/pkgconfig"
 
meta/classes/nativesdk.bbclass +2 -2
index 154bd82..7ed5ca5 100644
@@ -43,6 +43,8 @@ base_prefix = "${SDKPATHNATIVE}"
 prefix = "${SDKPATHNATIVE}${prefix_nativesdk}"
 exec_prefix = "${SDKPATHNATIVE}${prefix_nativesdk}"
 
+do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/${base_prefix}"
+
 FILES_${PN} += "${prefix}"
 FILES_${PN}-dbg += "${prefix}/.debug \
                     ${prefix}/bin/.debug \
@@ -78,5 +80,3 @@ python __anonymous () {
     bb.data.setVar("PROVIDES", provides, d)
     bb.data.setVar("OVERRIDES", bb.data.getVar("OVERRIDES", d, False) + ":virtclass-nativesdk", d)
 }
-
-
meta/classes/siteconfig.bbclass +3 -2
index 37d910e..e7cc9ae 100644
@@ -20,8 +20,9 @@ siteconfig_do_siteconfig_gencache () {
 	sed -n -e "/ac_cv_c_bigendian/p" -e "/ac_cv_sizeof_/p" \
 		-e "/ac_cv_type_/p" -e "/ac_cv_header_/p" -e "/ac_cv_func_/p" \
 		< ${PN}_cache > ${PN}_config
-	mkdir -p ${SYSROOT_DESTDIR}${SITECONFIG_SYSROOTCACHE}
-	cp ${PN}_config ${SYSROOT_DESTDIR}${SITECONFIG_SYSROOTCACHE}
+	mkdir -p ${SYSROOT_DESTDIR}${datadir}/${TARGET_SYS}_config_site.d
+	cp ${PN}_config ${SYSROOT_DESTDIR}${datadir}/${TARGET_SYS}_config_site.d
+
 }
 
 do_populate_sysroot[sstate-interceptfuncs] += "do_siteconfig "
meta/classes/staging.bbclass +14 -13
index 8432565..4d2991b 100644
@@ -45,25 +45,25 @@ sysroot_stage_dirs() {
 	from="$1"
 	to="$2"
 
-	sysroot_stage_dir $from${includedir} $to${STAGING_INCDIR}
+	sysroot_stage_dir $from${includedir} $to${includedir}
 	if [ "${BUILD_SYS}" = "${HOST_SYS}" ]; then
-		sysroot_stage_dir $from${bindir} $to${STAGING_DIR_HOST}${bindir}
-		sysroot_stage_dir $from${sbindir} $to${STAGING_DIR_HOST}${sbindir}
-		sysroot_stage_dir $from${base_bindir} $to${STAGING_DIR_HOST}${base_bindir}
-		sysroot_stage_dir $from${base_sbindir} $to${STAGING_DIR_HOST}${base_sbindir}
-		sysroot_stage_dir $from${libexecdir} $to${STAGING_DIR_HOST}${libexecdir}
-		sysroot_stage_dir $from${sysconfdir} $to${STAGING_DIR_HOST}${sysconfdir}
-		sysroot_stage_dir $from${localstatedir} $to${STAGING_DIR_HOST}${localstatedir}
+		sysroot_stage_dir $from${bindir} $to${bindir}
+		sysroot_stage_dir $from${sbindir} $to${sbindir}
+		sysroot_stage_dir $from${base_bindir} $to${base_bindir}
+		sysroot_stage_dir $from${base_sbindir} $to${base_sbindir}
+		sysroot_stage_dir $from${libexecdir} $to${libexecdir}
+		sysroot_stage_dir $from${sysconfdir} $to${sysconfdir}
+		sysroot_stage_dir $from${localstatedir} $to${localstatedir}
 	fi
 	if [ -d $from${libdir} ]
 	then
-		sysroot_stage_libdir $from/${libdir} $to${STAGING_LIBDIR}
+		sysroot_stage_libdir $from/${libdir} $to${libdir}
 	fi
 	if [ -d $from${base_libdir} ]
 	then
-		sysroot_stage_libdir $from${base_libdir} $to${STAGING_DIR_HOST}${base_libdir}
+		sysroot_stage_libdir $from${base_libdir} $to${base_libdir}
 	fi
-	sysroot_stage_dir $from${datadir} $to${STAGING_DATADIR}
+	sysroot_stage_dir $from${datadir} $to${datadir}
 }
 
 sysroot_stage_all() {
@@ -76,6 +76,7 @@ do_populate_sysroot[dirs] = "${STAGING_DIR_TARGET}/${bindir} ${STAGING_DIR_TARGE
 			     ${STAGING_INCDIR_NATIVE} \
 			     ${STAGING_DATADIR} \
                              ${SYSROOT_DESTDIR}${STAGING_DIR_TARGET} \
+                             ${SYSROOT_DESTDIR}${base_prefix} \
 			     ${S} ${B}"
 
 # Could be compile but populate_sysroot and do_install shouldn't run at the same time
@@ -103,8 +104,8 @@ python do_populate_sysroot () {
 
 SSTATETASKS += "do_populate_sysroot"
 do_populate_sysroot[sstate-name] = "populate-sysroot"
-do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${STAGING_DIR}"
-do_populate_sysroot[sstate-outputdirs] = "${TMPDIR}/sysroots"
+do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${base_prefix}"
+do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/"
 
 python do_populate_sysroot_setscene () {
 	sstate_setscene(d)
meta/recipes-connectivity/gupnp/gupnp_0.14.0.bb +2 -2
index 9beb46e..da9fc92 100644
@@ -23,7 +23,7 @@ FILES_${PN}-dev += "${bindir}/gupnp-binding-tool"
 SYSROOT_PREPROCESS_FUNCS += "gupnp_sysroot_preprocess"
 
 gupnp_sysroot_preprocess () {
-	install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/
-	install -m 755 ${D}${bindir}/gupnp-binding-tool ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/
+	install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts/
+	install -m 755 ${D}${bindir}/gupnp-binding-tool ${SYSROOT_DESTDIR}${bindir}/crossscripts/
 }
 
meta/recipes-devtools/gcc/gcc-cross-intermediate.inc +3 -0
index 72a4241..22bef73 100644
@@ -17,6 +17,9 @@ EXTRA_OECONF = "--with-local-prefix=${STAGING_DIR_TARGET}${target_prefix} \
 		${EXTRA_OECONF_INTERMEDIATE} \
 		${@get_gcc_fpu_setting(bb, d)}"
 
+do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${STAGING_DIR_HOST} ${SYSROOT_DESTDIR}/${STAGING_DIR_TARGET}"
+do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST} ${STAGING_DIR_TARGET}"
+
 do_compile () {
     oe_runmake
 }
meta/recipes-devtools/libtool/libtool-cross_2.4.bb +2 -2
index 6ea96a5..09a8693 100644
@@ -29,6 +29,6 @@ do_install () {
 SYSROOT_PREPROCESS_FUNCS += "libtoolcross_sysroot_preprocess"
 
 libtoolcross_sysroot_preprocess () {
-	install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/
-	install -m 755 ${D}${bindir}/${HOST_SYS}-libtool ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/${HOST_SYS}-libtool
+	install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts
+	install -m 755 ${D}${bindir}/${HOST_SYS}-libtool ${SYSROOT_DESTDIR}${bindir}/crossscripts/${HOST_SYS}-libtool
 }
meta/recipes-devtools/libtool/libtool-nativesdk_2.4.bb +2 -2
index c9f24a9..c17023a 100644
@@ -22,6 +22,6 @@ do_install () {
 SYSROOT_PREPROCESS_FUNCS += "libtoolnativesdk_sysroot_preprocess"
 
 libtoolnativesdk_sysroot_preprocess () {
-	install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/
-	install -m 755 ${D}${bindir}/${HOST_SYS}-libtool ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/${HOST_SYS}-libtool
+	install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts/
+	install -m 755 ${D}${bindir}/${HOST_SYS}-libtool ${SYSROOT_DESTDIR}${bindir}/crossscripts/${HOST_SYS}-libtool
 }
meta/recipes-devtools/libtool/libtool_2.4.bb +3 -3
index b5ec33e..3f73732 100644
@@ -26,9 +26,9 @@ SYSROOT_PREPROCESS_FUNCS += "libtool_sysroot_preprocess"
 
 libtool_sysroot_preprocess () {
 	if [ "${PN}" == "libtool" ]; then
-		rm -rf ${SYSROOT_DESTDIR}${STAGING_DIR_TARGET}${bindir}/*
-		rm -rf ${SYSROOT_DESTDIR}${STAGING_DIR_TARGET}${datadir}/aclocal/*
-		rm -rf ${SYSROOT_DESTDIR}${STAGING_DIR_TARGET}${datadir}/libtool/config/*
+		rm -rf ${SYSROOT_DESTDIR}${bindir}/*
+		rm -rf ${SYSROOT_DESTDIR}${datadir}/aclocal/*
+		rm -rf ${SYSROOT_DESTDIR}${datadir}/libtool/config/*
 	fi
 }
 
meta/recipes-gnome/gtk+/gtk+.inc +2 -2
index 425a21e..bd6fb19 100644
@@ -64,8 +64,8 @@ SYSROOT_PREPROCESS_FUNCS += "gtk_sysroot_preprocess"
 
 gtk_sysroot_preprocess () {
 	if [ -e ${D}${bindir}/gtk-builder-convert ]; then
-		install -d ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/
-		install -m 755 ${D}${bindir}/gtk-builder-convert ${SYSROOT_DESTDIR}${STAGING_BINDIR_CROSS}/
+		install -d ${SYSROOT_DESTDIR}${bindir}/crossscripts/
+		install -m 755 ${D}${bindir}/gtk-builder-convert ${SYSROOT_DESTDIR}${bindir}/crossscripts/
 	fi
 }
 
meta/recipes-support/apr/apr_1.3.3.bb +1 -1
index 26cc01f..fabaae4 100644
@@ -24,7 +24,7 @@ do_configure_prepend() {
 SYSROOT_PREPROCESS_FUNCS += "apr_sysroot_preprocess"
 
 apr_sysroot_preprocess () {
-	d=${SYSROOT_DESTDIR}${STAGING_DIR_TARGET}${datadir}/apr
+	d=${SYSROOT_DESTDIR}${datadir}/apr
 	install -d $d/
 	cp ${S}/build/apr_rules.mk $d/
 	sed -i s,apr_builddir=.*,apr_builddir=,g $d/apr_rules.mk
Signature · 1 line
1.6.3.3
#2 lore raw

PATCH 2/3 bitbake: Introduce stamp-extra-info into build stamp file

In reply to Dongxiao Xu

For certain tasks, we need additional information in build stamp file except the task name and file name. stamp-extra-info is introduced as an flag adding to the end of stamp file name.

Besides, if we need to add common flags for tasks, we can set value for macro BB_STAMP_EXTRA.

---
 bitbake/lib/bb/build.py    |   10 +++-------
 bitbake/lib/bb/cache.py    |   19 ++++++++++++++++++-
 bitbake/lib/bb/runqueue.py |   12 ++++++------
 bitbake/lib/bb/siggen.py   |   20 ++++++++++++++++++--
 4 files changed, 45 insertions(+), 16 deletions(-)
bitbake/lib/bb/build.py +3 -7
index f127796..3b297b3 100644
@@ -383,15 +383,11 @@ def stamp_internal(taskname, d, file_name):
     When called in task context, d will be a data store, file_name will not be set
     """
     if file_name:
-        stamp = d.stamp[file_name]
+        is_dict = 0
     else:
-        stamp = d.getVar('STAMP', True)
-        file_name = d.getVar('BB_FILENAME', True)
+        is_dict = 1
 
-    if not stamp:
-        return
-
-    stamp = bb.parse.siggen.stampfile(stamp, file_name, taskname)
+    stamp = bb.parse.siggen.stampfile(d, file_name, taskname, is_dict)
 
     bb.utils.mkdirhier(os.path.dirname(stamp))
 
bitbake/lib/bb/cache.py +18 -1
index 9a2e2d5..2437d4a 100644
@@ -55,6 +55,8 @@ recipe_fields = (
     'provides',
     'task_deps',
     'stamp',
+    'stamp_extra',
+    'stamp_extra_info',
     'broken',
     'not_world',
     'skipped',
@@ -102,6 +104,11 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)):
                     for task in tasks)
 
     @classmethod
+    def stampvar(cls, var, tasks, stamp, metadata):
+        return dict((stamp + '.' + task, metadata.getVarFlag(task, var, True))
+                    for task in tasks)
+
+    @classmethod
     def getvar(cls, var, metadata):
         return metadata.getVar(var, True) or ''
 
@@ -126,6 +133,7 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)):
         if not pn in packages:
             packages.append(pn)
 
+        stamp = cls.getvar('STAMP', metadata)
         return RecipeInfo(
             tasks            = tasks,
             basetaskhashes   = cls.taskvar('BB_BASEHASH', tasks, metadata),
@@ -147,7 +155,9 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)):
             defaultpref      = cls.intvar('DEFAULT_PREFERENCE', metadata),
             broken           = cls.getvar('BROKEN', metadata),
             not_world        = cls.getvar('EXCLUDE_FROM_WORLD', metadata),
-            stamp            = cls.getvar('STAMP', metadata),
+            stamp            = stamp,
+            stamp_extra      = cls.getvar('BB_STAMP_EXTRA', metadata),
+            stamp_extra_info = cls.stampvar('stamp-extra-info', tasks, stamp, metadata),
             packages_dynamic = cls.listvar('PACKAGES_DYNAMIC', metadata),
             depends          = cls.depvar('DEPENDS', metadata),
             provides         = cls.depvar('PROVIDES', metadata),
@@ -562,6 +572,8 @@ class CacheData(object):
         self.task_queues = {}
         self.task_deps = {}
         self.stamp = {}
+        self.stamp_extra = {}
+        self.stamp_extra_info = {}
         self.preferred = {}
         self.tasks = {}
         self.basetaskhash = {}
@@ -577,12 +589,17 @@ class CacheData(object):
         self.bbfile_config_priorities = []
 
     def add_from_recipeinfo(self, fn, info):
+        tasks = info.tasks
         self.task_deps[fn] = info.task_deps
         self.pkg_fn[fn] = info.pn
         self.pkg_pn[info.pn].append(fn)
         self.pkg_pepvpr[fn] = (info.pe, info.pv, info.pr)
         self.pkg_dp[fn] = info.defaultpref
         self.stamp[fn] = info.stamp
+        self.stamp_extra[fn] = info.stamp_extra
+
+        for task in tasks:
+            self.stamp_extra_info[self.stamp[fn] + '.' + task] = info.stamp_extra_info[self.stamp[fn] + '.' + task]
 
         provides = [info.pn]
         for provide in info.provides:
bitbake/lib/bb/runqueue.py +6 -6
index a465275..764ae7e 100644
@@ -794,7 +794,7 @@ class RunQueue:
                 continue
             fn = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[task]]
             taskname = self.rqdata.runq_task[task]
-            stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname)
+            stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn, taskname, 0)
             # If the stamp is missing its not current
             if not os.access(stampfile, os.F_OK):
                 del unchecked[task]
@@ -815,7 +815,7 @@ class RunQueue:
                 if task in unchecked:
                     fn = self.taskData.fn_index[self.rqdata.runq_fnid[task]]
                     taskname = self.rqdata.runq_task[task]
-                    stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname)
+                    stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn, taskname, 0)
                     iscurrent = True
 
                     t1 = os.stat(stampfile)[stat.ST_MTIME]
@@ -823,7 +823,7 @@ class RunQueue:
                         if iscurrent:
                             fn2 = self.taskData.fn_index[self.rqdata.runq_fnid[dep]]
                             taskname2 = self.rqdata.runq_task[dep]
-                            stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2)
+                            stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn2, taskname2, 0)
                             if fn == fn2 or (fulldeptree and fn2 not in stampwhitelist):
                                 if dep in notcurrent:
                                     iscurrent = False
@@ -875,7 +875,7 @@ class RunQueue:
         if taskname is None:
             taskname = self.rqdata.runq_task[task]
 
-        stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname)
+        stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn, taskname, 0)
 
         # If the stamp is missing its not current
         if not os.access(stampfile, os.F_OK):
@@ -896,8 +896,8 @@ class RunQueue:
             if iscurrent:
                 fn2 = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[dep]]
                 taskname2 = self.rqdata.runq_task[dep]
-                stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2)
-                stampfile3 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2 + "_setscene")
+                stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn2, taskname2, 0)
+                stampfile3 = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn2, taskname2 + "_setscene", 0)
                 t2 = get_timestamp(stampfile2)
                 t3 = get_timestamp(stampfile3)
                 if t3 and t3 > t2:
bitbake/lib/bb/siggen.py +18 -2
index 4dc09b3..48fe6af 100644
@@ -42,8 +42,24 @@ class SignatureGenerator(object):
     def set_taskdata(self, hashes, deps):
         return
 
-    def stampfile(self, stampbase, file_name, taskname):
-        return "%s.%s" % (stampbase, taskname)
+    def stampfile(self, d, file_name, taskname, is_dict):
+        if is_dict:
+            stamp = d.getVar('STAMP', True)
+            file_name = d.getVar('BB_FILENAME', True)
+            extra_info = taskname + '.' + (bb.data.getVarFlag(taskname, 'stamp-extra-info', d) or bb.data.getVar('BB_STAMP_EXTRA', d, True) or "")
+            extra_info = bb.data.expand(extra_info, d)
+        else:
+            stamp = d.stamp[file_name]
+            if stamp + "." + taskname in d.stamp_extra_info.keys():
+                extra_info = taskname + '.' + (d.stamp_extra_info[stamp + "." + taskname] or d.stamp_extra[file_name] or "")
+            else:
+                extra_info = taskname + '.' + (d.stamp_extra[file_name] or "")
+
+        if not stamp:
+            return
+
+        return "%s.%s" % (stamp, extra_info.rstrip('.'))
+
 
 class SignatureGeneratorBasic(SignatureGenerator):
     """
Signature · 1 line
1.6.3.3
#5 lore raw

PATCH 3/3 bitbake: machine specific sysroots implementation

In reply to Dongxiao Xu

This commit changes the sysroots path to be machine specific.

Changes includes:

1) STAGING_DIR_TARGET and STRAGING_DIR_HOST points to machine specific paths.

2) task stamp files. Adding ${MACHINE} info into stamp files for do_populate_sysroots and do_package tasks. Add a BB_STAMPTASK_BLACKLIST to keep native, nativesdk, crosssdk, and cross-canadian stamp unchanged.

3) siteconfig path. Separate the site config path for different machines to avoid one machine adopting the cache file of another machine.

4) sstate stamp file. Add machine name to sstate stamp file since some prebuild result could not be shared after sysroots are implemented as machine spedific. Keep native, nativesdk, crosssdk, and cross-canadian unchanged.

5) toolchain scripts. Change the environment path to point to machine specific sysroots in toolchain scripts bbclass.

---
 meta/classes/cross-canadian.bbclass    |    3 ++-
 meta/classes/cross.bbclass             |    5 +++++
 meta/classes/crosssdk.bbclass          |    2 ++
 meta/classes/native.bbclass            |    2 ++
 meta/classes/nativesdk.bbclass         |    5 +++++
 meta/classes/package.bbclass           |    1 +
 meta/classes/siteconfig.bbclass        |    6 +++---
 meta/classes/sstate.bbclass            |    6 ++++--
 meta/classes/staging.bbclass           |    1 +
 meta/classes/toolchain-scripts.bbclass |    4 ++--
 meta/conf/bitbake.conf                 |   11 ++++++-----
 11 files changed, 33 insertions(+), 13 deletions(-)
meta/classes/cross-canadian.bbclass +2 -1
index 9897105..33be2e3 100644
@@ -30,7 +30,6 @@ MULTIMACH_TARGET_SYS = "${MULTIMACH_ARCH}${HOST_VENDOR}-${HOST_OS}"
 INHIBIT_DEFAULT_DEPS = "1"
 
 STAGING_DIR_HOST = "${STAGING_DIR}/${HOST_SYS}-nativesdk"
-STAGING_DIR_TARGET = "${STAGING_DIR}/${OLD_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}"
 
 TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR}/${HOST_ARCH}-nativesdk${HOST_VENDOR}-${HOST_OS}"
 
@@ -84,3 +83,5 @@ export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR_HOST}"
 # Cross-canadian packages need to pull in nativesdk dynamic libs
 SHLIBSDIR = "${STAGING_DIR}/${SDK_ARCH}-nativesdk${SDK_VENDOR}-${BUILD_OS}/shlibs"
 
+do_populate_sysroot[stamp-extra-info] = ""
+do_package[stamp-extra-info] = ""
meta/classes/cross.bbclass +5 -0
index 7c20be0..085ce21 100644
@@ -20,6 +20,11 @@ HOST_OS = "${BUILD_OS}"
 HOST_PREFIX = "${BUILD_PREFIX}"
 HOST_CC_ARCH = "${BUILD_CC_ARCH}"
 
+STAGING_DIR_HOST = "${STAGING_DIR}/${BASEPKG_HOST_SYS}"
+
+export PKG_CONFIG_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}${libdir}/pkgconfig"
+export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}"
+
 CPPFLAGS = "${BUILD_CPPFLAGS}"
 CFLAGS = "${BUILD_CFLAGS}"
 CXXFLAGS = "${BUILD_CFLAGS}"
meta/classes/crosssdk.bbclass +2 -0
index 23db163..08ba823 100644
@@ -18,3 +18,5 @@ target_base_libdir = "${SDKPATHNATIVE}${base_libdir_nativesdk}"
 target_prefix = "${SDKPATHNATIVE}${prefix_nativesdk}"
 target_exec_prefix = "${SDKPATHNATIVE}${exec_prefix_nativesdk}"
 
+do_populate_sysroot[stamp-extra-info] = ""
+do_package[stamp-extra-info] = ""
meta/classes/native.bbclass +2 -0
index 4355db2..986c6ff 100644
@@ -119,3 +119,5 @@ do_package_write_ipk[noexec] = "1"
 do_package_write_deb[noexec] = "1"
 do_package_write_rpm[noexec] = "1"
 
+do_populate_sysroot[stamp-extra-info] = ""
+do_package[stamp-extra-info] = ""
meta/classes/nativesdk.bbclass +5 -0
index 7ed5ca5..e41a4cf 100644
@@ -19,6 +19,8 @@ python () {
 
 #STAGING_DIR_HOST = "${STAGING_DIR}/${HOST_SYS}-nativesdk"
 #STAGING_DIR_TARGET = "${STAGING_DIR}/${BASEPKG_TARGET_SYS}-nativesdk"
+STAGING_DIR_HOST = "${STAGING_DIR}/${BASEPKG_HOST_SYS}"
+STAGING_DIR_TARGET = "${STAGING_DIR}/${BASEPKG_TARGET_SYS}"
 
 HOST_ARCH = "${SDK_ARCH}"
 HOST_VENDOR = "${SDK_VENDOR}"
@@ -80,3 +82,6 @@ python __anonymous () {
     bb.data.setVar("PROVIDES", provides, d)
     bb.data.setVar("OVERRIDES", bb.data.getVar("OVERRIDES", d, False) + ":virtclass-nativesdk", d)
 }
+
+do_populate_sysroot[stamp-extra-info] = ""
+do_package[stamp-extra-info] = ""
meta/classes/package.bbclass +1 -0
index d39c694..7e901ec 100644
@@ -1091,6 +1091,7 @@ do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
 do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
 do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
 do_package[sstate-lockfile] = "${PACKAGELOCK}"
+do_package[stamp-extra-info] = "${MACHINE}"
 do_package_setscene[dirs] = "${STAGING_DIR}"
 
 python do_package_setscene () {
meta/classes/siteconfig.bbclass +3 -3
index e7cc9ae..c037bb5 100644
@@ -11,10 +11,10 @@ python siteconfig_do_siteconfig () {
 }
 
 siteconfig_do_siteconfig_gencache () {
-	mkdir -p ${WORKDIR}/site_config
+	mkdir -p ${WORKDIR}/site_config_${MACHINE}
 	gen-site-config ${FILE_DIRNAME}/site_config \
-		>${WORKDIR}/site_config/configure.ac
-	cd ${WORKDIR}/site_config
+		>${WORKDIR}/site_config_${MACHINE}/configure.ac
+	cd ${WORKDIR}/site_config_${MACHINE}
 	autoconf
         CONFIG_SITE="" ./configure ${CONFIGUREOPTS} --cache-file ${PN}_cache
 	sed -n -e "/ac_cv_c_bigendian/p" -e "/ac_cv_sizeof_/p" \
meta/classes/sstate.bbclass +4 -2
index b6e6c92..1a9805a 100644
@@ -5,7 +5,7 @@ SSTATE_MANFILEBASE = "${SSTATE_MANIFESTS}/manifest-${SSTATE_PKGARCH}-"
 SSTATE_MANFILEPREFIX = "${SSTATE_MANFILEBASE}${PN}"
 
 
-SSTATE_PKGARCH    = "${MULTIMACH_ARCH}"
+SSTATE_PKGARCH    = "${MULTIMACH_ARCH}_${MACHINE}"
 SSTATE_PKGSPEC    = "sstate-${PN}-${MULTIMACH_ARCH}${TARGET_VENDOR}-${TARGET_OS}-${PV}-${PR}-${SSTATE_PKGARCH}-${SSTATE_VERSION}-"
 SSTATE_PKGNAME    = "${SSTATE_PKGSPEC}${BB_TASKHASH}"
 SSTATE_PKG        = "${SSTATE_DIR}/${SSTATE_PKGNAME}"
@@ -17,7 +17,9 @@ BB_HASHFILENAME = "${SSTATE_PKGNAME}"
 python () {
     if bb.data.inherits_class('native', d):
         bb.data.setVar('SSTATE_PKGARCH', bb.data.getVar('BUILD_ARCH', d), d)
-    elif bb.data.inherits_class('cross', d) or bb.data.inherits_class('crosssdk', d):
+    elif bb.data.inherits_class('cross', d):
+        bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${BUILD_ARCH}_${BASE_PACKAGE_ARCH}_${MACHINE}", d), d)
+    elif bb.data.inherits_class('crosssdk', d):
         bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${BUILD_ARCH}_${BASE_PACKAGE_ARCH}", d), d)
     elif bb.data.inherits_class('nativesdk', d):
         bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${SDK_ARCH}", d), d)
meta/classes/staging.bbclass +1 -0
index 4d2991b..7ff4847 100644
@@ -106,6 +106,7 @@ SSTATETASKS += "do_populate_sysroot"
 do_populate_sysroot[sstate-name] = "populate-sysroot"
 do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}/${base_prefix}"
 do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/"
+do_populate_sysroot[stamp-extra-info] = "${MACHINE}"
 
 python do_populate_sysroot_setscene () {
 	sstate_setscene(d)
meta/classes/toolchain-scripts.bbclass +2 -2
index a5b2bd1..462f3d8 100644
@@ -49,8 +49,8 @@ toolchain_create_tree_env_script () {
 	echo 'export TARGET_PREFIX=${TARGET_PREFIX}' >> $script
 	echo 'export CONFIGURE_FLAGS="--target=${TARGET_SYS} --host=${TARGET_SYS} --build=${BUILD_SYS}"' >> $script
 	if [ "${TARGET_OS}" = "darwin8" ]; then
-		echo 'export TARGET_CFLAGS="-I${STAGING_DIR}${TARGET_SYS}${includedir}"' >> $script
-		echo 'export TARGET_LDFLAGS="-L${STAGING_DIR}${TARGET_SYS}${libdir}"' >> $script
+		echo 'export TARGET_CFLAGS="-I${STAGING_DIR}${MACHINE}${includedir}"' >> $script
+		echo 'export TARGET_LDFLAGS="-L${STAGING_DIR}${MACHINE}${libdir}"' >> $script
 		# Workaround darwin toolchain sysroot path problems
 		cd ${SDK_OUTPUT}${SDKTARGETSYSROOT}/usr
 		ln -s /usr/local local
meta/conf/bitbake.conf +6 -5
index 376e3cf..4d83258 100644
@@ -276,7 +276,7 @@ STAGING_DATADIR_NATIVE = "${STAGING_DIR_NATIVE}${datadir_native}"
 
 # This should really be MULTIMACH_HOST_SYS but that breaks "all" and machine 
 # specific packages - hack around it for now.
-STAGING_DIR_HOST = "${STAGING_DIR}/${BASEPKG_HOST_SYS}"
+STAGING_DIR_HOST = "${STAGING_DIR}/${MACHINE}"
 STAGING_BINDIR = "${STAGING_DIR_HOST}${bindir}"
 STAGING_LIBDIR = "${STAGING_DIR_HOST}${libdir}"
 STAGING_INCDIR = "${STAGING_DIR_HOST}${includedir}"
@@ -288,7 +288,7 @@ STAGING_PYDIR = "${STAGING_DIR}/lib/python2.4"
 
 # This should really be MULTIMACH_TARGET_SYS but that breaks "all" and machine 
 # specific packages - hack around it for now.
-STAGING_DIR_TARGET = "${STAGING_DIR}/${BASEPKG_TARGET_SYS}"
+STAGING_DIR_TARGET = "${STAGING_DIR}/${MACHINE}"
 
 # Setting DEPLOY_DIR outside of TMPDIR is helpful, when you are using
 # packaged staging and/or multimachine.
@@ -311,7 +311,7 @@ SDKPATHNATIVE = "${SDKPATH}/sysroots/${SDK_SYS}"
 ##################################################################
 
 OLDEST_KERNEL = "2.4.0"
-STAGING_KERNEL_DIR = "${STAGING_DIR}/${MULTIMACH_TARGET_SYS}/kernel"
+STAGING_KERNEL_DIR = "${STAGING_DIR}/${MACHINE}/kernel"
 
 ##################################################################
 # Specific image creation and rootfs population info.
@@ -596,10 +596,10 @@ SLOT = "0"
 
 # Other
 
-export PKG_CONFIG_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}${libdir}/pkgconfig"
+export PKG_CONFIG_DIR = "${STAGING_DIR}/${MACHINE}/${libdir}/pkgconfig"
 export PKG_CONFIG_PATH = "${PKG_CONFIG_DIR}:${STAGING_DATADIR}/pkgconfig"
 export PKG_CONFIG_LIBDIR = "${PKG_CONFIG_DIR}"
-export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR}/${BASE_PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}"
+export PKG_CONFIG_SYSROOT_DIR = "${STAGING_DIR}/${MACHINE}"
 export PKG_CONFIG_DISABLE_UNINSTALLED = "yes"
 
 export QMAKE_MKSPEC_PATH = "${STAGING_DATADIR_NATIVE}/qmake"
@@ -732,3 +732,4 @@ BB_SIGNATURE_HANDLER ?= "basic"
 BB_HASHTASK_WHITELIST ?= "(.*-cross$|.*-native$|.*-cross-initial$|.*-cross-intermediate$|^virtual:native:.*|^virtual:nativesdk:.*)"
 BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM USER"
 
+BB_STAMPTASK_BLACKLIST ?= "(.*-native_*|.*-nativesdk_*|^virtual:native:*|^virtual:nativesdk:*|.*-crosssdk_*|.*-cross-canadian-*)"
Signature · 1 line
1.6.3.3
#3 lore raw

Re: [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file

In reply to Dongxiao Xu

Hi Dongxiao,

I've done a quick style review of this patch, comments inline.

Quoted 25 lines · Dongxiao Xu

For certain tasks, we need additional information in build stamp file except the task name and file name. stamp-extra-info is introduced as an flag adding to the end of stamp file name.

Besides, if we need to add common flags for tasks, we can set value for macro BB_STAMP_EXTRA.

---
 bitbake/lib/bb/build.py    |   10 +++-------
 bitbake/lib/bb/cache.py    |   19 ++++++++++++++++++-
 bitbake/lib/bb/runqueue.py |   12 ++++++------
 bitbake/lib/bb/siggen.py   |   20 ++++++++++++++++++--
 4 files changed, 45 insertions(+), 16 deletions(-)
bitbake/lib/bb/build.py +1 -1
index f127796..3b297b3 100644
@@ -383,15 +383,11 @@ def stamp_internal(taskname, d, file_name):
     When called in task context, d will be a data store, file_name will not be set
     """
     if file_name:
-        stamp = d.stamp[file_name]
+        is_dict = 0

bitbake/lib/bb/build.py def stamp_internal(taskname, d, file_name): · line 390

Quoted 10 lines
bitbake/lib/bb/build.py +2 -6
     else:
-        stamp = d.getVar('STAMP', True)
-        file_name = d.getVar('BB_FILENAME', True)
+        is_dict = 1
 
-    if not stamp:
-        return
-
-    stamp = bb.parse.siggen.stampfile(stamp, file_name, taskname)
+    stamp = bb.parse.siggen.stampfile(d, file_name, taskname, is_dict)

Nitpick, but could you please use True/False rather than 1/0 - IMHO it makes the code more readable and the BitBake codebase has been gradually moving in the direction of using the boolean types of late.

Quoted 79 lines
bitbake/lib/bb/build.py +0 -0
     bb.utils.mkdirhier(os.path.dirname(stamp))
 
bitbake/lib/bb/cache.py +18 -1
index 9a2e2d5..2437d4a 100644
@@ -55,6 +55,8 @@ recipe_fields = (
     'provides',
     'task_deps',
     'stamp',
+    'stamp_extra',
+    'stamp_extra_info',
     'broken',
     'not_world',
     'skipped',
@@ -102,6 +104,11 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)):
                     for task in tasks)
 
     @classmethod
+    def stampvar(cls, var, tasks, stamp, metadata):
+        return dict((stamp + '.' + task, metadata.getVarFlag(task, var, True))
+                    for task in tasks)
+
+    @classmethod
     def getvar(cls, var, metadata):
         return metadata.getVar(var, True) or ''
 
@@ -126,6 +133,7 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)):
         if not pn in packages:
             packages.append(pn)
 
+        stamp = cls.getvar('STAMP', metadata)
         return RecipeInfo(
             tasks            = tasks,
             basetaskhashes   = cls.taskvar('BB_BASEHASH', tasks, metadata),
@@ -147,7 +155,9 @@ class RecipeInfo(namedtuple('RecipeInfo', recipe_fields)):
             defaultpref      = cls.intvar('DEFAULT_PREFERENCE', metadata),
             broken           = cls.getvar('BROKEN', metadata),
             not_world        = cls.getvar('EXCLUDE_FROM_WORLD', metadata),
-            stamp            = cls.getvar('STAMP', metadata),
+            stamp            = stamp,
+            stamp_extra      = cls.getvar('BB_STAMP_EXTRA', metadata),
+            stamp_extra_info = cls.stampvar('stamp-extra-info', tasks, stamp, metadata),
             packages_dynamic = cls.listvar('PACKAGES_DYNAMIC', metadata),
             depends          = cls.depvar('DEPENDS', metadata),
             provides         = cls.depvar('PROVIDES', metadata),
@@ -562,6 +572,8 @@ class CacheData(object):
         self.task_queues = {}
         self.task_deps = {}
         self.stamp = {}
+        self.stamp_extra = {}
+        self.stamp_extra_info = {}
         self.preferred = {}
         self.tasks = {}
         self.basetaskhash = {}
@@ -577,12 +589,17 @@ class CacheData(object):
         self.bbfile_config_priorities = []
 
     def add_from_recipeinfo(self, fn, info):
+        tasks = info.tasks
         self.task_deps[fn] = info.task_deps
         self.pkg_fn[fn] = info.pn
         self.pkg_pn[info.pn].append(fn)
         self.pkg_pepvpr[fn] = (info.pe, info.pv, info.pr)
         self.pkg_dp[fn] = info.defaultpref
         self.stamp[fn] = info.stamp
+        self.stamp_extra[fn] = info.stamp_extra
+
+        for task in tasks:
+            self.stamp_extra_info[self.stamp[fn] + '.' + task] = info.stamp_extra_info[self.stamp[fn] + '.' + task]
 
         provides = [info.pn]
         for provide in info.provides:
bitbake/lib/bb/runqueue.py +0 -0
index a465275..764ae7e 100644

bitbake/lib/bb/runqueue.py class RunQueue: · line 797

Quoted 5 lines
bitbake/lib/bb/runqueue.py +1 -1
@@ -794,7 +794,7 @@ class RunQueue:
                 continue
             fn = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[task]]
             taskname = self.rqdata.runq_task[task]
-            stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname)
+            stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn, taskname, 0)

False, rather than 0, here please.

bitbake/lib/bb/runqueue.py class RunQueue: · line 818

Quoted 9 lines
bitbake/lib/bb/runqueue.py +1 -1
             # If the stamp is missing its not current
             if not os.access(stampfile, os.F_OK):
                 del unchecked[task]
@@ -815,7 +815,7 @@ class RunQueue:
                 if task in unchecked:
                     fn = self.taskData.fn_index[self.rqdata.runq_fnid[task]]
                     taskname = self.rqdata.runq_task[task]
-                    stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname)
+                    stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn, taskname, 0)

And again.

bitbake/lib/bb/runqueue.py class RunQueue: · line 826

Quoted 9 lines
bitbake/lib/bb/runqueue.py +1 -1
                     iscurrent = True
 
                     t1 = os.stat(stampfile)[stat.ST_MTIME]
@@ -823,7 +823,7 @@ class RunQueue:
                         if iscurrent:
                             fn2 = self.taskData.fn_index[self.rqdata.runq_fnid[dep]]
                             taskname2 = self.rqdata.runq_task[dep]
-                            stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2)
+                            stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn2, taskname2, 0)

And again

bitbake/lib/bb/runqueue.py class RunQueue: · line 878

Quoted 9 lines
bitbake/lib/bb/runqueue.py +1 -1
                             if fn == fn2 or (fulldeptree and fn2 not in stampwhitelist):
                                 if dep in notcurrent:
                                     iscurrent = False
@@ -875,7 +875,7 @@ class RunQueue:
         if taskname is None:
             taskname = self.rqdata.runq_task[task]
 
-        stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn], fn, taskname)
+        stampfile = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn, taskname, 0)

And again

bitbake/lib/bb/runqueue.py class RunQueue: · line 900

Quoted 11 lines
bitbake/lib/bb/runqueue.py +2 -2
         # If the stamp is missing its not current
         if not os.access(stampfile, os.F_OK):
@@ -896,8 +896,8 @@ class RunQueue:
             if iscurrent:
                 fn2 = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[dep]]
                 taskname2 = self.rqdata.runq_task[dep]
-                stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2)
-                stampfile3 = bb.parse.siggen.stampfile(self.rqdata.dataCache.stamp[fn2], fn2, taskname2 + "_setscene")
+                stampfile2 = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn2, taskname2, 0)
+                stampfile3 = bb.parse.siggen.stampfile(self.rqdata.dataCache, fn2, taskname2 + "_setscene", 0)

And again, and again :-)

Quoted 8 lines
bitbake/lib/bb/runqueue.py +0 -0
                 t2 = get_timestamp(stampfile2)
                 t3 = get_timestamp(stampfile3)
                 if t3 and t3 > t2:
bitbake/lib/bb/siggen.py +0 -0
index 4dc09b3..48fe6af 100644

bitbake/lib/bb/siggen.py class SignatureGenerator(object): · line 45

Quoted 6 lines
bitbake/lib/bb/siggen.py +1 -2
@@ -42,8 +42,24 @@ class SignatureGenerator(object):
     def set_taskdata(self, hashes, deps):
         return
 
-    def stampfile(self, stampbase, file_name, taskname):
-        return "%s.%s" % (stampbase, taskname)
+    def stampfile(self, d, file_name, taskname, is_dict):

As you have a lot of calls to stampfile with is_dict set to False I'd recommend you define the function with a default parameter of False for is_dict:

def stampfile(self, d, file_name, taskname, is_dict=False):

that way you can avoid typing this parameter for most of the calls to this function. 6 times, by my count.

bitbake/lib/bb/siggen.py class SignatureGenerator(object): · line 50

Quoted 5 lines
bitbake/lib/bb/siggen.py +5 -0
+        if is_dict:
+            stamp = d.getVar('STAMP', True)
+            file_name = d.getVar('BB_FILENAME', True)
+            extra_info = taskname + '.' + (bb.data.getVarFlag(taskname, 'stamp-extra-info', d) or bb.data.getVar('BB_STAMP_EXTRA', d, True) or "")
+            extra_info = bb.data.expand(extra_info, d)

Haven't you already expanded this by passing True as the third argument to getVar() above?

Quoted 5 lines
bitbake/lib/bb/siggen.py +5 -0
+        else:
+            stamp = d.stamp[file_name]
+            if stamp + "." + taskname in d.stamp_extra_info.keys():
+                extra_info = taskname + '.' + (d.stamp_extra_info[stamp + "." + taskname] or d.stamp_extra[file_name] or "")
+            else:

bitbake/lib/bb/siggen.py class SignatureGenerator(object): · line 65

Quoted 10 lines
bitbake/lib/bb/siggen.py +7 -0
+                extra_info = taskname + '.' + (d.stamp_extra[file_name] or "")
+
+        if not stamp:
+            return
+
+        return "%s.%s" % (stamp, extra_info.rstrip('.'))
+
 
 class SignatureGeneratorBasic(SignatureGenerator):
     """

Cheers,
Joshua

Signature · 2 lines
Joshua Lock
        Intel Open Source Technology Centre
#4 lore raw

Re: [PATCH 2/3] bitbake: Introduce stamp-extra-info into build stamp file

Quoted 13 lines · Joshua Lock

Hi Dongxiao,

I've done a quick style review of this patch, comments inline.

Quoted 8 lines · Dongxiao Xu

For certain tasks, we need additional information in build stamp file except the task name and file name. stamp-extra-info is introduced as an flag adding to the end of stamp file name.

Besides, if we need to add common flags for tasks, we can set value for macro BB_STAMP_EXTRA.

I talked with Dongxiao about this patch and we agreed the best approach will be to make bb.build.stampfile() the main entry point so most users will be converted to use that and then the datacache vs d ugliness can be contained in one place (stamp_internal) which calls into siggen.

Josh is right about the 0/1 vs. False/True though. Values of 0/1 are used in many places in the code but we're trying to clean it up as we update things.

Cheers,

Richard