Skip to contentSkip to conversation

Fixes for shell message changes (OE-Core side)

PATCH New Paul Eggleton 1 participant 5 messages Jul 14, 2015

lore
Share & tools
lore
#0 lore raw

cover Fixes for shell message changes (OE-Core side)

Unfortunately the shell message changes caused a few minor regressions; this is a perfect example of a situation where what seems like a simple and logical change can have unintended consequences that you'll miss if you're not careful. More testing all round would probably be advisable in future. Having said that, it has had the effect of highlighting something that we need to be mindful of in future - don't call bbfatal() where you expect the rest of the error log to be printed - use die() or bbfatal_log() for that, or alternatively make your message meaningful enough that you don't need the log.

NOTE: the fix for die() and bbfatal_log() requires the BitBake change that implements bbfatal_log internally which has just sent to the bitbake-devel list. Having said that, logging.bbclass now works as it used to for older BitBake versions that didn't have the message support at all.

The following changes since commit 6be698b7270f73f40d38713ecf13f12aec0ced61:

dpkg: Fix for Fedora22 and new versions of tar (2015-07-13 13:46:45 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib paule/oecore-shell-logging-fixes
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/oecore-shell-logging-fixes

Paul Eggleton (4):
classes/useradd: don't read bbnote/bbwarn/bbfatal values classes/base: fix die() to print the full log
Use die() or bbfatal_log() where the log should definitely be printed classes/logging: allow shell message functions to work in devshell

 meta/classes/autotools.bbclass              |  4 +--
 meta/classes/base.bbclass                   |  2 +-
 meta/classes/kernel-yocto.bbclass           | 16 +++++-----
 meta/classes/logging.bbclass                | 48 +++++++++++++++++++++++++----
 meta/classes/scons.bbclass                  |  4 +--
 meta/classes/useradd.bbclass                |  6 ++--
 meta/recipes-extended/groff/groff_1.22.3.bb |  2 +-
 7 files changed, 59 insertions(+), 23 deletions(-)
Signature · 1 line
2.1.0
#1 lore raw

PATCH 1/4 classes/useradd: don't read bbnote/bbwarn/bbfatal values

Since these functions now log through BitBake's message logging system we must have standalone implementations here.

---
 meta/classes/useradd.bbclass | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
meta/classes/useradd.bbclass +3 -3
index aae038f..4577e56 100644
@@ -191,9 +191,9 @@ fakeroot python populate_packages_prepend () {
         preinst = d.getVar('pkg_preinst_%s' % pkg, True) or d.getVar('pkg_preinst', True)
         if not preinst:
             preinst = '#!/bin/sh\n'
-        preinst += 'bbnote () {\n%s}\n' % d.getVar('bbnote', True)
-        preinst += 'bbwarn () {\n%s}\n' % d.getVar('bbwarn', True)
-        preinst += 'bbfatal () {\n%s}\n' % d.getVar('bbfatal', True)
+        preinst += 'bbnote () {\n\techo "NOTE: $*"\n}\n'
+        preinst += 'bbwarn () {\n\techo "WARNING: $*"\n}\n'
+        preinst += 'bbfatal () {\n\techo "ERROR: $*"\n\texit 1\n}\n'
         preinst += 'perform_groupadd () {\n%s}\n' % d.getVar('perform_groupadd', True)
         preinst += 'perform_useradd () {\n%s}\n' % d.getVar('perform_useradd', True)
         preinst += 'perform_groupmems () {\n%s}\n' % d.getVar('perform_groupmems', True)
Signature · 1 line
2.1.0
#2 lore raw

PATCH 2/4 classes/base: fix die() to print the full log

In reply to Paul Eggleton

The recent change to connect through the shell logging functions caused a regression - bb.error() and bb.fatal() cause a flag to be set internally such that BitBake's UI will not print the full task log on failure; unfortunately we have in a lot of places called die() or bbfatal() within shell functions with a very terse message as a means of exiting out, where we still want to see the full log (and we were previously). We do still want to have fatal errors with proper messages that don't result in the full log being printed, however we can't ignore the typical usage of die(). Having added a mechanism to BitBake to log an error and reset the flag, create a bbfatal_log() function that uses this and call it from die() to restore the previous behaviour.

---
 meta/classes/base.bbclass    | 2 +-
 meta/classes/logging.bbclass | 8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)
meta/classes/base.bbclass +1 -1
index fe803f1..6676078 100644
@@ -47,7 +47,7 @@ def lsb_distro_identifier(d):
     return oe.lsb.distro_identifier(adjust_func)
 
 die() {
-	bbfatal "$*"
+	bbfatal_log "$*"
 }
 
 oe_runmake_call() {
meta/classes/logging.bbclass +8 -0
index f19eddd..6b24839 100644
@@ -40,6 +40,14 @@ bbfatal() {
 	exit 1
 }
 
+# Like bbfatal, except prevents the suppression of the error log by
+# bitbake's UI.
+# Output: logs console
+bbfatal_log() {
+	printf "%b\0" "bbfatal_log $*" > ${LOGFIFO}
+	exit 1
+}
+
 # Print debug messages. These are appropriate for progress checkpoint
 # messages to the logs. Depending on the debug log level, they may also
 # go to the console.
Signature · 1 line
2.1.0
#3 lore raw

PATCH 3/4 Use die() or bbfatal_log() where the log should definitely be printed

In reply to Paul Eggleton

Change calls to bbfatal() to either die() or bbfatal_log() where we know we want the full log to be printed by the UI (calling bberror or bbfatal would otherwise suppress it since the change to connect these functions through to the UI.) bbfatal() is still fine to use where there is enough context information in the message such that the log isn't needed.

---
 meta/classes/autotools.bbclass              |  4 ++--
 meta/classes/kernel-yocto.bbclass           | 16 ++++++++--------
 meta/classes/scons.bbclass                  |  4 ++--
 meta/recipes-extended/groff/groff_1.22.3.bb |  2 +-
 4 files changed, 13 insertions(+), 13 deletions(-)
meta/classes/autotools.bbclass +2 -2
index 454dcb6..6f51429 100644
@@ -87,7 +87,7 @@ oe_runconf () {
 		if [ "$?" != "0" ]; then
 			echo "Configure failed. The contents of all config.log files follows to aid debugging"
 			find ${S} -ignore_readdir_race -name config.log -print -exec cat {} \;
-			bbfatal "oe_runconf failed"
+			die "oe_runconf failed"
 		fi
 		set -e
 	else
@@ -287,7 +287,7 @@ autotools_do_configure() {
 			intltoolize --copy --force --automake
 		fi
 		bbnote Executing ACLOCAL=\"$ACLOCAL\" autoreconf --verbose --install --force ${EXTRA_AUTORECONF} $acpaths
-		ACLOCAL="$ACLOCAL" autoreconf -Wcross --verbose --install --force ${EXTRA_AUTORECONF} $acpaths || bbfatal "autoreconf execution failed."
+		ACLOCAL="$ACLOCAL" autoreconf -Wcross --verbose --install --force ${EXTRA_AUTORECONF} $acpaths || die "autoreconf execution failed."
 		cd $olddir
 	fi
 	if [ -e ${S}/configure ]; then
meta/classes/kernel-yocto.bbclass +8 -8
index 6fd025e..878b554 100644
@@ -120,7 +120,7 @@ do_kernel_metadata() {
 
 	createme -v -v ${createme_flags} ${ARCH} ${machine_branch}
 	if [ $? -ne 0 ]; then
-		bbfatal "Could not create ${machine_branch}"
+		bbfatal_log "Could not create ${machine_branch}"
 	fi
 
 	sccs="$sccs ${@" ".join(find_sccs(d))}"
@@ -152,7 +152,7 @@ do_kernel_metadata() {
 	updateme ${updateme_flags} -DKDESC=${KMACHINE}:${LINUX_KERNEL_TYPE} \
                          ${includes} ${addon_features} ${ARCH} ${KMACHINE} ${sccs} ${patches}
 	if [ $? -ne 0 ]; then
-		bbfatal "Could not update ${machine_branch}"
+		bbfatal_log "Could not update ${machine_branch}"
 	fi
 }
 
@@ -163,7 +163,7 @@ do_patch() {
 	patchme ${KMACHINE}
 	if [ $? -ne 0 ]; then
 		bberror "Could not apply patches for ${KMACHINE}."
-		bbfatal "Patch failures can be resolved in the linux source directory ${S})"
+		bbfatal_log "Patch failures can be resolved in the linux source directory ${S})"
 	fi
 
 	# check to see if the specified SRCREV is reachable from the final branch.
@@ -243,7 +243,7 @@ do_kernel_checkout() {
 		if [ $? -eq 1 ]; then
 			bberror "The branch '${KMETA}' is required and was not found"
 			bberror "Ensure that the SRC_URI points to a valid linux-yocto"
-			bbfatal "kernel repository"
+			bbfatal_log "kernel repository"
 		fi
 	fi
 	
@@ -283,7 +283,7 @@ do_kernel_configme() {
 	PATH=${PATH}:${S}/scripts/util
 	configme ${configmeflags} --reconfig --output ${B} ${LINUX_KERNEL_TYPE} ${KMACHINE}
 	if [ $? -ne 0 ]; then
-		bbfatal "Could not configure ${KMACHINE}-${LINUX_KERNEL_TYPE}"
+		bbfatal_log "Could not configure ${KMACHINE}-${LINUX_KERNEL_TYPE}"
 	fi
 	
 	echo "# Global settings from linux recipe" >> ${B}/.config
@@ -362,7 +362,7 @@ do_validate_branches() {
 		git cat-file -t ${machine_srcrev} > /dev/null
 		if [ $? -ne 0 ]; then
 			bberror "${machine_srcrev} is not a valid commit ID."
-			bbfatal "The kernel source tree may be out of sync"
+			bbfatal_log "The kernel source tree may be out of sync"
 		fi
 		force_srcrev=${machine_srcrev}
 	fi
@@ -377,14 +377,14 @@ do_validate_branches() {
 		git cat-file -t ${target_meta_head} > /dev/null
 		if [ $? -ne 0 ]; then
 			bberror "${target_meta_head} is not a valid commit ID"
-			bbfatal "The kernel source tree may be out of sync"
+			bbfatal_log "The kernel source tree may be out of sync"
 		fi
 		if [ "$meta_head" != "$target_meta_head" ]; then
 			bbnote "Setting branch ${KMETA} to ${target_meta_head}"
 			git branch -m ${KMETA} ${KMETA}-orig
 			git checkout -q -b ${KMETA} ${target_meta_head}
 			if [ $? -ne 0 ];then
-				bbfatal "Could not checkout ${KMETA} branch from known hash ${target_meta_head}"
+				bbfatal_log "Could not checkout ${KMETA} branch from known hash ${target_meta_head}"
 			fi
 		fi
 	fi
meta/classes/scons.bbclass +2 -2
index fc0f26b..b8de822 100644
@@ -4,12 +4,12 @@ EXTRA_OESCONS ?= ""
 
 scons_do_compile() {
         ${STAGING_BINDIR_NATIVE}/scons ${PARALLEL_MAKE} PREFIX=${prefix} prefix=${prefix} ${EXTRA_OESCONS} || \
-        bbfatal "scons build execution failed."
+        die "scons build execution failed."
 }
 
 scons_do_install() {
         ${STAGING_BINDIR_NATIVE}/scons PREFIX=${D}${prefix} prefix=${D}${prefix} install ${EXTRA_OESCONS}|| \
-        bbfatal "scons install execution failed."
+        die "scons install execution failed."
 }
 
 EXPORT_FUNCTIONS do_compile do_install
meta/recipes-extended/groff/groff_1.22.3.bb +1 -1
index e8cf524..4bffa81 100644
@@ -41,7 +41,7 @@ do_configure_append() {
     # generate gnulib configure script
     olddir=`pwd`
     cd ${S}/src/libs/gnulib/
-    ACLOCAL="$ACLOCAL" autoreconf -Wcross --verbose --install --force ${EXTRA_AUTORECONF} $acpaths || bbfatal "autoreconf execution failed."
+    ACLOCAL="$ACLOCAL" autoreconf -Wcross --verbose --install --force ${EXTRA_AUTORECONF} $acpaths || die "autoreconf execution failed."
     cd ${olddir}
 }
 
Signature · 1 line
2.1.0
#4 lore raw

PATCH 4/4 classes/logging: allow shell message functions to work in devshell

In reply to Paul Eggleton

Fix a regression caused by the shell message changes - if you run a shell task's runfile, the task isn't actually running in BitBake and thus the message FIFO won't exist - so we should just fall back to printing the message with echo as we did before so that the run files are still useful.

---
 meta/classes/logging.bbclass | 42 +++++++++++++++++++++++++++++++++++-------
 1 file changed, 35 insertions(+), 7 deletions(-)
meta/classes/logging.bbclass +35 -7
index 6b24839..06c7c31 100644
@@ -9,34 +9,54 @@ LOGFIFO = "${T}/fifo.${@os.getpid()}"
 # tasks that should be seen on the console. Use sparingly.
 # Output: logs console
 bbplain() {
-	printf "%b\0" "bbplain $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bbplain $*" > ${LOGFIFO}
+	else
+		echo "$*"
+	fi
 }
 
 # Notify the user of a noteworthy condition. 
 # Output: logs
 bbnote() {
-	printf "%b\0" "bbnote $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bbnote $*" > ${LOGFIFO}
+	else
+		echo "NOTE: $*"
+	fi
 }
 
 # Print a warning to the log. Warnings are non-fatal, and do not
 # indicate a build failure.
 # Output: logs console
 bbwarn() {
-	printf "%b\0" "bbwarn $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bbwarn $*" > ${LOGFIFO}
+	else
+		echo "WARNING: $*"
+	fi
 }
 
 # Print an error to the log. Errors are non-fatal in that the build can
 # continue, but they do indicate a build failure.
 # Output: logs console
 bberror() {
-	printf "%b\0" "bberror $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bberror $*" > ${LOGFIFO}
+	else
+		echo "ERROR: $*"
+	fi
 }
 
 # Print a fatal error to the log. Fatal errors indicate build failure
 # and halt the build, exiting with an error code.
 # Output: logs console
 bbfatal() {
-	printf "%b\0" "bbfatal $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bbfatal $*" > ${LOGFIFO}
+	else
+		echo "ERROR: $*"
+	fi
 	exit 1
 }
 
@@ -44,7 +64,11 @@ bbfatal() {
 # bitbake's UI.
 # Output: logs console
 bbfatal_log() {
-	printf "%b\0" "bbfatal_log $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bbfatal_log $*" > ${LOGFIFO}
+	else
+		echo "ERROR: $*"
+	fi
 	exit 1
 }
 
@@ -68,6 +92,10 @@ bbdebug() {
 	fi
 
 	# All debug output is printed to the logs
-	printf "%b\0" "bbdebug $DBGLVL $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bbdebug $DBGLVL $*" > ${LOGFIFO}
+	else
+		echo "DEBUG: $*"
+	fi
 }
 
Signature · 1 line
2.1.0