Skip to contentSkip to conversation

Improve oe-init-build-env/-memres

PATCH Superseded Peter Kjellerstedt 1 participant 7 messages Mar 15, 2016

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Improve oe-init-build-env/-memres

I started looking at the oe-init-build-env (and oe-init-build-env-memres) scripts since I wanted to add support for specifying $OEROOT from the outside. This is because we want to wrap those scripts with our own while keeping the official names (so anyone reading the official Poky documentation will not be too confused).

While fixing the above, I had noticed a number of inconsistencies in the scripts, so I continued to fix those. I also improved the code that adds paths to $PATH, and the code that adds variable names to $BB_ENV_EXTRAWHITE.

//Peter

The following changes since commit 8debfea81e69d038bd2d56314b272cb74f5582ed:

local.conf.sample: Disable prelink by default (2016-03-13 22:09:05 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib pkj/oe-init-build-env
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/oe-init-build-env

Peter Kjellerstedt (6):
oe-init-build-env*: Allow $OEROOT to be predefined oe-init-build-env*: Update/correct comment about specifying arguments oe-init-build-env*: Remove unnecessary differences between the scripts
oe-buildenv-internal: Add paths to $PATH individually
oe-buildenv-internal: Add variables individually to BB_ENV_EXTRAWHITE
oe-buildenv-internal: Some clean up

 oe-init-build-env            | 45 ++++++++++++------------
 oe-init-build-env-memres     | 54 ++++++++++++++--------------
 scripts/oe-buildenv-internal | 83 ++++++++++++++++++++++++++++----------------
 3 files changed, 102 insertions(+), 80 deletions(-)
Signature · 1 line
2.1.0
#1 lore raw

PATCH 1/6 oe-init-build-env*: Allow $OEROOT to be predefined

The current implementation of oe-init-build-env and oe-init-build-env-memres requires that they are sourced from the directory that will be known as $OEROOT. This makes it hard to write a wrapper script with the same name as the original OE script which, e.g., sources the original OE script from a sub-directory.

With this change, $OEROOT can be predefined when oe-init-build-env or oe-init-build-env-memres is sourced, allowing the original OE scripts to be anywhere.

---
 oe-init-build-env        | 16 ++++++++--------
 oe-init-build-env-memres | 16 ++++++++--------
 2 files changed, 16 insertions(+), 16 deletions(-)
oe-init-build-env +8 -8
index b7e2918..f511b57 100755
@@ -26,30 +26,30 @@
 # to sourcing this script.
 #
 if [ -n "$BASH_SOURCE" ]; then
-   OEROOT="`dirname $BASH_SOURCE`"
+    THIS_SCRIPT=$BASH_SOURCE
 elif [ -n "$ZSH_NAME" ]; then
-   OEROOT="`dirname $0`"
+    THIS_SCRIPT=$0
 else
-   OEROOT="`pwd`"
+    THIS_SCRIPT="$(pwd)/oe-init-build-env"
 fi
 if [ -n "$BBSERVER" ]; then
    unset BBSERVER
 fi
-THIS_SCRIPT=$OEROOT/oe-init-build-env
 
 if [ -z "$ZSH_NAME" ] && [ "$0" = "$THIS_SCRIPT" ]; then
    echo "Error: This script needs to be sourced. Please run as '. $THIS_SCRIPT'"
    exit 1
 fi
 
-OEROOT=`readlink -f "$OEROOT"`
+if [ -z "$OEROOT" ]; then
+    OEROOT=$(dirname "$THIS_SCRIPT")
+    OEROOT=$(readlink -f "$OEROOT")
+fi
 export OEROOT
 . $OEROOT/scripts/oe-buildenv-internal && \
      TEMPLATECONF="$TEMPLATECONF" $OEROOT/scripts/oe-setup-builddir && \
      [ -n "$BUILDDIR" ] && cd "$BUILDDIR"
-unset OEROOT
-unset BBPATH
-unset THIS_SCRIPT
+unset THIS_SCRIPT OEROOT BBPATH
 
 # Shutdown any bitbake server if the BBSERVER variable is not set
 if [ -z "$BBSERVER" ] && [ -f bitbake.lock ] ; then
oe-init-build-env-memres +8 -8
index c1dc4fe..6dd2b13 100755
@@ -34,30 +34,30 @@ else
 fi
 
 if [ -n "$BASH_SOURCE" ]; then
-  OEROOT="`dirname $BASH_SOURCE`"
+    THIS_SCRIPT=$BASH_SOURCE
 elif [ -n "$ZSH_NAME" ]; then
-  OEROOT="`dirname $0`"
+    THIS_SCRIPT=$0
 else
-  OEROOT="`pwd`"
+    THIS_SCRIPT="$(pwd)/oe-init-build-env"
 fi
 if [ -n "$BBSERVER" ]; then
   unset BBSERVER
 fi
 
-THIS_SCRIPT=$OEROOT/oe-init-build-env-memres
 if [ -z "$ZSH_NAME" ] && [ "x$0" = "x$THIS_SCRIPT" ]; then
    echo "Error: This script needs to be sourced. Please run as '. $THIS_SCRIPT'"
    exit 1
 fi
 
-OEROOT=`readlink -f "$OEROOT"`
+if [ -z "$OEROOT" ]; then
+    OEROOT=$(dirname "$THIS_SCRIPT")
+    OEROOT=$(readlink -f "$OEROOT")
+fi
 export OEROOT
 . $OEROOT/scripts/oe-buildenv-internal && \
     $OEROOT/scripts/oe-setup-builddir && \
     [ -n "$BUILDDIR" ] && cd $BUILDDIR
-unset OEROOT
-unset BBPATH
-unset THIS_SCRIPT
+unset THIS_SCRIPT OEROOT BBPATH
 
 res=1
 if [ -e bitbake.lock ] && grep : bitbake.lock > /dev/null ; then
Signature · 1 line
2.1.0
#2 lore raw

PATCH 2/6 oe-init-build-env*: Update/correct comment about specifying arguments

In reply to Peter Kjellerstedt
---
 oe-init-build-env        | 8 ++++----
 oe-init-build-env-memres | 8 ++++----
 2 files changed, 8 insertions(+), 8 deletions(-)
oe-init-build-env +4 -4
index f511b57..baab827 100755
@@ -19,11 +19,11 @@
 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 
 #
-# Normally this is called as '. ./oe-init-build-env builddir'
+# Normally this is called as '. ./oe-init-build-env <builddir>'
 #
-# This works in most shells (not dash), but not all of them pass arg1 when
-# being sourced.   To workaround the shell limitation use "set arg1" prior 
-# to sourcing this script.
+# This works in most shells (not dash), but not all of them pass the arguments
+# when being sourced.  To workaround the shell limitation use "set <builddir>"
+# prior to sourcing this script.
 #
 if [ -n "$BASH_SOURCE" ]; then
     THIS_SCRIPT=$BASH_SOURCE
oe-init-build-env-memres +4 -4
index 6dd2b13..b0c7421 100755
@@ -19,11 +19,11 @@
 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 
 #
-# Normally this is called as '. ./oe-init-build-env builddir <portnumber> <builddir>'
+# Normally this is called as '. ./oe-init-build-env-memres <portnumber> <builddir>'
 #
-# This works in most shells (not dash), but not all of them pass arg1 when
-# being sourced.   To workaround the shell limitation use "set arg1" prior 
-# to sourcing this script.
+# This works in most shells (not dash), but not all of them pass the arguments
+# when being sourced.  To workaround the shell limitation use "set <portnumber>
+# <builddir>" prior to sourcing this script.
 #
 if [ -z "$1" ]; then
     echo "No port specified, using dynamically selected port"
Signature · 1 line
2.1.0
#3 lore raw

PATCH 3/6 oe-init-build-env*: Remove unnecessary differences between the scripts

In reply to Peter Kjellerstedt

While at it, also fix:
* consistent indentation (four spaces)
* unset temporary variables
* use $(...) instead of ...

---
 oe-init-build-env        | 21 ++++++++++-----------
 oe-init-build-env-memres | 30 +++++++++++++++---------------
 2 files changed, 25 insertions(+), 26 deletions(-)
oe-init-build-env +10 -11
index baab827..53873e6 100755
@@ -33,12 +33,12 @@ else
     THIS_SCRIPT="$(pwd)/oe-init-build-env"
 fi
 if [ -n "$BBSERVER" ]; then
-   unset BBSERVER
+    unset BBSERVER
 fi
 
 if [ -z "$ZSH_NAME" ] && [ "$0" = "$THIS_SCRIPT" ]; then
-   echo "Error: This script needs to be sourced. Please run as '. $THIS_SCRIPT'"
-   exit 1
+    echo "Error: This script needs to be sourced. Please run as '. $THIS_SCRIPT'"
+    exit 1
 fi
 
 if [ -z "$OEROOT" ]; then
@@ -47,16 +47,15 @@ if [ -z "$OEROOT" ]; then
 fi
 export OEROOT
 . $OEROOT/scripts/oe-buildenv-internal && \
-     TEMPLATECONF="$TEMPLATECONF" $OEROOT/scripts/oe-setup-builddir && \
-     [ -n "$BUILDDIR" ] && cd "$BUILDDIR"
+    TEMPLATECONF="$TEMPLATECONF" $OEROOT/scripts/oe-setup-builddir && \
+    [ -n "$BUILDDIR" ] && cd "$BUILDDIR"
 unset THIS_SCRIPT OEROOT BBPATH
 
 # Shutdown any bitbake server if the BBSERVER variable is not set
-if [ -z "$BBSERVER" ] && [ -f bitbake.lock ] ; then
-    grep ":" bitbake.lock > /dev/null && BBSERVER=`cat bitbake.lock` bitbake --status-only
-    if [ $? = 0 ] ; then
-	echo "Shutting down bitbake memory resident server with bitbake -m"
-	BBSERVER=`cat bitbake.lock` bitbake -m
+if [ -z "$BBSERVER" ] && [ -f bitbake.lock ]; then
+    grep ":" bitbake.lock > /dev/null && BBSERVER=$(cat bitbake.lock) bitbake --status-only
+    if [ $? = 0 ]; then
+        echo "Shutting down bitbake memory resident server with bitbake -m"
+        BBSERVER=$(cat bitbake.lock) bitbake -m
     fi
 fi
-
oe-init-build-env-memres +15 -15
index b0c7421..08d255c 100755
@@ -41,12 +41,12 @@ else
     THIS_SCRIPT="$(pwd)/oe-init-build-env"
 fi
 if [ -n "$BBSERVER" ]; then
-  unset BBSERVER
+    unset BBSERVER
 fi
 
-if [ -z "$ZSH_NAME" ] && [ "x$0" = "x$THIS_SCRIPT" ]; then
-   echo "Error: This script needs to be sourced. Please run as '. $THIS_SCRIPT'"
-   exit 1
+if [ -z "$ZSH_NAME" ] && [ "$0" = "$THIS_SCRIPT" ]; then
+    echo "Error: This script needs to be sourced. Please run as '. $THIS_SCRIPT'"
+    exit 1
 fi
 
 if [ -z "$OEROOT" ]; then
@@ -55,30 +55,30 @@ if [ -z "$OEROOT" ]; then
 fi
 export OEROOT
 . $OEROOT/scripts/oe-buildenv-internal && \
-    $OEROOT/scripts/oe-setup-builddir && \
-    [ -n "$BUILDDIR" ] && cd $BUILDDIR
+    TEMPLATECONF="$TEMPLATECONF" $OEROOT/scripts/oe-setup-builddir && \
+    [ -n "$BUILDDIR" ] && cd "$BUILDDIR"
 unset THIS_SCRIPT OEROOT BBPATH
 
 res=1
-if [ -e bitbake.lock ] && grep : bitbake.lock > /dev/null ; then
-    BBSERVER=`cat bitbake.lock` bitbake --status-only
+if [ -e bitbake.lock ] && grep : bitbake.lock > /dev/null; then
+    BBSERVER=$(cat bitbake.lock) bitbake --status-only
     res=$?
 fi
 
-if [ $res != 0 ] ; then
+if [ $res != 0 ]; then
     bitbake --server-only -t xmlrpc -B localhost:$port
 fi
 
-if [ $port = -1 ] ; then
+if [ $port = -1 ]; then
     export BBSERVER=localhost:-1
     echo "Bitbake server started on demand as needed, use bitbake -m to shut it down"
 else
-    export BBSERVER=`cat bitbake.lock`
+    export BBSERVER=$(cat bitbake.lock)
 
-    if [ $res = 0 ] ; then
-	echo "Using existing bitbake server at: $BBSERVER, use bitbake -m to shut it down"
+    if [ $res = 0 ]; then
+        echo "Using existing bitbake server at: $BBSERVER, use bitbake -m to shut it down"
     else
-	echo "Bitbake server started at: $BBSERVER, use bitbake -m to shut it down"
+        echo "Bitbake server started at: $BBSERVER, use bitbake -m to shut it down"
     fi
-    unset res
 fi
+unset port res
Signature · 1 line
2.1.0
#4 lore raw

PATCH 4/6 oe-buildenv-internal: Add paths to $PATH individually

In reply to Peter Kjellerstedt

Instead of assuming that the path to the scripts directory always is in $PATH directly before the bitbake directory, treat them as separate paths and add them individually to $PATH.

---
 scripts/oe-buildenv-internal | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
scripts/oe-buildenv-internal +8 -3
index 457763b..cf33c49 100755
@@ -95,9 +95,14 @@ if ! (test -d "$BITBAKEDIR"); then
 fi
 
 # Make sure our paths are at the beginning of $PATH
-NEWPATHS="${OEROOT}/scripts:$BITBAKEDIR/bin:"
-PATH=$NEWPATHS$(echo $PATH | sed -e "s|:$NEWPATHS|:|g" -e "s|^$NEWPATHS||")
-unset BITBAKEDIR NEWPATHS
+for newpath in "$BITBAKEDIR/bin" "$OEROOT/scripts"; do
+    # Remove any existences of $newpath from $PATH
+    PATH=$(echo $PATH | sed -re "s#:$newpath(:|$)#\1#g" -e "s#^$newpath:##")
+
+    # Add $newpath to $PATH
+    PATH="$newpath:$PATH"
+done
+unset BITBAKEDIR newpath
 
 # Used by the runqemu script
 export BUILDDIR
Signature · 1 line
2.1.0
#5 lore raw

PATCH 5/6 oe-buildenv-internal: Add variables individually to BB_ENV_EXTRAWHITE

In reply to Peter Kjellerstedt

Instead of adding all variables to BB_ENV_EXTRAWHITE as one, treat them separately and add them one by one as needed.

---
 scripts/oe-buildenv-internal | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)
scripts/oe-buildenv-internal +21 -5
index cf33c49..6a63399 100755
@@ -108,13 +108,29 @@ unset BITBAKEDIR newpath
 export BUILDDIR
 export PATH
 
-BB_ENV_EXTRAWHITE_OE="MACHINE DISTRO TCMODE TCLIBC HTTP_PROXY http_proxy \
+add_extrawhite() {
+    # If the current shell is zsh, then temporarily set it to emulate sh in this
+    # function so that the for and case statements below work as expected.
+    [ -z "$ZSH_NAME" ] || emulate -L sh
+
+    local extrawhite="MACHINE DISTRO TCMODE TCLIBC HTTP_PROXY http_proxy \
 HTTPS_PROXY https_proxy FTP_PROXY ftp_proxy FTPS_PROXY ftps_proxy ALL_PROXY \
 all_proxy NO_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY \
 SDKMACHINE BB_NUMBER_THREADS BB_NO_NETWORK PARALLEL_MAKE GIT_PROXY_COMMAND \
 SOCKS5_PASSWD SOCKS5_USER SCREENDIR STAMPS_DIR"
 
-echo "$BB_ENV_EXTRAWHITE" | grep -q "${BB_ENV_EXTRAWHITE_OE}"
-if [ $? != 0 ]; then
-     export BB_ENV_EXTRAWHITE="${BB_ENV_EXTRAWHITE_OE} $BB_ENV_EXTRAWHITE"
-fi
+    local var
+    for var in $extrawhite; do
+        case " $BB_ENV_EXTRAWHITE " in
+            *[[:blank:]]$var[[:blank:]]*)
+                ;;
+            *)
+                BB_ENV_EXTRAWHITE="${BB_ENV_EXTRAWHITE:+$BB_ENV_EXTRAWHITE }$var"
+                ;;
+        esac
+    done
+}
+
+add_extrawhite
+unset -f add_extrawhite
+export BB_ENV_EXTRAWHITE
Signature · 1 line
2.1.0
#6 lore raw

PATCH 6/6 oe-buildenv-internal: Some clean up

In reply to Peter Kjellerstedt

* Consistent indentation (four spaces)
* Use [ -z ...] and [ -n ... ] where possible
* Unset temporary variables
* Use $(...) instead of ...
* Avoid an unnecessary call to expr

---
 scripts/oe-buildenv-internal | 46 +++++++++++++++++++++++---------------------
 1 file changed, 24 insertions(+), 22 deletions(-)
scripts/oe-buildenv-internal +24 -22
index 6a63399..14af872 100755
@@ -24,7 +24,7 @@ if [ -z "$OEROOT" ]; then
     return 1
 fi
 
-if [ -z "$OE_SKIP_SDK_CHECK" -a ! -z "$OECORE_SDK_VERSION" ]; then
+if [ -z "$OE_SKIP_SDK_CHECK" ] && [ -n "$OECORE_SDK_VERSION" ]; then
     echo >&2 "Error: The OE SDK/ADT was detected as already being present in this shell environment. Please use a clean shell when sourcing this environment script."
     return 1
 fi
@@ -33,24 +33,26 @@ fi
 # sanity.bbclass because bitbake's source code doesn't even pass
 # parsing stage when used with python v3, so we catch it here so we
 # can offer a meaningful error message.
-py_v3_check=`/usr/bin/env python --version 2>&1 | grep "Python 3"`
-if [ "$py_v3_check" != "" ]; then
-	echo >&2 "Bitbake is not compatible with python v3"
-	echo >&2 "Please set up python v2 as your default python interpreter"
-	return 1
+py_v3_check=$(/usr/bin/env python --version 2>&1 | grep "Python 3")
+if [ -n "$py_v3_check" ]; then
+    echo >&2 "Bitbake is not compatible with python v3"
+    echo >&2 "Please set up python v2 as your default python interpreter"
+    return 1
 fi
+unset py_v3_check
 
 # Similarly, we now have code that doesn't parse correctly with older
 # versions of Python, and rather than fixing that and being eternally
 # vigilant for any other new feature use, just check the version here.
-py_v26_check=`python -c 'import sys; print sys.version_info >= (2,7,3)'`
+py_v26_check=$(python -c 'import sys; print sys.version_info >= (2,7,3)')
 if [ "$py_v26_check" != "True" ]; then
-	echo >&2 "BitBake requires Python 2.7.3 or later"
-	return 1
+    echo >&2 "BitBake requires Python 2.7.3 or later"
+    return 1
 fi
+unset py_v26_check
 
-if [ "x$BDIR" = "x" ]; then
-    if [ "x$1" = "x" ]; then
+if [ -z "$BDIR" ]; then
+    if [ -z "$1" ]; then
         BDIR="build"
     else
         BDIR="$1"
@@ -62,34 +64,34 @@ if [ "x$BDIR" = "x" ]; then
         # Remove any possible trailing slashes. This is used to work around
         # buggy readlink in Ubuntu 10.04 that doesn't ignore trailing slashes
         # and hence "readlink -f new_dir_to_be_created/" returns empty.
-        BDIR=`echo $BDIR | sed -re 's|/+$||'`
+        BDIR=$(echo $BDIR | sed -re 's|/+$||')
 
-        BDIR=`readlink -f "$BDIR"`
+        BDIR=$(readlink -f "$BDIR")
         if [ -z "$BDIR" ]; then
-            PARENTDIR=`dirname "$1"`
+            PARENTDIR=$(dirname "$1")
             echo >&2 "Error: the directory $PARENTDIR does not exist?"
             return 1
         fi
     fi
-    if [ "x$2" != "x" ]; then
+    if [ -n "$2" ]; then
         BITBAKEDIR="$2"
     fi
 fi
-if expr "$BDIR" : '/.*' > /dev/null ; then
+if [ "${BDIR#/}" != "$BDIR" ]; then
     BUILDDIR="$BDIR"
 else
-    BUILDDIR="`pwd`/$BDIR"
+    BUILDDIR="$(pwd)/$BDIR"
 fi
 unset BDIR
 
-if [ "x$BITBAKEDIR" = "x" ]; then
-    BITBAKEDIR="$OEROOT/bitbake$BBEXTRA/"
+if [ -z "$BITBAKEDIR" ]; then
+    BITBAKEDIR="$OEROOT/bitbake$BBEXTRA"
 fi
 
-BITBAKEDIR=`readlink -f "$BITBAKEDIR"`
-BUILDDIR=`readlink -f "$BUILDDIR"`
+BITBAKEDIR=$(readlink -f "$BITBAKEDIR")
+BUILDDIR=$(readlink -f "$BUILDDIR")
 
-if ! (test -d "$BITBAKEDIR"); then
+if [ ! -d "$BITBAKEDIR" ]; then
     echo >&2 "Error: The bitbake directory ($BITBAKEDIR) does not exist!  Please ensure a copy of bitbake exists at this location"
     return 1
 fi
Signature · 1 line
2.1.0