Skip to contentSkip to conversation

Fixes for bitbake wrapper script

PATCH Applied Paul Eggleton 3 participants 10 messages Mar 27, 2012 – Apr 3, 2012

lore
Share & tools
lore
#0 lore raw

cover Fixes for bitbake wrapper script

Andreas, if you could review these that would be much appreciated.

The following changes since commit 644b7503c37fd73730dd3d7841463b158b8934ed:

guile: Deal with hardcoded path issues (2012-03-27 00:28:41 +0100)

are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/builddir
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/builddir

Paul Eggleton (3):
scripts/bitbake: try harder to check if pseudo exists scripts/bitbake: allow switching between build directories scripts/bitbake: add/fix some comments

 scripts/bitbake |   43 ++++++++++++++++++++++++++++++++-----------
 1 files changed, 32 insertions(+), 11 deletions(-)
Signature · 1 line
1.7.5.4
#1 lore raw

PATCH 1/3 scripts/bitbake: try harder to check if pseudo exists

If pseudodone doesn't exist, we can get STAGING_BINDIR_NATIVE by calling bitbake -e and use that as the path to check for pseudo before we give up and try to build it explicitly first.

This is useful for people who share TMPDIR between multiple build directories.

---
 scripts/bitbake |   12 +++++++++---
 1 files changed, 9 insertions(+), 3 deletions(-)
scripts/bitbake +9 -3
index 45c8697..1ba1839 100755
@@ -53,14 +53,21 @@ if [ "`pwd`" != "$BUILDDIR" ] ; then
 fi
 
 buildpseudo="1"
-if [ $needpseudo = "1" ] && [ -e "$BUILDDIR/pseudodone" ]; then
-    PSEUDOBINDIR=`cat $BUILDDIR/pseudodone`
+if [ $needpseudo = "1" ]; then
+    if [ -e "$BUILDDIR/pseudodone" ]; then
+        PSEUDOBINDIR=`cat $BUILDDIR/pseudodone`
+    else
+        PSEUDOBINDIR=`bitbake -e | grep STAGING_BINDIR_NATIVE=\" | cut -d '=' -f2 | cut -d '"' -f2`
+    fi
     if [ -e "$PSEUDOBINDIR/pseudo" -a -e "$PSEUDOBINDIR/tar" -a "$needtar" = "1" ]; then
         buildpseudo="0"
     fi
     if [ -e "$PSEUDOBINDIR/pseudo" -a $needtar = "0" ]; then
         buildpseudo="0"
     fi
+    if [ $buildpseudo = "0" -a ! -e "$BUILDDIR/pseudodone" ] ; then
+        echo $PSEUDOBINDIR > $BUILDDIR/pseudodone
+    fi
 fi
 if [ $needpseudo = "0" ]; then
     buildpseudo="0"
@@ -104,7 +111,6 @@ BITBAKE=`which bitbake`
 export PATH=$OLDPATH
 if [ $needpseudo = "1" ]; then
     export PSEUDO_BUILD=2
-    PSEUDOBINDIR=`cat $BUILDDIR/pseudodone`
     PSEUDO_BINDIR=$PSEUDOBINDIR PSEUDO_LIBDIR=$PSEUDOBINDIR/../lib/pseudo/lib PSEUDO_PREFIX=$PSEUDOBINDIR/../../ PSEUDO_DISABLED=1 $PSEUDOBINDIR/pseudo $BITBAKE $@
 else
     export PSEUDO_BUILD=0
Signature · 1 line
1.7.5.4
#4 lore raw

PATCH 2/3 scripts/bitbake: allow switching between build directories

In reply to Paul Eggleton

The recent addition of the check to ensure the user was in their build directory disabled the ability to switch between build directories without re-running the build environment setup script. We can rely upon checking for conf/bblayers.conf instead, so use this check.

This does allow BUILDDIR (which is normally set by the environment script) to be unset; however if it is set then it is assumed to be the correct build directory and will be used in the error message that is shown when we can't find conf/bblayers.conf.

---
 scripts/bitbake |   15 ++++++++++-----
 1 files changed, 10 insertions(+), 5 deletions(-)
scripts/bitbake +10 -5
index 1ba1839..f02e89b 100755
@@ -37,6 +37,16 @@ if [ "$py_v26_check" != "True" ]; then
 	exit 1
 fi
 
+if [ ! -e conf/bblayers.conf ] ; then
+    BDPRINT=""
+    [ -n "$BUILDDIR" ] && BDPRINT=": $BUILDDIR"
+    echo "Unable to find conf/bblayers.conf"
+    echo "BitBake must be run from within your build directory$BDPRINT"
+    exit 1
+elif [ -z "$BUILDDIR" ] ; then
+    BUILDDIR="`pwd`"
+fi
+
 needtar="1"
 TARVERSION=`tar --version | head -n 1 | cut -d ' ' -f 4`
 float_test() {
@@ -47,11 +57,6 @@ float_test() {
 # but earlier versions do not
 float_test "$TARVERSION > 1.23" && needtar="0"
 
-if [ "`pwd`" != "$BUILDDIR" ] ; then
-    echo "BitBake must be run from your build directory: $BUILDDIR"
-    exit 1
-fi
-
 buildpseudo="1"
 if [ $needpseudo = "1" ]; then
     if [ -e "$BUILDDIR/pseudodone" ]; then
Signature · 1 line
1.7.5.4
#6 lore raw

PATCH 3/3 scripts/bitbake: add/fix some comments

In reply to Paul Eggleton

Add some comments explaining what this script does, fix one grammatical error in a comment and make the tar-replacement-native comment give the full reason why it is needed.

---
 scripts/bitbake |   16 +++++++++++++---
 1 files changed, 13 insertions(+), 3 deletions(-)
scripts/bitbake +13 -3
index f02e89b..a8adeb7 100755
@@ -1,10 +1,20 @@
 #!/bin/sh
 
+# This is the bitbake wrapper script that ensures everything is set up
+# correctly in the environment, builds pseudo separately if it hasn't
+# already been built, and then runs bitbake within pseudo.
+
 export BBFETCH2=True
 export BB_ENV_EXTRAWHITE="PSEUDO_BUILD PSEUDO_DISABLED $BB_ENV_EXTRAWHITE"
 
+# For certain operations (i.e. that won't be actually running any tasks)
+# we don't need pseudo
 NO_BUILD_OPTS="--version -h --help -p --parse-only -s --show-versions -e --environment -g --graphviz"
+
+# Some options are useful to pass through to the initial pseudo build if
+# that needs to be run (for debugging)
 PASSTHROUGH_OPTS="-D -DD -DDD -DDDD -v"
+
 needpseudo="1"
 for opt in $@; do
 for key in $NO_BUILD_OPTS; do
@@ -29,7 +39,7 @@ if [ "$py_v3_check" != "" ]; then
 fi
 
 # Similarly, we now have code that doesn't parse correctly with older
-# versions of Python, and rather than fixing that and be eternally
+# versions of Python, and rather than fixing that and being eternally
 # vigilant for any other new feature use, just check the version here.
 py_v26_check=`python -c 'import sys; print sys.version_info >= (2,6,0)'`
 if [ "$py_v26_check" != "True" ]; then
@@ -53,8 +63,8 @@ float_test() {
      echo | awk 'END { exit ( !( '"$1"')); }'
 }
 
-# Tar version 1.24 and onwards handle symlinks in sstate packages correctly
-# but earlier versions do not
+# Tar version 1.24 and onwards handle overwriting symlinks correctly
+# but earlier versions do not; this needs to work properly for sstate
 float_test "$TARVERSION > 1.23" && needtar="0"
 
 buildpseudo="1"
Signature · 1 line
1.7.5.4
Acked-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] scripts/bitbake: try harder to check if pseudo exists

In reply to Paul Eggleton
Quoted 8 lines · Paul Eggleton

If pseudodone doesn't exist, we can get STAGING_BINDIR_NATIVE by calling bitbake -e and use that as the path to check for pseudo before we give up and try to build it explicitly first.

This is useful for people who share TMPDIR between multiple build directories.

This adds a 3s delay over the previous setup on my build system, which I deem acceptable. See one minor suggestion inline.

Quoted 9 lines
---
 scripts/bitbake |   12 +++++++++---
 1 files changed, 9 insertions(+), 3 deletions(-)
scripts/bitbake +0 -0
index 45c8697..1ba1839 100755

scripts/bitbake if [ "`pwd`" != "$BUILDDIR" ] ; then · line 60

Quoted 10 lines
scripts/bitbake +5 -2
@@ -53,14 +53,21 @@ if [ "`pwd`" != "$BUILDDIR" ] ; then
 fi
 
 buildpseudo="1"
-if [ $needpseudo = "1" ] && [ -e "$BUILDDIR/pseudodone" ]; then
-    PSEUDOBINDIR=`cat $BUILDDIR/pseudodone`
+if [ $needpseudo = "1" ]; then
+    if [ -e "$BUILDDIR/pseudodone" ]; then
+        PSEUDOBINDIR=`cat $BUILDDIR/pseudodone`
+    else
+        PSEUDOBINDIR=`bitbake -e | grep STAGING_BINDIR_NATIVE=\" | cut -d '=' -f2 | cut -d '"' -f2`

I think using sed might improve readability:

bitbake -e | grep ^STAGING_BINDIR_NATIVE= | sed -e 's,^STAGING_BINDIR_NATIVE="\(.*\)"$,\1,'

This may be a subjective impression, though.

Quoted 21 lines
scripts/bitbake +4 -1
+    fi
     if [ -e "$PSEUDOBINDIR/pseudo" -a -e "$PSEUDOBINDIR/tar" -a "$needtar" = "1" ]; then
         buildpseudo="0"
     fi
     if [ -e "$PSEUDOBINDIR/pseudo" -a $needtar = "0" ]; then
         buildpseudo="0"
     fi
+    if [ $buildpseudo = "0" -a ! -e "$BUILDDIR/pseudodone" ] ; then
+        echo $PSEUDOBINDIR > $BUILDDIR/pseudodone
+    fi
 fi
 if [ $needpseudo = "0" ]; then
     buildpseudo="0"
@@ -104,7 +111,6 @@ BITBAKE=`which bitbake`
 export PATH=$OLDPATH
 if [ $needpseudo = "1" ]; then
     export PSEUDO_BUILD=2
-    PSEUDOBINDIR=`cat $BUILDDIR/pseudodone`
     PSEUDO_BINDIR=$PSEUDOBINDIR PSEUDO_LIBDIR=$PSEUDOBINDIR/../lib/pseudo/lib PSEUDO_PREFIX=$PSEUDOBINDIR/../../ PSEUDO_DISABLED=1 $PSEUDOBINDIR/pseudo $BITBAKE $@
 else
     export PSEUDO_BUILD=0
Acked-by on v1 2/3
#5 lore raw

Re: [PATCH 2/3] scripts/bitbake: allow switching between build directories

In reply to Paul Eggleton
Quoted 11 lines · Paul Eggleton

The recent addition of the check to ensure the user was in their build directory disabled the ability to switch between build directories without re-running the build environment setup script. We can rely upon checking for conf/bblayers.conf instead, so use this check.

This does allow BUILDDIR (which is normally set by the environment script) to be unset; however if it is set then it is assumed to be the correct build directory and will be used in the error message that is shown when we can't find conf/bblayers.conf.

Quoted 37 lines
---
 scripts/bitbake |   15 ++++++++++-----
 1 files changed, 10 insertions(+), 5 deletions(-)
scripts/bitbake +10 -5
index 1ba1839..f02e89b 100755
@@ -37,6 +37,16 @@ if [ "$py_v26_check" != "True" ]; then
 	exit 1
 fi
 
+if [ ! -e conf/bblayers.conf ] ; then
+    BDPRINT=""
+    [ -n "$BUILDDIR" ] && BDPRINT=": $BUILDDIR"
+    echo "Unable to find conf/bblayers.conf"
+    echo "BitBake must be run from within your build directory$BDPRINT"
+    exit 1
+elif [ -z "$BUILDDIR" ] ; then
+    BUILDDIR="`pwd`"
+fi
+
 needtar="1"
 TARVERSION=`tar --version | head -n 1 | cut -d ' ' -f 4`
 float_test() {
@@ -47,11 +57,6 @@ float_test() {
 # but earlier versions do not
 float_test "$TARVERSION > 1.23" && needtar="0"
 
-if [ "`pwd`" != "$BUILDDIR" ] ; then
-    echo "BitBake must be run from your build directory: $BUILDDIR"
-    exit 1
-fi
-
 buildpseudo="1"
 if [ $needpseudo = "1" ]; then
     if [ -e "$BUILDDIR/pseudodone" ]; then
Acked-by on v1 3/3
#7 lore raw

Re: [PATCH 3/3] scripts/bitbake: add/fix some comments

In reply to Paul Eggleton
Quoted 5 lines · Paul Eggleton

Add some comments explaining what this script does, fix one grammatical error in a comment and make the tar-replacement-native comment give the full reason why it is needed.

Quoted 49 lines
---
 scripts/bitbake |   16 +++++++++++++---
 1 files changed, 13 insertions(+), 3 deletions(-)
scripts/bitbake +13 -3
index f02e89b..a8adeb7 100755
@@ -1,10 +1,20 @@
 #!/bin/sh
 
+# This is the bitbake wrapper script that ensures everything is set up
+# correctly in the environment, builds pseudo separately if it hasn't
+# already been built, and then runs bitbake within pseudo.
+
 export BBFETCH2=True
 export BB_ENV_EXTRAWHITE="PSEUDO_BUILD PSEUDO_DISABLED $BB_ENV_EXTRAWHITE"
 
+# For certain operations (i.e. that won't be actually running any tasks)
+# we don't need pseudo
 NO_BUILD_OPTS="--version -h --help -p --parse-only -s --show-versions -e --environment -g --graphviz"
+
+# Some options are useful to pass through to the initial pseudo build if
+# that needs to be run (for debugging)
 PASSTHROUGH_OPTS="-D -DD -DDD -DDDD -v"
+
 needpseudo="1"
 for opt in $@; do
 for key in $NO_BUILD_OPTS; do
@@ -29,7 +39,7 @@ if [ "$py_v3_check" != "" ]; then
 fi
 
 # Similarly, we now have code that doesn't parse correctly with older
-# versions of Python, and rather than fixing that and be eternally
+# versions of Python, and rather than fixing that and being eternally
 # vigilant for any other new feature use, just check the version here.
 py_v26_check=`python -c 'import sys; print sys.version_info >= (2,6,0)'`
 if [ "$py_v26_check" != "True" ]; then
@@ -53,8 +63,8 @@ float_test() {
      echo | awk 'END { exit ( !( '"$1"')); }'
 }
 
-# Tar version 1.24 and onwards handle symlinks in sstate packages correctly
-# but earlier versions do not
+# Tar version 1.24 and onwards handle overwriting symlinks correctly
+# but earlier versions do not; this needs to work properly for sstate
 float_test "$TARVERSION > 1.23" && needtar="0"
 
 buildpseudo="1"
#3 lore raw

Re: [PATCH 1/3] scripts/bitbake: try harder to check if pseudo exists

In reply to Andreas Oberritter
Quoted 9 lines · On Tuesday 27 March 2012 15:08:28 Andreas Oberritter wrote:
Quoted 2 lines · Paul Eggleton

+ PSEUDOBINDIR=`bitbake -e | grep STAGING_BINDIR_NATIVE=\" | cut -d
'=' -f2 | cut -d '"' -f2`

I think using sed might improve readability:

bitbake -e | grep ^STAGING_BINDIR_NATIVE= | sed -e 's,^STAGING_BINDIR_NATIVE="\(.*\)"$,\1,'

This may be a subjective impression, though.

I just copied this from further down in the script. Personally I find the original more readable although the sed version spawns less processes.

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#8 lore raw
In reply to Paul Eggleton
Quoted 13 lines · On Tuesday 27 March 2012 12:50:07 Paul Eggleton wrote:

Andreas, if you could review these that would be much appreciated.

The following changes since commit 644b7503c37fd73730dd3d7841463b158b8934ed:

guile: Deal with hardcoded path issues (2012-03-27 00:28:41 +0100)

are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/builddir

http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paul
e/builddir

Quoted 7 lines

Paul Eggleton (3):
scripts/bitbake: try harder to check if pseudo exists scripts/bitbake: allow switching between build directories scripts/bitbake: add/fix some comments

 scripts/bitbake |   43 ++++++++++++++++++++++++++++++++-----------
 1 files changed, 32 insertions(+), 11 deletions(-)

FYI I've rebased this patch on top of Mark's recent bitbake wrapper script changes that are now in master.

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
Applied
#9 lore raw
In reply to Paul Eggleton
Quoted 11 lines · Paul Eggleton

Andreas, if you could review these that would be much appreciated.

The following changes since commit 644b7503c37fd73730dd3d7841463b158b8934ed:

guile: Deal with hardcoded path issues (2012-03-27 00:28:41 +0100)

are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/builddir
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/builddir

Quoted 8 lines

Paul Eggleton (3):
scripts/bitbake: try harder to check if pseudo exists scripts/bitbake: allow switching between build directories scripts/bitbake: add/fix some comments

  scripts/bitbake |   43 ++++++++++++++++++++++++++++++++-----------
  1 files changed, 32 insertions(+), 11 deletions(-)

Merged into OE-Core

Thanks
Sau!