Skip to contentSkip to conversation

Remove bitbake's knowledge of ${B}

RFC PATCH In review Ross Burton 2 participants 11 messages Jul 13, 2015 – Jul 14, 2015

lore
Share & tools
lore
#0 lore raw

cover Remove bitbake's knowledge of ${B}

Hi,

So when starting a new task if you don't tell bitbake what directories it should create (and implicit, chdir() into) it uses ${B}. Which is bitbake knowing about openembedded variables, and generally frowned upon.

This branch series removes that knowledge, and if a task is executed without any directories to create or chdir into it just doesn't do anything.

This revealed a few classes and recipes which assume they're being ran in ${B} so the rest of the series is either adding back [dirs] flags to the tasks, or making paths absolute.

Testing has been quite minimal: core-image-sato builds but I've not verified the contents haven't changed yet.

Comments?

Ross

The following changes since commit 7f8286fb9ec9f9d9f606c69e5abd717078ace5b2:

classes/sstate.bbclass: Improve checkstatus using connection cache. (2015-07-12 22:55:46 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib ross/dirs

for you to fetch changes up to b306a4b740868835ff0e0ac14d8b7ed9afe83091:

kernel: fix path assumptions (2015-07-13 14:51:57 +0100)

----------------------------------------------------------------
Ross Burton (7):
build: don't automatically mkdir/chdir B
build: add remove/mkdir/chdir debugging
ptest: add dirs
boost: add dirs
python: add explicit directory when copying
libc-package: fix path assumption
kernel: fix path assumptions

 bitbake/lib/bb/build.py                       | 19 ++++++++++++++++++-
 meta/classes/kernel.bbclass                   |  8 ++++----
 meta/classes/libc-package.bbclass             |  3 +--
 meta/classes/ptest.bbclass                    |  3 +++
 meta/recipes-devtools/python/python3_3.4.3.bb |  2 +-
 meta/recipes-devtools/python/python_2.7.9.bb  |  2 +-
 meta/recipes-support/boost/boost.inc          |  1 +
 7 files changed, 29 insertions(+), 9 deletions(-)

Ross Burton (7):
build: don't automatically mkdir/chdir B
build: add remove/mkdir/chdir debugging
ptest: add dirs
boost: add dirs
python: add explicit directory when copying
libc-package: fix path assumption
kernel: fix path assumptions

 bitbake/lib/bb/build.py                       | 19 ++++++++++++++++++-
 meta/classes/kernel.bbclass                   |  8 ++++----
 meta/classes/libc-package.bbclass             |  3 +--
 meta/classes/ptest.bbclass                    |  3 +++
 meta/recipes-devtools/python/python3_3.4.3.bb |  2 +-
 meta/recipes-devtools/python/python_2.7.9.bb  |  2 +-
 meta/recipes-support/boost/boost.inc          |  1 +
 7 files changed, 29 insertions(+), 9 deletions(-)
Signature · 1 line
2.1.4
#1 lore raw

PATCH 1/7 build: don't automatically mkdir/chdir B

---
 bitbake/lib/bb/build.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
bitbake/lib/bb/build.py +3 -1
index cce01fe..7cefee7 100644
@@ -183,7 +183,9 @@ def exec_func(func, d, dirs = None):
         adir = dirs[-1]
     else:
         adir = d.getVar('B', True)
-        bb.utils.mkdirhier(adir)
+        bb.note("NOT creating implicit %s (for %s)" % (adir, func))
+        #bb.utils.mkdirhier(adir)
+        adir = None
 
     ispython = flags.get('python')
 
Signature · 1 line
2.1.4
#2 lore raw

PATCH 2/7 build: add remove/mkdir/chdir debugging

In reply to Ross Burton
---
 bitbake/lib/bb/build.py | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
bitbake/lib/bb/build.py +15 -0
index 7cefee7..fa297c3 100644
@@ -169,6 +169,7 @@ def exec_func(func, d, dirs = None):
     cleandirs = flags.get('cleandirs')
     if cleandirs:
         for cdir in d.expand(cleandirs).split():
+            bb.note("removing %s (for %s)" % (cdir, func))
             bb.utils.remove(cdir, True)
             bb.utils.mkdirhier(cdir)
 
@@ -179,6 +180,7 @@ def exec_func(func, d, dirs = None):
 
     if dirs:
         for adir in dirs:
+            bb.note("creating explicit %s (for %s)" % (adir, func))
             bb.utils.mkdirhier(adir)
         adir = dirs[-1]
     else:
@@ -251,7 +253,13 @@ def exec_func_python(func, d, runfile, cwd=None):
             olddir = os.getcwd()
         except OSError:
             olddir = None
+        bb.note("chdir(%s) for %s" % (cwd, func))
         os.chdir(cwd)
+    else:
+        try:
+            bb.note("pwd(%s) for %s" % (os.getcwd(), func))
+        except:
+            bb.note("no chdir for %s" % func)
 
     bb.debug(2, "Executing python function %s" % func)
 
@@ -301,6 +309,13 @@ def exec_func_shell(func, d, runfile, cwd=None):
     # Don't let the emitted shell script override PWD
     d.delVarFlag('PWD', 'export')
 
+    if cwd:
+        bb.note("shell for %s cd %s" % (func, cwd))
+    else:
+        try:
+            bb.note("pwd(%s) for %s" % (os.getcwd(), func))
+        except:
+            bb.note("no cd for %s" % func)
     with open(runfile, 'w') as script:
         script.write(shell_trap_code())
 
Signature · 1 line
2.1.4
#3 lore raw

PATCH 3/7 ptest: add dirs

In reply to Ross Burton
---
 meta/classes/ptest.bbclass | 3 +++
 1 file changed, 3 insertions(+)
meta/classes/ptest.bbclass +3 -0
index 101ab9d..2ac9143 100644
@@ -46,6 +46,9 @@ do_install_ptest_base() {
     fi
 }
 
+do_configure_ptest_base[dirs] = "${B}"
+do_compile_ptest_base[dirs] = "${B}"
+do_install_ptest_base[dirs] = "${B}"
 do_install_ptest_base[cleandirs] = "${D}${PTEST_PATH}"
 
 addtask configure_ptest_base after do_configure before do_compile
Signature · 1 line
2.1.4
#4 lore raw

PATCH 4/7 boost: add dirs

In reply to Ross Burton
---
 meta/recipes-support/boost/boost.inc | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-support/boost/boost.inc +1 -0
index cf4faca..27d489f 100644
@@ -154,6 +154,7 @@ do_boostconfig() {
 	sed -i '/^using python/d' project-config.jam
 }
 
+do_boostconfig[dirs] = "${S}"
 addtask do_boostconfig after do_patch before do_configure
 
 do_compile() {
Signature · 1 line
2.1.4
#5 lore raw

PATCH 5/7 python: add explicit directory when copying

In reply to Ross Burton
---
 meta/recipes-devtools/python/python3_3.4.3.bb | 2 +-
 meta/recipes-devtools/python/python_2.7.9.bb  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
meta/recipes-devtools/python/python3_3.4.3.bb +1 -1
index 3efa7de..dce778a 100644
@@ -180,7 +180,7 @@ PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess"
 
 py_package_preprocess () {
 	# copy back the old Makefile to fix target package
-	install -m 0644 Makefile.orig ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile
+	install -m 0644 ${B}/Makefile.orig ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile
 	# Remove references to buildmachine paths in target Makefile
 	sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile
 }
meta/recipes-devtools/python/python_2.7.9.bb +1 -1
index 19190c3..803a1fc 100644
@@ -135,7 +135,7 @@ PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess"
 
 py_package_preprocess () {
 	# copy back the old Makefile to fix target package
-	install -m 0644 Makefile.orig ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile
+	install -m 0644 ${B}/Makefile.orig ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile
 
 	# Remove references to buildmachine paths in target Makefile
 	sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile
Signature · 1 line
2.1.4
#6 lore raw

PATCH 6/7 libc-package: fix path assumption

In reply to Ross Burton

Relying on exec_func() doing a mkdir and chdir if you set ${B} is obscure, instead just pass -C <dir> to oe_runmake.

---
 meta/classes/libc-package.bbclass | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
meta/classes/libc-package.bbclass +1 -2
index 2076aa1..21c6b43 100644
@@ -363,10 +363,9 @@ python package_do_split_gconvs () {
             m.write(cmd + ":\n")
             m.write("\t" + commands[cmd] + "\n\n")
         m.close()
-        d.setVar("B", os.path.dirname(makefile))
         d.setVar("EXTRA_OEMAKE", "${PARALLEL_MAKE}")
         bb.note("Executing binary locale generation makefile")
-        bb.build.exec_func("oe_runmake", d)
+        bb.build.exec_func("oe_runmake", d, [os.path.dirname(makefile),])
         bb.note("collecting binary locales from locale tree")
         bb.build.exec_func("do_collect_bins_from_locale_tree", d)
         do_split_packages(d, binary_locales_dir, file_regex='(.*)', \
Signature · 1 line
2.1.4
#7 lore raw

Re: [PATCH 6/7] libc-package: fix path assumption

Quoted 16 lines · Ross Burton

Relying on exec_func() doing a mkdir and chdir if you set ${B} is obscure, instead just pass -C <dir> to oe_runmake.

---
 meta/classes/libc-package.bbclass | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
meta/classes/libc-package.bbclass +0 -0
index 2076aa1..21c6b43 100644
@@ -363,10 +363,9 @@ python package_do_split_gconvs () {
             m.write(cmd + ":\n")

meta/classes/libc-package.bbclass python package_do_split_gconvs () { · line 371

Quoted 10 lines
meta/classes/libc-package.bbclass +1 -2
             m.write("\t" + commands[cmd] + "\n\n")
         m.close()
-        d.setVar("B", os.path.dirname(makefile))
         d.setVar("EXTRA_OEMAKE", "${PARALLEL_MAKE}")
         bb.note("Executing binary locale generation makefile")
-        bb.build.exec_func("oe_runmake", d)
+        bb.build.exec_func("oe_runmake", d, [os.path.dirname(makefile),])
         bb.note("collecting binary locales from locale tree")
         bb.build.exec_func("do_collect_bins_from_locale_tree", d)
         do_split_packages(d, binary_locales_dir, file_regex='(.*)', \

This says you’re passing -C <dir> to oe_runmake, but actually passes the dir as dirs for exec_func.

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
#8 lore raw

Re: [PATCH 6/7] libc-package: fix path assumption

Quoted 3 lines · Christopher Larson

This says you’re passing -C <dir> to oe_runmake, but actually passes the dir as dirs for exec_func.

Good point - passing -C doesn't work as I was being stupid (bitbake can't find a function called "oe_runmake -C ${B}"). Fixing comment in the repo.

Ross

#9 lore raw

PATCH 7/7 kernel: fix path assumptions

In reply to Ross Burton
---
 meta/classes/kernel.bbclass | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
meta/classes/kernel.bbclass +4 -4
index 76d03dc..d06f6cf 100644
@@ -211,8 +211,8 @@ kernel_do_compile() {
 
 do_compile_kernelmodules() {
 	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
-	if (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
-		oe_runmake ${PARALLEL_MAKE} modules CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
+	if (grep -q -i -e '^CONFIG_MODULES=y$' ${B}/.config); then
+		oe_runmake -C ${B} ${PARALLEL_MAKE} modules CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
 	else
 		bbnote "no modules to compile"
 	fi
@@ -260,7 +260,7 @@ emit_depmod_pkgdata() {
 	# Stash data for depmod
 	install -d ${PKGDESTWORK}/kernel-depmod/
 	echo "${KERNEL_VERSION}" > ${PKGDESTWORK}/kernel-depmod/kernel-abiversion
-	cp System.map ${PKGDESTWORK}/kernel-depmod/System.map-${KERNEL_VERSION}
+	cp ${B}/System.map ${PKGDESTWORK}/kernel-depmod/System.map-${KERNEL_VERSION}
 }
 
 PACKAGEFUNCS += "emit_depmod_pkgdata"
@@ -336,7 +336,7 @@ kernel_do_configure() {
 }
 
 do_savedefconfig() {
-	oe_runmake savedefconfig
+	oe_runmake -C ${B} savedefconfig
 }
 do_savedefconfig[nostamp] = "1"
 addtask savedefconfig after do_configure
Signature · 1 line
2.1.4
#10 lore raw
In reply to Ross Burton
Quoted 9 lines · Ross Burton

Ross Burton (7):
build: don't automatically mkdir/chdir B
build: add remove/mkdir/chdir debugging
ptest: add dirs
boost: add dirs
python: add explicit directory when copying
libc-package: fix path assumption
kernel: fix path assumptions

FYI, I've fixed the libc-package fix (to pass -C to EXTRA_OEMAKE), elaborated on the commit messages, and merged the oe-core parts into my master-under-test branch. The actual change to bitbake is still not merged and open for comments.

Ross