Enable wic in eSDK

From: Chang Rebecca Swee Fun <[email protected]>

Hi all,

As the subject called out: this patch series enable wic in eSDK. The details of what I have done are documented within the commit message. Basically wic requires an OE build environment, but we are using a different environment setup script in eSDK. Hence, I have added some code for wic to explicitly export bitbake variables within eSDK. I have also make wic to use the shared code in scriptpath for oe lib and bitbake path addition to sys.path.

I have run the changes on wic oe-selftest and the tests are passing. What's next: I think it would better to have some test cases for wic within eSDK if this series are merged.

Thanks.

Regards,
Rebecca

The following changes since commit 205cfd702190026e64eed9cae27c05ff62d1637e:

bitbake: fetch2/__init__: Disable pseudo in runfetchcmd() (2018-01-08 08:48:54 +0000)

are available in the git repository at:

git://push.yoctoproject.org/poky-contrib rebeccas/wic-dev

Chang Rebecca Swee Fun (5):
scripts/wic: use scriptpath module to find bitbake path and oe lib
path
scripts/wic: append bitbake executable file path in eSDK environment scripts/wic: fix error of import wic module in eSDK environment scripts/wic: explicitly set BUILDDIR within eSDK classes/populate_sdk_ext: support wic in eSDK

 meta/classes/populate_sdk_ext.bbclass |  2 +-
 scripts/wic                           | 23 ++++++++++++++++++-----
 2 files changed, 19 insertions(+), 6 deletions(-)

2.7.4

From: Chang Rebecca Swee Fun <[email protected]>

Use the scriptpath module in order to standardize the adding of bitbake and meta/lib path to sys.path.

---
 scripts/wic | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

index 097084a..0d98875 100755
@@ -43,13 +43,12 @@ from distutils import spawn
 scripts_path = os.path.abspath(os.path.dirname(__file__))
 lib_path = scripts_path + '/lib'
 sys.path.insert(0, lib_path)
-oe_lib_path = os.path.join(os.path.dirname(scripts_path), 'meta', 'lib')
-sys.path.insert(0, oe_lib_path)
+import scriptpath
+scriptpath.add_oe_lib_path()
 
 bitbake_exe = spawn.find_executable('bitbake')
 if bitbake_exe:
-    bitbake_path = os.path.join(os.path.dirname(bitbake_exe), '../lib')
-    sys.path.insert(0, bitbake_path)
+    bitbake_path = scriptpath.add_bitbake_lib_path()
     from bb import cookerdata
     from bb.main import bitbake_main, BitBakeConfigParameters
 else:

2.7.4

From: Chang Rebecca Swee Fun <[email protected]>

wic needs a set of tools to be available from sysroots. wic will find bitbake executable within the environment, and wic was unable to locate bitbake executable within eSDK because it wasn't setup with the OE build environment script. Hence, we need to add bitbake file path into the environment PATH for wic to be able to discover it and import bb modules.

---
 scripts/wic | 12 ++++++++++++
 1 file changed, 12 insertions(+)

index 0d98875..293a216 100755
@@ -46,6 +46,18 @@ sys.path.insert(0, lib_path)
 import scriptpath
 scriptpath.add_oe_lib_path()
 
+# Check whether wic is running within eSDK environment
+sdkroot = scripts_path
+if os.environ.get('SDKTARGETSYSROOT'):
+    while sdkroot != '' and sdkroot != os.sep:
+        if os.path.exists(os.path.join(sdkroot, '.devtoolbase')):
+            # .devtoolbase only exists within eSDK
+            # If found, initialize bitbake path for eSDK environment and append to PATH
+            sdkroot = os.path.join(os.path.dirname(scripts_path), 'bitbake', 'bin')
+            os.environ['PATH'] += ":" + sdkroot
+            break
+        sdkroot = os.path.dirname(sdkroot)
+
 bitbake_exe = spawn.find_executable('bitbake')
 if bitbake_exe:
     bitbake_path = scriptpath.add_bitbake_lib_path()

2.7.4

From: Chang Rebecca Swee Fun <[email protected]>

wic modules in scripts/lib/ are needed for wic to work, but path to the python module is not exported in eSDK environment and we were using an absolutized path of wic script within the sysroots.

We now changed to use real script path instead, where the wic modules are located. This will also resolved the tracebacks found when running wic from within the eSDK environment.

Traceback (most recent call last):
File "/tmp/deploy/sdk/poky_sdk/sysroots/x86_64-pokysdk-linux/usr/bin/wic", line 58, in <module>
from wic import WicError
ImportError: No module named 'wic'

---
 scripts/wic | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index 293a216..d9bea22 100755
@@ -40,7 +40,7 @@ from collections import namedtuple
 from distutils import spawn
 
 # External modules
-scripts_path = os.path.abspath(os.path.dirname(__file__))
+scripts_path = os.path.dirname(os.path.realpath(__file__))
 lib_path = scripts_path + '/lib'
 sys.path.insert(0, lib_path)
 import scriptpath

2.7.4

From: Chang Rebecca Swee Fun <[email protected]>

When we run wic within eSDK:

$ wic create mkefidisk -e core-image-minimal

ERROR: BUILDDIR not found, exiting. (Did you forget to source oe-init-build-env?)

In order to figure out variable values, one must have sourced the OE build environment setup script. However, when we are in within the eSDK environment which isn't initialised like the normal OE build environment, we can't use wic utility with eSDK.

Reference:
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#wic-requirements

While wic ought to be fixed to be able to run without bitbake & native tools [YOCTO #11281], but this is a workaround to set BUILDDIR in the environment so that bitbake environment is setup for wic to build its required native tools.

---
 scripts/wic | 2 ++
 1 file changed, 2 insertions(+)

index d9bea22..b02d907 100755
@@ -55,6 +55,8 @@ if os.environ.get('SDKTARGETSYSROOT'):
             # If found, initialize bitbake path for eSDK environment and append to PATH
             sdkroot = os.path.join(os.path.dirname(scripts_path), 'bitbake', 'bin')
             os.environ['PATH'] += ":" + sdkroot
+            # Set BUILDDIR for wic to work within eSDK
+            os.environ['BUILDDIR'] = sdkroot
             break
         sdkroot = os.path.dirname(sdkroot)
 

2.7.4

From: Chang Rebecca Swee Fun <[email protected]>

Make 'wic' image creation tool/command available in eSDK environment. This would allow eSDK users to manipulate images within eSDK environment.

[YOCTO #12177]

---
 meta/classes/populate_sdk_ext.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index eabc365..4f7100d 100644
@@ -533,7 +533,7 @@ def get_sdk_required_utilities(buildtools_fn, d):
 
 install_tools() {
 	install -d ${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}
-	scripts="devtool recipetool oe-find-native-sysroot runqemu*"
+	scripts="devtool recipetool oe-find-native-sysroot runqemu* wic"
 	for script in $scripts; do
 		for scriptfn in `find ${SDK_OUTPUT}/${SDKPATH}/${scriptrelpath} -maxdepth 1 -executable -name "$script"`; do
 			lnr ${scriptfn} ${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/`basename $scriptfn`

2.7.4