wic: fix for #11017
cover wic: fix for #11017
Hi,
This patchset improves handling of wic native tool dependencies and fixes minor bugs in the wic code.
The following changes since commit 9a0ddad5e178d16f302af2d05c32368572849fd3:
wic: pluginbase: use global dictionary (2017-02-20 11:58:28 +0200)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib ed/wic/wip
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/wic/wip
Ed Bartosh (6):
wic: use 2 sysroots to find native executable
wic: use image recipe sysroot as default
wic: exec_native_cmd: improve debug message
wic: exec_native_cmd: fix undefined variable error
wic: add WKS_FILE_DEPENDS variable
wic: raise WicError instead of calling logger.error
meta-selftest/recipes-test/images/wic-image-minimal.bb | 2 +- meta/classes/image_types_wic.bbclass | 2 ++ scripts/lib/wic/plugins/source/bootimg-pcbios.py | 4 ++-- scripts/lib/wic/utils/misc.py | 13 +++++++++---- scripts/wic | 2 +- 5 files changed, 15 insertions(+), 8 deletions(-)
--
Regards,
Ed
PATCH 1/6 wic: use 2 sysroots to find native executable
Currently there is no way to specify a dependency on native tools for wic without modifying wic-tools recipe. Obvious way to make it more flexible is to use image sysroot and wic-tools together to find an executable.
Modified run_native_cmd to use image and wic-tools sysroots to find native executable.
[YOCTO #11017]
--- scripts/lib/wic/utils/misc.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
scripts/lib/wic/utils/misc.py +6 -2
@@ -105,9 +105,13 @@ def exec_native_cmd(cmd_and_args, native_sysroot, catch=3, pseudo=""): if pseudo: cmd_and_args = pseudo + cmd_and_args + + wtools_sysroot = get_bitbake_var("RECIPE_SYSROOT_NATIVE", "wic-tools") + native_paths = \ - "%s/sbin:%s/usr/sbin:%s/usr/bin" % \ - (native_sysroot, native_sysroot, native_sysroot) + "%s/sbin:%s/usr/sbin:%s/usr/bin:%s/sbin:%s/usr/sbin:%s/usr/bin" % \ + (wtools_sysroot, wtools_sysroot, wtools_sysroot, + native_sysroot, native_sysroot, native_sysroot) native_cmd_and_args = "export PATH=%s:$PATH;%s" % \ (native_paths, cmd_and_args) logger.debug("exec_native_cmd: %s", cmd_and_args)
Signature · 1 line
2.1.4
PATCH 2/6 wic: use image recipe sysroot as default
In reply to Ed BartoshSwitched from using wic-tools recipe sysroot to using image sysroot in wic script. This way is more logical and consistent with the way wic uses other artifacts.
[YOCTO #11017]
--- scripts/wic | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/wic +1 -1
@@ -190,7 +190,7 @@ def wic_create_subcommand(args, usage_str): rootfs_dir = get_bitbake_var("IMAGE_ROOTFS", options.image_name) kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE", options.image_name) native_sysroot = get_bitbake_var("RECIPE_SYSROOT_NATIVE", - "wic-tools", cache=False) + options.image_name) #, cache=False) else: if options.build_rootfs: raise WicError("Image name is not specified, exiting. "
Signature · 1 line
2.1.4
PATCH 3/6 wic: exec_native_cmd: improve debug message
In reply to Ed BartoshAdded search paths to the debug message to make it easier to see where the native command is searched.
[YOCTO #11017]
--- scripts/lib/wic/utils/misc.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/lib/wic/utils/misc.py +1 -1
@@ -114,7 +114,7 @@ def exec_native_cmd(cmd_and_args, native_sysroot, catch=3, pseudo=""): native_sysroot, native_sysroot, native_sysroot) native_cmd_and_args = "export PATH=%s:$PATH;%s" % \ (native_paths, cmd_and_args) - logger.debug("exec_native_cmd: %s", cmd_and_args) + logger.debug("exec_native_cmd: %s", native_cmd_and_args) # If the command isn't in the native sysroot say we failed. if spawn.find_executable(args[0], native_paths):
Signature · 1 line
2.1.4
PATCH 4/6 wic: exec_native_cmd: fix undefined variable error
In reply to Ed BartoshDefined variable 'out' to fix this exception:
UnboundLocalError: local variable 'out' referenced before assignment
--- scripts/lib/wic/utils/misc.py | 1 + 1 file changed, 1 insertion(+)
scripts/lib/wic/utils/misc.py +1 -0
@@ -121,6 +121,7 @@ def exec_native_cmd(cmd_and_args, native_sysroot, catch=3, pseudo=""): ret, out = _exec_cmd(native_cmd_and_args, True, catch) else: ret = 127 + out = "can't find native executable %s in %s" % (args[0], native_paths) prog = args[0] # shell command-not-found
Signature · 1 line
2.1.4
PATCH 5/6 wic: add WKS_FILE_DEPENDS variable
In reply to Ed BartoshThis variable should be used to specify dependencies that will be added to DEPENDS only if wic image build is 'active' i.e. when 'wic' is in IMAGE_FSTYPES.
--- meta-selftest/recipes-test/images/wic-image-minimal.bb | 2 +- meta/classes/image_types_wic.bbclass | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-)
meta-selftest/recipes-test/images/wic-image-minimal.bb +1 -1
@@ -6,7 +6,7 @@ IMAGE_INSTALL = "packagegroup-core-boot ${ROOTFS_PKGMANAGE_BOOTSTRAP}" IMAGE_FSTYPES = "wic" -DEPENDS = "syslinux syslinux-native dosfstools-native mtools-native gptfdisk-native" +WKS_FILE_DEPENDS = "syslinux syslinux-native dosfstools-native mtools-native gptfdisk-native" LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
meta/classes/image_types_wic.bbclass +2 -0
@@ -40,6 +40,8 @@ USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + ' '.join('wic.%s WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' % os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}" do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}" do_image_wic[depends] += "wic-tools:do_build" +WKS_FILE_DEPENDS ??= '' +DEPENDS += "${@ '${WKS_FILE_DEPENDS}' if d.getVar('USING_WIC') else '' }" python do_write_wks_template () { """Write out expanded template contents to WKS_FULL_PATH."""
Signature · 1 line
2.1.4
PATCH 6/6 wic: raise WicError instead of calling logger.error
In reply to Ed BartoshReplaced forgotten calls of logger.error with raising WicError exception. Otherwise errors will be ignored.
--- scripts/lib/wic/plugins/source/bootimg-pcbios.py | 4 ++-- scripts/lib/wic/utils/misc.py | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-)
scripts/lib/wic/plugins/source/bootimg-pcbios.py +2 -2
@@ -96,8 +96,8 @@ class BootimgPcbiosPlugin(SourcePlugin): logger.debug("Using custom configuration file %s " "for syslinux.cfg", bootloader.configfile) else: - logger.error("configfile is specified but failed to " - "get it from %s.", bootloader.configfile) + raise WicError("configfile is specified but failed to " + "get it from %s." % bootloader.configfile) if not custom_cfg: # Create syslinux configuration using parameters from wks file
scripts/lib/wic/utils/misc.py +1 -1
@@ -136,7 +136,7 @@ def exec_native_cmd(cmd_and_args, native_sysroot, catch=3, pseudo=""): else: msg += "Wic failed to find a recipe to build native %s. Please "\ "file a bug against wic.\n" % prog - logger.error(msg) + raise WicError(msg) return ret, out
Signature · 1 line
2.1.4
Hi Ed,
Something in your latest patchset is breaking selftest:
The broken commit is somewhere in ross/mut-wic. Can you verify that I didn't leave something out, or otherwise chase this?
Ross
Quoted 40 lines · Ed Bartosh
Hi,
This patchset improves handling of wic native tool dependencies and fixes minor bugs in the wic code.
The following changes since commit
9a0ddad5e178d16f302af2d05c32368572849fd3:wic: pluginbase: use global dictionary (2017-02-20 11:58:28 +0200)
are available in the git repository at:
git://git.yoctoproject.org/poky-contribed/wic/wip
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/wic/wipEd Bartosh (6):
wic: use 2 sysroots to find native executable
wic: use image recipe sysroot as default
wic:exec_native_cmd: improve debug message
wic:exec_native_cmd: fix undefined variable error
wic: addWKS_FILE_DEPENDSvariable
wic: raise WicError instead of calling logger.errormeta-selftest/recipes-test/images/wic-image-minimal.bb | 2 +- meta/classes/image_types_wic.bbclass | 2 ++ scripts/lib/wic/plugins/source/bootimg-pcbios.py | 4 ++-- scripts/lib/wic/utils/misc.py | 13 +++++++++---- scripts/wic | 2 +- 5 files changed, 15 insertions(+), 8 deletions(-)--
Regards,
Ed--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Quoted 9 lines · Ross
Hi Ed,
Something in your latest patchset is breaking selftest:
The broken commit is somewhere in ross/mut-wic. Can you verify that I didn't leave something out, or otherwise chase this?
I couldn't reproduce this so far. Will investigate further.
Can you merge this patchset please?
http://lists.openembedded.org/pipermail/openembedded-core/2017-February/132854.html
The failure is most probably caused by patchset from this thread.
Regards,
Ed
Quoted 42 lines · Ed Bartosh
Quoted 41 lines
Hi,
This patchset improves handling of wic native tool dependencies and fixes minor bugs in the wic code.
The following changes since commit
9a0ddad5e178d16f302af2d05c32368572849fd3:wic: pluginbase: use global dictionary (2017-02-20 11:58:28 +0200)
are available in the git repository at:
git://git.yoctoproject.org/poky-contribed/wic/wip
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/wic/wipEd Bartosh (6):
wic: use 2 sysroots to find native executable
wic: use image recipe sysroot as default
wic:exec_native_cmd: improve debug message
wic:exec_native_cmd: fix undefined variable error
wic: addWKS_FILE_DEPENDSvariable
wic: raise WicError instead of calling logger.errormeta-selftest/recipes-test/images/wic-image-minimal.bb | 2 +- meta/classes/image_types_wic.bbclass | 2 ++ scripts/lib/wic/plugins/source/bootimg-pcbios.py | 4 ++-- scripts/lib/wic/utils/misc.py | 13 +++++++++---- scripts/wic | 2 +- 5 files changed, 15 insertions(+), 8 deletions(-)--
Regards,
Ed--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 3 lines
-- Regards, Ed
Quoted 10 lines · Ed Bartosh
Quoted 9 lines · Ross
Hi Ed,
Something in your latest patchset is breaking selftest:
The broken commit is somewhere in ross/mut-wic. Can you verify that I didn't leave something out, or otherwise chase this?
Quoted 7 lines
I couldn't reproduce this so far. Will investigate further.
Can you merge this patchset please?
http://lists.openembedded.org/pipermail/openembedded-core/2017-February/132854.htmlThe failure is most probably caused by patchset from this thread.
Still not able to reproduce it. It works just fine in my environment. This time I used setup as close to AB as I could get: core-image-sato image, genericx86-64 machine, clean build.
[build-genericx86-64 (ed/wic/wip)]$ rm -rf tmp/
[build-genericx86-64 (ed/wic/wip)]$ bitbake core-image-sato
WARNING: Host distribution "opensuse-13.2" has not been validated with
this version of the build system; you may possibly experience unexpected
failures. It is recommended that you use a tested distribution.
Parsing recipes: 100% |###############################################################################################################################################|
Time: 0:00:13
Parsing of 856 .bb files complete (0 cached, 856 parsed). 1322 targets,
53 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
Build Configuration:
BB_VERSION = "1.33.1" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "opensuse-13.2" TARGET_SYS = "x86_64-poky-linux" MACHINE = "genericx86-64" DISTRO = "poky" DISTRO_VERSION = "2.2+snapshot-20170222" TUNE_FEATURES = "m64 core2" TARGET_FPU = ""
meta
meta-poky
meta-yocto-bsp =
"ed/wic/wip:5d96bc6d1af89dda11f3266548fe1547152c86e1"
Initialising tasks: 100% |############################################################################################################################################|
Time: 0:00:15
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 6116 tasks of which 2807 didn't need to
be rerun and all succeeded.
Summary: There was 1 WARNING message shown.
[build-genericx86-64 (ed/wic/wip)]$ wic create directdisk -e core-image-sato
INFO: Creating image(s)...
INFO: The new image(s) can be found here:
./directdisk-201702231207-sda.direct
The following build artifacts were used to create the image(s):
ROOTFS_DIR:/home/ed/git/yocto/poky/build-genericx86-64/tmp/work/genericx86_64-poky-linux/core-image-sato/1.0-r0/rootfs
BOOTIMG_DIR:/home/ed/git/yocto/poky/build-genericx86-64/tmp/work/core2-64-poky-linux/wic-tools/1.0-r0/recipe-sysroot/usr/share
KERNEL_DIR:/home/ed/git/yocto/poky/build-genericx86-64/tmp/deploy/images/genericx86-64
NATIVE_SYSROOT:/home/ed/git/yocto/poky/build-genericx86-64/tmp/work/genericx86_64-poky-linux/core-image-sato/1.0-r0/recipe-sysroot-native
INFO: The image(s) were created using OE kickstart file:
/home/ed/git/yocto/poky/scripts/lib/wic/canned-wks/directdisk.wks
Could it be because of something in AB setup?
Regards,
Ed
Quoted 52 lines
Quoted 42 lines · Ed Bartosh
Quoted 41 lines
Hi,
This patchset improves handling of wic native tool dependencies and fixes minor bugs in the wic code.
The following changes since commit
9a0ddad5e178d16f302af2d05c32368572849fd3:wic: pluginbase: use global dictionary (2017-02-20 11:58:28 +0200)
are available in the git repository at:
git://git.yoctoproject.org/poky-contribed/wic/wip
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/wic/wipEd Bartosh (6):
wic: use 2 sysroots to find native executable
wic: use image recipe sysroot as default
wic:exec_native_cmd: improve debug message
wic:exec_native_cmd: fix undefined variable error
wic: addWKS_FILE_DEPENDSvariable
wic: raise WicError instead of calling logger.errormeta-selftest/recipes-test/images/wic-image-minimal.bb | 2 +- meta/classes/image_types_wic.bbclass | 2 ++ scripts/lib/wic/plugins/source/bootimg-pcbios.py | 4 ++-- scripts/lib/wic/utils/misc.py | 13 +++++++++---- scripts/wic | 2 +- 5 files changed, 15 insertions(+), 8 deletions(-)--
Regards,
Ed--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-coreSignature · 8 lines
-- Regards, Ed -- _______________________________________________ Openembedded-core mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 3 lines
-- Regards, Ed
Quoted 77 lines · Ed Bartosh
Quoted 10 lines · Ed Bartosh
Quoted 9 lines · Ross
Hi Ed,
Something in your latest patchset is breaking selftest:
The broken commit is somewhere in ross/mut-wic. Can you verify that I didn't leave something out, or otherwise chase this?
Quoted 7 lines
I couldn't reproduce this so far. Will investigate further.
Can you merge this patchset please?
http://lists.openembedded.org/pipermail/openembedded-core/2017-February/132854.htmlThe failure is most probably caused by patchset from this thread.
Still not able to reproduce it. It works just fine in my environment. This time I used setup as close to AB as I could get: core-image-sato image, genericx86-64 machine, clean build.
[build-genericx86-64 (ed/wic/wip)]$ rm -rf tmp/
[build-genericx86-64 (ed/wic/wip)]$ bitbake core-image-sato
WARNING: Host distribution "opensuse-13.2" has not been validated with
this version of the build system; you may possibly experience unexpected
failures. It is recommended that you use a tested distribution.
Parsing recipes: 100% |###############################################################################################################################################|
Time: 0:00:13
Parsing of 856 .bb files complete (0 cached, 856 parsed). 1322 targets,
53 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependenciesBuild Configuration:
BB_VERSION = "1.33.1" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "opensuse-13.2" TARGET_SYS = "x86_64-poky-linux" MACHINE = "genericx86-64" DISTRO = "poky" DISTRO_VERSION = "2.2+snapshot-20170222" TUNE_FEATURES = "m64 core2" TARGET_FPU = ""meta
meta-poky
meta-yocto-bsp =
"ed/wic/wip:5d96bc6d1af89dda11f3266548fe1547152c86e1"Initialising tasks: 100% |############################################################################################################################################|
Time: 0:00:15
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 6116 tasks of which 2807 didn't need to
be rerun and all succeeded.Summary: There was 1 WARNING message shown.
[build-genericx86-64 (ed/wic/wip)]$ wic create directdisk -e core-image-sato
INFO: Creating image(s)...INFO: The new image(s) can be found here:
./directdisk-201702231207-sda.directThe following build artifacts were used to create the image(s):
ROOTFS_DIR:/home/ed/git/yocto/poky/build-genericx86-64/tmp/work/genericx86_64-poky-linux/core-image-sato/1.0-r0/rootfs
BOOTIMG_DIR:/home/ed/git/yocto/poky/build-genericx86-64/tmp/work/core2-64-poky-linux/wic-tools/1.0-r0/recipe-sysroot/usr/share
KERNEL_DIR:/home/ed/git/yocto/poky/build-genericx86-64/tmp/deploy/images/genericx86-64
NATIVE_SYSROOT:/home/ed/git/yocto/poky/build-genericx86-64/tmp/work/genericx86_64-poky-linux/core-image-sato/1.0-r0/recipe-sysroot-native
Quoted 6 lines
INFO: The image(s) were created using OE kickstart file:
/home/ed/git/yocto/poky/scripts/lib/wic/canned-wks/directdisk.wksCould it be because of something in AB setup?
Now I'm pretty sure it's because of AB setup.
I was not able to reproduce it as I used genericx86-64 machine mentioned on the build page:
https://autobuilder.yoctoproject.org/main/builders/nightly-wic/builds/623
However, AB switched to qemux86-64 machine despite of what's specified. This is very confusing.
Anyway, the reason of the failure is that AB builds set of tools for wic in step 12: BuildImages Building syslinux syslinux-native parted-native gptfdisk-native dosfstools-native mtools-native, but due to RSS those tools are not accessible by wic. AB should build wic-tools instead.
Regards,
Ed
Quoted 62 lines
Quoted 52 lines
Quoted 42 lines · Ed Bartosh
Quoted 41 lines
Hi,
This patchset improves handling of wic native tool dependencies and fixes minor bugs in the wic code.
The following changes since commit
9a0ddad5e178d16f302af2d05c32368572849fd3:wic: pluginbase: use global dictionary (2017-02-20 11:58:28 +0200)
are available in the git repository at:
git://git.yoctoproject.org/poky-contribed/wic/wip
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/wic/wipEd Bartosh (6):
wic: use 2 sysroots to find native executable
wic: use image recipe sysroot as default
wic:exec_native_cmd: improve debug message
wic:exec_native_cmd: fix undefined variable error
wic: addWKS_FILE_DEPENDSvariable
wic: raise WicError instead of calling logger.errormeta-selftest/recipes-test/images/wic-image-minimal.bb | 2 +- meta/classes/image_types_wic.bbclass | 2 ++ scripts/lib/wic/plugins/source/bootimg-pcbios.py | 4 ++-- scripts/lib/wic/utils/misc.py | 13 +++++++++---- scripts/wic | 2 +- 5 files changed, 15 insertions(+), 8 deletions(-)--
Regards,
Ed--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-coreSignature · 8 lines
-- Regards, Ed -- _______________________________________________ Openembedded-core mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-coreSignature · 8 lines
-- Regards, Ed -- _______________________________________________ Openembedded-core mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 3 lines
-- Regards, Ed
Quoted 2 lines · Ed Bartosh
This patchset improves handling of wic native tool dependencies and fixes minor bugs in the wic code.
I haven't tested it, but the patches themselves look good to me. Thanks!
Signature · 6 lines
Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.