Skip to contentSkip to conversation

core-image-tiny-initramfs failing to parse because of WIC (WKS_FULL_PATH, )

PATCH Superseded Martin Jansa 3 participants 8 messages Feb 9, 2017 – Feb 22, 2017

lore
Share & tools
lore
#0 lore raw

Running e.g. bitbake -e busybox with latest oe-core triggers this badly looking exception:

Can we add some weak assignment for people who don't care about wic or core-image-tiny-initramfs to continue using oe-core?

ERROR: ExpansionError during parsing /OE/oe-core/meta/recipes-core/images/ core-image-tiny-initramfs.bb
Traceback (most recent call last):
File "/OE/bitbake/lib/bb/data_smart.py", line 412, in
DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif [ -z "$wks" ]; then\n\t\tbbfatal "No kickstart files from WKS_FILES were
found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic create "$wks" --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}\n\tmv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm -rf "$out/"\n', varname='IMAGE_CMD'):
try:
> s = __expand_var_regexp__.sub(varparse.var_sub, s)
try:
File "/OE/bitbake/lib/bb/data_smart.py", line 111, in
VariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59), match='${WKS_FULL_PATH}'>):
else:
> var = self.d.getVarFlag(key, "_content")
self.references.add(key)
File "/OE/bitbake/lib/bb/data_smart.py", line 773, in
DataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content', expand=True, noweakdefault=False, parsing=False):
cachename = var + "[" + flag + "]"
> value = self.expand(value, cachename)

File "/OE/bitbake/lib/bb/data_smart.py", line 436, in
DataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
def expand(self, s, varname = None):
> return self.expandWithRefs(s, varname).value

File "/OE/bitbake/lib/bb/data_smart.py", line 426, in
DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or ''}", varname='WKS_FULL_PATH'):
except Exception as exc:
> raise ExpansionError(varname, s, exc) from exc

bb.data_smart.ExpansionError: Failure expanding variable WKS_FULL_PATH, expression was ${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or ''} which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)

#1 lore raw

Hi Martin,

Sorry for the delay. I missedd your message somehow.

I can't reproduce this. Tried with latest poky and latest oe-core + bitbake. I built core-image-tiny-initramfs from scratch and bitbake -e busybox works for me just fine.

Can you provide more details about your setup, please?

Quoted 48 lines · Martin Jansa

Running e.g. bitbake -e busybox with latest oe-core triggers this badly looking exception:

Can we add some weak assignment for people who don't care about wic or core-image-tiny-initramfs to continue using oe-core?

ERROR: ExpansionError during parsing /OE/oe-core/meta/recipes-core/images/ core-image-tiny-initramfs.bb
Traceback (most recent call last):
File "/OE/bitbake/lib/bb/data_smart.py", line 412, in
DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif [ -z "$wks" ]; then\n\t\tbbfatal "No kickstart files from WKS_FILES were
found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic create "$wks" --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}\n\tmv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm -rf "$out/"\n', varname='IMAGE_CMD'):
try:
> s = __expand_var_regexp__.sub(varparse.var_sub, s)
try:
File "/OE/bitbake/lib/bb/data_smart.py", line 111, in
VariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59), match='${WKS_FULL_PATH}'>):
else:
> var = self.d.getVarFlag(key, "_content")
self.references.add(key)
File "/OE/bitbake/lib/bb/data_smart.py", line 773, in
DataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content', expand=True, noweakdefault=False, parsing=False):
cachename = var + "[" + flag + "]"
> value = self.expand(value, cachename)

File "/OE/bitbake/lib/bb/data_smart.py", line 436, in
DataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
def expand(self, s, varname = None):
> return self.expandWithRefs(s, varname).value

File "/OE/bitbake/lib/bb/data_smart.py", line 426, in
DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or ''}", varname='WKS_FULL_PATH'):
except Exception as exc:
> raise ExpansionError(varname, s, exc) from exc

Quoted 8 lines

bb.data_smart.ExpansionError: Failure expanding variable WKS_FULL_PATH, expression was ${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or ''} which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)

--
Regards,
Ed

#2 lore raw
Quoted 9 lines · Ed Bartosh

Hi Martin,

Sorry for the delay. I missedd your message somehow.

I can't reproduce this. Tried with latest poky and latest oe-core + bitbake. I built core-image-tiny-initramfs from scratch and bitbake -e busybox works for me just fine.

Can you provide more details about your setup, please?

Hi Ed,

I was also having troubles reproducing it in different setup as well, until I've found out that my local.conf had this line:

IMAGE_FSTYPES_qemux86 = "vmdk"

which prevented ever using wic (USING_WIC variable was always empty).

Now with this line removed I can share the complete exception reproduced with just oe-core layer included:

$ bitbake -e core-image-tiny-initramfs | tee env.core-image-tiny-initramfs5
NOTE: Started PRServer with DBfile: /OE/build/oe-core/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 38869, PID: 21506
Parsing recipes...ERROR: ExpansionError during parsing /OE/build/oe-core/openembedded-core/meta/recipes-core/images/core-image-tiny-initramfs.bb
Traceback (most recent call last):
  File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 412, in DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif [ -z "$wks" ]; then\n\t\tbbfatal "No kickstart files from WKS_FILES were found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic create "$wks" --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}\n\tmv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm -rf "$out/"\n', varname='IMAGE_CMD'):
                 try:
Quoted 1 line
               s = __expand_var_regexp__.sub(varparse.var_sub, s)

try:
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 111, in VariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59), match='${WKS_FULL_PATH}'>):
else:
> var = self.d.getVarFlag(key, "_content")
self.references.add(key)
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 773, in DataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content', expand=True, noweakdefault=False, parsing=False):
cachename = var + "[" + flag + "]"
> value = self.expand(value, cachename)

File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 436, in DataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
def expand(self, s, varname = None):
> return self.expandWithRefs(s, varname).value

File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 426, in DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/build/oe-core/openembedded-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta:/OE/build/oe-core/openembedded-core'.split(':'))}') or ''}", varname='WKS_FULL_PATH'):
except Exception as exc:
> raise ExpansionError(varname, s, exc) from exc

bb.data_smart.ExpansionError: Failure expanding variable WKS_FULL_PATH, expression was ${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/build/oe-core/openembedded-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta:/OE/build/oe-core/openembedded-core'.split(':'))}') or ''} which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)

Summary: There was 1 ERROR message shown, returning a non-zero exit code.

The strange part of this exception is the unexpanded ${TMPDIR} in the WKS_SEARCH_PATH value, that lead me to following 2 lines in conf/bblayers.conf:

TMPDIR := "/OE/build/oe-core"
BBPATH = "${TOPDIR}:${TMPDIR}"

I'm not sure why it appears unexpanded in WKS_SEARCH_PATH, I see you do some magick with all these WKS_* variables, especially in that anonymous python, but the exception is actually from expanding IMAGE_CMD_wic () function and it goes away if I remove following line from it:

        wks="${WKS_FULL_PATH}"

And it also goes away if I use immediate expansion when setting that BBPATH in bblayers.conf.

Interesting variables from bitbake -e with immediate expansion:
# $BBPATH [4 operations]
# immediate /OE/build/oe-core/conf/bblayers.conf:6
# "${TOPDIR}:${TMPDIR}"
# postdot /OE/build/oe-core/openembedded-core/meta/conf/layer.conf:2
# ":${LAYERDIR}"
# set data_smart.py:911 [expandVarref]
# "/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-core/openembedded-core/meta"
# set /OE/build/oe-core/openembedded-core/meta/conf/documentation.conf:94
# [doc] "Used by BitBake to locate .bbclass and configuration files. This variable is analogous to the PATH variable."

# pre-expansion value:
#   "/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-core/openembedded-core/meta"
BBPATH="/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-core/openembedded-core/meta"

# $WKS_SEARCH_PATH
# set? /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.bbclass:10
# "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}"
WKS_SEARCH_PATH="/OE/build/oe-core/openembedded-core/meta/recipes-core/images:/OE/build/oe-core/wic:/OE/build/oe-core/wic:/OE/build/oe-core/openembedded-core/meta/wic:/OE/build/oe-core/scripts/lib/wic/canned-wks:/OE/build/oe-core/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/meta/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks"

# $WKS_FULL_PATH
# set /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.bbclass:11
# "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
WKS_FULL_PATH="/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks/directdisk.wks"

And the same without immediate expansion, but with 'wks="${WKS_FULL_PATH}"' line removed so that it doesn't fail to parse:
# $BBPATH [4 operations]
# set /OE/build/oe-core/conf/bblayers.conf:6
# "${TOPDIR}:${TMPDIR}"
# postdot /OE/build/oe-core/openembedded-core/meta/conf/layer.conf:2
# ":${LAYERDIR}"
# set data_smart.py:911 [expandVarref]
# "${TOPDIR}:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta"
# set /OE/build/oe-core/openembedded-core/meta/conf/documentation.conf:94
# [doc] "Used by BitBake to locate .bbclass and configuration files. This variable is analogous to the PATH variable."

# pre-expansion value:
#   "${TOPDIR}:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta"
BBPATH="/OE/build/oe-core:/OE/build/oe-core-glibc:/OE/build/oe-core/openembedded-core/meta"

# $WKS_SEARCH_PATH
# set? /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.bbclass:10
# "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}"
WKS_SEARCH_PATH="/OE/build/oe-core/openembedded-core/meta/recipes-core/images:/OE/build/oe-core/wic:/OE/build/oe-core-glibc/wic:/OE/build/oe-core/openembedded-core/meta/wic:/OE/build/oe-core/scripts/lib/wic/canned-wks:/OE/build/oe-core-glibc/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/meta/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks"

# $WKS_FULL_PATH
# set /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.bbclass:11
# "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
WKS_FULL_PATH="/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks/directdisk.wks"

Let me know if sharing "bitbake -e core-image-tiny-initramfs" from these 2 tests would be useful for you.

Regards,

Quoted 62 lines · Martin Jansa
Quoted 48 lines

Running e.g. bitbake -e busybox with latest oe-core triggers this badly looking exception:

Can we add some weak assignment for people who don't care about wic or core-image-tiny-initramfs to continue using oe-core?

ERROR: ExpansionError during parsing /OE/oe-core/meta/recipes-core/images/ core-image-tiny-initramfs.bb
Traceback (most recent call last):
File "/OE/bitbake/lib/bb/data_smart.py", line 412, in
DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif [ -z "$wks" ]; then\n\t\tbbfatal "No kickstart files from WKS_FILES were
found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic create "$wks" --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}\n\tmv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm -rf "$out/"\n', varname='IMAGE_CMD'):
try:
> s = __expand_var_regexp__.sub(varparse.var_sub, s)
try:
File "/OE/bitbake/lib/bb/data_smart.py", line 111, in
VariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59), match='${WKS_FULL_PATH}'>):
else:
> var = self.d.getVarFlag(key, "_content")
self.references.add(key)
File "/OE/bitbake/lib/bb/data_smart.py", line 773, in
DataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content', expand=True, noweakdefault=False, parsing=False):
cachename = var + "[" + flag + "]"
> value = self.expand(value, cachename)

File "/OE/bitbake/lib/bb/data_smart.py", line 436, in
DataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
def expand(self, s, varname = None):
> return self.expandWithRefs(s, varname).value

File "/OE/bitbake/lib/bb/data_smart.py", line 426, in
DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or ''}", varname='WKS_FULL_PATH'):
except Exception as exc:
> raise ExpansionError(varname, s, exc) from exc

Quoted 8 lines

bb.data_smart.ExpansionError: Failure expanding variable WKS_FULL_PATH, expression was ${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or ''} which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)

--
Regards,
Ed

Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#3 lore raw

You asked for bblayers.conf and local.conf on IRC, here it is:

bblayers.conf:
LCONF_VERSION = "7"
TMPDIR := "/OE/build/oe-core"
BBPATH = "${TOPDIR}:${TMPDIR}"
BBLAYERS = "${TOPDIR}/openembedded-core/meta"

local.conf:
empty

and in environment:
BB_ENV_EXTRAWHITE=MACHINE DISTRO http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS GIT_PROXY_COMMAND PSEUDO_DISABLED PSEUDO_BUILD MACHINE=qemux86
DISTRO is empty

Quoted 267 lines · Martin Jansa
Quoted 9 lines · Ed Bartosh

Hi Martin,

Sorry for the delay. I missedd your message somehow.

I can't reproduce this. Tried with latest poky and latest oe-core + bitbake. I built core-image-tiny-initramfs from scratch and bitbake -e busybox works for me just fine.

Can you provide more details about your setup, please?

Hi Ed,

I was also having troubles reproducing it in different setup as well, until I've
found out that my local.conf had this line:

IMAGE_FSTYPES_qemux86 = "vmdk"

which prevented ever using wic (USING_WIC variable was always empty).

Now with this line removed I can share the complete exception reproduced with just oe-core layer included:

$ bitbake -e core-image-tiny-initramfs | tee env.core-image-tiny-initramfs5
NOTE: Started PRServer with DBfile: /OE/build/oe-core/cache/prserv.sqlite3,
IP: 127.0.0.1, PORT: 38869, PID: 21506
Parsing recipes...ERROR: ExpansionError during parsing /OE/build/oe-core/
openembedded-core/meta/recipes-core/images/core-image-tiny-initramfs.bb
Traceback (most recent call last):
  File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 412, in
DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_
NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif [ -z "$wks" ]; then\n\t\tbbfatal
"No kickstart files from WKS_FILES were found: ${WKS_FILES}. Please set
WKS_FILE or WKS_FILES appropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic
create "$wks" --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e
"${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}\n\tmv
"$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm
-rf "$out/"\n', varname='IMAGE_CMD'):
                 try:
Quoted 1 line
               s = __expand_var_regexp__.sub(varparse.var_sub, s)

try:
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 111, in
VariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59), match='${WKS_FULL_PATH}'>):
else:
> var = self.d.getVarFlag(key, "_content")
self.references.add(key)
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 773, in
DataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content', expand=True, noweakdefault=False, parsing=False):
cachename = var + "[" + flag + "]"
> value = self.expand(value, cachename)

File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 436, in
DataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
def expand(self, s, varname = None):
> return self.expandWithRefs(s, varname).value

File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 426, in
DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/build/oe-core/ openembedded-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded- core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded- core/meta:/OE/build/oe-core/openembedded-core'.split(':'))}') or ''}", varname='WKS_FULL_PATH'):
except Exception as exc:
> raise ExpansionError(varname, s, exc) from exc

bb.data_smart.ExpansionError: Failure expanding variable WKS_FULL_PATH, expression was ${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/build/oe-core/openembedded-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded- core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded- core/meta:/OE/build/oe-core/openembedded-core'.split(':'))}') or ''} which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line
1)

Summary: There was 1 ERROR message shown, returning a non-zero exit code.

The strange part of this exception is the unexpanded ${TMPDIR} in the WKS_SEARCH_PATH value,
that lead me to following 2 lines in conf/bblayers.conf:

TMPDIR := "/OE/build/oe-core"
BBPATH = "${TOPDIR}:${TMPDIR}"

I'm not sure why it appears unexpanded in WKS_SEARCH_PATH, I see you do some magick with all these
WKS_* variables, especially in that anonymous python, but the exception is actually from
expanding IMAGE_CMD_wic () function and it goes away if I remove following line from it:

        wks="${WKS_FULL_PATH}"

And it also goes away if I use immediate expansion when setting that BBPATH in bblayers.conf.

Interesting variables from bitbake -e with immediate expansion:
# $BBPATH [4 operations]
# immediate /OE/build/oe-core/conf/bblayers.conf:6
# "${TOPDIR}:${TMPDIR}"
# postdot /OE/build/oe-core/openembedded-core/meta/conf/layer.conf:2
# ":${LAYERDIR}"
# set data_smart.py:911 [expandVarref]
# "/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-core/
openembedded-core/meta"
# set /OE/build/oe-core/openembedded-core/meta/conf/
documentation.conf:94
# [doc] "Used by BitBake to locate .bbclass and configuration files.
This variable is analogous to the PATH variable."

# pre-expansion value:
#   "/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-core/
openembedded-core/meta"
BBPATH="/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-
core/openembedded-core/meta"

# $WKS_SEARCH_PATH
# set? /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.
bbclass:10
# "${THISDIR}:${@':'.join('%s/wic' % p for p in
'${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}" WKS_SEARCH_PATH="/OE/build/oe-core/openembedded-core/meta/ recipes-core/images:/OE/build/oe-core/wic:/OE/build/oe-core/ wic:/OE/build/oe-core/openembedded-core/meta/wic:/ OE/build/oe-core/scripts/lib/wic/canned-wks:/OE/build/oe- core/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/meta/ scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded- core/scripts/lib/wic/canned-wks"

# $WKS_FULL_PATH
# set /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.
bbclass:11
# "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
WKS_FULL_PATH="/OE/build/oe-core/openembedded-core/ scripts/lib/wic/canned-wks/directdisk.wks"

And the same without immediate expansion, but with 'wks="${WKS_FULL_PATH}"' line removed so that it doesn't fail to parse:
# $BBPATH [4 operations]
# set /OE/build/oe-core/conf/bblayers.conf:6
# "${TOPDIR}:${TMPDIR}"
# postdot /OE/build/oe-core/openembedded-core/meta/conf/layer.conf:2
# ":${LAYERDIR}"
# set data_smart.py:911 [expandVarref]
# "${TOPDIR}:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta"
# set /OE/build/oe-core/openembedded-core/meta/conf/
documentation.conf:94
# [doc] "Used by BitBake to locate .bbclass and configuration files.
This variable is analogous to the PATH variable."

# pre-expansion value:
#   "${TOPDIR}:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta"
BBPATH="/OE/build/oe-core:/OE/build/oe-core-glibc:/OE/build/
oe-core/openembedded-core/meta"

# $WKS_SEARCH_PATH
# set? /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.
bbclass:10
# "${THISDIR}:${@':'.join('%s/wic' % p for p in
'${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}" WKS_SEARCH_PATH="/OE/build/oe-core/openembedded-core/meta/ recipes-core/images:/OE/build/oe-core/wic:/OE/build/oe-core- glibc/wic:/OE/build/oe-core/openembedded-core/meta/wic:/ OE/build/oe-core/scripts/lib/wic/canned-wks:/OE/build/oe- core-glibc/scripts/lib/wic/canned-wks:/OE/build/oe-core/ openembedded-core/meta/scripts/lib/wic/canned-wks:/ OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks"

# $WKS_FULL_PATH
# set /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.
bbclass:11
# "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
WKS_FULL_PATH="/OE/build/oe-core/openembedded-core/ scripts/lib/wic/canned-wks/directdisk.wks"

Let me know if sharing "bitbake -e core-image-tiny-initramfs" from these 2 tests would be useful for you.

Regards,

Quoted 9 lines · Martin Jansa
Quoted 7 lines

Running e.g. bitbake -e busybox with latest oe-core triggers this badly looking exception:

Can we add some weak assignment for people who don't care about wic or core-image-tiny-initramfs to continue using oe-core?

ERROR: ExpansionError during parsing /OE/oe-core/meta/recipes-core/

images/

Quoted 4 lines
Quoted 4 lines

core-image-tiny-initramfs.bb
Traceback (most recent call last):
File "/OE/bitbake/lib/bb/data_smart.py", line 412, in
DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_

NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif

Quoted 1 line
Quoted 1 line

[ -z "$wks" ]; then\n\t\tbbfatal "No kickstart files from WKS_FILES

were

Quoted 2 lines
Quoted 2 lines

found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic create "$wks"

--vars

Quoted 3 lines
Quoted 3 lines

"${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}\n\tmv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm -rf "$out/"\n',

varname='IMAGE_CMD'):

Quoted 2 lines
Quoted 2 lines

try:
> s = __expand_var_regexp__.sub(varparse.var_sub,

s)

Quoted 9 lines
Quoted 9 lines

try:
File "/OE/bitbake/lib/bb/data_smart.py", line 111, in
VariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59), match='${WKS_FULL_PATH}'>):
else:
> var = self.d.getVarFlag(key, "_content")
self.references.add(key)
File "/OE/bitbake/lib/bb/data_smart.py", line 773, in
DataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content',

expand=True,

Quoted 10 lines
Quoted 10 lines

noweakdefault=False, parsing=False):
cachename = var + "[" + flag + "]"
> value = self.expand(value, cachename)

File "/OE/bitbake/lib/bb/data_smart.py", line 436, in
DataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
def expand(self, s, varname = None):
> return self.expandWithRefs(s, varname).value

Quoted 4 lines
Quoted 4 lines

File "/OE/bitbake/lib/bb/data_smart.py", line 426, in
DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p

in

Quoted 1 line
Quoted 1 line

'/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.

join('%s/scripts/lib/wic/canned-wks'

Quoted 1 line
Quoted 1 line

% l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}')

or

Quoted 8 lines
Quoted 8 lines

''}", varname='WKS_FULL_PATH'):
except Exception as exc:
> raise ExpansionError(varname, s, exc) from exc

bb.data_smart.ExpansionError: Failure expanding variable WKS_FULL_PATH, expression was ${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p

in

Quoted 1 line
Quoted 1 line

'/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.

join('%s/scripts/lib/wic/canned-wks'

Quoted 1 line
Quoted 1 line

% l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}')

or

Quoted 1 line
Quoted 1 line

''} which triggered exception SyntaxError: invalid syntax

(WKS_FULL_PATH,

Quoted 5 lines
Quoted 1 line

line 1)

--
Regards,
Ed

--
Martin 'JaMa' Jansa jabber: [email protected]

#4 lore raw

PATCH 1/1 image_types_wic: fix expansion error

In reply to Martin Jansa

Variable WKS_SEARCH_PATH was not fully expanded when used directly in this expression:

    WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"

This caused expansion error: bb.data_smart.ExpansionError: Failure expanding variable WKS_FULL_PATH, .... which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)

Using d.getVar('WKS_SEARCH_PATH') instead of directly referencing WKS_SEARCH_PATH variable should fix the issue.

---
 meta/classes/image_types_wic.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/image_types_wic.bbclass +1 -1
index 43b2917..c6e92fb 100644
@@ -8,7 +8,7 @@ WICVARS ?= "\
 WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
 WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
 WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}"
-WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
+WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), d.getVar('WKS_SEARCH_PATH')) or ''}"
 
 def wks_search(files, search_path):
     for f in files:
Signature · 1 line
2.1.4
author Acked-by on v1 1/1
#5 lore raw

Re: [PATCH] image_types_wic: fix expansion error

Quoted 8 lines · Ed Bartosh

Variable WKS_SEARCH_PATH was not fully expanded when used directly in this expression:

    WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"

This caused expansion error: bb.data_smart.ExpansionError: Failure expanding variable WKS_FULL_PATH, .... which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)

Using d.getVar('WKS_SEARCH_PATH') instead of directly referencing WKS_SEARCH_PATH variable should fix the issue.

Allows me to parse oe-core again, thanks!

Quoted 26 lines
---
 meta/classes/image_types_wic.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/image_types_wic.bbclass +1 -1
index 43b2917..c6e92fb 100644
@@ -8,7 +8,7 @@ WICVARS ?= "\
 WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
 WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
 WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}"
-WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
+WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), d.getVar('WKS_SEARCH_PATH')) or ''}"
 
 def wks_search(files, search_path):
     for f in files:
Signature · 7 lines
2.1.4

-- 
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#6 lore raw

Re: [PATCH] image_types_wic: fix expansion error

In reply to Ed Bartosh
Quoted 5 lines · Ed Bartosh

-WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
+WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), d.getVar('WKS_SEARCH_PATH')) or ''}"

I think it would be safer to always use getVar instead of using the implicit expansion of WKS_FILES and the later expansion of WKS_SEARCH_PATH.

Ross

#7 lore raw

PATCH v2 1/1 image_types_wic: fix expansion error

Variable WKS_SEARCH_PATH was not fully expanded when used directly in this expression:

    WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"

This caused expansion error: bb.data_smart.ExpansionError:
Failure expanding variable WKS_FULL_PATH, .... which triggered exception
SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)

Calling d.getVar instead of directly referencing variables should fix the issue.

---
 meta/classes/image_types_wic.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/image_types_wic.bbclass +1 -1
index 43b2917..500c8c5 100644
@@ -8,7 +8,7 @@ WICVARS ?= "\
 WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
 WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
 WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}"
-WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
+WKS_FULL_PATH = "${@wks_search(d.getVar('WKS_FILES').split(), d.getVar('WKS_SEARCH_PATH')) or ''}"
 
 def wks_search(files, search_path):
     for f in files:
Signature · 1 line
2.1.4