core-image-tiny-initramfs failing to parse because of WIC (WKS_FULL_PATH, )
Running e.g. bitbake -e busybox with latest oe-core triggers this badly looking exception:
Can we add some weak assignment for people who don't care about wic or core-image-tiny-initramfs to continue using oe-core?
ERROR: ExpansionError during parsing /OE/oe-core/meta/recipes-core/images/ core-image-tiny-initramfs.bb
Traceback (most recent call last):
File "/OE/bitbake/lib/bb/data_smart.py", line 412, in
DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif [ -z "$wks" ]; then\n\t\tbbfatal "No kickstart files from WKS_FILES were
found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic create "$wks" --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}\n\tmv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm -rf "$out/"\n', varname='IMAGE_CMD'):
try:
> s = __expand_var_regexp__.sub(varparse.var_sub, s)
try:
File "/OE/bitbake/lib/bb/data_smart.py", line 111, inVariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59), match='${WKS_FULL_PATH}'>):
else:
> var = self.d.getVarFlag(key, "_content")
self.references.add(key)
File "/OE/bitbake/lib/bb/data_smart.py", line 773, inDataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content', expand=True, noweakdefault=False, parsing=False):
cachename = var + "[" + flag + "]"
> value = self.expand(value, cachename)
File "/OE/bitbake/lib/bb/data_smart.py", line 436, inDataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
def expand(self, s, varname = None):
> return self.expandWithRefs(s, varname).value
File "/OE/bitbake/lib/bb/data_smart.py", line 426, in
DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or ''}", varname='WKS_FULL_PATH'):
except Exception as exc:
> raise ExpansionError(varname, s, exc) from exc
bb.data_smart.ExpansionError: Failure expanding variable WKS_FULL_PATH, expression was ${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or ''} which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)
Hi Martin,
Sorry for the delay. I missedd your message somehow.
I can't reproduce this. Tried with latest poky and latest oe-core + bitbake. I built core-image-tiny-initramfs from scratch and bitbake -e busybox works for me just fine.
Can you provide more details about your setup, please?
Quoted 48 lines · Martin Jansa
Running e.g. bitbake -e busybox with latest oe-core triggers this badly looking exception:
Can we add some weak assignment for people who don't care about wic or core-image-tiny-initramfs to continue using oe-core?
ERROR: ExpansionError during parsing /OE/oe-core/meta/recipes-core/images/ core-image-tiny-initramfs.bb
Traceback (most recent call last):
File "/OE/bitbake/lib/bb/data_smart.py", line 412, in
DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif [ -z "$wks" ]; then\n\t\tbbfatal "No kickstart files fromWKS_FILESwere
found: ${WKS_FILES}. Please setWKS_FILEorWKS_FILESappropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic create "$wks" --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}\n\tmv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm -rf "$out/"\n', varname='IMAGE_CMD'):
try:
> s =__expand_var_regexp__.sub(varparse.var_sub, s)
try:
File "/OE/bitbake/lib/bb/data_smart.py", line 111, inVariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59), match='${WKS_FULL_PATH}'>):
else:
> var =self.d.getVarFlag(key, "_content")
self.references.add(key)
File "/OE/bitbake/lib/bb/data_smart.py", line 773, inDataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content', expand=True, noweakdefault=False, parsing=False):
cachename = var + "[" + flag + "]"
> value =self.expand(value, cachename)File "/OE/bitbake/lib/bb/data_smart.py", line 436, in
DataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
defexpand(self, s, varname = None):
> returnself.expandWithRefs(s, varname).valueFile "/OE/bitbake/lib/bb/data_smart.py", line 426, in
DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or ''}", varname='WKS_FULL_PATH'):
except Exception as exc:
> raiseExpansionError(varname, s, exc)from exc
Quoted 8 lines
bb.data_smart.ExpansionError: Failure expanding variable
WKS_FULL_PATH, expression was ${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or ''} which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)
--
Regards,
Ed
Quoted 9 lines · Ed Bartosh
Hi Martin,
Sorry for the delay. I missedd your message somehow.
I can't reproduce this. Tried with latest poky and latest oe-core + bitbake. I built core-image-tiny-initramfs from scratch and bitbake -e busybox works for me just fine.
Can you provide more details about your setup, please?
Hi Ed,
I was also having troubles reproducing it in different setup as well, until I've found out that my local.conf had this line:
IMAGE_FSTYPES_qemux86 = "vmdk"
which prevented ever using wic (USING_WIC variable was always empty).
Now with this line removed I can share the complete exception reproduced with just oe-core layer included:
$ bitbake -e core-image-tiny-initramfs | tee env.core-image-tiny-initramfs5
NOTE: Started PRServer with DBfile: /OE/build/oe-core/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 38869, PID: 21506
Parsing recipes...ERROR: ExpansionError during parsing /OE/build/oe-core/openembedded-core/meta/recipes-core/images/core-image-tiny-initramfs.bb
Traceback (most recent call last):
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 412, in DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif [ -z "$wks" ]; then\n\t\tbbfatal "No kickstart files from WKS_FILES were found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic create "$wks" --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}\n\tmv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm -rf "$out/"\n', varname='IMAGE_CMD'):
try:Quoted 1 line
s = __expand_var_regexp__.sub(varparse.var_sub, s)
try:
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 111, in VariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59), match='${WKS_FULL_PATH}'>):
else:
> var = self.d.getVarFlag(key, "_content")
self.references.add(key)
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 773, in DataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content', expand=True, noweakdefault=False, parsing=False):
cachename = var + "[" + flag + "]"
> value = self.expand(value, cachename)
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 436, in DataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
def expand(self, s, varname = None):
> return self.expandWithRefs(s, varname).value
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 426, in DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/build/oe-core/openembedded-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta:/OE/build/oe-core/openembedded-core'.split(':'))}') or ''}", varname='WKS_FULL_PATH'):
except Exception as exc:
> raise ExpansionError(varname, s, exc) from exc
bb.data_smart.ExpansionError: Failure expanding variable WKS_FULL_PATH, expression was ${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/build/oe-core/openembedded-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta:/OE/build/oe-core/openembedded-core'.split(':'))}') or ''} which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
The strange part of this exception is the unexpanded ${TMPDIR} in the WKS_SEARCH_PATH value, that lead me to following 2 lines in conf/bblayers.conf:
TMPDIR := "/OE/build/oe-core"
BBPATH = "${TOPDIR}:${TMPDIR}"
I'm not sure why it appears unexpanded in WKS_SEARCH_PATH, I see you do some magick with all these WKS_* variables, especially in that anonymous python, but the exception is actually from expanding IMAGE_CMD_wic () function and it goes away if I remove following line from it:
wks="${WKS_FULL_PATH}"And it also goes away if I use immediate expansion when setting that BBPATH in bblayers.conf.
Interesting variables from bitbake -e with immediate expansion:
# $BBPATH [4 operations]
# immediate /OE/build/oe-core/conf/bblayers.conf:6
# "${TOPDIR}:${TMPDIR}"
# postdot /OE/build/oe-core/openembedded-core/meta/conf/layer.conf:2
# ":${LAYERDIR}"
# set data_smart.py:911 [expandVarref]
# "/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-core/openembedded-core/meta"
# set /OE/build/oe-core/openembedded-core/meta/conf/documentation.conf:94
# [doc] "Used by BitBake to locate .bbclass and configuration files. This variable is analogous to the PATH variable."
# pre-expansion value: # "/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-core/openembedded-core/meta" BBPATH="/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-core/openembedded-core/meta"
# $WKS_SEARCH_PATH
# set? /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.bbclass:10
# "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}"
WKS_SEARCH_PATH="/OE/build/oe-core/openembedded-core/meta/recipes-core/images:/OE/build/oe-core/wic:/OE/build/oe-core/wic:/OE/build/oe-core/openembedded-core/meta/wic:/OE/build/oe-core/scripts/lib/wic/canned-wks:/OE/build/oe-core/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/meta/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks"
# $WKS_FULL_PATH
# set /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.bbclass:11
# "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
WKS_FULL_PATH="/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks/directdisk.wks"
And the same without immediate expansion, but with 'wks="${WKS_FULL_PATH}"' line removed so that it doesn't fail to parse:
# $BBPATH [4 operations]
# set /OE/build/oe-core/conf/bblayers.conf:6
# "${TOPDIR}:${TMPDIR}"
# postdot /OE/build/oe-core/openembedded-core/meta/conf/layer.conf:2
# ":${LAYERDIR}"
# set data_smart.py:911 [expandVarref]
# "${TOPDIR}:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta"
# set /OE/build/oe-core/openembedded-core/meta/conf/documentation.conf:94
# [doc] "Used by BitBake to locate .bbclass and configuration files. This variable is analogous to the PATH variable."
# pre-expansion value:
# "${TOPDIR}:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta"
BBPATH="/OE/build/oe-core:/OE/build/oe-core-glibc:/OE/build/oe-core/openembedded-core/meta"# $WKS_SEARCH_PATH
# set? /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.bbclass:10
# "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}"
WKS_SEARCH_PATH="/OE/build/oe-core/openembedded-core/meta/recipes-core/images:/OE/build/oe-core/wic:/OE/build/oe-core-glibc/wic:/OE/build/oe-core/openembedded-core/meta/wic:/OE/build/oe-core/scripts/lib/wic/canned-wks:/OE/build/oe-core-glibc/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/meta/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks"
# $WKS_FULL_PATH
# set /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.bbclass:11
# "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
WKS_FULL_PATH="/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks/directdisk.wks"
Let me know if sharing "bitbake -e core-image-tiny-initramfs" from these 2 tests would be useful for you.
Regards,
Quoted 62 lines · Martin Jansa
Quoted 48 lines
Running e.g. bitbake -e busybox with latest oe-core triggers this badly looking exception:
Can we add some weak assignment for people who don't care about wic or core-image-tiny-initramfs to continue using oe-core?
ERROR: ExpansionError during parsing /OE/oe-core/meta/recipes-core/images/ core-image-tiny-initramfs.bb
Traceback (most recent call last):
File "/OE/bitbake/lib/bb/data_smart.py", line 412, in
DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif [ -z "$wks" ]; then\n\t\tbbfatal "No kickstart files fromWKS_FILESwere
found: ${WKS_FILES}. Please setWKS_FILEorWKS_FILESappropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic create "$wks" --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}\n\tmv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm -rf "$out/"\n', varname='IMAGE_CMD'):
try:
> s =__expand_var_regexp__.sub(varparse.var_sub, s)
try:
File "/OE/bitbake/lib/bb/data_smart.py", line 111, inVariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59), match='${WKS_FULL_PATH}'>):
else:
> var =self.d.getVarFlag(key, "_content")
self.references.add(key)
File "/OE/bitbake/lib/bb/data_smart.py", line 773, inDataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content', expand=True, noweakdefault=False, parsing=False):
cachename = var + "[" + flag + "]"
> value =self.expand(value, cachename)File "/OE/bitbake/lib/bb/data_smart.py", line 436, in
DataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
defexpand(self, s, varname = None):
> returnself.expandWithRefs(s, varname).valueFile "/OE/bitbake/lib/bb/data_smart.py", line 426, in
DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or ''}", varname='WKS_FULL_PATH'):
except Exception as exc:
> raiseExpansionError(varname, s, exc)from excQuoted 8 lines
bb.data_smart.ExpansionError: Failure expanding variable
WKS_FULL_PATH, expression was ${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}') or ''} which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)--
Regards,
Ed
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]
You asked for bblayers.conf and local.conf on IRC, here it is:
bblayers.conf:LCONF_VERSION = "7"
TMPDIR := "/OE/build/oe-core"
BBPATH = "${TOPDIR}:${TMPDIR}"
BBLAYERS = "${TOPDIR}/openembedded-core/meta"
local.conf:
empty
and in environment:
BB_ENV_EXTRAWHITE=MACHINE DISTRO http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS GIT_PROXY_COMMAND PSEUDO_DISABLED PSEUDO_BUILD MACHINE=qemux86
DISTRO is empty
Quoted 267 lines · Martin Jansa
Quoted 9 lines · Ed Bartosh
Hi Martin,
Sorry for the delay. I missedd your message somehow.
I can't reproduce this. Tried with latest poky and latest oe-core + bitbake. I built core-image-tiny-initramfs from scratch and bitbake -e busybox works for me just fine.
Can you provide more details about your setup, please?
Hi Ed,
I was also having troubles reproducing it in different setup as well, until I've
found out that my local.conf had this line:
IMAGE_FSTYPES_qemux86= "vmdk"which prevented ever using wic (
USING_WICvariable was always empty).Now with this line removed I can share the complete exception reproduced with just oe-core layer included:
$ bitbake -e core-image-tiny-initramfs | tee env.core-image-tiny-initramfs5 NOTE: Started PRServer with DBfile: /OE/build/oe-core/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 38869, PID: 21506 Parsing recipes...ERROR: ExpansionError during parsing /OE/build/oe-core/ openembedded-core/meta/recipes-core/images/core-image-tiny-initramfs.bb Traceback (most recent call last): File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 412, in DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_ NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif [ -z "$wks" ]; then\n\t\tbbfatal "No kickstart files from WKS_FILES were found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic create "$wks" --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}\n\tmv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm -rf "$out/"\n', varname='IMAGE_CMD'): try:Quoted 1 line
s = __expand_var_regexp__.sub(varparse.var_sub, s)try:
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 111, inVariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59), match='${WKS_FULL_PATH}'>):
else:
> var =self.d.getVarFlag(key, "_content")
self.references.add(key)
File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 773, inDataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content', expand=True, noweakdefault=False, parsing=False):
cachename = var + "[" + flag + "]"
> value =self.expand(value, cachename)File "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 436, in
DataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
defexpand(self, s, varname = None):
> returnself.expandWithRefs(s, varname).valueFile "/OE/build/oe-core/bitbake/lib/bb/data_smart.py", line 426, in
DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/build/oe-core/ openembedded-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded- core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded- core/meta:/OE/build/oe-core/openembedded-core'.split(':'))}') or ''}", varname='WKS_FULL_PATH'):
except Exception as exc:
> raiseExpansionError(varname, s, exc)from excbb.data_smart.ExpansionError: Failure expanding variable
WKS_FULL_PATH, expression was ${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/build/oe-core/openembedded-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for p in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded- core/meta'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '/OE/build/oe-core:${TMPDIR}:/OE/build/oe-core/openembedded- core/meta:/OE/build/oe-core/openembedded-core'.split(':'))}') or ''} which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line
1)Summary: There was 1 ERROR message shown, returning a non-zero exit code.
The strange part of this exception is the unexpanded ${TMPDIR} in the
WKS_SEARCH_PATHvalue,
that lead me to following 2 lines inconf/bblayers.conf:TMPDIR := "/OE/build/oe-core"
BBPATH = "${TOPDIR}:${TMPDIR}"I'm not sure why it appears unexpanded in
WKS_SEARCH_PATH, I see you do some magick with all these
WKS_* variables, especially in that anonymous python, but the exception is actually from
expandingIMAGE_CMD_wic() function and it goes away if I remove following line from it:wks="${WKS_FULL_PATH}"And it also goes away if I use immediate expansion when setting that BBPATH in bblayers.conf.
Interesting variables from bitbake -e with immediate expansion:
# $BBPATH [4 operations]
# immediate /OE/build/oe-core/conf/bblayers.conf:6
# "${TOPDIR}:${TMPDIR}"
# postdot /OE/build/oe-core/openembedded-core/meta/conf/layer.conf:2
# ":${LAYERDIR}"
# set data_smart.py:911 [expandVarref]
# "/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-core/
openembedded-core/meta"
# set /OE/build/oe-core/openembedded-core/meta/conf/
documentation.conf:94
# [doc] "Used by BitBake to locate .bbclass and configuration files.
This variable is analogous to the PATH variable."# pre-expansion value: # "/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe-core/ openembedded-core/meta" BBPATH="/OE/build/oe-core:/OE/build/oe-core:/OE/build/oe- core/openembedded-core/meta"# $WKS_SEARCH_PATH
# set? /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.
bbclass:10
# "${THISDIR}:${@':'.join('%s/wic' % p for p in
'${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}" WKS_SEARCH_PATH="/OE/build/oe-core/openembedded-core/meta/ recipes-core/images:/OE/build/oe-core/wic:/OE/build/oe-core/ wic:/OE/build/oe-core/openembedded-core/meta/wic:/ OE/build/oe-core/scripts/lib/wic/canned-wks:/OE/build/oe- core/scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded-core/meta/ scripts/lib/wic/canned-wks:/OE/build/oe-core/openembedded- core/scripts/lib/wic/canned-wks"# $WKS_FULL_PATH
# set /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.
bbclass:11
# "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
WKS_FULL_PATH="/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks/directdisk.wks"And the same without immediate expansion, but with 'wks="${WKS_FULL_PATH}"' line removed so that it doesn't fail to parse:
# $BBPATH [4 operations]
# set /OE/build/oe-core/conf/bblayers.conf:6
# "${TOPDIR}:${TMPDIR}"
# postdot /OE/build/oe-core/openembedded-core/meta/conf/layer.conf:2
# ":${LAYERDIR}"
# set data_smart.py:911 [expandVarref]
# "${TOPDIR}:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta"
# set /OE/build/oe-core/openembedded-core/meta/conf/
documentation.conf:94
# [doc] "Used by BitBake to locate .bbclass and configuration files.
This variable is analogous to the PATH variable."# pre-expansion value: # "${TOPDIR}:${TMPDIR}:/OE/build/oe-core/openembedded-core/meta" BBPATH="/OE/build/oe-core:/OE/build/oe-core-glibc:/OE/build/ oe-core/openembedded-core/meta"# $WKS_SEARCH_PATH
# set? /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.
bbclass:10
# "${THISDIR}:${@':'.join('%s/wic' % p for p in
'${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}" WKS_SEARCH_PATH="/OE/build/oe-core/openembedded-core/meta/ recipes-core/images:/OE/build/oe-core/wic:/OE/build/oe-core- glibc/wic:/OE/build/oe-core/openembedded-core/meta/wic:/ OE/build/oe-core/scripts/lib/wic/canned-wks:/OE/build/oe- core-glibc/scripts/lib/wic/canned-wks:/OE/build/oe-core/ openembedded-core/meta/scripts/lib/wic/canned-wks:/ OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks"# $WKS_FULL_PATH
# set /OE/build/oe-core/openembedded-core/meta/classes/image_types_wic.
bbclass:11
# "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
WKS_FULL_PATH="/OE/build/oe-core/openembedded-core/scripts/lib/wic/canned-wks/directdisk.wks"Let me know if sharing "bitbake -e core-image-tiny-initramfs" from these 2 tests would be useful for you.
Regards,
Quoted 9 lines · Martin Jansa
Quoted 7 lines
Running e.g. bitbake -e busybox with latest oe-core triggers this badly looking exception:
Can we add some weak assignment for people who don't care about wic or core-image-tiny-initramfs to continue using oe-core?
ERROR: ExpansionError during parsing /OE/oe-core/meta/recipes-core/
images/
Quoted 4 lines
Quoted 4 lines
core-image-tiny-initramfs.bb
Traceback (most recent call last):
File "/OE/bitbake/lib/bb/data_smart.py", line 412, in
DataSmart.expandWithRefs(s='\tout="${IMGDEPLOYDIR}/${IMAGE_NAME}"\n\twks="${WKS_FULL_PATH}"\n\tif
Quoted 1 line
Quoted 1 line
[ -z "$wks" ]; then\n\t\tbbfatal "No kickstart files from
WKS_FILESwere
Quoted 2 lines
Quoted 2 lines
found: ${WKS_FILES}. Please set
WKS_FILEorWKS_FILESappropriately."\n\tfi\n\n\tBUILDDIR="${TOPDIR}" wic create "$wks"--vars
Quoted 3 lines
Quoted 3 lines
"${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}\n\tmv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"\n\trm -rf "$out/"\n',
varname='IMAGE_CMD'):
Quoted 2 lines
Quoted 2 lines
try:
> s = __expand_var_regexp__.sub(varparse.var_sub,s)
Quoted 9 lines
Quoted 9 lines
try:
File "/OE/bitbake/lib/bb/data_smart.py", line 111, inVariableParse.var_sub(match=<_sre.SRE_Match object; span=(43, 59), match='${WKS_FULL_PATH}'>):
else:
> var =self.d.getVarFlag(key, "_content")
self.references.add(key)
File "/OE/bitbake/lib/bb/data_smart.py", line 773, in
DataSmart.getVarFlag(var='WKS_FULL_PATH', flag='_content',expand=True,
Quoted 10 lines
Quoted 10 lines
noweakdefault=False, parsing=False):
cachename = var + "[" + flag + "]"
> value =self.expand(value, cachename)File "/OE/bitbake/lib/bb/data_smart.py", line 436, in
DataSmart.expand(s="${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}", varname='WKS_FULL_PATH'):
defexpand(self, s, varname = None):
> returnself.expandWithRefs(s, varname).valueQuoted 4 lines
Quoted 4 lines
File "/OE/bitbake/lib/bb/data_smart.py", line 426, in
DataSmart.expandWithRefs(s="${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for pin
Quoted 1 line
Quoted 1 line
'/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.
join('%s/scripts/lib/wic/canned-wks'
Quoted 1 line
Quoted 1 line
% l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}')
or
Quoted 8 lines
Quoted 8 lines
''}", varname='WKS_FULL_PATH'):
except Exception as exc:
> raiseExpansionError(varname, s, exc)from excbb.data_smart.ExpansionError: Failure expanding variable
WKS_FULL_PATH, expression was ${@wks_search('directdisk.wks core-image-tiny-initramfs.wks'.split(), '/OE/oe-core/meta/recipes-core/images:${@':'.join('%s/wic' % p for pin
Quoted 1 line
Quoted 1 line
'/OE:${TMPDIR}:/OE/oe-core/meta'.split(':'))}:${@':'.
join('%s/scripts/lib/wic/canned-wks'
Quoted 1 line
Quoted 1 line
% l for l in '/OE:${TMPDIR}:/OE/oe-core/meta:/OE/oe-core'.split(':'))}')
or
Quoted 1 line
Quoted 1 line
''} which triggered exception SyntaxError: invalid syntax
(
WKS_FULL_PATH,Quoted 5 lines
Quoted 1 line
line 1)
--
Regards,
Ed--
Martin 'JaMa' Jansa jabber: [email protected]
PATCH 1/1 image_types_wic: fix expansion error
In reply to Martin JansaVariable WKS_SEARCH_PATH was not fully expanded when used directly in this expression:
WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"This caused expansion error: bb.data_smart.ExpansionError: Failure expanding variable WKS_FULL_PATH, .... which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)
Using d.getVar('WKS_SEARCH_PATH') instead of directly referencing WKS_SEARCH_PATH variable should fix the issue.
--- meta/classes/image_types_wic.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/image_types_wic.bbclass +1 -1
@@ -8,7 +8,7 @@ WICVARS ?= "\ WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks" WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks" WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}" -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}" +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), d.getVar('WKS_SEARCH_PATH')) or ''}" def wks_search(files, search_path): for f in files:
Signature · 1 line
2.1.4
Re: [PATCH] image_types_wic: fix expansion error
Quoted 8 lines · Ed Bartosh
Variable
WKS_SEARCH_PATHwas not fully expanded when used directly in this expression:WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"This caused expansion error: bb.data_smart.ExpansionError: Failure expanding variable
WKS_FULL_PATH, .... which triggered exception SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)Using
d.getVar('WKS_SEARCH_PATH')instead of directly referencingWKS_SEARCH_PATHvariable should fix the issue.
Allows me to parse oe-core again, thanks!
Quoted 26 lines
Signed-off-by Ed Bartosh <[email protected]>--- meta/classes/image_types_wic.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/classes/image_types_wic.bbclass +1
-1@@ -8,7 +8,7 @@ WICVARS ?= "\ WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks" WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks" WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}" -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}" +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), d.getVar('WKS_SEARCH_PATH')) or ''}" def wks_search(files, search_path): for f in files:Signature · 7 lines
2.1.4 -- _______________________________________________ Openembedded-core mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]
Re: [PATCH] image_types_wic: fix expansion error
In reply to Ed BartoshQuoted 5 lines · Ed Bartosh
-WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
+WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),d.getVar('WKS_SEARCH_PATH')) or ''}"
I think it would be safer to always use getVar instead of using the implicit expansion of WKS_FILES and the later expansion of WKS_SEARCH_PATH.
Ross
PATCH v2 1/1 image_types_wic: fix expansion error
Variable WKS_SEARCH_PATH was not fully expanded when used directly in this expression:
WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"This caused expansion error: bb.data_smart.ExpansionError:
Failure expanding variable WKS_FULL_PATH, .... which triggered exception
SyntaxError: invalid syntax (WKS_FULL_PATH, line 1)
Calling d.getVar instead of directly referencing variables should fix the issue.
--- meta/classes/image_types_wic.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/image_types_wic.bbclass +1 -1
@@ -8,7 +8,7 @@ WICVARS ?= "\ WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks" WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks" WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}" -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}" +WKS_FULL_PATH = "${@wks_search(d.getVar('WKS_FILES').split(), d.getVar('WKS_SEARCH_PATH')) or ''}" def wks_search(files, search_path): for f in files:
Signature · 1 line
2.1.4