uboot-sign: Refactor do_deploy prefunc to do_deploy_prepend
PATCH 1/2 uboot-sign: Refactor do_deploy prefunc to do_deploy_prepend
bitbake removes cleandirs once per prefunc and then again for the actual task. By moving the concat_dtb step here from prefunc to main task we can add
do_deploy[cleandirs] = "${DEPLOYDIR}"to deploy.bbclass without losing the files produced by concat_dtb.
It looks like using do_deploy(_append) was the original goal anyways.
I tested this with poky, putting the following in local.conf:
UBOOT_SIGN_ENABLE = "1"
KERNEL_CLASSES = " kernel-fitimage "
KERNEL_IMAGETYPE = "fitImage"
and then doing
bitbake core-image-minimal
bitbake u-boot
It builds successfully, the kernel and uboot recipes' temp/run.do_deploy scripts look good (kernel's do_deploy is not broken, and uboot's do_deploy still calls concat_dtb).
--- meta/classes/uboot-sign.bbclass | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
meta/classes/uboot-sign.bbclass +6 -5
@@ -117,15 +117,16 @@ do_install_append() { fi } +do_deploy_prepend_pn-${UBOOT_PN}() { + if [ "${UBOOT_SIGN_ENABLE}" = "1" -a -n "${UBOOT_DTB_BINARY}" ]; then + concat_dtb + fi +} + python () { if d.getVar('UBOOT_SIGN_ENABLE') == '1' and d.getVar('PN') == d.getVar('UBOOT_PN') and d.getVar('UBOOT_DTB_BINARY'): kernel_pn = d.getVar('PREFERRED_PROVIDER_virtual/kernel') # Make "bitbake u-boot -cdeploy" deploys the signed u-boot.dtb d.appendVarFlag('do_deploy', 'depends', ' %s:do_deploy' % kernel_pn) - - # kernerl's do_deploy is a litle special, so we can't use - # do_deploy_append, otherwise it would override - # kernel_do_deploy. - d.appendVarFlag('do_deploy', 'prefuncs', ' concat_dtb') }
Signature · 1 line
2.17.1
PATCH 2/2 deploy.bbclass: Clean DEPLOYDIR before do_deploy
It seems convenient for do_deploy to clean up ${DEPLOYDIR} (its output directory) before running, just like do_install cleans up ${D} before running. This way we can be sure that a recipe's do_deploy output is not accidentally contaminated by previously existing files in DEPLOYDIR in case of incremental builds.
All recipes using deploy.bbclass (grep -r 'inherit .*deploy') in poky, meta-openembedded and meta-freescale look to me like they either benefit from this or are at least not affected negatively by it. The only exception I've noticed was uboot-sign.bbclass, which was however fixed by the previous patch.
I tested this by running "bitbake virtual/kernel u-boot":
1. on patched, but clean poky source tree
2. after adding "touch ${DEPLOYDIR}/foo-${PN}" to linux-yocto's/u-boot's do_deploy, to manually trigger an incremental build of do_deploy only.
3. after removing the modifications again.
The foo-${PN} files existed in tmp/deploy/images/qemux86-64 after step 2, and not anymore after step 3, as expected. Thus the cleandirs still works.
--- meta/classes/deploy.bbclass | 1 + meta/classes/kernel.bbclass | 2 -- meta/recipes-core/meta/signing-keys.bb | 2 -- meta/recipes-core/ovmf/ovmf_git.bb | 1 - 4 files changed, 1 insertion(+), 5 deletions(-)
meta/classes/deploy.bbclass +1 -0
@@ -8,4 +8,5 @@ python do_deploy_setscene () { } addtask do_deploy_setscene do_deploy[dirs] = "${DEPLOYDIR} ${B}" +do_deploy[cleandirs] = "${DEPLOYDIR}" do_deploy[stamp-extra-info] = "${MACHINE_ARCH}"
meta/classes/kernel.bbclass +0 -2
@@ -706,8 +706,6 @@ kernel_do_deploy() { done fi } -do_deploy[cleandirs] = "${DEPLOYDIR}" -do_deploy[dirs] = "${DEPLOYDIR} ${B}" do_deploy[prefuncs] += "package_get_auto_pr" addtask deploy after do_populate_sysroot do_packagedata
meta/recipes-core/meta/signing-keys.bb +0 -2
@@ -67,8 +67,6 @@ do_deploy () { fi } do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_RPM}" -# cleandirs should possibly be in deploy.bbclass but we need it -do_deploy[cleandirs] = "${DEPLOYDIR}" # clear stamp-extra-info since MACHINE_ARCH is normally put there by # deploy.bbclass do_deploy[stamp-extra-info] = ""
meta/recipes-core/ovmf/ovmf_git.bb +0 -1
@@ -223,7 +223,6 @@ do_deploy[depends] += "${DEPLOYDEP}" do_deploy() { } -do_deploy[cleandirs] = "${DEPLOYDIR}" do_deploy_class-target() { # For use with "runqemu ovmf". for i in \
Signature · 1 line
2.17.1
Re: [PATCH 2/2] deploy.bbclass: Clean DEPLOYDIR before do_deploy
Ping - any chance of getting this merged? (refering to
https://patchwork.openembedded.org/series/19533/)
I think it would still be useful, especially for meta-freescale, which uses do_deploy() in many recipes.
If not, then I can update the meta-freescale pull request that triggered this patch (https://github.com/Freescale/meta-freescale/pull/139).
Thanks.
Quoted 2 lines · Daniel Klauer
bitbake removes cleandirs once per prefunc and then again for the actual task.
That commit message isn't correct.
It removes cleandirs for the prefunc at the start of the prefunc and cleandirs for the main function at the start of that main function.
The code is in lib/bb/build.py:
for func in (prefuncs or '').split():
exec_func(func, localdata)
exec_func(task, localdata)
for func in (postfuncs or '').split():
exec_func(func, localdata)
with the cleandirs happening in exec_func() for each function.
This does mean the cleandirs happens after the prefunc is called but not as described above.
Quoted 4 lines
By moving the
concat_dtbstep here from prefunc to main task we can
adddo_deploy[cleandirs] = "${DEPLOYDIR}"to deploy.bbclass without losing the files produced by
concat_dtb.
I think you also need to state here that this is only expected to be run for the uboot recipe and not the kernel.
The patch itself is probably ok but I'd ask the commit message be corrected please.
Cheers,
Richard
Quoted 44 lines
It looks like using
do_deploy(_append)was the original goal anyways.I tested this with poky, putting the following in local.conf:
UBOOT_SIGN_ENABLE= "1"
KERNEL_CLASSES= " kernel-fitimage "
KERNEL_IMAGETYPE= "fitImage"
and then doing
bitbake core-image-minimal
bitbake u-boot
It builds successfully, the kernel and uboot recipes' temp/run.do_deploy scripts look good (kernel'sdo_deployis not broken, and uboot'sdo_deploystill callsconcat_dtb).Signed-off-by Daniel Klauer <[email protected]>--- meta/classes/uboot-sign.bbclass | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)meta/classes/uboot-sign.bbclass +6
-5@@ -117,15 +117,16 @@ do_install_append() { fi } +do_deploy_prepend_pn-${UBOOT_PN}() { + if [ "${UBOOT_SIGN_ENABLE}" = "1" -a -n "${UBOOT_DTB_BINARY}" ]; then + concat_dtb + fi +} + python () { if d.getVar('UBOOT_SIGN_ENABLE') == '1' and d.getVar('PN') == d.getVar('UBOOT_PN') and d.getVar('UBOOT_DTB_BINARY'): kernel_pn = d.getVar('PREFERRED_PROVIDER_virtual/kernel') # Make "bitbake u-boot -cdeploy" deploys the signed u-boot.dtb d.appendVarFlag('do_deploy', 'depends', ' %s:do_deploy' % kernel_pn) - - # kernerl's do_deploy is a litle special, so we can't use - # do_deploy_append, otherwise it would override - # kernel_do_deploy. - d.appendVarFlag('do_deploy', 'prefuncs', ' concat_dtb')
Quoted 4 lines
That commit message isn't correct.
It removes cleandirs for the prefunc at the start of the prefunc and cleandirs for the main function at the start of that main function.
Oh ok, so the prefuncs/postfuncs can have their own [cleandirs], that's interesting.
How about the following (trying to find a good way to explain it):
-----
When inherited by the u-boot recipe (UBOOT_PN), uboot-sign.bbclass adds a concat_dtb step, which places additional files into ${DEPLOYDIR} before do_deploy.
The use of prefuncs for this prevents us from adding
do_deploy[cleandirs] = "${DEPLOYDIR}"
because that would remove the files produced by the prefunc. (Each of prefuncs/task/postfuncs functions has their own cleandirs, and each function's cleandirs are removed at the start of that function.)
Thus, it seems good to make concat_dtb a part of do_deploy, such that it runs after removal of do_deploy's cleandirs. As before, care is taken to not interfere with the kernel's do_deploy definition, since concat_dtb was only needed for u-boot.
-----
Quoted 9 lines · Daniel Klauer
Quoted 4 lines
That commit message isn't correct.
It removes cleandirs for the prefunc at the start of the prefunc and cleandirs for the main function at the start of that main function.
Oh ok, so the prefuncs/postfuncs can have their own [cleandirs], that's interesting.
How about the following (trying to find a good way to explain it):
Yes, that sounds fine. The bit in brackets probably isn't needed.
Cheers,
Richard
Quoted 18 lines
-----
When inherited by the u-boot recipe (UBOOT_PN), uboot-sign.bbclass adds aconcat_dtbstep, which places additional files into ${DEPLOYDIR} beforedo_deploy.The use of prefuncs for this prevents us from adding
do_deploy[cleandirs] = "${DEPLOYDIR}"
because that would remove the files produced by the prefunc. (Each of prefuncs/task/postfuncs functions has their own cleandirs, and each function's cleandirs are removed at the start of that function.)Thus, it seems good to make
concat_dtba part ofdo_deploy, such that it runs after removal of do_deploy's cleandirs. As before, care is taken to not interfere with the kernel'sdo_deploydefinition, sinceconcat_dtbwas only needed for u-boot.
-----
PATCH v2 1/2 uboot-sign: Refactor do_deploy prefunc to do_deploy_prepend
When inherited by the u-boot recipe (UBOOT_PN), uboot-sign.bbclass adds a concat_dtb step, which places additional files into ${DEPLOYDIR} before do_deploy.
The use of prefuncs for this prevents us from adding
do_deploy[cleandirs] = "${DEPLOYDIR}"
because that would remove the files produced by the prefunc.
Thus, it seems good to make concat_dtb a part of do_deploy, such that it runs after removal of do_deploy's cleandirs. As before, care is taken to not interfere with the kernel's do_deploy definition, since concat_dtb was only needed for u-boot.
I tested this with poky, putting the following in local.conf:
UBOOT_SIGN_ENABLE = "1"
KERNEL_CLASSES = " kernel-fitimage " KERNEL_IMAGETYPE = "fitImage"
and then doing
bitbake core-image-minimal
bitbake u-boot
It built successfully and concat_dtb was still called.
--- meta/classes/uboot-sign.bbclass | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
meta/classes/uboot-sign.bbclass +6 -5
@@ -117,15 +117,16 @@ do_install_append() { fi } +do_deploy_prepend_pn-${UBOOT_PN}() { + if [ "${UBOOT_SIGN_ENABLE}" = "1" -a -n "${UBOOT_DTB_BINARY}" ]; then + concat_dtb + fi +} + python () { if d.getVar('UBOOT_SIGN_ENABLE') == '1' and d.getVar('PN') == d.getVar('UBOOT_PN') and d.getVar('UBOOT_DTB_BINARY'): kernel_pn = d.getVar('PREFERRED_PROVIDER_virtual/kernel') # Make "bitbake u-boot -cdeploy" deploys the signed u-boot.dtb d.appendVarFlag('do_deploy', 'depends', ' %s:do_deploy' % kernel_pn) - - # kernerl's do_deploy is a litle special, so we can't use - # do_deploy_append, otherwise it would override - # kernel_do_deploy. - d.appendVarFlag('do_deploy', 'prefuncs', ' concat_dtb') }
Signature · 1 line
2.17.1