rm_work: Improve handling for addto_recipe_sysroot
PATCH 1/5 rm_work: Improve handling for addto_recipe_sysroot
Rather than requiring each user to handle this individually, handle addto_recipe_sysroot in the core class. As well as preserving the sysroot directory, this also ensures the stamp is preserved rather than rerunning the task every time as currently happens.
--- meta/classes/rm_work.bbclass | 32 ++++++++++++++-------- .../qemu/qemu-helper-native_1.0.bb | 1 - .../systemtap/systemtap-native_git.bb | 1 - 3 files changed, 20 insertions(+), 14 deletions(-)
meta/classes/rm_work.bbclass +20 -12
@@ -35,22 +35,12 @@ do_rm_work () { fi done - cd ${WORKDIR} - for dir in * - do - # Retain only logs and other files in temp, safely ignore - # failures of removing pseudo folers on NFS2/3 server. - if [ $dir = 'pseudo' ]; then - rm -rf $dir 2> /dev/null || true - elif ! echo '${RM_WORK_EXCLUDE_ITEMS}' | grep -q -w "$dir"; then - rm -rf $dir - fi - done - # Need to add pseudo back or subsqeuent work in this workdir # might fail since setscene may not rerun to recreate it mkdir -p ${WORKDIR}/pseudo/ + excludes='${RM_WORK_EXCLUDE_ITEMS}' + # Change normal stamps into setscene stamps as they better reflect the # fact WORKDIR is now empty # Also leave noexec stamps since setscene stamps don't cover them
@@ -79,6 +69,12 @@ do_rm_work () { i=dummy break ;; + *do_addto_recipe_sysroot*) + # Preserve recipe-sysroot-native if do_addto_recipe_sysroot has been used + excludes="$excludes recipe-sysroot-native" + i=dummy + break + ;; # We remove do_package entirely, including any # sstate version since otherwise we'd need to leave 'plaindirs' around # such as 'packages' and 'packages-split' and these can be large. No end
@@ -101,6 +97,18 @@ do_rm_work () { done rm -f $i done + + cd ${WORKDIR} + for dir in * + do + # Retain only logs and other files in temp, safely ignore + # failures of removing pseudo folers on NFS2/3 server. + if [ $dir = 'pseudo' ]; then + rm -rf $dir 2> /dev/null || true + elif ! echo "$excludes" | grep -q -w "$dir"; then + rm -rf $dir + fi + done } do_rm_work_all () { :
meta/recipes-devtools/qemu/qemu-helper-native_1.0.bb +0 -1
@@ -20,6 +20,5 @@ do_install() { install tunctl ${D}${bindir}/ } -RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" DEPENDS += "qemu-native" addtask addto_recipe_sysroot after do_populate_sysroot before do_build
meta/recipes-kernel/systemtap/systemtap-native_git.bb +0 -1
@@ -3,5 +3,4 @@ require systemtap_git.bb inherit native -RM_WORK_EXCLUDE_ITEMS += "recipe-sysroot-native" addtask addto_recipe_sysroot after do_populate_sysroot before do_build
Signature · 1 line
2.7.4
PATCH 2/5 sstate: Improve SSTATE_SKIP_CREATION
Currently, dummy archives are created for skipped sstate tasks. We relied on these never being installed (the setscene task is missing) however this leads to odd behaviour as for example the setscene stamps are never looked at.
Rather than trying to continue with the two separate behaviours, really skip package creation. We do need the file manipulation steps to install files under sstate control from a manifest perspective so we only skip at the final creation step.
--- meta/classes/sstate.bbclass | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)
meta/classes/sstate.bbclass +4 -6
@@ -346,8 +346,6 @@ def sstate_installpkgdir(ss, d): oe.path.remove(dir) for state in ss['dirs']: - if d.getVar('SSTATE_SKIP_CREATION') == '1': - continue prepdir(state[1]) os.rename(sstateinst + state[0], state[1]) sstate_install(ss, d)
@@ -596,8 +594,6 @@ def sstate_package(ss, d): for state in ss['dirs']: if not os.path.exists(state[1]): continue - if d.getVar('SSTATE_SKIP_CREATION') == '1': - continue srcbase = state[0].rstrip("/").rsplit('/', 1)[0] # Find and error for absolute symlinks. We could attempt to relocate but its not # clear where the symlink is relative to in this context. We could add that markup
@@ -625,6 +621,10 @@ def sstate_package(ss, d): d.setVar('SSTATE_BUILDDIR', sstatebuild) d.setVar('SSTATE_PKG', sstatepkg) + d.setVar('SSTATE_INSTDIR', sstatebuild) + + if d.getVar('SSTATE_SKIP_CREATION') == '1': + return for f in (d.getVar('SSTATECREATEFUNCS') or '').split() + \ ['sstate_create_package', 'sstate_sign_package'] + \
@@ -634,8 +634,6 @@ def sstate_package(ss, d): bb.siggen.dump_this_task(sstatepkg + ".siginfo", d) - d.setVar('SSTATE_INSTDIR', sstatebuild) - return def pstaging_fetch(sstatefetch, sstatepkg, d):
Signature · 1 line
2.7.4
PATCH 3/5 image: Drop before do_build from do_rootfs and do_image
In reply to Richard Purdiedo_image_complete has the do_build dependency, these other two tasks don't need it. If this is present, it complicates the dependency tree and convinces rm_work that the tasks need to rerun even if they already did since the stamps are removed.
Drop the unneeded dependency to help resolve this issue.
--- meta/classes/image.bbclass | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/classes/image.bbclass +2 -2
@@ -261,7 +261,7 @@ fakeroot python do_rootfs () { do_rootfs[dirs] = "${TOPDIR}" do_rootfs[cleandirs] += "${S} ${IMGDEPLOYDIR}" do_rootfs[umask] = "022" -addtask rootfs before do_build after do_prepare_recipe_sysroot +addtask rootfs after do_prepare_recipe_sysroot fakeroot python do_image () { from oe.utils import execute_pre_post_process
@@ -272,7 +272,7 @@ fakeroot python do_image () { } do_image[dirs] = "${TOPDIR}" do_image[umask] = "022" -addtask do_image after do_rootfs before do_build +addtask do_image after do_rootfs fakeroot python do_image_complete () { from oe.utils import execute_pre_post_process
Signature · 1 line
2.7.4
PATCH 4/5 image/rm_work: Promote do_image_complete to be more sstate like
In reply to Richard PurdieWe relied on the missing do_image_complete_setscene task to ensure the dummy sstate tarball that was created would never be used. This lead to its own issues and a better fix for SSTATE_SKIP_CREATION has now been merged.
We can therefore make do_image_complete look like a more standard sstate task which means image generation doesn't keep rerunning when using rm_work.
We do need to turn do_image_complete's stamp into an sstate version to handle this (it otherwise matches the do_image_* glob).
--- meta/classes/image.bbclass | 4 ++++ meta/classes/rm_work.bbclass | 5 +++++ 2 files changed, 9 insertions(+)
meta/classes/image.bbclass +4 -0
@@ -289,6 +289,10 @@ do_image_complete[sstate-inputdirs] = "${IMGDEPLOYDIR}" do_image_complete[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}" do_image_complete[stamp-extra-info] = "${MACHINE}" addtask do_image_complete after do_image before do_build +python do_image_complete_setscene () { + sstate_setscene(d) +} +addtask do_image_complete_setscene # Add image-level QA/sanity checks to IMAGE_QA_COMMANDS #
meta/classes/rm_work.bbclass +5 -0
@@ -61,6 +61,11 @@ do_rm_work () { i=dummy break ;; + *do_image_complete*) + mv $i `echo $i | sed -e "s#${j}#${j}_setscene#"` + i=dummy + break + ;; *do_rootfs*|*do_image*|*do_bootimg*|*do_bootdirectdisk*|*do_vmimg*|*do_write_qemuboot_conf*) i=dummy break
Signature · 1 line
2.7.4
PATCH 5/5 image: Make do_image_qa a dummy sstate task to match do_image_complete
In reply to Richard PurdieSimilar to do_image_complete, make do_image_qa a dummy sstate task so that rm_work doesn't cause image generation to repeat on every new build command.
--- meta/classes/image.bbclass | 9 +++++++++ 1 file changed, 9 insertions(+)
meta/classes/image.bbclass +9 -0
@@ -324,6 +324,15 @@ fakeroot python do_image_qa () { } addtask do_image_qa after do_image_complete before do_build +SSTATETASKS += "do_image_qa" +SSTATE_SKIP_CREATION_task-image-qa = '1' +do_image_qa[sstate-inputdirs] = "" +do_image_qa[sstate-outputdirs] = "" +python do_image_qa_setscene () { + sstate_setscene(d) +} +addtask do_image_qa_setscene + def setup_debugfs_variables(d): d.appendVar('IMAGE_ROOTFS', '-dbg') d.appendVar('IMAGE_LINK_NAME', '-dbg')
Signature · 1 line
2.7.4