Skip to contentSkip to conversation

rm_work: add note about how to clean all built recipes' workdir

PATCH In review Chen Qi 2 participants 3 messages Dec 1, 2022 – Dec 2, 2022

lore
Share & tools
lore

Chen Qi

author
#0 lore raw

PATCH 1/1 rm_work: add note about how to clean all built recipes' workdir

The do_build recrdeptask dependencies are removed for a cleaner dependency relationship. This, however, makes rm_work behave differently than previous releases. Not all built recipes' ${WORKDIR} are cleaned. This is because we use the 'depends' varflag to specify some dependencies, and rm_work cannot handle such case. e.g.
do_rootfs[depends] += " makedevs-native:do_populate_sysroot ..."

I really don't see an easy way to restore previous behavior, but we can use `--runall build' to ensure all built recipes' ${WORKDIR} are cleaned.
e.g.
bitbake core-image-minimal --runall build

It would be helpful to add such note in this bbclass, as people who use rm_work might also want cleaning all workdirs.

The following testing is done to ensure that the extra tasks added by
specifying '--runall build' should be mostly do_build, do_rm_work_all
and do_rm_work.
'''
1. bitbake -g core-image-minimal && grep label task-depends.dot > current
2. bitbake -g core-image-minimal --runall build && grep label task-depends.dot > runall-build
3. diff current runall-build | grep '^>' | grep -v do_rm_work_all | grep -v do_build | grep -v do_rm_work
'''
In my testing, the output is empty, which demonstrates that all extra
tasks introduced by '--runall build' are these three tasks.

---
 meta/classes/rm_work.bbclass | 7 +++++++
 1 file changed, 7 insertions(+)
meta/classes/rm_work.bbclass +7 -0
index 4121a13279..b54d949e46 100644
@@ -23,6 +23,13 @@
 #     echo "bar" >${WORKDIR}/foo
 # }
 # RM_WORK_EXCLUDE_ITEMS += "foo"
+#
+# Note:
+# After removing the do_build recrdeptask dependencies, merely running
+# `bitbake recipeA' does not clean all built recipes' ${WORKDIR}.
+# To ensure all built recipes' ${WORKDIR} are cleaned up, run
+# `bitbake recipeA --runall build'
+#
 RM_WORK_EXCLUDE_ITEMS = "temp"
 
 # Use the completion scheduler by default when rm_work is active
Signature · 1 line
2.17.1
#1 lore raw
Quoted 38 lines · Chen Qi

The do_build recrdeptask dependencies are removed for a cleaner dependency relationship. This, however, makes rm_work behave differently than previous releases. Not all built recipes' ${WORKDIR} are cleaned. This is because we use the 'depends' varflag to specify some dependencies, and rm_work cannot handle such case. e.g.
do_rootfs[depends] += " makedevs-native:do_populate_sysroot ..."

I really don't see an easy way to restore previous behavior, but we can use `--runall build' to ensure all built recipes' ${WORKDIR} are cleaned.
e.g.
bitbake core-image-minimal --runall build

It would be helpful to add such note in this bbclass, as people who use rm_work might also want cleaning all workdirs.

The following testing is done to ensure that the extra tasks added by
specifying '--runall build' should be mostly do_build, do_rm_work_all
and do_rm_work.
'''
1. bitbake -g core-image-minimal && grep label task-depends.dot > current
2. bitbake -g core-image-minimal --runall build && grep label task-depends.dot > runall-build
3. diff current runall-build | grep '^>' | grep -v do_rm_work_all | grep -v do_build | grep -v do_rm_work
'''
In my testing, the output is empty, which demonstrates that all extra
tasks introduced by '--runall build' are these three tasks.

---
 meta/classes/rm_work.bbclass | 7 +++++++
 1 file changed, 7 insertions(+)
meta/classes/rm_work.bbclass +0 -0
index 4121a13279..b54d949e46 100644

meta/classes/rm_work.bbclass line 28

Quoted 6 lines
meta/classes/rm_work.bbclass +3 -0
@@ -23,6 +23,13 @@
 #     echo "bar" >${WORKDIR}/foo
 # }
 # RM_WORK_EXCLUDE_ITEMS += "foo"
+#
+# Note:
+# After removing the do_build recrdeptask dependencies,

I don't think this wording is going to age well, it refers to a previous event which nobody will remember in the future. I think if we do add anything, it needs to be worded differently.

meta/classes/rm_work.bbclass line 33

Quoted 6 lines
meta/classes/rm_work.bbclass +4 -0
 merely running
+# `bitbake recipeA' does not clean all built recipes' ${WORKDIR}.
+# To ensure all built recipes' ${WORKDIR} are cleaned up, run
+# `bitbake recipeA --runall build'
+#
 RM_WORK_EXCLUDE_ITEMS = "temp"

I am worried this is really something most users of rm_work will consider a bug and want the issue fixed "properly" too. As you say, finding a fix for it is going to be rather tricky though :(.

Cheers,

Richard

author
#2 lore raw

Hi Richard,

When I was trying to figure out a way to solve the problem, I found that with a single line change, things will just work fine.

- bb.build.addtask('do_rm_work', 'do_build', ' '.join(deps), d)
+ bb.build.addtask('do_rm_work', 'do_build do_rm_work_all', ' '.join(deps), d)

I really didn't expect the above change would make any difference, but it did.
Looking at the meaning of recrdeptask (from the comments in runqueue.py), it seems that the original codes should just work, even without this change, because we have:
do_rm_work_all[recrdeptask] = 'do_rm_work'.

Besides, bitbake's manual about recrdeptask does not seem accurate. It says it's only related to RDEPENDS and RRECOMMENDS, but in fact, DEPENDS and '[depends]' also has effect.

Do you have any clue on why such change would work?

Regards,
Qi

-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Richard Purdie via lists.openembedded.org
Sent: Thursday, December 1, 2022 7:41 PM
To: Chen, Qi <[email protected]>; [email protected]
Subject: Re: [OE-core][master][PATCH] rm_work: add note about how to clean all built recipes' workdir

Quoted 38 lines · Chen Qi

The do_build recrdeptask dependencies are removed for a cleaner dependency relationship. This, however, makes rm_work behave differently than previous releases. Not all built recipes' ${WORKDIR} are cleaned. This is because we use the 'depends' varflag to specify some dependencies, and rm_work cannot handle such case. e.g.
do_rootfs[depends] += " makedevs-native:do_populate_sysroot ..."

I really don't see an easy way to restore previous behavior, but we can use `--runall build' to ensure all built recipes' ${WORKDIR} are cleaned.
e.g.
bitbake core-image-minimal --runall build

It would be helpful to add such note in this bbclass, as people who use rm_work might also want cleaning all workdirs.

The following testing is done to ensure that the extra tasks added by specifying '--runall build' should be mostly do_build, do_rm_work_all and do_rm_work.
'''
1. bitbake -g core-image-minimal && grep label task-depends.dot >
current 2. bitbake -g core-image-minimal --runall build && grep label task-depends.dot > runall-build 3. diff current runall-build | grep '^>' | grep -v do_rm_work_all | grep -v do_build | grep -v do_rm_work ''' In my testing, the output is empty, which demonstrates that all extra tasks introduced by '--runall build' are these three tasks.

---
 meta/classes/rm_work.bbclass | 7 +++++++
 1 file changed, 7 insertions(+)
(quoted diff) +0 -0

b/meta/classes/rm_work.bbclass index 4121a13279..b54d949e46 100644

meta/classes/rm_work.bbclass line 28

Quoted 6 lines
meta/classes/rm_work.bbclass +3 -0
@@ -23,6 +23,13 @@
 #     echo "bar" >${WORKDIR}/foo
 # }
 # RM_WORK_EXCLUDE_ITEMS += "foo"
+#
+# Note:
+# After removing the do_build recrdeptask dependencies,

I don't think this wording is going to age well, it refers to a previous event which nobody will remember in the future. I think if we do add anything, it needs to be worded differently.

meta/classes/rm_work.bbclass line 33

Quoted 6 lines
meta/classes/rm_work.bbclass +4 -0
 merely running
+# `bitbake recipeA' does not clean all built recipes' ${WORKDIR}.
+# To ensure all built recipes' ${WORKDIR} are cleaned up, run # 
+`bitbake recipeA --runall build'
+#
 RM_WORK_EXCLUDE_ITEMS = "temp"

I am worried this is really something most users of rm_work will consider a bug and want the issue fixed "properly" too. As you say, finding a fix for it is going to be rather tricky though :(.

Cheers,

Richard