Skip to contentSkip to conversation

classes: add workdir_save class

RFC PATCH In review Paul Eggleton 3 participants 5 messages Jul 8, 2020 – Jul 9, 2020

lore
Share & tools
lore
#0 lore raw

cover classes: add workdir_save class

This is fairly trivial but I've found it useful in my setup - I'd like to know if others would as well. Improvement suggestions welcome.

Please review the following changes for suitability for inclusion. If you have any objections or suggestions for improvement, please respond to the patches. If you agree with the changes, please provide your Acked-by.

The following changes since commit b3c96103a5063eeefb0c537227eab3f77616b9c0:

libnl: Extend for native/nativesdk (2020-07-08 10:56:11 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib paule/workdir_save
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=paule/workdir_save

Paul Eggleton (1):
classes: add workdir_save class

 meta/classes/workdir_save.bbclass | 54 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)
 create mode 100644 meta/classes/workdir_save.bbclass
Signature · 1 line
1.8.3.1
#1 lore raw

RFC PATCH 1/1 classes: add workdir_save class

If you are running your builds inside an environment where you don't have access to the build tree (e.g. an autobuilder where you can only download final artifacts such as images), then debugging build failures can be difficult - you can't examine log files, the source tree or output files. When enabled, this class triggers on task failure and saves a tarball of the work directory for the task's recipe and puts it in a configurable location, where it can be picked up by a separate process and made available as a downloadable artifact.

---
 meta/classes/workdir_save.bbclass | 54 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)
 create mode 100644 meta/classes/workdir_save.bbclass
meta/classes/workdir_save.bbclass new file +54 -0
new file mode 100644
index 0000000..7007041
@@ -0,0 +1,54 @@
+# Save the work directory for a recipe when one of its tasks fails.
+# Useful in cases where the environment in which builds are run is
+# ephemeral or otherwise inaccessible for examination during
+# debugging.
+#
+# To enable, simply add the following to your configuration:
+#
+# INHERIT += "workdir_save"
+#
+# Notes:
+# * For this to work you also need corresponding logic in your build
+#   orchestration tool to pick up any files written out to WORKDIR_SAVE_DIR
+#   (with the other assumption being that no files are present there at
+#   the start of the build).
+# * Work directories can be quite large, so saving them can take some time
+#   and of course space.
+#
+# Copyright (c) 2020 Microsoft Corporation
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+WORKDIR_SAVE_DIR ?= "${TMPDIR}/workdir_save"
+WORKDIR_SAVE_ENABLED ?= "1"
+
+addhandler workdir_save_handler
+workdir_save_handler[eventmask] = "bb.build.TaskFailed"
+
+python workdir_save_handler() {
+    import datetime
+    if d.getVar('WORKDIR_SAVE_ENABLED') != '1':
+        return
+
+    base_workdir = d.getVar('BASE_WORKDIR')
+    workdir = d.getVar('WORKDIR')
+    outdir = d.getVar('WORKDIR_SAVE_DIR')
+    bb.utils.mkdirhier(outdir)
+    pn = d.getVar('PN')
+    tstamp = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
+    tarfn = 'workdir_%s_%s.tar.gz' % (pn, tstamp)
+    tarfp = os.path.join(outdir, tarfn)
+    taskname = d.getVar('BB_CURRENTTASK')
+    bb.plain('NOTE: Saving workdir for failed task %s.do_%s to %s' % (pn, taskname, tarfp))
+    try:
+        bb.process.run(['tar', 'czf', tarfp,
+                os.path.relpath(workdir, base_workdir)], cwd=base_workdir)
+    except bb.process.ExecutionError as e:
+        # It is possible for other tasks to be writing to the workdir
+        # while we are tarring it up, in which case tar will return 1,
+        # but we don't care in this situation (tar returns 2 for other
+        # errors so we we will see those)
+        if e.exitcode != 1:
+            bb.warn('workdir save error: %s' % str(e))
+}
Signature · 1 line
1.8.3.1
#2 lore raw
Quoted 12 lines · Paul Eggleton

If you are running your builds inside an environment where you don't have access to the build tree (e.g. an autobuilder where you can only download final artifacts such as images), then debugging build failures
can be difficult - you can't examine log files, the source tree or output
files. When enabled, this class triggers on task failure and saves a tarball of the work directory for the task's recipe and puts it in a configurable location, where it can be picked up by a separate process
and made available as a downloadable artifact.

Quoted 1 line

Seems like a useful idea. I think the name may need "failed" in the name to make it clearer what it does. We also need a section for the classes chapter in the manual.

As another data point, OEQA is using OEQA_DEBUGGING_SAVED_OUTPUT to trigger saving information for reproducible builds...

Cheers,

Richard

#3 lore raw
Quoted 27 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Richard Purdie
Sent: den 8 juli 2020 20:12
To: Paul Eggleton <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [RFC PATCH 1/1] classes: add workdir_save class

Quoted 12 lines · Paul Eggleton

If you are running your builds inside an environment where you don't have access to the build tree (e.g. an autobuilder where you can only download final artifacts such as images), then debugging build failures
can be difficult - you can't examine log files, the source tree or output
files. When enabled, this class triggers on task failure and saves a tarball of the work directory for the task's recipe and puts it in a configurable location, where it can be picked up by a separate process
and made available as a downloadable artifact.

Quoted 1 line

Seems like a useful idea. I think the name may need "failed" in the name to make it clearer what it does. We also need a section for the classes chapter in the manual.

Quoted 6 lines

As another data point, OEQA is using OEQA_DEBUGGING_SAVED_OUTPUT to trigger saving information for reproducible builds...

Cheers,

Richard

How about "save_workdir_on_failure.bbclass"?

//Peter

#4 lore raw
Quoted 9 lines · Peter Kjellerstedt
Quoted 7 lines · Richard Purdie

Seems like a useful idea. I think the name may need "failed" in the name to make it clearer what it does. We also need a section for the classes chapter in the manual.

As another data point, OEQA is using OEQA_DEBUGGING_SAVED_OUTPUT to trigger saving information for reproducible builds...

How about "save_workdir_on_failure.bbclass"?

I'd like to keep the name short if possible, it'll end up in the variable names as well. Thinking ahead we might want to save other information in future beyond the workdir, so we could possibly drop that from the name. What about "failure_save" or "failure_saver"? Also, with underscores or without?

Paul