Skip to contentSkip to conversation

archive.bbclass: archive work directory

PATCH Superseded [email protected] 3 participants 6 messages Jan 6, 2012 – Jan 9, 2012

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover archive.bbclass: archive work directory

From: Wenzong Fan <[email protected]>

Archive the sources and build scripts for license compliance checking by end users or legal departments.

The functionality is in a separate class which 'archive.bbclass' and the user can optionally include it for archiving any packages.

[Yocto #1590]

The following changes since commit 5797feac5f06f5cc363869cd440b82b3eaafd456:
Richard Purdie (1):
image-mklibs/package_ipk: Remove bashisms

are available in the git repository at:

git://git.pokylinux.org/poky-contrib wenzong/1590
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=wenzong/1590

Wenzong Fan (1):
archive.bbclass: archive work directory

 meta/classes/archive.bbclass |   42 ++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 42 insertions(+), 0 deletions(-)
 create mode 100644 meta/classes/archive.bbclass
#1 lore raw

PATCH 1/1 archive.bbclass: archive work directory

From: Wenzong Fan <[email protected]>

Some legal departments believe a complete archive of the work directory is required for certain license compliance issues. We could therefore do with a class which archives up the work directories in each build and provide them so those legal departments can be happy.

Implementations:

Add a new class named 'archive.bbclass' to provide task 'do_archive', and get it called after 'do_patch' before 'do_configure'.

Following cases should be considered to the sources dirs:
1) The sources dir is under $WORKDIR:
Just archive sources and temp/run.* up.

2) The sources dir is outside of $WORKDIR, the only package is gcc:
Copy its sources and temp/run.* to a temporary dir and then archive them up.

3) The sources dir is equal to $WORKDIR:
Just archive whole work dir up.

[YOCTO #1590]

---
 meta/classes/archive.bbclass |   42 ++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 42 insertions(+), 0 deletions(-)
 create mode 100644 meta/classes/archive.bbclass
meta/classes/archive.bbclass new file +42 -0
new file mode 100644
index 0000000..75ac090
@@ -0,0 +1,42 @@
+# Archive the patched sources and build scripts to assist in license
+# compliance by the end user or legal departments.
+
+ARCHIVE_DIR = "${TMPDIR}/archives/${MULTIMACH_TARGET_SYS}/"
+do_archive[dirs] = "${ARCHIVE_DIR}"
+
+archive_do_archive() {
+    # In mostly scenarios the $S is under $WORKDIR and has a separate
+    # dir for storing the sources; but gcc is a special case, its sources
+    # had been moved to the shared location 'tmp/work-shared/'
+    if [[ -d ${S} && ${S} != ${WORKDIR} ]]; then
+        if [[ ${S} =~ "/work-shared/gcc" ]]; then
+            # Create temporary sources directory for gcc
+            mkdir -p ${PF}/temp
+            cp -r ${S} ${PF}
+            cp -r ${S}/../temp/* ${PF}/temp
+            cp -r ${WORKDIR}/temp/* ${PF}/temp
+            tarbase=`pwd`
+        else
+            tarbase=`dirname ${WORKDIR}`
+        fi
+
+        sourcedir=`basename ${S}`
+        tar -C $tarbase -cjf ${PF}.tar.bz2 ${PF}/$sourcedir \
+            ${PF}/temp --exclude log.do_*
+
+        # Remove the temporary gcc sources directory
+        if [[ ${S} =~ "/work-shared/gcc" && -d ${PF} ]]; then
+            rm -rf ${PF}
+        fi
+    fi
+
+    # Just archive whole build directory up when $S is equal to $WORKDIR
+    if [[ -d ${S} && ${S} == ${WORKDIR} ]]; then
+        tarbase=`dirname ${WORKDIR}`
+        tar -C $tarbase -cjf ${PF}.tar.bz2 ${PF} --exclude log.do_*
+    fi
+}
+
+addtask do_archive after do_patch before do_configure
+
+EXPORT_FUNCTIONS do_archive
Signature · 1 line
1.7.0.4
#2 lore raw
Quoted 37 lines · On 01/06/2012 12:20 AM, [email protected] wrote:

From: Wenzong Fan<[email protected]>

Some legal departments believe a complete archive of the work directory is required for certain license compliance issues. We could therefore do with a class which archives up the work directories in each build and provide them so those legal departments can be happy.

Implementations:

Add a new class named 'archive.bbclass' to provide task 'do_archive', and get it called after 'do_patch' before 'do_configure'.

Following cases should be considered to the sources dirs:
1) The sources dir is under $WORKDIR:
Just archive sources and temp/run.* up.

2) The sources dir is outside of $WORKDIR, the only package is gcc:
Copy its sources and temp/run.* to a temporary dir and then archive them up.

3) The sources dir is equal to $WORKDIR:
Just archive whole work dir up.

[YOCTO #1590]

---
  meta/classes/archive.bbclass |   42 ++++++++++++++++++++++++++++++++++++++++++
  1 files changed, 42 insertions(+), 0 deletions(-)

create mode 100644 meta/classes/archive.bbclass

meta/classes/archive.bbclass new file +0 -0
new file mode 100644
index 0000000..75ac090

meta/classes/archive.bbclass line 6

Quoted 6 lines
meta/classes/archive.bbclass new file +6 -0
@@ -0,0 +1,42 @@
+# Archive the patched sources and build scripts to assist in license
+# compliance by the end user or legal departments.
+
+ARCHIVE_DIR = "${TMPDIR}/archives/${MULTIMACH_TARGET_SYS}/"
+do_archive[dirs] = "${ARCHIVE_DIR}"
+

I see you set ARCHIVE_DIR, but I don' see you move or copy any files to this directory, am I missing something?

Quoted 10 lines
meta/classes/archive.bbclass new file +10 -0
+archive_do_archive() {
+    # In mostly scenarios the $S is under $WORKDIR and has a separate
+    # dir for storing the sources; but gcc is a special case, its sources
+    # had been moved to the shared location 'tmp/work-shared/'
+    if [[ -d ${S}&&  ${S} != ${WORKDIR} ]]; then
+        if [[ ${S} =~ "/work-shared/gcc" ]]; then
+            # Create temporary sources directory for gcc
+            mkdir -p ${PF}/temp
+            cp -r ${S} ${PF}
+            cp -r ${S}/../temp/* ${PF}/temp

meta/classes/archive.bbclass line 26

Quoted 10 lines
meta/classes/archive.bbclass new file +10 -0
+            cp -r ${WORKDIR}/temp/* ${PF}/temp
+            tarbase=`pwd`
+        else
+            tarbase=`dirname ${WORKDIR}`
+        fi
+
+        sourcedir=`basename ${S}`
+        tar -C $tarbase -cjf ${PF}.tar.bz2 ${PF}/$sourcedir \
+            ${PF}/temp --exclude log.do_*
+

Also, do you want to name the tarball PF or BP, which deals with removing any multilib naming issues if the build it multilib.

Note BP = ${BPN}-${PV} so it removes the ${PR} if that is important (which it might be).

Quoted 16 lines
meta/classes/archive.bbclass new file +16 -0
+        # Remove the temporary gcc sources directory
+        if [[ ${S} =~ "/work-shared/gcc"&&  -d ${PF} ]]; then
+            rm -rf ${PF}
+        fi
+    fi
+
+    # Just archive whole build directory up when $S is equal to $WORKDIR
+    if [[ -d ${S}&&  ${S} == ${WORKDIR} ]]; then
+        tarbase=`dirname ${WORKDIR}`
+        tar -C $tarbase -cjf ${PF}.tar.bz2 ${PF} --exclude log.do_*
+    fi
+}
+
+addtask do_archive after do_patch before do_configure
+
+EXPORT_FUNCTIONS do_archive
#3 lore raw
Quoted 33 lines · Saul Wold
Quoted 32 lines · On 01/06/2012 12:20 AM, [email protected] wrote:

From: Wenzong Fan<[email protected]>

Some legal departments believe a complete archive of the work directory is required for certain license compliance issues. We could therefore do with a class which archives up the work directories in each build and provide them so those legal departments can be happy.

Implementations:

Add a new class named 'archive.bbclass' to provide task 'do_archive', and get it called after 'do_patch' before 'do_configure'.

Following cases should be considered to the sources dirs:
1) The sources dir is under $WORKDIR:
Just archive sources and temp/run.* up.

2) The sources dir is outside of $WORKDIR, the only package is gcc:
Copy its sources and temp/run.* to a temporary dir and then archive them up.

3) The sources dir is equal to $WORKDIR:
Just archive whole work dir up.

[YOCTO #1590]

---
  meta/classes/archive.bbclass |   42 
++++++++++++++++++++++++++++++++++++++++++
  1 files changed, 42 insertions(+), 0 deletions(-)

create mode 100644 meta/classes/archive.bbclass

Quoted 13 lines

meta/classes/archive.bbclass line 6

Quoted 12 lines
meta/classes/archive.bbclass new file +6 -0
new file mode 100644
index 0000000..75ac090
@@ -0,0 +1,42 @@
+# Archive the patched sources and build scripts to assist in license
+# compliance by the end user or legal departments.
+
+ARCHIVE_DIR = "${TMPDIR}/archives/${MULTIMACH_TARGET_SYS}/"
+do_archive[dirs] = "${ARCHIVE_DIR}"
+

I see you set ARCHIVE_DIR, but I don' see you move or copy any files to this directory, am I missing something?

It just gets bitbake creating '${ARCHIVE_DIR}' before the task started, and then 'do_archive' will be run in this dir.

Quoted 24 lines
Quoted 12 lines
meta/classes/archive.bbclass new file +3 -0
+archive_do_archive() {
+    # In mostly scenarios the $S is under $WORKDIR and has a separate
+    # dir for storing the sources; but gcc is a special case, its 

sources

meta/classes/archive.bbclass new file +7 -0
+    # had been moved to the shared location 'tmp/work-shared/'
+    if [[ -d ${S}&&  ${S} != ${WORKDIR} ]]; then
+        if [[ ${S} =~ "/work-shared/gcc" ]]; then
+            # Create temporary sources directory for gcc
+            mkdir -p ${PF}/temp
+            cp -r ${S} ${PF}
+            cp -r ${S}/../temp/* ${PF}/temp

meta/classes/archive.bbclass line 26

Quoted 10 lines
meta/classes/archive.bbclass new file +10 -0
+            cp -r ${WORKDIR}/temp/* ${PF}/temp
+            tarbase=`pwd`
+        else
+            tarbase=`dirname ${WORKDIR}`
+        fi
+
+        sourcedir=`basename ${S}`
+        tar -C $tarbase -cjf ${PF}.tar.bz2 ${PF}/$sourcedir \
+            ${PF}/temp --exclude log.do_*
+

Also, do you want to name the tarball PF or BP, which deals with removing any multilib naming issues if the build it multilib.

Quoted 1 line

Note BP = ${BPN}-${PV} so it removes the ${PR} if that is important (which it might be).

Sorry I'm not clear your exact meanings here, do you mean the tarball name with '${PF}' will have any issues?

Thanks
Wenzong

Quoted 19 lines
Quoted 18 lines
meta/classes/archive.bbclass new file +7 -0
+        # Remove the temporary gcc sources directory
+        if [[ ${S} =~ "/work-shared/gcc"&&  -d ${PF} ]]; then
+            rm -rf ${PF}
+        fi
+    fi
+
+    # Just archive whole build directory up when $S is equal to 

$WORKDIR

meta/classes/archive.bbclass new file +9 -0
+    if [[ -d ${S}&&  ${S} == ${WORKDIR} ]]; then
+        tarbase=`dirname ${WORKDIR}`
+        tar -C $tarbase -cjf ${PF}.tar.bz2 ${PF} --exclude log.do_*
+    fi
+}
+
+addtask do_archive after do_patch before do_configure
+
+EXPORT_FUNCTIONS do_archive
#4 lore raw
In reply to [email protected]

meta/classes/archive.bbclass line 18

Quoted 8 lines · On Fri, 2012-01-06 at 16:20 +0800, [email protected] wrote:
meta/classes/archive.bbclass new file +8 -0
+    if [[ -d ${S} && ${S} != ${WORKDIR} ]]; then
+        if [[ ${S} =~ "/work-shared/gcc" ]]; then
+            # Create temporary sources directory for gcc
+            mkdir -p ${PF}/temp
+            cp -r ${S} ${PF}
+            cp -r ${S}/../temp/* ${PF}/temp
+            cp -r ${WORKDIR}/temp/* ${PF}/temp
+            tarbase=`pwd`

Special-casing gcc like this seems a bit lame. Is that check really needed?

Also, will the above code work with all shells? It looks a bit bash-specific to me.

p.

#5 lore raw

Re: [PATCH 1/1] archive.bbclass: archive work directoryDistribution

Quoted 11 lines · Phil Blundell

meta/classes/archive.bbclass line 18

Quoted 9 lines · On Fri, 2012-01-06 at 16:20 +0800, [email protected] wrote:
meta/classes/archive.bbclass new file +8 -0
+    if [[ -d ${S}&&  ${S} != ${WORKDIR} ]]; then
+        if [[ ${S} =~ "/work-shared/gcc" ]]; then
+            # Create temporary sources directory for gcc
+            mkdir -p ${PF}/temp
+            cp -r ${S} ${PF}
+            cp -r ${S}/../temp/* ${PF}/temp
+            cp -r ${WORKDIR}/temp/* ${PF}/temp
+            tarbase=`pwd`
Quoted 3 lines

Special-casing gcc like this seems a bit lame. Is that check really needed?

Oh, we might have other packages under this dir, so I'll remove the special-casing 'gcc' here.

Quoted 3 lines

Also, will the above code work with all shells? It looks a bit bash-specific to me.

I assume that the bitbake have to run with 'bash', looks like this is not right on all distributions.

I'll update this.

Thanks
Wenzong

Quoted 10 lines

p.

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core