Skip to contentSkip to conversation

Fix some errors in archiver class

PATCH New [email protected] 1 participant 5 messages Mar 22, 2016

lore
Share & tools
lore
#0 lore raw

cover Fix some errors in archiver class

From: Mariano Lopez <[email protected]>

There were some errors in the archiver class that are fixed with this series:

- Don't expand python functions in dumpdata
Fixed a datastore expansion for python functions
- Fix use of ARCHIVER_WORKDIR and ARCHIVER_OUTDIR
In unpack and patch task the outdir was used as workdir
- Fix gcc-source corner case
gcc-source doesn't have do_configure task used for do_ar_configure
- Fix tar name for git repositories
For git repositories the file generated for the original source had the domain in the file name.

The following changes since commit 8debfea81e69d038bd2d56314b272cb74f5582ed:

local.conf.sample: Disable prelink by default (2016-03-13 22:09:05 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib mariano/bug9245
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug9245

Mariano Lopez (4):
archiver.bbclass: Don't expand python functions in dumpdata archiver.bbclass: Fix use of ARCHIVER_WORKDIR and ARCHIVER_OUTDIR archiver.bbclass: Fix gcc-source corner case archiver.bbclass: Fix tar name for git repositories

 meta/classes/archiver.bbclass | 76 +++++++++++++++++++++++--------------------
 1 file changed, 40 insertions(+), 36 deletions(-)
Signature · 1 line
2.6.2
#1 lore raw

PATCH 1/4 archiver.bbclass: Don't expand python functions in dumpdata

From: Mariano Lopez <[email protected]>

Currently the dumpdata task expands python data in the datastore, in some functions this causes a silent error and the task will fail.

The change also rewrite the function to make a bit clearer.

---
 meta/classes/archiver.bbclass | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)
meta/classes/archiver.bbclass +7 -8
index b95176b..221c7f4 100644
@@ -335,14 +335,13 @@ python do_dumpdata () {
     dumpfile = os.path.join(d.getVar('ARCHIVER_OUTDIR', True), \
         '%s-showdata.dump' % d.getVar('PF', True))
     bb.note('Dumping metadata into %s' % dumpfile)
-    f = open(dumpfile, 'w')
-    # emit variables and shell functions
-    bb.data.emit_env(f, d, True)
-    # emit the metadata which isn't valid shell
-    for e in d.keys():
-        if bb.data.getVarFlag(e, 'python', d):
-            f.write("\npython %s () {\n%s}\n" % (e, bb.data.getVar(e, d, True)))
-    f.close()
+    with open(dumpfile, "w") as f:
+        # emit variables and shell functions
+        bb.data.emit_env(f, d, True)
+        # emit the metadata which isn't valid shell
+        for e in d.keys():
+            if d.getVarFlag(e, "python", False):
+                f.write("\npython %s () {\n%s}\n" % (e, d.getVar(e, False)))
 }
 
 SSTATETASKS += "do_deploy_archives"
Signature · 1 line
2.6.2
#2 lore raw

PATCH 2/4 archiver.bbclass: Fix use of ARCHIVER_WORKDIR and ARCHIVER_OUTDIR

In reply to [email protected]

From: Mariano Lopez <[email protected]>

Currently do_unpack_and_patch() and do_ar_configured() are using the ARCHIVER_OUTDIR as the ARCHIVER_WORKDIR, this lead to have duplicated files inside the tars when using the archiver class for patched and configured source.

---
 meta/classes/archiver.bbclass | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)
meta/classes/archiver.bbclass +11 -11
index 221c7f4..4064674 100644
@@ -156,8 +156,9 @@ python do_ar_patched() {
 
     # Get the ARCHIVER_OUTDIR before we reset the WORKDIR
     ar_outdir = d.getVar('ARCHIVER_OUTDIR', True)
+    ar_workdir = d.getVar('ARCHIVER_WORKDIR', True)
     bb.note('Archiving the patched source...')
-    d.setVar('WORKDIR', ar_outdir)
+    d.setVar('WORKDIR', ar_workdir)
     create_tarball(d, d.getVar('S', True), 'patched', ar_outdir)
 }
 
@@ -250,21 +251,20 @@ python do_unpack_and_patch() {
             [ 'patched', 'configured'] and \
             d.getVarFlag('ARCHIVER_MODE', 'diff', True) != '1':
         return
-    # Change the WORKDIR to make do_unpack do_patch run in another dir.
     ar_outdir = d.getVar('ARCHIVER_OUTDIR', True)
-    d.setVar('WORKDIR', ar_outdir)
-
-    # The changed 'WORKDIR' also caused 'B' changed, create dir 'B' for the
-    # possibly requiring of the following tasks (such as some recipes's
-    # do_patch required 'B' existed).
-    bb.utils.mkdirhier(d.getVar('B', True))
+    ar_workdir = d.getVar('ARCHIVER_WORKDIR', True)
 
     # The kernel class functions require it to be on work-shared, so we dont change WORKDIR
     if not bb.data.inherits_class('kernel-yocto', d):
-        ar_outdir = d.getVar('ARCHIVER_OUTDIR', True)
-        d.setVar('WORKDIR', ar_outdir)
-        bb.build.exec_func('do_unpack', d)
+        # Change the WORKDIR to make do_unpack do_patch run in another dir.
+        d.setVar('WORKDIR', ar_workdir)
 
+        # The changed 'WORKDIR' also caused 'B' changed, create dir 'B' for the
+        # possibly requiring of the following tasks (such as some recipes's
+        # do_patch required 'B' existed).
+        bb.utils.mkdirhier(d.getVar('B', True))
+
+        bb.build.exec_func('do_unpack', d)
 
     # Save the original source for creating the patches
     if d.getVarFlag('ARCHIVER_MODE', 'diff', True) == '1':
Signature · 1 line
2.6.2
#3 lore raw

PATCH 3/4 archiver.bbclass: Fix gcc-source corner case

In reply to [email protected]

From: Mariano Lopez <[email protected]>

Bitbake couldn't add the task ar_configured when trying to archive the configured source for gcc-source-${PV} recipes. This is because the task depended in the do_configure and this task doesn't exist for gcc-source.

This fix allows to archive configured gcc-source recipe.

---
 meta/classes/archiver.bbclass | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)
meta/classes/archiver.bbclass +16 -2
index 4064674..d709f78 100644
@@ -73,8 +73,15 @@ python () {
         # We can't use "addtask do_ar_configured after do_configure" since it
         # will cause the deptask of do_populate_sysroot to run not matter what
         # archives we need, so we add the depends here.
-        d.appendVarFlag('do_ar_configured', 'depends', ' %s:do_configure' % pn)
+
+        # There is a corner case with "gcc-source-${PV}" recipes, they don't have
+        # the "do_configure" task, so we need to use "do_preconfigure"
+        if pn.startswith("gcc-source-"):
+            d.appendVarFlag('do_ar_configured', 'depends', ' %s:do_preconfigure' % pn)
+        else:
+            d.appendVarFlag('do_ar_configured', 'depends', ' %s:do_configure' % pn)
         d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_configured' % pn)
+
     elif ar_src:
         bb.fatal("Invalid ARCHIVER_MODE[src]: %s" % ar_src)
 
@@ -168,11 +175,18 @@ python do_ar_configured() {
     ar_outdir = d.getVar('ARCHIVER_OUTDIR', True)
     if d.getVarFlag('ARCHIVER_MODE', 'src', True) == 'configured':
         bb.note('Archiving the configured source...')
+        pn = d.getVar('PN', True)
+        # "gcc-source-${PV}" recipes don't have "do_configure"
+        # task, so we need to run "do_preconfigure" instead
+        if pn.startswith("gcc-source-"):
+            d.setVar('WORKDIR', d.getVar('ARCHIVER_WORKDIR', True))
+            bb.build.exec_func('do_preconfigure', d)
+
         # The libtool-native's do_configure will remove the
         # ${STAGING_DATADIR}/aclocal/libtool.m4, so we can't re-run the
         # do_configure, we archive the already configured ${S} to
         # instead of.
-        if d.getVar('PN', True) != 'libtool-native':
+        elif pn != 'libtool-native':
             # Change the WORKDIR to make do_configure run in another dir.
             d.setVar('WORKDIR', d.getVar('ARCHIVER_WORKDIR', True))
             if bb.data.inherits_class('kernel-yocto', d):
Signature · 1 line
2.6.2
#4 lore raw

PATCH 4/4 archiver.bbclass: Fix tar name for git repositories

In reply to [email protected]

From: Mariano Lopez <[email protected]>

When archiving the original source, the git repositories have the name as they are in the $DL_DIR plus the source revision; i.e. "git.yoctoproject.org.linux-yocto-4.4.git.89419d8b90_dadb436904.tar.gz". This change set the tar name to $PF.tar.gz instead, to have consistency with the others archives created by the class.

---
 meta/classes/archiver.bbclass | 21 ++++++---------------
 1 file changed, 6 insertions(+), 15 deletions(-)
meta/classes/archiver.bbclass +6 -15
index d709f78..7758f3c 100644
@@ -126,21 +126,9 @@ python do_ar_original() {
         if os.path.isfile(local):
             shutil.copy(local, ar_outdir)
         elif os.path.isdir(local):
-            basename = os.path.basename(local)
-
             tmpdir = tempfile.mkdtemp(dir=d.getVar('ARCHIVER_WORKDIR', True))
             fetch.unpack(tmpdir, (url,))
-
-            os.chdir(tmpdir)
-            # We eliminate any AUTOINC+ in the revision.
-            try:
-                src_rev = bb.fetch2.get_srcrev(d).replace('AUTOINC+','')
-            except:
-                src_rev = 'NOREV'
-            tarname = os.path.join(ar_outdir, basename + '.' + src_rev + '.tar.gz')
-            tar = tarfile.open(tarname, 'w:gz')
-            tar.add('.')
-            tar.close()
+            create_tarball(d, tmpdir + '/.', '', ar_outdir)
 
     # Emit patch series files for 'original'
     bb.note('Writing patch series files...')
@@ -222,8 +210,11 @@ def create_tarball(d, srcdir, suffix, ar_outdir):
         return
 
     bb.utils.mkdirhier(ar_outdir)
-    tarname = os.path.join(ar_outdir, '%s-%s.tar.gz' % \
-            (d.getVar('PF', True), suffix))
+    if suffix:
+        filename = '%s-%s.tar.gz' % (d.getVar('PF', True), suffix)
+    else:
+        filename = '%s.tar.gz' % d.getVar('PF', True)
+    tarname = os.path.join(ar_outdir, filename)
 
     srcdir = srcdir.rstrip('/')
     dirname = os.path.dirname(srcdir)
Signature · 1 line
2.6.2