Skip to contentSkip to conversation

dizzy backports

PATCH New Paul Eggleton 1 participant 3 messages May 14, 2015

lore
Share & tools
lore
#0 lore raw

cover dizzy backports

A couple of backports from master; Fabrice requested his one and I noticed the other hadn't yet been backported in the process.

The following changes since commit 4a44c9287d80dec0973b31d30d3d6250ce4b4df4:

init-install-efi.sh: fix gummiboot entry installation (2015-05-01 12:35:26 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib paule/dizzy-strip
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/dizzy-strip

Ed Bartosh (1):
split_and_strip_files: regroup hardlinks to make build deterministic

Fabrice Coulon (1):
meta/lib/oe/package.py: fix files ownership in packages

 meta/classes/package.bbclass | 48 ++++++++++++++++++++++++--------------------
 meta/lib/oe/package.py       |  3 +--
 2 files changed, 27 insertions(+), 24 deletions(-)
Signature · 1 line
2.1.0
#1 lore raw

PATCH 1/2 meta/lib/oe/package.py: fix files ownership in packages

From: Fabrice Coulon <[email protected]>

This fix solves the problem with the ownership of files in packages. The do_install task was producing correct and expected output but when the files were being put in, e.g. a rpm package, the ownership could be different than that in the do_install task.

[YOCTO #7428]

(From OE-Core master rev: 1a50cc5aeafff0d8ee6c4a41dd2770ecd31455f0)

---
 meta/lib/oe/package.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oe/package.py +1 -1
index a26a631..db8dc28 100644
@@ -31,7 +31,7 @@ def runstrip(arg):
         extraflags = "--remove-section=.comment --remove-section=.note"
 
     # Use mv to break hardlinks
-    stripcmd = "'%s' %s '%s' -o '%s.tmp' && mv '%s.tmp' '%s'" % (strip, extraflags, file, file, file, file)
+    stripcmd = "'%s' %s '%s' -o '%s.tmp' && chown --reference='%s' '%s.tmp' && mv '%s.tmp' '%s'" % (strip, extraflags, file, file, file, file, file, file)
     bb.debug(1, "runstrip: %s" % stripcmd)
 
     ret = subprocess.call(stripcmd, shell=True)
Signature · 1 line
2.1.0
#2 lore raw

PATCH 2/2 split_and_strip_files: regroup hardlinks to make build deterministic

In reply to Paul Eggleton

From: Ed Bartosh <[email protected]>

Reverted 7c0fd561bad0250a00cef63e3d787573112a59cf

Created separate group of hardlinks for the files inside the same package. This should prevent stripped files to be populated outside of package directories.

This turns out not to be straightforward and has overlap with the other hardlink handling code in this area. The code is condensed into a more concise and documented form.

[Original patch from Ed with tweaks from RP]

[YOCTO #7586]

(From OE-Core master rev: 82d00f7254b7d3bb6a167d675d798134884d1b19)

---
 meta/classes/package.bbclass | 48 ++++++++++++++++++++++++--------------------
 meta/lib/oe/package.py       |  3 +--
 2 files changed, 27 insertions(+), 24 deletions(-)
meta/classes/package.bbclass +26 -22
index 4cddbef..b81f4f9 100644
@@ -815,8 +815,8 @@ python split_and_strip_files () {
     #
     elffiles = {}
     symlinks = {}
-    hardlinks = {}
     kernmods = []
+    inodes = {}
     libdir = os.path.abspath(dvar + os.sep + d.getVar("libdir", True))
     baselibdir = os.path.abspath(dvar + os.sep + d.getVar("base_libdir", True))
     if (d.getVar('INHIBIT_PACKAGE_STRIP', True) != '1'):
@@ -854,6 +854,7 @@ python split_and_strip_files () {
                             #bb.note("Sym: %s (%d)" % (ltarget, isELF(ltarget)))
                             symlinks[file] = target
                         continue
+
                     # It's a file (or hardlink), not a link
                     # ...but is it ELF, and is it already stripped?
                     elf_file = isELF(file)
@@ -865,28 +866,30 @@ python split_and_strip_files () {
                                 msg = "File '%s' from %s was already stripped, this will prevent future debugging!" % (file[len(dvar):], pn)
                                 package_qa_handle_error("already-stripped", msg, d)
                             continue
-                        # Check if it's a hard link to something else
-                        if s.st_nlink > 1:
-                            file_reference = "%d_%d" % (s.st_dev, s.st_ino)
-                            # Hard link to something else
-                            hardlinks[file] = file_reference
-                            continue
-                        elffiles[file] = elf_file
+
+                        # At this point we have an unstripped elf file. We need to:
+                        #  a) Make sure any file we strip is not hardlinked to anything else outside this tree
+                        #  b) Only strip any hardlinked file once (no races)
+                        #  c) Track any hardlinks between files so that we can reconstruct matching debug file hardlinks
+
+                        # Use a reference of device ID and inode number to indentify files
+                        file_reference = "%d_%d" % (s.st_dev, s.st_ino)
+                        if file_reference in inodes:
+                            os.unlink(file)
+                            os.link(inodes[file_reference][0], file)
+                            inodes[file_reference].append(file)
+                        else:
+                            inodes[file_reference] = [file]
+                            # break hardlink
+                            bb.utils.copyfile(file, file)
+                            elffiles[file] = elf_file
+                        # Modified the file so clear the cache
+                        cpath.updatecache(file)
 
     #
     # First lets process debug splitting
     #
     if (d.getVar('INHIBIT_PACKAGE_DEBUG_SPLIT', True) != '1'):
-        hardlinkmap = {}
-        # For hardlinks, process only one of the files
-        for file in hardlinks:
-            file_reference = hardlinks[file]
-            if file_reference not in hardlinkmap:
-                # If this is a new file, add it as a reference, and
-                # update it's type, so we can fall through and split
-                elffiles[file] = isELF(file)
-                hardlinkmap[file_reference] = file
-
         for file in elffiles:
             src = file[len(dvar):]
             dest = debuglibdir + os.path.dirname(src) + debugdir + "/" + os.path.basename(src) + debugappend
@@ -899,13 +902,14 @@ python split_and_strip_files () {
             splitdebuginfo(file, fpath, debugsrcdir, sourcefile, d)
 
         # Hardlink our debug symbols to the other hardlink copies
-        for file in hardlinks:
-            if file not in elffiles:
+        for ref in inodes:
+            if len(inodes[ref]) == 1:
+                continue
+            for file in inodes[ref][1:]:
                 src = file[len(dvar):]
                 dest = debuglibdir + os.path.dirname(src) + debugdir + "/" + os.path.basename(src) + debugappend
                 fpath = dvar + dest
-                file_reference = hardlinks[file]
-                target = hardlinkmap[file_reference][len(dvar):]
+                target = inodes[ref][0][len(dvar):]
                 ftarget = dvar + debuglibdir + os.path.dirname(target) + debugdir + "/" + os.path.basename(target) + debugappend
                 bb.utils.mkdirhier(os.path.dirname(fpath))
                 #bb.note("Link %s -> %s" % (fpath, ftarget))
meta/lib/oe/package.py +1 -2
index db8dc28..f8b5322 100644
@@ -30,8 +30,7 @@ def runstrip(arg):
     elif elftype & 8 or elftype & 4:
         extraflags = "--remove-section=.comment --remove-section=.note"
 
-    # Use mv to break hardlinks
-    stripcmd = "'%s' %s '%s' -o '%s.tmp' && chown --reference='%s' '%s.tmp' && mv '%s.tmp' '%s'" % (strip, extraflags, file, file, file, file, file, file)
+    stripcmd = "'%s' %s '%s'" % (strip, extraflags, file)
     bb.debug(1, "runstrip: %s" % stripcmd)
 
     ret = subprocess.call(stripcmd, shell=True)
Signature · 1 line
2.1.0