Skip to contentSkip to conversation

staging: Handle races between binaries and their libs

PATCH In review Richard Purdie 3 participants 6 messages Feb 14, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 staging: Handle races between binaries and their libs

There is a long standing issue where a binary could be installed into the sysroot before its library dependencies. We've always argued nothing should use the binary until its been installed by a dependency but there are issues around binaries which conflict with the host system, for example patch, python3, gzip and more.

With the recent patch changes we've see issues like:
ERROR: gdb-cross-canadian-powerpc-8.3.1-r0 do_patch: Command Error: 'quilt --quiltrc /home/pokybuild/yocto-worker/qemuppc/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-powerpc/8.3.1-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output:
Applying patch 0009-Change-order-of-CFLAGS.patch
patch: /lib64/libattr.so.1: version `ATTR_1.3' not found (required by patch)
Patch 0009-Change-order-of-CFLAGS.patch does not apply (enforce with -f)

which is a symptom of this issue (libattr-native is a dependency of patch-native).

There are other ways to fix this such as disabling libattr in patch, installing patch to a subdirectory and requiring PATH manipulation and so on.

We can simply fix the staging code to handle /bin/ after everything else so lets do that and avoid all these other complications.

---
 meta/classes/staging.bbclass | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
meta/classes/staging.bbclass +11 -1
index 4dd2ed01010..530e23b1853 100644
@@ -470,6 +470,7 @@ python extend_recipe_sysroot() {
         elif os.path.lexists(depdir + "/" + c):
             os.unlink(depdir + "/" + c)
 
+    binfiles = {}
     # Now handle installs
     for dep in configuredeps:
         c = setscenedeps[dep][0]
@@ -562,7 +563,16 @@ python extend_recipe_sysroot() {
                     if l.endswith("/"):
                         staging_copydir(l, targetdir, dest, seendirs)
                         continue
-                    staging_copyfile(l, targetdir, dest, postinsts, seendirs)
+                    if "/bin/" in l:
+                        # defer /bin/* files until last in case they need libs
+                        binfiles[l] = (targetdir, dest)
+                    else:
+                        staging_copyfile(l, targetdir, dest, postinsts, seendirs)
+
+    # Handle deferred binfiles
+    for l in binfiles:
+        (targetdir, dest) = binfiles[l]
+        staging_copyfile(l, targetdir, dest, postinsts, seendirs)
 
     bb.note("Installed into sysroot: %s" % str(msg_adding))
     bb.note("Skipping as already exists in sysroot: %s" % str(msg_exists))
Signature · 1 line
2.20.1
#1 lore raw
Quoted 7 lines · Richard Purdie

There is a long standing issue where a binary could be installed into the sysroot before its library dependencies. We've always argued nothing should use the binary until its been installed by a dependency but there are issues around binaries which conflict with the host system, for example patch, python3, gzip and more.

Quoted 8 lines

With the recent patch changes we've see issues like:
ERROR: gdb-cross-canadian-powerpc-8.3.1-r0 do_patch: Command Error: 'quilt --quiltrc /home/pokybuild/yocto-worker/qemuppc/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-powerpc/8.3.1-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output:
Applying patch 0009-Change-order-of-CFLAGS.patch
patch: /lib64/libattr.so.1: version `ATTR_1.3' not found (required by patch)
Patch 0009-Change-order-of-CFLAGS.patch does not apply (enforce with -f)

which is a symptom of this issue (libattr-native is a dependency of patch-native).

ah so this was the problem I wondered

Quoted 49 lines

There are other ways to fix this such as disabling libattr in patch, installing patch to a subdirectory and requiring PATH manipulation and so on.

We can simply fix the staging code to handle /bin/ after everything else so lets do that and avoid all these other complications.

---
 meta/classes/staging.bbclass | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
meta/classes/staging.bbclass +11 -2
index 4dd2ed01010..530e23b1853 100644
@@ -470,6 +470,7 @@ python extend_recipe_sysroot() {
         elif os.path.lexists(depdir + "/" + c):
             os.unlink(depdir + "/" + c)
 
+    binfiles = {}
     # Now handle installs
     for dep in configuredeps:
         c = setscenedeps[dep][0]
@@ -562,7 +563,16 @@ python extend_recipe_sysroot() {
                     if l.endswith("/"):
                         staging_copydir(l, targetdir, dest, seendirs)
                         continue
-                    staging_copyfile(l, targetdir, dest, postinsts, seendirs)
+                    if "/bin/" in l:
+                        # defer /bin/* files until last in case they need libs
+                        binfiles[l] = (targetdir, dest)
+                    else:
+                        staging_copyfile(l, targetdir, dest, postinsts, seendirs)
+
+    # Handle deferred binfiles
+    for l in binfiles:
+        (targetdir, dest) = binfiles[l]
+        staging_copyfile(l, targetdir, dest, postinsts, seendirs)
 
     bb.note("Installed into sysroot: %s" % str(msg_adding))
     bb.note("Skipping as already exists in sysroot: %s" % str(msg_exists))
--

2.20.1

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#2 lore raw
Quoted 24 lines · Khem Raj
Quoted 22 lines · Richard Purdie

There is a long standing issue where a binary could be installed into the
sysroot before its library dependencies. We've always argued nothing should
use the binary until its been installed by a dependency but there are issues
around binaries which conflict with the host system, for example patch,
python3, gzip and more.

With the recent patch changes we've see issues like:
ERROR: gdb-cross-canadian-powerpc-8.3.1-r0 do_patch: Command Error:
'quilt --quiltrc /home/pokybuild/yocto- worker/qemuppc/build/build/tmp/work/x86_64-nativesdk-pokysdk- linux/gdb-cross-canadian-powerpc/8.3.1-r0/recipe-sysroot- native/etc/quiltrc push' exited with 0 Output:
Applying patch 0009-Change-order-of-CFLAGS.patch
patch: /lib64/libattr.so.1: version `ATTR_1.3' not found (required by patch)
Patch 0009-Change-order-of-CFLAGS.patch does not apply (enforce with -f)

Quoted 5 lines
Quoted 3 lines

which is a symptom of this issue (libattr-native is a dependency of patch-native).

ah so this was the problem I wondered

Its one problem we're seeing, I'm still not sure those two other patches of yours apply cleanly though... :/

Cheers,

Richard

#3 lore raw
Quoted 26 lines · Richard Purdie
Quoted 24 lines · Khem Raj
Quoted 22 lines · Richard Purdie

There is a long standing issue where a binary could be installed into the
sysroot before its library dependencies. We've always argued nothing should
use the binary until its been installed by a dependency but there are issues
around binaries which conflict with the host system, for example patch,
python3, gzip and more.

With the recent patch changes we've see issues like:
ERROR: gdb-cross-canadian-powerpc-8.3.1-r0 do_patch: Command Error:
'quilt --quiltrc /home/pokybuild/yocto- worker/qemuppc/build/build/tmp/work/x86_64-nativesdk-pokysdk- linux/gdb-cross-canadian-powerpc/8.3.1-r0/recipe-sysroot- native/etc/quiltrc push' exited with 0 Output:
Applying patch 0009-Change-order-of-CFLAGS.patch
patch: /lib64/libattr.so.1: version `ATTR_1.3' not found (required by patch)
Patch 0009-Change-order-of-CFLAGS.patch does not apply (enforce with -f)

Quoted 9 lines
Quoted 5 lines
Quoted 3 lines

which is a symptom of this issue (libattr-native is a dependency of patch-native).

ah so this was the problem I wondered

Its one problem we're seeing, I'm still not sure those two other patches of yours apply cleanly though... :/

I have resent the patches for musl upgrade using git send-mail instead of ./scripts/send-pull-request, can you try these from mailing list ?

Quoted 4 lines

Cheers,

Richard

#4 lore raw
In reply to Richard Purdie
Quoted 11 lines

-----Original Message-----
From: [email protected] <openembedded-core- [email protected]> On Behalf Of Richard Purdie
Sent: den 14 februari 2020 14:03
To: [email protected]
Subject: [OE-core] [PATCH] staging: Handle races between binaries and their libs

There is a long standing issue where a binary could be installed into the sysroot before its library dependencies. We've always argued nothing should use the binary until its been installed by a dependency but there are issues

Change "its" to "it has".

Quoted 4 lines

around binaries which conflict with the host system, for example patch, python3, gzip and more.

With the recent patch changes we've see issues like:

Change "see" to "seen".

Quoted 11 lines

ERROR: gdb-cross-canadian-powerpc-8.3.1-r0 do_patch: Command Error: 'quilt --quiltrc /home/pokybuild/yocto-worker/qemuppc/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-powerpc/8.3.1-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output:
Applying patch 0009-Change-order-of-CFLAGS.patch
patch: /lib64/libattr.so.1: version `ATTR_1.3' not found (required by patch)
Patch 0009-Change-order-of-CFLAGS.patch does not apply (enforce with -f)

which is a symptom of this issue (libattr-native is a dependency of patch-native).

There are other ways to fix this such as disabling libattr in patch, installing patch to a subdirectory and requiring PATH manipulation and so on.

We can simply fix the staging code to handle /bin/ after everything else so lets

Change "lets" to "let's".

Quoted 1 line

do that and avoid all these other complications.

Shouldn't /sbin/ be treated the same way as /bin/?

//Peter

Quoted 36 lines
---
 meta/classes/staging.bbclass | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
meta/classes/staging.bbclass +11 -2
index 4dd2ed01010..530e23b1853 100644
@@ -470,6 +470,7 @@ python extend_recipe_sysroot() {
         elif os.path.lexists(depdir + "/" + c):
             os.unlink(depdir + "/" + c)
 
+    binfiles = {}
     # Now handle installs
     for dep in configuredeps:
         c = setscenedeps[dep][0]
@@ -562,7 +563,16 @@ python extend_recipe_sysroot() {
                     if l.endswith("/"):
                         staging_copydir(l, targetdir, dest, seendirs)
                         continue
-                    staging_copyfile(l, targetdir, dest, postinsts, seendirs)
+                    if "/bin/" in l:
+                        # defer /bin/* files until last in case they need libs
+                        binfiles[l] = (targetdir, dest)
+                    else:
+                        staging_copyfile(l, targetdir, dest, postinsts, seendirs)
+
+    # Handle deferred binfiles
+    for l in binfiles:
+        (targetdir, dest) = binfiles[l]
+        staging_copyfile(l, targetdir, dest, postinsts, seendirs)
 
     bb.note("Installed into sysroot: %s" % str(msg_adding))
     bb.note("Skipping as already exists in sysroot: %s" % str(msg_exists))
--
Quoted 1 line

2.20.1

//Peter

#5 lore raw
Quoted 1 line · Peter Kjellerstedt

Shouldn't /sbin/ be treated the same way as /bin/?

I guess so, yes. I've done that.

Cheers,

Richard