Skip to contentSkip to conversation

lib/oe/rootfs.py: fix RPM multilib issue

PATCH Superseded Laurentiu Palcu 3 participants 6 messages Feb 12, 2014 – Feb 13, 2014

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover lib/oe/rootfs.py: fix RPM multilib issue

The following changes since commit b70a10ca3ee4a0add90d387e2c8288e531aaa47c:

systemd-serialgetty: Update to match version in recent systemd (2014-02-11 22:59:38 +0000)

are available in the git repository at:

git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib lpalcu/rootfs_refactoring_rpm_multilib_fix

for you to fetch changes up to cc9a8b3a006120ba793051f1a790c3cb3681c28e:

lib/oe/rootfs.py: fix RPM multilib issue (2014-02-12 20:46:37 +0200)

----------------------------------------------------------------
Laurentiu Palcu (1):
lib/oe/rootfs.py: fix RPM multilib issue

 meta/lib/oe/rootfs.py |   16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

Laurentiu Palcu (1):
lib/oe/rootfs.py: fix RPM multilib issue

 meta/lib/oe/rootfs.py |   16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH 1/1 lib/oe/rootfs.py: fix RPM multilib issue

For some odd reason (at least I couldn't find an explanation to this, yet), if a multilib version of a package is installed after the main one (that is: in a different smart session), the main package binaries are not overwritten.

This commit restores the functionality to the original one, before migrating to python: feed all the packages to smart, apart from attempt only ones which are installed separately.

---
 meta/lib/oe/rootfs.py |   16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
meta/lib/oe/rootfs.py +12 -4
index b6baf77..9162e52 100644
@@ -317,10 +317,18 @@ class RpmRootfs(Rootfs):
 
         self.pm.update()
 
-        for pkg_type in self.install_order:
-            if pkg_type in pkgs_to_install:
-                self.pm.install(pkgs_to_install[pkg_type],
-                                [False, True][pkg_type == "aop"])
+        pkgs = []
+        pkgs_attempt = []
+        for pkg_type in pkgs_to_install:
+            if pkg_type == Manifest.PKG_TYPE_ATTEMPT_ONLY:
+                pkgs_attempt = pkgs_to_install[pkg_type]
+                continue
+
+            pkgs += pkgs_to_install[pkg_type]
+
+        self.pm.install(pkgs)
+
+        self.pm.install(pkgs_attempt, True)
 
         self.pm.install_complementary()
 
Signature · 1 line
1.7.9.5
#2 lore raw
Quoted 4 lines · Laurentiu Palcu

For some odd reason (at least I couldn't find an explanation to this, yet), if a multilib version of a package is installed after the main one (that is: in a different smart session), the main package binaries are not overwritten.

For two packages with the same name, but different architectures -- the non-ELF files must be identical (or different file names in each package.)

The ELF files are identified by type and there is a resolution mechanism within RPM to determine which is the 'preferred' version.

We do NOT have the resolution mechanism set, so the first package installed 'wins'. So if you do two packages in two separate transactions, the first version installed will be kept. If you do it in one transaction, then I believe it ends up being the last version [or maybe it's random due to sort order?]

There is a Yocto Project defect to configure the RPM multilib settings, but so far it's just sat there, as nobody has either needed it -- or cared enough to implement it. The reality is installed two packages with the same /bin, /sbin binaries is rare these days.. it's much more useful to install the libraries.

The code you have below may turn out to be a performance improvement.. (each smart/rpm transaction takes setup and cleanup time, so multiple transactions are slower then one larger transaction.) But it's likely not fixing the actual problem you have.

# The default transaction color. This value is a set of bits to
# determine file and dependency affinity for this arch.

#               0       uncolored (i.e. use only arch as install hint)
#               1       Elf32 permitted
#               2       Elf64 permitted
#               4       MIPS reserved
%_transaction_color     3

(4 BTW is MIPS64 - n32)

_transaction_color of 3 indicates we allow Elf32 and Elf64 to be installed..

There is a second item:

_prefer_color that is used to define which item should be installed when there is that ELF conflict. The numbers above follow there as well. By default I believe it sets itself to '2' when doing a single transaction installed, preferring Elf64.

--Mark

Quoted 37 lines

This commit restores the functionality to the original one, before migrating to python: feed all the packages to smart, apart from attempt only ones which are installed separately.

---
  meta/lib/oe/rootfs.py |   16 ++++++++++++----
  1 file changed, 12 insertions(+), 4 deletions(-)
meta/lib/oe/rootfs.py +12 -4
index b6baf77..9162e52 100644
@@ -317,10 +317,18 @@ class RpmRootfs(Rootfs):
 
          self.pm.update()
 
-        for pkg_type in self.install_order:
-            if pkg_type in pkgs_to_install:
-                self.pm.install(pkgs_to_install[pkg_type],
-                                [False, True][pkg_type == "aop"])
+        pkgs = []
+        pkgs_attempt = []
+        for pkg_type in pkgs_to_install:
+            if pkg_type == Manifest.PKG_TYPE_ATTEMPT_ONLY:
+                pkgs_attempt = pkgs_to_install[pkg_type]
+                continue
+
+            pkgs += pkgs_to_install[pkg_type]
+
+        self.pm.install(pkgs)
+
+        self.pm.install(pkgs_attempt, True)
 
          self.pm.install_complementary()
 
#3 lore raw

Hi Mark,

Quoted 19 lines · Mark Hatle
Quoted 4 lines · Laurentiu Palcu

For some odd reason (at least I couldn't find an explanation to this, yet), if a multilib version of a package is installed after the main one (that is: in a different smart session), the main package binaries are not overwritten.

For two packages with the same name, but different architectures -- the non-ELF files must be identical (or different file names in each package.)

The ELF files are identified by type and there is a resolution mechanism within RPM to determine which is the 'preferred' version.

We do NOT have the resolution mechanism set, so the first package installed 'wins'. So if you do two packages in two separate transactions, the first version installed will be kept. If you do it in one transaction, then I believe it ends up being the last version [or maybe it's random due to sort order?]

Quoted 3 lines

There is a Yocto Project defect to configure the RPM multilib settings, but so far it's just sat there, as nobody has either needed it -- or cared enough to implement it.

There are several rpm related multilib issues in bugzilla left hanging, unfortunately. :|

Quoted 3 lines

The reality is
installed two packages with the same /bin, /sbin binaries is rare these days.. it's much more useful to install the libraries.

Apparently, the multilib test we currently have on AB is to install lib32-connman-gnome and test the ELF class of the /usr/bin/connman-applet binary...

Quoted 6 lines

The code you have below may turn out to be a performance improvement.. (each smart/rpm transaction takes setup and cleanup time, so multiple transactions are slower then one larger transaction.) But it's likely not fixing the actual problem you have.

True, this problem should be properly fixed. This patch, however, just restores the behavior we already used in the old bash code which people seemed to have used for a long time now.

Quoted 11 lines

# The default transaction color. This value is a set of bits to
# determine file and dependency affinity for this arch.

#               0       uncolored (i.e. use only arch as install hint)
#               1       Elf32 permitted
#               2       Elf64 permitted
#               4       MIPS reserved
%_transaction_color     3

(4 BTW is MIPS64 - n32)

Quoted 8 lines

_transaction_color of 3 indicates we allow Elf32 and Elf64 to be installed..

There is a second item:

_prefer_color that is used to define which item should be installed when there is that ELF conflict. The numbers above follow there as well. By default I believe it sets itself to '2' when doing a single transaction installed, preferring Elf64.

If these parameters (_transaction_color and _prefer_color) can be easily set before each transaction, I suppose we could alter them accordingly, depending on the type of packages we're going to install. Doesn't seem very complicated, in theory...

laurentiu

Quoted 45 lines

--Mark

Quoted 27 lines

This commit restores the functionality to the original one, before migrating to python: feed all the packages to smart, apart from attempt only ones which are installed separately.

---
 meta/lib/oe/rootfs.py |   16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
meta/lib/oe/rootfs.py +5 -4
index b6baf77..9162e52 100644
@@ -317,10 +317,18 @@ class RpmRootfs(Rootfs):
 
         self.pm.update()
 
-        for pkg_type in self.install_order:
-            if pkg_type in pkgs_to_install:
-                self.pm.install(pkgs_to_install[pkg_type],
-                                [False, True][pkg_type == "aop"])
+        pkgs = []
+        pkgs_attempt = []
+        for pkg_type in pkgs_to_install:
+            if pkg_type == Manifest.PKG_TYPE_ATTEMPT_ONLY:
+                pkgs_attempt = pkgs_to_install[pkg_type]

meta/lib/oe/rootfs.py class RpmRootfs(Rootfs): · line 334

Quoted 10 lines
meta/lib/oe/rootfs.py +7 -0
+                continue
+
+            pkgs += pkgs_to_install[pkg_type]
+
+        self.pm.install(pkgs)
+
+        self.pm.install(pkgs_attempt, True)
 
         self.pm.install_complementary()
 

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#4 lore raw
In reply to Laurentiu Palcu
Quoted 21 lines · Laurentiu Palcu

For some odd reason (at least I couldn't find an explanation to this, yet), if a multilib version of a package is installed after the main one (that is: in a different smart session), the main package binaries are not overwritten.

This commit restores the functionality to the original one, before migrating to python: feed all the packages to smart, apart from attempt only ones which are installed separately.

---
 meta/lib/oe/rootfs.py |   16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
meta/lib/oe/rootfs.py +0 -0
index b6baf77..9162e52 100644
@@ -317,10 +317,18 @@ class RpmRootfs(Rootfs):
 
         self.pm.update()

meta/lib/oe/rootfs.py class RpmRootfs(Rootfs): · line 324

Quoted 10 lines
meta/lib/oe/rootfs.py +5 -4
 
-        for pkg_type in self.install_order:
-            if pkg_type in pkgs_to_install:
-                self.pm.install(pkgs_to_install[pkg_type],
-                                [False, True][pkg_type == "aop"])
+        pkgs = []
+        pkgs_attempt = []
+        for pkg_type in pkgs_to_install:
+            if pkg_type == Manifest.PKG_TYPE_ATTEMPT_ONLY:
+                pkgs_attempt = pkgs_to_install[pkg_type]

Should this be a += ?

This might be clearer as an if: else:

Quoted 10 lines
meta/lib/oe/rootfs.py +7 -0
+                continue
+
+            pkgs += pkgs_to_install[pkg_type]
+
+        self.pm.install(pkgs)
+
+        self.pm.install(pkgs_attempt, True)
 
         self.pm.install_complementary()
 
#5 lore raw
Quoted 23 lines · Richard Purdie
Quoted 22 lines · Laurentiu Palcu

For some odd reason (at least I couldn't find an explanation to this, yet), if a multilib version of a package is installed after the main one (that is: in a different smart session), the main package binaries are not overwritten.

This commit restores the functionality to the original one, before migrating to python: feed all the packages to smart, apart from attempt only ones which are installed separately.

---
 meta/lib/oe/rootfs.py |   16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
meta/lib/oe/rootfs.py +0 -0
index b6baf77..9162e52 100644
@@ -317,10 +317,18 @@ class RpmRootfs(Rootfs):
 
         self.pm.update()
 
Quoted 11 lines

meta/lib/oe/rootfs.py class RpmRootfs(Rootfs): · line 324

Quoted 9 lines
meta/lib/oe/rootfs.py +5 -4
-        for pkg_type in self.install_order:
-            if pkg_type in pkgs_to_install:
-                self.pm.install(pkgs_to_install[pkg_type],
-                                [False, True][pkg_type == "aop"])
+        pkgs = []
+        pkgs_attempt = []
+        for pkg_type in pkgs_to_install:
+            if pkg_type == Manifest.PKG_TYPE_ATTEMPT_ONLY:
+                pkgs_attempt = pkgs_to_install[pkg_type]

Should this be a += ?

Indeed, I guess I was thinking that there can be only one attempt only list. Nevertheless, should be += there. Thanks!

Quoted 2 lines

This might be clearer as an if: else:

Agreed.

Incoming v2...

laurentiu

Quoted 13 lines
Quoted 11 lines
meta/lib/oe/rootfs.py +7 -0
+                continue
+
+            pkgs += pkgs_to_install[pkg_type]
+
+        self.pm.install(pkgs)
+
+        self.pm.install(pkgs_attempt, True)
 
         self.pm.install_complementary()