Skip to contentSkip to conversation

Multilib, duplicate deps, and misc cleanup

PATCH In review Mark Hatle 3 participants 21 messages Sep 30, 2012 – Oct 1, 2012

lore
Share & tools
lore

Mark Hatle

author
#0 lore raw

cover Multilib, duplicate deps, and misc cleanup

The patch set (01-03) implementes SDK multilib support. This was tested by building core-image-minimal and core-image-sato w/ the -c populate_sdk task.

The patch (04) removes unintentional duplicate entries from the various dependency variables. This works along side a related bitbake patch. Even if the bitbake patch is not accepted, this is still useful as it resolves some potential failures with deps changing behavior. This was tested with buildhistory -- there were NO changes in the produced files.

The patch (05) fixes an issue with the deb and ipk implementation. Use of '<' and '>' is deprecated, with '<<' and '>>' replacing them -- meaning greater then and less then. ('<' and '>' mean greater then or equal and less then or equal -- which is counter intuitive.)

The patch (06) basic cleanup. Remove duplicate dependencies, behavior verified by using buildhistory.

The patch (07) cleans up the old usage for checking for target packages, PN == BPN. This unfortunately does not work with multilib packages. New CLASSOVERRIDE == class-target is used instead. This resolves a number of minor issues when building packages with multilibs enabled. Verified by using buildhistory -- only expected changes occured. (lib32 on x86_64)

The patch (08) fixes a defect, when multilibs are enabled, with kernel and kernel related packages. The STAGING_DIR_KERNEL was set, using := before all of the pieces had their final values -- move this to a bbclass to ensure that the full configuration is loaded.

The patch set (09-12) implement a number of minor RPM fixes. 12 is a workaround for a BerkleyDB problem. It is intentional that the workaround still display errors to the end user to make it clear the workaround is being used.

The following changes since commit d83e218dc480a09befddf8b934d774519cdbacb5:

xserver-xorg: Remove RCONFLICTS against xserver-xorg (2012-09-28 15:16:52 +0100)

are available in the git repository at:
git://git.yoctoproject.org/poky-contrib mhatle/dep_cleanup
http://git.yoctoproject.org/cgit.cgi//log/?h=mhatle/dep_cleanup

Mark Hatle (12):
multilib: Add support for cross-canadian multilib packages multilib - crosssdk: Stop building multilib for crosssdk packages
populate_sdk_base: Update extraction script for multilibs bb.utils.explode_dep_versions: Update to ensure we avoid duplicate
deps
package_deb/ipk: Remap < and > to << and >>
libxcb: Update DEPENDS to avoid duplicate entries
Cleanup: fix PN == BPN cases
multilib: Move redefinition of STAGING_DIR_KERNEL
rpm-native: Fix 'uuid_rc_t' undeclared error when compiling
rpm: Fix file contention issue
rpm: Add rpm patch to fix git_strerror issues
rpm: Implement workaround for DB_BUFFER_SMALL error

 meta/classes/base.bbclass                          |   43 ++++++++++-
 meta/classes/distutils-base.bbclass                |    2 +-
 meta/classes/insane.bbclass                        |   55 ++++++++++++++-
 meta/classes/kernel.bbclass                        |   20 +++--
 meta/classes/libc-common.bbclass                   |   13 +++-
 meta/classes/multilib.bbclass                      |   22 +++++--
 meta/classes/package.bbclass                       |   20 ++++-
 meta/classes/package_deb.bbclass                   |   18 +++++
 meta/classes/package_ipk.bbclass                   |   18 +++++
 meta/classes/package_rpm.bbclass                   |   61 +++++++++-------
 meta/classes/populate_sdk_base.bbclass             |    5 +-
 meta/conf/multilib.conf                            |    1 -
 meta/lib/oe/classextend.py                         |    4 +-
 meta/recipes-core/ncurses/ncurses.inc              |    6 +-
 .../rpm/rpm/makefile-am-exec-hook.patch            |   31 ++++++++
 .../rpm/rpm/rpm-db_buffer_small.patch              |   77 ++++++++++++++++++++
 .../rpm/rpm/rpm-stub-out-git_strerror.patch        |   60 +++++++++++++++
 .../rpm/rpm/rpm-uuid-include.patch                 |   35 +++++++++
 meta/recipes-devtools/rpm/rpm_5.4.9.bb             |    6 ++-
 meta/recipes-extended/pigz/pigz.inc                |    4 +-
 meta/recipes-graphics/xcb/libxcb.inc               |    2 +-
 meta/recipes-graphics/xcb/libxcb_1.1.91.bb         |    2 -
 meta/recipes-graphics/xcb/libxcb_1.8.1.bb          |    2 +-
 meta/recipes-graphics/xcb/libxcb_git.bb            |    2 -
 meta/recipes-support/apr/apr-util_1.4.1.bb         |    4 +-
 25 files changed, 441 insertions(+), 72 deletions(-)
 create mode 100644 meta/recipes-devtools/rpm/rpm/makefile-am-exec-hook.patch
 create mode 100644 meta/recipes-devtools/rpm/rpm/rpm-db_buffer_small.patch
 create mode 100644 meta/recipes-devtools/rpm/rpm/rpm-stub-out-git_strerror.patch
 create mode 100644 meta/recipes-devtools/rpm/rpm/rpm-uuid-include.patch
Signature · 1 line
1.7.3.4
author
#1 lore raw

PATCH 1/12 multilib: Add support for cross-canadian multilib packages

Add support for the generation of cross-canadian packages.

Each cross-canadian package has:
PN = "pkg-cross-canadian-${TRANSLATED_TARGET_ARCH}"

in order for that to be evaluated properly with multilibs enabled, it was necessary to detect both the presence of the cross-canadian packages and then update the vars using the OVERRIDE for the multilib. Additional checks were made to ensure that any dependency that sais "cross-canadian" did not get prefixed with the MLPREFIX.

Also, make sure that even when building multilib cross-canadian packages, we only use the single SDK PACKAGE_ARCH, we don't want or need variants.

---
 meta/classes/base.bbclass     |   39 +++++++++++++++++++++++++++++++++++++--
 meta/classes/multilib.bbclass |   18 ++++++++++++++----
 meta/lib/oe/classextend.py    |    4 ++--
 3 files changed, 53 insertions(+), 8 deletions(-)
meta/classes/base.bbclass +37 -2
index 801896a..43a352d 100644
@@ -183,6 +183,8 @@ def preferred_ml_updates(d):
             providers.append(v)
 
     for pkg, reason in blacklists.items():
+        if pkg.endswith("-native") or pkg.startswith("nativesdk-") or 'cross-canadian' in pkg:
+            continue
         for p in prefixes:
             newpkg = p + "-" + pkg
             if not d.getVarFlag('PNBLACKLIST', newpkg, True):
@@ -193,6 +195,17 @@ def preferred_ml_updates(d):
         pkg = v.replace("PREFERRED_VERSION_", "")
         if pkg.endswith("-native") or pkg.startswith("nativesdk-"):
             continue
+        if 'cross-canadian' in pkg:
+            for p in prefixes:
+                localdata = bb.data.createCopy(d)
+                override = ":virtclass-multilib-" + p
+                localdata.setVar("OVERRIDES", localdata.getVar("OVERRIDES", False) + override)
+                bb.data.update_data(localdata)
+                newname = localdata.expand(v)
+                if newname != v:
+                    newval = localdata.getVar(v, True)
+                    d.setVar(newname, newval)
+            continue
         for p in prefixes:
             newname = "PREFERRED_VERSION_" + p + "-" + pkg
             if not d.getVar(newname, False):
@@ -203,14 +216,36 @@ def preferred_ml_updates(d):
         pkg = prov.replace("PREFERRED_PROVIDER_", "")
         if pkg.endswith("-native") or pkg.startswith("nativesdk-"):
             continue
+        if 'cross-canadian' in pkg:
+            for p in prefixes:
+                localdata = bb.data.createCopy(d)
+                override = ":virtclass-multilib-" + p
+                localdata.setVar("OVERRIDES", localdata.getVar("OVERRIDES", False) + override)
+                bb.data.update_data(localdata)
+                newname = localdata.expand(prov)
+                if newname != prov:
+                    newval = localdata.expand(val)
+                    d.setVar(newname, newval)
+            continue
         virt = ""
         if pkg.startswith("virtual/"):
             pkg = pkg.replace("virtual/", "")
             virt = "virtual/"
         for p in prefixes:
-            newname = "PREFERRED_PROVIDER_" + virt + p + "-" + pkg
             if pkg != "kernel":
                 val = p + "-" + val
+
+            # implement variable keys
+            localdata = bb.data.createCopy(d)
+            override = ":virtclass-multilib-" + p
+            localdata.setVar("OVERRIDES", localdata.getVar("OVERRIDES", False) + override)
+            bb.data.update_data(localdata)
+            newname = localdata.expand(prov)
+            if newname != prov:
+                d.setVar(newname, localdata.expand(val))
+
+            # implement alternative multilib name
+            newname = localdata.expand("PREFERRED_PROVIDER_" + virt + p + "-" + pkg)
             if not d.getVar(newname, False):
                 d.setVar(newname, val)
 
@@ -218,7 +253,7 @@ def preferred_ml_updates(d):
     mp = (d.getVar("MULTI_PROVIDER_WHITELIST", True) or "").split()
     extramp = []
     for p in mp:
-        if p.endswith("-native") or p.startswith("nativesdk-"):
+        if p.endswith("-native") or p.startswith("nativesdk-") or 'cross-canadian' in p:
             continue
         virt = ""
         if p.startswith("virtual/"):
meta/classes/multilib.bbclass +14 -4
index 76cbac6..0109d53 100644
@@ -16,6 +16,13 @@ python multilib_virtclass_handler () {
         e.data.setVar("PN", variant + "-" + e.data.getVar("PN", False))
         return
 
+    if bb.data.inherits_class('cross-canadian', e.data):
+        e.data.setVar("MLPREFIX", variant + "-")
+        override = ":virtclass-multilib-" + variant
+        e.data.setVar("OVERRIDES", e.data.getVar("OVERRIDES", False) + override)
+        bb.data.update_data(e.data)
+        return
+
     if bb.data.inherits_class('native', e.data):
         raise bb.parse.SkipPackage("We can't extend native recipes")
 
@@ -67,12 +74,16 @@ python __anonymous () {
     if bb.data.inherits_class('image', d) or bb.data.inherits_class('populate_sdk_base', d):
         return
 
+    clsextend.map_depends_variable("DEPENDS")
+    clsextend.map_variable("PROVIDES")
+
+    if bb.data.inherits_class('cross-canadian', d):
+        return
+
     clsextend.rename_packages()
     clsextend.rename_package_variables((d.getVar("PACKAGEVARS", True) or "").split())
 
-    clsextend.map_depends_variable("DEPENDS")
     clsextend.map_packagevars()
-    clsextend.map_variable("PROVIDES")
     clsextend.map_variable("PACKAGES_DYNAMIC")
     clsextend.map_variable("PACKAGE_INSTALL")
     clsextend.map_variable("INITSCRIPT_PACKAGES")
@@ -88,7 +99,7 @@ python do_package_qa_multilib() {
         for i in values.keys():
             if i.startswith('virtual/'):
                 i = i[len('virtual/'):]
-            if (not i.startswith('kernel-module')) and (not i.startswith(mlprefix)):
+            if (not i.startswith('kernel-module')) and (not i.startswith(mlprefix)) and (not 'cross-canadian' in i):
                 candidates.append(i)
         if len(candidates) > 0:
             bb.warn("Multilib QA Issue: %s package %s - suspicious values '%s' in %s" 
@@ -107,4 +118,3 @@ python do_package_qa_multilib() {
         check_mlprefix(pkg, 'RREPLACES', ml)
         check_mlprefix(pkg, 'RCONFLICTS', ml)
 }
-
meta/lib/oe/classextend.py +2 -2
index 86b1e8a..1be0df5 100644
@@ -34,7 +34,7 @@ class ClassExtender(object):
         return newdata
 
     def map_depends(self, dep):
-        if dep.endswith(("-native", "-native-runtime")):
+        if dep.endswith(("-native", "-native-runtime")) or ('nativesdk-' in dep) or ('-nativesdk') in dep or ('cross-canadian' in dep):
             return dep
         else:
             return self.extend_name(dep)
@@ -77,7 +77,7 @@ class ClassExtender(object):
 
 class NativesdkClassExtender(ClassExtender):
     def map_depends(self, dep):
-        if dep.endswith(("-native", "-native-runtime", "-cross")):
+        if dep.endswith(("-native", "-native-runtime", "-cross")) or ('nativesdk-' in dep) or ('-nativesdk') in dep:
             return dep
         elif dep.endswith(("-gcc-intermediate", "-gcc-initial", "-gcc", "-g++")):
             return dep + "-crosssdk"
Signature · 1 line
1.7.3.4
author
#2 lore raw

PATCH 2/12 multilib - crosssdk: Stop building multilib for crosssdk packages

In reply to Mark Hatle

Crosssdk packages are not actually multilib packages, so treat them the same as other nativesdk packages in the multilib, base, and classextend components.

---
 meta/classes/base.bbclass     |    8 ++++----
 meta/classes/multilib.bbclass |    4 ++--
 meta/lib/oe/classextend.py    |    4 ++--
 3 files changed, 8 insertions(+), 8 deletions(-)
meta/classes/base.bbclass +4 -4
index 43a352d..2fc04da 100644
@@ -183,7 +183,7 @@ def preferred_ml_updates(d):
             providers.append(v)
 
     for pkg, reason in blacklists.items():
-        if pkg.endswith("-native") or pkg.startswith("nativesdk-") or 'cross-canadian' in pkg:
+        if pkg.endswith(("-native", "-crosssdk")) or pkg.startswith("nativesdk-") or 'cross-canadian' in pkg:
             continue
         for p in prefixes:
             newpkg = p + "-" + pkg
@@ -193,7 +193,7 @@ def preferred_ml_updates(d):
     for v in versions:
         val = d.getVar(v, False)
         pkg = v.replace("PREFERRED_VERSION_", "")
-        if pkg.endswith("-native") or pkg.startswith("nativesdk-"):
+        if pkg.endswith(("-native", "-crosssdk")) or pkg.startswith("nativesdk-"):
             continue
         if 'cross-canadian' in pkg:
             for p in prefixes:
@@ -214,7 +214,7 @@ def preferred_ml_updates(d):
     for prov in providers:
         val = d.getVar(prov, False)
         pkg = prov.replace("PREFERRED_PROVIDER_", "")
-        if pkg.endswith("-native") or pkg.startswith("nativesdk-"):
+        if pkg.endswith(("-native", "-crosssdk")) or pkg.startswith("nativesdk-"):
             continue
         if 'cross-canadian' in pkg:
             for p in prefixes:
@@ -253,7 +253,7 @@ def preferred_ml_updates(d):
     mp = (d.getVar("MULTI_PROVIDER_WHITELIST", True) or "").split()
     extramp = []
     for p in mp:
-        if p.endswith("-native") or p.startswith("nativesdk-") or 'cross-canadian' in p:
+        if p.endswith(("-native", "-crosssdk")) or p.startswith("nativesdk-") or 'cross-canadian' in p:
             continue
         virt = ""
         if p.startswith("virtual/"):
meta/classes/multilib.bbclass +2 -2
index 0109d53..fb03fca 100644
@@ -26,7 +26,7 @@ python multilib_virtclass_handler () {
     if bb.data.inherits_class('native', e.data):
         raise bb.parse.SkipPackage("We can't extend native recipes")
 
-    if bb.data.inherits_class('nativesdk', e.data):
+    if bb.data.inherits_class('nativesdk', e.data) or bb.data.inherits_class('crosssdk', e.data):
         raise bb.parse.SkipPackage("We can't extend nativesdk recipes")
 
     save_var_name=e.data.getVar("MULTILIB_SAVE_VARNAME", True) or ""
@@ -99,7 +99,7 @@ python do_package_qa_multilib() {
         for i in values.keys():
             if i.startswith('virtual/'):
                 i = i[len('virtual/'):]
-            if (not i.startswith('kernel-module')) and (not i.startswith(mlprefix)) and (not 'cross-canadian' in i):
+            if (not i.startswith('kernel-module')) and (not i.startswith(mlprefix)) and (not 'cross-canadian' in i) and (not i.startswith("nativesdk-")):
                 candidates.append(i)
         if len(candidates) > 0:
             bb.warn("Multilib QA Issue: %s package %s - suspicious values '%s' in %s" 
meta/lib/oe/classextend.py +2 -2
index 1be0df5..64d3aa0 100644
@@ -34,7 +34,7 @@ class ClassExtender(object):
         return newdata
 
     def map_depends(self, dep):
-        if dep.endswith(("-native", "-native-runtime")) or ('nativesdk-' in dep) or ('-nativesdk') in dep or ('cross-canadian' in dep):
+        if dep.endswith(("-native", "-native-runtime", "-crosssdk")) or ('nativesdk-' in dep) or ('-nativesdk') in dep or ('cross-canadian' in dep):
             return dep
         else:
             return self.extend_name(dep)
@@ -77,7 +77,7 @@ class ClassExtender(object):
 
 class NativesdkClassExtender(ClassExtender):
     def map_depends(self, dep):
-        if dep.endswith(("-native", "-native-runtime", "-cross")) or ('nativesdk-' in dep) or ('-nativesdk') in dep:
+        if dep.endswith(("-native", "-native-runtime", "-cross", "-crosssdk")) or ('nativesdk-' in dep) or ('-nativesdk') in dep:
             return dep
         elif dep.endswith(("-gcc-intermediate", "-gcc-initial", "-gcc", "-g++")):
             return dep + "-crosssdk"
Signature · 1 line
1.7.3.4
author
#3 lore raw

PATCH 3/12 populate_sdk_base: Update extraction script for multilibs

In reply to Mark Hatle

When multilibs are enabled, there will be more then one environment file created. We need to be sure to process each environment file. The next function can simply use the last environment file processed to get the magic value(s) that it requires.

---
 meta/classes/populate_sdk_base.bbclass |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)
meta/classes/populate_sdk_base.bbclass +3 -2
index 6eb6726..1bc1438 100644
@@ -158,8 +158,9 @@ echo "done"
 
 printf "Setting it up..."
 # fix environment paths
-env_setup_script=$(find $target_sdk_dir/ -name "environment-setup-${REAL_MULTIMACH_TARGET_SYS}")
-sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+for env_setup_script in `find $target_sdk_dir/ -name "environment-setup-*"` ; do
+  sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+done
 
 # fix dynamic loader paths in all ELF SDK binaries
 native_sysroot=$(cat $env_setup_script |grep OECORE_NATIVE_SYSROOT|cut -d'=' -f2|tr -d '"')
Signature · 1 line
1.7.3.4
#4 lore raw

Re: [PATCH 03/12] populate_sdk_base: Update extraction script for multilibs

Quoted 16 lines · Mark Hatle

When multilibs are enabled, there will be more then one environment file created. We need to be sure to process each environment file. The next function can simply use the last environment file processed to get the magic value(s) that it requires.

---
 meta/classes/populate_sdk_base.bbclass |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)
meta/classes/populate_sdk_base.bbclass +0 -0
index 6eb6726..1bc1438 100644
@@ -158,8 +158,9 @@ echo "done"
 

meta/classes/populate_sdk_base.bbclass echo "done" · line 166

Quoted 10 lines
meta/classes/populate_sdk_base.bbclass +3 -2
 printf "Setting it up..."
 # fix environment paths
-env_setup_script=$(find $target_sdk_dir/ -name "environment-setup-${REAL_MULTIMACH_TARGET_SYS}")
-sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+for env_setup_script in `find $target_sdk_dir/ -name "environment-setup-*"` ; do
+  sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+done
 
 # fix dynamic loader paths in all ELF SDK binaries
 native_sysroot=$(cat $env_setup_script |grep OECORE_NATIVE_SYSROOT|cut -d'=' -f2|tr -d '"')

This is on course to conflict with

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=1b6019086c4242c550b4e0551c7b5d206a0d52e1

Can you please talk with Laurentiu and come up with a solution that works for everyone.

Cheers,

Richard

author
#5 lore raw

Re: [PATCH 03/12] populate_sdk_base: Update extraction script for multilibs

Quoted 30 lines · Richard Purdie
Quoted 16 lines · Mark Hatle

When multilibs are enabled, there will be more then one environment file created. We need to be sure to process each environment file. The next function can simply use the last environment file processed to get the magic value(s) that it requires.

---
  meta/classes/populate_sdk_base.bbclass |    5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)
meta/classes/populate_sdk_base.bbclass +0 -0
index 6eb6726..1bc1438 100644
@@ -158,8 +158,9 @@ echo "done"
 

meta/classes/populate_sdk_base.bbclass echo "done" · line 166

Quoted 10 lines
meta/classes/populate_sdk_base.bbclass +3 -2
  printf "Setting it up..."
  # fix environment paths
-env_setup_script=$(find $target_sdk_dir/ -name "environment-setup-${REAL_MULTIMACH_TARGET_SYS}")
-sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+for env_setup_script in `find $target_sdk_dir/ -name "environment-setup-*"` ; do
+  sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+done
 
  # fix dynamic loader paths in all ELF SDK binaries
  native_sysroot=$(cat $env_setup_script |grep OECORE_NATIVE_SYSROOT|cut -d'=' -f2|tr -d '"')

This is on course to conflict with

Quoted 4 lines

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=1b6019086c4242c550b4e0551c7b5d206a0d52e1

Can you please talk with Laurentiu and come up with a solution that works for everyone.

I think this is a better fix for the problem. The other change limits the environment file to a single file. However, there is nothing in the function -- other then the use of 'env_setuo_script' -- that wants or needs a single file loaded.

So it's better to find htem all and just iterate over them all. Also the order of processing, and the last item processed does not matter. The later chunks of functionality just look for a static value that is supposed to be the same in all of the environment files.

--Mark

Quoted 4 lines

Cheers,

Richard

#6 lore raw

Re: [PATCH 03/12] populate_sdk_base: Update extraction script for multilibs

Quoted 18 lines · Mark Hatle
Quoted 17 lines · Richard Purdie
Quoted 16 lines · Mark Hatle

When multilibs are enabled, there will be more then one environment file created. We need to be sure to process each environment file. The next function can simply use the last environment file processed to get the magic value(s) that it requires.

---
  meta/classes/populate_sdk_base.bbclass |    5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)
meta/classes/populate_sdk_base.bbclass +0 -0
index 6eb6726..1bc1438 100644
@@ -158,8 +158,9 @@ echo "done"
 
Quoted 3 lines
Quoted 3 lines

meta/classes/populate_sdk_base.bbclass echo "done" · line 161

Quoted 3 lines
meta/classes/populate_sdk_base.bbclass +0 -1
  printf "Setting it up..."
  # fix environment paths
-env_setup_script=$(find $target_sdk_dir/ -name "environment-setup-${REAL_MULTIMACH_TARGET_SYS}")

The reason to replace globbing with ${REAL_MULTIMACH_TARGET_SYS} was to match only the file we're interested in and avoid matching other files whose names start with environment-setup. However, since all the environment-setup directories reside in $target_sdk_dir/ we would probably be safer, and faster, by doing a simple ls and avoid searching all the tree.

env_setup_script=$(ls $target_sdk_dir/environment-setup-*)

Quoted 4 lines
Quoted 4 lines

meta/classes/populate_sdk_base.bbclass echo "done" · line 163

Quoted 4 lines
meta/classes/populate_sdk_base.bbclass +3 -1
-sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+for env_setup_script in `find $target_sdk_dir/ -name "environment-setup-*"` ; do
+  sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+done

We could probably do this without the for loop since sed accepts multiple input files.

Quoted 28 lines
Quoted 10 lines

meta/classes/populate_sdk_base.bbclass echo "done" · line 166

Quoted 3 lines
meta/classes/populate_sdk_base.bbclass +0 -0
  # fix dynamic loader paths in all ELF SDK binaries
  native_sysroot=$(cat $env_setup_script |grep OECORE_NATIVE_SYSROOT|cut -d'=' -f2|tr -d '"')

This is on course to conflict with

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=1b6019086c4242c550b4e0551c7b5d206a0d52e1

Can you please talk with Laurentiu and come up with a solution that works for everyone.

I think this is a better fix for the problem. The other change limits the environment file to a single file. However, there is nothing in the function -- other then the use of 'env_setuo_script' -- that wants or needs a single file loaded.

So it's better to find htem all and just iterate over them all. Also the order of processing, and the last item processed does not matter. The later chunks of functionality just look for a static value that is supposed to be the same in all of the environment files.

--Mark

Quoted 4 lines

Cheers,

Richard

author
#7 lore raw

Re: [PATCH 03/12] populate_sdk_base: Update extraction script for multilibs

Quoted 31 lines · Laurentiu Palcu
Quoted 18 lines · Mark Hatle
Quoted 17 lines · Richard Purdie
Quoted 16 lines · Mark Hatle

When multilibs are enabled, there will be more then one environment file created. We need to be sure to process each environment file. The next function can simply use the last environment file processed to get the magic value(s) that it requires.

---
   meta/classes/populate_sdk_base.bbclass |    5 +++--
   1 files changed, 3 insertions(+), 2 deletions(-)
meta/classes/populate_sdk_base.bbclass +0 -0
index 6eb6726..1bc1438 100644
@@ -158,8 +158,9 @@ echo "done"
 
Quoted 3 lines
Quoted 3 lines

meta/classes/populate_sdk_base.bbclass echo "done" · line 161

Quoted 3 lines
meta/classes/populate_sdk_base.bbclass +0 -1
   printf "Setting it up..."
   # fix environment paths
-env_setup_script=$(find $target_sdk_dir/ -name "environment-setup-${REAL_MULTIMACH_TARGET_SYS}")

The reason to replace globbing with ${REAL_MULTIMACH_TARGET_SYS} was to match only the file we're interested in and avoid matching other files whose names start with environment-setup. However, since all the environment-setup directories reside in $target_sdk_dir/ we would probably be safer, and faster, by doing a simple ls and avoid searching all the tree.

Quoted 1 line

env_setup_script=$(ls $target_sdk_dir/environment-setup-*)

That seems reasonable to me, and a lot faster then the find.

Quoted 6 lines
Quoted 4 lines
Quoted 4 lines

meta/classes/populate_sdk_base.bbclass echo "done" · line 163

Quoted 4 lines
meta/classes/populate_sdk_base.bbclass +3 -1
-sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+for env_setup_script in `find $target_sdk_dir/ -name "environment-setup-*"` ; do
+  sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+done

We could probably do this without the for loop since sed accepts multiple input files.

The end result of the env_setups_script though, needs to only contain -one- environment script (doesn't matter which one). That's why I used the loop.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

meta/classes/populate_sdk_base.bbclass echo "done" · line 166

Quoted 3 lines
meta/classes/populate_sdk_base.bbclass +0 -0
   # fix dynamic loader paths in all ELF SDK binaries
   native_sysroot=$(cat $env_setup_script |grep OECORE_NATIVE_SYSROOT|cut -d'=' -f2|tr -d '"')

The code snippet above is the reason why it can only contain the one item.. but it happens that OECORE_NATIVE_SYSROOT is the same in all environment files (and has to be...) So it's safe to just use the last processed item.

A for loop shouldn't be any slower then calling the set w/ multilib inputs and then later adjusting the variable to only be one item. (We're talking at a maximum 3 environment scripts.. thus why it shouldn't be an issue.)

Quoted 25 lines
Quoted 25 lines
Quoted 7 lines

This is on course to conflict with

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=1b6019086c4242c550b4e0551c7b5d206a0d52e1

Can you please talk with Laurentiu and come up with a solution that works for everyone.

I think this is a better fix for the problem. The other change limits the environment file to a single file. However, there is nothing in the function -- other then the use of 'env_setuo_script' -- that wants or needs a single file loaded.

So it's better to find htem all and just iterate over them all. Also the order of processing, and the last item processed does not matter. The later chunks of functionality just look for a static value that is supposed to be the same in all of the environment files.

--Mark

Quoted 4 lines

Cheers,

Richard

#8 lore raw

Re: [PATCH 03/12] populate_sdk_base: Update extraction script for multilibs

Quoted 32 lines · Mark Hatle
Quoted 31 lines · Laurentiu Palcu
Quoted 18 lines · Mark Hatle
Quoted 17 lines · Richard Purdie
Quoted 16 lines · Mark Hatle

When multilibs are enabled, there will be more then one environment file created. We need to be sure to process each environment file. The next function can simply use the last environment file processed to get the magic value(s) that it requires.

---
   meta/classes/populate_sdk_base.bbclass |    5 +++--
   1 files changed, 3 insertions(+), 2 deletions(-)
meta/classes/populate_sdk_base.bbclass +0 -0
index 6eb6726..1bc1438 100644
@@ -158,8 +158,9 @@ echo "done"
 
Quoted 3 lines
Quoted 3 lines

meta/classes/populate_sdk_base.bbclass echo "done" · line 161

Quoted 3 lines
meta/classes/populate_sdk_base.bbclass +0 -1
   printf "Setting it up..."
   # fix environment paths
-env_setup_script=$(find $target_sdk_dir/ -name "environment-setup-${REAL_MULTIMACH_TARGET_SYS}")

The reason to replace globbing with ${REAL_MULTIMACH_TARGET_SYS} was to match only the file we're interested in and avoid matching other files whose names start with environment-setup. However, since all the environment-setup directories reside in $target_sdk_dir/ we would probably be safer, and faster, by doing a simple ls and avoid searching all the tree.

Quoted 3 lines
Quoted 1 line

env_setup_script=$(ls $target_sdk_dir/environment-setup-*)

That seems reasonable to me, and a lot faster then the find.

Also, we should replace 'find' with 'ls' in adt_installer_internal script (line 212) too. So that both the toolchain tarball installer and adt-installer scripts are in sync.

Quoted 15 lines
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines

meta/classes/populate_sdk_base.bbclass echo "done" · line 163

Quoted 5 lines
meta/classes/populate_sdk_base.bbclass +3 -1
-sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+for env_setup_script in `find $target_sdk_dir/ -name "environment-setup-*"` ; do
+  sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
+done

We could probably do this without the for loop since sed accepts multiple input files.

The end result of the env_setups_script though, needs to only contain -one- environment script (doesn't matter which one). That's why I used the loop.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

meta/classes/populate_sdk_base.bbclass echo "done" · line 166

Quoted 3 lines
meta/classes/populate_sdk_base.bbclass +0 -0
   # fix dynamic loader paths in all ELF SDK binaries
   native_sysroot=$(cat $env_setup_script |grep OECORE_NATIVE_SYSROOT|cut -d'=' -f2|tr -d '"')
Quoted 7 lines

The code snippet above is the reason why it can only contain the one item.. but it happens that OECORE_NATIVE_SYSROOT is the same in all environment files (and has to be...) So it's safe to just use the last processed item.

A for loop shouldn't be any slower then calling the set w/ multilib inputs and then later adjusting the variable to only be one item. (We're talking at a maximum 3 environment scripts.. thus why it shouldn't be an issue.)

Agreed! I'm OK with the solution.

Thanks,
Laurentiu

Quoted 27 lines
Quoted 26 lines
Quoted 26 lines
Quoted 8 lines

This is on course to conflict with

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=1b6019086c4242c550b4e0551c7b5d206a0d52e1

Can you please talk with Laurentiu and come up with a solution that works for everyone.

I think this is a better fix for the problem. The other change limits the environment file to a single file. However, there is nothing in the function -- other then the use of 'env_setuo_script' -- that wants or needs a single file loaded.

So it's better to find htem all and just iterate over them all. Also the order of processing, and the last item processed does not matter. The later chunks of functionality just look for a static value that is supposed to be the same in all of the environment files.

--Mark

Quoted 4 lines

Cheers,

Richard

author
#9 lore raw

PATCH 4/12 bb.utils.explode_dep_versions: Update to ensure we avoid duplicate deps

In reply to Mark Hatle

Due to a recent change in bb.utils.explode_dep_version, we need to make sure that we do not have any duplicates in things that use explode_dep_versions.

---
 meta/classes/insane.bbclass      |   55 +++++++++++++++++++++++++++++++++-
 meta/classes/kernel.bbclass      |   20 +++++++-----
 meta/classes/libc-common.bbclass |   13 ++++++--
 meta/classes/package.bbclass     |   20 ++++++++++--
 meta/classes/package_rpm.bbclass |   61 +++++++++++++++++++++-----------------
 5 files changed, 125 insertions(+), 44 deletions(-)
meta/classes/insane.bbclass +53 -2
index 1fb8970..17d887a 100644
@@ -114,7 +114,7 @@ def package_qa_get_machine_dict():
 
 # Currently not being used by default "desktop"
 WARN_QA ?= "ldflags useless-rpaths rpaths unsafe-references-in-binaries unsafe-references-in-scripts staticdev libdir"
-ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch la2 pkgconfig la perms"
+ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch la2 pkgconfig la perms dep-cmp"
 
 ALL_QA = "${WARN_QA} ${ERROR_QA}"
 
@@ -659,6 +659,54 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, d):
 
     return sane
 
+def package_qa_check_deps(pkg, pkgdest, skip, d):
+    sane = True
+
+    # Copied from package_ipk.bbclass
+    # boiler plate to update the data
+    localdata = bb.data.createCopy(d)
+    root = "%s/%s" % (pkgdest, pkg)
+
+    localdata.setVar('ROOT', '') 
+    localdata.setVar('ROOT_%s' % pkg, root)
+    pkgname = localdata.getVar('PKG_%s' % pkg, True)
+    if not pkgname:
+        pkgname = pkg
+    localdata.setVar('PKG', pkgname)
+
+    localdata.setVar('OVERRIDES', pkg)
+
+    bb.data.update_data(localdata)
+
+    def check_valid_deps(var):
+        sane = True
+        try:
+            rvar = bb.utils.explode_dep_versions(localdata.getVar(var, True) or "")
+        except ValueError as e:
+            bb.fatal("%s_%s: %s" % (var, pkgname, e))
+            raise e
+        for dep in rvar:
+            if rvar[dep] and not rvar[dep].startswith(('< ', '= ', '> ', '<= ', '>=')):
+                error_msg = "%s_%s is invalid: %s (%s)   only comparisons <, =, >, <=, and >= are allowed" % (var, pkgname, dep, rvar[dep])
+                sane = package_qa_handle_error("dep-cmp", error_msg, d)
+        return sane
+
+    sane = True
+    if not check_valid_deps('RDEPENDS'):
+        sane = False
+    if not check_valid_deps('RRECOMMENDS'):
+        sane = False
+    if not check_valid_deps('RSUGGESTS'):
+        sane = False
+    if not check_valid_deps('RPROVIDES'):
+        sane = False
+    if not check_valid_deps('RREPLACES'):
+        sane = False
+    if not check_valid_deps('RCONFLICTS'):
+        sane = False
+
+    return sane
+
 # The PACKAGE FUNC to scan each package
 python do_package_qa () {
     import subprocess
@@ -699,6 +747,7 @@ python do_package_qa () {
     g = globals()
     walk_sane = True
     rdepends_sane = True
+    deps_sane = True
     for package in packages.split():
         skip = (d.getVar('INSANE_SKIP_' + package, True) or "").split()
         if skip:
@@ -722,12 +771,14 @@ python do_package_qa () {
             walk_sane  = False
         if not package_qa_check_rdepends(package, pkgdest, skip, d):
             rdepends_sane = False
+        if not package_qa_check_deps(package, pkgdest, skip, d):
+            deps_sane = False
 
 
     if 'libdir' in d.getVar("ALL_QA", True).split():
         package_qa_check_libdir(d)
 
-    if not walk_sane or not rdepends_sane:
+    if not walk_sane or not rdepends_sane or not deps_sane:
         bb.fatal("QA run found fatal errors. Please consider fixing them.")
     bb.note("DONE with PACKAGE QA")
 }
meta/classes/kernel.bbclass +12 -8
index fdef1be..878fd6c 100644
@@ -20,6 +20,13 @@ python __anonymous () {
     image = d.getVar('INITRAMFS_IMAGE', True)
     if image:
         d.setVar('INITRAMFS_TASK', '${INITRAMFS_IMAGE}:do_rootfs')
+
+    # RPROVIDES_kernel-base += "kernel-${KERNEL_VERSION}"
+    rprovides = bb.utils.explode_dep_versions(d.getVar("RPROVIDES_kernel-base", True) or "")
+    dep = d.expand("kernel-${KERNEL_VERSION}")
+    if not dep in rprovides:
+        rprovides[dep] = ""
+    d.setVar("RPROVIDES_kernel-base", bb.utils.join_deps(rprovides, commasep=False))
 }
 
 inherit kernel-arch deploy
@@ -269,7 +276,6 @@ RDEPENDS_kernel = "kernel-base"
 RDEPENDS_kernel-base ?= "kernel-image"
 PKG_kernel-image = "kernel-image-${@legitimize_package_name('${KERNEL_VERSION}')}"
 PKG_kernel-base = "kernel-${@legitimize_package_name('${KERNEL_VERSION}')}"
-RPROVIDES_kernel-base += "kernel-${KERNEL_VERSION}"
 ALLOW_EMPTY_kernel = "1"
 ALLOW_EMPTY_kernel-base = "1"
 ALLOW_EMPTY_kernel-image = "1"
@@ -429,13 +435,11 @@ python populate_packages_prepend () {
             old_desc = d.getVar('DESCRIPTION_' + pkg, True) or ""
             d.setVar('DESCRIPTION_' + pkg, old_desc + "; " + vals["description"])
 
-        rdepends_str = d.getVar('RDEPENDS_' + pkg, True)
-        if rdepends_str:
-            rdepends = rdepends_str.split()
-        else:
-            rdepends = []
-        rdepends.extend(get_dependencies(file, pattern, format))
-        d.setVar('RDEPENDS_' + pkg, ' '.join(rdepends))
+        rdepends = bb.utils.explode_dep_versions(d.getVar('RDEPENDS_' + pkg, True) or "")
+        for dep in get_dependencies(file, pattern, format):
+            if not dep in rdepends:
+                rdepends[dep] = ""
+        d.setVar('RDEPENDS_' + pkg, bb.utils.join_deps(rdepends, commasep=False))
 
     module_deps = parse_depmod()
     module_regex = '^(.*)\.k?o$'
meta/classes/libc-common.bbclass +10 -3
index 0f49936..dc32c81 100644
@@ -29,7 +29,14 @@ python populate_packages_prepend () {
         d.setVar('PKG_'+bpn+'-dev', 'libc6-dev')
         d.setVar('PKG_'+bpn+'-dbg', 'libc6-dbg')
         # For backward compatibility with old -dbg package
-        d.appendVar('RPROVIDES_' + bpn + '-dbg', ' libc-dbg')
-        d.appendVar('RCONFLICTS_' + bpn + '-dbg', ' libc-dbg')
-        d.appendVar('RREPLACES_' + bpn + '-dbg', ' libc-dbg')
+
+        def add_dep(var, dep):
+            deps = bb.utils.explode_dep_versions(d.getVar(var + '_' + bpn, True) or "")
+            if not dep in deps:
+                deps[dep] = ""
+                d.setVar(var + '_' + bpn, bb.utils.join_deps(deps, commasep=False))
+
+        add_dep('RPROVIDES', 'libc-dbg')
+        add_dep('RCONFLICTS', 'libc-dbg')
+        add_dep('RREPLACES', 'libc-dbg')
 }
meta/classes/package.bbclass +16 -4
index c8aafc9..6b28a15 100644
@@ -1637,14 +1637,19 @@ def read_libdep_files(d):
     pkglibdeps = {}
     packages = d.getVar('PACKAGES', True).split()
     for pkg in packages:
-        pkglibdeps[pkg] = []
+        pkglibdeps[pkg] = {}
         for extension in ".shlibdeps", ".pcdeps", ".clilibdeps":
             depsfile = d.expand("${PKGDEST}/" + pkg + extension)
             if os.access(depsfile, os.R_OK):
                 fd = file(depsfile)
                 lines = fd.readlines()
                 fd.close()
-                pkglibdeps[pkg].extend([l.rstrip() for l in lines])
+                for l in lines:
+                    l.rstrip()
+                    deps = bb.utils.explode_dep_versions(l)
+                    for dep in deps:
+                        if not dep in pkglibdeps[pkg]:
+                            pkglibdeps[pkg][dep] = deps[dep]
     return pkglibdeps
 
 python read_shlibdeps () {
@@ -1654,7 +1659,10 @@ python read_shlibdeps () {
     for pkg in packages:
         rdepends = bb.utils.explode_dep_versions(d.getVar('RDEPENDS_' + pkg, False) or d.getVar('RDEPENDS', False) or "")
         for dep in pkglibdeps[pkg]:
-            rdepends[dep] = ""
+            # Add the dep if it's not already there, or if no comparison is set
+            if not dep in rdepends or not rdepends[dep]:
+                rdepends[dep] = pkglibdeps[pkg][dep]
+
         d.setVar('RDEPENDS_' + pkg, bb.utils.join_deps(rdepends, commasep=False))
 }
 
@@ -1761,7 +1769,11 @@ python package_depchains() {
         pkglibdeplist = []
         for pkg in pkglibdeps:
             for dep in pkglibdeps[pkg]:
-                add_dep(pkglibdeplist, dep)
+                cmp = pkglibdeps[pkg][dep]
+                if cmp:
+                    add_dep(pkglibdeplist, dep)
+                else:
+                    add_dep(pkglibdeplist, "%s (%s)" % (dep, cmp))
         # FIXME this should not look at PN once all task recipes inherit from task.bbclass
         dbgdefaultdeps = ((d.getVar('DEPCHAIN_DBGDEFAULTDEPS', True) == '1') or (d.getVar('PN', True) or '').startswith('packagegroup-'))
 
meta/classes/package_rpm.bbclass +34 -27

91 lines not on this page · raw message

Signature · 1 line
1.7.3.4
#10 lore raw

Re: [PATCH 04/12] bb.utils.explode_dep_versions: Update to ensure we avoid duplicate deps

Quoted 30 lines · Mark Hatle

Due to a recent change in bb.utils.explode_dep_version, we need to make sure that we do not have any duplicates in things that use explode_dep_versions.

---
 meta/classes/insane.bbclass      |   55 +++++++++++++++++++++++++++++++++-
 meta/classes/kernel.bbclass      |   20 +++++++-----
 meta/classes/libc-common.bbclass |   13 ++++++--
 meta/classes/package.bbclass     |   20 ++++++++++--
 meta/classes/package_rpm.bbclass |   61 +++++++++++++++++++++-----------------
 5 files changed, 125 insertions(+), 44 deletions(-)
meta/classes/insane.bbclass +1 -1
index 1fb8970..17d887a 100644
@@ -114,7 +114,7 @@ def package_qa_get_machine_dict():
 
 # Currently not being used by default "desktop"
 WARN_QA ?= "ldflags useless-rpaths rpaths unsafe-references-in-binaries unsafe-references-in-scripts staticdev libdir"
-ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch la2 pkgconfig la perms"
+ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch la2 pkgconfig la perms dep-cmp"
 
 ALL_QA = "${WARN_QA} ${ERROR_QA}"
 
@@ -659,6 +659,54 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, d):
 
     return sane
 

meta/classes/insane.bbclass def package_qa_check_rdepends(pkg, pkgdest, skip, d): · line 671

Quoted 10 lines
meta/classes/insane.bbclass +10 -0
+def package_qa_check_deps(pkg, pkgdest, skip, d):
+    sane = True
+
+    # Copied from package_ipk.bbclass
+    # boiler plate to update the data
+    localdata = bb.data.createCopy(d)
+    root = "%s/%s" % (pkgdest, pkg)
+
+    localdata.setVar('ROOT', '') 
+    localdata.setVar('ROOT_%s' % pkg, root)

Nothing here uses ROOT or root so these can be removed?

meta/classes/insane.bbclass def package_qa_check_rdepends(pkg, pkgdest, skip, d): · line 675

Quoted 4 lines
meta/classes/insane.bbclass +4 -0
+    pkgname = localdata.getVar('PKG_%s' % pkg, True)
+    if not pkgname:
+        pkgname = pkg
+    localdata.setVar('PKG', pkgname)

Again, these look surplus to requirements?

meta/classes/insane.bbclass def package_qa_check_rdepends(pkg, pkgdest, skip, d): · line 680

Quoted 4 lines
meta/classes/insane.bbclass +4 -0
+    localdata.setVar('OVERRIDES', pkg)
+
+    bb.data.update_data(localdata)
+

Cheers,

Richard

author
#11 lore raw

Re: [PATCH 04/12] bb.utils.explode_dep_versions: Update to ensure we avoid duplicate deps

Quoted 50 lines · Richard Purdie
Quoted 30 lines · Mark Hatle

Due to a recent change in bb.utils.explode_dep_version, we need to make sure that we do not have any duplicates in things that use explode_dep_versions.

---
  meta/classes/insane.bbclass      |   55 +++++++++++++++++++++++++++++++++-
  meta/classes/kernel.bbclass      |   20 +++++++-----
  meta/classes/libc-common.bbclass |   13 ++++++--
  meta/classes/package.bbclass     |   20 ++++++++++--
  meta/classes/package_rpm.bbclass |   61 +++++++++++++++++++++-----------------
  5 files changed, 125 insertions(+), 44 deletions(-)
meta/classes/insane.bbclass +1 -1
index 1fb8970..17d887a 100644
@@ -114,7 +114,7 @@ def package_qa_get_machine_dict():
 
  # Currently not being used by default "desktop"
  WARN_QA ?= "ldflags useless-rpaths rpaths unsafe-references-in-binaries unsafe-references-in-scripts staticdev libdir"
-ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch la2 pkgconfig la perms"
+ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch la2 pkgconfig la perms dep-cmp"
 
  ALL_QA = "${WARN_QA} ${ERROR_QA}"
 
@@ -659,6 +659,54 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, d):
 
      return sane
 

meta/classes/insane.bbclass def package_qa_check_rdepends(pkg, pkgdest, skip, d): · line 671

Quoted 10 lines
meta/classes/insane.bbclass +10 -0
+def package_qa_check_deps(pkg, pkgdest, skip, d):
+    sane = True
+
+    # Copied from package_ipk.bbclass
+    # boiler plate to update the data
+    localdata = bb.data.createCopy(d)
+    root = "%s/%s" % (pkgdest, pkg)
+
+    localdata.setVar('ROOT', '')
+    localdata.setVar('ROOT_%s' % pkg, root)

Nothing here uses ROOT or root so these can be removed?

meta/classes/insane.bbclass def package_qa_check_rdepends(pkg, pkgdest, skip, d): · line 675

Quoted 4 lines
meta/classes/insane.bbclass +4 -0
+    pkgname = localdata.getVar('PKG_%s' % pkg, True)
+    if not pkgname:
+        pkgname = pkg
+    localdata.setVar('PKG', pkgname)
Quoted 6 lines

Again, these look surplus to requirements?

meta/classes/insane.bbclass def package_qa_check_rdepends(pkg, pkgdest, skip, d): · line 680

Quoted 4 lines
meta/classes/insane.bbclass +4 -0
+    localdata.setVar('OVERRIDES', pkg)
+
+    bb.data.update_data(localdata)
+

I'll get this fixed and resubmitted.

--Mark

Quoted 4 lines

Cheers,

Richard

#12 lore raw

Re: [PATCH 04/12] bb.utils.explode_dep_versions: Update to ensure we avoid duplicate deps

Quoted 51 lines · Mark Hatle
Quoted 50 lines · Richard Purdie
Quoted 30 lines · Mark Hatle

Due to a recent change in bb.utils.explode_dep_version, we need to make sure that we do not have any duplicates in things that use explode_dep_versions.

---
  meta/classes/insane.bbclass      |   55 +++++++++++++++++++++++++++++++++-
  meta/classes/kernel.bbclass      |   20 +++++++-----
  meta/classes/libc-common.bbclass |   13 ++++++--
  meta/classes/package.bbclass     |   20 ++++++++++--
  meta/classes/package_rpm.bbclass |   61 +++++++++++++++++++++-----------------
  5 files changed, 125 insertions(+), 44 deletions(-)
meta/classes/insane.bbclass +1 -1
index 1fb8970..17d887a 100644
@@ -114,7 +114,7 @@ def package_qa_get_machine_dict():
 
  # Currently not being used by default "desktop"
  WARN_QA ?= "ldflags useless-rpaths rpaths unsafe-references-in-binaries unsafe-references-in-scripts staticdev libdir"
-ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch la2 pkgconfig la perms"
+ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch la2 pkgconfig la perms dep-cmp"
 
  ALL_QA = "${WARN_QA} ${ERROR_QA}"
 
@@ -659,6 +659,54 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, d):
 
      return sane
 

meta/classes/insane.bbclass def package_qa_check_rdepends(pkg, pkgdest, skip, d): · line 671

Quoted 10 lines
meta/classes/insane.bbclass +10 -0
+def package_qa_check_deps(pkg, pkgdest, skip, d):
+    sane = True
+
+    # Copied from package_ipk.bbclass
+    # boiler plate to update the data
+    localdata = bb.data.createCopy(d)
+    root = "%s/%s" % (pkgdest, pkg)
+
+    localdata.setVar('ROOT', '')
+    localdata.setVar('ROOT_%s' % pkg, root)

Nothing here uses ROOT or root so these can be removed?

Quoted 4 lines
meta/classes/insane.bbclass +4 -0
+    pkgname = localdata.getVar('PKG_%s' % pkg, True)
+    if not pkgname:
+        pkgname = pkg
+    localdata.setVar('PKG', pkgname)
Quoted 8 lines
Quoted 6 lines

Again, these look surplus to requirements?

meta/classes/insane.bbclass def package_qa_check_rdepends(pkg, pkgdest, skip, d): · line 680

Quoted 4 lines
meta/classes/insane.bbclass +4 -0
+    localdata.setVar('OVERRIDES', pkg)
+
+    bb.data.update_data(localdata)
+

I'll get this fixed and resubmitted.

I'm trying to find you on irc as we need to talk a bit more about things.

With the above, I realised you copy and pasted some other code so I have a cleanup patch for both now.

I don't think the ValueError is going to cut it, particularly this close to release. I'm working on replacing the function usage with an improved one which can cope with the duplicates.

Cheers,

Richard

author
#13 lore raw

PATCH 5/12 package_deb/ipk: Remap < and > to << and >>

In reply to Mark Hatle

In deb and ipk, < means <=, while > means >=... there is a different operator << and >> that means < and >, so we map them when constructing the packages.

---
 meta/classes/package_deb.bbclass |   18 ++++++++++++++++++
 meta/classes/package_ipk.bbclass |   18 ++++++++++++++++++
 2 files changed, 36 insertions(+), 0 deletions(-)
meta/classes/package_deb.bbclass +18 -0
index 3b42d3b..a25e5d7 100644
@@ -334,18 +334,36 @@ python do_package_deb () {
 
         mapping_rename_hook(localdata)
 
+        def debian_cmp_remap(var):
+            # In debian '>' and '<' do not mean what it appears they mean
+            #   '<' = less or equal
+            #   '>' = greater or equal
+            # adjust these to the '<<' and '>>' equivalents
+            #
+            for dep in var:
+                if (var[dep] or "").startswith("< "):
+                    var[dep] = var[dep].replace("< ", "<< ")
+                elif (var[dep] or "").startswith("> "):
+                    var[dep] = var[dep].replace("> ", ">> ")
+
         rdepends = bb.utils.explode_dep_versions(localdata.getVar("RDEPENDS", True) or "")
+        debian_cmp_remap(rdepends)
         for dep in rdepends:
                 if '*' in dep:
                         del rdepends[dep]
         rrecommends = bb.utils.explode_dep_versions(localdata.getVar("RRECOMMENDS", True) or "")
+        debian_cmp_remap(rrecommends)
         for dep in rrecommends:
                 if '*' in dep:
                         del rrecommends[dep]
         rsuggests = bb.utils.explode_dep_versions(localdata.getVar("RSUGGESTS", True) or "")
+        debian_cmp_remap(rsuggests)
         rprovides = bb.utils.explode_dep_versions(localdata.getVar("RPROVIDES", True) or "")
+        debian_cmp_remap(rprovides)
         rreplaces = bb.utils.explode_dep_versions(localdata.getVar("RREPLACES", True) or "")
+        debian_cmp_remap(rreplaces)
         rconflicts = bb.utils.explode_dep_versions(localdata.getVar("RCONFLICTS", True) or "")
+        debian_cmp_remap(rconflicts)
         if rdepends:
             ctrlfile.write("Depends: %s\n" % unicode(bb.utils.join_deps(rdepends)))
         if rsuggests:
meta/classes/package_ipk.bbclass +18 -0
index 59cb856..281ce59 100644
@@ -372,12 +372,30 @@ python do_package_ipk () {
 
         mapping_rename_hook(localdata)
 
+        def debian_cmp_remap(var):
+            # In debian '>' and '<' do not mean what it appears they mean
+            #   '<' = less or equal
+            #   '>' = greater or equal
+            # adjust these to the '<<' and '>>' equivalents
+            #
+            for dep in var:
+                if (var[dep] or "").startswith("< "):
+                    var[dep] = var[dep].replace("< ", "<< ")
+                elif (var[dep] or "").startswith("> "):
+                    var[dep] = var[dep].replace("> ", ">> ")
+
         rdepends = bb.utils.explode_dep_versions(localdata.getVar("RDEPENDS", True) or "")
+        debian_cmp_remap(rdepends)
         rrecommends = bb.utils.explode_dep_versions(localdata.getVar("RRECOMMENDS", True) or "")
+        debian_cmp_remap(rrecommends)
         rsuggests = bb.utils.explode_dep_versions(localdata.getVar("RSUGGESTS", True) or "")
+        debian_cmp_remap(rsuggests)
         rprovides = bb.utils.explode_dep_versions(localdata.getVar("RPROVIDES", True) or "")
+        debian_cmp_remap(rprovides)
         rreplaces = bb.utils.explode_dep_versions(localdata.getVar("RREPLACES", True) or "")
+        debian_cmp_remap(rreplaces)
         rconflicts = bb.utils.explode_dep_versions(localdata.getVar("RCONFLICTS", True) or "")
+        debian_cmp_remap(rconflicts)
 
         if rdepends:
             ctrlfile.write("Depends: %s\n" % bb.utils.join_deps(rdepends))
Signature · 1 line
1.7.3.4
author
#14 lore raw

PATCH 6/12 libxcb: Update DEPENDS to avoid duplicate entries

In reply to Mark Hatle

Some items were listed multiple times in DEPENDS, avoid this situation.

Note, PR was not incremented as no change to the build process occurs.

---
 meta/recipes-graphics/xcb/libxcb.inc       |    2 +-
 meta/recipes-graphics/xcb/libxcb_1.1.91.bb |    2 --
 meta/recipes-graphics/xcb/libxcb_1.8.1.bb  |    2 +-
 meta/recipes-graphics/xcb/libxcb_git.bb    |    2 --
 4 files changed, 2 insertions(+), 6 deletions(-)
meta/recipes-graphics/xcb/libxcb.inc +1 -1
index 1517f8a..5f4df69 100644
@@ -11,7 +11,7 @@ SECTION = "x11/libs"
 LICENSE = "MIT-X"
 
 PARALLEL_MAKE = ""
-DEPENDS = "xcb-proto xproto libxau libxslt-native xcb-proto-native"
+DEPENDS = "xcb-proto xproto libxau libxslt-native xcb-proto-native libpthread-stubs"
 # DEPENDS += "xsltproc-native gperf-native"
 
 SRC_URI = "http://xcb.freedesktop.org/dist/libxcb-${PV}.tar.bz2"
meta/recipes-graphics/xcb/libxcb_1.1.91.bb +0 -2
index 7ea7451..b74243e 100644
@@ -1,6 +1,4 @@
 include libxcb.inc
 PR = "r2"
 
-DEPENDS += "libpthread-stubs xcb-proto-native"
-
 PACKAGES =+ "libxcb-xinerama"
meta/recipes-graphics/xcb/libxcb_1.8.1.bb +1 -1
index 1f39ca0..52ff0b0 100644
@@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=d763b081cb10c223435b01e00dc0aba7"
 
 PR = "r0"
 
-DEPENDS += "libpthread-stubs xcb-proto-native libxdmcp"
+DEPENDS += "libxdmcp"
 
 PACKAGES =+ "libxcb-xinerama"
 
meta/recipes-graphics/xcb/libxcb_git.bb +0 -2
index f8cf2a5..66934b4 100644
@@ -6,8 +6,6 @@ SRCREV = "625ed596cae6dd8175aeb6cb6f26784928042f22"
 PV = "1.1.90.1+gitr${SRCPV}"
 PR = "r1"
 
-DEPENDS += "libpthread-stubs xcb-proto-native"
-
 SRC_URI = "git://anongit.freedesktop.org/git/xcb/libxcb;protocol=git"
 S = "${WORKDIR}/git"
 
Signature · 1 line
1.7.3.4
author
#15 lore raw

PATCH 7/12 Cleanup: fix PN == BPN cases

In reply to Mark Hatle

When building target packages, it used to be enought to check for PN == BPN, however with the multilib configurations, this can lead to subtle errors. Change instances of PN == BPN, to ${CLASSOVERRIDE} == 'cross-target'.

---
 meta/classes/distutils-base.bbclass        |    2 +-
 meta/recipes-core/ncurses/ncurses.inc      |    6 +++---
 meta/recipes-extended/pigz/pigz.inc        |    4 +++-
 meta/recipes-support/apr/apr-util_1.4.1.bb |    4 ++--
 4 files changed, 9 insertions(+), 7 deletions(-)
meta/classes/distutils-base.bbclass +1 -1
index c761b36..3b43e76 100644
@@ -1,5 +1,5 @@
 DEPENDS  += "${@["python-native python", ""][(d.getVar('PACKAGES', True) == '')]}"
-RDEPENDS_${PN} += "${@['', 'python-core']['${PN}' == '${BPN}']}"
+RDEPENDS_${PN} += "${@['', 'python-core']['${CLASSOVERRIDE}' == 'class-target']}"
 
 inherit distutils-common-base pythonnative
 
meta/recipes-core/ncurses/ncurses.inc +3 -3
index 91b1f34..ed7f001 100644
@@ -6,7 +6,7 @@ LIC_FILES_CHKSUM = "file://ncurses/base/version.c;beginline=1;endline=27;md5=cbc
 SECTION = "libs"
 DEPENDS = "ncurses-native"
 DEPENDS_virtclass-native = ""
-INC_PR = "r10"
+INC_PR = "r11"
 
 inherit autotools binconfig multilib_header
 
@@ -244,8 +244,8 @@ FILES_${PN}-tools = "\
   ${bindir}/infotocap \
   ${bindir}/captoinfo \
   ${bindir}/infocmp \
-  ${bindir}/clear${@['', '.${BPN\x7d'][d.getVar('PN', 1) == 'ncurses']} \
-  ${bindir}/reset${@['', '.${BPN\x7d'][d.getVar('PN', 1) == 'ncurses']} \
+  ${bindir}/clear${@['', '.${BPN\x7d']['${CLASSOVERRIDE}' == 'class-target']} \
+  ${bindir}/reset${@['', '.${BPN\x7d']['${CLASSOVERRIDE}' == 'class-target']} \
   ${bindir}/tack \
   ${bindir}/tabs \
 "
meta/recipes-extended/pigz/pigz.inc +3 -1
index be37e8a..3721c97 100644
@@ -7,6 +7,8 @@ HOMEPAGE = "http://zlib.net/pigz/"
 SECTION = "console/utils"
 LICENSE = "Zlib"
 
+PR = "r1"
+
 SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
            file://ldflags.patch"
 
@@ -15,7 +17,7 @@ PROVIDES_virtclass-native += "gzip-native"
 DEPENDS = "zlib"
 
 do_install () {
-	if [ "${PN}" = "${BPN}" ] ; then
+	if [ "${CLASSOVERRIDE}" = "class-target" ] ; then
 		# Install files into /bin (FHS), which is typical place for gzip
 	        install -d ${D}${base_bindir}	        
 		install ${B}/pigz ${D}${base_bindir}/gzip
meta/recipes-support/apr/apr-util_1.4.1.bb +2 -2
index 0516bc4..238af97 100644
@@ -9,7 +9,7 @@ LICENSE = "Apache-2.0"
 LIC_FILES_CHKSUM = "file://LICENSE;md5=519e0a18e03f7c023070568c14b077bb \
                     file://include/apu_version.h;endline=17;md5=806685a84e71f10c80144c48eb35df42"
 
-PR = "r1"
+PR = "r2"
 
 SRC_URI = "${APACHE_MIRROR}/apr/${BPN}-${PV}.tar.gz \
            file://configfix.patch \
@@ -33,7 +33,7 @@ inherit autotools lib_package binconfig
 OE_BINCONFIG_EXTRA_MANGLE = " -e 's:location=source:location=installed:'"
 
 do_configure_append() {
-	if [ "${PN}" == "${BPN}" ]; then
+	if [ "${CLASSOVERRIDE}" = "class-target" ]; then
 		cp ${STAGING_DATADIR}/apr/apr_rules.mk ${S}/build/rules.mk
 	fi
 }
Signature · 1 line
1.7.3.4
author
#16 lore raw

PATCH 8/12 multilib: Move redefinition of STAGING_DIR_KERNEL

In reply to Mark Hatle

If the STAGING_DIR_KERNEL is set in the multilib.conf, then it may be set incorrected. The evaluation happens before TMPDIR and LIBC are defined in other components.

Moving the definition process to the multilib.bbclass ensures that everything has been loaded before it is set.

---
 meta/classes/multilib.bbclass |    2 ++
 meta/conf/multilib.conf       |    1 -
 2 files changed, 2 insertions(+), 1 deletions(-)
meta/classes/multilib.bbclass +2 -0
index fb03fca..8187e18 100644
@@ -7,6 +7,8 @@ python multilib_virtclass_handler () {
     if cls != "multilib" or not variant:
         return
 
+    e.data.setVar('STAGING_KERNEL_DIR', e.data.getVar('STAGING_KERNEL_DIR', True))
+
     # There should only be one kernel in multilib configs
     if bb.data.inherits_class('kernel', e.data) or bb.data.inherits_class('module-base', e.data):
         raise bb.parse.SkipPackage("We shouldn't have multilib variants for the kernel")
meta/conf/multilib.conf +0 -1
index ea5ed71..b22a4cb 100644
@@ -6,7 +6,6 @@ MULTILIB_SAVE_VARNAME = "DEFAULTTUNE"
 
 MULTILIBS ??= "multilib:lib32"
 
-STAGING_KERNEL_DIR := "${STAGING_KERNEL_DIR}"
 STAGING_DIR_HOST = "${STAGING_DIR}/${MLPREFIX}${MACHINE}"
 STAGING_DIR_TARGET = "${STAGING_DIR}/${MLPREFIX}${MACHINE}"
 
Signature · 1 line
1.7.3.4
author
#17 lore raw

PATCH 9/12 rpm-native: Fix 'uuid_rc_t' undeclared error when compiling

In reply to Mark Hatle

When attempting to build with uuid and all tests compiling will error because uuid.h doesn't exist in the rpm tarball. Fix this by changing the include to use the one in ossp which solves the issue.

The recipe already depends on ossp so ossp/uuid.h will be there when rpm-native is built.

---
 .../rpm/rpm/rpm-uuid-include.patch                 |   35 ++++++++++++++++++++
 meta/recipes-devtools/rpm/rpm_5.4.9.bb             |    3 +-
 2 files changed, 37 insertions(+), 1 deletions(-)
 create mode 100644 meta/recipes-devtools/rpm/rpm/rpm-uuid-include.patch
meta/recipes-devtools/rpm/rpm/rpm-uuid-include.patch new file +35 -0
new file mode 100644
index 0000000..0e45ed7
@@ -0,0 +1,35 @@
+A patch to fix a problem when compiling with uuid.
+
+When attempting to build with uuid and all tests compiling will error because
+uuid.h doesn't exist in the rpm tarball. Fix this by changing the include to
+use the one in ossp which solves the issue. 
+
+The recipe already depends on ossp so ossp/uuid.h will be there when rpm-native
+is built.
+
+Signed-off-by: Morgan Little <[email protected]>
+
+
+--- a/lua/local/luuid.c.orig
++++ b/lua/local/luuid.c
+@@ -21,7 +21,7 @@
+ 
+ #ifdef LUA_USE_UUID
+ 
+-#include "uuid.h"
++#include <ossp/uuid.h>
+ 
+ /*  Lua method: <uuid> = uuid.generate(<version>[, <uuid-namespace>, <data>])  */
+ static int luuid_generate(lua_State *L)
+
+--- a/rpmio/rpmuuid.c.orig
++++ b/rpmio/rpmuuid.c
+@@ -16,7 +16,7 @@
+ #include "rpmlog.h"
+ #include "rpmuuid.h"
+ #ifdef WITH_UUID
+-#include "uuid.h"
++#include <ossp/uuid.h>
+ #endif
+ #include "debug.h"
+
meta/recipes-devtools/rpm/rpm_5.4.9.bb +2 -1
index 5435445..2a4c8d8 100644
@@ -43,7 +43,7 @@ LICENSE = "LGPLv2.1"
 LIC_FILES_CHKSUM = "file://COPYING.LIB;md5=2d5025d4aa3495befef8f17206a5b0a1"
 
 DEPENDS = "libpcre attr acl popt ossp-uuid file bison-native"
-PR = "r48"
+PR = "r49"
 
 # rpm2cpio is a shell script, which is part of the rpm src.rpm.  It is needed
 # in order to extract the distribution SRPM into a format we can extract...
@@ -76,6 +76,7 @@ SRC_URI = "http://www.rpm5.org/files/rpm/rpm-5.4/rpm-5.4.9-0.20120508.src.rpm;ex
 	   file://rpmatch.patch \
 	   file://fstack-protector-configure-check.patch \
 	   file://dbconvert.patch \
+	   file://rpm-uuid-include.patch \
 	  "
 
 SRC_URI[md5sum] = "60d56ace884340c1b3fcac6a1d58e768"
Signature · 1 line
1.7.3.4
author
#18 lore raw

PATCH 10/12 rpm: Fix file contention issue

In reply to Mark Hatle

There is an issue that is caused when doing the install step of rpm on systems with high parallelization where two jobs of make will fight for the same file while installing the sub-directory lua. This is caused by the same makefile rule being called twice in a way that both could be trying to install at the same time.

This fix renames the linking rule so it will always be run after the needed files are added and removed it's dependency so the required rule would only run once.

This was tested heavily using ppss to run mutliple installs in parallel. This wouldn't happen in practise but it was tested will all the individual rules as well.

---
 .../rpm/rpm/makefile-am-exec-hook.patch            |   31 ++++++++++++++++++++
 meta/recipes-devtools/rpm/rpm_5.4.9.bb             |    3 +-
 2 files changed, 33 insertions(+), 1 deletions(-)
 create mode 100644 meta/recipes-devtools/rpm/rpm/makefile-am-exec-hook.patch
meta/recipes-devtools/rpm/rpm/makefile-am-exec-hook.patch new file +31 -0
new file mode 100644
index 0000000..12f85b7
@@ -0,0 +1,31 @@
+rpm: Resolve parallel install issue when installing lua
+
+When lua is being installed on some systems that are being run with high levels
+of parallelization there are cases where install-data-am and install-exec-hook
+are processed at the same or a very short time apart. This causes
+install-pkgbinPROGRAMS, which is a dependency of both, to be run at around the
+same time This sometimes causes file contention and will sometimes be in a state
+where install-exec-hook is looking for a file that is being installed or both
+install-pkgbinPROGRAMS being run are installing the same file and fail because
+it cannon create a new file.
+
+This patch allows install-exec-hook to be called by install-data-am instead of
+install-exec-am. It also removed the dependency in install-data-hook on
+install-pkgbinPROGRAMS. This means install-pkgbinPROGRAMS will only be run once
+so there whould be any file contention.
+
+Upstream-Status: Pending
+
+Signed-off-by: Morgan Little <[email protected]>
+
+--- a/lua/Makefile.am.orig
++++ b/lua/Makefile.am
+@@ -320,7 +320,7 @@
+ # XXX Build & install as rpmlua/rpmluac with hardlinks to lua/luac post install.
+ # XXX CVS has lua/luac sub-directories in the Attic that collide with the
+ # XXX lua/luac executable names when using cvs update.
+-install-exec-hook:	install-pkgbinPROGRAMS
++install-data-hook:	
+ 	mkdir -p $(DESTDIR)$(pkgbindir)
+ 	$(__RM) -f $(DESTDIR)$(pkgbindir)/lua
+ 	$(__LN) $(DESTDIR)$(pkgbindir)/rpmlua $(DESTDIR)$(pkgbindir)/lua
meta/recipes-devtools/rpm/rpm_5.4.9.bb +2 -1
index 2a4c8d8..4aea2f6 100644
@@ -43,7 +43,7 @@ LICENSE = "LGPLv2.1"
 LIC_FILES_CHKSUM = "file://COPYING.LIB;md5=2d5025d4aa3495befef8f17206a5b0a1"
 
 DEPENDS = "libpcre attr acl popt ossp-uuid file bison-native"
-PR = "r49"
+PR = "r50"
 
 # rpm2cpio is a shell script, which is part of the rpm src.rpm.  It is needed
 # in order to extract the distribution SRPM into a format we can extract...
@@ -77,6 +77,7 @@ SRC_URI = "http://www.rpm5.org/files/rpm/rpm-5.4/rpm-5.4.9-0.20120508.src.rpm;ex
 	   file://fstack-protector-configure-check.patch \
 	   file://dbconvert.patch \
 	   file://rpm-uuid-include.patch \
+	   file://makefile-am-exec-hook.path \
 	  "
 
 SRC_URI[md5sum] = "60d56ace884340c1b3fcac6a1d58e768"
Signature · 1 line
1.7.3.4
author
#19 lore raw

PATCH 11/12 rpm: Add rpm patch to fix git_strerror issues

In reply to Mark Hatle

Remove the optimzation append from recipe and add the patch that is in the rpm cvs repo, http://www.mail-archive.com/[email protected]/msg08907.html. The -O2 optimzation append is removed since it can limit debugging options that are provided when -O0 is used.

This was tested by setting: SELECTED_OPTIMIZATION = "-O0"

---
 .../rpm/rpm/rpm-stub-out-git_strerror.patch        |   60 ++++++++++++++++++++
 meta/recipes-devtools/rpm/rpm_5.4.9.bb             |    3 +-
 2 files changed, 62 insertions(+), 1 deletions(-)
 create mode 100644 meta/recipes-devtools/rpm/rpm/rpm-stub-out-git_strerror.patch
meta/recipes-devtools/rpm/rpm/rpm-stub-out-git_strerror.patch new file +60 -0
new file mode 100644
index 0000000..2cfc163
@@ -0,0 +1,60 @@
+git: fix: stud-out git_strerror when --without-git
+
+Patch taken from the cvs repo,http://www.mail-archive.com/[email protected]/msg08907.html. 
+
+Upstream-status: Backport
+
+Signed-off-by: Morgan Little <[email protected]>
+
+Server: rpm5.org                         Name:   Jeff Johnson
+Root:   /v/rpm/cvs                       Email:  [email protected]
+Module: rpm                              Date:   07-Jun-2012 16:17:02
+Branch: rpm-5_4                          Handle: 2012060714165803
+
+Modified files:           (Branch: rpm-5_4)
+  rpm                     CHANGES
+  rpm/rpmio               rpmgit.c tgit.c
+
+Log:
+  - git: fix: stub-out git_strerror when --without-git.
+
+Summary:
+  Revision    Changes     Path
+  2.1.2.11    +3  -1      rpm/rpmio/rpmgit.c
+  1.1.2.7     +0  -1      rpm/rpmio/tgit.c
+____________________________________________________________________________
+Index: rpm/rpmio/rpmgit.c
+============================================================================
+--- rpm/rpmio/rpmgit.c
++++ rpm/rpmio/rpmgit.c
+@@ -145,6 +145,9 @@
+ #endif       /* defined(WITH_LIBGT2) */
+ 
+ /*==============================================================*/
++#if !defined(WITH_LIBGIT2)
++#define      git_strerror(_rc) ""
++#endif
+ 
+ static int Xchkgit(/*@unused@*/ rpmgit git, const char * msg,
+                 int error, int printit,
+@@ -157,7 +157,6 @@
+     int rc = error;
+ 
+     if (printit && rc) {
+-	/* XXX git_strerror? */
+         rpmlog(RPMLOG_ERR, "%s:%s:%u: %s(%d): %s\n",
+                 func, fn, ln, msg, rc, git_strerror(rc));
+     }
+Index: rpm/rpmio/tgit.c
+============================================================================
+--- rpm/rpmio/tgit.c
++++ rpm/rpmio/tgit.c
+@@ -34,7 +34,6 @@
+     int rc = error;
+ 
+     if (printit && rc) {
+-	/* XXX git_strerror? */
+         rpmlog(RPMLOG_ERR, "%s:%s:%u: %s(%d): %s\n",
+                 func, fn, ln, msg, rc, git_strerror(rc));
+     }
+
meta/recipes-devtools/rpm/rpm_5.4.9.bb +2 -1
index 4aea2f6..dda5611 100644
@@ -43,7 +43,7 @@ LICENSE = "LGPLv2.1"
 LIC_FILES_CHKSUM = "file://COPYING.LIB;md5=2d5025d4aa3495befef8f17206a5b0a1"
 
 DEPENDS = "libpcre attr acl popt ossp-uuid file bison-native"
-PR = "r50"
+PR = "r51"
 
 # rpm2cpio is a shell script, which is part of the rpm src.rpm.  It is needed
 # in order to extract the distribution SRPM into a format we can extract...
@@ -78,6 +78,7 @@ SRC_URI = "http://www.rpm5.org/files/rpm/rpm-5.4/rpm-5.4.9-0.20120508.src.rpm;ex
 	   file://dbconvert.patch \
 	   file://rpm-uuid-include.patch \
 	   file://makefile-am-exec-hook.path \
+	   file://rpm-stub-out-git_strerror.patch \
 	  "
 
 SRC_URI[md5sum] = "60d56ace884340c1b3fcac6a1d58e768"
Signature · 1 line
1.7.3.4
author
#20 lore raw

PATCH 12/12 rpm: Implement workaround for DB_BUFFER_SMALL error

In reply to Mark Hatle

In certain cases with BerkleyDB 5.3.x we are getting the error:

db3.c:1443: dbcursor->pget(-30999): BDB0063 DB_BUFFER_SMALL: User memory too small fo

See https://bugs.launchpad.net/rpm/+bug/934420 for more information.

It appears to be some type of a bug in the BerkleyDB 5.3.x. In an attempt to workaround the problem, when we encounter this situation we attempt to adjust the size of the mmap buffer until the call works, or we end up trying 10 times. The new size is either the updated vp->size from the failed pget call, or the previous size + 1024.

If DBI debugging is enabled, additional diagnostics are printed, otherwise a basic retry and success message is added to show that the failure was resolved.

---
 .../rpm/rpm/rpm-db_buffer_small.patch              |   77 ++++++++++++++++++++
 meta/recipes-devtools/rpm/rpm_5.4.9.bb             |    3 +-
 2 files changed, 79 insertions(+), 1 deletions(-)
 create mode 100644 meta/recipes-devtools/rpm/rpm/rpm-db_buffer_small.patch
meta/recipes-devtools/rpm/rpm/rpm-db_buffer_small.patch new file +77 -0
new file mode 100644
index 0000000..d2ed853
@@ -0,0 +1,77 @@
+In certain cases with BerkleyDB 5.3.x we are getting the error:
+
+db3.c:1443: dbcursor->pget(-30999): BDB0063 DB_BUFFER_SMALL: User memory too small for return value
+
+See https://bugs.launchpad.net/rpm/+bug/934420 for more information.
+
+It appears to be some type of a bug in the BerkleyDB 5.3.x.  In an attempt
+to workaround the problem, when we encounter this situation we attempt
+to adjust the size of the mmap buffer until the call works, or we
+end up trying 10 times.  The new size is either the updated vp->size
+from the failed pget call, or the previous size + 1024.
+
+If DBI debugging is enabled, additional diagnostics are printed, otherwise
+a basic retry and success message is added to show that the failure was
+resolved.
+
+Upstream-status: Inappropriate (workaround)
+
+Signed-off-by: Mark Hatle <[email protected]>
+
+Index: rpm-5.4.9/rpmdb/rpmdb.c
+===================================================================
+--- rpm-5.4.9.orig/rpmdb/rpmdb.c
++++ rpm-5.4.9/rpmdb/rpmdb.c
+@@ -2212,8 +2212,12 @@ static int rpmmiGet(dbiIndex dbi, DBC * 
+ 	vp->flags |= DB_DBT_USERMEM;
+ 	rc = dbiGet(dbi, dbcursor, kp, vp, flags);
+ 	if (rc == DB_BUFFER_SMALL) {
++	    int retry = 0;
++	    size_t origlen = vp->size;
+ 	    size_t uhlen = vp->size;
+-	    void * uh = mmap(NULL, uhlen, _prot, _flags, _fdno, _off);
++	    void * uh;
++retry_get:
++	    uh = mmap(NULL, uhlen, _prot, _flags, _fdno, _off);
+ 	    if (uh == NULL || uh == (void *)-1)
+ 		fprintf(stderr,
+ 		    "==> mmap(%p[%u], 0x%x, 0x%x, %d, 0x%x) error(%d): %s\n",
+@@ -2235,6 +2239,25 @@ static int rpmmiGet(dbiIndex dbi, DBC * 
+ 		if (munmap(uh, uhlen) != 0)
+ 		    fprintf(stderr, "==> munmap(%p[%u]) error(%d): %s\n",
+                 	uh, (unsigned)uhlen, errno, strerror(errno));
++	        /* We want to be sure to limit the number of retry attempts to avoid a loop! */
++	        if (rc == DB_BUFFER_SMALL && retry < 10) {
++		   /* If we got a largr vp-size back, use that, otherwise increment the size by 1k */
++	           uhlen = vp->size > uhlen ? vp->size : uhlen + 1024;
++		   retry++;
++	           if ((dbi)->dbi_debug)
++	               fprintf(stderr, "==> DB_BUFFER_SMALL orig requested (%d), configured (%d), forcing larger buffer (%d), new size (%d)\n",
++	                    origlen, vp->ulen, uhlen, vp->size);
++	           else
++	               fprintf(stderr, "==> retry (%d) db3cpget (%d)\n", retry, uhlen);
++	           goto retry_get;
++	        }
++	    }
++	    if (retry) {
++	        if ((dbi)->dbi_debug)
++	           fprintf(stderr, "==> success orig requested (%d), configured buffer (%d), buffer (%d), size after dbiGet (%d)\n",
++			origlen, vp->ulen, uhlen, vp->size);
++	        else
++	           fprintf(stderr, "==> success\n");
+ 	    }
+ 	}
+     } else
+Index: rpm-5.4.9/rpmdb/db3.c
+===================================================================
+--- rpm-5.4.9.orig/rpmdb/db3.c
++++ rpm-5.4.9/rpmdb/db3.c
+@@ -1452,7 +1452,7 @@ assert(db != NULL);
+ #endif
+     }
+ 
+-DBIDEBUG(dbi, (stderr, "<-- %s(%p,%p,%p,%p,%p,0x%x) rc %d %s%s\n", __FUNCTION__, dbi, dbcursor, key, pkey, data, flags, rc, _DBCFLAGS(flags), _KEYDATA(key, pkey, data, NULL)));
++DBIDEBUG(dbi, (stderr, "<-- %s(%p,%p,%p,%p,%p,0x%x) rc %d %s%s\n", __FUNCTION__, dbi, dbcursor, key, pkey, data, flags, rc, _DBCFLAGS(flags), _KEYDATA(key, pkey, rc == DB_BUFFER_SMALL ? NULL : data, NULL)));
+     return rc;
+ }
+ /*@=mustmod@*/
meta/recipes-devtools/rpm/rpm_5.4.9.bb +2 -1
index dda5611..2dbdf54 100644
@@ -43,7 +43,7 @@ LICENSE = "LGPLv2.1"
 LIC_FILES_CHKSUM = "file://COPYING.LIB;md5=2d5025d4aa3495befef8f17206a5b0a1"
 
 DEPENDS = "libpcre attr acl popt ossp-uuid file bison-native"
-PR = "r51"
+PR = "r52"
 
 # rpm2cpio is a shell script, which is part of the rpm src.rpm.  It is needed
 # in order to extract the distribution SRPM into a format we can extract...
@@ -79,6 +79,7 @@ SRC_URI = "http://www.rpm5.org/files/rpm/rpm-5.4/rpm-5.4.9-0.20120508.src.rpm;ex
 	   file://rpm-uuid-include.patch \
 	   file://makefile-am-exec-hook.path \
 	   file://rpm-stub-out-git_strerror.patch \
+	   file://rpm-db_buffer_small.patch \
 	  "
 
 SRC_URI[md5sum] = "60d56ace884340c1b3fcac6a1d58e768"
Signature · 1 line
1.7.3.4