Fix two regressions with RPM introduced with Pyro
PATCH 1/13 package_manager.py: Generate correct RPM package names again
In reply to Peter KjellerstedtDuring the transition to RPM4, the package names returned by RpmPM.list_installed() changed from the expected names of the packages that were installed into the image to some fictitious source RPM names.
This restores the original functionality so that the installed-packages.txt files produced by inheriting buildhistory yet again contains a list of the names of the installed packages.
--- meta/lib/oe/package_manager.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
meta/lib/oe/package_manager.py +3 -3
@@ -647,7 +647,7 @@ class RpmPM(PackageManager): symlinks=True) def list_installed(self): - output = self._invoke_dnf(["repoquery", "--installed", "--queryformat", "Package: %{name} %{arch} %{version} %{sourcerpm}\nDependencies:\n%{requires}\nRecommendations:\n%{recommends}\nDependenciesEndHere:\n"], + output = self._invoke_dnf(["repoquery", "--installed", "--queryformat", "Package: %{name} %{arch} %{version} %{name}-%{version}-%{release}.%{arch}.rpm\nDependencies:\n%{requires}\nRecommendations:\n%{recommends}\nDependenciesEndHere:\n"], print_output = False) packages = {} current_package = None
@@ -659,8 +659,8 @@ class RpmPM(PackageManager): current_package = package_info[0] package_arch = package_info[1] package_version = package_info[2] - package_srpm = package_info[3] - packages[current_package] = {"arch":package_arch, "ver":package_version, "filename":package_srpm} + package_rpm = package_info[3] + packages[current_package] = {"arch":package_arch, "ver":package_version, "filename":package_rpm} current_deps = [] elif line.startswith("Dependencies:"): current_state = "dependencies"
Signature · 1 line
2.12.0
PATCH 2/13 libxml2: Make ptest run the Python tests if Python support is enabled
In reply to Peter KjellerstedtSince we go through the trouble of copying the Python tests, we may as well actually run them...
This also avoids the following QA issue:
ERROR: libxml2-2.9.4-r0 do_package_qa: QA Issue:
/usr/lib/libxml2/ptest/python/tests/push.py contained in package libxml2-ptest requires /usr/bin/python, but no providers found in RDEPENDS_libxml2-ptest? [file-rdeps]
--- ...run-the-python-tests-if-python-is-enabled.patch | 99 ++++++++++++++++++++++ meta/recipes-core/libxml/libxml2_2.9.4.bb | 11 ++- 2 files changed, 108 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-core/libxml/libxml2/0001-Make-ptest-run-the-python-tests-if-python-is-enabled.patch
meta/recipes-core/libxml/libxml2/0001-Make-ptest-run-the-python-tests-if-python-is-enabled.patch new file +99 -0
@@ -0,0 +1,99 @@ +From 78dbd4c09d617a9cb730d796f94ee4d93840d3cc Mon Sep 17 00:00:00 2001 +From: Peter Kjellerstedt <[email protected]> +Date: Fri, 9 Jun 2017 17:50:46 +0200 +Subject: [PATCH] Make ptest run the python tests if python is enabled + +One of the tests (tstLastError.py) needed a minor correction. It might +be due to the fact that the tests are forced to run with Python 3. + +Upstream-Status: Inappropriate [OE specific] +Signed-off-by: Peter Kjellerstedt <[email protected]> +--- + Makefile.am | 2 +- + python/Makefile.am | 9 +++++++++ + python/tests/Makefile.am | 12 ++++++++++-- + python/tests/tstLastError.py | 2 +- + 4 files changed, 21 insertions(+), 4 deletions(-) + +diff --git a/Makefile.am b/Makefile.am +index e181ee0..7960e7d 100644 +--- a/Makefile.am ++++ b/Makefile.am +@@ -207,9 +207,9 @@ install-ptest: + install $(noinst_PROGRAMS) $(DESTDIR)) + cp -r $(srcdir)/test $(DESTDIR) + cp -r $(srcdir)/result $(DESTDIR) +- cp -r $(srcdir)/python $(DESTDIR) + cp Makefile $(DESTDIR) + sed -i -e 's|^Makefile:|_Makefile:|' $(DESTDIR)/Makefile ++ $(MAKE) -C python install-ptest + + runtests: + [ -d test ] || $(LN_S) $(srcdir)/test . +diff --git a/python/Makefile.am b/python/Makefile.am +index 34aed96..ba3ec6a 100644 +--- a/python/Makefile.am ++++ b/python/Makefile.am +@@ -48,7 +48,16 @@ GENERATED = libxml2class.py libxml2class.txt $(BUILT_SOURCES) + + $(GENERATED): $(srcdir)/generator.py $(API_DESC) + $(PYTHON) $(srcdir)/generator.py $(srcdir) ++ ++install-ptest: ++ cp -r $(srcdir) $(DESTDIR) ++ sed -e 's|^Makefile:|_Makefile:|' \ ++ -e 's|^\(tests test:\) all|\1|' Makefile >$(DESTDIR)/python/Makefile ++ $(MAKE) -C tests install-ptest ++else ++install-ptest: + endif + ++.PHONY: tests test + tests test: all + cd tests && $(MAKE) tests +diff --git a/python/tests/Makefile.am b/python/tests/Makefile.am +index 95ebead..418e21e 100644 +--- a/python/tests/Makefile.am ++++ b/python/tests/Makefile.am +@@ -59,6 +59,11 @@ XMLS= \ + CLEANFILES = core tmp.xml *.pyc + + if WITH_PYTHON ++install-ptest: ++ cp -r $(srcdir) $(DESTDIR)/python ++ sed -e 's|^Makefile:|_Makefile:|' \ ++ -e 's|^\(srcdir = \).*|\1.|' Makefile >$(DESTDIR)/python/tests/Makefile ++ + tests: $(PYTESTS) + @for f in $(XMLS) ; do test -f $$f || $(LN_S) $(srcdir)/$$f . ; done + @echo "## running Python regression tests" +@@ -68,8 +73,11 @@ tests: $(PYTESTS) + export LD_LIBRARY_PATH; \ + for test in $(PYTESTS) ; \ + do log=`$(PYTHON) $(srcdir)/$$test` ; \ +- if [ "`echo $$log | grep OK`" = "" ] ; then \ +- echo "-- $$test" ; echo "$$log" ; fi ; done) ++ if [ "`echo $$log | grep OK`" ]; then \ ++ echo "PASS: $$test"; else \ ++ echo "$$log"; echo "FAIL: $$test"; fi; done) + else ++install-ptest: ++ + tests: + endif +diff --git a/python/tests/tstLastError.py b/python/tests/tstLastError.py +index d5f9be7..3e5bef8 100755 +--- a/python/tests/tstLastError.py ++++ b/python/tests/tstLastError.py +@@ -25,7 +25,7 @@ class TestCase(unittest.TestCase): + when the exception is raised, check the libxml2.lastError for + expected values.""" + # disable the default error handler +- libxml2.registerErrorHandler(None,None) ++ libxml2.registerErrorHandler(lambda ctx,str: None,None) + try: + f(*args) + except exc: +-- +2.12.0 +
meta/recipes-core/libxml/libxml2_2.9.4.bb +9 -2
@@ -24,7 +24,8 @@ SRC_URI = "ftp://xmlsoft.org/libxml2/libxml2-${PV}.tar.gz;name=libtar \ file://libxml2-CVE-2016-4658.patch \ file://libxml2-fix_NULL_pointer_derefs.patch \ file://CVE-2016-9318.patch \ - " + file://0001-Make-ptest-run-the-python-tests-if-python-is-enabled.patch \ + " SRC_URI[libtar.md5sum] = "ae249165c173b1ff386ee8ad676815f5" SRC_URI[libtar.sha256sum] = "ffb911191e509b966deb55de705387f14156e1a56b21824357cdf0053233633c"
@@ -43,7 +44,7 @@ inherit autotools pkgconfig binconfig-disabled ptest inherit ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3native', '', d)} -RDEPENDS_${PN}-ptest += "make ${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-core', '', d)}" +RDEPENDS_${PN}-ptest += "make ${@bb.utils.contains('PACKAGECONFIG', 'python', 'libgcc python3-core python3-argparse python3-logging python3-shell python3-signal python3-stringold python3-threading python3-unittest ${PN}-python', '', d)}" RDEPENDS_${PN}-python += "${@bb.utils.contains('PACKAGECONFIG', 'python', 'python3-core', '', d)}"
@@ -77,6 +78,12 @@ do_configure_prepend () { do_install_ptest () { cp -r ${WORKDIR}/xmlconf ${D}${PTEST_PATH} + if [ "${@bb.utils.filter('PACKAGECONFIG', 'python', d)}" ]; then + sed -i -e 's|^\(PYTHON = \).*|\1${USRBINPATH}/${PYTHON_PN}|' \ + ${D}${PTEST_PATH}/python/tests/Makefile + grep -lrZ '#!/usr/bin/python' ${D}${PTEST_PATH}/python | + xargs -0 sed -i -e 's|/usr/bin/python|${USRBINPATH}/${PYTHON_PN}|' + fi } do_install_append_class-native () {
Signature · 1 line
2.12.0
cover Fix two regressions with RPM introduced with Pyro
Right before the release of Pyro (about a month ago), we noticed a problem with missing automatically determined runtime file dependencies after the transition to rpm4 and dnf. Unfortunately it has taken me quite a while to work out all the details, but here is the result finally.
During the search for the missing dependencies, I also stumbled on another regression where the installed-packages.txt file that is generated by buildhistory turned out to no longer contain the actual package names that were used to create the image, but rather some fictitious source RPM names... The first patch in the series addresses that.
The following patches to the libxml2 recipe, insane.bbclass, buildhistory.bbclass and toaster.bbclass prepare the stage for the changes to rpm and package.bbclass so that they will not break once the automatically generated dependencies are restored.
Then rpm is updated to make it possible to yet again access the runtime dependencies we need, and finally package.bbclass is updated to make use of them.
See the patches, especially the last one, for more in depth information on what went wrong during the transition to rpm4 and dnf, and how to solve it.
I would also like to thank Mark Hatle and Martin Jansa who set me on the right track to find the problem.
//Peter
The following changes since commit b1d5267f0e31605b346af40778da0ac1ff298b78:
README.hardware: update flashing instructions for MPC8315 (2017-06-09 17:13:45 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib pkj/rpm_regressions
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/rpm_regressions
Peter Kjellerstedt (13):
package_manager.py: Generate correct RPM package names again
libxml2: Make ptest run the Python tests if Python support is enabled insane.bbclass: Report all file-rdeps errors, not just the first insane.bbclass: Improve the handling of runtime file dependencies buildhistory.bbclass: Improve the generated depends.dot file toaster.bbclass: Simplify parsing of depends.dot toaster.bbclass: Ignore some dependencies in
toaster_buildhistory_dump()
rpm: Simplify the creation of wrappers for the native tools
rpm: Create a wrapper for the native rpmdeps tool
rpm: Do not require that ELF binaries are executable to be
identifiable
rpm: Use conditional to access %{_docdir} in macros.in
rpm: Add a new option --alldeps to rpmdeps package.bbclass: Restore functionality to detect RPM dependencies
meta/classes/buildhistory.bbclass | 15 +- meta/classes/insane.bbclass | 18 ++- meta/classes/package.bbclass | 5 +- meta/classes/toaster.bbclass | 36 +++-- meta/lib/oe/package.py | 47 ++++--- meta/lib/oe/package_manager.py | 6 +- ...run-the-python-tests-if-python-is-enabled.patch | 99 ++++++++++++++ meta/recipes-core/libxml/libxml2_2.9.4.bb | 11 +- ...ire-that-ELF-binaries-are-executable-to-b.patch | 32 +++++ ...onditional-to-access-_docdir-in-macros.in.patch | 36 +++++ .../0013-Add-a-new-option-alldeps-to-rpmdeps.patch | 152 +++++++++++++++++++++ meta/recipes-devtools/rpm/rpm_git.bb | 77 ++++------- 12 files changed, 426 insertions(+), 108 deletions(-) create mode 100644 meta/recipes-core/libxml/libxml2/0001-Make-ptest-run-the-python-tests-if-python-is-enabled.patch create mode 100644 meta/recipes-devtools/rpm/files/0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch create mode 100644 meta/recipes-devtools/rpm/files/0012-Use-conditional-to-access-_docdir-in-macros.in.patch create mode 100644 meta/recipes-devtools/rpm/files/0013-Add-a-new-option-alldeps-to-rpmdeps.patch
Signature · 1 line
2.12.0
PATCH 7/13 toaster.bbclass: Ignore some dependencies in toaster_buildhistory_dump()
When using RPM, depends.dot may contain dependencies such as "/bin/sh", which will confuse _toaster_load_pkgdatafile(). Ignore them. While at it, also ignore dependencies that contain parentheses, e.g., "libc.so.6(GLIBC_2.7)".
--- meta/classes/toaster.bbclass | 8 ++++++++ 1 file changed, 8 insertions(+)
meta/classes/toaster.bbclass +8 -0
@@ -279,6 +279,14 @@ python toaster_buildhistory_dump() { dependsname = m.group('dep') deptype = 'recommends' if m.group('rec') else 'depends' + # If RPM is used for packaging, then there may be + # dependencies such as "/bin/sh", which will confuse + # _toaster_load_pkgdatafile() later on. While at it, ignore + # any dependencies that contain parentheses, e.g., + # "libc.so.6(GLIBC_2.7)". + if dependsname.startswith('/') or '(' in dependsname: + continue + if not pname in images[target]: images[target][pname] = {'size': 0, 'depends' : []} if not dependsname in images[target]:
Signature · 1 line
2.12.0
PATCH 11/13 rpm: Use conditional to access %{_docdir} in macros.in
In reply to Peter KjellerstedtThis avoids the following warning:
warning: Ignoring invalid regex %{_docdir}
when runing rpmdeps -R <file>, since %{_docdir} is only defined when parsing a spec file.
--- ...onditional-to-access-_docdir-in-macros.in.patch | 36 ++++++++++++++++++++++ meta/recipes-devtools/rpm/rpm_git.bb | 1 + 2 files changed, 37 insertions(+) create mode 100644 meta/recipes-devtools/rpm/files/0012-Use-conditional-to-access-_docdir-in-macros.in.patch
meta/recipes-devtools/rpm/files/0012-Use-conditional-to-access-_docdir-in-macros.in.patch new file +36 -0
@@ -0,0 +1,36 @@ +From 77808db4036dc4a012c47aca36255549ed764a6a Mon Sep 17 00:00:00 2001 +From: Peter Kjellerstedt <[email protected]> +Date: Tue, 16 May 2017 10:58:18 +0200 +Subject: [PATCH 12/13] Use conditional to access %{_docdir} in macros.in + +This avoids the following warning: + +warning: Ignoring invalid regex %{_docdir} + +when runing `rpmdeps -R <file>`, since %{_docdir} is only defined when +parsing a spec file (in parseSpec()). + +Upstream-Status: Accepted [https://github.com/rpm-software-management/rpm/pull/216] +Signed-off-by: Peter Kjellerstedt <[email protected]> +--- + macros.in | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/macros.in b/macros.in +index 84ae25275..cca67a500 100644 +--- a/macros.in ++++ b/macros.in +@@ -200,8 +200,8 @@ package or when debugging this package.\ + # Their purpouse is to set up global filtering for all packages. If you need + # to set up specific filtering for your package use %__requires_exclude_from + # and %__provides_exclude_from instead. +-%__global_requires_exclude_from %{_docdir} +-%__global_provides_exclude_from %{_docdir} ++%__global_requires_exclude_from %{?_docdir:%{_docdir}} ++%__global_provides_exclude_from %{?_docdir:%{_docdir}} + + # The path to the gzip executable (legacy, use %{__gzip} instead). + %_gzipbin %{__gzip} +-- +2.12.0 +
meta/recipes-devtools/rpm/rpm_git.bb +1 -0
@@ -36,6 +36,7 @@ SRC_URI = "git://github.com/rpm-software-management/rpm \ file://0001-Add-a-color-setting-for-mips64_n32-binaries.patch \ file://0001-Add-PYTHON_ABI-when-searching-for-python-libraries.patch \ file://0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch \ + file://0012-Use-conditional-to-access-_docdir-in-macros.in.patch \ " PV = "4.13.90+git${SRCPV}"
Signature · 1 line
2.12.0
PATCH 13/13 package.bbclass: Restore functionality to detect RPM dependencies
In reply to Peter KjellerstedtDuring the transition to dnf and rpm4, the functionality to automatically make RPM determine dependencies was lost.
Before the transition, an OE specific tool called rpmdeps-oecore had been added to the rpm suit. It was based on the rpmdeps tool that is part of rpm. For each file specified on its command line, it would output the provides and requires that RPM could determine.
During the transition to rpm4, rpmdeps-oecore was replaced with the standard rpmdeps. However, what no one noticed was that unless rpmdeps is given options, e.g., -P or -R, to tell it what it should output, it will not output anything. Thus, it would do all the work to determine the requirements, but would keep silent about it. And since no output from rpmdeps is expected unless there are requirements, there were no warnings indicating that everything was not working as expected.
Porting the old rpmdeps-oecore to work with rpm4 is not really possible since it relied on being able to access internals of RPM that are no longer available. However, it turned out that rpmdeps had a debug option, --rpmfcdebug, that would output exactly the information that we need, albeit in a different format and to stderr. To make this usable, rpmdeps has now received a new option, --alldeps, which sends the information we need to stdout.
--- meta/classes/package.bbclass | 5 ++--- meta/lib/oe/package.py | 47 +++++++++++++++++++++++++------------------- 2 files changed, 29 insertions(+), 23 deletions(-)
meta/classes/package.bbclass +2 -3
@@ -1434,7 +1434,7 @@ if [ x"$D" = "x" ]; then fi } -RPMDEPS = "${STAGING_LIBDIR_NATIVE}/rpm/rpmdeps --rcfile ${STAGING_LIBDIR_NATIVE}/rpm/rpmrc --macros ${STAGING_LIBDIR_NATIVE}/rpm/macros --define '_rpmconfigdir ${STAGING_LIBDIR_NATIVE}/rpm/'" +RPMDEPS = "${STAGING_LIBDIR_NATIVE}/rpm/rpmdeps --alldeps" # Collect perfile run-time dependency metadata # Output:
@@ -1451,7 +1451,6 @@ python package_do_filedeps() { pkgdest = d.getVar('PKGDEST') packages = d.getVar('PACKAGES') rpmdeps = d.getVar('RPMDEPS') - magic = d.expand("${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc") def chunks(files, n): return [files[i:i+n] for i in range(0, len(files), n)]
@@ -1463,7 +1462,7 @@ python package_do_filedeps() { if pkg.endswith('-dbg') or pkg.endswith('-doc') or pkg.find('-locale-') != -1 or pkg.find('-localedata-') != -1 or pkg.find('-gconv-') != -1 or pkg.find('-charmap-') != -1 or pkg.startswith('kernel-module-'): continue for files in chunks(pkgfiles[pkg], 100): - pkglist.append((pkg, files, rpmdeps, pkgdest, magic)) + pkglist.append((pkg, files, rpmdeps, pkgdest)) processed = oe.utils.multiprocess_exec( pkglist, oe.package.filedeprunner)
meta/lib/oe/package.py +27 -20
71 lines not on this page · raw message
Signature · 1 line
2.12.0
PATCH 3/13 insane.bbclass: Report all file-rdeps errors, not just the first
In reply to Peter Kjellerstedt--- meta/classes/insane.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/insane.bbclass +1 -1
@@ -968,7 +968,7 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d): for key in filerdepends: error_msg = "%s contained in package %s requires %s, but no providers found in RDEPENDS_%s?" % \ (filerdepends[key].replace("_%s" % pkg, "").replace("@underscore@", "_"), pkg, key, pkg) - package_qa_handle_error("file-rdeps", error_msg, d) + package_qa_handle_error("file-rdeps", error_msg, d) def package_qa_check_deps(pkg, pkgdest, skip, d):
Signature · 1 line
2.12.0
PATCH 5/13 buildhistory.bbclass: Improve the generated depends.dot file
In reply to Peter Kjellerstedt* Convert incorrectly formatted dependencies such as:
"bar -> "foo" ">=" "1.2.3"
into dependencies with edge labels:
"bar -> "foo" [label=">= 1.2.3"]
* Remove rpmlib() and config() dependencies such as:
"foo" -> "rpmlib(CompressedFileNames)" [label="<= 3.0.4-1"]
and:
"base-files" -> "config(base-files)" [label="= 3.0.14-r89.49"]* Remove the trailing semicolon that was added to each line. It fills no purpose.
--- meta/classes/buildhistory.bbclass | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
meta/classes/buildhistory.bbclass +11 -4
@@ -434,13 +434,20 @@ buildhistory_get_installed() { # Produce dependency graph # First, quote each name to handle characters that cause issues for dot - sed 's:\([^| ]*\):"\1":g' ${WORKDIR}/bh_installed_pkgs_deps.txt > $1/depends.tmp && \ + sed 's:\([^| ]*\):"\1":g' ${WORKDIR}/bh_installed_pkgs_deps.txt > $1/depends.tmp && rm ${WORKDIR}/bh_installed_pkgs_deps.txt - # Change delimiter from pipe to -> and set style for recommend lines - sed -i -e 's:|: -> :' -e 's:"\[REC\]":[style=dotted]:' -e 's:$:;:' $1/depends.tmp + # Remove lines with rpmlib(...) and config(...) dependencies, change the + # delimiter from pipe to "->", set the style for recommend lines and + # turn versioned dependencies into edge labels. + sed -i -e '/rpmlib(/d' \ + -e '/config(/d' \ + -e 's:|: -> :' \ + -e 's:"\[REC\]":[style=dotted]:' \ + -e 's:"\([<>=]\+\)" "\([^"]*\)":[label="\1 \2"]:' \ + $1/depends.tmp # Add header, sorted and de-duped contents and footer and then delete the temp file printf "digraph depends {\n node [shape=plaintext]\n" > $1/depends.dot - cat $1/depends.tmp | sort | uniq >> $1/depends.dot + cat $1/depends.tmp | sort -u >> $1/depends.dot echo "}" >> $1/depends.dot rm $1/depends.tmp
Signature · 1 line
2.12.0
PATCH 4/13 insane.bbclass: Improve the handling of runtime file dependencies
In reply to Peter KjellerstedtThis makes the file-rdeps test support:
* versioned dependencies, e.g., "perl (>= 5.000)", and
* package dependencies among the file dependencies, e.g., "perl".
It also ignores all "perl(...)" dependencies since it is expected that these are generated and handled by rpm itself and there is no reason to second guess what it is doing.
--- meta/classes/insane.bbclass | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
meta/classes/insane.bbclass +10 -6
@@ -912,8 +912,9 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d): rdep_data = oe.packagedata.read_subpkgdata(pkg, d) for key in rdep_data: if key.startswith("FILERDEPENDS_"): - for subkey in rdep_data[key].split(): - if subkey not in ignored_file_rdeps: + for subkey in bb.utils.explode_deps(rdep_data[key]): + if subkey not in ignored_file_rdeps and \ + not subkey.startswith('perl('): # We already know it starts with FILERDEPENDS_ filerdepends[subkey] = key[13:]
@@ -928,11 +929,10 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d): sub_rdeps = rdep_data.get("RDEPENDS_" + rdep) if not sub_rdeps: continue - for sub_rdep in sub_rdeps.split(): + for sub_rdep in bb.utils.explode_deps(sub_rdeps): if sub_rdep in done: continue - if not sub_rdep.startswith('(') and \ - oe.packagedata.has_subpkgdata(sub_rdep, d): + if oe.packagedata.has_subpkgdata(sub_rdep, d): # It's a new rdep done.append(sub_rdep) new.append(sub_rdep)
@@ -950,11 +950,15 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d): filerdepends.pop("/usr/bin/python",None) done.remove(py) for rdep in done: + # The file dependencies may contain package names, e.g., + # perl + filerdepends.pop(rdep,None) + # For Saving the FILERPROVIDES, RPROVIDES and FILES_INFO rdep_data = oe.packagedata.read_subpkgdata(rdep, d) for key in rdep_data: if key.startswith("FILERPROVIDES_") or key.startswith("RPROVIDES_"): - for subkey in rdep_data[key].split(): + for subkey in bb.utils.explode_deps(rdep_data[key]): filerdepends.pop(subkey,None) # Add the files list to the rprovides if key == "FILES_INFO":
Signature · 1 line
2.12.0
PATCH 9/13 rpm: Create a wrapper for the native rpmdeps tool
In reply to Peter KjellerstedtRather than trying to call rpmdeps with the correct arguments to work with the sysroot as was done in package.bbclass, create a wrapper for it like all the other native tools already had.
--- meta/recipes-devtools/rpm/rpm_git.bb | 1 + 1 file changed, 1 insertion(+)
meta/recipes-devtools/rpm/rpm_git.bb +1 -0
@@ -77,6 +77,7 @@ do_install_append_class-native() { ${bindir}/rpmkeys \ ${bindir}/rpmsign \ ${bindir}/rpmspec \ + ${libdir}/rpm/rpmdeps \ " for tool in $tools; do
Signature · 1 line
2.12.0
PATCH 12/13 rpm: Add a new option --alldeps to rpmdeps
In reply to Peter KjellerstedtThis will send the output from rpmfcPrint() to stdout. This is an alternative to using the --rpmfcdebug option, which will send the same output to stderr. The two options have totally different use cases though. While --alldeps is used when the output from rpmfcPrint() is what is wanted, --rpmfcdebug can be used together with the other output options, e.g., --requires, without affecting their output.
--- .../0013-Add-a-new-option-alldeps-to-rpmdeps.patch | 152 +++++++++++++++++++++ meta/recipes-devtools/rpm/rpm_git.bb | 1 + 2 files changed, 153 insertions(+) create mode 100644 meta/recipes-devtools/rpm/files/0013-Add-a-new-option-alldeps-to-rpmdeps.patch
meta/recipes-devtools/rpm/files/0013-Add-a-new-option-alldeps-to-rpmdeps.patch new file +152 -0
@@ -0,0 +1,152 @@ +From 3bf20a6116ae3e1a5a3a6907bee7e881b17efb2f Mon Sep 17 00:00:00 2001 +From: Peter Kjellerstedt <[email protected]> +Date: Mon, 15 May 2017 11:23:26 +0200 +Subject: [PATCH 13/13] Add a new option --alldeps to rpmdeps + +This will send the output from rpmfcPrint() to stdout. This is an +alternative to using the --rpmfcdebug option, which will send the same +output to stderr. The two options have totally different use cases +though. While --alldeps is used when the output from rpmfcPrint() is +what is wanted, --rpmfcdebug can be used together with the other +output options, e.g., --requires, without affecting their output. + +Upstream-Status: Submitted [https://github.com/rpm-software-management/rpm/pull/220] +Signed-off-by: Peter Kjellerstedt <[email protected]> +--- + build/rpmfc.c | 27 ++++++++++++++------------- + build/rpmfc.h | 1 - + tools/rpmdeps.c | 44 +++++++++++++++++++++++++------------------- + 3 files changed, 39 insertions(+), 33 deletions(-) + +diff --git a/build/rpmfc.c b/build/rpmfc.c +index c8e2f876a..44f1cdc9a 100644 +--- a/build/rpmfc.c ++++ b/build/rpmfc.c +@@ -732,7 +732,6 @@ static rpm_color_t rpmfcColor(const char * fmstr) + + void rpmfcPrint(const char * msg, rpmfc fc, FILE * fp) + { +- rpm_color_t fcolor; + int ndx; + int dx; + int fx; +@@ -744,21 +743,23 @@ void rpmfcPrint(const char * msg, rpmfc fc, FILE * fp) + + if (fc) + for (fx = 0; fx < fc->nfiles; fx++) { +- rpmsid cx = fc->fcdictx[fx] + 1; /* id's are one off */ +- fcolor = fc->fcolor[fx]; +- ARGV_t fattrs = fc->fattrs[fx]; +- + fprintf(fp, "%3d %s", fx, fc->fn[fx]); +- if (fcolor != RPMFC_BLACK) ++ if (_rpmfc_debug) { ++ rpmsid cx = fc->fcdictx[fx] + 1; /* id's are one off */ ++ rpm_color_t fcolor = fc->fcolor[fx]; ++ ARGV_t fattrs = fc->fattrs[fx]; ++ ++ if (fcolor != RPMFC_BLACK) + fprintf(fp, "\t0x%x", fc->fcolor[fx]); +- else ++ else + fprintf(fp, "\t%s", rpmstrPoolStr(fc->cdict, cx)); +- if (fattrs) { +- char *attrs = argvJoin(fattrs, ","); +- fprintf(fp, " [%s]", attrs); +- free(attrs); +- } else { +- fprintf(fp, " [none]"); ++ if (fattrs) { ++ char *attrs = argvJoin(fattrs, ","); ++ fprintf(fp, " [%s]", attrs); ++ free(attrs); ++ } else { ++ fprintf(fp, " [none]"); ++ } + } + fprintf(fp, "\n"); + +diff --git a/build/rpmfc.h b/build/rpmfc.h +index dae8ea5b1..3d87b31cf 100644 +--- a/build/rpmfc.h ++++ b/build/rpmfc.h +@@ -45,7 +45,6 @@ typedef const struct rpmfcTokens_s * rpmfcToken; + + /** \ingroup rpmfc + * Print results of file classification. +- * @todo Remove debugging routine. + * @param msg message prefix (NULL for none) + * @param fc file classifier + * @param fp output file handle (NULL for stderr) +diff --git a/tools/rpmdeps.c b/tools/rpmdeps.c +index a414b6343..f260a38c4 100644 +--- a/tools/rpmdeps.c ++++ b/tools/rpmdeps.c +@@ -23,6 +23,8 @@ static int print_conflicts; + + static int print_obsoletes; + ++static int print_alldeps; ++ + static void rpmdsPrint(const char * msg, rpmds ds, FILE * fp) + { + if (fp == NULL) fp = stderr; +@@ -57,6 +59,8 @@ static struct poptOption optionsTable[] = { + NULL, NULL }, + { "obsoletes", '\0', POPT_ARG_VAL, &print_obsoletes, -1, + NULL, NULL }, ++ { "alldeps", '\0', POPT_ARG_VAL, &print_alldeps, -1, ++ NULL, NULL }, + + POPT_AUTOALIAS + POPT_AUTOHELP +@@ -100,25 +104,27 @@ main(int argc, char *argv[]) + if (rpmfcClassify(fc, av, NULL) || rpmfcApply(fc)) + goto exit; + +- if (_rpmfc_debug) +- rpmfcPrint(buf, fc, NULL); +- +- if (print_provides) +- rpmdsPrint(NULL, rpmfcProvides(fc), stdout); +- if (print_requires) +- rpmdsPrint(NULL, rpmfcRequires(fc), stdout); +- if (print_recommends) +- rpmdsPrint(NULL, rpmfcRecommends(fc), stdout); +- if (print_suggests) +- rpmdsPrint(NULL, rpmfcSuggests(fc), stdout); +- if (print_supplements) +- rpmdsPrint(NULL, rpmfcSupplements(fc), stdout); +- if (print_enhances) +- rpmdsPrint(NULL, rpmfcEnhances(fc), stdout); +- if (print_conflicts) +- rpmdsPrint(NULL, rpmfcConflicts(fc), stdout); +- if (print_obsoletes) +- rpmdsPrint(NULL, rpmfcObsoletes(fc), stdout); ++ if (print_alldeps || _rpmfc_debug) ++ rpmfcPrint(NULL, fc, print_alldeps ? stdout : NULL); ++ ++ if (!print_alldeps) { ++ if (print_provides) ++ rpmdsPrint(NULL, rpmfcProvides(fc), stdout); ++ if (print_requires) ++ rpmdsPrint(NULL, rpmfcRequires(fc), stdout); ++ if (print_recommends) ++ rpmdsPrint(NULL, rpmfcRecommends(fc), stdout); ++ if (print_suggests) ++ rpmdsPrint(NULL, rpmfcSuggests(fc), stdout); ++ if (print_supplements) ++ rpmdsPrint(NULL, rpmfcSupplements(fc), stdout); ++ if (print_enhances) ++ rpmdsPrint(NULL, rpmfcEnhances(fc), stdout); ++ if (print_conflicts) ++ rpmdsPrint(NULL, rpmfcConflicts(fc), stdout); ++ if (print_obsoletes) ++ rpmdsPrint(NULL, rpmfcObsoletes(fc), stdout); ++ } + + ec = 0; + +-- +2.12.0 +
meta/recipes-devtools/rpm/rpm_git.bb +1 -0
@@ -37,6 +37,7 @@ SRC_URI = "git://github.com/rpm-software-management/rpm \ file://0001-Add-PYTHON_ABI-when-searching-for-python-libraries.patch \ file://0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch \ file://0012-Use-conditional-to-access-_docdir-in-macros.in.patch \ + file://0013-Add-a-new-option-alldeps-to-rpmdeps.patch \ " PV = "4.13.90+git${SRCPV}"
Signature · 1 line
2.12.0
PATCH 8/13 rpm: Simplify the creation of wrappers for the native tools
In reply to Peter KjellerstedtUse a loop rather than calling create_wrapper for each individual tool.
--- meta/recipes-devtools/rpm/rpm_git.bb | 73 ++++++++++-------------------------- 1 file changed, 19 insertions(+), 54 deletions(-)
meta/recipes-devtools/rpm/rpm_git.bb +19 -54
@@ -67,62 +67,27 @@ BBCLASSEXTEND = "native nativesdk" # Direct rpm-native to read configuration from our sysroot, not the one it was compiled in # libmagic also has sysroot path contamination, so override it do_install_append_class-native() { - create_wrapper ${D}/${bindir}/rpmbuild \ - RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \ - RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \ - MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \ - RPM_NO_CHROOT_FOR_SCRIPTS=1 - - create_wrapper ${D}/${bindir}/rpmsign \ - RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \ - RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \ - MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \ - RPM_NO_CHROOT_FOR_SCRIPTS=1 - - create_wrapper ${D}/${bindir}/rpmkeys \ - RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \ - RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \ - MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \ - RPM_NO_CHROOT_FOR_SCRIPTS=1 - - create_wrapper ${D}/${bindir}/rpm \ - RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \ - RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \ - MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \ - RPM_NO_CHROOT_FOR_SCRIPTS=1 - - create_wrapper ${D}/${bindir}/rpm2archive \ - RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \ - RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \ - MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \ - RPM_NO_CHROOT_FOR_SCRIPTS=1 - - create_wrapper ${D}/${bindir}/rpm2cpio \ - RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \ - RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \ - MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \ - RPM_NO_CHROOT_FOR_SCRIPTS=1 - - create_wrapper ${D}/${bindir}/rpmdb \ - RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \ - RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \ - MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \ - RPM_NO_CHROOT_FOR_SCRIPTS=1 - - create_wrapper ${D}/${bindir}/rpmgraph \ - RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \ - RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \ - MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \ - RPM_NO_CHROOT_FOR_SCRIPTS=1 - - create_wrapper ${D}/${bindir}/rpmspec \ - RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \ - RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \ - MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \ - RPM_NO_CHROOT_FOR_SCRIPTS=1 + tools="\ + ${bindir}/rpm \ + ${bindir}/rpm2archive \ + ${bindir}/rpm2cpio \ + ${bindir}/rpmbuild \ + ${bindir}/rpmdb \ + ${bindir}/rpmgraph \ + ${bindir}/rpmkeys \ + ${bindir}/rpmsign \ + ${bindir}/rpmspec \ + " + + for tool in $tools; do + create_wrapper ${D}$tool \ + RPM_CONFIGDIR=${STAGING_LIBDIR_NATIVE}/rpm \ + RPM_ETCCONFIGDIR=${STAGING_DIR_NATIVE} \ + MAGIC=${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc \ + RPM_NO_CHROOT_FOR_SCRIPTS=1 + done } - # Rpm's make install creates var/tmp which clashes with base-files packaging do_install_append_class-target() { rm -rf ${D}/var
Signature · 1 line
2.12.0
PATCH 6/13 toaster.bbclass: Simplify parsing of depends.dot
In reply to Peter KjellerstedtBy using a single regular expression, the parsing of the depends.dot file can be simplified a lot. This should also make it less susceptible to formatting changes in that file.
--- meta/classes/toaster.bbclass | 28 +++++++++++++--------------- 1 file changed, 13 insertions(+), 15 deletions(-)
meta/classes/toaster.bbclass +13 -15
@@ -270,22 +270,20 @@ python toaster_buildhistory_dump() { images[target][pname.strip()] = {'size':int(psize)*1024, 'depends' : []} with open("%s/depends.dot" % installed_img_path, "r") as fin: - p = re.compile(r' -> ') - dot = re.compile(r'.*style=dotted') + p = re.compile(r'\s*"(?P<name>[^"]+)"\s*->\s*"(?P<dep>[^"]+)"(?P<rec>.*?\[style=dotted\])?') for line in fin: - line = line.rstrip(';') - linesplit = p.split(line) - if len(linesplit) == 2: - pname = linesplit[0].rstrip('"').strip('"') - dependsname = linesplit[1].split(" ")[0].strip().strip(";").strip('"').rstrip('"') - deptype = "depends" - if dot.match(line): - deptype = "recommends" - if not pname in images[target]: - images[target][pname] = {'size': 0, 'depends' : []} - if not dependsname in images[target]: - images[target][dependsname] = {'size': 0, 'depends' : []} - images[target][pname]['depends'].append((dependsname, deptype)) + m = p.match(line) + if not m: + continue + pname = m.group('name') + dependsname = m.group('dep') + deptype = 'recommends' if m.group('rec') else 'depends' + + if not pname in images[target]: + images[target][pname] = {'size': 0, 'depends' : []} + if not dependsname in images[target]: + images[target][dependsname] = {'size': 0, 'depends' : []} + images[target][pname]['depends'].append((dependsname, deptype)) # files-in-image.txt is only generated if an image file is created, # so the file entries ('syms', 'dirs', 'files') for a target will be
Signature · 1 line
2.12.0
PATCH 10/13 rpm: Do not require that ELF binaries are executable to be identifiable
In reply to Peter KjellerstedtThere is nothing that requires, e.g., a DSO to be executable, but it is still an ELF binary and should be identified as such.
--- ...ire-that-ELF-binaries-are-executable-to-b.patch | 32 ++++++++++++++++++++++ meta/recipes-devtools/rpm/rpm_git.bb | 1 + 2 files changed, 33 insertions(+) create mode 100644 meta/recipes-devtools/rpm/files/0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch
meta/recipes-devtools/rpm/files/0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch new file +32 -0
@@ -0,0 +1,32 @@ +From d65d6e8760afbd7f70b22a1f3297a037bc475fea Mon Sep 17 00:00:00 2001 +From: Peter Kjellerstedt <[email protected]> +Date: Mon, 15 May 2017 10:21:08 +0200 +Subject: [PATCH 11/13] Do not require that ELF binaries are executable to be + identifiable + +There is nothing that requires, e.g., a DSO to be executable, but it +is still an ELF binary and should be identified as such. + +Upstream probably expects all ELF binaries to be marked as executable, +but rather than imposing such a limitation for OE, allow any file to +be identified as an ELF binary regardless of whether it is executable +or not. + +Upstream-Status: Inappropriate +Signed-off-by: Peter Kjellerstedt <[email protected]> +--- + fileattrs/elf.attr | 1 - + 1 file changed, 1 deletion(-) + +diff --git a/fileattrs/elf.attr b/fileattrs/elf.attr +index 595b33e09..bac52649d 100644 +--- a/fileattrs/elf.attr ++++ b/fileattrs/elf.attr +@@ -1,4 +1,3 @@ + %__elf_provides %{_rpmconfigdir}/elfdeps --provides %{?__filter_GLIBC_PRIVATE:--filter-private} + %__elf_requires %{_rpmconfigdir}/elfdeps --requires %{?__filter_GLIBC_PRIVATE:--filter-private} + %__elf_magic ^(setuid )?(setgid )?(sticky )?ELF (32|64)-bit.*$ +-%__elf_flags exeonly +-- +2.12.0 +
meta/recipes-devtools/rpm/rpm_git.bb +1 -0
@@ -35,6 +35,7 @@ SRC_URI = "git://github.com/rpm-software-management/rpm \ file://0001-Fix-build-with-musl-C-library.patch \ file://0001-Add-a-color-setting-for-mips64_n32-binaries.patch \ file://0001-Add-PYTHON_ABI-when-searching-for-python-libraries.patch \ + file://0011-Do-not-require-that-ELF-binaries-are-executable-to-b.patch \ " PV = "4.13.90+git${SRCPV}"
Signature · 1 line
2.12.0
Re: [PATCH 13/13] package.bbclass: Restore functionality to detect RPM dependencies
In reply to Peter KjellerstedtQuoted 7 lines · Peter Kjellerstedt
During the transition to rpm4, rpmdeps-oecore was replaced with the standard rpmdeps. However, what no one noticed was that unless rpmdeps is given options, e.g., -P or -R, to tell it what it should output, it will not output anything. Thus, it would do all the work to determine the requirements, but would keep silent about it. And since no output from rpmdeps is expected unless there are requirements, there were no warnings indicating that everything was not working as expected.
Apologies for that, however, this is not tested in any of our autobuilder scenarios or otherwise, and if it's not tested, then it's not broken. :-)
I would recommend you add some kind of test to this patchset to make sure it doesn't happen again (for example, when I would update rpm to a new upstream release).
Alex
Re: [PATCH 11/13] rpm: Use conditional to access %{_docdir} in macros.in
In reply to Peter KjellerstedtQuoted 6 lines · Peter Kjellerstedt
This avoids the following warning:
warning: Ignoring invalid regex %{_docdir}
when runing
rpmdeps -R <file>, since %{_docdir} is only defined when parsing a spec file.
Is this possible to do by adding a -D/--define option to rpmdeps? The less 'Inappropriate' patches we have to carry, the better.
Alex
Re: [PATCH 11/13] rpm: Use conditional to access %{_docdir} in macros.in
Quoted 10 lines · Alexander Kanavin
Quoted 6 lines · Peter Kjellerstedt
This avoids the following warning:
warning: Ignoring invalid regex %{_docdir}
when runing
rpmdeps -R <file>, since %{_docdir} is only defined when parsing a spec file.Is this possible to do by adding a -D/--define option to rpmdeps? The less 'Inappropriate' patches we have to carry, the better.
Nevermind, got this mixed up :)
Alex