More fixes for file dependency generation with RPM

This is the second take on restoring the file dependency support for RPM that was lost in Pyro. These changes should fix most (all?) problems that were seen on the autobuilders after the first set of changes were introduced.

The first patch is not strictly needed for this, but it fixes a build problem due to a renamed file in a patch file that I got when building core-image-sato. No idea why no one else has had problems with this...

//Peter

The following changes since commit 2bf2ab9fcd12b179c396d2d4b4603b774a37eef6:

bitbake: bitbake-user-manual: Updated BBLAYERS_FETCH_DIR variable description (2017-06-15 11:15:59 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib pkj/rpm_regressions2
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/rpm_regressions2

Peter Kjellerstedt (11):
gtk+3: Update the patches to work without PATCHTOOL = "git"
perl: Do not generate file dependencies for perl-ptest
rpm: Add dependencies on bash, perl and python3-core
linux-firmware: Avoid a dependency on python-core
texi2html: Add a dependency on perl
git: Add a dependency on perl for gitweb
git: Do not install git cvsserver and git svn
texinfo: Avoid a problem with a dependency on perl(Locale::gettext_xs) package_rpm.bbclass: Filter out unwanted file deps for nativesdk
packages
insane.bbclass: Ignore perl as dependency for nativesdk packages package.bbclass: Restore functionality to detect RPM dependencies

 meta/classes/insane.bbclass                        |   2 +-
 meta/classes/package.bbclass                       |   5 +-
 meta/classes/package_rpm.bbclass                   |  25 +-
 meta/lib/oe/package.py                             |  60 +++--
 meta/recipes-devtools/git/git.inc                  |  16 +-
 meta/recipes-devtools/perl/perl-ptest.inc          |   4 +
 meta/recipes-devtools/rpm/rpm_git.bb               |   5 +
 meta/recipes-extended/texi2html/texi2html_5.0.bb   |   2 +
 meta/recipes-extended/texinfo/texinfo_6.3.bb       |   6 +
 .../gtk+/gtk+3/0001-Hardcoded-libtool.patch        |  12 +-
 ...Do-not-try-to-initialize-GL-without-libGL.patch |   8 +-
 .../0003-Add-disable-opengl-configure-option.patch | 268 +++++++++++++++------
 ...4-configure.ac-Fix-wayland-protocols-path.patch |   8 +-
 .../linux-firmware/linux-firmware_git.bb           |   3 +
 14 files changed, 307 insertions(+), 117 deletions(-)

2.12.0

The patch "0003-Add-disable-opengl-configure-option.patch" used Git rename syntax, which is not supported by quilt.

---
 .../gtk+/gtk+3/0001-Hardcoded-libtool.patch        |  12 +-
 ...Do-not-try-to-initialize-GL-without-libGL.patch |   8 +-
 .../0003-Add-disable-opengl-configure-option.patch | 268 +++++++++++++++------
 ...4-configure.ac-Fix-wayland-protocols-path.patch |   8 +-
 4 files changed, 207 insertions(+), 89 deletions(-)

index 0bda1f1074..c210bbc7d5 100644
@@ -1,4 +1,4 @@
-From 5a73a5b92566e314bbc8a1bd40f1ec204837c111 Mon Sep 17 00:00:00 2001
+From 0ecaa5bab162abf0cb2057d77beeb7b89d5873b4 Mon Sep 17 00:00:00 2001
 From: Jussi Kukkonen <[email protected]>
 Date: Tue, 21 Jun 2016 14:53:56 +0300
 Subject: [PATCH 1/4] Hardcoded libtool

@@ -12,10 +12,10 @@ Signed-off-by: Jussi Kukkonen <[email protected]>
  1 file changed, 3 insertions(+), 3 deletions(-)
 
 diff --git a/configure.ac b/configure.ac
-index 23d3077..b6931d6 100644
+index 6628e21..f43ac09 100644
 --- a/configure.ac
 +++ b/configure.ac
-@@ -610,7 +610,7 @@ AC_MSG_CHECKING([whether to write dependencies into .pc files])
+@@ -617,7 +617,7 @@ AC_MSG_CHECKING([whether to write dependencies into .pc files])
  case $enable_explicit_deps in
    auto)
      export SED

@@ -24,7 +24,7 @@ index 23d3077..b6931d6 100644
      if test "x$deplibs_check_method" != xpass_all || test "x$enable_static" = xyes ; then
        enable_explicit_deps=yes
      else
-@@ -876,7 +876,7 @@ else
+@@ -895,7 +895,7 @@ else
      dnl Now we check to see if our libtool supports shared lib deps
      dnl (in a rather ugly way even)
      if $dynworks; then

@@ -33,7 +33,7 @@ index 23d3077..b6931d6 100644
          module_deplibs_check=`$module_libtool_config | \
              grep '^[[a-z_]]*check[[a-z_]]*_method=[['\''"]]' | \
              sed 's/.*[['\''"]]\(.*\)[['\''"]]$/\1/'`
-@@ -1630,7 +1630,7 @@ AC_SUBST(GTK_PRINT_BACKENDS)
+@@ -1649,7 +1649,7 @@ AC_SUBST(GTK_PRINT_BACKENDS)
  # We are using gmodule-no-export now, but I'm leaving the stripping
  # code in place for now, since pango and atk still require gmodule.
  export SED

@@ -43,5 +43,5 @@ index 23d3077..b6931d6 100644
    GDK_DEP_LIBS=`echo $GDK_DEP_LIBS | sed -e "s/$export_dynamic//"`
    GTK_DEP_LIBS=`echo $GTK_DEP_LIBS | sed -e "s/$export_dynamic//"`
 -- 
-2.1.4
+2.12.0
 

index 989716e513..ab7b659414 100644
@@ -1,4 +1,4 @@
-From d63b926f8c8d8b5c9f9ec33d078b775f0409d88a Mon Sep 17 00:00:00 2001
+From 2d2361f24be1e51201063dbbda570debe8702b6f Mon Sep 17 00:00:00 2001
 From: Jussi Kukkonen <[email protected]>
 Date: Fri, 16 Oct 2015 16:35:16 +0300
 Subject: [PATCH 2/4] Do not try to initialize GL without libGL

@@ -21,10 +21,10 @@ Signed-off-by: Jussi Kukkonen <[email protected]>
  2 files changed, 12 insertions(+)
 
 diff --git a/configure.ac b/configure.ac
-index b6931d6..e27da49 100644
+index f43ac09..965eec9 100644
 --- a/configure.ac
 +++ b/configure.ac
-@@ -342,6 +342,13 @@ AC_ARG_ENABLE(mir-backend,
+@@ -346,6 +346,13 @@ AC_ARG_ENABLE(mir-backend,
                                [enable the Mir gdk backend])],
  			      [backend_set=yes])
  

@@ -56,5 +56,5 @@ index 81479d8..3c8c5c0 100644
  
  gint
 -- 
-2.1.4
+2.12.0
 

index 634b3a46ff..aca55ceaf4 100644
@@ -1,4 +1,4 @@
-From 97b78e566634579bbae51be914aeaaa921137a8b Mon Sep 17 00:00:00 2001
+From 34f4d3928e8206a89b0a601bdf79d6194b3cc77e Mon Sep 17 00:00:00 2001
 From: Jussi Kukkonen <[email protected]>
 Date: Thu, 9 Jun 2016 11:21:36 +0300
 Subject: [PATCH 4/4] configure.ac: Fix wayland-protocols path

@@ -19,10 +19,10 @@ Signed-off-by: Jussi Kukkonen <[email protected]>
  1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/configure.ac b/configure.ac
-index a3e9beb..dde9dc5 100644
+index 00bf544..5515915 100644
 --- a/configure.ac
 +++ b/configure.ac
-@@ -461,7 +461,7 @@ fi
+@@ -465,7 +465,7 @@ fi
  if test "$enable_wayland_backend" = "yes"; then
    # For the cairo image backend
    cairo_backends="$cairo_backends cairo"

@@ -32,5 +32,5 @@ index a3e9beb..dde9dc5 100644
    GDK_WINDOWING="$GDK_WINDOWING
  #define GDK_WINDOWING_WAYLAND"
 -- 
-2.1.4
+2.12.0
 

2.12.0

But quilt is the default, so how did this ever work?

Ross

I have absolutely no idea…

//Peter

From: Burton, Ross [mailto:[email protected]]
Sent: den 15 juni 2017 15:58
To: Peter Kjellerstedt <[email protected]>
Cc: OE-core <[email protected]>
Subject: Re: [OE-core] [PATCH 01/11] gtk+3: Update the patches to work without PATCHTOOL = "git"

On 15 June 2017 at 14:53, Peter Kjellerstedt <[email protected]<mailto:[email protected]>> wrote:
The patch "0003-Add-disable-opengl-configure-option.patch" used Git
rename syntax, which is not supported by quilt.

But quilt is the default, so how did this ever work?

Ross

My apologies, but I do not think this is a satisfactory answer: it may mean you are fixing the wrong problem, or a problem that does not exist.

Alex

Well, I am pretty sure Git patches that rename files are not supposed to be used, since quilt does not support this AFAIK. So the change in this patch should be correct.

However, I would be more than happy if anyone can explain why others are not seeing this (or does everyone have PATCHTOOL = "git" in their configurations somehow?)

//Peter

This is unlikely; I certainly don't. Perhaps if you provide the exact command that is being executed, and what is the error it prints, I could try to do the same?

Alex

Well, I was running "bitbake core-image-sato", but "bitbake gtk+3" should do...

//Peter

What I wanted to see is the failing command (quilt, or patch, or what is it specifically), and the error you get. gtk+3 is patched and built here with no issues at all.

Alex

It turns out the problem is only seen if opengl is enabled in DISTRO_FEATURES. However, since that is the default for poky, that still does not explain why others do not see the problem

I have included the log.do_compile file where one can see that the file gdkx-with-gl-context.h is missing (that is the file that was supposed to be renamed from gdkx.h by the patch). When I look in tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/gtk+-3.22.15/gdk/x11 I see gdkx.h and gdkx-without-gl-context.h, instead of the expected gdkx-with-gl-context.h and gdkx-without-gl-context.h...

//Peter

DEBUG: SITE files ['endian-little', 'bit-32', 'ix86-common', 'common-linux', 'common-glibc', 'i586-linux', 'common']
DEBUG: Executing shell function do_compile
+ cd /home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build
+ do_compile
+ export GIR_EXTRA_LIBS_PATH=/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build/gdk/.libs
+ GIR_EXTRA_LIBS_PATH=/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build/gdk/.libs
+ qemu_binary='PSEUDO_UNLOAD=1 qemu-i386 -r 3.2.0 -L /home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/recipe-sysroot -E LD_LIBRARY_PATH=$GIR_EXTRA_LIBS_PATH:/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/recipe-sysroot//usr/lib:/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/recipe-sysroot//lib '
+ cat
+ chmod +x /home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build/gtkdoc-qemuwrapper
+ autotools_do_compile
+ oe_runmake
+ oe_runmake_call
+ bbnote make -j 8
+ '[' -p /home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/temp/fifo.10262 ']'
NOTE: make -j 8
+ printf '%b\0' 'bbnote make -j 8'
+ make -j 8
make all-recursive
make[1]: Entering directory `/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build'
Making all in po
make[2]: Entering directory `/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build/po'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build/po'
Making all in po-properties
make[2]: Entering directory `/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build/po-properties'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build/po-properties'
Making all in gdk
Failed to open file “./gdk.gresource.xml”: No such file or directory
make[2]: Entering directory `/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build/gdk'

( cd ../../gtk+-3.22.15/gdk && glib-mkenums --template gdkenumtypes.h.template \
	gdk.h gdk-autocleanup.h gdkapplaunchcontext.h gdkcairo.h gdkcursor.h gdkdevice.h gdkdevicepad.h gdkdevicetool.h gdkdevicemanager.h gdkdisplay.h gdkdisplaymanager.h gdkdnd.h gdkdrawingcontext.h gdkevents.h gdkframetimings.h gdkglcontext.h gdkkeys.h gdkkeysyms.h gdkkeysyms-compat.h gdkmain.h gdkmonitor.h gdkpango.h gdkframeclock.h gdkpixbuf.h gdkprivate.h gdkproperty.h gdkrectangle.h gdkrgba.h gdkscreen.h gdkseat.h gdkselection.h gdktestutils.h gdkthreads.h gdktypes.h gdkvisual.h gdkwindow.h deprecated/gdkcolor.h ) >> xgen-geth \

&& (cmp -s xgen-geth gdkenumtypes.h || cp xgen-geth gdkenumtypes.h ) \
&& rm -f xgen-geth \
&& echo timestamp > stamp-gdkenumtypes.h
echo "<?xml version='1.0' encoding='UTF-8'?>" > gdk.gresource.xml; \
echo "<gresources>" >> gdk.gresource.xml; \
echo " <gresource prefix='/org/gtk/libgdk'>" >> gdk.gresource.xml; \
for f in ../../gtk+-3.22.15/gdk/resources/glsl/gles2-texture.fs.glsl ../../gtk+-3.22.15/gdk/resources/glsl/gl2-texture-2d.fs.glsl ../../gtk+-3.22.15/gdk/resources/glsl/gl2-texture-2d.vs.glsl ../../gtk+-3.22.15/gdk/resources/glsl/gl2-texture-rect.fs.glsl ../../gtk+-3.22.15/gdk/resources/glsl/gles2-texture.vs.glsl ../../gtk+-3.22.15/gdk/resources/glsl/gl3-texture-rect.fs.glsl ../../gtk+-3.22.15/gdk/resources/glsl/gl2-texture-rect.vs.glsl ../../gtk+-3.22.15/gdk/resources/glsl/gl3-texture-2d.vs.glsl ../../gtk+-3.22.15/gdk/resources/glsl/gl3-texture-2d.fs.glsl ../../gtk+-3.22.15/gdk/resources/glsl/gl3-texture-rect.vs.glsl; do \
n=basename $f; \
echo " <file alias='glsl/$n'>resources/glsl/$n</file>" >> gdk.gresource.xml; \
done; \
echo " </gresource>" >> gdk.gresource.xml; \
echo "</gresources>" >> gdk.gresource.xml;
cd .. && /bin/sh ./config.status gdk/gdkconfig.h \
&& echo timestamp > gdk/stamp-gc-h

glib-compile-resources gdk.gresource.xml --target=gdkresources.h \
	--sourcedir=../../gtk+-3.22.15/gdk --c-name _gdk --generate-header --manual-register
glib-compile-resources gdk.gresource.xml --target=gdkresources.c \
	--sourcedir=../../gtk+-3.22.15/gdk --c-name _gdk --generate-source --manual-register

config.status: executing gdk/gdkconfig.h commands
config.status: gdk/gdkconfig.h is unchanged
if test -f gdkconfig.h; then :; \
else rm -f stamp-gc-h; make stamp-gc-h; fi
make all-recursive
make[3]: Entering directory `/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build/gdk'
Making all in x11
make[4]: Entering directory `/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build/gdk/x11'
cd ../../../gtk+-3.22.15/gdk/x11 \
&& (cmp -s gdkx-with-gl-context.h gdkx.h || cp gdkx-with-gl-context.h gdkx.h )
cp: cannot stat `gdkx-with-gl-context.h': No such file or directory
make[4]: *** [gdkx.h] Error 1
make[4]: Leaving directory `/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build/gdk/x11'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory `/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build/gdk'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build/gdk'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/build'
make: *** [all] Error 2
+ die 'oe_runmake failed'
+ bbfatal_log 'oe_runmake failed'
+ '[' -p /home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/temp/fifo.10262 ']'
ERROR: oe_runmake failed
+ printf '%b\0' 'bbfatal_log oe_runmake failed'
+ exit 1
+ bb_exit_handler
+ ret=1
+ case $ret in
+ case $BASH_VERSION in
+ echo 'WARNING: /home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/temp/run.do_compile.10262:1 exit 1 from '\''exit 1'\'''
WARNING: /home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/temp/run.do_compile.10262:1 exit 1 from 'exit 1'
+ exit 1
ERROR: Function failed: do_compile (log file is located at /home/pkj/dists/poky-master/builds/qemux86/tmp/work/i586-poky-linux/gtk+3/3.22.15-r0/temp/log.do_compile.10262)

I guess Jussi needs to look at this, as he's maintaining the patches and the recipe.

Alex

Ok, I have finally figured out what is going on. The problem is the version of patch that is installed on the host. Support for Git style patches was added to patch in version 2.7, but on the host where I failed to build gtk+3, the installed version of patch is 2.6.1...

However, since this is the only patch in all of Poky and OpenEmbedded that uses the Git style format, I still think the correct thing to do is to apply my change. It does not change the contents of the patch, only the format.

I will send an updated change with an updated commit message to clarify what is going on.

//Peter

Good to know!

Thanks,
Ross

By default (i.e. nothing special in my local.conf), I have
PATCHTOOL = "quilt"

------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

The perl-ptest package contains Perl internal modules and generating file dependencies for it causes problems.

---
 meta/recipes-devtools/perl/perl-ptest.inc | 4 ++++
 1 file changed, 4 insertions(+)

index 04406d5744..1f549af1b3 100644
@@ -39,3 +39,7 @@ python populate_packages_prepend() {
 }
 
 RDEPENDS_${PN}-ptest += "${PN}-modules ${PN}-doc ${PN}-misc sed libssp"
+
+# The perl-ptest package contains Perl internal modules and generating file
+# dependencies for it causes problems.
+SKIP_FILEDEPS_${PN}-ptest = '1'

2.12.0

Also modify a Python script (pythondistdeps.py) to use Python 3.

---
 meta/recipes-devtools/rpm/rpm_git.bb | 5 +++++
 1 file changed, 5 insertions(+)

index 2a0dcafac7..30702a7507 100644
@@ -103,6 +103,9 @@ do_install_append_class-target() {
 
 do_install_append () {
 	sed -i -e 's:${HOSTTOOLS_DIR}/::g' ${D}/${libdir}/rpm/macros
+
+	sed -i -e 's|/usr/bin/python|${USRBINPATH}/env ${PYTHON_PN}|' \
+	    ${D}${libdir}/rpm/pythondistdeps.py
 }
 
 FILES_${PN} += "${libdir}/rpm-plugins/*.so \

@@ -117,3 +120,5 @@ FILES_python3-rpm = "${PYTHON_SITEPACKAGES_DIR}/rpm/*"
 
 # rpm 5.x was packaging the rpm build tools separately
 RPROVIDES_${PN} += "rpm-build"
+
+RDEPENDS_${PN} = "bash perl python3-core"

2.12.0

Remove the check_whence.py script since it is only needed to validate the WHENCE file, and only if explicitly running make check.

---
 meta/recipes-kernel/linux-firmware/linux-firmware_git.bb | 3 +++
 1 file changed, 3 insertions(+)

index 4558dafdac..53089f2767 100644
@@ -210,6 +210,9 @@ do_install() {
 	# Remove pointless bash script
 	rm ${D}${nonarch_base_libdir}/firmware/configure
 
+	# Remove python script used to check the WHENCE file
+	rm ${D}${nonarch_base_libdir}/firmware/check_whence.py
+
 	# Libertas sd8686
 	ln -sf libertas/sd8686_v9.bin ${D}${nonarch_base_libdir}/firmware/sd8686.bin
 	ln -sf libertas/sd8686_v9_helper.bin ${D}${nonarch_base_libdir}/firmware/sd8686_helper.bin

2.12.0

---
 meta/recipes-extended/texi2html/texi2html_5.0.bb | 2 ++
 1 file changed, 2 insertions(+)

index ae64816f82..afd125b08f 100644
@@ -27,3 +27,5 @@ do_install_append () {
 }
 
 FILES_${PN}-doc += "${datadir}/texinfo"
+
+RDEPENDS_${PN} = "perl"

2.12.0

---
 meta/recipes-devtools/git/git.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index ade99ecfe0..9b0d328bad 100644
@@ -118,6 +118,6 @@ FILES_${PN}-tk = " \
 
 PACKAGES =+ "gitweb"
 FILES_gitweb = "${datadir}/gitweb/"
-
+RDEPENDS_gitweb = "perl"
 
 BBCLASSEXTEND = "native nativesdk"

2.12.0

These git commands require Perl modules that do not exist in OE-Core.

---
 meta/recipes-devtools/git/git.inc | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

index 9b0d328bad..3ce4642ae7 100644
@@ -54,6 +54,17 @@ perl_native_fixup () {
 	mkdir -p ${D}${libdir}
 	mv ${D}${exec_prefix}/lib/perl-native/perl ${D}${libdir}
 	rmdir -p ${D}${exec_prefix}/lib/perl-native || true
+
+	# Remove git cvsserver as it requires the DBI Perl module, which does
+	# not exist in OE-Core.
+	rm ${D}${libexecdir}/git-core/git-cvsserver \
+	   ${D}${bindir}/git-cvsserver
+
+	# Remove git svn and all Git::SVN Perl modules as they require the
+	# SVN::Core Perl module, which does not exist in OE-Core.
+	rm -r ${D}${libexecdir}/git-core/git-svn \
+	      ${D}${libdir}/perl/site_perl/*/Git/SVN*
+	sed -i -e '/SVN/d' ${D}${libdir}/perl/site_perl/*/auto/Git/.packlist
 }
 
 REL_GIT_EXEC_PATH = "${@os.path.relpath(libexecdir, bindir)}/git-core"

@@ -83,12 +94,9 @@ PERLTOOLS = " \
     ${libexecdir}/git-core/git-archimport \
     ${libexecdir}/git-core/git-cvsexportcommit \
     ${libexecdir}/git-core/git-cvsimport \
-    ${libexecdir}/git-core/git-cvsserver \
-    ${bindir}/git-cvsserver \
     ${libexecdir}/git-core/git-difftool \
     ${libexecdir}/git-core/git-relink \
     ${libexecdir}/git-core/git-send-email \
-    ${libexecdir}/git-core/git-svn \
     ${libexecdir}/git-core/git-instaweb \
     ${libexecdir}/git-core/git-submodule \
     ${libexecdir}/git-core/git-am \

2.12.0

Is this a good reason to remove the binaries though? What if someone added the modules and wanted to use these binaries?

Ross

Oh, the new filedeps logic will add module-level dependencies.

Maybe PACKAGECONFIG these?

Ross

Well, I guess I can do that…

//Peter

From: Burton, Ross [mailto:[email protected]]
Sent: den 15 juni 2017 16:01
To: Peter Kjellerstedt <[email protected]>
Cc: OE-core <[email protected]>
Subject: Re: [OE-core] [PATCH 07/11] git: Do not install git cvsserver and git svn

On 15 June 2017 at 14:59, Burton, Ross <[email protected]<mailto:[email protected]>> wrote:
Is this a good reason to remove the binaries though? What if someone added the modules and wanted to use these binaries?

Oh, the new filedeps logic will add module-level dependencies.

Maybe PACKAGECONFIG these?

Ross

We do not build the Locale::gettext_xs Perl module and the code will test for it and happily use Locale::gettext_pp instead if it is not found. However, this still causes a file dependency on perl(Locale::gettext_xs) to be generated, which must be satisfied by adding an explicit provide for it.

---
 meta/recipes-extended/texinfo/texinfo_6.3.bb | 6 ++++++
 1 file changed, 6 insertions(+)

index d82731e11e..f58df928aa 100644
@@ -79,4 +79,10 @@ FILES_${PN}-doc = "${infodir}/texinfo* \
                    ${datadir}/${tex_texinfo} \
                    ${mandir}/man1 ${mandir}/man5"
 
+# Lie about providing the Locale::gettext_xs module. It is not actually built,
+# but the code will test for it and if not found use Locale::gettext_pp instead.
+# However, this causes a file dependency on perl(Locale::gettext_xs) to be
+# generated, which must be satisfied.
+RPROVIDES_${PN} += "perl(Locale::gettext_xs)"
+
 BBCLASSEXTEND = "native nativesdk"

2.12.0

Filter out any file dependencies on absolute paths and any dependencies on Perl modules for nativesdk packages. It is assumed that they will be provided by the native host if needed, and they mess up the dependency handling if they are present.

---
 meta/classes/package_rpm.bbclass | 25 +++++++++++++++++++++++--
 1 file changed, 23 insertions(+), 2 deletions(-)

index dc241975c3..5f29b45180 100644
@@ -7,10 +7,28 @@ RPMBUILD="rpmbuild"
 
 PKGWRITEDIRRPM = "${WORKDIR}/deploy-rpms"
 
-# Maintaining the perfile dependencies has singificant overhead when writing the 
+# Maintaining the perfile dependencies has singificant overhead when writing the
 # packages. When set, this value merges them for efficiency.
 MERGEPERFILEDEPS = "1"
 
+# Filter dependencies based on a provided function.
+def filter_deps(var, f):
+    import collections
+
+    depends_dict = bb.utils.explode_dep_versions2(var)
+    newdeps_dict = collections.OrderedDict()
+    for dep in depends_dict:
+        if f(dep):
+            newdeps_dict[dep] = depends_dict[dep]
+    return bb.utils.join_deps(newdeps_dict, commasep=False)
+
+# Filter out absolute paths (typically /bin/sh and /usr/bin/env) and any perl
+# dependencies for nativesdk packages.
+def filter_nativesdk_deps(srcname, var):
+    if var and srcname.startswith("nativesdk-"):
+        var = filter_deps(var, lambda dep: not dep.startswith('/') and dep != 'perl' and not dep.startswith('perl('))
+    return var
+
 # Construct per file dependencies file
 def write_rpm_perfiledata(srcname, d):
     workdir = d.getVar('WORKDIR')

@@ -26,7 +44,8 @@ def write_rpm_perfiledata(srcname, d):
             dependsflist = (d.getVar(dependsflist_key) or "")
             for dfile in dependsflist.split():
                 key = "FILE" + varname + "_" + dfile + "_" + pkg
-                depends_dict = bb.utils.explode_dep_versions(d.getVar(key) or "")
+                deps = filter_nativesdk_deps(srcname, d.getVar(key) or "")
+                depends_dict = bb.utils.explode_dep_versions(deps)
                 file = dfile.replace("@underscore@", "_")
                 file = file.replace("@closebrace@", "]")
                 file = file.replace("@openbrace@", "[")

@@ -359,6 +378,8 @@ python write_specfile () {
             splitrdepends = splitrdepends + " " + get_perfile('RDEPENDS', pkg, d)
             splitrprovides = splitrprovides + " " + get_perfile('RPROVIDES', pkg, d)
 
+        splitrdepends = filter_nativesdk_deps(srcname, splitrdepends)
+
         # Gather special src/first package data
         if srcname == splitname:
             srcrdepends    = splitrdepends

2.12.0

Does this mean you can also drop the "bin/sh" hack from here in the same file?

print_deps(srcrprovides + (" /bin/sh" if
srcname.startswith("nativesdk-") else ""), "Provides", spec_preamble_top, d)

Alex

No, unfortunately not. I tried it, but it turns out rpm adds the dependency itself even if it is not specified in the .spec file due to the postinst scriptlet.

//Peter

---
 meta/classes/insane.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index f6fd7c74d8..d57b98e04c 100644
@@ -906,7 +906,7 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d):
         if "file-rdeps" not in skip:
             ignored_file_rdeps = set(['/bin/sh', '/usr/bin/env', 'rtld(GNU_HASH)'])
             if bb.data.inherits_class('nativesdk', d):
-                ignored_file_rdeps |= set(['/bin/bash', '/usr/bin/perl'])
+                ignored_file_rdeps |= set(['/bin/bash', '/usr/bin/perl', 'perl'])
             # For Saving the FILERDEPENDS
             filerdepends = {}
             rdep_data = oe.packagedata.read_subpkgdata(pkg, d)

2.12.0

During the transition to dnf and rpm4, the functionality to automatically make RPM determine dependencies was lost.

Before the transition, an OE specific tool called rpmdeps-oecore had been added to the rpm suit. It was based on the rpmdeps tool that is part of rpm. For each file specified on its command line, it would output the provides and requires that RPM could determine.

During the transition to rpm4, rpmdeps-oecore was replaced with the standard rpmdeps. However, what no one noticed was that unless rpmdeps is given options, e.g., -P or -R, to tell it what it should output, it will not output anything. Thus, it would do all the work to determine the requirements, but would keep silent about it. And since no output from rpmdeps is expected unless there are requirements, there were no warnings indicating that everything was not working as expected.

Porting the old rpmdeps-oecore to work with rpm4 is not really possible since it relied on being able to access internals of RPM that are no longer available. However, it turned out that rpmdeps had a debug option, --rpmfcdebug, that would output exactly the information that we need, albeit in a different format and to stderr. To make this usable, rpmdeps has now received a new option, --alldeps, which sends the information we need to stdout.

---
 meta/classes/package.bbclass |  5 ++--
 meta/lib/oe/package.py       | 60 +++++++++++++++++++++++++++++++-------------
 2 files changed, 44 insertions(+), 21 deletions(-)

index cc466bd1b2..65cf5fac5f 100644
@@ -1434,7 +1434,7 @@ if [ x"$D" = "x" ]; then
 fi
 }
 
-RPMDEPS = "${STAGING_LIBDIR_NATIVE}/rpm/rpmdeps --rcfile ${STAGING_LIBDIR_NATIVE}/rpm/rpmrc  --macros ${STAGING_LIBDIR_NATIVE}/rpm/macros --define '_rpmconfigdir ${STAGING_LIBDIR_NATIVE}/rpm/'"
+RPMDEPS = "${STAGING_LIBDIR_NATIVE}/rpm/rpmdeps --alldeps"
 
 # Collect perfile run-time dependency metadata
 # Output:

@@ -1451,7 +1451,6 @@ python package_do_filedeps() {
     pkgdest = d.getVar('PKGDEST')
     packages = d.getVar('PACKAGES')
     rpmdeps = d.getVar('RPMDEPS')
-    magic = d.expand("${STAGING_DIR_NATIVE}${datadir_native}/misc/magic.mgc")
 
     def chunks(files, n):
         return [files[i:i+n] for i in range(0, len(files), n)]

@@ -1463,7 +1462,7 @@ python package_do_filedeps() {
         if pkg.endswith('-dbg') or pkg.endswith('-doc') or pkg.find('-locale-') != -1 or pkg.find('-localedata-') != -1 or pkg.find('-gconv-') != -1 or pkg.find('-charmap-') != -1 or pkg.startswith('kernel-module-'):
             continue
         for files in chunks(pkgfiles[pkg], 100):
-            pkglist.append((pkg, files, rpmdeps, pkgdest, magic))
+            pkglist.append((pkg, files, rpmdeps, pkgdest))
 
     processed = oe.utils.multiprocess_exec( pkglist, oe.package.filedeprunner)
 

2.12.0

I'd like you to address my previous comment: how do you ensure this will continue to work going forward? Is it taken into use by default, and if it breaks, will we notice?

Alex

Yes, once the last commit that changes package.bbclass is integrated, it will be activated for all builds that use RPM. And given how red the autobuilders got when RP did it the first time, I do not think there is any chance we will miss problems due to it...

I do agree that some kind of tests for this would probably be good. However, I do not have any experience with the oe-selftests, and how to implement some tests for this, and I am pressed for time as I go on summer vacation next Friday and would very much like for this to be fixed before then. Especially as we need these changes to make it into Pyro as well...

//Peter