core: ditch LD_LIBRARY_PATH (branch yem/no-ld-library-path)

mkfs.axfs uses zlib, but does not have an rpath to our host dir.

That's because:
- we're not passing our host CFLAGS or LDFLAGS
- it is forcibly setting CFLAGS in the Makefile, overriding anything specified by the user
- it is not using LDFLAGS at all

Add two patches so that CFLAGS and LDFLAGS from the environment are used if present.

Signed-off-by "Yann E. MORIN" <[email protected]>

---
 package/axfsutils/0001-fix-cflags.patch  | 19 +++++++++++++++++++
 package/axfsutils/0002-use-ldflags.patch | 20 ++++++++++++++++++++
 package/axfsutils/axfsutils.mk           |  2 +-
 3 files changed, 40 insertions(+), 1 deletion(-)
 create mode 100644 package/axfsutils/0001-fix-cflags.patch
 create mode 100644 package/axfsutils/0002-use-ldflags.patch

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

mkfs.axfs uses zlib, but does not have an rpath to our host dir. That's because:
- we're not passing our host CFLAGS or LDFLAGS
- it is forcibly setting CFLAGS in the Makefile, overriding anything specified by the user
- it is not using LDFLAGS at all

Add two patches so that CFLAGS and LDFLAGS from the environment are used if present.

Signed-off-by "Yann E. MORIN" <[email protected]>

To build the host-mysql, we only build parts of the source, just the strictly minimum required to then cross-compile it.

However, the host variables (conf opts, build and install cmds) are only defined when the mysql server is enabled in the configuration.

So, this breaks:
make defconfig; make host-mysql

Even though it is not much use to have that partial host-mysql on its own, it is still very interesting to be able to build it, if at least for testing changes in the core package infrastructures (like new step hooks or the likes...)

Move the definitions of the host variant out of the server conditional block.

Signed-off-by "Yann E. MORIN" <[email protected]>

Cc Thomas Petazzoni <[email protected]>

Cc Marcelo Guti?rrez(UTN/FRH) <[email protected]>

---
 package/mysql/mysql.mk | 39 ++++++++++++++++++++-------------------
 1 file changed, 20 insertions(+), 19 deletions(-)

package/mysql/mysql.mk +20 -19

index c6d9f3a..00fab85 100644
@@ -33,6 +33,26 @@ MYSQL_CONF_OPTS = \
 	--enable-thread-safe-client \
 	--disable-mysql-maintainer-mode
 
+HOST_MYSQL_DEPENDENCIES = host-zlib host-ncurses
+
+HOST_MYSQL_CONF_OPTS = \
+	--with-embedded-server \
+	--disable-mysql-maintainer-mode
+
+define HOST_MYSQL_BUILD_CMDS
+	$(MAKE) -C $(@D)/include my_config.h
+	$(MAKE) -C $(@D)/mysys libmysys.a
+	$(MAKE) -C $(@D)/strings libmystrings.a
+	$(MAKE) -C $(@D)/vio libvio.a
+	$(MAKE) -C $(@D)/dbug libdbug.a
+	$(MAKE) -C $(@D)/regex libregex.a
+	$(MAKE) -C $(@D)/sql gen_lex_hash
+endef
+
+define HOST_MYSQL_INSTALL_CMDS
+	$(INSTALL) -m 0755  $(@D)/sql/gen_lex_hash $(HOST_DIR)/usr/bin/
+endef
+
 ifeq ($(BR2_PACKAGE_OPENSSL),y)
 MYSQL_DEPENDENCIES += openssl
 endif

@@ -43,11 +63,6 @@ endif
 
 ifeq ($(BR2_PACKAGE_MYSQL_SERVER),y)
 MYSQL_DEPENDENCIES += host-mysql host-bison
-HOST_MYSQL_DEPENDENCIES = host-zlib host-ncurses
-
-HOST_MYSQL_CONF_OPTS = \
-	--with-embedded-server \
-	--disable-mysql-maintainer-mode
 
 MYSQL_CONF_OPTS += \
 	--localstatedir=/var/mysql \

@@ -74,20 +89,6 @@ else
 MYSQL_CONF_OPTS += --without-debug
 endif
 
-define HOST_MYSQL_BUILD_CMDS
-	$(MAKE) -C $(@D)/include my_config.h
-	$(MAKE) -C $(@D)/mysys libmysys.a
-	$(MAKE) -C $(@D)/strings libmystrings.a
-	$(MAKE) -C $(@D)/vio libvio.a
-	$(MAKE) -C $(@D)/dbug libdbug.a
-	$(MAKE) -C $(@D)/regex libregex.a
-	$(MAKE) -C $(@D)/sql gen_lex_hash
-endef
-
-define HOST_MYSQL_INSTALL_CMDS
-	$(INSTALL) -m 0755  $(@D)/sql/gen_lex_hash  $(HOST_DIR)/usr/bin/
-endef
-
 define MYSQL_USERS
 	mysql -1 nogroup -1 * /var/mysql - - MySQL daemon
 endef

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

To build the host-mysql, we only build parts of the source, just the strictly minimum required to then cross-compile it.

However, the host variables (conf opts, build and install cmds) are only defined when the mysql server is enabled in the configuration.

So, this breaks:
make defconfig; make host-mysql

Even though it is not much use to have that partial host-mysql on its own, it is still very interesting to be able to build it, if at least for testing changes in the core package infrastructures (like new step hooks or the likes...)

Move the definitions of the host variant out of the server conditional block.

Signed-off-by "Yann E. MORIN" <[email protected]>

Cc Thomas Petazzoni <[email protected]>

Cc Marcelo Guti?rrez(UTN/FRH) <[email protected]>

It does not build, and no one depends on it.

Signed-off-by "Yann E. MORIN" <[email protected]>

Cc Francois Perrad <[email protected]>

---
 package/perl-file-util/perl-file-util.mk | 1 -
 1 file changed, 1 deletion(-)

package/perl-file-util/perl-file-util.mk +0 -1

index 2712aa4..fe83585 100644
@@ -12,4 +12,3 @@ PERL_FILE_UTIL_LICENSE = Artistic or GPLv1+
 PERL_FILE_UTIL_LICENSE_FILES = COPYING LICENSE
 
 $(eval $(perl-package))
-$(eval $(host-perl-package))

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

It does not build, and no one depends on it.

Signed-off-by "Yann E. MORIN" <[email protected]>

Cc Francois Perrad <[email protected]>

To build libcurl, we need to override LD_LIBRARY and force it to a sane value, otherwise libcurl is confused when target == host (see a51ce319,
libcurl: fix configure with openssl when target == host).

That is currently OK, since we always set LD_LIBRARY_PATH to a non-empty value.

However, we're soon to stop setting it at all.

So, if the user has an empty (or no) LD_LIBRARY_PATH in his envirnment, we'd end up adding the current working directory to LD_LIBRARY_PATH (as an empty entry in a colon-separated list is most probably interpreted as meaning the currentworking directory, which we do know can cause issue, and which we expfressely check against in support/dependencies/dependencies.sh

Fix that by only using an existing LD_LIBRARY_PATH if it is not empty. Also use a Makefile construct as it is easier to read than a shell one (we can do that, as all variables from the environment as available as make variables).

Signed-off-by "Yann E. MORIN" <[email protected]>

Cc Peter Korsgaard <[email protected]>

Cc Thomas Petazzoni <[email protected]>

---
 package/libcurl/libcurl.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

package/libcurl/libcurl.mk +1 -1

index 8587baa..7c259b2 100644
@@ -30,7 +30,7 @@ LIBCURL_CONF_ENV += ac_cv_lib_crypto_CRYPTO_lock=yes
 # native stuff during the rest of configure when target == host.
 # Fix it by setting LD_LIBRARY_PATH to something sensible so those libs
 # are found first.
-LIBCURL_CONF_ENV += LD_LIBRARY_PATH=$$LD_LIBRARY_PATH:/lib:/usr/lib
+LIBCURL_CONF_ENV += LD_LIBRARY_PATH=$(if $(LD_LIBRARY_PATH),$(LD_LIBRARY_PATH):)/lib:/usr/lib
 LIBCURL_CONF_OPTS += --with-ssl=$(STAGING_DIR)/usr \
 	--with-ca-path=/etc/ssl/certs
 else ifeq ($(BR2_PACKAGE_GNUTLS),y)

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

To build libcurl, we need to override LD_LIBRARY and force it to a sane value, otherwise libcurl is confused when target == host (see a51ce319,
libcurl: fix configure with openssl when target == host).

That is currently OK, since we always set LD_LIBRARY_PATH to a non-empty value.

However, we're soon to stop setting it at all.

So, if the user has an empty (or no) LD_LIBRARY_PATH in his envirnment, we'd end up adding the current working directory to LD_LIBRARY_PATH (as an empty entry in a colon-separated list is most probably interpreted as meaning the currentworking directory, which we do know can cause issue, and which we expfressely check against in support/dependencies/dependencies.sh

Fix that by only using an existing LD_LIBRARY_PATH if it is not empty. Also use a Makefile construct as it is easier to read than a shell one (we can do that, as all variables from the environment as available as make variables).

Signed-off-by "Yann E. MORIN" <[email protected]>

Cc Peter Korsgaard <[email protected]>

Cc Thomas Petazzoni <[email protected]>

When we build our host programs, and they depend on a host library we also build, we want to ensure that program actually uses that library at runtime, and not the one from the system.

We currently ensure that in two ways:
- we add a RPATH tag that points to our host library directory,
- we export LD_LIBRARY_PATH to point to that same directory.

With thse two in place, we're pretty much confident that our host libraries will be used by our host programs.

However, it turns our that not all the host programs we build end up with an RPATH tag:
- some packages do not use our $(HOST_LDFLAGS)
- some packages' build system are oblivious to those LDFLAGS

In this case, there are two situation:
- the program is not linked to one of our host libraries: it in fact does not need an RPATH tag [0]
- the program actually uses one of our host libraries: in that case it should have had an RPATH tag pointing to the host directory.

As for libraries, it is unclear whether they should or should not have an RPATH pointing to our host directory. as for programs, it is only important they have such an RPATH if they have a dependency on another host lbrary we build. But even though, in practice this is not an issue, because the program that loads such a libray does have an RPATH (it did find that library!), so the RPATH from the program is also used to search for second-level (and third-level...) dependencies, as well as for libraries loaded via dlopen().

We add a new support script that checks that all ELF executables have a proper DT_RPATH (or DT_RUNPATH) tag when they link to our host libraries, and reports those file that are missing an RPATH. If a file missing an RPATH is an executable, the script aborts; if only libraries are are missing an RPATH, the script does not abort.

[0] Except if it were to dlopen() it, of course, but the only program I'm aware of that does that is openssl, and it has a correct RPATH tag.

Signed-off-by "Yann E. MORIN" <[email protected]>

Cc Thomas Petazzoni <[email protected]>

Cc Arnout Vandecappelle <[email protected]>

Cc Peter Korsgaard <[email protected]>

---
 package/pkg-generic.mk           |  8 +++++
 support/scripts/check-host-rpath | 71 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 79 insertions(+)
 create mode 100755 support/scripts/check-host-rpath

package/pkg-generic.mk +8 -0

index a5d0e57..ccb0d26 100644
@@ -87,6 +87,14 @@ define step_pkg_size
 endef
 GLOBAL_INSTRUMENTATION_HOOKS += step_pkg_size
 
+# This hook checks that host packages that need libraries that we build
+# have a proper DT_RPATH or DT_RUNPATH tag
+define check_host_rpath
+	$(if $(filter install-host,$(2)),\
+		$(if $(filter end,$(1)),support/scripts/check-host-rpath $(3) $(HOST_DIR)))
+endef
+GLOBAL_INSTRUMENTATION_HOOKS += check_host_rpath
+
 # User-supplied script
 ifneq ($(BR2_INSTRUMENTATION_SCRIPTS),)
 define step_user

support/scripts/check-host-rpath new file +71 -0

new file mode 100755
index 0000000..b140974
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+
+# This script scans $(HOST_DIR)/{bin,sbin} for all ELF files, and checks
+# they have an RPATH to $(HOT_DIR)/usr/lib if they need libraries from
+# there.
+
+# Override the user's locale so we are sure we can parse the output of
+# readelf(1) and file(1)
+export LC_ALL=C
+
+main() {
+    local pkg="${1}"
+    local hostdir="${2}"
+    local file ret
+
+    # Remove duplicate and trailing '/' for proper match
+    hostdir="$( sed -r -e 's:/+:/:g;' <<<"${hostdir}" )"
+
+    ret=0
+    while read file; do
+        elf_needs_rpath "${file}" "${hostdir}" || continue
+        check_elf_has_rpath "${file}" "${hostdir}" && continue
+        if [ ${ret} -eq 0 ]; then
+            ret=1
+            printf "***\n"
+            printf "*** ERROR: package %s installs executables without proper RPATH:\n" "${pkg}"
+        fi
+        printf "***   %s\n" "${file}"
+    done < <( find "${hostdir}"/usr/{bin,sbin} -type f -exec file {} + 2>/dev/null \
+              |sed -r -e '/^([^:]+):.*\<ELF\>.*\<executable\>.*/!d'                \
+                      -e 's//\1/'                                                  \
+            )
+
+    return ${ret}
+}
+
+elf_needs_rpath() {
+    local file="${1}"
+    local hostdir="${2}"
+    local lib
+
+    while read lib; do
+        [ -e "${hostdir}/usr/lib/${lib}" ] && return 0
+    done < <( readelf -d "${file}"                                         \
+              |sed -r -e '/^.* \(NEEDED\) .*Shared library: \[(.+)\]$/!d;' \
+                     -e 's//\1/;'                                          \
+            )
+
+    return 1
+}
+
+check_elf_has_rpath() {
+    local file="${1}"
+    local hostdir="${2}"
+    local rpath dir
+
+    while read rpath; do
+        for dir in ${rpath//:/ }; do
+            # Remove duplicate and trailing '/' for proper match
+            dir="$( sed -r -e 's:/+:/:g; s:/$::;' <<<"${dir}" )"
+            [ "${dir}" = "${hostdir}/usr/lib" ] && return 0
+        done
+    done < <( readelf -d "${file}"                                              \
+              |sed -r -e '/.* \(R(UN)?PATH\) +Library r(un)?path: \[(.+)\]$/!d' \
+                      -e 's//\3/;'                                              \
+            )
+
+    return 1
+}
+
+main "${@}"

Signature · 1 line

1.9.1

When we build our host programs, and they depend on a host library we also build, we want to ensure that program actually uses that library at runtime, and not the one from the system.

We currently ensure that in two ways:
- we add a RPATH tag that points to our host library directory,
- we export LD_LIBRARY_PATH to point to that same directory.

With thse two in place, we're pretty much confident that our host

libraries will be used by our host programs.

However, it turns our that not all the host programs we build end up with an RPATH tag:
- some packages do not use our $(HOST_LDFLAGS)
- some packages' build system are oblivious to those LDFLAGS

In this case, there are two situation:

- the program is not linked to one of our host libraries: it in fact does not need an RPATH tag [0]
- the program actually uses one of our host libraries: in that case it should have had an RPATH tag pointing to the host directory.

As for libraries, it is unclear whether they should or should not have an RPATH pointing to our host directory. as for programs, it is only important they have such an RPATH if they have a dependency on another host lbrary we build. But even though, in practice this is not an issue, because the program that loads such a libray does have an RPATH (it did find that library!), so the RPATH from the program is also used to search for second-level (and third-level...) dependencies, as well as for libraries loaded via dlopen().

We add a new support script that checks that all ELF executables have a proper DT_RPATH (or DT_RUNPATH) tag when they link to our host libraries, and reports those file that are missing an RPATH. If a file missing an RPATH is an executable, the script aborts; if only libraries are are missing an RPATH, the script does not abort.

[0] Except if it were to dlopen() it, of course, but the only program I'm aware of that does that is openssl, and it has a correct RPATH tag.

Signed-off-by "Yann E. MORIN" <[email protected]>

Cc Thomas Petazzoni <[email protected]>

Cc Arnout Vandecappelle <[email protected]>

Cc Peter Korsgaard <[email protected]>

---
 package/pkg-generic.mk           |  8 +++++
 support/scripts/check-host-rpath | 71 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 79 insertions(+)
 create mode 100755 support/scripts/check-host-rpath

package/pkg-generic.mk +0 -0

index a5d0e57..ccb0d26 100644

package/pkg-generic.mk +1 -0

+
 # User-supplied script
 ifneq ($(BR2_INSTRUMENTATION_SCRIPTS),)
 define step_user

support/scripts/check-host-rpath new file +1 -0

new file mode 100755
index 0000000..b140974
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash

support/scripts/check-host-rpath new file +9 -0

+
+    return 1
+}
+
+check_elf_has_rpath() {
+    local file="${1}"
+    local hostdir="${2}"
+    local rpath dir
+

support/scripts/check-host-rpath new file +5 -0

+
+    return 1
+}
+
+main "${@}"

Some comments on this one (as could be expected :-P )

Quoted 8 lines

As for libraries, it is unclear whether they should or should not have an RPATH pointing to our host directory. as for programs, it is only important they have such an RPATH if they have a dependency on another host lbrary we build. But even though, in practice this is not an issue, because the program that loads such a libray does have an RPATH (it did find that library!), so the RPATH from the program is also used to search for second-level (and third-level...) dependencies, as well as for libraries loaded via dlopen().

This paragraph isn't clear enough. How about:

For libraries, they only need an RPATH if they depend on another library that is not installed in the standard library path. However, any system library will already be in the standard library path, and any library we install ourselves is in $(HOST_DIR)/usr/lib so already in RPATH.

Also, I think it would be good to repeat this explanation in the script itself.

Quoted 8 lines

We add a new support script that checks that all ELF executables have a proper DT_RPATH (or DT_RUNPATH) tag when they link to our host libraries, and reports those file that are missing an RPATH. If a file missing an RPATH is an executable, the script aborts; if only libraries are are missing an RPATH, the script does not abort.

[0] Except if it were to dlopen() it, of course, but the only program I'm aware of that does that is openssl, and it has a correct RPATH tag.

cmake and debugfs link with dlopen() as well, so possibly they will dlopen

libraries. Therefore, I'd check for dlopen as well.

package/pkg-generic.mk define step_pkg_size · line 96

Quoted 5 lines

package/pkg-generic.mk +5 -0

+define check_host_rpath
+	$(if $(filter install-host,$(2)),\
+		$(if $(filter end,$(1)),support/scripts/check-host-rpath $(3) $(HOST_DIR)))
+endef
+GLOBAL_INSTRUMENTATION_HOOKS += check_host_rpath

As usual, I prefer it to be part of the .stamp_host_installed commands directly
instead of as a hook, because it is IMHO much more readable and simpler to understand. Not to mention that it is 6 lines shorter.

More importantly, though, there are also some packages that install stuff in
host during target or staging install, e.g. cppcms.

Also, it is usually fairly
clear where the executable comes from, and you should only really see this error while adding a package. So it seems to me that it would be sufficient to do this in the finailization step instead of after each host package install.

Quoted 14 lines

# User-supplied script
ifneq ($(BR2_INSTRUMENTATION_SCRIPTS),)
define step_user

support/scripts/check-host-rpath new file +5 -0

new file mode 100755
index 0000000..b140974
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+
+# This script scans $(HOST_DIR)/{bin,sbin} for all ELF files, and checks
+# they have an RPATH to $(HOT_DIR)/usr/lib if they need libraries from
+# there.

support/scripts/check-host-rpath line 11

Quoted 6 lines

support/scripts/check-host-rpath new file +6 -0

+
+# Override the user's locale so we are sure we can parse the output of
+# readelf(1) and file(1)
+export LC_ALL=C
+
+main() {

Not sure if I like this approach of a main() function, but OK.

Quoted 4 lines

support/scripts/check-host-rpath new file +4 -0

+    local pkg="${1}"
+    local hostdir="${2}"
+    local file ret
+

support/scripts/check-host-rpath line 20

Quoted 5 lines

support/scripts/check-host-rpath new file +5 -0

+    # Remove duplicate and trailing '/' for proper match
+    hostdir="$( sed -r -e 's:/+:/:g;' <<<"${hostdir}" )"
+
+    ret=0
+    while read file; do

I definitely don't like this while read ... <( ... ) approach, because it is
IMHO much harder to understand

(like a German sentence where all the verbs are
at the end :-).

So I would prefer a much simpler:

   for file in $(find "${hostdir}"/usr/{bin,sbin} -type f); do
       if file $file | grep -q -E '^([^:]+):.*\<ELF\>.*\<executable\>.*'; then
           ...

If you're worried about spaces in filenames, just add at the top of the file:

IFS=$(printf '\n')

support/scripts/check-host-rpath line 31

Quoted 11 lines

support/scripts/check-host-rpath new file +11 -0

+        elf_needs_rpath "${file}" "${hostdir}" || continue
+        check_elf_has_rpath "${file}" "${hostdir}" && continue
+        if [ ${ret} -eq 0 ]; then
+            ret=1
+            printf "***\n"
+            printf "*** ERROR: package %s installs executables without proper RPATH:\n" "${pkg}"
+        fi
+        printf "***   %s\n" "${file}"
+    done < <( find "${hostdir}"/usr/{bin,sbin} -type f -exec file {} + 2>/dev/null \
+              |sed -r -e '/^([^:]+):.*\<ELF\>.*\<executable\>.*/!d'                \
+                      -e 's//\1/'                                                  \

As shown above, I prefer a simple grep over this complicated sed expression.

In fact I also don't really like extended regexps (because less people are
familiar with them) but in this case it really makes it simpler.

support/scripts/check-host-rpath line 43

Quoted 1 line

support/scripts/check-host-rpath new file +1 -0

+        [ -e "${hostdir}/usr/lib/${lib}" ] && return 0

Nite: I would only use [] inside if constructs, and test if you use it like here.

support/scripts/check-host-rpath line 47

Quoted 4 lines

support/scripts/check-host-rpath new file +4 -0

+    done < <( readelf -d "${file}"                                         \
+              |sed -r -e '/^.* \(NEEDED\) .*Shared library: \[(.+)\]$/!d;' \
+                     -e 's//\1/;'                                          \
+            )

This is also where the check for dlopen should be added:

    if readelf -s "${file}" | grep -q 'UND dlopen'; then
        return 0
    else
        return 1
    fi

Well, actually it would be enough to put

     readelf -s "${file}" | grep -q 'UND dlopen'

(because the return value of a function is the return value of the last pipeline) but that's in fact harder to understand so I don't like it.

support/scripts/check-host-rpath line 66

Quoted 4 lines

support/scripts/check-host-rpath new file +4 -0

+    done < <( readelf -d "${file}"                                              \
+              |sed -r -e '/.* \(R(UN)?PATH\) +Library r(un)?path: \[(.+)\]$/!d' \
+                      -e 's//\3/;'                                              \
+            )

I stopped trying to parse this :-)

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

When we build our host programs, and they depend on a host library we also build, we want to ensure that program actually uses that library at runtime, and not the one from the system.

We currently ensure that in two ways:
- we add a RPATH tag that points to our host library directory,
- we export LD_LIBRARY_PATH to point to that same directory.

With thse two in place, we're pretty much confident that our host libraries will be used by our host programs.

However, it turns our that not all the host programs we build end up with an RPATH tag:
- some packages do not use our $(HOST_LDFLAGS)
- some packages' build system are oblivious to those LDFLAGS

In this case, there are two situation:
- the program is not linked to one of our host libraries: it in fact does not need an RPATH tag [0]
- the program actually uses one of our host libraries: in that case it should have had an RPATH tag pointing to the host directory.

As for libraries, it is unclear whether they should or should not have an RPATH pointing to our host directory. as for programs, it is only important they have such an RPATH if they have a dependency on another host lbrary we build. But even though, in practice this is not an issue, because the program that loads such a libray does have an RPATH (it did find that library!), so the RPATH from the program is also used to search for second-level (and third-level...) dependencies, as well as for libraries loaded via dlopen().

We add a new support script that checks that all ELF executables have a proper DT_RPATH (or DT_RUNPATH) tag when they link to our host libraries, and reports those file that are missing an RPATH. If a file missing an RPATH is an executable, the script aborts; if only libraries are are missing an RPATH, the script does not abort.

[0] Except if it were to dlopen() it, of course, but the only program I'm aware of that does that is openssl, and it has a correct RPATH tag.

Signed-off-by "Yann E. MORIN" <[email protected]>

Cc Thomas Petazzoni <[email protected]>

Cc Arnout Vandecappelle <[email protected]>

Cc Peter Korsgaard <[email protected]>

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

From: Ben Boeckel <[email protected]>
If system tools are selected, the host's lib/ directory may shadow libraries from the system which are configured differently and do not have all of the symbols required by the system tool.

Since buildroot now uses rpath everywhere, LD_LIBRARY_PATH should not be necessary anyways.

Signed-off-by Ben Boeckel <[email protected]>

Acked-by Arnout Vandecappelle (Essensium/Mind) <[email protected]>

Acked-by "Yann E. MORIN" <[email protected]>

Signed-off-by "Yann E. MORIN" <[email protected]>

Hello All!

This series is an attempt to ditch LD_LIBRARY_PATH for host binaries.

Using LD_LIBRARY_PATH is causing build issues on certain distributions, like Fedore 23, see:

    https://bugs.busybox.net/show_bug.cgi?id=8456
    http://lists.busybox.net/pipermail/buildroot/2015-October/142898.html

Basically, on Fedora 23, ls (from coreutils) is now linked to libcap, which is also linked to libattr. However, outr own libcap is not linked to libattr, and ls is too. However, because of our LD_LIBRARY_PATH, the host ls is using our libcap, and thus fails to run.

This series:
- ?xes a few host packages, add RPATH to packages where it is missing
- fixes the host-mysql definition, allow it to be built always
- removes host-perl-file-util, it is not used anywhere and is broken
- cleanups libcurl about LD_LIBRARY_PATH
- adds a script to check that host binaries have appropriate RPATH
- finally removes LD_LIBRARY_PATH

Even though this is a relatively intrusive change, it in my opinion should go in master now, otherwise, we'd get a full release cycle broken on Fedora, which would really be a shame... :-/

Regards,
Yann E. MORIN.

The following changes since commit 2bc7c2e009ac08ae80cbf3ce736ade16a6cfcb26:

zxing-cpp: needs dynamic library (2015-11-13 16:39:29 +0100)

are available in the git repository at:

git://git.busybox.net/~ymorin/git/buildroot yem/no-ld-library-path

for you to fetch changes up to 0a14a9ff7f39c78d2a39f8ccde40a9f7a6d020b2:

core/pkg-infrastructures: remove LD_LIBRARY_PATH from the environment (2015-11-13 19:51:07 +0100)

----------------------------------------------------------------
Ben Boeckel (1):
core/pkg-infrastructures: remove LD_LIBRARY_PATH from the environment

Yann E. MORIN (5):
package/axfsutils: fix Makefile package/mysql: unconditionally define host variables package/perl-file-util: remove host variant package/libcurl: carefully override LD_LIBRARY_PATH
core: check host executables have appropriate RPATH

 package/Makefile.in                      |  2 -
 package/axfsutils/0001-fix-cflags.patch  | 19 +++++++++
 package/axfsutils/0002-use-ldflags.patch | 20 +++++++++
 package/axfsutils/axfsutils.mk           |  2 +-
 package/libcurl/libcurl.mk               |  2 +-
 package/mysql/mysql.mk                   | 39 +++++++++---------
 package/perl-file-util/perl-file-util.mk |  1 -
 package/pkg-generic.mk                   |  8 ++++
 support/scripts/check-host-rpath         | 71 ++++++++++++++++++++++++++++++++
 9 files changed, 140 insertions(+), 24 deletions(-)
 create mode 100644 package/axfsutils/0001-fix-cflags.patch
 create mode 100644 package/axfsutils/0002-use-ldflags.patch
 create mode 100755 support/scripts/check-host-rpath