core: install foo-config scripts early in the PATH (branch yem/foo-config-in-PATH)

Hello All!

This RFC series is an attempt at solving once and for all the problem we encounter with foo-config scripts.

The foo-config scripts are usually installed in staging, so are out of the PATH. However, most packages will just simply want to run them, and most are not configurable in where to look for that script and just expect to find them in the PATH.

Up until now, we used to patch those packages so that they accept an environment variable.

Arnout has suggested [0] that we move all the target-related foo-config scripts early in the PATH, so that they are found before the host ones and before the system ones.

This would allow us to drop our local patches, and to no longer set the custom FOO_CONFIG or the ac_cv_foo_config variables from our package recipes.

This series is my proposal to implement Arnout's suggestion, and is articulated as thus:

- first, we ensure that we install the munged foo-config scripts in their own location (patches 1-3),

- then we separate again the host and target PATHs variables (patch 4)

- finally, we use the proper host or target PATH everywhere we set

    PATH (patches 5-7).

- a preliminary cleanup patch against libhid concludes the series

    (patch 8)

This series is RFC: it is a "technology-preview" for early comments and to see if the solution is viable.

The series makes it so that our currently workarounds still work. When/if the topic is accepted (and the series is fixed after the reviews), we can then (un)fix / (un)patch packages in follow-up patches.

[0] http://lists.busybox.net/pipermail/buildroot/2015-December/146959.html

Regards,
Yann E. MORIN.

The following changes since commit 0848e9e89c2c427ab6ca35fab25996d7a1cd8c5b:

luvi: new package (2015-12-20 23:13:02 +0100)

are available in the git repository at:

git://git.busybox.net/~ymorin/git/buildroot yem/foo-config-in-PATH

for you to fetch changes up to 5927b9370935b82aedcdcade5cdd5d9c280528d5:

package/libhid: no need for custom PATH (2015-12-21 12:34:06 +0100)

----------------------------------------------------------------
Yann E. MORIN (8):
core: split long line of directories
core/pkg-generic: use a $(foreach) loop to fix foo-config scripts core/pkg-generic: install foo-config scripts in their own dir
core: re-instate different target and host PATHs
fs: use HOST_PATH
packages: use HOST_PATH
packages: use TARGET_PATH
package/libhid: no need for custom PATH

 Makefile                     | 15 ++++++++++++---
 fs/common.mk                 |  6 +++---
 fs/ext2/ext2.mk              |  2 +-
 package/Makefile.in          | 10 +++++-----
 package/faifa/faifa.mk       |  2 +-
 package/freetype/freetype.mk |  2 +-
 package/gpm/gpm.mk           |  2 +-
 package/kodi/kodi.mk         |  2 +-
 package/libhid/libhid.mk     |  4 +---
 package/pkg-cmake.mk         |  4 ++--
 package/pkg-generic.mk       |  5 ++++-
 package/pkg-python.mk        |  8 ++++----
 package/sdl/sdl.mk           |  2 +-
 13 files changed, 37 insertions(+), 27 deletions(-)

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

More are coming, so split for readability.

---
 Makefile | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

index dc657d9..f2b09ad 100644
@@ -445,7 +445,9 @@ world: target-post-image
 # dependencies anywhere else
 #
 ################################################################################
-$(BUILD_DIR) $(TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST):
+$(BUILD_DIR) $(TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR):
+	@mkdir -p $@
+$(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST):
 	@mkdir -p $@
 
 # We make a symlink lib32->lib or lib64->lib as appropriate

1.9.1

Currently, we fix all the foo-config scripts of a package in one sed invocation. We can do so because we fix them in-place.

However, we're in the process of moving the fixed foo-config scripts to their own directory, so we'll need to be able to sed them one by one.

---
 package/pkg-generic.mk | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

index 1e024d3..69349f6 100644
@@ -238,6 +238,7 @@ $(BUILD_DIR)/%/.stamp_staging_installed:
 	$(foreach hook,$($(PKG)_POST_INSTALL_STAGING_HOOKS),$(call $(hook))$(sep))
 	$(Q)if test -n "$($(PKG)_CONFIG_SCRIPTS)" ; then \
 		$(call MESSAGE,"Fixing package configuration files") ;\
+		$(foreach s,$($(PKG)_CONFIG_SCRIPTS),\
 			$(SED)  "s,$(BASE_DIR), at BASE_DIR@,g" \
 				-e "s,$(STAGING_DIR), at STAGING_DIR@,g" \
 				-e "s,^\(exec_\)\?prefix=.*,\1prefix=@STAGING_DIR@/usr,g" \

@@ -245,7 +246,7 @@ $(BUILD_DIR)/%/.stamp_staging_installed:
 				-e "s,-L/usr/,-L at STAGING_DIR@/usr/,g" \
 				-e "s, at STAGING_DIR@,$(STAGING_DIR),g" \
 				-e "s, at BASE_DIR@,$(BASE_DIR),g" \
-				$(addprefix $(STAGING_DIR)/usr/bin/,$($(PKG)_CONFIG_SCRIPTS)) ;\
+				$(STAGING_DIR)/usr/bin/$(s); ) \
 	fi
 	@$(call MESSAGE,"Fixing libtool files")
 	$(Q)find $(STAGING_DIR)/usr/lib* -name "*.la" | xargs --no-run-if-empty \

1.9.1

After we fixup the foo-config scripts of a package, copy them to their own directory that we put early in the PATH, so that packges that hard-code calls to foo-config will find them automatically and avoid finding those of the host.

Note that we do keep the fixed-up scripts in staging, because we already pass 'FOO_CONFIG=$(STAGING)/usr/bin/foo-config' to a lot of packages, some for which we even have patches to accept the FOO_CONFIG environment variables. We need to unpatch those packages before we can just move the fixed-up foo-config scripts out of staging.

---
 Makefile               | 7 +++++--
 package/pkg-generic.mk | 4 +++-
 2 files changed, 8 insertions(+), 3 deletions(-)

index f2b09ad..780b596 100644
@@ -363,6 +363,9 @@ TAR_OPTIONS = $(call qstrip,$(BR2_TAR_OPTIONS)) -xf
 # packages compiled for the host go here
 HOST_DIR := $(call qstrip,$(BR2_HOST_DIR))
 
+# foo-config scripts for the target go here
+FOO_CONFIG_DIR := $(HOST_DIR)/opt/foo-config
+
 # Quotes are needed for spaces and all in the original PATH content.
 BR_PATH = "$(HOST_DIR)/bin:$(HOST_DIR)/sbin:$(HOST_DIR)/usr/bin:$(HOST_DIR)/usr/sbin:$(PATH)"
 

@@ -426,7 +429,7 @@ include fs/common.mk
 include $(BR2_EXTERNAL)/external.mk
 
 dirs: $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
-	$(HOST_DIR) $(BINARIES_DIR)
+	$(HOST_DIR) $(BINARIES_DIR) $(FOO_CONFIG_DIR)
 
 $(BUILD_DIR)/buildroot-config/auto.conf: $(BR2_CONFIG)
 	$(MAKE1) $(EXTRAMAKEARGS) HOSTCC="$(HOSTCC_NOCCACHE)" HOSTCXX="$(HOSTCXX_NOCCACHE)" silentoldconfig

@@ -445,7 +448,7 @@ world: target-post-image
 # dependencies anywhere else
 #
 ################################################################################
-$(BUILD_DIR) $(TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR):
+$(BUILD_DIR) $(TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(FOO_CONFIG_DIR):
 	@mkdir -p $@
 $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST):
 	@mkdir -p $@

index 69349f6..6e3aee1 100644
@@ -246,7 +246,9 @@ $(BUILD_DIR)/%/.stamp_staging_installed:
 				-e "s,-L/usr/,-L at STAGING_DIR@/usr/,g" \
 				-e "s, at STAGING_DIR@,$(STAGING_DIR),g" \
 				-e "s, at BASE_DIR@,$(BASE_DIR),g" \
-				$(STAGING_DIR)/usr/bin/$(s); ) \
+				$(STAGING_DIR)/usr/bin/$(s); \
+			$(INSTALL) -m 0755 -D $(STAGING_DIR)/usr/bin/$(s) \
+				$(FOO_CONFIG_DIR)/$(s); ) \
 	fi
 	@$(call MESSAGE,"Fixing libtool files")
 	$(Q)find $(STAGING_DIR)/usr/lib* -name "*.la" | xargs --no-run-if-empty \

1.9.1

Hi Yann,

     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -

Baruch, All,

On 2015-12-21 20:51 +0200, Baruch Siach spake thusly:

Oh, well, 'foo-config' just has stuck, and I tried to keep it short. Since it is used only in a very few locations, LIB_CONFIG_SCRIPT_DIR is pretty OK, so I don't minf renaming.

Thanks!

Regards,
Yann E. MORIN.

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

In b989976 and previous commits, the differentiation between host and target PATH has been removed, because there was no reason at that time to have two different PATHs.

We now have the foo-config scripts installed in their own directory, and that has to come first in the PATH when building for the target. However, we do not want to use those directories for the host packages, as those foo-config scripts are only for the target packages.

So, we need to differentiate the host and target PATHs again.

This is a preparatory commit, which only introduces the two new variables. Follow-up patches will make use of those variables in the proper locations.

---
 Makefile | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

index 780b596..b5fadf4 100644
@@ -366,8 +366,12 @@ HOST_DIR := $(call qstrip,$(BR2_HOST_DIR))
 # foo-config scripts for the target go here
 FOO_CONFIG_DIR := $(HOST_DIR)/opt/foo-config
 
+# Host PATH is just BR_PATH, but target PATH must find the foo-config
+# scripts first.
 # Quotes are needed for spaces and all in the original PATH content.
-BR_PATH = "$(HOST_DIR)/bin:$(HOST_DIR)/sbin:$(HOST_DIR)/usr/bin:$(HOST_DIR)/usr/sbin:$(PATH)"
+BR_PATH = $(HOST_DIR)/bin:$(HOST_DIR)/sbin:$(HOST_DIR)/usr/bin:$(HOST_DIR)/usr/sbin:$(PATH)
+HOST_PATH = "$(BR_PATH)"
+TARGET_PATH = "$(FOO_CONFIG_DIR):$(BR_PATH)"
 
 # Location of a file giving a big fat warning that output/target
 # should not be used as the root filesystem.

1.9.1

---
 fs/common.mk    | 6 +++---
 fs/ext2/ext2.mk | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

index 528e194..b513f63 100644
@@ -90,14 +90,14 @@ ifneq ($$(ROOTFS_USERS_TABLES),)
 	cat $$(ROOTFS_USERS_TABLES) >> $$(USERS_TABLE)
 endif
 	printf '$$(subst $$(sep),\n,$$(PACKAGES_USERS))' >> $$(USERS_TABLE)
-	PATH=$$(BR_PATH) $$(TOPDIR)/support/scripts/mkusers $$(USERS_TABLE) $$(TARGET_DIR) >> $$(FAKEROOT_SCRIPT)
+	PATH=$$(HOST_PATH) $$(TOPDIR)/support/scripts/mkusers $$(USERS_TABLE) $$(TARGET_DIR) >> $$(FAKEROOT_SCRIPT)
 	echo "$$(ROOTFS_$(2)_CMD)" >> $$(FAKEROOT_SCRIPT)
 	chmod a+x $$(FAKEROOT_SCRIPT)
-	PATH=$$(BR_PATH) $$(HOST_DIR)/usr/bin/fakeroot -- $$(FAKEROOT_SCRIPT)
+	PATH=$$(HOST_PATH) $$(HOST_DIR)/usr/bin/fakeroot -- $$(FAKEROOT_SCRIPT)
 	$$(INSTALL) -m 0644 support/misc/target-dir-warning.txt $$(TARGET_DIR_WARNING_FILE)
 	- at rm -f $$(FAKEROOT_SCRIPT) $$(FULL_DEVICE_TABLE)
 ifneq ($$(ROOTFS_$(2)_COMPRESS_CMD),)
-	PATH=$$(BR_PATH) $$(ROOTFS_$(2)_COMPRESS_CMD) $$@ > $$@$$(ROOTFS_$(2)_COMPRESS_EXT)
+	PATH=$$(HOST_PATH) $$(ROOTFS_$(2)_COMPRESS_CMD) $$@ > $$@$$(ROOTFS_$(2)_COMPRESS_EXT)
 endif
 	$$(foreach hook,$$(ROOTFS_$(2)_POST_GEN_HOOKS),$$(call $$(hook))$$(sep))
 

index 7417f81..ab876b7 100644
@@ -31,7 +31,7 @@ endif
 ROOTFS_EXT2_DEPENDENCIES = host-mke2img
 
 define ROOTFS_EXT2_CMD
-	PATH=$(BR_PATH) mke2img -d $(TARGET_DIR) $(EXT2_OPTS) -o $@
+	PATH=$(HOST_PATH) mke2img -d $(TARGET_DIR) $(EXT2_OPTS) -o $@
 endef
 
 rootfs-ext2-symlink:

1.9.1

---
 package/Makefile.in   | 6 +++---
 package/pkg-cmake.mk  | 2 +-
 package/pkg-python.mk | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

index 82a66c2..7517a62 100644
@@ -272,7 +272,7 @@ TARGET_MAKE_ENV = PATH=$(BR_PATH)
 
 
 HOST_CONFIGURE_OPTS = \
-	PATH=$(BR_PATH) \
+	PATH=$(HOST_PATH) \
 	AR="$(HOSTAR)" \
 	AS="$(HOSTAS)" \
 	LD="$(HOSTLD)" \

@@ -295,7 +295,7 @@ HOST_CONFIGURE_OPTS = \
 	INTLTOOL_PERL=$(PERL)
 
 HOST_MAKE_ENV = \
-	PATH=$(BR_PATH) \
+	PATH=$(HOST_PATH) \
 	PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \
 	PKG_CONFIG_SYSROOT_DIR="/" \
 	PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig"

@@ -305,7 +305,7 @@ HOST_MAKE_ENV = \
 # explicitly pass it to user-supplied external hooks (eg. post-build,
 # post-images)
 EXTRA_ENV = \
-	PATH=$(BR_PATH) \
+	PATH=$(HOST_PATH) \
 	BR2_DL_DIR=$(BR2_DL_DIR) \
 	BUILD_DIR=$(BUILD_DIR) \
 	BR2_EXTERNAL=$(BR2_EXTERNAL)

index 81dcfcc..101a551 100644
@@ -109,7 +109,7 @@ define $(2)_CONFIGURE_CMDS
 	(mkdir -p $$($$(PKG)_BUILDDIR) && \
 	cd $$($$(PKG)_BUILDDIR) && \
 	rm -f CMakeCache.txt && \
-	PATH=$$(BR_PATH) \
+	PATH=$$(HOST_PATH) \
 	$$(HOST_DIR)/usr/bin/cmake $$($$(PKG)_SRCDIR) \
 		-DCMAKE_INSTALL_SO_NO_EXE=0 \
 		-DCMAKE_FIND_ROOT_PATH="$$(HOST_DIR)" \

index b7a702d..d94a485 100644
@@ -43,7 +43,7 @@ PKG_PYTHON_DISTUTILS_INSTALL_STAGING_OPTS = \
 
 # Host distutils-based packages
 HOST_PKG_PYTHON_DISTUTILS_ENV = \
-	PATH=$(BR_PATH)
+	PATH=$(HOST_PATH)
 
 HOST_PKG_PYTHON_DISTUTILS_INSTALL_OPTS = \
 	--prefix=$(HOST_DIR)/usr

@@ -70,7 +70,7 @@ PKG_PYTHON_SETUPTOOLS_INSTALL_STAGING_OPTS = \
 
 # Host setuptools-based packages
 HOST_PKG_PYTHON_SETUPTOOLS_ENV = \
-	PATH=$(BR_PATH)
+	PATH=$(HOST_PATH)
 
 HOST_PKG_PYTHON_SETUPTOOLS_INSTALL_OPTS = \
 	--prefix=$(HOST_DIR)/usr

1.9.1

Note that libhid should no longer need that trick, but this woill be fixed independently in a follow-up patch.

---
 package/Makefile.in          | 4 ++--
 package/faifa/faifa.mk       | 2 +-
 package/freetype/freetype.mk | 2 +-
 package/gpm/gpm.mk           | 2 +-
 package/kodi/kodi.mk         | 2 +-
 package/libhid/libhid.mk     | 2 +-
 package/pkg-cmake.mk         | 2 +-
 package/pkg-python.mk        | 4 ++--
 package/sdl/sdl.mk           | 2 +-
 9 files changed, 11 insertions(+), 11 deletions(-)

index 7517a62..e211001 100644
@@ -231,7 +231,7 @@ export PERL=$(shell which perl)
 export PERL5LIB=$(HOST_DIR)/usr/lib/perl
 
 TARGET_CONFIGURE_OPTS = \
-	PATH=$(BR_PATH) \
+	PATH=$(TARGET_PATH) \
 	AR="$(TARGET_AR)" \
 	AS="$(TARGET_AS)" \
 	LD="$(TARGET_LD)" \

@@ -268,7 +268,7 @@ TARGET_CONFIGURE_OPTS = \
 	STAGING_DIR="$(STAGING_DIR)" \
 	INTLTOOL_PERL=$(PERL)
 
-TARGET_MAKE_ENV = PATH=$(BR_PATH)
+TARGET_MAKE_ENV = PATH=$(TARGET_PATH)
 
 
 HOST_CONFIGURE_OPTS = \

index a56a8c0..1c9ccb5 100644
@@ -18,7 +18,7 @@ FAIFA_MAKE_OPTS += GIT_REV=$(FAIFA_VERSION)
 # installation commands.
 
 define FAIFA_RUN_AUTOGEN
-	cd $(@D) && PATH=$(BR_PATH) ./autogen.sh
+	cd $(@D) && PATH=$(TARGET_PATH) ./autogen.sh
 endef
 FAIFA_PRE_CONFIGURE_HOOKS += FAIFA_RUN_AUTOGEN
 

index fa11b0c..a611b83 100644
@@ -23,7 +23,7 @@ HOST_FREETYPE_CONF_OPTS = --without-zlib --without-bzip2 --without-png
 # even AUTORECONF with _OPTS doesn't do it properly.
 # POST_PATCH is because we still need to patch libtool after the regen.
 define FREETYPE_RUN_AUTOGEN
-	cd $(@D) && PATH=$(BR_PATH) ./autogen.sh
+	cd $(@D) && PATH=$(TARGET_PATH) ./autogen.sh
 endef
 FREETYPE_POST_PATCH_HOOKS += FREETYPE_RUN_AUTOGEN
 HOST_FREETYPE_POST_PATCH_HOOKS += FREETYPE_RUN_AUTOGEN

index 6d3b5d2..e753ce7 100644
@@ -36,7 +36,7 @@ GPM_CONF_OPTS = --without-curses
 # configure is missing but gpm seems not compatible with our autoreconf
 # mechanism so we have to do it manually instead of using GPM_AUTORECONF = YES
 define GPM_RUN_AUTOGEN
-	cd $(@D) && PATH=$(BR_PATH) ./autogen.sh
+	cd $(@D) && PATH=$(TARGET_PATH) ./autogen.sh
 endef
 GPM_PRE_CONFIGURE_HOOKS += GPM_RUN_AUTOGEN
 

index bf3aa7a..95442ad 100644
@@ -224,7 +224,7 @@ endif
 define KODI_BOOTSTRAP
 	$(HOST_CONFIGURE_OPTS) $(MAKE) -C $(@D)/tools/depends/native/JsonSchemaBuilder
 	$(HOST_CONFIGURE_OPTS) $(MAKE) -C $(@D)/tools/depends/native/TexturePacker
-	cd $(@D) && PATH=$(BR_PATH) ./bootstrap
+	cd $(@D) && PATH=$(TARGET_PATH) ./bootstrap
 endef
 KODI_PRE_CONFIGURE_HOOKS += KODI_BOOTSTRAP
 

index 8b3cb2a..92a3049 100644
@@ -13,7 +13,7 @@ LIBHID_LICENSE = GPLv2
 LIBHID_LICENSE_FILES = COPYING
 # configure runs libusb-config for cflags/ldflags. Ensure it picks up
 # the target version
-LIBHID_CONF_ENV = PATH=$(STAGING_DIR)/usr/bin:$(BR_PATH)
+LIBHID_CONF_ENV = PATH=$(STAGING_DIR)/usr/bin:$(TARGET_PATH)
 LIBHID_CONF_OPTS = \
 	--disable-swig \
 	--disable-werror \

index 101a551..ddef2d8 100644
@@ -84,7 +84,7 @@ define $(2)_CONFIGURE_CMDS
 	(mkdir -p $$($$(PKG)_BUILDDIR) && \
 	cd $$($$(PKG)_BUILDDIR) && \
 	rm -f CMakeCache.txt && \
-	PATH=$$(BR_PATH) \
+	PATH=$$(TARGET_PATH) \
 	$$($$(PKG)_CONF_ENV) $$(HOST_DIR)/usr/bin/cmake $$($$(PKG)_SRCDIR) \
 		-DCMAKE_TOOLCHAIN_FILE="$$(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake" \
 		-DCMAKE_BUILD_TYPE=$$(if $$(BR2_ENABLE_DEBUG),Debug,Release) \

index d94a485..e0d9650 100644
@@ -22,7 +22,7 @@
 
 # Target distutils-based packages
 PKG_PYTHON_DISTUTILS_ENV = \
-	PATH=$(BR_PATH) \
+	PATH=$(TARGET_PATH) \
 	CC="$(TARGET_CC)" \
 	CFLAGS="$(TARGET_CFLAGS)" \
 	LDFLAGS="$(TARGET_LDFLAGS)" \

@@ -50,7 +50,7 @@ HOST_PKG_PYTHON_DISTUTILS_INSTALL_OPTS = \
 
 # Target setuptools-based packages
 PKG_PYTHON_SETUPTOOLS_ENV = \
-	PATH=$(BR_PATH) \
+	PATH=$(TARGET_PATH) \
 	PYTHONPATH="$(if $(BR2_PACKAGE_PYTHON3),$(PYTHON3_PATH),$(PYTHON_PATH))" \
 	_python_sysroot=$(STAGING_DIR) \
 	_python_prefix=/usr \

index cc1ad06..5896fb1 100644
@@ -14,7 +14,7 @@ SDL_INSTALL_STAGING = YES
 # we're patching configure.in, but package cannot autoreconf with our version of
 # autotools, so we have to do it manually instead of setting SDL_AUTORECONF = YES
 define SDL_RUN_AUTOGEN
-	cd $(@D) && PATH=$(BR_PATH) ./autogen.sh
+	cd $(@D) && PATH=$(TARGET_PATH) ./autogen.sh
 endef
 
 SDL_PRE_CONFIGURE_HOOKS += SDL_RUN_AUTOGEN

1.9.1

Hi Yann,

So can BR_PATH be removed entirely from top Makefile after this patch?

baruch

     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -

Baruch, All,

On 2015-12-21 20:44 +0200, Baruch Siach spake thusly:

I wondered about that.

Still, I think we should keep it, in case we need to update both PATHs with new entries.

We could obviously do:

    TARGET_PATH = $(FOO_CONFIG_DIR):$(HOST_PATH)

but that would not fly too far if we need to add dirs to HOST_PATH that should not be in TARGET_PATH.

So I prefer to keep BR_PATH (which we could rename as BR_COMMON_PATH).

Regards,
Yann E. MORIN.

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Now that we have all the target foo-config scripts early in the PATH, we no longer need a cutom PATH.

---
 package/libhid/libhid.mk | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

index 92a3049..3c65814 100644
@@ -11,9 +11,7 @@ LIBHID_INSTALL_STAGING = YES
 LIBHID_AUTORECONF = YES
 LIBHID_LICENSE = GPLv2
 LIBHID_LICENSE_FILES = COPYING
-# configure runs libusb-config for cflags/ldflags. Ensure it picks up
-# the target version
-LIBHID_CONF_ENV = PATH=$(STAGING_DIR)/usr/bin:$(TARGET_PATH)
+
 LIBHID_CONF_OPTS = \
 	--disable-swig \
 	--disable-werror \

1.9.1

Yann,

Unfortunately, I believe it doesn't work completely, because there are some <foo>-config scripts that are not listed in <pkg>_CONFIG_SCRIPTS because they are for some reason not compatible with the fixup that we do. One example that comes to mind is python-*config (they are written in shell but don't follow the usual pattern). I believe there might be other cases as well.

Also, this solution doesn't solve the (admittedly unlikely) case of a package calling /usr/bin/<foo>-config directly.

But I agree that it might solve many of the usual cases. I don't know if the additional complexity of yet another special mechanism really makes it worth it. I'd like to hear from Peter about this.

Thomas

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Thomas, All,

On 2015-12-21 23:20 +0100, Thomas Petazzoni spake thusly:

And how do we handle them today? If we don't do anything about it, then this series would indeed not solve those, but it would not break it either, since it would already be broken today.

Unless we do have code to manually install them in $(HOST_DIR)/usr/bin ?

Again, what are we doing for those today?

Indeed. But it is already broken today, and the series does not break it any more.

Yes, this series is really just an RFC. It was relatively easy to do (even a bit easier than what I expected, aha!), so I just spun it on the list just to "feel the temperature".

Regards,
Yann E. MORIN.

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Yann,

What? No, we are just installing them in $(STAGING_DIR), like all libraries. You can look at package/rtai/rtai.mk for an example.

It is not broken today. Such special *-config scripts get naturally installed in $(STAGING_DIR), they might be fixed up by a patch or some custom hook. And then on the consumer side, we pass some environment variable or other trick to get the consumer build system to use this specific -config script rather than the one in the PATH. Nothing special.

Thomas

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Thomas, All,

On 2015-12-21 23:34 +0100, Thomas Petazzoni spake thusly:

That one is interesting, indeed. We "fix" rtai-config in-place in staging, but then we never pass $(STAGING_DIR)/usr/bin/rtai-config to any variable of any package, which means that any consumer of rtai-config, if any, is called with $(STAGING_DIR)/usr/bin in its PATH.

I think this is an abomination. There are three cases there:

- cross-compiling for a different architecture with shared libraries:
this is not really a problem, binaries in there will not work; yet, this might still cause problems, because there might be binaries in there that would shadow those of the host, and that would be a problem.

- cross-compiling for the same architecture: binaries in staging would probably work if the host system is recent enough that the C library id "compatible" (same or newer glibc on the host) or if doing a static build.

- cross-compiling with static libs: damn, that one would be awfully trricky to debug. For example, my machine is setup to run qemu-user for some architectures, so I'd totaly be able to run ARM binaries on my build machine. Fortunately, this is a rare-enough case that we were not caught by it yet...

And we indeed do such abonminations. For example, the libhid stuff that I already fixed in my series.

Then those patch-or-hook fixups should be complemented by a post-install hook that also installs the -config script in the newly-introduced FOO_CONFIG_DIR.

Again, nothing that this series would *break*; existing "workarounds" would continue to work as-is. It's only a new opportunity to cleanup the mess, but will need much more pathces later on.

Ah, that's probably what I forgot to write in my cover later: this 8-patch series only introduces the "infra" and does not actually fix the packages, or undo our workarounds, or removes our patches, of add new fixes. Hmm. Well, actually I did:

When/if the topic is accepted (and the series is fixed after the reviews), we can then (un)fix / (un)patch packages in follow-up patches.

Or maybe I completely missed the point...

Regards,
Yann E. MORIN.

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Hello,

Absolutely not.

There are no packages in Buildroot that use the RTAI libraries. The rtai-config script in $(STAGING_DIR) is meant to be used:

1/ by custom packages (i.e not in Buildroot mainline)

2/ by people building their stuff outside of Buildroot

There is absolutely no abomination here. This is a regular *-config script, installed in STAGING_DIR like any other. It is just not used by any package in the upstream Buildroot.

Correct.

Yes, yes, this is fully understood, I do understand that we will be able to remove a number of patches, or custom variable passing.

Thomas

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Thomas, All,

On 2015-12-22 11:53 +0100, Thomas Petazzoni spake thusly:

Oh, sorry, what I meant is: having staging/usr/bin in the PATH is an abomination.

Of course, the rtai-config script is not the abomination I was referring too. My bad, I was not carefull enough in proof-reading what I wrote.

s/later/letter/ Damit... :-)

Cheers! ;-)

Regards,
Yann E. MORIN.

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'