Skip to contentSkip to conversation

toolchain-ext: rm shared libs for static builds

PATCH New Matt Weber 3 participants 3 messages Oct 26, 2019

lore
Share & tools
lore

Matt Weber

author
#0 lore raw

PATCH v2 1/1 toolchain-ext: rm shared libs for static builds

For cases where Buildroot is generating a build with BR2_STATIC_LIBS=y and there is an external toolchain being used, the STAGING_DIR and HOSTDIR need to be scrubbed of shared library *.so*. This patch updates the toolchain-external staging step to first clean out all shared libraries before a sysroot is created. Before this patch, if shared libraries were found in the GCC library search path(s), build systems might still pick these up during build/link and fail with an error like "ld: attempted static link of dynamic object".

NOTE: The Meson build system by default prefers external libraries to be shared libraries unless the developer has explicitly in their meson.build set each find_library() invocation to contain the static keyword (requires meson 0.51.0+). One example where this was noticed was with iputils-20190709 where a link time dependency on libatomic occurred because the prebuilt toolchain had provided both a static and shared option. Meson then generated a compile string including a fixed path to the toolchain's shared libatomic.so instead of static.

Fixed:
http://autobuild.buildroot.net/results/db1740b4777f436324218c52bc7b08e5c21b667d/

---
Changes v1 -> v2
[Yann
- Cleaned up weird Unicode chars
- Updated comment about dirs existing

---
 .../toolchain-external/pkg-toolchain-external.mk   | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
toolchain/toolchain-external/pkg-toolchain-external.mk +14 -0
index c00211d59c..a87d359d32 100644
@@ -444,6 +444,19 @@ define TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS
 	$(call copy_toolchain_sysroot,$${SYSROOT_DIR},$${ARCH_SYSROOT_DIR},$${ARCH_SUBDIR},$${ARCH_LIB_DIR},$${SUPPORT_LIB_DIR})
 endef
 
+# NOTE: below the readlink call follows/builds each absolute path with any
+# invalid paths failing (falling out of the list). Thus no checking of paths
+# existing is required before doing the find. This assumes that GCC returns
+# at least one directory as a working toolchain should always have search
+# paths.
+ifeq ($(BR2_STATIC_LIBS),y)
+define TOOLCHAIN_EXTERNAL_REMOVE_SHARED_LIBS
+	$(Q)$(call MESSAGE,"Removing shared libraries from toolchain...")
+	GCC_LIBRARY_PATHS=`$(TOOLCHAIN_EXTERNAL_CC) -print-search-dirs | sed -r -e '/libraries: =(.+)/!d; s//\1/; s/:/\n/g' | xargs readlink -f | grep -v 'gcc\|/[0-9.]\+$$'` ; \
+	find $${GCC_LIBRARY_PATHS} -name *.so* -delete
+endef
+endif
+
 # Create a symlink from (usr/)$(ARCH_LIB_DIR) to lib.
 # Note: the skeleton package additionally creates lib32->lib or lib64->lib
 # (as appropriate)
@@ -565,6 +578,7 @@ $(2)_TOOLCHAIN_WRAPPER_ARGS += $$(TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS)
 $(2)_BUILD_CMDS = $$(TOOLCHAIN_WRAPPER_BUILD)
 
 define $(2)_INSTALL_STAGING_CMDS
+	$$(TOOLCHAIN_EXTERNAL_REMOVE_SHARED_LIBS)
 	$$(TOOLCHAIN_WRAPPER_INSTALL)
 	$$(TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK)
 	$$(TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS)
Signature · 1 line
2.17.1
#1 lore raw

PATCH v2 1/1 toolchain-ext: rm shared libs for static builds

Matt, All,

On 2019-10-26 03:52 -0500, Matt Weber spake thusly:

Quoted 18 lines

For cases where Buildroot is generating a build with BR2_STATIC_LIBS=y and there is an external toolchain being used, the STAGING_DIR and HOSTDIR need to be scrubbed of shared library *.so*. This patch updates the toolchain-external staging step to first clean out all shared libraries before a sysroot is created. Before this patch, if shared libraries were found in the GCC library search path(s), build systems might still pick these up during build/link and fail with an error like "ld: attempted static link of dynamic object".

NOTE: The Meson build system by default prefers external libraries to be shared libraries unless the developer has explicitly in their meson.build set each find_library() invocation to contain the static keyword (requires meson 0.51.0+). One example where this was noticed was with iputils-20190709 where a link time dependency on libatomic occurred because the prebuilt toolchain had provided both a static and shared option. Meson then generated a compile string including a fixed path to the toolchain's shared libatomic.so instead of static.

Quoted 5 lines

Fixed:
http://autobuild.buildroot.net/results/db1740b4777f436324218c52bc7b08e5c21b667d/

Regards,
Yann E. MORIN.

Quoted 44 lines

---
Changes v1 -> v2
[Yann
- Cleaned up weird Unicode chars
- Updated comment about dirs existing

---
 .../toolchain-external/pkg-toolchain-external.mk   | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
toolchain/toolchain-external/pkg-toolchain-external.mk +14 -0
index c00211d59c..a87d359d32 100644
@@ -444,6 +444,19 @@ define TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS
 	$(call copy_toolchain_sysroot,$${SYSROOT_DIR},$${ARCH_SYSROOT_DIR},$${ARCH_SUBDIR},$${ARCH_LIB_DIR},$${SUPPORT_LIB_DIR})
 endef
 
+# NOTE: below the readlink call follows/builds each absolute path with any
+# invalid paths failing (falling out of the list). Thus no checking of paths
+# existing is required before doing the find. This assumes that GCC returns
+# at least one directory as a working toolchain should always have search
+# paths.
+ifeq ($(BR2_STATIC_LIBS),y)
+define TOOLCHAIN_EXTERNAL_REMOVE_SHARED_LIBS
+	$(Q)$(call MESSAGE,"Removing shared libraries from toolchain...")
+	GCC_LIBRARY_PATHS=`$(TOOLCHAIN_EXTERNAL_CC) -print-search-dirs | sed -r -e '/libraries: =(.+)/!d; s//\1/; s/:/\n/g' | xargs readlink -f | grep -v 'gcc\|/[0-9.]\+$$'` ; \
+	find $${GCC_LIBRARY_PATHS} -name *.so* -delete
+endef
+endif
+
 # Create a symlink from (usr/)$(ARCH_LIB_DIR) to lib.
 # Note: the skeleton package additionally creates lib32->lib or lib64->lib
 # (as appropriate)
@@ -565,6 +578,7 @@ $(2)_TOOLCHAIN_WRAPPER_ARGS += $$(TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS)
 $(2)_BUILD_CMDS = $$(TOOLCHAIN_WRAPPER_BUILD)
 
 define $(2)_INSTALL_STAGING_CMDS
+	$$(TOOLCHAIN_EXTERNAL_REMOVE_SHARED_LIBS)
 	$$(TOOLCHAIN_WRAPPER_INSTALL)
 	$$(TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK)
 	$$(TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS)
Signature · 1 line
2.17.1
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#2 lore raw

PATCH v2 1/1 toolchain-ext: rm shared libs for static builds

In reply to Matt Weber

Hello,

toolchain/toolchain-external/pkg-toolchain-external.mk define TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS · line 456

Quoted 10 lines · On Sat, 26 Oct 2019 03:52:22 -0500 Matt Weber…
toolchain/toolchain-external/pkg-toolchain-external.mk +10 -0
+# NOTE: below the readlink call follows/builds each absolute path with any
+# invalid paths failing (falling out of the list). Thus no checking of paths
+# existing is required before doing the find. This assumes that GCC returns
+# at least one directory as a working toolchain should always have search
+# paths.
+ifeq ($(BR2_STATIC_LIBS),y)
+define TOOLCHAIN_EXTERNAL_REMOVE_SHARED_LIBS
+	$(Q)$(call MESSAGE,"Removing shared libraries from toolchain...")
+	GCC_LIBRARY_PATHS=`$(TOOLCHAIN_EXTERNAL_CC) -print-search-dirs | sed -r -e '/libraries: =(.+)/!d; s//\1/; s/:/\n/g' | xargs readlink -f | grep -v 'gcc\|/[0-9.]\+$$'` ; \
+	find $${GCC_LIBRARY_PATHS} -name *.so* -delete

As discussed live during the meeting, this doesn't work for pre-installed toolchain, because you are removing files directly from the original toolchain location, which could be read-only. And even if it's not read-only, we don't want to tamper with the pre-installed toolchain of the user.

So another solution is needed I'm afraid.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com