toolchain-ext: rm shared libs for static builds
PATCH v2 1/1 toolchain-ext: rm shared libs for static builds
For cases where Buildroot is generating a build with BR2_STATIC_LIBS=y and there is an external toolchain being used, the STAGING_DIR and HOSTDIR need to be scrubbed of shared library *.so*. This patch updates the toolchain-external staging step to first clean out all shared libraries before a sysroot is created. Before this patch, if shared libraries were found in the GCC library search path(s), build systems might still pick these up during build/link and fail with an error like "ld: attempted static link of dynamic object".
NOTE: The Meson build system by default prefers external libraries to be shared libraries unless the developer has explicitly in their meson.build set each find_library() invocation to contain the static keyword (requires meson 0.51.0+). One example where this was noticed was with iputils-20190709 where a link time dependency on libatomic occurred because the prebuilt toolchain had provided both a static and shared option. Meson then generated a compile string including a fixed path to the toolchain's shared libatomic.so instead of static.
Fixed:
http://autobuild.buildroot.net/results/db1740b4777f436324218c52bc7b08e5c21b667d/
---
Changes v1 -> v2
[Yann
- Cleaned up weird Unicode chars
- Updated comment about dirs existing
--- .../toolchain-external/pkg-toolchain-external.mk | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
toolchain/toolchain-external/pkg-toolchain-external.mk +14 -0
@@ -444,6 +444,19 @@ define TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS $(call copy_toolchain_sysroot,$${SYSROOT_DIR},$${ARCH_SYSROOT_DIR},$${ARCH_SUBDIR},$${ARCH_LIB_DIR},$${SUPPORT_LIB_DIR}) endef +# NOTE: below the readlink call follows/builds each absolute path with any +# invalid paths failing (falling out of the list). Thus no checking of paths +# existing is required before doing the find. This assumes that GCC returns +# at least one directory as a working toolchain should always have search +# paths. +ifeq ($(BR2_STATIC_LIBS),y) +define TOOLCHAIN_EXTERNAL_REMOVE_SHARED_LIBS + $(Q)$(call MESSAGE,"Removing shared libraries from toolchain...") + GCC_LIBRARY_PATHS=`$(TOOLCHAIN_EXTERNAL_CC) -print-search-dirs | sed -r -e '/libraries: =(.+)/!d; s//\1/; s/:/\n/g' | xargs readlink -f | grep -v 'gcc\|/[0-9.]\+$$'` ; \ + find $${GCC_LIBRARY_PATHS} -name *.so* -delete +endef +endif + # Create a symlink from (usr/)$(ARCH_LIB_DIR) to lib. # Note: the skeleton package additionally creates lib32->lib or lib64->lib # (as appropriate)
@@ -565,6 +578,7 @@ $(2)_TOOLCHAIN_WRAPPER_ARGS += $$(TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS) $(2)_BUILD_CMDS = $$(TOOLCHAIN_WRAPPER_BUILD) define $(2)_INSTALL_STAGING_CMDS + $$(TOOLCHAIN_EXTERNAL_REMOVE_SHARED_LIBS) $$(TOOLCHAIN_WRAPPER_INSTALL) $$(TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK) $$(TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS)
Signature · 1 line
2.17.1
PATCH v2 1/1 toolchain-ext: rm shared libs for static builds
Matt, All,
On 2019-10-26 03:52 -0500, Matt Weber spake thusly:
Quoted 18 lines
For cases where Buildroot is generating a build with BR2_STATIC_LIBS=y and there is an external toolchain being used, the
STAGING_DIRand HOSTDIR need to be scrubbed of shared library *.so*. This patch updates the toolchain-external staging step to first clean out all shared libraries before a sysroot is created. Before this patch, if shared libraries were found in the GCC library search path(s), build systems might still pick these up during build/link and fail with an error like "ld: attempted static link of dynamic object".NOTE: The Meson build system by default prefers external libraries to be shared libraries unless the developer has explicitly in their meson.build set each
find_library()invocation to contain the static keyword (requires meson 0.51.0+). One example where this was noticed was with iputils-20190709 where a link time dependency on libatomic occurred because the prebuilt toolchain had provided both a static and shared option. Meson then generated a compile string including a fixed path to the toolchain's shared libatomic.so instead of static.
Quoted 5 lines
Fixed:
http://autobuild.buildroot.net/results/db1740b4777f436324218c52bc7b08e5c21b667d/Cc Yann E. MORIN <[email protected]>Signed-off-by Matthew Weber <[email protected]>
Regards,
Yann E. MORIN.
Quoted 44 lines
---
Changes v1 -> v2
[Yann
- Cleaned up weird Unicode chars
- Updated comment about dirs existing--- .../toolchain-external/pkg-toolchain-external.mk | 14 ++++++++++++++ 1 file changed, 14 insertions(+)toolchain/toolchain-external/pkg-toolchain-external.mk +14
-0@@ -444,6 +444,19 @@ define TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS $(call copy_toolchain_sysroot,$${SYSROOT_DIR},$${ARCH_SYSROOT_DIR},$${ARCH_SUBDIR},$${ARCH_LIB_DIR},$${SUPPORT_LIB_DIR}) endef +# NOTE: below the readlink call follows/builds each absolute path with any +# invalid paths failing (falling out of the list). Thus no checking of paths +# existing is required before doing the find. This assumes that GCC returns +# at least one directory as a working toolchain should always have search +# paths. +ifeq ($(BR2_STATIC_LIBS),y) +define TOOLCHAIN_EXTERNAL_REMOVE_SHARED_LIBS + $(Q)$(call MESSAGE,"Removing shared libraries from toolchain...") + GCC_LIBRARY_PATHS=`$(TOOLCHAIN_EXTERNAL_CC) -print-search-dirs | sed -r -e '/libraries: =(.+)/!d; s//\1/; s/:/\n/g' | xargs readlink -f | grep -v 'gcc\|/[0-9.]\+$$'` ; \ + find $${GCC_LIBRARY_PATHS} -name *.so* -delete +endef +endif + # Create a symlink from (usr/)$(ARCH_LIB_DIR) to lib. # Note: the skeleton package additionally creates lib32->lib or lib64->lib # (as appropriate)@@ -565,6 +578,7 @@ $(2)_TOOLCHAIN_WRAPPER_ARGS += $$(TOOLCHAIN_EXTERNAL_TOOLCHAIN_WRAPPER_ARGS) $(2)_BUILD_CMDS = $$(TOOLCHAIN_WRAPPER_BUILD) define $(2)_INSTALL_STAGING_CMDS + $$(TOOLCHAIN_EXTERNAL_REMOVE_SHARED_LIBS) $$(TOOLCHAIN_WRAPPER_INSTALL) $$(TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK) $$(TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS)Signature · 1 line
2.17.1
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH v2 1/1 toolchain-ext: rm shared libs for static builds
In reply to Matt WeberHello,
toolchain/toolchain-external/pkg-toolchain-external.mk define TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS · line 456
Quoted 10 lines · On Sat, 26 Oct 2019 03:52:22 -0500 Matt Weber…
toolchain/toolchain-external/pkg-toolchain-external.mk +10
-0+# NOTE: below the readlink call follows/builds each absolute path with any +# invalid paths failing (falling out of the list). Thus no checking of paths +# existing is required before doing the find. This assumes that GCC returns +# at least one directory as a working toolchain should always have search +# paths. +ifeq ($(BR2_STATIC_LIBS),y) +define TOOLCHAIN_EXTERNAL_REMOVE_SHARED_LIBS + $(Q)$(call MESSAGE,"Removing shared libraries from toolchain...") + GCC_LIBRARY_PATHS=`$(TOOLCHAIN_EXTERNAL_CC) -print-search-dirs | sed -r -e '/libraries: =(.+)/!d; s//\1/; s/:/\n/g' | xargs readlink -f | grep -v 'gcc\|/[0-9.]\+$$'` ; \ + find $${GCC_LIBRARY_PATHS} -name *.so* -delete
As discussed live during the meeting, this doesn't work for pre-installed toolchain, because you are removing files directly from the original toolchain location, which could be read-only. And even if it's not read-only, we don't want to tamper with the pre-installed toolchain of the user.
So another solution is needed I'm afraid.
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com