toolchain-external: don't exclude too much lib in sysroot rsync

PATCH New Thomas De Schampheleire 3 participants 7 messages Jan 28, 2016 – Feb 1, 2016

lore

Share & tools

lore

Thomas De Schampheleire

author

2016-01-28 12:32 UTC #0 lore raw

PATCH v12 1/3 toolchain-external: don't exclude too much lib in sysroot rsync

From: Thomas De Schampheleire <[email protected]>

The copy_toolchain_sysroot helper in toolchain/helpers.mk performs an rsync of various directories from the extracted external toolchain to the corresponding directory in staging.

The relevant (simplified) snippet is:

    for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
            rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale' \
                    --exclude lib --exclude lib32 --exclude lib64 \
                    $${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
    done ; \

The exclusion logic of lib/lib32/lib64 has been added by commit 5628776c4a4d29d0715633ea463b64cc19e19c5a with the purpose of only copying the relevant usr/lib* directory from the toolchain to staging, instead of all. For example, if ARCH_LIB_DIR is 'lib64', then only usr/lib64 would be copied and usr/lib and usr/lib32 are ignored. It works by ignoring any lib/lib32/lib64 subdirectory on the rsync of 'usr' and then separately copying usr/{lib,lib32,lib64} as appropriate. (The exclusion rules only have impact on the files beneath the main source directory.)

However, on the rsync of 'usr', ANY of the following directories AND files would be excluded:

    lib/
    lib
    lib32/
    foobar/something/lib/
    something-else/lib64/

while it is only the intention to skip directories directly under usr.

Therefore, add a leading (to restrict the scope to first-level) and trailing (to restrict to directories) slash to the exclude pattern. From 'man rsync':

- if the pattern starts with a / then it is anchored to [..] the root of the transfer.
- if the pattern ends with a / then it will only match a directory, not a regular file, symlink, or device.

Signed-off-by Thomas De Schampheleire <[email protected]>

Cc Samuel Martin <[email protected]>

Cc Thomas Petazzoni <[email protected]>

Cc Arnout Vandecappelle <[email protected]>

---
v12: unchanged
v11: unchanged
v10: new patch

 toolchain/helpers.mk | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

toolchain/helpers.mk +2 -1

index 70695ee..906993d 100644
@@ -154,7 +154,8 @@ copy_toolchain_sysroot = \
 	for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
 		if [ -d $${ARCH_SYSROOT_DIR}/$$i ] ; then \
 			rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale' \
-				--exclude lib --exclude lib32 --exclude lib64 \
+				--exclude '/lib/' --exclude '/lib32/' \
+				--exclude '/lib64/' \
 				$${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
 		fi ; \
 	done ; \

Signature · 1 line

2.4.10

Thomas De Schampheleire

author

2016-01-28 12:32 UTC #1 lore raw

PATCH v12 2/3 toolchain-external: improve sysroot rsync if ARCH_LIB_DIR != lib/lib32/lib64

From: Thomas De Schampheleire <[email protected]>

The copy_toolchain_sysroot helper in toolchain/helpers.mk performs an rsync of various directories from the extracted external toolchain to the corresponding directory in staging.

The relevant (simplified) snippet is:

	for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
		rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale' \
			--exclude '/lib/' --exclude '/lib32/' \
			--exclude '/lib64/' \
			$${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
	done ; \

The exclusion logic of lib/lib32/lib64 has originally been added by commit 5628776c4a4d29d0715633ea463b64cc19e19c5a with the purpose of only copying the relevant usr/lib* directory from the toolchain to staging, instead of all. For example, if ARCH_LIB_DIR is 'lib64', then only usr/lib64 would be copied and usr/lib and usr/lib32 are ignored. It works by ignoring any lib/lib32/lib64 subdirectory on the rsync of 'usr' and then separately copying usr/{lib,lib32,lib64} as appropriate. (The exclusion rules only have impact on the files beneath the main source directory.)

However, ARCH_LIB_DIR can take other values than (lib, lib32, lib64), for example lib32-fp or lib64-fp (Octeon III toolchain with -march=octeon3). In the existing code, the rsync for 'usr' would then already copy these lib directories, and the next rsync for 'usr/$${ARCH_LIB_DIR}' does nothing.

By itself, this is not a very big problem: the staging directory simply has some extra directories. However, a subsequent patch will create a staging symlink from $${ARCH_LIB_DIR} to lib. The first rsync would then overwrite that symlink with the real directory usr/$${ARCH_LIB_DIR} from the toolchain, which is not correct.

Assuming the patch that creates the symlink ARCH_LIB_DIR->lib is applied, the original situation after 'make clean toolchain' with an ARCH_LIB_DIR=lib32-fp is:

$ ls -ld output/staging/{,usr/}lib* output/target/{usr/,}lib*
drwxr-xr-x 2  4096 May 26  2015 output/staging/lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/staging/lib32 -> lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/staging/lib32-fp -> lib
drwxr-xr-x 2  4096 Jan 20 13:47 output/staging/usr/lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/staging/usr/lib32 -> lib
drwxr-xr-x 4  4096 May 26  2015 output/staging/usr/lib32-fp
drwxr-xr-x 4  4096 May 26  2015 output/staging/usr/lib64-fp
drwxr-xr-x 4  4096 May 26  2015 output/staging/usr/libexec
drwxr-xr-x 3  4096 May 26  2015 output/staging/usr/libexec32
drwxr-xr-x 3  4096 May 26  2015 output/staging/usr/libexec32-fp
drwxr-xr-x 3  4096 May 26  2015 output/staging/usr/libexec64-fp
drwxr-xr-x 2  4096 Jan 20 13:48 output/target/lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/target/lib32 -> lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/target/lib32-fp -> lib
drwxr-xr-x 2  4096 Jan 20 13:48 output/target/usr/lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/target/usr/lib32 -> lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/target/usr/lib32-fp -> lib

Notice how usr/lib32-fp is not a symlink but a directory, and the presence of an unnecessary directory usr/lib64-fp.

This patch improves the rsync exclusion rules by excluding any lib* directory on the first rsync. As this would also exclude any libexec/libexec32/... directory, explicitly include them first (first match takes precedence). This (as is already the case today) results in more usr/libexec* directories than needed, but it is not touched by this patch.

With the fix applied, the situation becomes:

drwxr-xr-x 2 4096 May 26 2015 output/staging/lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/staging/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/staging/lib32-fp -> lib
drwxr-xr-x 4 4096 May 26 2015 output/staging/usr/lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/staging/usr/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/staging/usr/lib32-fp -> lib

drwxr-xr-x 4 4096 May 26  2015 output/staging/usr/libexec
drwxr-xr-x 3 4096 May 26  2015 output/staging/usr/libexec32
drwxr-xr-x 3 4096 May 26  2015 output/staging/usr/libexec32-fp
drwxr-xr-x 3 4096 May 26  2015 output/staging/usr/libexec64-fp

drwxr-xr-x 2 4096 Jan 20 14:27 output/target/lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/target/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/target/lib32-fp -> lib
drwxr-xr-x 2 4096 Jan 20 14:27 output/target/usr/lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/target/usr/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/target/usr/lib32-fp -> lib

For cases where ARCH_LIB_DIR is one of lib, lib32 or lib64 this fix makes no difference, and likewise for internal toolchains.

Signed-off-by Thomas De Schampheleire <[email protected]>

Cc Samuel Martin <[email protected]>

Cc Thomas Petazzoni <[email protected]>

Cc Arnout Vandecappelle <[email protected]>

---
v12: unchanged
v11: unchanged
v10: new patch

 toolchain/helpers.mk | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

toolchain/helpers.mk +1 -2

index 906993d..02cc0bb 100644
@@ -154,8 +154,7 @@ copy_toolchain_sysroot = \
 	for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
 		if [ -d $${ARCH_SYSROOT_DIR}/$$i ] ; then \
 			rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale' \
-				--exclude '/lib/' --exclude '/lib32/' \
-				--exclude '/lib64/' \
+				--include '/libexec*/' --exclude '/lib*/' \
 				$${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
 		fi ; \
 	done ; \

Signature · 1 line

2.4.10

Romain Naour

2016-01-31 19:57 UTC #2 lore raw

PATCH v12 2/3 toolchain-external: improve sysroot rsync if ARCH_LIB_DIR != lib/lib32/lib64

Hi Thomas,

Le 28/01/2016 13:32, Thomas De Schampheleire a ?crit :

Quoted 85 lines

From: Thomas De Schampheleire <[email protected]>
The copy_toolchain_sysroot helper in toolchain/helpers.mk performs an rsync of various directories from the extracted external toolchain to the corresponding directory in staging.
The relevant (simplified) snippet is:
	for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
		rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale' \
			--exclude '/lib/' --exclude '/lib32/' \
			--exclude '/lib64/' \
			$${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
	done ; \
The exclusion logic of lib/lib32/lib64 has originally been added by commit 5628776c4a4d29d0715633ea463b64cc19e19c5a with the purpose of only copying the relevant usr/lib* directory from the toolchain to staging, instead of all. For example, if ARCH_LIB_DIR is 'lib64', then only usr/lib64 would be copied and usr/lib and usr/lib32 are ignored. It works by ignoring any lib/lib32/lib64 subdirectory on the rsync of 'usr' and then separately copying usr/{lib,lib32,lib64} as appropriate. (The exclusion rules only have impact on the files beneath the main source directory.)
However, ARCH_LIB_DIR can take other values than (lib, lib32, lib64), for example lib32-fp or lib64-fp (Octeon III toolchain with -march=octeon3). In the existing code, the rsync for 'usr' would then already copy these lib directories, and the next rsync for 'usr/$${ARCH_LIB_DIR}' does nothing.
By itself, this is not a very big problem: the staging directory simply has some extra directories. However, a subsequent patch will create a staging symlink from $${ARCH_LIB_DIR} to lib. The first rsync would then overwrite that symlink with the real directory usr/$${ARCH_LIB_DIR} from the toolchain, which is not correct.
Assuming the patch that creates the symlink ARCH_LIB_DIR->lib is applied, the original situation after 'make clean toolchain' with an ARCH_LIB_DIR=lib32-fp is:
$ ls -ld output/staging/{,usr/}lib* output/target/{usr/,}lib*
drwxr-xr-x 2  4096 May 26  2015 output/staging/lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/staging/lib32 -> lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/staging/lib32-fp -> lib
drwxr-xr-x 2  4096 Jan 20 13:47 output/staging/usr/lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/staging/usr/lib32 -> lib
drwxr-xr-x 4  4096 May 26  2015 output/staging/usr/lib32-fp
drwxr-xr-x 4  4096 May 26  2015 output/staging/usr/lib64-fp
drwxr-xr-x 4  4096 May 26  2015 output/staging/usr/libexec
drwxr-xr-x 3  4096 May 26  2015 output/staging/usr/libexec32
drwxr-xr-x 3  4096 May 26  2015 output/staging/usr/libexec32-fp
drwxr-xr-x 3  4096 May 26  2015 output/staging/usr/libexec64-fp
drwxr-xr-x 2  4096 Jan 20 13:48 output/target/lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/target/lib32 -> lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/target/lib32-fp -> lib
drwxr-xr-x 2  4096 Jan 20 13:48 output/target/usr/lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/target/usr/lib32 -> lib
lrwxrwxrwx 1     3 Jan 20 13:47 output/target/usr/lib32-fp -> lib
Notice how usr/lib32-fp is not a symlink but a directory, and the presence of an unnecessary directory usr/lib64-fp.
This patch improves the rsync exclusion rules by excluding any lib* directory on the first rsync. As this would also exclude any libexec/libexec32/... directory, explicitly include them first (first match takes precedence). This (as is already the case today) results in more usr/libexec* directories than needed, but it is not touched by this patch.
With the fix applied, the situation becomes:
drwxr-xr-x 2 4096 May 26 2015 output/staging/lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/staging/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/staging/lib32-fp -> lib
drwxr-xr-x 4 4096 May 26 2015 output/staging/usr/lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/staging/usr/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/staging/usr/lib32-fp -> lib
drwxr-xr-x 4 4096 May 26  2015 output/staging/usr/libexec
drwxr-xr-x 3 4096 May 26  2015 output/staging/usr/libexec32
drwxr-xr-x 3 4096 May 26  2015 output/staging/usr/libexec32-fp
drwxr-xr-x 3 4096 May 26  2015 output/staging/usr/libexec64-fp
drwxr-xr-x 2 4096 Jan 20 14:27 output/target/lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/target/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/target/lib32-fp -> lib
drwxr-xr-x 2 4096 Jan 20 14:27 output/target/usr/lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/target/usr/lib32 -> lib
lrwxrwxrwx 1 3 Jan 20 14:27 output/target/usr/lib32-fp -> lib

Quoted 8 lines

For cases where ARCH_LIB_DIR is one of lib, lib32 or lib64 this fix makes no difference, and likewise for internal toolchains.
Signed-off-by Thomas De Schampheleire <[email protected]>
Cc Samuel Martin <[email protected]>
Cc Thomas Petazzoni <[email protected]>
Cc Arnout Vandecappelle <[email protected]>

Reviewed-by Romain Naour <[email protected]>

Best regards,
Romain

Quoted 23 lines

---
v12: unchanged
v11: unchanged
v10: new patch

 toolchain/helpers.mk | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

toolchain/helpers.mk +1 -2

index 906993d..02cc0bb 100644
@@ -154,8 +154,7 @@ copy_toolchain_sysroot = \
 	for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
 		if [ -d $${ARCH_SYSROOT_DIR}/$$i ] ; then \
 			rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale' \
-				--exclude '/lib/' --exclude '/lib32/' \
-				--exclude '/lib64/' \
+				--include '/libexec*/' --exclude '/lib*/' \
 				$${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
 		fi ; \
 	done ; \

Thomas De Schampheleire

author

2016-01-28 12:32 UTC #3 lore raw

PATCH v12 3/3 toolchain-external: create symlink ARCH_LIB_DIR->lib

In reply to Thomas De Schampheleire

From: Thomas De Schampheleire <[email protected]>

Currently, following symbolic links are created in both target and staging directories:
- lib(32|64) --> lib
- usr/lib(32|64) --> lib

The decision for lib32 or lib64 is based on the target architecture configuration in buildroot (BR2_ARCH_IS_64).

In at least one case this is not correct: when building for a Cavium Octeon III processor using the toolchain from the Cavium Networks SDK, and specifying -march=octeon3 in BR2_TARGET_OPTIMIZATION, libraries are expected in directory 'lib32-fp' rather than 'lib32' (ABI=n32; likewise for lib64-fp in case of ABI=n64)

More generally the correct symbolic link is from (usr/)${ARCH_LIB_DIR}->lib. However, feedback from Arnout Vandecappelle is that there are packages that do depend on the lib32/lib64 symlink, even if ARCH_LIB_DIR is different. Hence, these links must be kept.

Fix the problem as follows:
- For internal toolchains: no change
- For external toolchains: create a symlink ARCH_LIB_DIR->lib if (usr/)ARCH_LIB_DIR does not exist yet.

Signed-off-by Thomas De Schampheleire <[email protected]>

Cc Thomas Petazzoni <[email protected]>

Cc Arnout Vandecappelle <[email protected]>

Cc "Yann E. Morin" <[email protected]>

Cc Romain Naour <[email protected]>

Cc Peter Korsgaard <[email protected]>

---
v12:
- 'test -e' instead of 'test -f' to avoid broken symlinks lib->lib (noticed by Romain)
v11:
- check for existence of destination instead of explicitly checking on the known values lib/lib32/lib64. (ThomasP)
v10:
- simplify after realization that skeleton symlink creation can be kept (thanks Thomas Petazzoni for noticing this)
v9:
- remove redundant mkdir's (handled by skeleton) (Yann)
v8:
- use helper only for external toolchain and incorporate ARCH_LIB_DIR definition (Arnout)
- keep lib32/lib64->lib symlink anyway
v7: rebase
v6: rebase only
v5:
- move internal toolchain logic into gcc-initial.mk
- also silence the internal toolchain link steps with $(Q)
v4:
- merge both helpers into one
- remove the separate target for the internal toolchain and hook into gcc-initial
- re-add deleted comment about MIPS64/n32
v3:
- update commit message wrapping
- change dependency on $(BUILD_DIR) to a order-only dependency v2:
- fix 'lib32-fp' leftover in toolchain-buildroot
- silence commands creating symlink with $(Q)
- fix case where ARCH_LIB_DIR is 'lib'

Note: in output/staging/usr/ there would still be more directories than I think are really necessary. This behavior is not changed by this patch, it was already present before.
For example, with the mentioned Octeon III toolchain, output/staging/usr/ contains:

    bin      bin32      bin32-fp      bin64-fp,
    lib      lib32      lib32-fp      lib64-fp
    libexec  libexec32  libexec32-fp  libexec64-fp
    sbin     sbin32     sbin32-fp     sbin64-fp

where bin/lib/libexec/sbin seem to be the 64-bit equivalents of bin32/lib32/libexec32/sbin32.
This is related to the behavior of copy_toolchain_sysroot in toolchain/helpers.mk. It already attempts to filter out the unnecessary lib* directories, but does not care about any bin/sbin/libexec directories. As this poses no known problem and is not impacted by this patch, I make no attempt to change it.

 toolchain/toolchain-external/toolchain-external.mk | 23 ++++++++++++++++++++++
 1 file changed, 23 insertions(+)

toolchain/toolchain-external/toolchain-external.mk +23 -0

index ddefd01..518afd6 100644
@@ -517,6 +517,27 @@ endef
 TOOLCHAIN_EXTERNAL_POST_INSTALL_STAGING_HOOKS += TOOLCHAIN_EXTERNAL_MUSL_LD_LINK
 endif
 
+# Create a symlink from (usr/)$(ARCH_LIB_DIR) to lib.
+# Note: the skeleton package additionally creates lib32->lib or lib64->lib
+# (as appropriate)
+#
+# $1: destination directory (TARGET_DIR / STAGING_DIR)
+create_lib_symlinks = \
+       $(Q)DESTDIR="$(strip $1)" ; \
+       ARCH_LIB_DIR="$(call toolchain_find_libdir,$(TOOLCHAIN_EXTERNAL_CC) $(TOOLCHAIN_EXTERNAL_CFLAGS))" ; \
+       if [ ! -e "$${DESTDIR}/$${ARCH_LIB_DIR}" -a ! -e "$${DESTDIR}/usr/$${ARCH_LIB_DIR}" ]; then \
+               ln -snf lib "$${DESTDIR}/$${ARCH_LIB_DIR}" ; \
+               ln -snf lib "$${DESTDIR}/usr/$${ARCH_LIB_DIR}" ; \
+       fi
+
+define TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK
+       $(call create_lib_symlinks,$(STAGING_DIR))
+endef
+
+define TOOLCHAIN_EXTERNAL_CREATE_TARGET_LIB_SYMLINK
+       $(call create_lib_symlinks,$(TARGET_DIR))
+endef
+
 # Integration of the toolchain into Buildroot: find the main sysroot
 # and the variant-specific sysroot, then copy the needed libraries to
 # the $(TARGET_DIR) and copy the whole sysroot (libraries and headers)

@@ -732,6 +753,7 @@ endef
 TOOLCHAIN_EXTERNAL_BUILD_CMDS = $(TOOLCHAIN_BUILD_WRAPPER)
 
 define TOOLCHAIN_EXTERNAL_INSTALL_STAGING_CMDS
+	$(TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_GDBINIT)

@@ -741,6 +763,7 @@ endef
 # and the target directory, we do everything within the
 # install-staging step, arbitrarily.
 define TOOLCHAIN_EXTERNAL_INSTALL_TARGET_CMDS
+	$(TOOLCHAIN_EXTERNAL_CREATE_TARGET_LIB_SYMLINK)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_TARGET_LIBS)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FDPIC)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FLAT)

Signature · 1 line

2.4.10

Romain Naour

2016-01-31 19:57 UTC #4 lore raw

PATCH v12 3/3 toolchain-external: create symlink ARCH_LIB_DIR->lib

Hi Thomas,

Le 28/01/2016 13:32, Thomas De Schampheleire a ?crit :

Quoted 26 lines

From: Thomas De Schampheleire <[email protected]>
Currently, following symbolic links are created in both target and staging directories:
- lib(32|64) --> lib
- usr/lib(32|64) --> lib
The decision for lib32 or lib64 is based on the target architecture configuration in buildroot (BR2_ARCH_IS_64).
In at least one case this is not correct: when building for a Cavium Octeon III processor using the toolchain from the Cavium Networks SDK, and specifying -march=octeon3 in BR2_TARGET_OPTIMIZATION, libraries are expected in directory 'lib32-fp' rather than 'lib32' (ABI=n32; likewise for lib64-fp in case of ABI=n64)
More generally the correct symbolic link is from (usr/)${ARCH_LIB_DIR}->lib. However, feedback from Arnout Vandecappelle is that there are packages that do depend on the lib32/lib64 symlink, even if ARCH_LIB_DIR is different. Hence, these links must be kept.
Fix the problem as follows:
- For internal toolchains: no change
- For external toolchains: create a symlink ARCH_LIB_DIR->lib if (usr/)ARCH_LIB_DIR does not exist yet.

Quoted 6 lines

Signed-off-by Thomas De Schampheleire <[email protected]>
Cc Thomas Petazzoni <[email protected]>
Cc Arnout Vandecappelle <[email protected]>
Cc "Yann E. Morin" <[email protected]>
Cc Romain Naour <[email protected]>
Cc Peter Korsgaard <[email protected]>

Thanks for this new version!

Reviewed-by Romain Naour <[email protected]>

Tested-by Romain Naour <[email protected]>

Best regards,
Romain

Quoted 105 lines

---
v12:
- 'test -e' instead of 'test -f' to avoid broken symlinks lib->lib (noticed by Romain)
v11:
- check for existence of destination instead of explicitly checking on the known values lib/lib32/lib64. (ThomasP)
v10:
- simplify after realization that skeleton symlink creation can be kept (thanks Thomas Petazzoni for noticing this)
v9:
- remove redundant mkdir's (handled by skeleton) (Yann)
v8:
- use helper only for external toolchain and incorporate ARCH_LIB_DIR definition (Arnout)
- keep lib32/lib64->lib symlink anyway
v7: rebase
v6: rebase only
v5:
- move internal toolchain logic into gcc-initial.mk
- also silence the internal toolchain link steps with $(Q)
v4:
- merge both helpers into one
- remove the separate target for the internal toolchain and hook into gcc-initial
- re-add deleted comment about MIPS64/n32
v3:
- update commit message wrapping
- change dependency on $(BUILD_DIR) to a order-only dependency v2:
- fix 'lib32-fp' leftover in toolchain-buildroot
- silence commands creating symlink with $(Q)
- fix case where ARCH_LIB_DIR is 'lib'
Note: in output/staging/usr/ there would still be more directories than I think are really necessary. This behavior is not changed by this patch, it was already present before.
For example, with the mentioned Octeon III toolchain, output/staging/usr/ contains:
    bin      bin32      bin32-fp      bin64-fp,
    lib      lib32      lib32-fp      lib64-fp
    libexec  libexec32  libexec32-fp  libexec64-fp
    sbin     sbin32     sbin32-fp     sbin64-fp
where bin/lib/libexec/sbin seem to be the 64-bit equivalents of bin32/lib32/libexec32/sbin32.
This is related to the behavior of copy_toolchain_sysroot in toolchain/helpers.mk. It already attempts to filter out the unnecessary lib* directories, but does not care about any bin/sbin/libexec directories. As this poses no known problem and is not impacted by this patch, I make no attempt to change it.
 toolchain/toolchain-external/toolchain-external.mk | 23 ++++++++++++++++++++++
 1 file changed, 23 insertions(+)
toolchain/toolchain-external/toolchain-external.mk +23 -0
index ddefd01..518afd6 100644
@@ -517,6 +517,27 @@ endef
 TOOLCHAIN_EXTERNAL_POST_INSTALL_STAGING_HOOKS += TOOLCHAIN_EXTERNAL_MUSL_LD_LINK
 endif
 
+# Create a symlink from (usr/)$(ARCH_LIB_DIR) to lib.
+# Note: the skeleton package additionally creates lib32->lib or lib64->lib
+# (as appropriate)
+#
+# $1: destination directory (TARGET_DIR / STAGING_DIR)
+create_lib_symlinks = \
+       $(Q)DESTDIR="$(strip $1)" ; \
+       ARCH_LIB_DIR="$(call toolchain_find_libdir,$(TOOLCHAIN_EXTERNAL_CC) $(TOOLCHAIN_EXTERNAL_CFLAGS))" ; \
+       if [ ! -e "$${DESTDIR}/$${ARCH_LIB_DIR}" -a ! -e "$${DESTDIR}/usr/$${ARCH_LIB_DIR}" ]; then \
+               ln -snf lib "$${DESTDIR}/$${ARCH_LIB_DIR}" ; \
+               ln -snf lib "$${DESTDIR}/usr/$${ARCH_LIB_DIR}" ; \
+       fi
+
+define TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK
+       $(call create_lib_symlinks,$(STAGING_DIR))
+endef
+
+define TOOLCHAIN_EXTERNAL_CREATE_TARGET_LIB_SYMLINK
+       $(call create_lib_symlinks,$(TARGET_DIR))
+endef
+
 # Integration of the toolchain into Buildroot: find the main sysroot
 # and the variant-specific sysroot, then copy the needed libraries to
 # the $(TARGET_DIR) and copy the whole sysroot (libraries and headers)
@@ -732,6 +753,7 @@ endef
 TOOLCHAIN_EXTERNAL_BUILD_CMDS = $(TOOLCHAIN_BUILD_WRAPPER)
 
 define TOOLCHAIN_EXTERNAL_INSTALL_STAGING_CMDS
+	$(TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_GDBINIT)
@@ -741,6 +763,7 @@ endef
 # and the target directory, we do everything within the
 # install-staging step, arbitrarily.
 define TOOLCHAIN_EXTERNAL_INSTALL_TARGET_CMDS
+	$(TOOLCHAIN_EXTERNAL_CREATE_TARGET_LIB_SYMLINK)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_TARGET_LIBS)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FDPIC)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FLAT)

Romain Naour

2016-01-31 19:56 UTC #5 lore raw

PATCH v12 1/3 toolchain-external: don't exclude too much lib in sysroot rsync

In reply to Thomas De Schampheleire

Hi Thomas,

Le 28/01/2016 13:32, Thomas De Schampheleire a ?crit :

Quoted 40 lines

From: Thomas De Schampheleire <[email protected]>
The copy_toolchain_sysroot helper in toolchain/helpers.mk performs an rsync of various directories from the extracted external toolchain to the corresponding directory in staging.
The relevant (simplified) snippet is:
    for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
            rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale' \
                    --exclude lib --exclude lib32 --exclude lib64 \
                    $${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
    done ; \
The exclusion logic of lib/lib32/lib64 has been added by commit 5628776c4a4d29d0715633ea463b64cc19e19c5a with the purpose of only copying the relevant usr/lib* directory from the toolchain to staging, instead of all. For example, if ARCH_LIB_DIR is 'lib64', then only usr/lib64 would be copied and usr/lib and usr/lib32 are ignored. It works by ignoring any lib/lib32/lib64 subdirectory on the rsync of 'usr' and then separately copying usr/{lib,lib32,lib64} as appropriate. (The exclusion rules only have impact on the files beneath the main source directory.)
However, on the rsync of 'usr', ANY of the following directories AND files would be excluded:
    lib/
    lib
    lib32/
    foobar/something/lib/
    something-else/lib64/
while it is only the intention to skip directories directly under usr.
Therefore, add a leading (to restrict the scope to first-level) and trailing (to restrict to directories) slash to the exclude pattern. From 'man rsync':
- if the pattern starts with a / then it is anchored to [..] the root of the transfer.
- if the pattern ends with a / then it will only match a directory, not a regular file, symlink, or device.

Quoted 5 lines

Signed-off-by Thomas De Schampheleire <[email protected]>
Cc Samuel Martin <[email protected]>
Cc Thomas Petazzoni <[email protected]>
Cc Arnout Vandecappelle <[email protected]>

Reviewed-by Romain Naour <[email protected]>

Best regards,
Romain

Quoted 23 lines

---
v12: unchanged
v11: unchanged
v10: new patch

 toolchain/helpers.mk | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

toolchain/helpers.mk +2 -1

index 70695ee..906993d 100644
@@ -154,7 +154,8 @@ copy_toolchain_sysroot = \
 	for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
 		if [ -d $${ARCH_SYSROOT_DIR}/$$i ] ; then \
 			rsync -au --chmod=u=rwX,go=rX --exclude 'usr/lib/locale' \
-				--exclude lib --exclude lib32 --exclude lib64 \
+				--exclude '/lib/' --exclude '/lib32/' \
+				--exclude '/lib64/' \
 				$${ARCH_SYSROOT_DIR}/$$i/ $(STAGING_DIR)/$$i/ ; \
 		fi ; \
 	done ; \

Thomas Petazzoni

2016-02-01 13:26 UTC #6 lore raw

PATCH v12 1/3 toolchain-external: don't exclude too much lib in sysroot rsync

In reply to Thomas De Schampheleire

Dear Thomas De Schampheleire,

Quoted 5 lines · Thomas De Schampheleire

From: Thomas De Schampheleire <[email protected]>
The copy_toolchain_sysroot helper in toolchain/helpers.mk performs an rsync of various directories from the extracted external toolchain to the corresponding directory in staging.

[...]

Better late than never, I've finally applied your patch series, after making sure that a build + minimal run-time test of a minimal ARM system based on a Linaro toolchain still works fine.

Thanks for this work, and thanks to Romain for being brave enough to analyze, review and test those patches!

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com