Skip to contentSkip to conversation

Add support for Cavium Octeon III

PATCH New Thomas De Schampheleire 3 participants 10 messages Feb 18, 2015 – Jul 14, 2015 Kbuild

lore
Share & tools
lore
#0 lore raw

v3 cover Add support for Cavium Octeon III

From: Thomas De Schampheleire <[email protected]>

This series adds support for building for the Cavium Octeon III processor, using the Cavium Networks SDK. The specific thing about this toolchain is that the libraries for the hardware-floating point capable processors are in lib32-fp and lib64-fp, rather than lib32/lib64.

The first patch was submitted earlier, and is resent with updated commit message.

v3: fix successive make invocations due to dependency on BUILD_DIR
v2: fix some minor issues in the second patch (see patch description)

Bai Yingjie (1):
toolchain-external: improve lib subdirectory matching

Thomas De Schampheleire (1):
toolchain: create symlink to 'lib' from ARCH_LIB_DIR iso fixed

    lib32/lib64
 Makefile                                             | 14 +-------------
 toolchain/helpers.mk                                 | 16 ++++++++++++++++
 toolchain/toolchain-buildroot/toolchain-buildroot.mk | 11 +++++++++++
 toolchain/toolchain-external/toolchain-external.mk   | 16 ++++++++++++++--
 4 files changed, 42 insertions(+), 15 deletions(-)
Signature · 1 line
1.8.5.1
#1 lore raw

PATCH v3 1/2 toolchain-external: improve lib subdirectory matching

From: Bai Yingjie <[email protected]>

The toolchain from the Cavium Networks Octeon SDK provides a sysroot with library directories lib32, lib32-fp, lib64 and lib64-fp. The -fp variants are used for processors with hardware floating point unit, such as the Octeon III variants.

When specifying -march=octeon3 in BR2_TARGET_OPTIMIZATION, the toolchain will use lib32-fp, but currently Buildroot does not accept that pattern.

This patch improves the matching by accepting lib(32|64)?([^/]*)? as lib directory name.

[ThomasDS: update commit message]
---
 toolchain/toolchain-external/toolchain-external.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
toolchain/toolchain-external/toolchain-external.mk +2 -2
index 999a354..e05957c 100644
@@ -434,13 +434,13 @@ endef
 
 # Returns the sysroot location for the given compiler + flags
 define toolchain_find_sysroot
-$$(echo -n $(call toolchain_find_libc_a,$(1)) | sed -r -e 's:(usr/)?lib(32|64)?/([^/]*/)?libc\.a::')
+$$(echo -n $(call toolchain_find_libc_a,$(1)) | sed -r -e 's:(usr/)?lib(32|64)?([^/]*)?/([^/]*/)?libc\.a::')
 endef
 
 # Returns the lib subdirectory for the given compiler + flags (i.e
 # typically lib32 or lib64 for some toolchains)
 define toolchain_find_libdir
-$$(echo -n $(call toolchain_find_libc_a,$(1)) | sed -r -e 's:.*/(usr/)?(lib(32|64)?)/([^/]*/)?libc.a:\2:')
+$$(echo -n $(call toolchain_find_libc_a,$(1)) | sed -r -e 's:.*/(usr/)?(lib(32|64)?([^/]*)?)/([^/]*/)?libc.a:\2:')
 endef
 
 # Checks for an already installed toolchain: check the toolchain
Signature · 1 line
1.8.5.1
#3 lore raw

PATCH v3 2/2 toolchain: create symlink to 'lib' from ARCH_LIB_DIR iso fixed lib32/lib64

In reply to Thomas De Schampheleire

From: Thomas De Schampheleire <[email protected]>

Currently, following symbolic links are created in both target and staging directories:
- lib(32|64) --> lib
- usr/lib(32|64) --> lib

The decision for lib32 or lib64 is based on the target architecture configuration in buildroot (BR2_ARCH_IS_64).

In at least one case this is not correct: when building for a Cavium Octeon III processor using the toolchain from the Cavium Networks SDK, and specifying -march=octeon3 in BR2_TARGET_OPTIMIZATION, libraries are expected in directory 'lib32-fp' rather than 'lib32' (likewise for lib64-fp).

More generally, for external toolchains, the correct symbolic link is from (usr/)${ARCH_LIB_DIR} to lib. For internal toolchains, current toolchains always use either lib32 or lib64.

Fix the problem as follows:
- create symlink creation helpers in toolchain/helpers.mk
- for external toolchains, call these helpers based on ARCH_LIB_DIR
- for internal toolchains, call these helpers based on the existing fixed lib32/lib64 logic, moved from Makefile
- to fix build order problems, add the correct dependency on $(BUILD_DIR) from $(BUILD_DIR)/.root

---
v3:
- update commit message wrapping
- change dependency on $(BUILD_DIR) to a order-only dependency v2:
- fix 'lib32-fp' leftover in toolchain-buildroot
- silence commands creating symlink with $(Q)
- fix case where ARCH_LIB_DIR is 'lib'

Note: the handling in the internal toolchain is a bit awkward because it explicitly adds a new target, but I don't see a much better way: it must be done _before_ the toolchain is being built. The entire logic cannot be done in Makefile as ARCH_LIB_DIR is not available there.

---
 Makefile                                             | 14 +-------------
 toolchain/helpers.mk                                 | 16 ++++++++++++++++
 toolchain/toolchain-buildroot/toolchain-buildroot.mk | 11 +++++++++++
 toolchain/toolchain-external/toolchain-external.mk   | 12 ++++++++++++
 4 files changed, 40 insertions(+), 13 deletions(-)
Makefile +1 -13
index 338c992..de202b0 100644
@@ -449,20 +449,10 @@ world: target-post-image
 $(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST):
 	@mkdir -p $@
 
-# We make a symlink lib32->lib or lib64->lib as appropriate
-# MIPS64/n32 requires lib32 even though it's a 64-bit arch.
-ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y)
-LIB_SYMLINK = lib64
-else
-LIB_SYMLINK = lib32
-endif
-
 $(STAGING_DIR):
 	@mkdir -p $(STAGING_DIR)/bin
 	@mkdir -p $(STAGING_DIR)/lib
-	@ln -snf lib $(STAGING_DIR)/$(LIB_SYMLINK)
 	@mkdir -p $(STAGING_DIR)/usr/lib
-	@ln -snf lib $(STAGING_DIR)/usr/$(LIB_SYMLINK)
 	@mkdir -p $(STAGING_DIR)/usr/include
 	@mkdir -p $(STAGING_DIR)/usr/bin
 	@ln -snf $(STAGING_DIR) $(BASE_DIR)/staging
@@ -475,15 +465,13 @@ RSYNC_VCS_EXCLUSIONS = \
 	--exclude .svn --exclude .git --exclude .hg --exclude .bzr \
 	--exclude CVS
 
-$(BUILD_DIR)/.root:
+$(BUILD_DIR)/.root: | $(BUILD_DIR)
 	mkdir -p $(TARGET_DIR)
 	rsync -a --ignore-times $(RSYNC_VCS_EXCLUSIONS) \
 		--chmod=Du+w --exclude .empty --exclude '*~' \
 		$(TARGET_SKELETON)/ $(TARGET_DIR)/
 	$(INSTALL) -m 0644 support/misc/target-dir-warning.txt $(TARGET_DIR_WARNING_FILE)
-	@ln -snf lib $(TARGET_DIR)/$(LIB_SYMLINK)
 	@mkdir -p $(TARGET_DIR)/usr
-	@ln -snf lib $(TARGET_DIR)/usr/$(LIB_SYMLINK)
 	touch $@
 
 $(TARGET_DIR): $(BUILD_DIR)/.root
toolchain/helpers.mk +16 -0
index 3121da4..b4879e0 100644
@@ -1,5 +1,21 @@
 # This Makefile fragment declares toolchain related helper functions.
 
+# Create the necessary symlink from (usr/)lib32|lib64|lib32-fp|... to lib
+# In general, for external toolchains, the correct link name is $ARCH_LIB_DIR.
+create_staging_lib_symlink = \
+	LIB_SYMLINK="$(strip $1)" ; \
+	if [ "$${LIB_SYMLINK}" != "lib" ]; then \
+		ln -snf lib $(STAGING_DIR)/$${LIB_SYMLINK} ; \
+		ln -snf lib $(STAGING_DIR)/usr/$${LIB_SYMLINK} ; \
+	fi
+
+create_target_lib_symlink = \
+	LIB_SYMLINK="$(strip $1)" ; \
+	if [ "$${LIB_SYMLINK}" != "lib" ]; then \
+	    ln -snf lib $(TARGET_DIR)/$${LIB_SYMLINK} ; \
+	    ln -snf lib $(TARGET_DIR)/usr/$${LIB_SYMLINK} ; \
+	fi
+
 # The copy_toolchain_lib_root function copies a toolchain library and
 # its symbolic links from the sysroot directory to the target
 # directory. Note that this function is used both by the external
toolchain/toolchain-buildroot/toolchain-buildroot.mk +11 -0
index b30cc33..3ccb030 100644
@@ -12,6 +12,17 @@ BR_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC))
 
 TOOLCHAIN_BUILDROOT_DEPENDENCIES = host-gcc-final
 
+ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y)
+LIB_SYMLINK = lib64
+else
+LIB_SYMLINK = lib32
+endif
+
+host-gcc-final: create-lib-symlinks
+create-lib-symlinks: $(STAGING_DIR) $(TARGET_DIR)
+	$(call create_staging_lib_symlink,$(LIB_SYMLINK))
+	$(call create_target_lib_symlink,$(LIB_SYMLINK))
+
 TOOLCHAIN_BUILDROOT_ADD_TOOLCHAIN_DEPENDENCY = NO
 
 $(eval $(virtual-package))
toolchain/toolchain-external/toolchain-external.mk +12 -0
index e05957c..16e398c 100644
@@ -600,6 +600,16 @@ define TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS
 	$(call copy_toolchain_sysroot,$${SYSROOT_DIR},$${ARCH_SYSROOT_DIR},$${ARCH_SUBDIR},$${ARCH_LIB_DIR},$${SUPPORT_LIB_DIR})
 endef
 
+define TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK
+	$(Q)ARCH_LIB_DIR="$(call toolchain_find_libdir,$(TOOLCHAIN_EXTERNAL_CC) $(TOOLCHAIN_EXTERNAL_CFLAGS))" ; \
+	$(call create_staging_lib_symlink,$${ARCH_LIB_DIR})
+endef
+
+define TOOLCHAIN_EXTERNAL_CREATE_TARGET_LIB_SYMLINK
+	$(Q)ARCH_LIB_DIR="$(call toolchain_find_libdir,$(TOOLCHAIN_EXTERNAL_CC) $(TOOLCHAIN_EXTERNAL_CFLAGS))" ; \
+	$(call create_target_lib_symlink,$${ARCH_LIB_DIR})
+endef
+
 # Special installation target used on the Blackfin architecture when
 # FDPIC is not the primary binary format being used, but the user has
 # nonetheless requested the installation of the FDPIC libraries to the
@@ -699,6 +709,7 @@ define TOOLCHAIN_EXTERNAL_INSTALL_GDBINIT
 endef
 
 define TOOLCHAIN_EXTERNAL_INSTALL_STAGING_CMDS
+	$(TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_GDBINIT)
@@ -708,6 +719,7 @@ endef
 # and the target directory, we do everything within the
 # install-staging step, arbitrarily.
 define TOOLCHAIN_EXTERNAL_INSTALL_TARGET_CMDS
+	$(TOOLCHAIN_EXTERNAL_CREATE_TARGET_LIB_SYMLINK)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_TARGET_LIBS)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FDPIC)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FLAT)
Signature · 1 line
1.8.5.1
#2 lore raw

PATCH v3 1/2 toolchain-external: improve lib subdirectory matching

In reply to Thomas De Schampheleire

Dear Thomas De Schampheleire,

Quoted 13 lines · Thomas De Schampheleire

From: Bai Yingjie <[email protected]>

The toolchain from the Cavium Networks Octeon SDK provides a sysroot with library directories lib32, lib32-fp, lib64 and lib64-fp. The -fp variants are used for processors with hardware floating point unit, such as the Octeon III variants.

When specifying -march=octeon3 in BR2_TARGET_OPTIMIZATION, the toolchain will use lib32-fp, but currently Buildroot does not accept that pattern.

This patch improves the matching by accepting lib(32|64)?([^/]*)? as lib directory name.

Quoted 6 lines
[ThomasDS: update commit message]
---
 toolchain/toolchain-external/toolchain-external.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

I've added a comment above the change in the code to illustrate a bit the various cases we are trying to handle in these complicated sed expressions. And then I applied your patch. Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#4 lore raw

PATCH v3 2/2 toolchain: create symlink to 'lib' from ARCH_LIB_DIR iso fixed lib32/lib64

In reply to Thomas De Schampheleire

Hi Thomas,

Sorry that it is taking so long to make progress on this, but the external
toolchain stuff is, as you know, horribly demotivating... We've spent quite a bit of time to understand what this patch does and how it could be done better...

Quoted 19 lines · Thomas De Schampheleire

From: Thomas De Schampheleire <[email protected]>

Currently, following symbolic links are created in both target and staging directories:
- lib(32|64) --> lib
- usr/lib(32|64) --> lib

The decision for lib32 or lib64 is based on the target architecture configuration in buildroot (BR2_ARCH_IS_64).

In at least one case this is not correct: when building for a Cavium Octeon III processor using the toolchain from the Cavium Networks SDK, and specifying -march=octeon3 in BR2_TARGET_OPTIMIZATION, libraries are expected in directory 'lib32-fp' rather than 'lib32' (likewise for lib64-fp).

More generally, for external toolchains, the correct symbolic link is from (usr/)${ARCH_LIB_DIR} to lib. For internal toolchains, current toolchains always use either lib32 or lib64.

Quoted 7 lines

Fix the problem as follows:
- create symlink creation helpers in toolchain/helpers.mk
- for external toolchains, call these helpers based on ARCH_LIB_DIR
- for internal toolchains, call these helpers based on the existing fixed lib32/lib64 logic, moved from Makefile
- to fix build order problems, add the correct dependency on $(BUILD_DIR) from $(BUILD_DIR)/.root

It's not clear to us which build order problems are solved by this, and
especially how these build order problems are introduced by this patch... Could you explain that a little?

Quoted 28 lines

---
v3:
- update commit message wrapping
- change dependency on $(BUILD_DIR) to a order-only dependency v2:
- fix 'lib32-fp' leftover in toolchain-buildroot
- silence commands creating symlink with $(Q)
- fix case where ARCH_LIB_DIR is 'lib'

Note: the handling in the internal toolchain is a bit awkward because it explicitly adds a new target, but I don't see a much better way: it must be done _before_ the toolchain is being built. The entire logic cannot be done in Makefile as ARCH_LIB_DIR is not available there.

---
 Makefile                                             | 14 +-------------
 toolchain/helpers.mk                                 | 16 ++++++++++++++++
 toolchain/toolchain-buildroot/toolchain-buildroot.mk | 11 +++++++++++
 toolchain/toolchain-external/toolchain-external.mk   | 12 ++++++++++++
 4 files changed, 40 insertions(+), 13 deletions(-)
Makefile +0 -0
index 338c992..de202b0 100644

Makefile world: target-post-image · line 458

Quoted 10 lines
Makefile +0 -7
@@ -449,20 +449,10 @@ world: target-post-image
 $(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST):
 	@mkdir -p $@
 
-# We make a symlink lib32->lib or lib64->lib as appropriate
-# MIPS64/n32 requires lib32 even though it's a 64-bit arch.
-ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y)
-LIB_SYMLINK = lib64
-else
-LIB_SYMLINK = lib32
-endif

Yeah, it's good to see this disappear from top-level Makefile!

Quoted 39 lines
Makefile +1 -6
-
 $(STAGING_DIR):
 	@mkdir -p $(STAGING_DIR)/bin
 	@mkdir -p $(STAGING_DIR)/lib
-	@ln -snf lib $(STAGING_DIR)/$(LIB_SYMLINK)
 	@mkdir -p $(STAGING_DIR)/usr/lib
-	@ln -snf lib $(STAGING_DIR)/usr/$(LIB_SYMLINK)
 	@mkdir -p $(STAGING_DIR)/usr/include
 	@mkdir -p $(STAGING_DIR)/usr/bin
 	@ln -snf $(STAGING_DIR) $(BASE_DIR)/staging
@@ -475,15 +465,13 @@ RSYNC_VCS_EXCLUSIONS = \
 	--exclude .svn --exclude .git --exclude .hg --exclude .bzr \
 	--exclude CVS
 
-$(BUILD_DIR)/.root:
+$(BUILD_DIR)/.root: | $(BUILD_DIR)
 	mkdir -p $(TARGET_DIR)
 	rsync -a --ignore-times $(RSYNC_VCS_EXCLUSIONS) \
 		--chmod=Du+w --exclude .empty --exclude '*~' \
 		$(TARGET_SKELETON)/ $(TARGET_DIR)/
 	$(INSTALL) -m 0644 support/misc/target-dir-warning.txt $(TARGET_DIR_WARNING_FILE)
-	@ln -snf lib $(TARGET_DIR)/$(LIB_SYMLINK)
 	@mkdir -p $(TARGET_DIR)/usr
-	@ln -snf lib $(TARGET_DIR)/usr/$(LIB_SYMLINK)
 	touch $@
 
 $(TARGET_DIR): $(BUILD_DIR)/.root
toolchain/helpers.mk +5 -0
index 3121da4..b4879e0 100644
@@ -1,5 +1,21 @@
 # This Makefile fragment declares toolchain related helper functions.
 
+# Create the necessary symlink from (usr/)lib32|lib64|lib32-fp|... to lib
+# In general, for external toolchains, the correct link name is $ARCH_LIB_DIR.
+create_staging_lib_symlink = \
+	LIB_SYMLINK="$(strip $1)" ; \
+	if [ "$${LIB_SYMLINK}" != "lib" ]; then \

toolchain/helpers.mk line 17

Quoted 10 lines
toolchain/helpers.mk +10 -0
+		ln -snf lib $(STAGING_DIR)/$${LIB_SYMLINK} ; \
+		ln -snf lib $(STAGING_DIR)/usr/$${LIB_SYMLINK} ; \
+	fi
+
+create_target_lib_symlink = \
+	LIB_SYMLINK="$(strip $1)" ; \
+	if [ "$${LIB_SYMLINK}" != "lib" ]; then \
+	    ln -snf lib $(TARGET_DIR)/$${LIB_SYMLINK} ; \
+	    ln -snf lib $(TARGET_DIR)/usr/$${LIB_SYMLINK} ; \
+	fi

These two functions are identical except for the target dir, so create just one
function with the directory as a second parameter.

toolchain/toolchain-buildroot/toolchain-buildroot.mk BR_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC)) · line 15

Quoted 13 lines
toolchain/helpers.mk +1 -0
+
 # The copy_toolchain_lib_root function copies a toolchain library and
 # its symbolic links from the sysroot directory to the target
 # directory. Note that this function is used both by the external
toolchain/toolchain-buildroot/toolchain-buildroot.mk +1 -0
index b30cc33..3ccb030 100644
@@ -12,6 +12,17 @@ BR_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC))
 
 TOOLCHAIN_BUILDROOT_DEPENDENCIES = host-gcc-final
 
+ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y)

The comment about MIPS64/n32 should have been retained here.

toolchain/toolchain-buildroot/toolchain-buildroot.mk BR_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC)) · line 19

Quoted 4 lines
toolchain/toolchain-buildroot/toolchain-buildroot.mk +4 -0
+LIB_SYMLINK = lib64
+else
+LIB_SYMLINK = lib32
+endif

Since now it is something specific to this .mk file, perhaps it should be
prefixed with TOOLCHAIN_BUILDROOT_

toolchain/toolchain-buildroot/toolchain-buildroot.mk BR_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC)) · line 21

Quoted 2 lines
toolchain/toolchain-buildroot/toolchain-buildroot.mk +2 -0
+
+host-gcc-final: create-lib-symlinks

This is not just awkward, it also breaks top-level parallel build. At least, if
host-gcc-final really does need this to be done. In top-level parallel build, the order between the dependencies is not set, so it's possible that the symlinks are only created after the build of gcc has already started. Of course, it's vanishingly unlikely that it will not yet have finished by the time the gcc install step starts... Actually, we wonder if it is really needed before gcc-final, or even before the libc? Because the way that make evaluates things in the non-parallel case, this step would have been done before anything else, even before host-binutils...

If it really should be done before host-gcc-final, the full solution would be
to add this as a post-install hook to the libc. You could do something like this:

$(call uppercase,$(BR_LIBC))_POST_INSTALL_STAGING_HOOKS += \
	$(call create_staging_lib_symlink,$(LIB_SYMLINK))

Isn't that beautiful :-) This also has the advantage of splitting the staging and target installs into their appropriate locations.

Alternatively, you could add a tiny package without source that does this and
add that as a dependency to host-gcc-final.

Do you still have the energy to make some changes and resubmit?

Regards,
Arnout

Quoted 45 lines
toolchain/toolchain-buildroot/toolchain-buildroot.mk +4 -0
+create-lib-symlinks: $(STAGING_DIR) $(TARGET_DIR)
+	$(call create_staging_lib_symlink,$(LIB_SYMLINK))
+	$(call create_target_lib_symlink,$(LIB_SYMLINK))
+
 TOOLCHAIN_BUILDROOT_ADD_TOOLCHAIN_DEPENDENCY = NO
 
 $(eval $(virtual-package))
toolchain/toolchain-external/toolchain-external.mk +12 -0
index e05957c..16e398c 100644
@@ -600,6 +600,16 @@ define TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS
 	$(call copy_toolchain_sysroot,$${SYSROOT_DIR},$${ARCH_SYSROOT_DIR},$${ARCH_SUBDIR},$${ARCH_LIB_DIR},$${SUPPORT_LIB_DIR})
 endef
 
+define TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK
+	$(Q)ARCH_LIB_DIR="$(call toolchain_find_libdir,$(TOOLCHAIN_EXTERNAL_CC) $(TOOLCHAIN_EXTERNAL_CFLAGS))" ; \
+	$(call create_staging_lib_symlink,$${ARCH_LIB_DIR})
+endef
+
+define TOOLCHAIN_EXTERNAL_CREATE_TARGET_LIB_SYMLINK
+	$(Q)ARCH_LIB_DIR="$(call toolchain_find_libdir,$(TOOLCHAIN_EXTERNAL_CC) $(TOOLCHAIN_EXTERNAL_CFLAGS))" ; \
+	$(call create_target_lib_symlink,$${ARCH_LIB_DIR})
+endef
+
 # Special installation target used on the Blackfin architecture when
 # FDPIC is not the primary binary format being used, but the user has
 # nonetheless requested the installation of the FDPIC libraries to the
@@ -699,6 +709,7 @@ define TOOLCHAIN_EXTERNAL_INSTALL_GDBINIT
 endef
 
 define TOOLCHAIN_EXTERNAL_INSTALL_STAGING_CMDS
+	$(TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_GDBINIT)
@@ -708,6 +719,7 @@ endef
 # and the target directory, we do everything within the
 # install-staging step, arbitrarily.
 define TOOLCHAIN_EXTERNAL_INSTALL_TARGET_CMDS
+	$(TOOLCHAIN_EXTERNAL_CREATE_TARGET_LIB_SYMLINK)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_TARGET_LIBS)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FDPIC)
 	$(TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FLAT)
Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
#5 lore raw

PATCH v3 2/2 toolchain: create symlink to 'lib' from ARCH_LIB_DIR iso fixed lib32/lib64

Quoted 34 lines · Arnout Vandecappelle

Hi Thomas,

Sorry that it is taking so long to make progress on this, but the external
toolchain stuff is, as you know, horribly demotivating... We've spent quite a bit of time to understand what this patch does and how it could be done better...

Quoted 19 lines · Thomas De Schampheleire

From: Thomas De Schampheleire <[email protected]>

Currently, following symbolic links are created in both target and staging directories:
- lib(32|64) --> lib
- usr/lib(32|64) --> lib

The decision for lib32 or lib64 is based on the target architecture configuration in buildroot (BR2_ARCH_IS_64).

In at least one case this is not correct: when building for a Cavium Octeon III processor using the toolchain from the Cavium Networks SDK, and specifying -march=octeon3 in BR2_TARGET_OPTIMIZATION, libraries are expected in directory 'lib32-fp' rather than 'lib32' (likewise for lib64-fp).

More generally, for external toolchains, the correct symbolic link is from (usr/)${ARCH_LIB_DIR} to lib. For internal toolchains, current toolchains always use either lib32 or lib64.

Quoted 7 lines

Fix the problem as follows:
- create symlink creation helpers in toolchain/helpers.mk
- for external toolchains, call these helpers based on ARCH_LIB_DIR
- for internal toolchains, call these helpers based on the existing fixed lib32/lib64 logic, moved from Makefile
- to fix build order problems, add the correct dependency on $(BUILD_DIR) from $(BUILD_DIR)/.root

Quoted 3 lines

It's not clear to us which build order problems are solved by this, and
especially how these build order problems are introduced by this patch... Could you explain that a little?

As you may imagine, my memory of these issues is not so crisp anymore, but IIRC without these build-order changes it would not build correctly. I guess I was using 'make clean toolchain' and it would fail, but I would need to retest explicitly to be sure.

Quoted 86 lines
Quoted 29 lines

---
v3:
- update commit message wrapping
- change dependency on $(BUILD_DIR) to a order-only dependency v2:
- fix 'lib32-fp' leftover in toolchain-buildroot
- silence commands creating symlink with $(Q)
- fix case where ARCH_LIB_DIR is 'lib'

Note: the handling in the internal toolchain is a bit awkward because it explicitly adds a new target, but I don't see a much better way: it must be done _before_ the toolchain is being built. The entire logic cannot be done in Makefile as ARCH_LIB_DIR is not available there.

---
 Makefile                                             | 14 +-------------
 toolchain/helpers.mk                                 | 16 ++++++++++++++++
 toolchain/toolchain-buildroot/toolchain-buildroot.mk | 11 +++++++++++
 toolchain/toolchain-external/toolchain-external.mk   | 12 ++++++++++++
 4 files changed, 40 insertions(+), 13 deletions(-)
Makefile +0 -0
index 338c992..de202b0 100644

Makefile world: target-post-image · line 458

Quoted 10 lines
Makefile +0 -7
@@ -449,20 +449,10 @@ world: target-post-image
 $(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST):
      @mkdir -p $@
 
-# We make a symlink lib32->lib or lib64->lib as appropriate
-# MIPS64/n32 requires lib32 even though it's a 64-bit arch.
-ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y)
-LIB_SYMLINK = lib64
-else
-LIB_SYMLINK = lib32
-endif

Yeah, it's good to see this disappear from top-level Makefile!

Quoted 44 lines
Makefile +1 -6
-
 $(STAGING_DIR):
      @mkdir -p $(STAGING_DIR)/bin
      @mkdir -p $(STAGING_DIR)/lib
-     @ln -snf lib $(STAGING_DIR)/$(LIB_SYMLINK)
      @mkdir -p $(STAGING_DIR)/usr/lib
-     @ln -snf lib $(STAGING_DIR)/usr/$(LIB_SYMLINK)
      @mkdir -p $(STAGING_DIR)/usr/include
      @mkdir -p $(STAGING_DIR)/usr/bin
      @ln -snf $(STAGING_DIR) $(BASE_DIR)/staging
@@ -475,15 +465,13 @@ RSYNC_VCS_EXCLUSIONS = \
      --exclude .svn --exclude .git --exclude .hg --exclude .bzr \
      --exclude CVS
 
-$(BUILD_DIR)/.root:
+$(BUILD_DIR)/.root: | $(BUILD_DIR)
      mkdir -p $(TARGET_DIR)
      rsync -a --ignore-times $(RSYNC_VCS_EXCLUSIONS) \
              --chmod=Du+w --exclude .empty --exclude '*~' \
              $(TARGET_SKELETON)/ $(TARGET_DIR)/
      $(INSTALL) -m 0644 support/misc/target-dir-warning.txt $(TARGET_DIR_WARNING_FILE)
-     @ln -snf lib $(TARGET_DIR)/$(LIB_SYMLINK)
      @mkdir -p $(TARGET_DIR)/usr
-     @ln -snf lib $(TARGET_DIR)/usr/$(LIB_SYMLINK)
      touch $@
 
 $(TARGET_DIR): $(BUILD_DIR)/.root
toolchain/helpers.mk +10 -0
index 3121da4..b4879e0 100644
@@ -1,5 +1,21 @@
 # This Makefile fragment declares toolchain related helper functions.
 
+# Create the necessary symlink from (usr/)lib32|lib64|lib32-fp|... to lib
+# In general, for external toolchains, the correct link name is $ARCH_LIB_DIR.
+create_staging_lib_symlink = \
+     LIB_SYMLINK="$(strip $1)" ; \
+     if [ "$${LIB_SYMLINK}" != "lib" ]; then \
+             ln -snf lib $(STAGING_DIR)/$${LIB_SYMLINK} ; \
+             ln -snf lib $(STAGING_DIR)/usr/$${LIB_SYMLINK} ; \
+     fi
+
+create_target_lib_symlink = \
Quoted 8 lines

toolchain/helpers.mk line 17

Quoted 5 lines
toolchain/helpers.mk +5 -0
+     LIB_SYMLINK="$(strip $1)" ; \
+     if [ "$${LIB_SYMLINK}" != "lib" ]; then \
+         ln -snf lib $(TARGET_DIR)/$${LIB_SYMLINK} ; \
+         ln -snf lib $(TARGET_DIR)/usr/$${LIB_SYMLINK} ; \
+     fi

These two functions are identical except for the target dir, so create just one
function with the directory as a second parameter.

OK

Quoted 11 lines
Quoted 11 lines
toolchain/helpers.mk +1 -0
+
 # The copy_toolchain_lib_root function copies a toolchain library and
 # its symbolic links from the sysroot directory to the target
 # directory. Note that this function is used both by the external
toolchain/toolchain-buildroot/toolchain-buildroot.mk +0 -0
index b30cc33..3ccb030 100644
@@ -12,6 +12,17 @@ BR_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC))
 
Quoted 5 lines

toolchain/toolchain-buildroot/toolchain-buildroot.mk BR_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC)) · line 15

Quoted 3 lines
toolchain/toolchain-buildroot/toolchain-buildroot.mk +1 -0
 TOOLCHAIN_BUILDROOT_DEPENDENCIES = host-gcc-final
 
+ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y)

The comment about MIPS64/n32 should have been retained here.

True.

Quoted 8 lines

toolchain/toolchain-buildroot/toolchain-buildroot.mk BR_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC)) · line 19

Quoted 5 lines
toolchain/toolchain-buildroot/toolchain-buildroot.mk +4 -0
+LIB_SYMLINK = lib64
+else
+LIB_SYMLINK = lib32
+endif

Since now it is something specific to this .mk file, perhaps it should be
prefixed with TOOLCHAIN_BUILDROOT_

OK.

Quoted 8 lines

toolchain/toolchain-buildroot/toolchain-buildroot.mk BR_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC)) · line 21

Quoted 3 lines
toolchain/toolchain-buildroot/toolchain-buildroot.mk +2 -0
+
+host-gcc-final: create-lib-symlinks

This is not just awkward, it also breaks top-level parallel build. At least, if
host-gcc-final really does need this to be done. In top-level parallel build, the order between the dependencies is not set, so it's possible that the symlinks are only created after the build of gcc has already started.

Here I don't follow: the line above expresses that host-gcc-final needs create-lib-symlinks. So even with top-level parallel build, _first_ create-lib-symlinks is executed, and only then host-gcc-final.

Your statement seems to refer to cases like:

foo: bar baz

Where bar and baz can indeed be executed in parallel. Or I'm misunderstanding you or the situation.

Quoted 6 lines

Of course,
it's vanishingly unlikely that it will not yet have finished by the time the gcc install step starts... Actually, we wonder if it is really needed before gcc-final, or even before the libc? Because the way that make evaluates things in the non-parallel case, this step would have been done before anything else, even before host-binutils...

For external toolchains the links are created after extraction, when installing the toolchain in target and staging.
For internal toolchain I don't remember if it needs to be done before host-gcc-final, or if it's enough that it is done after host-gcc-final but I couldn't find a better way to achieve that because host-gcc-final is the target used for toolchain/toolchain-buildroot, so there wasn't a lot of hooking possibility without changing a lot of code. I would need to retest to be sure.

Quoted 10 lines

If it really should be done before host-gcc-final, the full solution would be
to add this as a post-install hook to the libc. You could do something like this:

$(call uppercase,$(BR_LIBC))_POST_INSTALL_STAGING_HOOKS += \
        $(call create_staging_lib_symlink,$(LIB_SYMLINK))

Isn't that beautiful :-) This also has the advantage of splitting the staging and target installs into their appropriate locations.

Quoted 6 lines

Alternatively, you could add a tiny package without source that does this and
add that as a dependency to host-gcc-final.

Do you still have the energy to make some changes and resubmit?

I'll see if I can look at some of this tomorrow but I can't make any promises. But if you happen to be in the heat of the moment tonight and can't resist reworking some of this then be my guest :-) I'll log into IRC tomorrow...

Thanks for the feedback,
Thomas

#6 lore raw

PATCH v3 2/2 toolchain: create symlink to 'lib' from ARCH_LIB_DIR iso fixed lib32/lib64

Quoted 261 lines · Thomas De Schampheleire
Quoted 3 lines · Arnout Vandecappelle

Hi Thomas,

Sorry that it is taking so long to make progress on this, but the

external

Quoted 1 line

toolchain stuff is, as you know, horribly demotivating... We've spent

quite a

Quoted 1 line

bit of time to understand what this patch does and how it could be

done better...

Quoted 9 lines · Thomas De Schampheleire
Quoted 7 lines

From: Thomas De Schampheleire <[email protected]>

Currently, following symbolic links are created in both target and staging directories:
- lib(32|64) --> lib
- usr/lib(32|64) --> lib

Quoted 4 lines
Quoted 4 lines

The decision for lib32 or lib64 is based on the target architecture configuration in buildroot (BR2_ARCH_IS_64).

In at least one case this is not correct: when building for a Cavium

Octeon

Quoted 2 lines
Quoted 2 lines

III processor using the toolchain from the Cavium Networks SDK, and specifying -march=octeon3 in BR2_TARGET_OPTIMIZATION, libraries are

expected

Quoted 3 lines
Quoted 3 lines

in directory 'lib32-fp' rather than 'lib32' (likewise for lib64-fp).

More generally, for external toolchains, the correct symbolic link

is

Quoted 11 lines
Quoted 10 lines

from (usr/)${ARCH_LIB_DIR} to lib. For internal toolchains, current toolchains always use either lib32 or lib64.

Fix the problem as follows:
- create symlink creation helpers in toolchain/helpers.mk
- for external toolchains, call these helpers based on ARCH_LIB_DIR
- for internal toolchains, call these helpers based on the existing fixed lib32/lib64 logic, moved from Makefile
- to fix build order problems, add the correct dependency on $(BUILD_DIR) from $(BUILD_DIR)/.root

Quoted 1 line

It's not clear to us which build order problems are solved by this,

and

Quoted 1 line

especially how these build order problems are introduced by this

patch... Could

Quoted 1 line

you explain that a little?

As you may imagine, my memory of these issues is not so crisp anymore, but IIRC without these build-order changes it would not build correctly. I guess I was using 'make clean toolchain' and it would fail, but I would need to retest explicitly to be sure.

Quoted 3 lines
Quoted 3 lines
Signed-off-by Thomas De Schampheleire

<[email protected]>

Quoted 11 lines
Quoted 11 lines

---
v3:
- update commit message wrapping
- change dependency on $(BUILD_DIR) to a order-only dependency v2:
- fix 'lib32-fp' leftover in toolchain-buildroot
- silence commands creating symlink with $(Q)
- fix case where ARCH_LIB_DIR is 'lib'

Note: the handling in the internal toolchain is a bit awkward

because it

Quoted 1 line
Quoted 1 line

explicitly adds a new target, but I don't see a much better way: it

must

Quoted 1 line
Quoted 1 line

be done _before_ the toolchain is being built. The entire logic

cannot

Quoted 3 lines
Quoted 3 lines

be done in Makefile as ARCH_LIB_DIR is not available there.

---
 Makefile                                             | 14
+-------------
>>  toolchain/helpers.mk                                 | 16

++++++++++++++++

Quoted 1 line
Quoted 1 line
 toolchain/toolchain-buildroot/toolchain-buildroot.mk | 11

+++++++++++

Quoted 1 line
Quoted 1 line
 toolchain/toolchain-external/toolchain-external.mk   | 12

++++++++++++

Quoted 8 lines

Makefile world: target-post-image · line 449

Quoted 8 lines
 4 files changed, 40 insertions(+), 13 deletions(-)
Makefile +0 -0
index 338c992..de202b0 100644
@@ -449,20 +449,10 @@ world: target-post-image
 $(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR)

$(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST):

Quoted 26 lines

Makefile world: target-post-image · line 458

Quoted 9 lines
Makefile +0 -7
      @mkdir -p $@
 
-# We make a symlink lib32->lib or lib64->lib as appropriate
-# MIPS64/n32 requires lib32 even though it's a 64-bit arch.
-ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y)
-LIB_SYMLINK = lib64
-else
-LIB_SYMLINK = lib32
-endif

Yeah, it's good to see this disappear from top-level Makefile!

Quoted 14 lines
Makefile +0 -3
-
 $(STAGING_DIR):
      @mkdir -p $(STAGING_DIR)/bin
      @mkdir -p $(STAGING_DIR)/lib
-     @ln -snf lib $(STAGING_DIR)/$(LIB_SYMLINK)
      @mkdir -p $(STAGING_DIR)/usr/lib
-     @ln -snf lib $(STAGING_DIR)/usr/$(LIB_SYMLINK)
      @mkdir -p $(STAGING_DIR)/usr/include
      @mkdir -p $(STAGING_DIR)/usr/bin
      @ln -snf $(STAGING_DIR) $(BASE_DIR)/staging
@@ -475,15 +465,13 @@ RSYNC_VCS_EXCLUSIONS = \
      --exclude .svn --exclude .git --exclude .hg --exclude .bzr \
      --exclude CVS
 
Quoted 7 lines

Makefile RSYNC_VCS_EXCLUSIONS = \

Quoted 7 lines
Makefile +1 -1
-$(BUILD_DIR)/.root:
+$(BUILD_DIR)/.root: | $(BUILD_DIR)
      mkdir -p $(TARGET_DIR)
      rsync -a --ignore-times $(RSYNC_VCS_EXCLUSIONS) \
              --chmod=Du+w --exclude .empty --exclude '*~' \
              $(TARGET_SKELETON)/ $(TARGET_DIR)/
      $(INSTALL) -m 0644 support/misc/target-dir-warning.txt

$(TARGET_DIR_WARNING_FILE)

Quoted 5 lines
Quoted 5 lines
Makefile +0 -2
-     @ln -snf lib $(TARGET_DIR)/$(LIB_SYMLINK)
      @mkdir -p $(TARGET_DIR)/usr
-     @ln -snf lib $(TARGET_DIR)/usr/$(LIB_SYMLINK)
      touch $@
 
Quoted 7 lines

toolchain/helpers.mk line 1

Quoted 7 lines
Makefile +0 -0
 $(TARGET_DIR): $(BUILD_DIR)/.root
toolchain/helpers.mk +0 -0
index 3121da4..b4879e0 100644
@@ -1,5 +1,21 @@
 # This Makefile fragment declares toolchain related helper

functions.

Quoted 2 lines

toolchain/helpers.mk line 2

Quoted 2 lines
toolchain/helpers.mk +1 -0
+# Create the necessary symlink from (usr/)lib32|lib64|lib32-fp|...

to lib

Quoted 1 line

toolchain/helpers.mk line 3

Quoted 1 line
toolchain/helpers.mk +1 -0
+# In general, for external toolchains, the correct link name is

$ARCH_LIB_DIR.

Quoted 7 lines
Quoted 7 lines
toolchain/helpers.mk +7 -0
+create_staging_lib_symlink = \
+     LIB_SYMLINK="$(strip $1)" ; \
+     if [ "$${LIB_SYMLINK}" != "lib" ]; then \
+             ln -snf lib $(STAGING_DIR)/$${LIB_SYMLINK} ; \
+             ln -snf lib $(STAGING_DIR)/usr/$${LIB_SYMLINK} ; \
+     fi
+
Quoted 8 lines

toolchain/helpers.mk line 17

Quoted 6 lines
toolchain/helpers.mk +6 -0
+create_target_lib_symlink = \
+     LIB_SYMLINK="$(strip $1)" ; \
+     if [ "$${LIB_SYMLINK}" != "lib" ]; then \
+         ln -snf lib $(TARGET_DIR)/$${LIB_SYMLINK} ; \
+         ln -snf lib $(TARGET_DIR)/usr/$${LIB_SYMLINK} ; \
+     fi

These two functions are identical except for the target dir, so

create just one

Quoted 1 line

function with the directory as a second parameter.

OK

Quoted 3 lines

toolchain/helpers.mk line 18

Quoted 3 lines
toolchain/helpers.mk +1 -0
+
 # The copy_toolchain_lib_root function copies a toolchain library

and

Quoted 3 lines

Review

Quoted 3 lines
toolchain/helpers.mk +0 -0
 # its symbolic links from the sysroot directory to the target
 # directory. Note that this function is used both by the external
(quoted diff) +0 -0

b/toolchain/toolchain-buildroot/toolchain-buildroot.mk

Quoted 4 lines

toolchain/toolchain-buildroot/toolchain-buildroot.mk BR_LIBC = $(call

Quoted 4 lines

index b30cc33..3ccb030 100644

toolchain/toolchain-buildroot/toolchain-buildroot.mk +0 -0
@@ -12,6 +12,17 @@ BR_LIBC = $(call

qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC))

Quoted 6 lines

toolchain/toolchain-buildroot/toolchain-buildroot.mk BR_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC)) · line 15

Quoted 4 lines
toolchain/toolchain-buildroot/toolchain-buildroot.mk +1 -0
 TOOLCHAIN_BUILDROOT_DEPENDENCIES = host-gcc-final
 
+ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y)

The comment about MIPS64/n32 should have been retained here.

True.

Quoted 7 lines

toolchain/toolchain-buildroot/toolchain-buildroot.mk BR_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC)) · line 19

Quoted 5 lines
toolchain/toolchain-buildroot/toolchain-buildroot.mk +4 -0
+LIB_SYMLINK = lib64
+else
+LIB_SYMLINK = lib32
+endif

Since now it is something specific to this .mk file, perhaps it

should be

Quoted 1 line

prefixed with TOOLCHAIN_BUILDROOT_

OK.

Quoted 5 lines

toolchain/toolchain-buildroot/toolchain-buildroot.mk BR_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC)) · line 21

Quoted 3 lines
toolchain/toolchain-buildroot/toolchain-buildroot.mk +2 -0
+
+host-gcc-final: create-lib-symlinks

This is not just awkward, it also breaks top-level parallel build.

At least, if

Quoted 1 line

host-gcc-final really does need this to be done. In top-level

parallel build,

Quoted 1 line

the order between the dependencies is not set, so it's possible that

the

Quoted 1 line

symlinks are only created after the build of gcc has already started.

Here I don't follow: the line above expresses that host-gcc-final needs create-lib-symlinks. So even with top-level parallel build, _first_ create-lib-symlinks is executed, and only then host-gcc-final.

Your statement seems to refer to cases like:

foo: bar baz

Where bar and baz can indeed be executed in parallel. Or I'm misunderstanding you or the situation.

Quoted 2 lines

Of course,
it's vanishingly unlikely that it will not yet have finished by the

time the gcc

Quoted 1 line

install step starts... Actually, we wonder if it is really needed

before

Quoted 1 line

gcc-final, or even before the libc? Because the way that make

evaluates things

Quoted 1 line

in the non-parallel case, this step would have been done before

anything else,

Quoted 1 line

even before host-binutils...

For external toolchains the links are created after extraction, when installing the toolchain in target and staging.
For internal toolchain I don't remember if it needs to be done before host-gcc-final, or if it's enough that it is done after host-gcc-final but I couldn't find a better way to achieve that because host-gcc-final is the target used for toolchain/toolchain-buildroot, so there wasn't a lot of hooking possibility without changing a lot of code. I would need to retest to be sure.

Quoted 2 lines

If it really should be done before host-gcc-final, the full solution

would be

Quoted 1 line

to add this as a post-install hook to the libc. You could do

something like this:

Quoted 5 lines
$(call uppercase,$(BR_LIBC))_POST_INSTALL_STAGING_HOOKS += \
        $(call create_staging_lib_symlink,$(LIB_SYMLINK))

Isn't that beautiful :-) This also has the advantage of splitting the

staging

Quoted 3 lines

and target installs into their appropriate locations.

Alternatively, you could add a tiny package without source that does

this and

Quoted 4 lines

add that as a dependency to host-gcc-final.

Quoted 5 lines
Quoted 1 line

Do you still have the energy to make some changes and resubmit?

I'll see if I can look at some of this tomorrow but I can't make any promises. But if you happen to be in the heat of the moment tonight and can't resist reworking some of this then be my guest :-)

'if you happen to be in the zone tonight' would probably qualify as better use of English expressions, but I'm sure you got the idea :-)

#8 lore raw

PATCH v3 2/2 toolchain: create symlink to 'lib' from ARCH_LIB_DIR iso fixed lib32/lib64

In reply to Arnout Vandecappelle
Quoted 34 lines · Arnout Vandecappelle

Hi Thomas,

Sorry that it is taking so long to make progress on this, but the external
toolchain stuff is, as you know, horribly demotivating... We've spent quite a bit of time to understand what this patch does and how it could be done better...

Quoted 19 lines · Thomas De Schampheleire

From: Thomas De Schampheleire <[email protected]>

Currently, following symbolic links are created in both target and staging directories:
- lib(32|64) --> lib
- usr/lib(32|64) --> lib

The decision for lib32 or lib64 is based on the target architecture configuration in buildroot (BR2_ARCH_IS_64).

In at least one case this is not correct: when building for a Cavium Octeon III processor using the toolchain from the Cavium Networks SDK, and specifying -march=octeon3 in BR2_TARGET_OPTIMIZATION, libraries are expected in directory 'lib32-fp' rather than 'lib32' (likewise for lib64-fp).

More generally, for external toolchains, the correct symbolic link is from (usr/)${ARCH_LIB_DIR} to lib. For internal toolchains, current toolchains always use either lib32 or lib64.

Quoted 7 lines

Fix the problem as follows:
- create symlink creation helpers in toolchain/helpers.mk
- for external toolchains, call these helpers based on ARCH_LIB_DIR
- for internal toolchains, call these helpers based on the existing fixed lib32/lib64 logic, moved from Makefile
- to fix build order problems, add the correct dependency on $(BUILD_DIR) from $(BUILD_DIR)/.root

Quoted 3 lines

It's not clear to us which build order problems are solved by this, and
especially how these build order problems are introduced by this patch... Could you explain that a little?

If you use this patch but revert the order-only dependency of .root on $(BUILD_DIR), then attempt building an internal toolchain using 'make clean toolchain', output is:

rm -rf /repo/tdescham/reborn/buildroot-git/output/target /repo/tdescham/reborn/buildroot-git/output/images /repo/tdescham/reborn/buildroot-git/output/host \
/repo/tdescham/reborn/buildroot-git/output/build
/repo/tdescham/reborn/buildroot-git/output/staging \
/repo/tdescham/reborn/buildroot-git/output/legal-info
/repo/tdescham/reborn/buildroot-git/output/graphs
mkdir -p /repo/tdescham/reborn/buildroot-git/output/target rsync -a --ignore-times --exclude .svn --exclude .git --exclude .hg

--exclude .bzr --exclude CVS \
                --chmod=u=rwX,go=rX --exclude .empty --exclude '*~' \
                /repo/tdescham/reborn/buildroot-git/system/skeleton/

/repo/tdescham/reborn/buildroot-git/output/target/
/usr/bin/install -m 0644 support/misc/target-dir-warning.txt /repo/tdescham/reborn/buildroot-git/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM touch /repo/tdescham/reborn/buildroot-git/output/build/.root
touch: cannot touch `/repo/tdescham/reborn/buildroot-git/output/build/.root': No such file or directory
make: *** [/repo/tdescham/reborn/buildroot-git/output/build/.root] Error 1

For external toolchains there is no issue.
It is caused by the fact that the patch adds a dependency on $(TARGET_DIR) to host-gcc-final via create-lib-symlinks, and $(TARGET_DIR) has a dependency on $(BUILD_DIR)/.root (see Makefile).

If the host-gcc-final change is fixed in another way (I have to check) this may not be needed after all. However, technically I think the expression of the dependency is correct: $(BUILD_DIR)/.root indeed needs BUILD_DIR first. Alternatively you could add a dependency on 'dirs' somewhere.

/Thomas

#9 lore raw

PATCH v3 2/2 toolchain: create symlink to 'lib' from ARCH_LIB_DIR iso fixed lib32/lib64

Quoted 8 lines · Thomas De Schampheleire
Quoted 7 lines · Arnout Vandecappelle

Hi Thomas,

Sorry that it is taking so long to make progress on this, but the external
toolchain stuff is, as you know, horribly demotivating... We've spent quite a bit of time to understand what this patch does and how it could be done better...

On 02/18/15 12:36, Thomas De Schampheleire wrote:

[snip]

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

- to fix build order problems, add the correct dependency on $(BUILD_DIR) from $(BUILD_DIR)/.root

It's not clear to us which build order problems are solved by this, and
especially how these build order problems are introduced by this patch... Could you explain that a little?

[snip]

Quoted 9 lines

It is caused by the fact that the patch adds a dependency on $(TARGET_DIR) to host-gcc-final via create-lib-symlinks, and $(TARGET_DIR) has a dependency on $(BUILD_DIR)/.root (see Makefile).

If the host-gcc-final change is fixed in another way (I have to check) this may not be needed after all. However, technically I think the expression of the dependency is correct: $(BUILD_DIR)/.root indeed needs BUILD_DIR first. Alternatively you could add a dependency on 'dirs' somewhere.

Or add an mkdir -p $(BUILD_DIR) in the commands of $(BUILD_DIR)/.root itself.
I'm personally a bit opposed against having directories as dependencies, precisely because of the need for order-only dependencies that you also discovered.

And I think that could be a separate patch - it may be a problem for top-level
parallel build as well, and it can be merged quickly.

Regards,
Arnout

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
#7 lore raw

PATCH v3 2/2 toolchain: create symlink to 'lib' from ARCH_LIB_DIR iso fixed lib32/lib64

In reply to Thomas De Schampheleire

[Now I understand Thomas why you didn't understand what I was talking about on IRC - seems I forgot to send this mail...]

Quoted 47 lines · Thomas De Schampheleire
Quoted 34 lines · Arnout Vandecappelle

Hi Thomas,

Sorry that it is taking so long to make progress on this, but the external
toolchain stuff is, as you know, horribly demotivating... We've spent quite a bit of time to understand what this patch does and how it could be done better...

Quoted 19 lines · Thomas De Schampheleire

From: Thomas De Schampheleire <[email protected]>

Currently, following symbolic links are created in both target and staging directories:
- lib(32|64) --> lib
- usr/lib(32|64) --> lib

The decision for lib32 or lib64 is based on the target architecture configuration in buildroot (BR2_ARCH_IS_64).

In at least one case this is not correct: when building for a Cavium Octeon III processor using the toolchain from the Cavium Networks SDK, and specifying -march=octeon3 in BR2_TARGET_OPTIMIZATION, libraries are expected in directory 'lib32-fp' rather than 'lib32' (likewise for lib64-fp).

More generally, for external toolchains, the correct symbolic link is from (usr/)${ARCH_LIB_DIR} to lib. For internal toolchains, current toolchains always use either lib32 or lib64.

Quoted 7 lines

Fix the problem as follows:
- create symlink creation helpers in toolchain/helpers.mk
- for external toolchains, call these helpers based on ARCH_LIB_DIR
- for internal toolchains, call these helpers based on the existing fixed lib32/lib64 logic, moved from Makefile
- to fix build order problems, add the correct dependency on $(BUILD_DIR) from $(BUILD_DIR)/.root

Quoted 3 lines

It's not clear to us which build order problems are solved by this, and
especially how these build order problems are introduced by this patch... Could you explain that a little?

As you may imagine, my memory of these issues is not so crisp anymore, but IIRC without these build-order changes it would not build correctly. I guess I was using 'make clean toolchain' and it would fail, but I would need to retest explicitly to be sure.

Quoted 3 lines
Quoted 3 lines

[snip]

Quoted 9 lines
Quoted 6 lines
Quoted 1 line

+host-gcc-final: create-lib-symlinks

This is not just awkward, it also breaks top-level parallel build. At least, if
host-gcc-final really does need this to be done. In top-level parallel build, the order between the dependencies is not set, so it's possible that the symlinks are only created after the build of gcc has already started.

Here I don't follow: the line above expresses that host-gcc-final needs create-lib-symlinks.

Semantically, yes. But technically, it doesn't. host-gcc-final is a PHONY
target that depends on the actual build steps of gcc. So in parallel build, create-lib-symlinks will be executed in parallel with host-gcc-final-install.

Quoted 8 lines

So even with top-level parallel build,
_first_ create-lib-symlinks is executed, and only then host-gcc-final.

Your statement seems to refer to cases like:

foo: bar baz

Where bar and baz can indeed be executed in parallel.

That's exactly the case we are in.

In toolchain-buildroot.mk:

host-gcc-final: create-lib-symlinks

Generated from generic-package:

host-gcc-final: host-gcc-final-install

and no commands.

Quoted 10 lines

Or I'm misunderstanding you or the situation.

Quoted 6 lines

Of course,
it's vanishingly unlikely that it will not yet have finished by the time the gcc install step starts... Actually, we wonder if it is really needed before gcc-final, or even before the libc? Because the way that make evaluates things in the non-parallel case, this step would have been done before anything else, even before host-binutils...

Quoted 8 lines

For external toolchains the links are created after extraction, when installing the toolchain in target and staging.
For internal toolchain I don't remember if it needs to be done before host-gcc-final, or if it's enough that it is done after host-gcc-final but I couldn't find a better way to achieve that because host-gcc-final is the target used for toolchain/toolchain-buildroot, so there wasn't a lot of hooking possibility without changing a lot of code. I would need to retest to be sure.

I was actually surprised that it was even needed for the internal toolchain
itself, I thought we refactored the symlinking of the external toolchain just to make things consistent. The current symlink was introduced in 5628776c4, before that there was only some (partially broken) symlinking for the external toolchain.

Quoted 15 lines
Quoted 15 lines

If it really should be done before host-gcc-final, the full solution would be
to add this as a post-install hook to the libc. You could do something like this:

$(call uppercase,$(BR_LIBC))_POST_INSTALL_STAGING_HOOKS += \
        $(call create_staging_lib_symlink,$(LIB_SYMLINK))

Isn't that beautiful :-) This also has the advantage of splitting the staging and target installs into their appropriate locations.

Alternatively, you could add a tiny package without source that does this and
add that as a dependency to host-gcc-final.

Quoted 6 lines
Quoted 1 line

Do you still have the energy to make some changes and resubmit?

I'll see if I can look at some of this tomorrow but I can't make any promises. But if you happen to be in the heat of the moment tonight and can't resist reworking some of this then be my guest :-) I'll log into IRC tomorrow...

We actually have other cats to whip :-) and to be honest, nobody is really
excited about touching the external toolchain stuff. I'll shout in the room here and see if anyone jumps up.

Regards,
Arnout

Quoted 4 lines

Thanks for the feedback,
Thomas

Signature · 6 lines
Arnout Vandecappelle      arnout dot vandecappelle at essensium dot com
Senior Embedded Software Architect . . . . . . +32-478-010353 (mobile)
Essensium, Mind division . . . . . . . . . . . . . . http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium . . . . . BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF