Speed-up gettext and host-gettext build
cover Speed-up gettext and host-gettext build
Hello,
This series aims at speeding up the build of the gettext and host-gettext packages, which are particularly long currently.
It has been tested with the minidlna build against a uClibc toolchain, minidlna being a package that needs both gettext and host-gettext.
It would be nice if a few other people could test these changes in their own configuration, to confirm that they don't break anything.
Best regards,
Thomas
Thomas Petazzoni (3):
minidlna: do not select BR2_PACKAGE_GETTEXT_TOOLS
gettext: remove support for gettext-tools on target
gettext: optimize build time
Config.in.legacy | 12 +++++++ package/gettext/Config.in | 14 -------- .../gettext-03-option-to-disable-tools.patch | 42 ---------------------- .../gettext/gettext-04-remove-examples-tests.patch | 20 ----------- package/gettext/gettext.mk | 27 +++++++------- package/minidlna/Config.in | 1 - 6 files changed, 27 insertions(+), 89 deletions(-) delete mode 100644 package/gettext/gettext-03-option-to-disable-tools.patch delete mode 100644 package/gettext/gettext-04-remove-examples-tests.patch
Signature · 1 line
1.9.3
PATCH 1/3 minidlna: do not select BR2_PACKAGE_GETTEXT_TOOLS
Commit 9ff640ae30ca588228030faf2c6fd13af292cb66 ("minidlna: needs gettext and host-gettext tools if locale is enabled") modified the minidlna package to fix gettext related issues. As part of this patch, a select of BR2_PACKAGE_GETTEXT_TOOLS was added, which according to the commit title is used to have host-gettext tools installed.
However, this is not what this option is about: this option is about having gettext tools installed on the target.
Since this is not what minidlna needs, and we anyway plan to remove this BR2_PACKAGE_GETTEXT_TOOLS option, this commit removes this incorrect select statement.
--- package/minidlna/Config.in | 1 - 1 file changed, 1 deletion(-)
package/minidlna/Config.in +0 -1
@@ -14,7 +14,6 @@ config BR2_PACKAGE_MINIDLNA select BR2_PACKAGE_LIBJPEG select BR2_PACKAGE_SQLITE select BR2_PACKAGE_GETTEXT if BR2_NEEDS_GETTEXT_IF_LOCALE - select BR2_PACKAGE_GETTEXT_TOOLS if BR2_NEEDS_GETTEXT_IF_LOCALE help MiniDLNA (aka ReadyDLNA) is server software with the aim of being fully compliant with DLNA/UPnP-AV clients.
Signature · 1 line
1.9.3
PATCH 2/3 gettext: remove support for gettext-tools on target
In reply to Thomas PetazzoniThis commit removes the BR2_PACKAGE_GETTEXT_TOOLS option, which could be used to install gettext tools on the target. This is not needed, because Buildroot is not designed to provide a full development environment on the target, and gettext translation files should be processed on the build machine, using the host gettext tools.
Remove this option will allow to optimize the build time of gettext on the target, by only building the gettext runtime libraries.
--- Config.in.legacy | 12 ++++++++++++ package/gettext/Config.in | 14 -------------- package/gettext/gettext.mk | 4 ---- 3 files changed, 12 insertions(+), 18 deletions(-)
Config.in.legacy +12 -0
@@ -99,6 +99,18 @@ comment "----------------------------------------------------" endif ############################################################################### +comment "Legacy options removed in 2014.08" + +config BR2_PACKAGE_GETTEXT_TOOLS + bool "gettext-tools on target support removed" + select BR2_LEGACY + help + The option to install the gettext utilities on the target + has been removed. This is not necessary as Buildroot is not + designed to provide a full development environment on the + target. gettext tools should be used on the build machine + instead. + comment "Legacy options removed in 2014.05" config BR2_PACKAGE_EVTEST_CAPTURE
package/gettext/Config.in +0 -14
@@ -13,20 +13,6 @@ config BR2_PACKAGE_GETTEXT http://www.gnu.org/software/gettext/ -if BR2_PACKAGE_GETTEXT - -config BR2_PACKAGE_GETTEXT_TOOLS - bool "Install gettext tools" - depends on BR2_USE_MMU # fork() - depends on BR2_LARGEFILE - select BR2_PACKAGE_LIBICONV if !BR2_ENABLE_LOCALE - help - This option allows to install the complete gettext suite in - the target filesystem. This is typically not useful for - correct operation of programs. - -endif - comment "gettext needs a toolchain w/ wchar" depends on BR2_NEEDS_GETTEXT depends on !BR2_USE_WCHAR
package/gettext/gettext.mk +0 -4
@@ -46,16 +46,12 @@ ifeq ($(BR2_ENABLE_LOCALE),) GETTEXT_CONF_OPT += --enable-nls endif -# When the gettext tools are not enabled in the configuration, we only -# install libintl to the target. -ifeq ($(BR2_PACKAGE_GETTEXT_TOOLS),) # When static libs are preferred the .so files aren't created ifeq ($(BR2_PREFER_STATIC_LIB),) define GETTEXT_INSTALL_TARGET_CMDS cp -dpf $(STAGING_DIR)/usr/lib/libintl*.so* $(TARGET_DIR)/usr/lib/ endef endif -endif # GETTEXT_TOOLS = n # Disable interactive confirmation in host gettextize for package fixups define HOST_GETTEXT_GETTEXTIZE_CONFIRMATION
Signature · 1 line
1.9.3
PATCH 2/3 gettext: remove support for gettext-tools on target
Quoted 21 lines · Thomas Petazzoni
This commit removes the
BR2_PACKAGE_GETTEXT_TOOLSoption, which could be used to install gettext tools on the target. This is not needed, because Buildroot is not designed to provide a full development environment on the target, and gettext translation files should be processed on the build machine, using the host gettext tools.Remove this option will allow to optimize the build time of gettext on the target, by only building the gettext runtime libraries.
Signed-off-by Thomas Petazzoni <[email protected]>--- Config.in.legacy | 12 ++++++++++++ package/gettext/Config.in | 14 -------------- package/gettext/gettext.mk | 4 ---- 3 files changed, 12 insertions(+), 18 deletions(-)Config.in.legacy +0
-0
Config.in.legacy comment "----------------------------------------------------" · line 105
Quoted 7 lines
Config.in.legacy +4
-0@@ -99,6 +99,18 @@ comment "----------------------------------------------------" endif ############################################################################### +comment "Legacy options removed in 2014.08" + +config BR2_PACKAGE_GETTEXT_TOOLS + bool "gettext-tools on target support removed"
I would rewrite this as any of the following:
support for gettext-tools on target has been removed gettext-tools target support has been removed gettext-tools target option has been removed
Best regards,
Thomas
PATCH 3/3 gettext: optimize build time
In reply to Thomas PetazzoniThis commit significantly reduces the build time of host-gettext and gettext, by using the capacity of gettext to handle build things in a certain subdirectory:
- For the host variant of gettext, we only need the gettext-tools, available in the directory of the same name in the gettext sources.
- For the target variant of gettext, we only need the gettext library libintl, available in the gettext-runtime directory in the gettext sources.
So by using appropriate values of GETTEXT_SUBDIR and HOST_GETTEXT_SUBDIR, we only build what's necessary. Moreover, by manually patching gettext-tools/Makefile.in and gettext-runtime/Makefile.in, we make sure to not build and install things like examples, documentation and so on.
In addition to this, these changes avoid the need to autoreconfigure the gettext package, which was particularly long.
Thanks to these changes, the build time of gettext goes from 1 minutes and 37 seconds to just 24 seconds, and the build of host-gettext goes from 2 minutes and 18 seconds to 1 minute and 13 seconds.
--- .../gettext-03-option-to-disable-tools.patch | 42 ---------------------- .../gettext/gettext-04-remove-examples-tests.patch | 20 ----------- package/gettext/gettext.mk | 23 +++++++----- 3 files changed, 15 insertions(+), 70 deletions(-) delete mode 100644 package/gettext/gettext-03-option-to-disable-tools.patch delete mode 100644 package/gettext/gettext-04-remove-examples-tests.patch
package/gettext/gettext-03-option-to-disable-tools.patch deleted +0 -42
@@ -1,42 +0,0 @@ -Add a --disable-tools option - -This patch adds a --disable-tools option that allows to disable the -compilation of the gettext tools, which may not be necessary on a -small embedded Linux system. - -Signed-off-by: Thomas Petazzoni <[email protected]> - -Index: b/configure.ac -=================================================================== ---- a/configure.ac -+++ b/configure.ac -@@ -37,6 +37,10 @@ - - dnl Checks for library functions. - -+AC_ARG_ENABLE([tools], [--disable-tools do not build tools], -+ [enable_tools=$enableval], [enable_tools=yes]) -+AM_CONDITIONAL([ENABLE_TOOLS], [test "$enable_tools" = "yes"]) -+ - AC_CONFIG_SUBDIRS([gettext-runtime gettext-tools]) - - dnl Ensure that "configure --help" lists all the command line options that -Index: b/Makefile.am -=================================================================== ---- a/Makefile.am -+++ b/Makefile.am -@@ -20,7 +20,13 @@ - ACLOCAL = build-aux/fixaclocal @ACLOCAL@ - ACLOCAL_AMFLAGS = -I m4 - --SUBDIRS = gnulib-local gettext-runtime gettext-tools -+if ENABLE_TOOLS -+TOOLS_SUBDIR = gettext-tools -+else -+TOOLS_SUBDIR = -+endif -+ -+SUBDIRS = gnulib-local gettext-runtime $(TOOLS_SUBDIR) - - EXTRA_DIST = \ - version.sh DEPENDENCIES PACKAGING HACKING ChangeLog.0 autogen.sh \
package/gettext/gettext-04-remove-examples-tests.patch deleted +0 -20
@@ -1,20 +0,0 @@ -Disable tests and examples - -This patch disables the gettext-tools tests and examples, which are -generally not necessary. - -Signed-off-by: Thomas Petazzoni <[email protected]> - -Index: b/gettext-tools/Makefile.am -=================================================================== ---- a/gettext-tools/Makefile.am -+++ b/gettext-tools/Makefile.am -@@ -20,7 +20,7 @@ - ACLOCAL = ../build-aux/fixaclocal @ACLOCAL@ - ACLOCAL_AMFLAGS = -I m4 -I ../gettext-runtime/m4 -I ../m4 -I gnulib-m4 -I libgrep/gnulib-m4 -I libgettextpo/gnulib-m4 - --SUBDIRS = doc intl gnulib-lib libgrep src libgettextpo po projects styles misc man m4 tests gnulib-tests examples -+SUBDIRS = doc intl gnulib-lib libgrep src libgettextpo po projects styles misc man m4 - - EXTRA_DIST = misc/DISCLAIM - MOSTLYCLEANFILES = core *.stackdump
package/gettext/gettext.mk +15 -8
@@ -9,7 +9,6 @@ GETTEXT_SITE = $(BR2_GNU_MIRROR)/gettext GETTEXT_INSTALL_STAGING = YES GETTEXT_LICENSE = GPLv2+ GETTEXT_LICENSE_FILES = COPYING -GETTEXT_AUTORECONF = YES GETTEXT_DEPENDENCIES = $(if $(BR2_PACKAGE_LIBICONV),libiconv) HOST_GETTEXT_DEPENDENCIES = # we don't want the libiconv dependency
@@ -37,6 +36,21 @@ HOST_GETTEXT_CONF_OPT = \ --disable-relocatable \ --without-emacs +# For the target version, we only need the runtime, and for the host +# version, we only need the tools. +GETTEXT_SUBDIR = gettext-runtime +HOST_GETTEXT_SUBDIR = gettext-tools + +# Disable the build of documentation and examples of gettext-tools, +# and the build of documentation and tests of gettext-runtime. +define HOST_GETTEXT_DISABLE_UNNEEDED + $(SED) '/^SUBDIRS/s/ doc //;/^SUBDIRS/s/examples$$//' $(@D)/gettext-tools/Makefile.in + $(SED) '/^SUBDIRS/s/ doc //;/^SUBDIRS/s/tests$$//' $(@D)/gettext-runtime/Makefile.in +endef + +GETTEXT_POST_PATCH_HOOKS += HOST_GETTEXT_DISABLE_UNNEEDED +HOST_GETTEXT_POST_PATCH_HOOKS += HOST_GETTEXT_DISABLE_UNNEEDED + # Force build with NLS support, otherwise libintl is not built # This is needed because some packages (eg. libglib2) requires # locales, but do not properly depend on BR2_ENABLE_LOCALE, and
@@ -46,13 +60,6 @@ ifeq ($(BR2_ENABLE_LOCALE),) GETTEXT_CONF_OPT += --enable-nls endif -# When static libs are preferred the .so files aren't created -ifeq ($(BR2_PREFER_STATIC_LIB),) -define GETTEXT_INSTALL_TARGET_CMDS - cp -dpf $(STAGING_DIR)/usr/lib/libintl*.so* $(TARGET_DIR)/usr/lib/ -endef -endif - # Disable interactive confirmation in host gettextize for package fixups define HOST_GETTEXT_GETTEXTIZE_CONFIRMATION $(SED) '/read dummy/d' $(HOST_DIR)/usr/bin/gettextize
Signature · 1 line
1.9.3
PATCH 3/3 gettext: optimize build time
Hi Thomas,
Quoted 20 lines · Thomas Petazzoni
This commit significantly reduces the build time of host-gettext and gettext, by using the capacity of gettext to handle build things in a certain subdirectory:
- For the host variant of gettext, we only need the gettext-tools, available in the directory of the same name in the gettext sources.
- For the target variant of gettext, we only need the gettext library libintl, available in the gettext-runtime directory in the gettext sources.
So by using appropriate values of
GETTEXT_SUBDIRandHOST_GETTEXT_SUBDIR, we only build what's necessary. Moreover, by manually patching gettext-tools/Makefile.in and gettext-runtime/Makefile.in, we make sure to not build and install things like examples, documentation and so on.In addition to this, these changes avoid the need to autoreconfigure the gettext package, which was particularly long.
Quoted 7 lines
Thanks to these changes, the build time of gettext goes from 1 minutes and 37 seconds to just 24 seconds, and the build of host-gettext goes from 2 minutes and 18 seconds to 1 minute and 13 seconds.
Signed-off-by Thomas Petazzoni <[email protected]>--- .../gettext-03-option-to-disable-tools.patch | 42 ----------------------
Why not remove --disable-tools from GETTEXT_CONF_OPT then?
baruch
Signature · 3 lines
http://baruch.siach.name/blog/ ~. .~ Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
- baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -PATCH 3/3 gettext: optimize build time
Dear Baruch Siach,
Quoted 9 lines · Baruch Siach
Quoted 7 lines
Thanks to these changes, the build time of gettext goes from 1 minutes and 37 seconds to just 24 seconds, and the build of host-gettext goes from 2 minutes and 18 seconds to 1 minute and 13 seconds.
Signed-off-by Thomas Petazzoni <[email protected]>--- .../gettext-03-option-to-disable-tools.patch | 42 ----------------------Why not remove --disable-tools from
GETTEXT_CONF_OPTthen?
Indeed, thanks for noticing. I'll fix that up in v2.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
cover Speed-up gettext and host-gettext build
In reply to Thomas PetazzoniQuoted 2 lines · Thomas Petazzoni
It would be nice if a few other people could test these changes in their own configuration, to confirm that they don't break anything.
Cuts my build of host-gettext from ~115 seconds to ~55 seconds.
Thanks,
Andy