Suppress Texinfo utilities invoked at build time for -cross & native recipes (revised)

Hello list,

As per discussion with Richard Purdie and others, I'm submitting a patch to suppress some unnecessary invocations of utilities from the Texinfo suite during Yocto builds. Specifically, it targets -native and -cross recipes that inherit texinfo.bbclass. This series is revised to fix issues that came up on the autobuilder when I first tried submitting it.

The Texinfo utilities are suppressed by supplying doppelgänger executables which understand the same command-line options as the genuine Texinfo utilites, and which can create blank output files where expected, but which do no actual processing, formatting, or markup. To see how this is done, look in the texinfo-dummy-native recipe (under meta/recipes-extended,) added in this patch series, as well as the modifications to texinfo.bbclass, also in this patch series.

Suppressing the Texinfo utilities for -native and -cross recipes does provide a noticable performance gain, as documented in bug 2421. [1] You could make the case that this patch series still does not completely address bug 2421, as there are other utilities not from the Texinfo suite (groff, pod2man, etc) which are not suppressed by these changes.

Target-architecture recipes continue to use the host system's Texinfo utilities as before, so that dependency is not eliminated by this patch series.

Regards,
-Max Eliaser

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=2421
The following changes since commit e7114046be6e71fb326477dc882adb814ef6a751:

busybox: fix meta-yocto's bbappend's FILESEXTRAPATH (2014-06-03 17:53:13 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib meliaser/texinfo-build-2
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=meliaser/texinfo-build-2

Max Eliaser (3):
texinfo: Use texinfo-replacement-native as an alias for
texinfo-native.
texinfo-dummy-native: Create recipe w/ scripts to stand in for Texinfo
utils.
texinfo.bbclass: native/cross uses dummy texinfo; target uses host's
Texinfo.

 meta/classes/texinfo.bbclass                       |  15 ++-
 meta/conf/bitbake.conf                             |   1 +
 .../texinfo-dummy-native/texinfo-dummy-native.bb   |  30 +++++
 .../texinfo-dummy-native/texinfo-dummy/COPYING     |  19 ++++
 .../texinfo-dummy-native/texinfo-dummy/template.py | 122 +++++++++++++++++++++
 meta/recipes-extended/texinfo/texinfo_5.2.bb       |   4 +-
 6 files changed, 189 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-extended/texinfo-dummy-native/texinfo-dummy-native.bb
 create mode 100644 meta/recipes-extended/texinfo-dummy-native/texinfo-dummy/COPYING
 create mode 100644 meta/recipes-extended/texinfo-dummy-native/texinfo-dummy/template.py

1.8.3.2

If texinfo-native is added to ASSUME_PROVIDED, this will be needed for two reasons. First, a world build will still build the native texinfo recipe, so it will still get test coverage and will not rot. Second, when the target architecture texinfo recipe is built, the native recipe must be built first so its makedoc binary can be extracted, ASSUME_PROVIDED or no.

---
 meta/recipes-extended/texinfo/texinfo_5.2.bb | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

index c1ccaa7..be90aba 100644
@@ -7,7 +7,9 @@ SECTION = "console/utils"
 LICENSE = "GPLv3+"
 LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
 
-DEPENDS = "zlib ncurses texinfo-native"
+PROVIDES_append_class-native = " texinfo-replacement-native"
+
+DEPENDS = "zlib ncurses texinfo-replacement-native"
 DEPENDS_class-native = "zlib-native ncurses-native"
 
 TARGET_PATCH = "file://use_host_makedoc.patch"

1.8.3.2

More work toward eliminating the dependency on the host system's Texinfo-- Python scripts that understand the same command-line options as the Texinfo utilities, and create blank output files if appropriate, but don't actually do any of the work done by those utilities.

This will be necessary to avoid circular dependencies when we start explicitly tracking dependencies on texinfo-native; i.e. texinfo-native -> autoconf-native -> texinfo-native. If we have all native recipes that inherit texinfo.bbclass depend on texinfo-dummy-native instead of texinfo-native, the cycle is broken.

It may also provide a performance gain by skipping the actual work of formatting and generating documentation files.

---
 .../texinfo-dummy-native/texinfo-dummy-native.bb   |  30 +++++
 .../texinfo-dummy-native/texinfo-dummy/COPYING     |  19 ++++
 .../texinfo-dummy-native/texinfo-dummy/template.py | 122 +++++++++++++++++++++
 3 files changed, 171 insertions(+)
 create mode 100644 meta/recipes-extended/texinfo-dummy-native/texinfo-dummy-native.bb
 create mode 100644 meta/recipes-extended/texinfo-dummy-native/texinfo-dummy/COPYING
 create mode 100644 meta/recipes-extended/texinfo-dummy-native/texinfo-dummy/template.py

new file mode 100644
index 0000000..b5420a3
@@ -0,0 +1,30 @@
+DESCRIPTION = "Fake version of the texinfo utility suite"
+SECTION = "console/utils"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://COPYING;md5=d6bb62e73ca8b901d3f2e9d71542f4bb"
+DEPENDS = ""
+PV = "1.0"
+
+SRC_URI = "file://template.py file://COPYING"
+
+S = "${WORKDIR}"
+
+NATIVE_PACKAGE_PATH_SUFFIX = "/${PN}"
+
+inherit native
+
+do_install_name() {
+    FILENAME="${D}${bindir}/$1"
+    # Using ln causes problems with rm_work
+    cp -T "${S}/template.py" "$FILENAME"
+    chmod +x $FILENAME
+}
+
+do_install() {
+    mkdir -p "${D}${bindir}"
+    for i in makeinfo pod2texi texi2dvi pdftexi2dvi texindex texi2pdf \
+             txixml2texi texi2any install-info ginstall-info \
+             update-info-dir; do
+        do_install_name $i
+    done
+}

new file mode 100644
index 0000000..531cbe7
@@ -0,0 +1,19 @@
+Copyright (c) 2014 Intel Corp.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

1.8.3.2

To unpack that to more than a single line: -native and -cross recipes are made to use the dummy Texinfo utilities provided by texinfo-dummy-native if they invoke those utilities at build time. The target-architecture (cross-compiled) recipes still use the genuine Texinfo utilites. Right now, they still use the host system's Texinfo utilities, but could be made to use the texinfo-native recipe we already ship with some config file changes.

---
 meta/classes/texinfo.bbclass | 15 ++++++++++++++-
 meta/conf/bitbake.conf       |  1 +
 2 files changed, 15 insertions(+), 1 deletion(-)

index 60cba0f..92efbcc 100644
@@ -1,2 +1,15 @@
 # This class is inherited by recipes whose upstream packages invoke the
-# texinfo utilities at build-time. This class currently does nothing.
+# texinfo utilities at build-time. Native and cross recipes are made to use the
+# dummy scripts provided by texinfo-dummy-native, for improved performance. 
+# Target architecture recipes use the genuine Texinfo utilities. By default, 
+# they use the Texinfo utilities on the host system. If you want to use the
+# Texinfo recipe shipped with yoco, you can remove texinfo-native from 
+# ASSUME_PROVIDED and makeinfo from SANITY_REQUIRED_UTILITIES.
+
+TEXDEP = "texinfo-native"
+TEXDEP_class-native = "texinfo-dummy-native"
+TEXDEP_class-cross = "texinfo-dummy-native"
+DEPENDS_append = " ${TEXDEP}"
+PATH_prepend_class-native = "${STAGING_BINDIR_NATIVE}/texinfo-dummy-native:"
+PATH_prepend_class-cross = "${STAGING_BINDIR_NATIVE}/texinfo-dummy-native:"
+

index 1d70d3c..cd6f940 100644
@@ -181,6 +181,7 @@ ASSUME_PROVIDED = "\
     python-native-runtime \
     tar-native \
     virtual/libintl-native \
+    texinfo-native\
     "
 # gzip-native should be listed above?
 

1.8.3.2

There are a few cases, where I've wanted texinfo processed files to be shipped as part of an SDK. Specifically nativesdk and the SDK cross compiler packages.
Will this affect either of those cases?

(Looking at the code, I don't think it will, but I wanted to check.)

--Mark

Shouldn't be a problem. It's only if the class is "native" or "cross" that the impostor texinfo utils get used. I believe "nativesdk" and "crosssdk" are separate from "native" and "cross".

-Max
________________________________________
From: [email protected] [[email protected]] on behalf of Mark Hatle [[email protected]]
Sent: Tuesday, June 10, 2014 2:06 PM
To: [email protected]
Subject: Re: [OE-core] [PATCH 0/3] Suppress Texinfo utilities invoked at build time for -cross & native recipes (revised)

There are a few cases, where I've wanted texinfo processed files to be shipped as part of an SDK. Specifically nativesdk and the SDK cross compiler packages.
Will this affect either of those cases?

(Looking at the code, I don't think it will, but I wanted to check.)

--Mark