autoconf: Raise line length limit for sed/awk
PATCH 1/1 autoconf: Raise line length limit for sed/awk
Latest autoconf is splitting long cmd lines in config.status it generates using sed/awk to have compatibility with platform like HPUX,Solaris,AIX which is understandable given autotools try to achieve wider portability, however, this does mean that in OE builds we might get these newlines added in places dividing commands including buildpaths in them and the methods deployed to scrub them from output artifacts will fail since the string is now on two lines resulting in build QA to pass but the strings do sneak in affecting reproducibility one such example is e2tools recipe in meta-filesystems and there perhaps are more
--- ...-command-limit-to-prevent-line-wrapp.patch | 41 +++++++++++++++++++ .../autoconf/autoconf_2.72.bb | 3 +- 2 files changed, 43 insertions(+), 1 deletion(-) create mode 100644 meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch new file +41 -0
@@ -0,0 +1,41 @@ +From 139f757a6244d3b1e07f7fddc6d07d9c17f45313 Mon Sep 17 00:00:00 2001 +From: Khem Raj <[email protected]> +Date: Tue, 25 Nov 2025 21:03:38 -0800 +Subject: [PATCH] Increase awk/sed command limit to prevent line wrapping + +Autoconf 2.70+ wraps long variable assignments in config.status +using backslash-newline continuations. This breaks when LDFLAGS +or other variables contain special characters. + +Increase _AC_SED_CMD_LIMIT from 99 and _AC_AWK_LITERAL_LIMIT +from 148 to 4000 to prevent wrapping. + +Upstream-Status: Inappropriate [OE-Specific] + +Signed-off-by: Khem Raj <[email protected]> +--- + lib/autoconf/status.m4 | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/lib/autoconf/status.m4 b/lib/autoconf/status.m4 +index 2bfaf32..be444b2 100644 +--- a/lib/autoconf/status.m4 ++++ b/lib/autoconf/status.m4 +@@ -297,7 +297,7 @@ AC_DEFUN([AC_CONFIG_FILES], [_AC_CONFIG_FOOS([FILES], $@)]) + # Some seds have small command number limits, like on Digital OSF/1 and HP-UX. + m4_define([_AC_SED_CMD_LIMIT], + dnl One cannot portably go further than 99 commands because of HP-UX. +-[99]) ++[4000]) + + + # _AC_AWK_LITERAL_LIMIT +@@ -307,7 +307,7 @@ dnl One cannot portably go further than 99 commands because of HP-UX. + # + # Some awk's have small limits, such as Solaris and AIX awk. + m4_define([_AC_AWK_LITERAL_LIMIT], +-[148]) ++[4000]) + + + # _AC_OUTPUT_FILES_PREPARE
meta/recipes-devtools/autoconf/autoconf_2.72.bb +2 -1
@@ -1,5 +1,5 @@ SUMMARY = "A GNU tool that produce shell scripts to automatically configure software" -DESCRIPTION = "Autoconf is an extensible package of M4 macros that produce shell scripts to automatically \ +DESCRIPTION = "Autoconf is an extensible package of M4 macros that produce shell scripts to automatically \ configure software source code packages. Autoconf creates a configuration script for a package from a template \ file that lists the operating system features that the package can use, in the form of M4 macro calls." LICENSE = "GPL-3.0-or-later"
@@ -19,6 +19,7 @@ SRC_URI = " \ file://preferbash.patch \ file://autotest-automake-result-format.patch \ file://man-host-perl.patch \ + file://0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch \ " SRC_URI:append:class-native = " file://no-man.patch"
Quoted 12 lines · Khem Raj via lists.openembedded.org
Latest autoconf is splitting long cmd lines in config.status it generates using sed/awk to have compatibility with platform like HPUX,Solaris,AIX which is understandable given autotools try to achieve wider portability, however, this does mean that in OE builds we might get these newlines added in places dividing commands including buildpaths in them and the methods deployed to scrub them from output artifacts will fail since the string is now on two lines resulting in build QA to pass but the strings do sneak in affecting reproducibility one such example is e2tools recipe in meta-filesystems and there perhaps are more
Quoted 2 lines
Signed-off-by Khem Raj <[email protected]>
For the impact on e2tools:
=> It does indeed make it reproducible by letting the sed replacing non reproducible paths work.
Quoted 98 lines
--- ...-command-limit-to-prevent-line-wrapp.patch | 41 +++++++++++++++++++ .../autoconf/autoconf_2.72.bb | 3 +- 2 files changed, 43 insertions(+), 1 deletion(-) create mode 100644
meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patchdiff --git
a/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patchb/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patchnew file mode 100644
index 00000000000..41468ba316c
---/dev/null
+++b/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patchmeta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch new file +24
-0@@ -0,0 +1,41 @@ +From 139f757a6244d3b1e07f7fddc6d07d9c17f45313 Mon Sep 17 00:00:00 2001 +From: Khem Raj <[email protected]> +Date: Tue, 25 Nov 2025 21:03:38 -0800 +Subject: [PATCH] Increase awk/sed command limit to prevent line wrapping + +Autoconf 2.70+ wraps long variable assignments in config.status +using backslash-newline continuations. This breaks when LDFLAGS +or other variables contain special characters. + +Increase _AC_SED_CMD_LIMIT from 99 and _AC_AWK_LITERAL_LIMIT +from 148 to 4000 to prevent wrapping. + +Upstream-Status: Inappropriate [OE-Specific] + +Signed-off-by: Khem Raj <[email protected]> +--- + lib/autoconf/status.m4 | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/lib/autoconf/status.m4 b/lib/autoconf/status.m4 +index 2bfaf32..be444b2 100644 +--- a/lib/autoconf/status.m4 ++++ b/lib/autoconf/status.m4 +@@ -297,7 +297,7 @@ AC_DEFUN([AC_CONFIG_FILES], [_AC_CONFIG_FOOS([FILES],$@)])
+ # Some seds have small command number limits, like on Digital OSF/1 and HP-UX.
+ m4_define([_AC_SED_CMD_LIMIT],
+ dnl One cannot portably go further than 99 commands because of HP-UX. +-[99])
++[4000])meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch new file +4
-0+ + + # _AC_AWK_LITERAL_LIMIT +@@ -307,7 +307,7 @@ dnl One cannot portably go further than 99 commandsbecause of HP-UX.
+ #
+ # Some awk's have small limits, such as Solaris and AIX awk.
+ m4_define([_AC_AWK_LITERAL_LIMIT],
+-[148])
++[4000])meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch new file +3
-0+ + + # _AC_OUTPUT_FILES_PREPAREmeta/recipes-devtools/autoconf/autoconf_2.72.bb +0
-0@@ -1,5 +1,5 @@ SUMMARY = "A GNU tool that produce shell scripts to automaticallyconfigure software"
-DESCRIPTION = "Autoconf is an extensible package of M4 macros that produce shell scripts to automatically \
+DESCRIPTION = "Autoconf is an extensible package of M4 macros that produce shell scripts to automatically \
configure software source code packages. Autoconf creates a configuration
script for a package from a template \
file that lists the operating system features that the package can use,
in the form of M4 macro calls."
LICENSE = "GPL-3.0-or-later"meta/recipes-devtools/autoconf/autoconf_2.72.bb +1
-1@@ -19,6 +19,7 @@ SRC_URI = " \ file://preferbash.patch \ file://autotest-automake-result-format.patch \ file://man-host-perl.patch \ + file://0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch \ " SRC_URI:append:class-native = " file://no-man.patch" -=-=-=-=-=-=-=-=-=-=-=-Links: You receive all messages sent to this group. View/Reply Online (#226850):
https://lists.openembedded.org/g/openembedded-core/message/226850
Mute This Topic: https://lists.openembedded.org/mt/116503258/4316185
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 2 lines
Yoann Congal Smile ECS
Quoted 12 lines · Khem Raj via lists.openembedded.org
Latest autoconf is splitting long cmd lines in config.status it generates using sed/awk to have compatibility with platform like HPUX,Solaris,AIX which is understandable given autotools try to achieve wider portability, however, this does mean that in OE builds we might get these newlines added in places dividing commands including buildpaths in them and the methods deployed to scrub them from output artifacts will fail since the string is now on two lines resulting in build QA to pass but the strings do sneak in affecting reproducibility one such example is e2tools recipe in meta-filesystems and there perhaps are more
Quoted 2 lines
Signed-off-by Khem Raj <[email protected]>---
Hi Khem,
Thanks for your patch.
I've had some issues on the autobuilder for a few days, and it looks like it was triggered by this patch. While there was no specific error, the -tc build jobs had been running for several days and never ending.
One example:
https://autobuilder.yoctoproject.org/valkyrie/?#/builders/66/builds/2740
I've had a look on the worker: it looks like the host nfs server, used to share files with the tests running in qemu, had been failing:
pokybui+ 4115014 1.1 0.0 0 0 ? Z 14:23 4:32 [unfsd] <defunct>
I clearly can't see how this can be linked to this patch, yet, git bisect points to it, and after its removal I don't have the issue anymore.
So I'm not sure there is anything you can do, but I will accept any clue about what might going on here.
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Quoted 31 lines · Mathieu Dubois-Briand via lists.openembedded.org
Quoted 12 lines · Khem Raj via lists.openembedded.org
Latest autoconf is splitting long cmd lines in config.status it generates using sed/awk to have compatibility with platform like HPUX,Solaris,AIX which is understandable given autotools try to achieve wider portability, however, this does mean that in OE builds we might get these newlines added in places dividing commands including buildpaths in them and the methods deployed to scrub them from output artifacts will fail since the string is now on two lines resulting in build QA to pass but the strings do sneak in affecting reproducibility one such example is e2tools recipe in meta-filesystems and there perhaps are more
Quoted 2 lines
Signed-off-by Khem Raj <[email protected]>---
Hi Khem,
Thanks for your patch.
I've had some issues on the autobuilder for a few days, and it looks like it was triggered by this patch. While there was no specific error, the -tc build jobs had been running for several days and never ending.
One example:
https://autobuilder.yoctoproject.org/valkyrie/?#/builders/66/builds/2740I've had a look on the worker: it looks like the host nfs server, used to share files with the tests running in qemu, had been failing:
Quoted 8 lines
pokybui+ 4115014 1.1 0.0 0 0 ? Z 14:23 4:32 [unfsd] <defunct>
I clearly can't see how this can be linked to this patch, yet, git bisect points to it, and after its removal I don't have the issue anymore.
So I'm not sure there is anything you can do, but I will accept any clue about what might going on here.
Well, that's unfortunate... I've created a ticket related to this to avoid losing this in our mailboxes:
16093 – Compatibility with old sed/awk interferes with reproducibility
https://bugzilla.yoctoproject.org/show_bug.cgi?id=16093
Quoted 19 lines
Thanks,
Mathieu--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#227203): https://lists.openembedded.org/g/openembedded-core/message/227203
Mute This Topic: https://lists.openembedded.org/mt/116503258/4316185
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 2 lines
Yoann Congal Smile ECS