Skip to contentSkip to conversation

autoconf: Raise line length limit for sed/awk

PATCH In review Khem Raj 3 participants 4 messages Nov 27, 2025 – Dec 8, 2025

lore
Share & tools
lore

Khem Raj

author
#0 lore raw

PATCH 1/1 autoconf: Raise line length limit for sed/awk

Latest autoconf is splitting long cmd lines in config.status it generates using sed/awk to have compatibility with platform like HPUX,Solaris,AIX which is understandable given autotools try to achieve wider portability, however, this does mean that in OE builds we might get these newlines added in places dividing commands including buildpaths in them and the methods deployed to scrub them from output artifacts will fail since the string is now on two lines resulting in build QA to pass but the strings do sneak in affecting reproducibility one such example is e2tools recipe in meta-filesystems and there perhaps are more

---
 ...-command-limit-to-prevent-line-wrapp.patch | 41 +++++++++++++++++++
 .../autoconf/autoconf_2.72.bb                 |  3 +-
 2 files changed, 43 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch new file +41 -0
new file mode 100644
index 00000000000..41468ba316c
@@ -0,0 +1,41 @@
+From 139f757a6244d3b1e07f7fddc6d07d9c17f45313 Mon Sep 17 00:00:00 2001
+From: Khem Raj <[email protected]>
+Date: Tue, 25 Nov 2025 21:03:38 -0800
+Subject: [PATCH] Increase awk/sed command limit to prevent line wrapping
+
+Autoconf 2.70+ wraps long variable assignments in config.status
+using backslash-newline continuations. This breaks when LDFLAGS
+or other variables contain special characters.
+
+Increase _AC_SED_CMD_LIMIT from 99 and _AC_AWK_LITERAL_LIMIT
+from 148 to 4000 to prevent wrapping.
+
+Upstream-Status: Inappropriate [OE-Specific]
+
+Signed-off-by: Khem Raj <[email protected]>
+---
+ lib/autoconf/status.m4 | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/lib/autoconf/status.m4 b/lib/autoconf/status.m4
+index 2bfaf32..be444b2 100644
+--- a/lib/autoconf/status.m4
++++ b/lib/autoconf/status.m4
+@@ -297,7 +297,7 @@ AC_DEFUN([AC_CONFIG_FILES], [_AC_CONFIG_FOOS([FILES], $@)])
+ # Some seds have small command number limits, like on Digital OSF/1 and HP-UX.
+ m4_define([_AC_SED_CMD_LIMIT],
+ dnl One cannot portably go further than 99 commands because of HP-UX.
+-[99])
++[4000])
+ 
+ 
+ # _AC_AWK_LITERAL_LIMIT
+@@ -307,7 +307,7 @@ dnl One cannot portably go further than 99 commands because of HP-UX.
+ #
+ # Some awk's have small limits, such as Solaris and AIX awk.
+ m4_define([_AC_AWK_LITERAL_LIMIT],
+-[148])
++[4000])
+ 
+ 
+ # _AC_OUTPUT_FILES_PREPARE
meta/recipes-devtools/autoconf/autoconf_2.72.bb +2 -1
index b599f270c79..20197d6f793 100644
@@ -1,5 +1,5 @@
 SUMMARY = "A GNU tool that produce shell scripts to automatically configure software"
-DESCRIPTION = "Autoconf is an extensible package of M4 macros that produce shell scripts to automatically \ 
+DESCRIPTION = "Autoconf is an extensible package of M4 macros that produce shell scripts to automatically \
 configure software source code packages. Autoconf creates a configuration script for a package from a template \
 file that lists the operating system features that the package can use, in the form of M4 macro calls."
 LICENSE = "GPL-3.0-or-later"
@@ -19,6 +19,7 @@ SRC_URI = " \
            file://preferbash.patch \
            file://autotest-automake-result-format.patch \
            file://man-host-perl.patch \
+		   file://0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch \
 "
 SRC_URI:append:class-native = " file://no-man.patch"
 
Tested-by on v1 1/1
#1 lore raw
Quoted 12 lines · Khem Raj via lists.openembedded.org

Latest autoconf is splitting long cmd lines in config.status it generates using sed/awk to have compatibility with platform like HPUX,Solaris,AIX which is understandable given autotools try to achieve wider portability, however, this does mean that in OE builds we might get these newlines added in places dividing commands including buildpaths in them and the methods deployed to scrub them from output artifacts will fail since the string is now on two lines resulting in build QA to pass but the strings do sneak in affecting reproducibility one such example is e2tools recipe in meta-filesystems and there perhaps are more

Quoted 2 lines

For the impact on e2tools:

=> It does indeed make it reproducible by letting the sed replacing non reproducible paths work.

Quoted 98 lines
---
 ...-command-limit-to-prevent-line-wrapp.patch | 41 +++++++++++++++++++
 .../autoconf/autoconf_2.72.bb                 |  3 +-
 2 files changed, 43 insertions(+), 1 deletion(-)
 create mode 100644

meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch

diff --git a/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch b/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch new file mode 100644
index 00000000000..41468ba316c
--- /dev/null
+++
b/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch

meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch new file +24 -0
@@ -0,0 +1,41 @@
+From 139f757a6244d3b1e07f7fddc6d07d9c17f45313 Mon Sep 17 00:00:00 2001
+From: Khem Raj <[email protected]>
+Date: Tue, 25 Nov 2025 21:03:38 -0800
+Subject: [PATCH] Increase awk/sed command limit to prevent line wrapping
+
+Autoconf 2.70+ wraps long variable assignments in config.status
+using backslash-newline continuations. This breaks when LDFLAGS
+or other variables contain special characters.
+
+Increase _AC_SED_CMD_LIMIT from 99 and _AC_AWK_LITERAL_LIMIT
+from 148 to 4000 to prevent wrapping.
+
+Upstream-Status: Inappropriate [OE-Specific]
+
+Signed-off-by: Khem Raj <[email protected]>
+---
+ lib/autoconf/status.m4 | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/lib/autoconf/status.m4 b/lib/autoconf/status.m4
+index 2bfaf32..be444b2 100644
+--- a/lib/autoconf/status.m4
++++ b/lib/autoconf/status.m4
+@@ -297,7 +297,7 @@ AC_DEFUN([AC_CONFIG_FILES], [_AC_CONFIG_FOOS([FILES],

$@)])
+ # Some seds have small command number limits, like on Digital OSF/1 and HP-UX.
+ m4_define([_AC_SED_CMD_LIMIT],
+ dnl One cannot portably go further than 99 commands because of HP-UX. +-[99])
++[4000])

meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch new file +4 -0
+
+
+ # _AC_AWK_LITERAL_LIMIT
+@@ -307,7 +307,7 @@ dnl One cannot portably go further than 99 commands

because of HP-UX.
+ #
+ # Some awk's have small limits, such as Solaris and AIX awk.
+ m4_define([_AC_AWK_LITERAL_LIMIT],
+-[148])
++[4000])

meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch new file +3 -0
+
+
+ # _AC_OUTPUT_FILES_PREPARE
meta/recipes-devtools/autoconf/autoconf_2.72.bb +0 -0
index b599f270c79..20197d6f793 100644
@@ -1,5 +1,5 @@
 SUMMARY = "A GNU tool that produce shell scripts to automatically

configure software"
-DESCRIPTION = "Autoconf is an extensible package of M4 macros that produce shell scripts to automatically \
+DESCRIPTION = "Autoconf is an extensible package of M4 macros that produce shell scripts to automatically \
configure software source code packages. Autoconf creates a configuration
script for a package from a template \
file that lists the operating system features that the package can use,
in the form of M4 macro calls."
LICENSE = "GPL-3.0-or-later"

meta/recipes-devtools/autoconf/autoconf_2.72.bb +1 -1
@@ -19,6 +19,7 @@ SRC_URI = " \
            file://preferbash.patch \
            file://autotest-automake-result-format.patch \
            file://man-host-perl.patch \
+
 file://0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch \
 "
 SRC_URI:append:class-native = " file://no-man.patch"
 
 
-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group. View/Reply Online (#226850):
https://lists.openembedded.org/g/openembedded-core/message/226850
Mute This Topic: https://lists.openembedded.org/mt/116503258/4316185
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 2 lines
Yoann Congal
Smile ECS
#2 lore raw
In reply to Khem Raj
Quoted 12 lines · Khem Raj via lists.openembedded.org

Latest autoconf is splitting long cmd lines in config.status it generates using sed/awk to have compatibility with platform like HPUX,Solaris,AIX which is understandable given autotools try to achieve wider portability, however, this does mean that in OE builds we might get these newlines added in places dividing commands including buildpaths in them and the methods deployed to scrub them from output artifacts will fail since the string is now on two lines resulting in build QA to pass but the strings do sneak in affecting reproducibility one such example is e2tools recipe in meta-filesystems and there perhaps are more

Quoted 2 lines

---

Hi Khem,

Thanks for your patch.

I've had some issues on the autobuilder for a few days, and it looks like it was triggered by this patch. While there was no specific error, the -tc build jobs had been running for several days and never ending.

One example:
https://autobuilder.yoctoproject.org/valkyrie/?#/builders/66/builds/2740

I've had a look on the worker: it looks like the host nfs server, used to share files with the tests running in qemu, had been failing:

pokybui+ 4115014 1.1 0.0 0 0 ? Z 14:23 4:32 [unfsd] <defunct>

I clearly can't see how this can be linked to this patch, yet, git bisect points to it, and after its removal I don't have the issue anymore.

So I'm not sure there is anything you can do, but I will accept any clue about what might going on here.

Thanks,
Mathieu

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#3 lore raw
Quoted 31 lines · Mathieu Dubois-Briand via lists.openembedded.org
Quoted 12 lines · Khem Raj via lists.openembedded.org

Latest autoconf is splitting long cmd lines in config.status it generates using sed/awk to have compatibility with platform like HPUX,Solaris,AIX which is understandable given autotools try to achieve wider portability, however, this does mean that in OE builds we might get these newlines added in places dividing commands including buildpaths in them and the methods deployed to scrub them from output artifacts will fail since the string is now on two lines resulting in build QA to pass but the strings do sneak in affecting reproducibility one such example is e2tools recipe in meta-filesystems and there perhaps are more

Quoted 2 lines

---

Hi Khem,

Thanks for your patch.

I've had some issues on the autobuilder for a few days, and it looks like it was triggered by this patch. While there was no specific error, the -tc build jobs had been running for several days and never ending.

One example:
https://autobuilder.yoctoproject.org/valkyrie/?#/builders/66/builds/2740

I've had a look on the worker: it looks like the host nfs server, used to share files with the tests running in qemu, had been failing:

Quoted 8 lines

pokybui+ 4115014 1.1 0.0 0 0 ? Z 14:23 4:32 [unfsd] <defunct>

I clearly can't see how this can be linked to this patch, yet, git bisect points to it, and after its removal I don't have the issue anymore.

So I'm not sure there is anything you can do, but I will accept any clue about what might going on here.

Well, that's unfortunate... I've created a ticket related to this to avoid losing this in our mailboxes:
16093 – Compatibility with old sed/awk interferes with reproducibility
https://bugzilla.yoctoproject.org/show_bug.cgi?id=16093

Quoted 19 lines

Thanks,
Mathieu

--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#227203): https://lists.openembedded.org/g/openembedded-core/message/227203
Mute This Topic: https://lists.openembedded.org/mt/116503258/4316185
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 2 lines
Yoann Congal
Smile ECS