Skip to contentSkip to conversation

package/dhcp: specify host-gawk path for dhcp build environment

PATCH New Sergey Matyukevich 2 participants 5 messages Jun 23, 2020 – Sep 14, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 package/dhcp: specify host-gawk path for dhcp build environment

DHCP package silently fails to install its binaries if gawk tool is not installed on build server. This is still the case even if host-gawk package is already available: dhcp Makefiles detect its presence but still attempt to use gawk from host system.

Add host-gawk to the list of dependencies for dhcp and explicitely specify host-gawk location.

---
 package/dhcp/dhcp.mk | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
package/dhcp/dhcp.mk +3 -1
index ad59804d3b..779d56801f 100644
@@ -9,7 +9,7 @@ DHCP_SITE = http://ftp.isc.org/isc/dhcp/$(DHCP_VERSION)
 DHCP_INSTALL_STAGING = YES
 DHCP_LICENSE = MPL-2.0
 DHCP_LICENSE_FILES = LICENSE
-DHCP_DEPENDENCIES = bind
+DHCP_DEPENDENCIES = bind host-gawk
 
 # use libtool-enabled configure.ac
 define DHCP_LIBTOOL_AUTORECONF
@@ -21,6 +21,8 @@ DHCP_CONF_ENV = \
 		-D_PATH_DHCLIENT_CONF=\"/etc/dhcp/dhclient.conf\"' \
 	CFLAGS='$(TARGET_CFLAGS) -DISC_CHECK_NONE=1'
 
+DHCP_CONF_ENV += ac_cv_prog_AWK=$(HOST_DIR)/bin/gawk
+
 DHCP_CONF_OPTS = \
 	--with-libbind=$(STAGING_DIR)/usr \
 	--with-randomdev=/dev/random \
Signature · 1 line
2.27.0
#1 lore raw

PATCH 1/1 package/dhcp: specify host-gawk path for dhcp build environment

Hello Sergey,

Quoted 9 lines · On Tue, 23 Jun 2020 10:39:18 +0300 Sergey Matyukevich <[email protected]>…

DHCP package silently fails to install its binaries if gawk tool is not installed on build server. This is still the case even if host-gawk package is already available: dhcp Makefiles detect its presence but still attempt to use gawk from host system.

Add host-gawk to the list of dependencies for dhcp and explicitely specify host-gawk location.

Thanks for the patch. However, could you be more specific about what's missing on the target?

Indeed, I just did two builds of the DHCP package, will all sub-options enabled, one inside a Docker container that only has the "mawk" implementation of awk, and another on my machine, which as "gawk" installed as awk.

And in both cases, the set of files installed to target/ was exactly identical.

Could you provide more details ?

Thanks a lot!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#2 lore raw

PATCH 1/1 package/dhcp: specify host-gawk path for dhcp build environment

Hello Thomas and all,

Quoted 23 lines

Hello Sergey,

Quoted 9 lines · On Tue, 23 Jun 2020 10:39:18 +0300 Sergey Matyukevich <[email protected]>…

DHCP package silently fails to install its binaries if gawk tool is not installed on build server. This is still the case even if host-gawk package is already available: dhcp Makefiles detect its presence but still attempt to use gawk from host system.

Add host-gawk to the list of dependencies for dhcp and explicitely specify host-gawk location.

Thanks for the patch. However, could you be more specific about what's missing on the target?

Indeed, I just did two builds of the DHCP package, will all sub-options enabled, one inside a Docker container that only has the "mawk" implementation of awk, and another on my machine, which as "gawk" installed as awk.

Quoted 4 lines

And in both cases, the set of files installed to target/ was exactly identical.

Could you provide more details ?

But what were the files installed in your tests? Did you see dhcpd and dhclient amount installed files in output/target ?

I still can reproduce the issue. The procedure is similar to yours. I enable both client and server for isc-dhcp in the board config. Finally, I do not have both dhcpd and dhclient in output/target if I do not have gawk installed on my build server. Moreover, in this case the following warnings appear in the end of dhcp build log:

bin/sh: line 19: gawk: command not found

Such warning are emitted by DHCP_INSTALL_SERVER and DHCP_INSTALL_CLIENT commands from dhcp.mk. The same result can be reproduced manually:

$ make -C /home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server DESTDIR=/home/matsi/devel/code/buildroot/buildroot-release/output/target install-sbinPROGRAMS         
make: Entering directory '/home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server'
 /usr/bin/mkdir -p  '/home/matsi/devel/code/buildroot/buildroot-release/output/target/usr/sbin'
/bin/sh: line 19: gawk: command not found
make: Leaving directory '/home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server'

What exactly goes on can be figured out using '-n' make option. Makefiles for isc-dhcp components use $(AWK) which is properly detected by configure script. It can any of gawk, mawk, nawk, awk. However in generated Makefiles that tool is used without specific path. As a result, *awk tool can be available in buildroot build environment, but missing on the build server.

Regards,
Sergey

#3 lore raw

PATCH 1/1 package/dhcp: specify host-gawk path for dhcp build environment

Hello Sergey,

Quoted 2 lines · On Sat, 12 Sep 2020 23:01:14 +0300 Sergey Matyukevich <[email protected]>…

But what were the files installed in your tests? Did you see dhcpd and dhclient amount installed files in output/target ?

Yes, both dhcpd and dhclient are installed in output/target. I'm attaching to this e-mail the list of files that were installed when building with awk==mawk and awk==gawk.

Quoted 17 lines

I still can reproduce the issue. The procedure is similar to yours. I enable both client and server for isc-dhcp in the board config. Finally, I do not have both dhcpd and dhclient in output/target if I do not have gawk installed on my build server. Moreover, in this case the following warnings appear in the end of dhcp build log:

bin/sh: line 19: gawk: command not found

Such warning are emitted by DHCP_INSTALL_SERVER and DHCP_INSTALL_CLIENT commands from dhcp.mk. The same result can be reproduced manually:

$ make -C /home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server DESTDIR=/home/matsi/devel/code/buildroot/buildroot-release/output/target install-sbinPROGRAMS         
make: Entering directory '/home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server'
 /usr/bin/mkdir -p  '/home/matsi/devel/code/buildroot/buildroot-release/output/target/usr/sbin'
/bin/sh: line 19: gawk: command not found
make: Leaving directory '/home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server'
Quoted 6 lines

What exactly goes on can be figured out using '-n' make option. Makefiles for isc-dhcp components use $(AWK) which is properly detected by configure script. It can any of gawk, mawk, nawk, awk. However in generated Makefiles that tool is used without specific path. As a result, *awk tool can be available in buildroot build environment, but missing on the build server.

I've posted at https://paste.ack.tf/52f197 at raw the full build of my Buildroot configuration, which has just BR2_PACKAGE_DHCP=y + client + server. awk is really mawk:

checking for gawk... no
checking for mawk... mawk

Both dhpcd and dhclient are installed:

$ find target/ -name 'dhcpd' -o -name 'dhclient'
target/usr/sbin/dhcpd
target/sbin/dhclient

I have this version of mawk installed:

ii mawk 1.3.3-17+b3 amd64 a pattern scanning and text processing language

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#4 lore raw

PATCH 1/1 package/dhcp: specify host-gawk path for dhcp build environment

Hello Thomas,

Quoted 45 lines

Hello Sergey,

Quoted 2 lines · On Sat, 12 Sep 2020 23:01:14 +0300 Sergey Matyukevich <[email protected]>…

But what were the files installed in your tests? Did you see dhcpd and dhclient amount installed files in output/target ?

Yes, both dhcpd and dhclient are installed in output/target. I'm attaching to this e-mail the list of files that were installed when building with awk==mawk and awk==gawk.

Quoted 17 lines

I still can reproduce the issue. The procedure is similar to yours. I enable both client and server for isc-dhcp in the board config. Finally, I do not have both dhcpd and dhclient in output/target if I do not have gawk installed on my build server. Moreover, in this case the following warnings appear in the end of dhcp build log:

bin/sh: line 19: gawk: command not found

Such warning are emitted by DHCP_INSTALL_SERVER and DHCP_INSTALL_CLIENT commands from dhcp.mk. The same result can be reproduced manually:

$ make -C /home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server DESTDIR=/home/matsi/devel/code/buildroot/buildroot-release/output/target install-sbinPROGRAMS         
make: Entering directory '/home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server'
 /usr/bin/mkdir -p  '/home/matsi/devel/code/buildroot/buildroot-release/output/target/usr/sbin'
/bin/sh: line 19: gawk: command not found
make: Leaving directory '/home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server'
Quoted 6 lines

What exactly goes on can be figured out using '-n' make option. Makefiles for isc-dhcp components use $(AWK) which is properly detected by configure script. It can any of gawk, mawk, nawk, awk. However in generated Makefiles that tool is used without specific path. As a result, *awk tool can be available in buildroot build environment, but missing on the build server.

I've posted at https://paste.ack.tf/52f197 at raw the full build of my Buildroot configuration, which has just BR2_PACKAGE_DHCP=y + client + server. awk is really mawk:

checking for gawk... no
checking for mawk... mawk

Both dhpcd and dhclient are installed:

Quoted 7 lines
$ find target/ -name 'dhcpd' -o -name 'dhclient'
target/usr/sbin/dhcpd
target/sbin/dhclient

I have this version of mawk installed:

ii mawk 1.3.3-17+b3 amd64 a pattern scanning and text processing language

Let me clarify a possible root cause. IIUC, the problem occurs when buildroot configuration and build host environment provide _different_ flavors of awk. For instance, mawk is available on host and gawk is available in buildroot output/host. In this case isc-dhcp configure script detects gawk in output/host and generates Makefiles specifying gawk without explicit path. Finally, during Buildroot installation phase, those Makefiles are used to install dhcp binaries. They attempt to use gawk without specific path, but host does not provide gawk.

Could you please take another look at your tests and check whether you have different awk flavors in buildroot config and on build host ? If yes, then my understanding is not correct. In this case I would suggest to mark this patch as rejected for the time being.

Regards,
Sergey