Parallel build failures with util-linux 2.40

Hi,

Since updating from util-linux 2.39.3 to 2.40.1 in Nixpkgs, we've been seeing intermittent parallel build failures like the following:

libtool: warning(B: relinking 'pam_lastlog2.la'(B
libtool: install: (cd /build/util-linux-2.40.1; /nix/store/306znyj77fv49kwnkpxmb0j2znqpa8bj-bash-5.2p26/bin/bash "/build/util-linux-2.40.1/libtool" --silent --tag CC --mode=relink gcc -fsigned-char -fno-common -Wall -Wextra -Waddress-of-packed-member -Wdiscarded-qualifiers -Wformat-security -Wimplicit-function-declaration -Wmissing-declarations -Wmissing-parameter-type -Wmissing-prototypes -Wnested-externs -Wno-missing-field-initializers -Wold-style-definition -Wpointer-arith -Wredundant-decls -Wsign-compare -Wstrict-prototypes -Wtype-limits -Wuninitialized -Wunused-but-set-parameter -Wunused-but-set-variable -Wunused-parameter -Wunused-result -Wunused-variable -Wvla -Walloca -Werror=sequence-point -I./liblastlog2/src -g -O2 -module -avoid-version -shared pam_lastlog2_la_LDFLAGS += -Wl,--version-script,./pam_lastlog2/src/pam_lastlog2.sym -o pam_lastlog2.la -rpath /nix/store/ri9wphsskq6ygxigvlbahb9mz379qiz5-util-linux-2.40.1-lib/lib/security pam_lastlog2/src/la-pam_lastlog2.lo liblastlog2.la )
libtool: warning(B: 'libmount.la' has not been installed in '/nix/store/ri9wphsskq6ygxigvlbahb9mz379qiz5-util-linux-2.40.1-lib/lib'(B
libtool: warning(B: '/build/util-linux-2.40.1/libblkid.la' has not been installed in '/nix/store/ri9wphsskq6ygxigvlbahb9mz379qiz5-util-linux-2.40.1-lib/lib'(B
libtool: install: /nix/store/php4qidg2bxzmm79vpri025bqi0fa889-coreutils-9.5/bin/install -c .libs/mount /nix/store/v6ff7704fchg30kj4z422693lgnnmmbn-util-linux-2.40.1-bin/bin/mount
/nix/store/bgcaxhhxswzvmxjbbgvvaximm5hwghz1-binutils-2.41/bin/ld: cannot find -llastlog2: No such file or directory
collect2: error: ld returned 1 exit status
libtool: warning(B: 'libmount.la' has not been installed in '/nix/store/ri9wphsskq6ygxigvlbahb9mz379qiz5-util-linux-2.40.1-lib/lib'(B
libtool: warning(B: '/build/util-linux-2.40.1/libblkid.la' has not been installed in '/nix/store/ri9wphsskq6ygxigvlbahb9mz379qiz5-util-linux-2.40.1-lib/lib'(B
libtool: error(B: error: relink 'pam_lastlog2.la' with the above command before installing it(B
make[3]: *** [Makefile:7598: install-securelibLTLIBRARIES] Error 1

Full log:
https://cache.nixos.org/log/pp817g4wf18pn7hrn4a7377yxf06bprw-util-linux-2.40.1.drv

Alyssa Ross

What does it mean "parallel build"? make -j ?

I'm not sure, but it seems like you're being affected by an extra "+=" in the LDFLAGS. This should be fixed by...

    https://github.com/util-linux/util-linux/commit/290748729dc3edf9ea1c680c8954441a5e367a44
    https://github.com/util-linux/util-linux/commit/597e8b246ae31366514ead6cca240a09fe5e1528

Can you try your use case with the latest git tree?

Karel

 Karel Zak  <[email protected]>
 http://karelzak.blogspot.com

Yeah.

Thanks for pointing these out — I've applied the patches to our build, but since the failure only happened intermittently, I'm not going to be able to tell if it's fixed the issue right away. We'll just have to wait and see, and no news will be good news.

Hi, I'm afraid we're still seeing similar failures with those two patches backported on top of 2.40.1:

	  /nix/store/3c54hwp1zji0h8yl66860cp8zp2vsrzv-bash-5.2p26/bin/bash ./libtool   --mode=install /nix/store/dm0wz420s9n1nbp3731y9iqya0zswyy8-coreutils-9.5/bin/install -c login dmesg mount umount wdctl mountpoint lsblk findmnt kill lsfd pipesz '/nix/store/hxd66fx3rvq1km5hjq6dgx8a6yw7czyd-util-linux-minimal-2.40.1-bin/bin'
	libtool: warning: relinking 'pam_lastlog2.la'
	libtool: install: (cd /build/util-linux-2.40.1; /nix/store/3c54hwp1zji0h8yl66860cp8zp2vsrzv-bash-5.2p26/bin/bash "/build/util-linux-2.40.1/libtool"  --silent --tag CC --mode=relink gcc -fsigned-char -fno-common -Wall -Wextra -Waddress-of-packed-member -Wdiscarded-qualifiers -Wformat-security -Wimplicit-function-declaration -Wmissing-declarations -Wmissing-parameter-type -Wmissing-prototypes -Wnested-externs -Wno-missing-field-initializers -Wold-style-definition -Wpointer-arith -Wredundant-decls -Wsign-compare -Wstrict-prototypes -Wtype-limits -Wuninitialized -Wunused-but-set-parameter -Wunused-but-set-variable -Wunused-parameter -Wunused-result -Wunused-variable -Wvla -Walloca -Werror=sequence-point -I./liblastlog2/src -g -O2 -module -avoid-version -shared -Wl,--version-script,./pam_lastlog2/src/pam_lastlog2.sym -o pam_lastlog2.la -rpath /nix/store/dsxjv3a5aj1f4pifv5a6rvpj7zk00vyx-util-linux-minimal-2.40.1-lib/lib/security pam_lastlog2/src/la-pam_lastlog2.lo liblastlog2.la )
	libtool: install: /nix/store/dm0wz420s9n1nbp3731y9iqya0zswyy8-coreutils-9.5/bin/install -c login /nix/store/hxd66fx3rvq1km5hjq6dgx8a6yw7czyd-util-linux-minimal-2.40.1-bin/bin/login
	libtool: install: /nix/store/dm0wz420s9n1nbp3731y9iqya0zswyy8-coreutils-9.5/bin/install -c dmesg /nix/store/hxd66fx3rvq1km5hjq6dgx8a6yw7czyd-util-linux-minimal-2.40.1-bin/bin/dmesg
	libtool: warning: 'libmount.la' has not been installed in '/nix/store/dsxjv3a5aj1f4pifv5a6rvpj7zk00vyx-util-linux-minimal-2.40.1-lib/lib'
	libtool: warning: '/build/util-linux-2.40.1/libblkid.la' has not been installed in '/nix/store/dsxjv3a5aj1f4pifv5a6rvpj7zk00vyx-util-linux-minimal-2.40.1-lib/lib'
	libtool: install: /nix/store/dm0wz420s9n1nbp3731y9iqya0zswyy8-coreutils-9.5/bin/install -c .libs/mount /nix/store/hxd66fx3rvq1km5hjq6dgx8a6yw7czyd-util-linux-minimal-2.40.1-bin/bin/mount
	libtool: warning: 'libmount.la' has not been installed in '/nix/store/dsxjv3a5aj1f4pifv5a6rvpj7zk00vyx-util-linux-minimal-2.40.1-lib/lib'
	libtool: warning: '/build/util-linux-2.40.1/libblkid.la' has not been installed in '/nix/store/dsxjv3a5aj1f4pifv5a6rvpj7zk00vyx-util-linux-minimal-2.40.1-lib/lib'
	libtool: install: /nix/store/dm0wz420s9n1nbp3731y9iqya0zswyy8-coreutils-9.5/bin/install -c .libs/umount /nix/store/hxd66fx3rvq1km5hjq6dgx8a6yw7czyd-util-linux-minimal-2.40.1-bin/bin/umount
	libtool: warning: 'libsmartcols.la' has not been installed in '/nix/store/dsxjv3a5aj1f4pifv5a6rvpj7zk00vyx-util-linux-minimal-2.40.1-lib/lib'
	libtool: install: /nix/store/dm0wz420s9n1nbp3731y9iqya0zswyy8-coreutils-9.5/bin/install -c .libs/wdctl /nix/store/hxd66fx3rvq1km5hjq6dgx8a6yw7czyd-util-linux-minimal-2.40.1-bin/bin/wdctl
	/nix/store/8qjkvnam69dm0r9lhirxy1ngzbsh6a8z-binutils-2.41/bin/ld: cannot find -llastlog2: No such file or directory
	libtool: warning: 'libmount.la' has not been installed in '/nix/store/dsxjv3a5aj1f4pifv5a6rvpj7zk00vyx-util-linux-minimal-2.40.1-lib/lib'
	collect2: error: ld returned 1 exit status
	libtool: warning: '/build/util-linux-2.40.1/libblkid.la' has not been installed in '/nix/store/dsxjv3a5aj1f4pifv5a6rvpj7zk00vyx-util-linux-minimal-2.40.1-lib/lib'
	libtool:   error: error: relink 'pam_lastlog2.la' with the above command before installing it
	make[3]: *** [Makefile:7603: install-securelibLTLIBRARIES] Error 1

Full log:
https://hydra.nixos.org/build/260133082/nixlog/11/raw

I wish I could provide more helpful information, but since it doesn't happen consistently it's hard to collect any.

Hello.

In some cases, make --shuffle option (available in GNU make 4.4) "amplifies" the probability that a Makefile bug shows up. I suggest that you people give it a try.

There is a blog entry from the author explaining --shuffle option:

https://trofi.github.io/posts/238-new-make-shuffle-mode.html

[ I'm just a long-time lurker in this list, but maybe this could help ]

Thanks.

Thanks for the suggestion, but I tried running with --shuffle for hours before submitting my report, but wasn't able to reproduce that way. :(