support/dependencies: add a check for a suitable gzip
PATCH 1/1 support/dependencies: add a check for a suitable gzip
Recently, some hash mismatch have been reported, both by users as well as autobuilder failures, about tarballs generated from git repositories.
This turned out to be caused by users having the 'gzip' command somehow aliased to 'pigz' (which stand for: parallel implementation of gzip, which takes advantage of multi-processor system to parallelise the compression).
Unfortunately, the output of pigz-compressed archives differ from that of gzip (even though they *are* valid gzip-compressed streams).
Add a dependency check that ensures that gzip is not pigz. If that is the case, bail out and refuse to build.
This is a stop-gap measure in preparation of the release. A complete solution would accept pigz as a decompressor (because that is totally OK), and ensure that we do build a host-gzip package should that be needed. This is a much bigger endeavour, so this simple solution is deemed enough for the release (after all, use of pigz is just atypical enough that it should not pose such a problem for users to reverti to using plain gzip).
Fixes:
http://autobuild.buildroot.org/results/330/3308271fc641cadb59dbf1b5ee529a84f79e6d5c/
--- support/dependencies/check-host-gzip.mk | 3 +++ support/dependencies/check-host-gzip.sh | 21 +++++++++++++++++++++ 2 files changed, 24 insertions(+) create mode 100644 support/dependencies/check-host-gzip.mk create mode 100755 support/dependencies/check-host-gzip.sh
support/dependencies/check-host-gzip.mk new file +3 -0
@@ -0,0 +1,3 @@ +ifeq (,$(call suitable-host-package,gzip)) +$(error No suitable gzip found) +endif
support/dependencies/check-host-gzip.sh new file +21 -0
@@ -0,0 +1,21 @@ +#!/bin/sh + +candidate="$1" # ignored + +gzip="$(which gzip)" +if [ ! -x "${gzip}" ]; then + exit 1 +fi + +# gzip displays its version string on stdout +# pigz displays its version string on stderr +version="$("${gzip}" --version 2>&1)" +case "${version}" in + (*pigz*) + printf "%s (%s) is not acceptable as a gzip implementation\n" \ + "${gzip}" "${version}" >&2 + exit 1 + ;; +esac + +printf "%s" "${gzip}"
Signature · 1 line
2.14.1
PATCH 1/1 support/dependencies: add a check for a suitable gzip
Hello,
Quoted 14 lines · Yann E. MORIN
Recently, some hash mismatch have been reported, both by users as well as autobuilder failures, about tarballs generated from git repositories.
This turned out to be caused by users having the 'gzip' command somehow aliased to 'pigz' (which stand for: parallel implementation of gzip, which takes advantage of multi-processor system to parallelise the compression).
Unfortunately, the output of pigz-compressed archives differ from that of gzip (even though they *are* valid gzip-compressed streams).
Add a dependency check that ensures that gzip is not pigz. If that is the case, bail out and refuse to build.
Quoted 7 lines
This is a stop-gap measure in preparation of the release. A complete solution would accept pigz as a decompressor (because that is totally OK), and ensure that we do build a host-gzip package should that be needed. This is a much bigger endeavour, so this simple solution is deemed enough for the release (after all, use of pigz is just atypical enough that it should not pose such a problem for users to reverti to using plain gzip).
Is it really that more complicated to add and use host-gzip ?
(This is a real question, not one asked with some irony.)
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 support/dependencies: add a check for a suitable gzip
Thomas, All,
On 2018-11-16 16:32 +0100, Thomas Petazzoni spake thusly:
Quoted 23 lines · Yann E. MORIN
Quoted 14 lines
Recently, some hash mismatch have been reported, both by users as well as autobuilder failures, about tarballs generated from git repositories.
This turned out to be caused by users having the 'gzip' command somehow aliased to 'pigz' (which stand for: parallel implementation of gzip, which takes advantage of multi-processor system to parallelise the compression).
Unfortunately, the output of pigz-compressed archives differ from that of gzip (even though they *are* valid gzip-compressed streams).
Add a dependency check that ensures that gzip is not pigz. If that is the case, bail out and refuse to build.
Quoted 7 lines
This is a stop-gap measure in preparation of the release. A complete solution would accept pigz as a decompressor (because that is totally OK), and ensure that we do build a host-gzip package should that be needed. This is a much bigger endeavour, so this simple solution is deemed enough for the release (after all, use of pigz is just atypical enough that it should not pose such a problem for users to reverti to using plain gzip).
Quoted 1 line
Is it really that more complicated to add and use host-gzip ?
We need to add an actual host-gzip package, then ensure it is used when needed (e.g. in _DOWNLOAD_DEPENDENCIES of git/svn/cvs based packages).
Definitiely not overly complex as I stated, no, but I was mostly looking through the release-incoming prism, with a goal to do the strict minimum to fail in a sane way, and was seeing the host-gzip as too far reaching for the release.
But I can do that if you prefer.
Quoted 1 line
(This is a real question, not one asked with some irony.)
;-)
Even then, I can stomach a bit of irony. Even if I am very often incapable of noticing it. :-]
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'