support/download: be more aggressive on missing hashes (branch yem/dl-hash)
cover support/download: be more aggressive on missing hashes (branch yem/dl-hash)
Hello All!
This series makes hashes mandatory when a .hash file exists.
Currently, we treat missing hashes as a mere warning. Unfortunately, that often goes un-noticed by most users, and thus we get a lot of package bumps that are missing the new hashes corresponding to the new version.
We now make that a hard error, so users really notice something is wrong.
Of course, if no .hash file exists, the behaviour is as yet unchanged.
Regards,
Yann E. MORIN.
The following changes since commit 2d05afa42792193fa392c9f5417e8effc73d1e38:
cups: deprecate package due to security issues (2015-03-16 22:26:30 +0100)
are available in the git repository at:
git://git.busybox.net/~ymorin/git/buildroot yem/dl-hash
for you to fetch changes up to e443dfcec4c084f823c718aa4c7740eddb3b628e:
support/download: always fail when there's no hash (2015-03-16 23:33:48 +0100)
----------------------------------------------------------------
Yann E. MORIN (3):
support/download: return different exit codes for different failures support/download: properly catch missing hashes support/download: always fail when there's no hash
support/download/check-hash | 15 +++++++++------ support/download/dl-wrapper | 11 ++++++++++- 2 files changed, 19 insertions(+), 7 deletions(-)
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/3 support/download: return different exit codes for different failures
Return different exit codes depending on the error that occured:
0: no error (hash file missing, or all hashes match) 1: hash file exists, but at least one hash in error 3: hash file exists, but no hash for file to check
--- support/download/check-hash | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
support/download/check-hash +8 -1
@@ -9,6 +9,13 @@ set -e # $3: the final basename of the file, to which it will be ultimately # saved as, to be able to match it to the corresponding hashes # in the .hash file +# +# Exits with: +# 0: the hash file exists and the file to check matches all its hashes, +# or the hash file does not exist +# 1: the hash file exists and the file to check does not match at least +# of its hashes +# 2: the hash file exists and there was no hash to check the fiel against while getopts :q OPT; do case "${OPT}" in
@@ -83,7 +90,7 @@ done <"${h_file}" if [ ${nb_checks} -eq 0 ]; then if [ -n "${BR2_ENFORCE_CHECK_HASH}" ]; then printf "ERROR: No hash found for %s\n" "${base}" >&2 - exit 1 + exit 2 else printf "WARNING: No hash found for %s\n" "${base}" >&2 fi
Signature · 1 line
1.9.1
PATCH 1/3 support/download: return different exit codes for different failures
Yann,
Just few typos ;)
Quoted 6 lines · Yann E. MORIN
Return different exit codes depending on the error that occured:
0: no error (hash file missing, or all hashes match) 1: hash file exists, but at least one hash in error 3: hash file exists, but no hash for file to check
s/3/2/
Quoted 12 lines
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Thomas Petazzoni <[email protected]>--- support/download/check-hash | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)support/download/check-hash +0
-0
support/download/check-hash set -e · line 18
Quoted 10 lines
support/download/check-hash +7
-0@@ -9,6 +9,13 @@ set -e # $3: the final basename of the file, to which it will be ultimately # saved as, to be able to match it to the corresponding hashes # in the .hash file +# +# Exits with: +# 0: the hash file exists and the file to check matches all its hashes, +# or the hash file does not exist +# 1: the hash file exists and the file to check does not match at least +# of its hashes +# 2: the hash file exists and there was no hash to check the fiel against
s/fiel/file/
Quoted 15 lines
support/download/check-hash +1
-2while getopts :q OPT; do case "${OPT}" in@@ -83,7 +90,7 @@ done <"${h_file}" if [ ${nb_checks} -eq 0 ]; then if [ -n "${BR2_ENFORCE_CHECK_HASH}" ]; then printf "ERROR: No hash found for %s\n" "${base}" >&2 - exit 1 + exit 2 else printf "WARNING: No hash found for %s\n" "${base}" >&2 fi --1.9.1
Quoted 4 lines
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Regards,
Signature · 1 line
Samuel
PATCH 2/3 support/download: properly catch missing hashes
In reply to Yann E. MORINWhen checking hashes reports no hash for a file, and this is treated as an error (now: because BR2_ENFORCE_CHECK_HASH is set; later: becasue that will be the new and only behaviour), exit promptly in error.
--- support/download/dl-wrapper | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
support/download/dl-wrapper +10 -1
@@ -48,10 +48,19 @@ main() { error "no hash-file specified, use -H\n" fi - # If the output file already exists, do not download it again + # If the output file already exists and: + # - there's no .hash file: do not download it again and exit promptly + # - matches all its hashes: do not download it again and exit promptly + # - fails at least one of its hashes: force a re-download + # - there's no hash (but a .hash file): consider it a hard error if [ -e "${output}" ]; then if support/download/check-hash ${quiet} "${hfile}" "${output}" "${output##*/}"; then exit 0 + elif [ ${?} -eq 2 ]; then + # Do not remove the file, otherwise it might get re-downloaded + # from a later location (i.e. primary -> upstream -> mirror). + # Do not print a message, check-hash already did + exit 2 fi rm -f "${output}" warn "Re-downloading '%s'...\n" "${output##*/}"
Signature · 1 line
1.9.1
PATCH 3/3 support/download: always fail when there's no hash
In reply to Yann E. MORINAt the time we introduced hashes, we did not want to be too harsh in the beginning, and give people some time to adapt and accept to hashes. So we so far only whined^Wwarned about a missing hash file.
Some time has passed now, and people are still missing updating hashes when bumping packages.
Let's make that warning a little bit more annoying...
--- support/download/check-hash | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
support/download/check-hash +2 -6
@@ -88,10 +88,6 @@ while read t h f; do done <"${h_file}" if [ ${nb_checks} -eq 0 ]; then - if [ -n "${BR2_ENFORCE_CHECK_HASH}" ]; then - printf "ERROR: No hash found for %s\n" "${base}" >&2 - exit 2 - else - printf "WARNING: No hash found for %s\n" "${base}" >&2 - fi + printf "ERROR: No hash found for %s\n" "${base}" >&2 + exit 2 fi
Signature · 1 line
1.9.1
PATCH 3/3 support/download: always fail when there's no hash
Quoted 11 lines · Yann E. MORIN
At the time we introduced hashes, we did not want to be too harsh in the beginning, and give people some time to adapt and accept to hashes. So we so far only whined^Wwarned about a missing hash file.
Some time has passed now, and people are still missing updating hashes when bumping packages.
Let's make that warning a little bit more annoying...
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Thomas Petazzoni <[email protected]>
Quoted 24 lines
--- support/download/check-hash | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)support/download/check-hash +2
-7@@ -88,10 +88,6 @@ while read t h f; do done <"${h_file}" if [ ${nb_checks} -eq 0 ]; then - if [ -n "${BR2_ENFORCE_CHECK_HASH}" ]; then - printf "ERROR: No hash found for %s\n" "${base}" >&2 - exit 2 - else - printf "WARNING: No hash found for %s\n" "${base}" >&2 - fi + printf "ERROR: No hash found for %s\n" "${base}" >&2 + exit 2 fi --1.9.1
Quoted 4 lines
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Regards,
Signature · 1 line
Samuel
PATCH 3/3 support/download: always fail when there's no hash
In reply to Yann E. MORINQuoted 8 lines · Yann E. MORIN
At the time we introduced hashes, we did not want to be too harsh in the beginning, and give people some time to adapt and accept to hashes. So we so far only whined^Wwarned about a missing hash file.
Some time has passed now, and people are still missing updating hashes when bumping packages.
Let's make that warning a little bit more annoying...
If you ever revive this series (which I think would be a good idea), don't
forget to remove the reference to BR2_ENFORCE_CHECK_HASH from the manual as well.
Regards,
Arnout
Quoted 25 lines
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Thomas Petazzoni <[email protected]>--- support/download/check-hash | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)support/download/check-hash +2
-6@@ -88,10 +88,6 @@ while read t h f; do done <"${h_file}" if [ ${nb_checks} -eq 0 ]; then - if [ -n "${BR2_ENFORCE_CHECK_HASH}" ]; then - printf "ERROR: No hash found for %s\n" "${base}" >&2 - exit 2 - else - printf "WARNING: No hash found for %s\n" "${base}" >&2 - fi + printf "ERROR: No hash found for %s\n" "${base}" >&2 + exit 2 fi
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
PATCH 3/3 support/download: always fail when there's no hash
Arnout, All,
On 2015-03-19 00:15 +0100, Arnout Vandecappelle spake thusly:
Quoted 5 lines · Yann E. MORIN
Quoted 4 lines
At the time we introduced hashes, we did not want to be too harsh in the beginning, and give people some time to adapt and accept to hashes. So we so far only whined^Wwarned about a missing hash file.
Quoted 7 lines
Quoted 4 lines
Some time has passed now, and people are still missing updating hashes when bumping packages.
Let's make that warning a little bit more annoying...
If you ever revive this series (which I think would be a good idea), don't
forget to remove the reference toBR2_ENFORCE_CHECK_HASHfrom the manual as well.
Already done in v2:
http://patchwork.ozlabs.org/patch/450979/
;-)
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
cover support/download: be more aggressive on missing hashes (branch yem/dl-hash)
In reply to Yann E. MORINAll,
On 2015-03-16 23:41 +0100, Yann E. MORIN spake thusly:
Quoted 1 line
This series makes hashes mandatory when a .hash file exists.
Forget this series: it is broken for git downloads, and potentially for all non-tarball downloads as well...
Sorry...
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'