Skip to contentSkip to conversation

Fix git download helper (branch yem/download-fixes)

PATCH New Yann E. MORIN 3 participants 7 messages Aug 6, 2014 – Aug 15, 2014

lore
Share & tools
lore

Yann E. MORIN

author
#0 lore raw

v3 cover Fix git download helper (branch yem/download-fixes)

Hello All!

This series fixes the git helper after some fall-overs when the wrapper was introduced.

Changes v2 -> v3:
- include initial patch from Peter (Peter, Rohit)
- fix the output file name (Graham)

Regards,
Yann E. MORIN.

The following changes since commit 10b6d10009c70262cfb492b2abaa3a091a8fb4aa:

espeak: new package (2014-08-04 22:40:45 +0200)

are available in the git repository at:

git://gitorious.org/buildroot/buildroot.git yem/download-fixes

for you to fetch changes up to 2205cccc9b1580208e604067224448da94599151:

support/download: fix the git helper output file format (2014-08-06 18:33:33 +0200)

----------------------------------------------------------------
Peter Seiderer (1):
support/download: fix the git helper

Yann E. MORIN (1):
support/download: fix the git helper output file format

 support/download/git | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
author
#1 lore raw

PATCH v3 1/2 support/download: fix the git helper

From: Peter Seiderer <[email protected]>

Re-add the git_done variable (lost in commit [1]).

Fixes download problem reported by Rohit Kumar [2].

[1] http://git.buildroot.net/buildroot/commit/?id=7e40a1103a919a8177f00ddca2b46b4439953511
[2] http://lists.busybox.net/pipermail/buildroot/2014-August/103733.html

Signed-off-by Peter Seiderer <[email protected]>
Acked-by "Yann E. MORIN" <[email protected]>
---
 support/download/git | 1 +
 1 file changed, 1 insertion(+)
support/download/git +1 -0
index 6edaa90..d451530 100755
@@ -19,6 +19,7 @@ basename="${4}"
 
 # Try to see if we can do a shallow clone, since it is faster
 # than a full clone.
+git_done=0
 if [ -n "$(${GIT} ls-remote "${repo}" "${cset}" 2>&1)" ]; then
     printf "Doing shallow clone\n"
     if ${GIT} clone --depth 1 -b "${cset}" --bare "${repo}" "${basename}"; then
Signature · 1 line
1.9.1
author
#2 lore raw

PATCH v3 2/2 support/download: fix the git helper output file format

In reply to Yann E. MORIN

The git helper uses gzip to compress the intermediate tarball. But gzip removes the source file, and create a new file named by appending .gz to the original file name.

Thus, we end up with output.gz, while the download wrapper expects jsut output, and thus believes the downlaod failed.

Fix that by storing the tar from git to a temporary file, then pipe this file to gzip's stdin, and redirect gzip's stdout to the output file.

Reported-by Graham Newton <[email protected]>
Signed-off-by "Yann E. MORIN" <[email protected]>
Cc Peter Seiderer <[email protected]>
---
 support/download/git | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
support/download/git +2 -2
index d451530..ff6b5c3 100755
@@ -34,6 +34,6 @@ if [ ${git_done} -eq 0 ]; then
 fi
 
 GIT_DIR="${basename}" \
-${GIT} archive --prefix="${basename}/" -o "${output}" --format=tar "${cset}"
+${GIT} archive --prefix="${basename}/" -o "${output}.tmp" --format=tar "${cset}"
 
-gzip "${output}"
+gzip <"${output}.tmp" >"${output}"
Signature · 1 line
1.9.1
#3 lore raw

PATCH v3 2/2 support/download: fix the git helper output file format

Quoted 24 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

The git helper uses gzip to compress the intermediate tarball. But gzip removes the source file, and create a new file named by appending .gz to the original file name.

Thus, we end up with output.gz, while the download wrapper expects jsut output, and thus believes the downlaod failed.

Fix that by storing the tar from git to a temporary file, then pipe this file to gzip's stdin, and redirect gzip's stdout to the output file.

Reported-by Graham Newton <[email protected]>
Signed-off-by "Yann E. MORIN" <[email protected]>
Cc Peter Seiderer <[email protected]>
---
 support/download/git | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
support/download/git +0 -0
index d451530..ff6b5c3 100755

support/download/git if [ ${git_done} -eq 0 ]; then · line 39

Quoted 8 lines
support/download/git +2 -2
@@ -34,6 +34,6 @@ if [ ${git_done} -eq 0 ]; then
 fi
 
 GIT_DIR="${basename}" \
-${GIT} archive --prefix="${basename}/" -o "${output}" --format=tar "${cset}"
+${GIT} archive --prefix="${basename}/" -o "${output}.tmp" --format=tar "${cset}"
 
-gzip "${output}"
+gzip <"${output}.tmp" >"${output}"

Don't we then end up with a bunch of foo.tmp files?

Signature · 1 line
Bye, Peter Korsgaard
author
#4 lore raw

PATCH v3 2/2 support/download: fix the git helper output file format

Peter, All,

On 2014-08-15 19:22 +0200, Peter Korsgaard spake thusly:

Quoted 26 lines
Quoted 25 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

The git helper uses gzip to compress the intermediate tarball. But gzip removes the source file, and create a new file named by appending .gz to the original file name.

Thus, we end up with output.gz, while the download wrapper expects jsut output, and thus believes the downlaod failed.

Fix that by storing the tar from git to a temporary file, then pipe this file to gzip's stdin, and redirect gzip's stdout to the output file.

Reported-by Graham Newton <[email protected]>
Signed-off-by "Yann E. MORIN" <[email protected]>
Cc Peter Seiderer <[email protected]>
---
 support/download/git | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
support/download/git +0 -0
index d451530..ff6b5c3 100755
@@ -34,6 +34,6 @@ if [ ${git_done} -eq 0 ]; then
 fi
Quoted 8 lines

support/download/git if [ ${git_done} -eq 0 ]; then · line 39

Quoted 6 lines
support/download/git +2 -2
 GIT_DIR="${basename}" \
-${GIT} archive --prefix="${basename}/" -o "${output}" --format=tar "${cset}"
+${GIT} archive --prefix="${basename}/" -o "${output}.tmp" --format=tar "${cset}"
 
-gzip "${output}"
+gzip <"${output}.tmp" >"${output}"

Don't we then end up with a bunch of foo.tmp files?

Nope, because the wrapper scripts first 'cd' into a temp dir, that ets removed at the end of the wrapper.

See:
http://git.buildroot.org/buildroot/tree/support/download/wrapper#n42
and:
http://git.buildroot.org/buildroot/tree/support/download/wrapper#n91

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#5 lore raw

PATCH v3 2/2 support/download: fix the git helper output file format

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

Hi,

Quoted 5 lines
Quoted 4 lines

support/download/git if [ ${git_done} -eq 0 ]; then · line 39

Quoted 2 lines
support/download/git +1 -1
-gzip "${output}"
+gzip <"${output}.tmp" >"${output}"

Don't we then end up with a bunch of foo.tmp files?

Quoted 7 lines

Nope, because the wrapper scripts first 'cd' into a temp dir, that ets removed at the end of the wrapper.

See:
http://git.buildroot.org/buildroot/tree/support/download/wrapper#n42
and:
http://git.buildroot.org/buildroot/tree/support/download/wrapper#n91

Ahh yes, good - Thanks.

Signature · 1 line
Bye, Peter Korsgaard
#6 lore raw

v3 cover Fix git download helper (branch yem/download-fixes)

In reply to Yann E. MORIN

Dear Yann E. MORIN,

Quoted 5 lines · Yann E. MORIN

Peter Seiderer (1):
support/download: fix the git helper

Yann E. MORIN (1):
support/download: fix the git helper output file format

Thanks, both patches have been applied.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com