Skip to contentSkip to conversation

support/download: reintroduce 'source-check' target

PATCH Superseded Thomas De Schampheleire 2 participants 11 messages Feb 15, 2019 – Feb 16, 2019 Kbuild

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/8 support/download: reintroduce 'source-check' target

From: Thomas De Schampheleire <[email protected]>

Commit bf28a165d992681a85b43a886005e669b3cb579b removed the source-check target to allow easier refactoring of the download infrastructure. It was partly based on the fact that no-one really used this target.

However, it turns out that there _are_ good uses for it. In a work environment where many people are submitting changes to a Buildroot repository, one cannot always blindly trust every change. A typical case of human error is when bumping the version of a package but forgetting to upload the tarball to the internal BR2_PRIMARY_SITE or (in case of a repository) pushing the new changesets to the repository.
If a user cannot directly push to the Buildroot repository but needs to queue their change via an automatic validation system, that validation system can use 'make source-check' on the relevant defconfigs to protect against such errors. A full download would also be possible but takes longer and unnecessarily uses internal network bandwidth.

With that use case in mind, this commit reintroduces the 'source-check' target, but embedded in the current situation with a dl-wrapper. The changes to the wrapper are minimal (not considering additional indentation). A new option '-C' means 'check only' and will be passed to the download backends. If the backend supports the option, no download will happen. If it does not, then the backend will actually perform a download as a means of checking that the source exists (a form of graceful degradation). In neither case, though, hash checking is performed (as the standard case is without download thus without file to calculate hashes on).

Subsequent commits will actually implement -C in the backends.

---
 Makefile                    |   7 +++
 package/pkg-download.mk     |   6 +++
 package/pkg-generic.mk      |  14 ++++-
 support/download/dl-wrapper | 100 ++++++++++++++++++++----------------
 4 files changed, 82 insertions(+), 45 deletions(-)

v4: no changes

v3: (feedback Arnout Vandecappelle)
- remove tmpd at the end of dl-wrapper
- remove duplication between DOWNLOAD and SOURCE_CHECK macros

v2:
- add a trailing dot on all usage texts like for other options
- split off the reintroduction of BR2_SSH to a separate patch
- fix the source-check for scp: it seems there were some broken things in v1
- drop incomplete implementations for git/svn/bzr/cvs so it falls back to actual download.

Makefile +7 -0
index 0d2659c46e..9ae047bfc6 100644
@@ -131,6 +131,7 @@ noconfig_targets := menuconfig nconfig gconfig xconfig config oldconfig randconf
 # (default target is to build), or when MAKECMDGOALS contains
 # something else than one of the nobuild_targets.
 nobuild_targets := source %-source \
+	source-check %-source-check %-all-source-check \
 	legal-info %-legal-info external-deps _external-deps \
 	clean distclean help show-targets graph-depends \
 	%-graph-depends %-show-depends %-show-version \
@@ -818,6 +819,9 @@ target-post-image: $(TARGETS_ROOTFS) target-finalize staging-finalize
 .PHONY: source
 source: $(foreach p,$(PACKAGES),$(p)-all-source)
 
+.PHONY: source-check
+source-check: $(foreach p,$(PACKAGES),$(p)-all-source-check)
+
 .PHONY: _external-deps external-deps
 _external-deps: $(foreach p,$(PACKAGES),$(p)-all-external-deps)
 external-deps:
@@ -1103,6 +1107,8 @@ help:
 	@echo '  <pkg>-dirclean         - Remove <pkg> build directory'
 	@echo '  <pkg>-reconfigure      - Restart the build from the configure step'
 	@echo '  <pkg>-rebuild          - Restart the build from the build step'
+	@echo '  <pkg>-source-check     - Check package for valid download URLs'
+	@echo '  <pkg>-all-source-check - Check package and its dependencies for valid download URLs'
 	$(foreach p,$(HELP_PACKAGES), \
 		@echo $(sep) \
 		@echo '$($(p)_NAME):' $(sep) \
@@ -1122,6 +1128,7 @@ help:
 	@echo
 	@echo 'Miscellaneous:'
 	@echo '  source                 - download all sources needed for offline-build'
+	@echo '  source-check           - check selected packages for valid download URLs'
 	@echo '  external-deps          - list external packages used'
 	@echo '  legal-info             - generate info about license compliance'
 	@echo '  printvars              - dump all the internal variables'
package/pkg-download.mk +6 -0
index 7cd87c38ff..5384fc2af4 100644
@@ -70,6 +70,7 @@ export BR_NO_CHECK_HASH_FOR =
 # 3) BR2_BACKUP_SITE if enabled, unless BR2_PRIMARY_SITE_ONLY is set
 #
 # Argument 1 is the source location
+# Argument 2 (optionally) provides extra arguments to pass to DL_WRAPPER
 #
 ################################################################################
 
@@ -92,6 +93,7 @@ endif
 define DOWNLOAD
 	$(Q)mkdir -p $($(PKG)_DL_DIR)
 	$(Q)$(EXTRA_ENV) $(FLOCK) $(DL_WRAPPER) \
+		$(2) \
 		-c '$($(PKG)_DL_VERSION)' \
 		-d '$($(PKG)_DL_DIR)' \
 		-D '$(DL_DIR)' \
@@ -106,3 +108,7 @@ define DOWNLOAD
 		-- \
 		$($(PKG)_DL_OPTS)
 endef
+
+define SOURCE_CHECK
+	$(call DOWNLOAD,$(1),-C)
+endef
package/pkg-generic.mk +13 -1
index 6168b40e89..bc57956867 100644
@@ -788,6 +788,10 @@ $(1)-legal-source:	$$($(2)_TARGET_ACTUAL_SOURCE)
 endif # actual sources != sources
 endif # actual sources != ""
 
+$(1)-source-check: PKG=$(2)
+$(1)-source-check:
+	$$(foreach p,$$($(2)_ALL_DOWNLOADS),$$(call SOURCE_CHECK,$$(p))$$(sep))
+
 $(1)-external-deps:
 	@for p in $$($(2)_SOURCE) $$($(2)_PATCH) $$($(2)_EXTRA_DOWNLOADS) ; do \
 		echo `basename $$$$p` ; \
@@ -812,6 +816,9 @@ $(1)-rsync:		$$($(2)_TARGET_RSYNC)
 $(1)-source:
 $(1)-legal-source:
 
+$(1)-source-check:
+	test -d $$($(2)_OVERRIDE_SRCDIR)
+
 $(1)-external-deps:
 	@echo "file://$$($(2)_OVERRIDE_SRCDIR)"
 endif
@@ -846,6 +853,9 @@ $(1)-graph-rdepends: graph-depends-requirements
 $(1)-all-source:	$(1)-source
 $(1)-all-source:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-source)
 
+$(1)-all-source-check:	$(1)-source-check
+$(1)-all-source-check:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-source-check)
+
 $(1)-all-external-deps:	$(1)-external-deps
 $(1)-all-external-deps:	$$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-external-deps)
 
@@ -1044,6 +1054,7 @@ DL_TOOLS_DEPENDENCIES += $$(call extractor-dependency,$$($(2)_SOURCE))
 	$(1)-all-external-deps \
 	$(1)-all-legal-info \
 	$(1)-all-source \
+	$(1)-all-source-check \
 	$(1)-build \
 	$(1)-clean-for-rebuild \
 	$(1)-clean-for-reconfigure \
@@ -1068,7 +1079,8 @@ DL_TOOLS_DEPENDENCIES += $$(call extractor-dependency,$$($(2)_SOURCE))
 	$(1)-rsync \
 	$(1)-show-depends \
 	$(1)-show-version \
-	$(1)-source
+	$(1)-source \
+	$(1)-source-check
 
 ifneq ($$($(2)_SOURCE),)
 ifeq ($$($(2)_SITE),)
support/download/dl-wrapper +56 -44

142 lines not on this page · raw message

Signature · 1 line
2.19.2
#1 lore raw

PATCH v4 2/8 support/download/hg: implement source-check

From: Thomas De Schampheleire <[email protected]>

Note: the implementation is different (better) than what used to be in Buildroot before source-check was removed.

The original implementation:

    hg incoming --force -l1 <URL>

would only verify that the repository exists, not that the requested revision is present.

An already better implementation is:

    hg incoming --force -l1 -r <revision> <URL>

but compared to the next solution it has a large resource consumption on the local machine. In the background, the full repository is first downloaded.

The implemented solution is:
hg identify -r <revision> <URL>
which operates directly on the remote repository.

---
 support/download/hg | 12 ++++++++++++
 1 file changed, 12 insertions(+)

v4: (feedback Yann E. Morin)
- use true/false as values to 'checkonly'
- always check revision, even if no source-check
- replace incorrect 'exit $?' by explicit 'exit 0'

v3: no changes

support/download/hg +12 -0
index efb515fca5..8f7d1a5a90 100755
@@ -7,6 +7,7 @@ set -e
 #
 # Options:
 #   -q          Be quiet.
+#   -C          Only check that the changeset exists in the remote repository.
 #   -o FILE     Generate archive in FILE.
 #   -u URI      Clone from repository at URI.
 #   -c CSET     Use changeset (or revision) CSET.
@@ -16,9 +17,11 @@ set -e
 #   HG       : the hg command to call
 
 verbose=
+checkonly=false
 while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do
     case "${OPT}" in
     q)  verbose=-q;;
+    C)  checkonly=true;;
     o)  output="${OPTARG}";;
     u)  uri="${OPTARG}";;
     c)  cset="${OPTARG}";;
@@ -36,6 +39,15 @@ _hg() {
     eval ${HG} "${@}"
 }
 
+if ! _hg identify ${verbose} "${@}" --rev "'${cset}'" "'${uri}'" > /dev/null; then
+    printf "Commit '%s' does not exist in this repository\n." "${cset}"
+    exit 1
+fi
+
+if ${checkonly}; then
+    exit 0
+fi
+
 _hg clone ${verbose} "${@}" --noupdate "'${uri}'" "'${basename}'"
 
 _hg archive ${verbose} --repository "'${basename}'" --type tgz \
Signature · 1 line
2.19.2
#2 lore raw

PATCH v4 3/8 support/download/wget: implement source-check

In reply to Thomas De Schampheleire

From: Thomas De Schampheleire <[email protected]>

---
 support/download/wget | 8 ++++++++
 1 file changed, 8 insertions(+)

v4: (feedback Yann E. Morin)
- use true/false as values to 'checkonly'
- replace incorrect 'exit $?' by explicit 'exit 0'

v3: no changes

support/download/wget +8 -0
index c69e6071aa..277eab6660 100755
@@ -7,6 +7,7 @@ set -e
 #
 # Options:
 #   -q          Be quiet.
+#   -C          Only check that the file exists remotely.
 #   -o FILE     Save into file FILE.
 #   -f FILENAME The filename of the tarball to get at URL
 #   -u URL      Download file at URL.
@@ -16,9 +17,11 @@ set -e
 #   WGET     : the wget command to call
 
 verbose=
+checkonly=false
 while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do
     case "${OPT}" in
     q)  verbose=-q;;
+    C)  checkonly=true;;
     o)  output="${OPTARG}";;
     f)  filename="${OPTARG}";;
     u)  url="${OPTARG}";;
@@ -40,4 +43,9 @@ _wget() {
 # mirror
 [ -n "${encode}" ] && filename=${filename//\?/%3F}
 
+if ${checkonly}; then
+    _wget --spider ${verbose} "${@}" "'${url}/${filename}'"
+    exit 0
+fi
+
 _wget ${verbose} "${@}" -O "'${output}'" "'${url}/${filename}'"
Signature · 1 line
2.19.2
#3 lore raw

PATCH v4 4/8 support/download/file: implement source-check

In reply to Thomas De Schampheleire

From: Thomas De Schampheleire <[email protected]>

---
 support/download/file | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

v4:
- use true/false as values to 'checkonly' (Yann E. Morin)
- replace incorrect 'exit $?' by explicit 'exit 0' (Yann E. Morin)
- replace 'test -e' by 'test -f' which is more specific

v3: no changes

support/download/file +12 -1
index e52fcf2c8c..059a8dde4c 100755
@@ -7,6 +7,7 @@ set -e
 #
 # Options:
 #   -q          Be quiet.
+#   -C          Only check that the source file exists.
 #   -o FILE     Copy to file FILE.
 #   -f FILE     Copy from basename file FILE.
 #   -u DIR      Copy from FILE in DIR.
@@ -20,9 +21,11 @@ set -e
 # Make 'cp' verbose by default, so it behaves a bit like the others.
 verbose=-v
 
+checkonly=false
 while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do
     case "${OPT}" in
     q)  verbose=;;
+    C)  checkonly=true;;
     o)  output="${OPTARG}";;
     f)  file="${OPTARG}";;
     u)  dir="${OPTARG}";;
@@ -39,4 +42,12 @@ _localfiles() {
     eval ${LOCALFILES} "${@}"
 }
 
-_localfiles ${verbose} "'${dir##file://}/${file}'" "'${output}'"
+# Remove any scheme prefix
+dir="${dir##file://}"
+
+if ${checkonly}; then
+    test -f "'${dir}/${file}'"
+    exit 0
+fi
+
+_localfiles ${verbose} "'${dir}/${file}'" "'${output}'"
Signature · 1 line
2.19.2
#4 lore raw

PATCH v4 5/8 Config.in: reintroduce BR2_SSH

In reply to Thomas De Schampheleire

From: Thomas De Schampheleire <[email protected]>

The BR2_SSH command was removed in commit db9473bf6cd7bd12aa1f9faad0a917c973c33827 ("core/download: drop the SSH command") but will be needed again to support 'source-check' for the scp download backend.

---
 Config.in               | 4 ++++
 package/pkg-download.mk | 1 +
 2 files changed, 5 insertions(+)

v4: no changes
v3: no changes

Config.in +4 -0
index d58d8dc04a..91d56907fe 100644
@@ -136,6 +136,10 @@ config BR2_SCP
 	string "Secure copy (scp) command"
 	default "scp"
 
+config BR2_SSH
+	string "Secure shell (ssh) command"
+	default "ssh"
+
 config BR2_HG
 	string "Mercurial (hg) command"
 	default "hg"
package/pkg-download.mk +1 -0
index 5384fc2af4..17d1ca99ee 100644
@@ -15,6 +15,7 @@ export BZR := $(call qstrip,$(BR2_BZR))
 export GIT := $(call qstrip,$(BR2_GIT))
 export HG := $(call qstrip,$(BR2_HG))
 export SCP := $(call qstrip,$(BR2_SCP))
+export SSH := $(call qstrip,$(BR2_SSH))
 export LOCALFILES := $(call qstrip,$(BR2_LOCALFILES))
 
 DL_WRAPPER = support/download/dl-wrapper
Signature · 1 line
2.19.2
#5 lore raw

PATCH v4 5/8 Config.in: reintroduce BR2_SSH

Thomas, All,

On 2019-02-15 22:08 +0100, Thomas De Schampheleire spake thusly:

Quoted 6 lines

From: Thomas De Schampheleire <[email protected]>

The BR2_SSH command was removed in commit db9473bf6cd7bd12aa1f9faad0a917c973c33827 ("core/download: drop the SSH command") but will be needed again to support 'source-check' for the scp download backend.

Rather than do a new commit, why did you not use "git revert"?

Quoted 29 lines
---
 Config.in               | 4 ++++
 package/pkg-download.mk | 1 +
 2 files changed, 5 insertions(+)

v4: no changes
v3: no changes

Config.in +4 -0
index d58d8dc04a..91d56907fe 100644
@@ -136,6 +136,10 @@ config BR2_SCP
 	string "Secure copy (scp) command"
 	default "scp"
 
+config BR2_SSH
+	string "Secure shell (ssh) command"
+	default "ssh"
+
 config BR2_HG
 	string "Mercurial (hg) command"
 	default "hg"
package/pkg-download.mk +0 -0
index 5384fc2af4..17d1ca99ee 100644

package/pkg-download.mk export BZR := $(call qstrip,$(BR2_BZR)) · line 18

Quoted 4 lines
package/pkg-download.mk +1 -0
@@ -15,6 +15,7 @@ export BZR := $(call qstrip,$(BR2_BZR))
 export GIT := $(call qstrip,$(BR2_GIT))
 export HG := $(call qstrip,$(BR2_HG))
 export SCP := $(call qstrip,$(BR2_SCP))
+export SSH := $(call qstrip,$(BR2_SSH))

To be noted: the commit you are (manually) reverting did not export SSH, but now you do need to do so, because it will now be used from one of a download backends, when the original use of SSH was done in Makefile code.

Regards,
Yann E. MORIN.

Quoted 6 lines
package/pkg-download.mk +0 -0
 export LOCALFILES := $(call qstrip,$(BR2_LOCALFILES))
 
 DL_WRAPPER = support/download/dl-wrapper
Signature · 1 line
2.19.2
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#6 lore raw

PATCH v4 5/8 Config.in: reintroduce BR2_SSH

El s?b., 16 feb. 2019 a las 13:34, Yann E. MORIN
(<[email protected]>) escribi?:

Quoted 4 lines

Thomas, All,

On 2019-02-15 22:08 +0100, Thomas De Schampheleire spake thusly:

Quoted 8 lines
Quoted 6 lines

From: Thomas De Schampheleire <[email protected]>

The BR2_SSH command was removed in commit db9473bf6cd7bd12aa1f9faad0a917c973c33827 ("core/download: drop the SSH command") but will be needed again to support 'source-check' for the scp download backend.

Rather than do a new commit, why did you not use "git revert"?

How does this make a difference when sending patches via email?

The background is that originally this change was put together with the scp source-check implementation, which was based on the implementation I had in my work repository, where BR2_SSH was not yet removed. Only later I split this patch apart and didn't think of using 'git revert'.

Quoted 31 lines
Quoted 31 lines
---
 Config.in               | 4 ++++
 package/pkg-download.mk | 1 +
 2 files changed, 5 insertions(+)

v4: no changes
v3: no changes

Config.in +4 -0
index d58d8dc04a..91d56907fe 100644
@@ -136,6 +136,10 @@ config BR2_SCP
      string "Secure copy (scp) command"
      default "scp"
 
+config BR2_SSH
+     string "Secure shell (ssh) command"
+     default "ssh"
+
 config BR2_HG
      string "Mercurial (hg) command"
      default "hg"
package/pkg-download.mk +0 -0
index 5384fc2af4..17d1ca99ee 100644
@@ -15,6 +15,7 @@ export BZR := $(call qstrip,$(BR2_BZR))
 export GIT := $(call qstrip,$(BR2_GIT))
Quoted 8 lines

package/pkg-download.mk export BZR := $(call qstrip,$(BR2_BZR)) · line 18

Quoted 3 lines
package/pkg-download.mk +1 -0
 export HG := $(call qstrip,$(BR2_HG))
 export SCP := $(call qstrip,$(BR2_SCP))
+export SSH := $(call qstrip,$(BR2_SSH))

To be noted: the commit you are (manually) reverting did not export SSH, but now you do need to do so, because it will now be used from one of a download backends, when the original use of SSH was done in Makefile code.

Yes indeed, that is true.

Best regards,
Thomas

#7 lore raw

PATCH v4 5/8 Config.in: reintroduce BR2_SSH

Thomas, All,

On 2019-02-16 22:23 +0100, Thomas De Schampheleire spake thusly:

Quoted 5 lines

El s?b., 16 feb. 2019 a las 13:34, Yann E. MORIN
(<[email protected]>) escribi?:

Quoted 3 lines

On 2019-02-15 22:08 +0100, Thomas De Schampheleire spake thusly:

Quoted 2 lines

From: Thomas De Schampheleire <[email protected]>

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

The BR2_SSH command was removed in commit db9473bf6cd7bd12aa1f9faad0a917c973c33827 ("core/download: drop the SSH command") but will be needed again to support 'source-check' for the scp download backend.

Rather than do a new commit, why did you not use "git revert"?

How does this make a difference when sending patches via email?

git-revert prepares the git commit with appropriate information about the revert:

Revert "core/download: drop the SSH command"

This reverts commit db9473bf6cd7bd12aa1f9faad0a917c973c33827.

And then you can amend the commit log to explain why it is reverted.

Quoted 5 lines

The background is that originally this change was put together with the scp source-check implementation, which was based on the implementation I had in my work repository, where BR2_SSH was not yet removed. Only later I split this patch apart and didn't think of using 'git revert'.

OK, that explains it.

Quoted 6 lines
Quoted 5 lines
Quoted 1 line

+export SSH := $(call qstrip,$(BR2_SSH))

To be noted: the commit you are (manually) reverting did not export SSH, but now you do need to do so, because it will now be used from one of a download backends, when the original use of SSH was done in Makefile code.

Yes indeed, that is true.

And hence, as you are not reinstating the code exactly as it was prior to the "reverted" commit, that should have been part of the commit log.

Anyway:

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#8 lore raw

PATCH v4 6/8 support/download/scp: implement source-check

In reply to Thomas De Schampheleire

From: Thomas De Schampheleire <[email protected]>

We use 'test -f' rather than 'ls' because 'test' is a mandatory POSIX utility while 'ls' is not.

---
 support/download/scp | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

v4: (feedback Yann E. Morin)
- use true/false as values to 'checkonly'
- replace incorrect 'exit $?' by explicit 'exit 0'
- use 'test -f' instead of 'ls' which is more certain to be present (POSIX mandatory utility)

v3: no changes

support/download/scp +15 -0
index 80cf495c4e..52ccf06d6e 100755
@@ -7,17 +7,21 @@ set -e
 #
 # Options:
 #   -q          Be quiet.
+#   -C          Only check that the file exists remotely.
 #   -o FILE     Copy to local file FILE.
 #   -f FILE     Copy from remote file FILE.
 #   -u URI      Download file at URI.
 #
 # Environment:
 #   SCP       : the scp command to call
+#   SSH       : the ssh command to use for checkonly
 
 verbose=
+checkonly=false
 while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do
     case "${OPT}" in
     q)  verbose=-q;;
+    C)  checkonly=true;;
     o)  output="${OPTARG}";;
     f)  filename="${OPTARG}";;
     u)  uri="${OPTARG}";;
@@ -33,8 +37,19 @@ shift $((OPTIND-1)) # Get rid of our options
 _scp() {
     eval ${SCP} "${@}"
 }
+_ssh() {
+    eval ${SSH} "${@}"
+}
 
 # Remove any scheme prefix
 uri="${uri##scp://}"
 
+if ${checkonly}; then
+    # uri now looks like:  foo.example.org:some/directory
+    domain="${uri%%:*}"
+    path="${uri#*:}/${filename}"
+    _ssh ${verbose} "${@}" "'${domain}'" test -f "'${path}'" > /dev/null
+    exit 0
+fi
+
 _scp ${verbose} "${@}" "'${uri}/${filename}'" "'${output}'"
Signature · 1 line
2.19.2
#9 lore raw

PATCH v4 7/8 support/download/svn: implement source-check

In reply to Thomas De Schampheleire

From: Thomas De Schampheleire <[email protected]>

---
 support/download/svn | 12 ++++++++++++
 1 file changed, 12 insertions(+)

v4: new patch

support/download/svn +12 -0
index 542b25c0a2..93984b6c45 100755
@@ -7,6 +7,7 @@ set -e
 #
 # Options:
 #   -q          Be quiet.
+#   -C          Only check that the revision exists in the remote repository.
 #   -o FILE     Generate archive in FILE.
 #   -u URI      Checkout from repository at URI.
 #   -c REV      Use revision REV.
@@ -16,9 +17,11 @@ set -e
 #   SVN      : the svn command to call
 
 verbose=
+checkonly=false
 while getopts "${BR_BACKEND_DL_GETOPTS}" OPT; do
     case "${OPT}" in
     q)  verbose=-q;;
+    C)  checkonly=true;;
     o)  output="${OPTARG}";;
     u)  uri="${OPTARG}";;
     c)  rev="${OPTARG}";;
@@ -36,6 +39,15 @@ _svn() {
     eval ${SVN} "${@}"
 }
 
+if ! _svn ls ${verbose} -r "'${rev}'" "'${uri}'" > /dev/null; then
+    printf "Revision '%s' does not exist in this repository\n." "${rev}"
+    exit 1
+fi
+
+if ${checkonly}; then
+    exit 0
+fi
+
 _svn export ${verbose} "${@}" "'${uri}@${rev}'" "'${basename}'"
 
 tar czf "${output}" "${basename}"
Signature · 1 line
2.19.2
#10 lore raw

PATCH v4 8/8 support/download/{bzr, cvs, git}: highlight unimplemented source-check

In reply to Thomas De Schampheleire

From: Thomas De Schampheleire <[email protected]>

For bzr, cvs and git, there is no method of checking whether the revision exists remotely, without actually cloning the repository. Therefore, there is no actual implementation for '-C' (checkonly) for these download helpers. The script will fall back to the normal download logic, which would effectively fail if the revision does not exist.

For completeness, mention this unimplemented feature in the usage text.

---
 support/download/bzr | 2 ++
 support/download/cvs | 2 ++
 support/download/git | 2 ++
 3 files changed, 6 insertions(+)

v4: remove 'svn' which now has an actual implementation

v3: no changes

support/download/bzr +2 -0
index 5289a421cd..278d8778af 100755
@@ -7,6 +7,8 @@ set -e
 #
 # Options:
 #   -q          Be quiet
+#   -C          (unimplemented) Only check that the revision exists in the
+#               remote repository.
 #   -o FILE     Generate archive in FILE.
 #   -u URI      Clone from repository at URI.
 #   -c CSET     Use changeset (or revision) CSET.
support/download/cvs +2 -0
index 9d0dc3cb3a..d9a586293a 100755
@@ -7,6 +7,8 @@ set -e
 #
 # Options:
 #   -q          Be quiet
+#   -C          (unimplemented) Only check that the revision exists in the
+#               remote repository.
 #   -o FILE     Generate archive in FILE.
 #   -u URI      Checkout from repository at URI.
 #   -c REV      Use revision REV.
support/download/git +2 -0
index 17ca04eb98..e9c02d6712 100755
@@ -7,6 +7,8 @@ set -E
 #
 # Options:
 #   -q          Be quiet.
+#   -C          (unimplemented) Only check that the revision exists in the
+#               remote repository.
 #   -r          Clone and archive sub-modules.
 #   -o FILE     Generate archive in FILE.
 #   -u URI      Clone from repository at URI.
Signature · 1 line
2.19.2