Skip to contentSkip to conversation

gitorious helper

RFC PATCH New Alexandre Belloni 3 participants 9 messages Dec 25, 2013 – Apr 29, 2014

lore
Share & tools
lore

Alexandre Belloni

author
#0 lore raw

cover gitorious helper

The following patches introduce a gitroious helper to download tarballs for gitorious. I currently see two shortcomings:
- the tarball URL is https://gitorious.org/<project>/<repo>/archive/<version>.tar.gz
so the downloaded tarball doesn't have any name appart from <version>.tar.gz
- It needs two separate functions for target and host packages.

Alexandre Belloni (2):
infra: introduce gitorious helper function
packages: use gitorious helper when applicable

 boot/xloader/xloader.mk                 |  2 +-
 docs/manual/adding-packages-generic.txt | 10 +++++-----
 docs/manual/adding-packages-tips.txt    | 18 ++++++++++++++++++
 package/can-utils/can-utils.mk          |  2 +-
 package/lpc3250loader/lpc3250loader.mk  |  2 +-
 package/pkg-download.mk                 |  8 ++++++++
 package/ti-uim/ti-uim.mk                |  2 +-
 7 files changed, 35 insertions(+), 9 deletions(-)
Signature · 1 line
1.8.3.2
author
#1 lore raw

PATCH 1/2 infra: introduce gitorious helper function

Introduce a gitorious helper function to make it easy to retrieve gitorious packages without using git whose port is often blocked on enterprise networks.

usage: $(eval $(call gitorious,project,repository))

Signed-off-by Alexandre Belloni <[email protected]>
---
 docs/manual/adding-packages-generic.txt | 10 +++++-----
 docs/manual/adding-packages-tips.txt    | 18 ++++++++++++++++++
 package/pkg-download.mk                 |  8 ++++++++
 3 files changed, 31 insertions(+), 5 deletions(-)
docs/manual/adding-packages-generic.txt +5 -5
index 14971b502ba9..d92e49bdcdc3 100644
@@ -231,11 +231,11 @@ information is (assuming the package name is +libfoo+) :
   types for retrieving package tarballs. Git, Subversion, Mercurial,
   and Bazaar are supported URL types for retrieving packages directly
   from source code management systems. There is a helper function to make
-  it easier to download source tarballs from github (refer to
-  xref:github-download-url[] for details). A filesystem path may be used
-  to specify either a tarball or a directory containing the package
-  source code. See +LIBFOO_SITE_METHOD+ below for more details on how
-  retrieval works. +
+  it easier to download source tarballs from github or gitorious (refer
+  to xref:github-download-url[] and xref:gitorious-download-url[] for
+  details). A filesystem path may be used to specify either a tarball or
+  a directory containing the package source code. See
+  +LIBFOO_SITE_METHOD+ below for more details on how retrieval works. +
   Note that SCP URLs should be of the form
   +scp://[user@]host:filepath+, and that filepath is relative to the
   user's home directory, so you may want to prepend the path with a
docs/manual/adding-packages-tips.txt +18 -0
index 73c25be015ab..b51105eae9f2 100644
@@ -55,3 +55,21 @@ FOO_SITE = $(call github,<user>,<package>,$(FOO_VERSION))
   Buildroot (e.g.: +foo-f6fb6654af62045239caed5950bc6c7971965e60.tar.gz+),
   so it is not necessary to specify it in the +.mk+ file.
 - When using a commit ID as version, you should use the full 40 hex characters.
+
+
+[[gitorious-download-url]]
+How to add a package from gitorious
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+As with github, you can use the 'gitorious' helper function to download
+tarballs from gitorious:
+
+------------------------
+FOO_VERSION = v1.0 # tag or full commit ID
+$(eval $(call gitorious,<project>,<repo>))
+------------------------
+
+.Notes
+- For host packages, use host-gitorious
+- The FOO_VERSION can either be a tag or a commit ID.
+- When using a commit ID as version, you should use the full 40 hex characters.
package/pkg-download.mk +8 -0
index 758b211097e8..531df49b60f2 100644
@@ -58,6 +58,14 @@ domainseparator=$(if $(1),$(1),/)
 # github(user,package,version): returns site of github repository
 github = https://github.com/$(1)/$(2)/tarball/$(3)
 
+# gitorious(project,repo): sets <foo>_SITE and <foo>_SOURCE for a gitorious repository 
+define inner-gitorious
+$(1)_SITE = https://gitorious.org/$(2)/$(3)/archive
+$(1)_SOURCE = $($(1)_VERSION).tar.gz
+endef
+gitorious = $(call inner-gitorious,$(call UPPERCASE,$(call pkgname)),$(1),$(2))
+host-gitorious = $(call inner-gitorious,$(call UPPERCASE,host-$(call pkgname)),$(1),$(2))
+
 ################################################################################
 # The DOWNLOAD_* helpers are in charge of getting a working copy
 # of the source repository for their corresponding SCM,
Signature · 1 line
1.8.3.2
author
#2 lore raw

PATCH 2/2 packages: use gitorious helper when applicable

In reply to Alexandre Belloni
Signed-off-by Alexandre Belloni <[email protected]>
---
 boot/xloader/xloader.mk                | 2 +-
 package/can-utils/can-utils.mk         | 2 +-
 package/lpc3250loader/lpc3250loader.mk | 2 +-
 package/ti-uim/ti-uim.mk               | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
boot/xloader/xloader.mk +1 -1
index 00c63b1efb91..0ec11de1d6af 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 XLOADER_VERSION    = 6f3a26101303051e0f91b6213735b68ce804e94e
-XLOADER_SITE       = git://gitorious.org/x-loader/x-loader.git
+$(eval $(call gitorious,x-loader,x-loader))
 XLOADER_BOARD_NAME = $(call qstrip,$(BR2_TARGET_XLOADER_BOARDNAME))
 
 XLOADER_LICENSE = GPLv2+
package/can-utils/can-utils.mk +1 -1
index 743fe7ce0b2b..d5f6673dd25f 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 CAN_UTILS_VERSION = 16c970d40e6c19dde705bad4751bab1a3a4f3a0d
-CAN_UTILS_SITE = git://gitorious.org/linux-can/can-utils.git
+$(eval $(call gitorious,linux-can,can-utils))
 CAN_UTILS_AUTORECONF = YES
 
 $(eval $(autotools-package))
package/lpc3250loader/lpc3250loader.mk +1 -1
index 42beb7bc84b4..606e446c20bb 100644
@@ -4,8 +4,8 @@
 #
 ################################################################################
 
-HOST_LPC3250LOADER_SITE = git://gitorious.org/lpc3250loader/lpc3250loader.git
 HOST_LPC3250LOADER_VERSION = 1.0
+$(eval $(call host-gitorious,lpc3250loader,lpc3250loader))
 
 define HOST_LPC3250LOADER_INSTALL_CMDS
 	$(INSTALL) -m 0755 -D $(@D)/LPC3250loader.py \
package/ti-uim/ti-uim.mk +1 -1
index b49818004989..034388a08d1e 100644
@@ -5,7 +5,7 @@
 ################################################################################
 
 TI_UIM_VERSION = c73894456df5def97111cb33d2106b684b8b7959
-TI_UIM_SITE = git://gitorious.org/uim/uim.git
+$(eval $(call gitorious,uim,uim))
 TI_UIM_LICENSE = GPLv2+
 
 define TI_UIM_BUILD_CMDS
Signature · 1 line
1.8.3.2
#3 lore raw

cover gitorious helper

In reply to Alexandre Belloni

Hi Alexandre,

Quoted 9 lines · Alexandre Belloni

The following patches introduce a gitroious helper to download tarballs for gitorious. I currently see two shortcomings:
- the tarball URL is https://gitorious.org/<project>/<repo>/archive/<version>.tar.gz
so the downloaded tarball doesn't have any name appart from <version>.tar.gz
- It needs two separate functions for target and host packages.

Alexandre Belloni (2):
infra: introduce gitorious helper function
packages: use gitorious helper when applicable

Just a quick note: I'm going to review this (hopefully soon)...

Best regards,
Thomas

#4 lore raw

cover gitorious helper

In reply to Alexandre Belloni

Hi Alexandre,

Quoted 4 lines · Alexandre Belloni

The following patches introduce a gitroious helper to download tarballs for gitorious. I currently see two shortcomings:
- the tarball URL is https://gitorious.org/<project>/<repo>/archive/<version>.tar.gz
so the downloaded tarball doesn't have any name appart from <version>.tar.gz

This is indeed a pity. Not only because it makes the helper more complex, but also because the dl/ directory will contain a 'magic' file called <version>.tar.gz; the package it belongs to is not clear to the user.

I propose two alternatives:
1. discuss the situation with the Gitorious people, requesting them to provide a downloadable tarball with the name <package>-<version>.tar.gz (or whichever extension), like the rest of the software world does.

2. accept the fact that the URL cannot be changed, but modify the wget download helper so that it can rename the file after download (clarifying the list of files in dl/)

In the first alternative, the gitorious helper can be a one-liner just as for github, and the .mk files only need to specify FOO_SITE. In the second alternative, we need to set two variables FOO_SITE and FOO_SOURCE. In your current patchset, you have done this with one helper that sets both variables. As a result, the usage compared to the github helper is fundamentally different.
For github:
FOO_SITE = $(call github,username,package,$(FOO_VERSION)) while for gitorious:
$(call gitorious,username,package,$(FOO_VERSION)

I don't like this difference, and suggest two alternatives:
A. split the helper in two, so that:
FOO_SITE = $(call gitorious-site,username,package,$(FOO_VERSION)) FOO_SOURCE = $(call gitorious-version,username,package,$(FOO_VERSION))

This is then equivalent to github except the new FOO_SOURCE, which is simply not present for github (at the moment).

B. Keep the current gitorious helper, but apply the same strategy to the github helper, so that its usage also becomes
$(call github,username,package,$(FOO_VERSION))

Quoted 2 lines

- It needs two separate functions for target and host packages.

Can you clarify this?
In package/pkg-generic.mk, if HOST_FOO_VERSION/SOURCE is not defined and we try to build a host package, its value is taken from FOO_VERSION/SOURCE. Then why do we need to have a separate helper?

Thanks,
Thomas

#5 lore raw

cover gitorious helper

Dear Thomas De Schampheleire,

Quoted 4 lines · Thomas De Schampheleire

1. discuss the situation with the Gitorious people, requesting them to provide a downloadable tarball with the name <package>-<version>.tar.gz (or whichever extension), like the rest of the software world does.

This would be an interesting thing to do, indeed.

Quoted 15 lines

2. accept the fact that the URL cannot be changed, but modify the wget download helper so that it can rename the file after download (clarifying the list of files in dl/)

In the first alternative, the gitorious helper can be a one-liner just as for github, and the .mk files only need to specify FOO_SITE. In the second alternative, we need to set two variables FOO_SITE and FOO_SOURCE. In your current patchset, you have done this with one helper that sets both variables. As a result, the usage compared to the github helper is fundamentally different.
For github:
FOO_SITE = $(call github,username,package,$(FOO_VERSION)) while for gitorious:
$(call gitorious,username,package,$(FOO_VERSION)

Quoted 4 lines

I don't like this difference, and suggest two alternatives:
A. split the helper in two, so that:
FOO_SITE = $(call gitorious-site,username,package,$(FOO_VERSION)) FOO_SOURCE = $(call gitorious-version,username,package,$(FOO_VERSION))

I very much prefer this solution. I don't really like the helper that magically defines FOO_SITE and FOO_VERSION.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
author
#6 lore raw

cover gitorious helper

Hi,

Quoted 15 lines · Thomas Petazzoni
Quoted 15 lines

2. accept the fact that the URL cannot be changed, but modify the wget download helper so that it can rename the file after download (clarifying the list of files in dl/)

In the first alternative, the gitorious helper can be a one-liner just as for github, and the .mk files only need to specify FOO_SITE. In the second alternative, we need to set two variables FOO_SITE and FOO_SOURCE. In your current patchset, you have done this with one helper that sets both variables. As a result, the usage compared to the github helper is fundamentally different.
For github:
FOO_SITE = $(call github,username,package,$(FOO_VERSION)) while for gitorious:
$(call gitorious,username,package,$(FOO_VERSION)

Quoted 8 lines
Quoted 4 lines

I don't like this difference, and suggest two alternatives:
A. split the helper in two, so that:
FOO_SITE = $(call gitorious-site,username,package,$(FOO_VERSION)) FOO_SOURCE = $(call gitorious-version,username,package,$(FOO_VERSION))

I very much prefer this solution. I don't really like the helper that magically defines FOO_SITE and FOO_VERSION.

After discussing quickly with Thomas, we feel that we can drop it as this would not really help with readability.

Regards,

Signature · 3 lines
Alexandre Belloni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#7 lore raw

cover gitorious helper

Alexandre Belloni <[email protected]> schreef:

Quoted 23 lines

Hi,

Quoted 20 lines · Thomas Petazzoni
Quoted 19 lines

2. accept the fact that the URL cannot be changed, but modify the wget download helper so that it can rename the file after download (clarifying the list of files in dl/)

In the first alternative, the gitorious helper can be a one-liner just as for github, and the .mk files only need to specify FOO_SITE. In the second alternative, we need to set two variables FOO_SITE and FOO_SOURCE. In your current patchset, you have done this with one helper that sets both variables. As a result, the usage compared to the github helper is fundamentally different.
For github:
FOO_SITE = $(call github,username,package,$(FOO_VERSION)) while for gitorious:
$(call gitorious,username,package,$(FOO_VERSION)

I don't like this difference, and suggest two alternatives:
A. split the helper in two, so that:
FOO_SITE = $(call gitorious-site,username,package,$(FOO_VERSION)) FOO_SOURCE = $(call gitorious-version,username,package,$(FOO_VERSION))

Quoted 6 lines
Quoted 3 lines

I very much prefer this solution. I don't really like the helper that magically defines FOO_SITE and FOO_VERSION.

After discussing quickly with Thomas, we feel that we can drop it as this would not really help with readability.

Readability wasn't the only reason such helpers were introduced. For github, the download urls changed a few times, and having it coded in only one place made changing this much easier. So if that us the only reason I prefer to keep this patch. I'm willing to adopt it by the way...

Best regards,
Thomas

author
#8 lore raw

cover gitorious helper

Quoted 7 lines · Thomas De Schampheleire

Alexandre Belloni <[email protected]> schreef:

Quoted 2 lines

After discussing quickly with Thomas, we feel that we can drop it as this would not really help with readability.

Readability wasn't the only reason such helpers were introduced. For github, the download urls changed a few times, and having it coded in only one place made changing this much easier. So if that us the only reason I prefer to keep this patch. I'm willing to adopt it by the way...

Feel free to adopt it but I'm not sure we can have something easy to use and elegant.

Signature · 3 lines
Alexandre Belloni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com