Introduce alternative archive format

SITE_METHOD is always set in inner-generic-package. So, it is useless to test it in pkg-download.

Signed-off-by J?r?me Pouiller <[email protected]>

---
 package/pkg-download.mk | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

package/pkg-download.mk +1 -6

index c86b9ab..757bafe 100644
@@ -222,12 +222,7 @@ define DOWNLOAD_INNER
 		exit 1 ; \
 	fi ; \
 	if test -n "$(1)" ; then \
-		if test -z "$($(PKG)_SITE_METHOD)" ; then \
-			scheme="$(call geturischeme,$(1))" ; \
-		else \
-			scheme="$($(PKG)_SITE_METHOD)" ; \
-		fi ; \
-		case "$$scheme" in \
+		case "$($(PKG)_SITE_METHOD)" in \
 			git) $($(3)_GIT) && exit ;; \
 			svn) $($(3)_SVN) && exit ;; \
 			cvs) $($(3)_CVS) && exit ;; \

Signature · 6 lines

2.1.4

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

SITE_METHOD is always set in inner-generic-package. So, it is useless to test it in pkg-download.

Signed-off-by J?r?me Pouiller <[email protected]>

---
 package/pkg-download.mk | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

As suggested by Arnout [1], this series provide an alternative archive format. This new format contains a shallowed version of upstream repository. This format is a little bigger and a little longer to create but it allow a better workflow with upstream. I describe some good practice in patch 2.

Notice projects hosted by github don't yet benefit of this feature since I have not found any elegant way to do it :-(.

During my tests, I have noticed current shallow clone is mostly broken (at least with git < 2.5 [2]). Indeed, shallow clone only work with symbolic references (HEAD, a tag or a branch). However, we avoid use of symbolic references in VERSION.

J?r?me,

Quoted 6 lines · J?r?me Pouiller

As suggested by Arnout [1], this series provide an alternative archive format. This new format contains a shallowed version of upstream repository. This format is a little bigger and a little longer to create but it allow a better workflow with upstream. I describe some good practice in patch 2.

Quoted 7 lines

Notice projects hosted by github don't yet benefit of this feature since I have not found any elegant way to do it :-(.

During my tests, I have noticed current shallow clone is mostly broken (at least with git < 2.5 [2]). Indeed, shallow clone only work with symbolic references (HEAD, a tag or a branch). However, we avoid use of symbolic references in VERSION.

Thanks for this contribution.

Your justification in PATCH 2 is just "That simplify workflow with upstream". However, we already have the <pkg>_OVERRIDE_SRCDIR mechanism (and <pkg>_SITE_METHOD = local, which is the same) to specifically address this use case.

The idea with <pkg>_OVERRIDE_SRCDIR is that if you are actively developing on a software component, then it should not be Buildroot's responsibility to download/extract/patch it, but it should instead use a locally available source directory, which is managed completely separately from Buildroot. There you can do whatever Git, Subversion or Mercurial version control you want, Buildroot will simply rsync to the build directory.

I think doing development in the build directory, as encouraged by your patch, is a bad practice. The build directory is a temporary location, people should not be encouraged to work from there.

And I fail to see what your solution brings compared to using <pkg>_OVERRIDE_SRCDIR or <pkg>_SITE_METHOD = local. To me, the existing solutions are in fact more flexible and don't encourage the practice of hacking in the build directory.

Of course, if there is a specific workflow that you could describe that doesn't work with <pkg>_OVERRIDE_SRCDIR, then I'm definitely interested, and from this discussion we can decide whether improvements to OVERRIDE_SRCDIR are needed, or if a completely different solution is needed.

Quoted 6 lines

As suggested by Arnout [1], this series provide an alternative archive format. This new format contains a shallowed version of upstream repository. This format is a little bigger and a little longer to create but it allow a better workflow with upstream. I describe some good practice in patch 2.

Quoted 7 lines

Notice projects hosted by github don't yet benefit of this feature since I have not found any elegant way to do it :-(.

During my tests, I have noticed current shallow clone is mostly broken (at least with git < 2.5 [2]). Indeed, shallow clone only work with symbolic references (HEAD, a tag or a branch). However, we avoid use of symbolic references in VERSION.

Thanks for this contribution.

Your justification in PATCH 2 is just "That simplify workflow with upstream". However, we already have the <pkg>_OVERRIDE_SRCDIR mechanism (and <pkg>_SITE_METHOD = local, which is the same) to specifically address this use case.

The idea with <pkg>_OVERRIDE_SRCDIR is that if you are actively developing on a software component, then it should not be Buildroot's responsibility to download/extract/patch it, but it should instead use a locally available source directory, which is managed completely separately from Buildroot. There you can do whatever Git, Subversion or Mercurial version control you want, Buildroot will simply rsync to the build directory.

I think doing development in the build directory, as encouraged by your patch, is a bad practice. The build directory is a temporary location, people should not be encouraged to work from there.

And I fail to see what your solution brings compared to using <pkg>_OVERRIDE_SRCDIR or <pkg>_SITE_METHOD = local. To me, the existing solutions are in fact more flexible and don't encourage the practice of hacking in the build directory.

Of course, if there is a specific workflow that you could describe that doesn't work with <pkg>_OVERRIDE_SRCDIR, then I'm definitely interested, and from this discussion we can decide whether improvements to OVERRIDE_SRCDIR are needed, or if a completely different solution is needed.

J?r?me, Thomas, All,

On 2015-11-19 13:02 +0100, Thomas Petazzoni spake thusly:

Quoted 40 lines · J?r?me Pouiller

Quoted 6 lines

As suggested by Arnout [1], this series provide an alternative archive format. This new format contains a shallowed version of upstream repository. This format is a little bigger and a little longer to create but it allow a better workflow with upstream. I describe some good practice in patch 2.

Quoted 7 lines

Notice projects hosted by github don't yet benefit of this feature since I have not found any elegant way to do it :-(.

During my tests, I have noticed current shallow clone is mostly broken (at least with git < 2.5 [2]). Indeed, shallow clone only work with symbolic references (HEAD, a tag or a branch). However, we avoid use of symbolic references in VERSION.

Thanks for this contribution.

Your justification in PATCH 2 is just "That simplify workflow with upstream". However, we already have the <pkg>_OVERRIDE_SRCDIR mechanism (and <pkg>_SITE_METHOD = local, which is the same) to specifically address this use case.

The idea with <pkg>_OVERRIDE_SRCDIR is that if you are actively developing on a software component, then it should not be Buildroot's
responsibility to download/extract/patch it, but it should instead use a locally available source directory, which is managed completely separately from Buildroot. There you can do whatever Git, Subversion or Mercurial version control you want, Buildroot will simply rsync to the build directory.

I think doing development in the build directory, as encouraged by your patch, is a bad practice. The build directory is a temporary location, people should not be encouraged to work from there.

And I fail to see what your solution brings compared to using <pkg>_OVERRIDE_SRCDIR or <pkg>_SITE_METHOD = local. To me, the existing solutions are in fact more flexible and don't encourage the practice of hacking in the build directory.

Quoted 5 lines

Of course, if there is a specific workflow that you could describe that doesn't work with <pkg>_OVERRIDE_SRCDIR, then I'm definitely interested, and from this discussion we can decide whether improvements to OVERRIDE_SRCDIR are needed, or if a completely different solution is needed.

I have to agree with Thomas: we already have one mechanism to do actual development on packages, and I believe that it is the best we can offer:

- it is not removed on 'make clean' (the killing feature for it)

- it can be managed however the developer wants to (that too is a killing feature)

- it is not too complex to setup

Also, I believe it covers all use-cases we can imagine.

The one thing that is made a bit more complex is gdb-ing, because path in the build directory are referenced, rather than in the actual source dir. IMHO, that's a minor annoyance, and it is easy to mentally match the former to the latter.

So, except for the first patch which is indeed a nice cleanup, I'm not too favourable to that series as a whole...

Hello,

As suggested by Arnout [1], this series provide an alternative archive format. This new format contains a shallowed version of upstream repository. This format is a little bigger and a little longer to create but it allow a better workflow with upstream. I describe some good practice in patch 2.

Notice projects hosted by github don't yet benefit of this feature since I have not found any elegant way to do it :-(.

During my tests, I have noticed current shallow clone is mostly broken (at least with git < 2.5 [2]). Indeed, shallow clone only work with symbolic references (HEAD, a tag or a branch). However, we avoid use of symbolic references in VERSION.

[1] http://lists.busybox.net/pipermail/buildroot/2015-November/144296.html
[2] http://stackoverflow.com/a/30701724/301717

J?r?me Pouiller (5):
pkg-download: do not test SITE_METHOD
download/git: allow to create archives containing shallowed git repos
pkg-generic: allow to populate build directory from a git archive
pkg-generic: provide an option to use git archives
pkg-generic: tag sources if git is used

 Config.in               | 13 +++++++++++
 package/pkg-download.mk |  7 +-----
 package/pkg-generic.mk  | 60 ++++++++++++++++++++++++++++++++++---------------
 support/download/git    | 30 ++++++++++++++++++++++---
 4 files changed, 83 insertions(+), 27 deletions(-)

As suggested by Arnout [1], this series provide an alternative archive format. This new format contains a shallowed version of upstream repository. This format is a little bigger and a little longer to create but it allow a better workflow with upstream. I describe some good practice in patch 2.

Notice projects hosted by github don't yet benefit of this feature since I have not found any elegant way to do it :-(.

During my tests, I have noticed current shallow clone is mostly broken (at least with git < 2.5 [2]). Indeed, shallow clone only work with symbolic references (HEAD, a tag or a branch). However, we avoid use of symbolic references in VERSION.

[1] http://lists.busybox.net/pipermail/buildroot/2015-November/144296.html
[2] http://stackoverflow.com/a/30701724/301717

J?r?me Pouiller (5):
pkg-download: do not test SITE_METHOD
download/git: allow to create archives containing shallowed git repos
pkg-generic: allow to populate build directory from a git archive
pkg-generic: provide an option to use git archives
pkg-generic: tag sources if git is used