Skip to contentSkip to conversation

Makefile.package.in: GIT: add support to fetch git submodules

PATCH New Alexandre Pereira da Silva 3 participants 4 messages Feb 28, 2012 – Jul 30, 2012

lore
Share & tools
lore

Alexandre Pereira da Silva

author
#0 lore raw

PATCH 1/1 Makefile.package.in: GIT: add support to fetch git submodules

Signed-off-by Alexandre Pereira da Silva <[email protected]>
---
 package/Makefile.package.in |   10 ++++++----
 1 files changed, 6 insertions(+), 4 deletions(-)
package/Makefile.package.in +6 -4
index ea44c6c..863e866 100644
@@ -147,13 +147,15 @@ domainseparator=$(if $(1),$(1),/)
 define DOWNLOAD_GIT
 	test -e $(DL_DIR)/$($(PKG)_SOURCE) || \
 	(pushd $(DL_DIR) > /dev/null && \
-	$(GIT) clone --bare $($(PKG)_SITE) $($(PKG)_BASE_NAME) && \
+	$(GIT) clone -n $($(PKG)_SITE) $($(PKG)_BASE_NAME) && \
 	pushd $($(PKG)_BASE_NAME) > /dev/null && \
-	$(GIT) archive --format=tar --prefix=$($(PKG)_BASE_NAME)/ $($(PKG)_DL_VERSION) | \
-		gzip -c > $(DL_DIR)/$($(PKG)_SOURCE) && \
+	$(GIT) checkout $($(PKG)_VERSION) && \
+	$(GIT) submodule init && \
+	$(GIT) submodule update && \
+	$(TAR) czf $(DL_DIR)/$($(PKG)_SOURCE) . &&\
 	popd > /dev/null && \
 	rm -rf $($(PKG)_DL_DIR) && \
-	popd > /dev/null)
+	popd > /dev/null)    
 endef
 
 # TODO: improve to check that the given PKG_DL_VERSION exists on the remote
Signature · 1 line
1.7.9
#1 lore raw

PATCH 1/1 Makefile.package.in: GIT: add support to fetch git submodules

Quoted 19 lines · On Tuesday 28 February 2012 13:25:19 Alexandre Pereira da Silva wrote:
Signed-off-by Alexandre Pereira da Silva <[email protected]>
---
 package/Makefile.package.in |   10 ++++++----
 1 files changed, 6 insertions(+), 4 deletions(-)
package/Makefile.package.in +2 -3
index ea44c6c..863e866 100644
@@ -147,13 +147,15 @@ domainseparator=$(if $(1),$(1),/)
 define DOWNLOAD_GIT
 	test -e $(DL_DIR)/$($(PKG)_SOURCE) || \
 	(pushd $(DL_DIR) > /dev/null && \
-	$(GIT) clone --bare $($(PKG)_SITE) $($(PKG)_BASE_NAME) && \
+	$(GIT) clone -n $($(PKG)_SITE) $($(PKG)_BASE_NAME) && \
 	pushd $($(PKG)_BASE_NAME) > /dev/null && \
-	$(GIT) archive --format=tar --prefix=$($(PKG)_BASE_NAME)/ $($(PKG)_DL_VERSION) | \
-		gzip -c > $(DL_DIR)/$($(PKG)_SOURCE) && \
+	$(GIT) checkout $($(PKG)_VERSION) && \

package/Makefile.package.in domainseparator=$(if $(1),$(1),/) · line 161

Quoted 10 lines
package/Makefile.package.in +4 -1
+	$(GIT) submodule init && \
+	$(GIT) submodule update && \
+	$(TAR) czf $(DL_DIR)/$($(PKG)_SOURCE) . &&\
 	popd > /dev/null && \
 	rm -rf $($(PKG)_DL_DIR) && \
-	popd > /dev/null)
+	popd > /dev/null)    
 endef
 
 # TODO: improve to check that the given PKG_DL_VERSION exists on the remote

I'm a bit concerned with the impact this will have on performance, e.g.
when cloning a linux git. Could you evaluate that?

Regards,
Arnout

Signature · 6 lines
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
author
#2 lore raw

PATCH 1/1 Makefile.package.in: GIT: add support to fetch git submodules

Quoted 5 lines

?I'm a bit concerned with the impact this will have on performance, e.g. when cloning a linux git. ?Could you evaluate that?

?Regards,
?Arnout

Arnout,

I just saw a problem in this first implementation. I am storing not only the checkout but the git repository as well. I will exclude those files and resend the patch.

I did some tests, with the optimization that I will implement. Here are the figures comparing the time needed to checkout a git repository versus the time to just archive it, excluding any network overhead:
rm -rf -- *; rm -f ../linux.tar.gz; time (git checkout -f master && tar czf ../linux.tar.gz --exclude-vcs . && git submodule init && git submodule update); rm -f ../linux.tar.gz; time git archive -o ../linux.tar.gz master
Checking out files: 100% (37083/37083), done.
Already on 'master'

real 1m24.828s
user 1m9.588s
sys 0m13.233s

real 1m0.601s
user 1m7.676s
sys 0m1.904s

So, there is a 24s overhead or 40% more time needed to checkout the copy.

We could avoid this extra delay if we check for the presence of any git submodule. If this kind of delay in the download phase is important I can go in that direction.

Regards

#3 lore raw

PATCH 1/1 Makefile.package.in: GIT: add support to fetch git submodules

Hi Alexandre and all,

Quoted 9 lines · Alexandre Pereira da Silva
Quoted 5 lines

I'm a bit concerned with the impact this will have on performance, e.g.
when cloning a linux git. Could you evaluate that?

Regards,
Arnout

I just saw a problem in this first implementation. I am storing not only the checkout but the git repository as well. I will exclude those files and resend the patch.

I was wondering if there had been any progress or further thoughts on adding support for fetching git submodules. I just ran into a situation where your patch would have been convenient.

Thanks,
Frank