Skip to contentSkip to conversation

git.py: skip smudging if lfs=0 is set

PATCH Superseded Mauro Queirós 2 participants 5 messages May 28, 2020 – May 29, 2020

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/3 git.py: skip smudging if lfs=0 is set

Git-LFS objects were being fetched even when lfs=0 was not set. This patch disables LFS smudging when lfs=0. That way, only the LFS pointers are downloaded during checkout.

---
 lib/bb/fetch2/git.py | 3 +++
 1 file changed, 3 insertions(+)
lib/bb/fetch2/git.py +3 -0
index 5b3793a7..4c7d388e 100644
@@ -475,6 +475,9 @@ class Git(FetchMethod):
 
         need_lfs = ud.parm.get("lfs", "1") == "1"
 
+        if not need_lfs:
+            ud.basecmd = "GIT_LFS_SKIP_SMUDGE=1 " + ud.basecmd
+
         source_found = False
         source_error = []
 
Signature · 1 line
2.17.1
#1 lore raw

PATCH 2/3 git.py: LFS bitbake note should not be printed if need_lfs is not set.

The message "Repository %s has LFS content but it is not being fetched" was being printed, even when Git-LFS was available and "lfs=1" was set. In those situations, we want to fetch LFS content, so that message would not make sense.

---
 lib/bb/fetch2/git.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/bb/fetch2/git.py +1 -1
index 4c7d388e..eab76a10 100644
@@ -509,7 +509,7 @@ class Git(FetchMethod):
         if self._contains_lfs(ud, d, destdir):
             if need_lfs and not self._find_git_lfs(d):
                 raise bb.fetch2.FetchError("Repository %s has LFS content, install git-lfs on host to download (or set lfs=0 to ignore it)" % (repourl))
-            else:
+            elif not need_lfs:
                 bb.note("Repository %s has LFS content but it is not being fetched" % (repourl))
 
         if not ud.nocheckout:
Signature · 1 line
2.17.1
#2 lore raw

PATCH 3/3 git.py: Use the correct branch to check if the repository has LFS objects.

In reply to Mauro Queirós

Function "contains_lfs" was only looking at the master branch when searching for LFS content. LFS may be configured in specific branches only, so we need to use the correct branch.

---
 lib/bb/fetch2/git.py | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
lib/bb/fetch2/git.py +9 -2
index eab76a10..b7841c43 100644
@@ -566,8 +566,15 @@ class Git(FetchMethod):
         """
         Check if the repository has 'lfs' (large file) content
         """
-        cmd = "%s grep lfs HEAD:.gitattributes | wc -l" % (
-                ud.basecmd)
+
+        if not ud.nobranch:
+            branchname = ud.branches[ud.names[0]]
+        else:
+            branchname = "master"
+
+        cmd = "%s show %s:.lfsconfig" % (
+                ud.basecmd, branchname)
+
         try:
             output = runfetchcmd(cmd, d, quiet=True, workdir=wd)
             if int(output) > 0:
Signature · 1 line
2.17.1
#3 lore raw
In reply to Mauro Queirós
Quoted 15 lines · Mauro Queirós

Git-LFS objects were being fetched even when lfs=0 was not set. This patch disables LFS smudging when lfs=0. That way, only the LFS pointers
are downloaded during checkout.

---
 lib/bb/fetch2/git.py | 3 +++
 1 file changed, 3 insertions(+)
lib/bb/fetch2/git.py +0 -0
index 5b3793a7..4c7d388e 100644

lib/bb/fetch2/git.py class Git(FetchMethod): · line 482

Quoted 8 lines
lib/bb/fetch2/git.py +3 -0
@@ -475,6 +475,9 @@ class Git(FetchMethod):
 
         need_lfs = ud.parm.get("lfs", "1") == "1"
 
+        if not need_lfs:
+            ud.basecmd = "GIT_LFS_SKIP_SMUDGE=1 " + ud.basecmd
+
         source_found = False
         source_error = []

Thanks for the patches, the sound good. Unfortunately they cause bitbake-selftest to fail:

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/991/steps/8/logs/step1d

Cheers,

Richard

#4 lore raw

That's because I was using .lfsconfig instead of .gitattributes in "contains_lfs" function. self-tests only use the latter. I've send v2 patches.

Richard Purdie <[email protected]> escreveu no dia sexta, 29/05/2020 à(s) 06:46:

Quoted 36 lines · Mauro Queirós
Quoted 15 lines

Git-LFS objects were being fetched even when lfs=0 was not set. This patch disables LFS smudging when lfs=0. That way, only the LFS pointers
are downloaded during checkout.

---
 lib/bb/fetch2/git.py | 3 +++
 1 file changed, 3 insertions(+)
lib/bb/fetch2/git.py +0 -0
index 5b3793a7..4c7d388e 100644

lib/bb/fetch2/git.py class Git(FetchMethod): · line 482

Quoted 8 lines
lib/bb/fetch2/git.py +3 -0
@@ -475,6 +475,9 @@ class Git(FetchMethod):
 
         need_lfs = ud.parm.get("lfs", "1") == "1"
 
+        if not need_lfs:
+            ud.basecmd = "GIT_LFS_SKIP_SMUDGE=1 " + ud.basecmd
+
         source_found = False
         source_error = []

Thanks for the patches, the sound good. Unfortunately they cause bitbake-selftest to fail:

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/991/steps/8/logs/step1d

Cheers,

Richard