Skip to contentSkip to conversation

fetch/git: return the shallow tarball as localpath when there is no clone

PATCH New [email protected] 1 participant 1 message Nov 17

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fetch/git: return the shallow tarball as localpath when there is no clone

From: Etienne Cordonnier <[email protected]>

Usually localpath() exists once do_fetch has completed, but not for a shallow fetch which uses the shallow tarball without creating the clone:
either the fast initial shallow clone, or a restore from a mirror. download() sets ud.localpath to the tarball, but a new Fetch object, as in the archiver, gets the non-existent clone from localpath() again.

Return the shallow tarball from localpath() when the clone does not exist. If the clone exists, even if it is outdated, localpath() still returns it.

The names of the .done and .lock files are derived from localpath, so they would then depend on whether the clone exists, and fetchers of the same url would not exclude each other. Add FetchMethod.stamppath(), which defaults to localpath, and let git always use the clone.

Do not follow the symlink of the shallow tarball in clean(), which would remove the tarball in the mirror.

AI-Generated: Claude Sonnet 5.5

---
 lib/bb/fetch/__init__.py | 17 ++++++++---
 lib/bb/fetch/git.py      | 14 +++++++--
 lib/bb/tests/fetch.py    | 66 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 91 insertions(+), 6 deletions(-)
lib/bb/fetch/__init__.py +13 -4
index ce2bd62ad..88c244ba4 100644
@@ -1387,10 +1387,11 @@ class FetchData(object):
             return
 
         # Note: .done and .lock files should always be in DL_DIR whereas localpath may not be.
-        if self.localpath and self.localpath.startswith(dldir):
-            basepath = self.localpath
-        elif self.localpath:
-            basepath = dldir + os.sep + os.path.basename(self.localpath)
+        stamppath = self.method.stamppath(self, d) if self.localpath else None
+        if stamppath and stamppath.startswith(dldir):
+            basepath = stamppath
+        elif stamppath:
+            basepath = dldir + os.sep + os.path.basename(stamppath)
         elif self.basepath or self.basename:
             basepath = dldir + os.sep + (self.basepath or self.basename)
         else:
@@ -1431,6 +1432,14 @@ class FetchMethod(object):
         """
         return 0
 
+    def stamppath(self, urldata, d):
+        """
+        Return the path which the names of the .done and .lock files are based
+        on. This must not change between calls, as the lock is only effective
+        if every fetcher of the same url uses the same lock file.
+        """
+        return urldata.localpath
+
     def localpath(self, urldata, d):
         """
         Return the local filename of a given url assuming a successful fetch.
lib/bb/fetch/git.py +12 -2
index 75ce1eda0..4a12c4bfa 100644
@@ -308,7 +308,16 @@ class Git(FetchMethod):
             ud.fullshallow = os.path.join(dl_dir, ud.shallowtarball)
             ud.mirrortarballs.insert(0, ud.shallowtarball)
 
+    def stamppath(self, ud, d):
+        # localpath() depends on whether the clone exists, the stamp and lock
+        # file names must not.
+        return ud.clonedir
+
     def localpath(self, ud, d):
+        # When do_fetch used the shallow tarball without creating the clone,
+        # the tarball is the result of the fetch.
+        if ud.shallow and not os.path.exists(ud.clonedir) and os.path.exists(ud.fullshallow):
+            return ud.fullshallow
         return ud.clonedir
 
     def need_update(self, ud, d):
@@ -805,8 +814,9 @@ class Git(FetchMethod):
 
         to_remove = [ud.localpath, ud.fullmirror, ud.fullmirror + ".done"]
         # The localpath is a symlink to clonedir when it is cloned from a
-        # mirror, so remove both of them.
-        if os.path.islink(ud.localpath):
+        # mirror, so remove both of them. When it is the shallow tarball, it
+        # can be a symlink to the tarball in the mirror, which must be kept.
+        if not (ud.shallow and ud.localpath == ud.fullshallow) and os.path.islink(ud.localpath):
             clonedir = os.path.realpath(ud.localpath)
             to_remove.append(clonedir)
 
lib/bb/tests/fetch.py +66 -0

72 lines not on this page · raw message

Signature · 1 line
2.43.0