fetch2/git: debug updates and minor cleanup

PATCH Applied Darren Hart 2 participants 5 messages May 20, 2011 – May 27, 2011

lore

Share & tools

lore

Darren Hart

author

2011-05-20 23:06 UTC #0 lore raw

cover fetch2/git: debug updates and minor cleanup

From: Darren Hart <[email protected]>

Yu Ke,

Please consider the following patches against the fetch2/git fetcher. The one point I would appreciate your review of is in 2/3. Is there a concern that the urldata has not been initialized at the point where I made the change?

Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: dvhart/git2
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dvhart/git2

Thanks,
Darren Hart <[email protected]>
---

Darren Hart (3):
fetch2/git: ensure network check log matches actual command fetch2/git: use urldata basecmd when available fetch2/git: use logging.debug() and clarify messages

 bitbake/lib/bb/fetch2/git.py |   24 +++++++++++++++---------
 1 files changed, 15 insertions(+), 9 deletions(-)

Darren Hart

author

2011-05-20 23:06 UTC #1 lore raw

PATCH 1/3 fetch2/git: ensure network check log matches actual command

From: Darren Hart <[email protected]>

The git command string logged via check_network_access() does not match the actual command executed in a few places. Ensure that it does.

Signed-off-by Darren Hart <[email protected]>

Cc Yu Ke <[email protected]>

---
 bitbake/lib/bb/fetch2/git.py |   16 ++++++++++------
 1 files changed, 10 insertions(+), 6 deletions(-)

bitbake/lib/bb/fetch2/git.py +10 -6

index 82721c6..c6c4cc9 100644
@@ -137,8 +137,10 @@ class Git(FetchMethod):
 
         # If the repo still doesn't exist, fallback to cloning it
         if not os.path.exists(ud.clonedir):
-            bb.fetch2.check_network_access(d, "git clone --bare %s%s" % (ud.host, ud.path))
-            runfetchcmd("%s clone --bare %s://%s%s%s %s" % (ud.basecmd, ud.proto, username, ud.host, ud.path, ud.clonedir), d)
+            clone_cmd = "%s clone --bare %s://%s%s%s %s" % \
+                  (ud.basecmd, ud.proto, username, ud.host, ud.path, ud.clonedir)
+            bb.fetch2.check_network_access(d, clone_cmd)
+            runfetchcmd(clone_cmd, d)
 
         os.chdir(ud.clonedir)
         # Update the checkout if needed

@@ -147,7 +149,6 @@ class Git(FetchMethod):
             if not self._contains_ref(ud.revisions[name], d):
                 needupdate = True
         if needupdate:
-            bb.fetch2.check_network_access(d, "git fetch %s%s" % (ud.host, ud.path), ud.url)
             try: 
                 runfetchcmd("%s remote prune origin" % ud.basecmd, d) 
                 runfetchcmd("%s remote rm origin" % ud.basecmd, d)

@@ -155,7 +156,9 @@ class Git(FetchMethod):
                 logger.debug(1, "No Origin")
             
             runfetchcmd("%s remote add origin %s://%s%s%s" % (ud.basecmd, ud.proto, username, ud.host, ud.path), d)
-            runfetchcmd("%s fetch --all -t" % ud.basecmd, d)
+            fetch_cmd = "%s fetch --all -t" % ud.basecmd
+            bb.fetch2.check_network_access(d, fetch_cmd, ud.url)
+            runfetchcmd(fetch_cmd, d)
             runfetchcmd("%s prune-packed" % ud.basecmd, d)
             runfetchcmd("%s pack-redundant --all | xargs -r rm" % ud.basecmd, d)
             ud.repochanged = True

@@ -216,9 +219,10 @@ class Git(FetchMethod):
         else:
             username = ""
 
-        bb.fetch2.check_network_access(d, "git ls-remote %s%s %s" % (ud.host, ud.path, ud.branches[name]))
         basecmd = data.getVar("FETCHCMD_git", d, True) or "git"
-        cmd = "%s ls-remote %s://%s%s%s %s" % (basecmd, ud.proto, username, ud.host, ud.path, ud.branches[name])
+        cmd = "%s ls-remote %s://%s%s%s %s" % \
+              (basecmd, ud.proto, username, ud.host, ud.path, ud.branches[name])
+        bb.fetch2.check_network_access(d, cmd)
         output = runfetchcmd(cmd, d, True)
         if not output:
             raise bb.fetch2.FetchError("The command %s gave empty output unexpectedly" % cmd, url)

Signature · 1 line

1.7.1

Darren Hart

author

2011-05-20 23:06 UTC #2 lore raw

PATCH 2/3 fetch2/git: use urldata basecmd when available

In reply to Darren Hart

From: Darren Hart <[email protected]>

If the urldata object is passed in, use it to determine basecmd rather than looking it up again.

Signed-off-by Darren Hart <[email protected]>

Cc Yu Ke <[email protected]>

---
 bitbake/lib/bb/fetch2/git.py |   15 +++++++--------
 1 files changed, 7 insertions(+), 8 deletions(-)

bitbake/lib/bb/fetch2/git.py +7 -8

index c6c4cc9..7ffaa5e 100644
@@ -137,10 +137,10 @@ class Git(FetchMethod):
 
         # If the repo still doesn't exist, fallback to cloning it
         if not os.path.exists(ud.clonedir):
-            clone_cmd = "%s clone --bare %s://%s%s%s %s" % \
+            cmd = "%s clone --bare %s://%s%s%s %s" % \
                   (ud.basecmd, ud.proto, username, ud.host, ud.path, ud.clonedir)
-            bb.fetch2.check_network_access(d, clone_cmd)
-            runfetchcmd(clone_cmd, d)
+            bb.fetch2.check_network_access(d, cmd)
+            runfetchcmd(cmd, d)
 
         os.chdir(ud.clonedir)
         # Update the checkout if needed

@@ -156,9 +156,9 @@ class Git(FetchMethod):
                 logger.debug(1, "No Origin")
             
             runfetchcmd("%s remote add origin %s://%s%s%s" % (ud.basecmd, ud.proto, username, ud.host, ud.path), d)
-            fetch_cmd = "%s fetch --all -t" % ud.basecmd
-            bb.fetch2.check_network_access(d, fetch_cmd, ud.url)
-            runfetchcmd(fetch_cmd, d)
+            cmd = "%s fetch --all -t" % ud.basecmd
+            bb.fetch2.check_network_access(d, cmd, ud.url)
+            runfetchcmd(cmd, d)
             runfetchcmd("%s prune-packed" % ud.basecmd, d)
             runfetchcmd("%s pack-redundant --all | xargs -r rm" % ud.basecmd, d)
             ud.repochanged = True

@@ -219,9 +219,8 @@ class Git(FetchMethod):
         else:
             username = ""
 
-        basecmd = data.getVar("FETCHCMD_git", d, True) or "git"
         cmd = "%s ls-remote %s://%s%s%s %s" % \
-              (basecmd, ud.proto, username, ud.host, ud.path, ud.branches[name])
+              (ud.basecmd, ud.proto, username, ud.host, ud.path, ud.branches[name])
         bb.fetch2.check_network_access(d, cmd)
         output = runfetchcmd(cmd, d, True)
         if not output:

Signature · 1 line

1.7.1

Darren Hart

author

2011-05-20 23:06 UTC #3 lore raw

PATCH 3/3 fetch2/git: use logging.debug() and clarify messages

In reply to Darren Hart

From: Darren Hart <[email protected]>

Replace a call to print() with logging.debug() and flesh out the message to clarify the state being reported.

Signed-off-by Darren Hart <[email protected]>

Cc Yu Ke <[email protected]>

---
 bitbake/lib/bb/fetch2/git.py |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

bitbake/lib/bb/fetch2/git.py +5 -2

index 7ffaa5e..c14346b 100644
@@ -241,10 +241,13 @@ class Git(FetchMethod):
         # Check if we have the rev already
 
         if not os.path.exists(ud.clonedir):
-            print("no repo")
+            logging.debug("GIT repository for %s does not exist in %s.  \
+                          Downloading.", url, ud.clonedir)
             self.download(None, ud, d)
             if not os.path.exists(ud.clonedir):
-                logger.error("GIT repository for %s doesn't exist in %s, cannot get sortable buildnumber, using old value", url, ud.clonedir)
+                logger.error("GIT repository for %s does not exist in %s after \
+                             download. Cannot get sortable buildnumber, using \
+                             old value", url, ud.clonedir)
                 return None

Signature · 1 line

1.7.1

Richard Purdie

Applied

2011-05-27 17:19 UTC #4 lore raw

In reply to Darren Hart

Quoted 12 lines · Darren Hart

From: Darren Hart <[email protected]>
Yu Ke,
Please consider the following patches against the fetch2/git fetcher. The one point I would appreciate your review of is in 2/3. Is there a concern that the urldata has not been initialized at the point where I made the change?
Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: dvhart/git2
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dvhart/git2

Quoted 8 lines

Thanks,
Darren Hart <[email protected]>
---
Darren Hart (3):
fetch2/git: ensure network check log matches actual command fetch2/git: use logging.debug() and clarify messages

I merged these into bitbake master, thanks.

fetch2/git: use urldata basecmd when available

This one we a had a good reason for, at least a long time ago. It needs some further checking into the callers (and better documentation if it is deliberate). The fetcher world has changed a lot since those times...

I'm therefore holding off on it for now.

Cheers,

Richard