Skip to contentSkip to conversation

fetch2: accept sha256 revisions

PATCH Superseded Nick Owens 2 participants 2 messages Jun 18

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Nick Owens

author
#0 lore raw

PATCH 1/1 fetch2: accept sha256 revisions

regexes in fetch2's git support currently don't allow for fetching git repos with the sha256 object format.

to allow fetching sha256 git repos, adjust the regular expression and fix all of the references.

tested on my gitea server with a sha256 repo

fixes https://bugzilla.yoctoproject.org/show_bug.cgi?id=16322

---
 lib/bb/fetch2/git.py | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
lib/bb/fetch2/git.py +5 -5
index cd045a3a9..0fbd85441 100644
@@ -82,7 +82,7 @@ from   bb.fetch2 import logger
 from   bb.fetch2 import trusted_network
 
 
-sha1_re = re.compile(r'^[0-9a-f]{40}$')
+git_hash_re = re.compile(r'^[0-9a-f]{40,64}$')
 slash_re = re.compile(r"/+")
 
 class GitProgressHandler(bb.progress.LineFilterProgressHandler):
@@ -257,8 +257,8 @@ class Git(FetchMethod):
 
         ud.setup_revisions(d)
 
-        # Ensure any revision that doesn't look like a SHA-1 is translated into one
-        if not sha1_re.match(ud.revision or ''):
+        # Ensure any revision that doesn't look like a git hash (SHA-1 or SHA-256) is resolved
+        if not git_hash_re.match(ud.revision or ''):
             if ud.revision:
                 ud.unresolvedrev = ud.revision
             ud.revision = self.latest_revision(ud, d, ud.name)
@@ -752,7 +752,7 @@ class Git(FetchMethod):
 
         # If there is a tag parameter in the url and we also have a fixed srcrev, check the tag
         # matches the revision
-        if 'tag' in ud.parm and sha1_re.match(ud.revision):
+        if 'tag' in ud.parm and git_hash_re.match(ud.revision):
             output = runfetchcmd(ud.basecmd + ['rev-list', '-n', '1', ud.parm['tag']], d, workdir=destdir, extraenv=extraenv)
             output = output.strip()
             if output != ud.revision:
@@ -976,7 +976,7 @@ class Git(FetchMethod):
             bb.note("Could not list remote: %s" % str(e))
             return pupver
 
-        rev_tag_re = re.compile(r"([0-9a-f]{40})\s+refs/tags/(.*)")
+        rev_tag_re = re.compile(r"([0-9a-f]{40,64})\s+refs/tags/(.*)")
         pver_re = re.compile(d.getVar('UPSTREAM_CHECK_GITTAGREGEX') or r"(?P<pver>([0-9][\.|_]?)+)")
         nonrel_re = re.compile(r"(alpha|beta|rc|final)+")
 
Signature · 1 line
2.52.0
#1 lore raw
Quoted 6 lines · Nick Owens via lists.openembedded.org

regexes in fetch2's git support currently don't allow for fetching git repos with the sha256 object format.

to allow fetching sha256 git repos, adjust the regular expression and fix all of the references.

Quoted 6 lines

tested on my gitea server with a sha256 repo

fixes https://bugzilla.yoctoproject.org/show_bug.cgi?id=16322

---

Hi Nick,

Thanks for your patch.

It looks like we still have some use of sha1_re in bitbake-setup, and so have a failing bitbake selftest:

ERROR: test_setup (bb.tests.setup.BitbakeSetupTest.test_setup)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/lib/bb/tests/setup.py", line 389, in test_setup
self.check_setupdir_files(setuppath, test_file_content)
File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/lib/bb/tests/setup.py", line 296, in check_setupdir_files
self.config_is_unchanged(setuppath)
File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/lib/bb/tests/setup.py", line 237, in config_is_unchanged
out = self.runbbsetup(["status"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/lib/bb/tests/setup.py", line 106, in runbbsetup
return bb.process.run([bbsetup, '--global-settings', os.path.join(self.tempdir, 'global-config')] + cmd, env=env, cwd=self.tempdir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/lib/bb/process.py", line 189, in run
raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
bb.process.ExecutionError: Execution of '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/bin/bitbake-setup --global-settings /tmp/bitbake-fetch-irglhhir/global-config status' failed with exit code 1:
NOTE: Loading settings from /tmp/bitbake-fetch-irglhhir/global-config
NOTE: Loading settings from /tmp/bitbake-fetch-irglhhir/bitbake-builds/settings.conf
NOTE: Bitbake-setup is using /tmp/bitbake-fetch-irglhhir/bitbake-builds as top directory.
NOTE: Looking up config test-config-1 in configuration registry
NOTE: Fetching configuration registry
git:///tmp/bitbake-fetch-irglhhir/bitbake-setup-configurations;protocol=file;branch=master;rev=master
into
/tmp/bitbake-fetch-irglhhir/bitbake-builds/.bitbake-setup-cache/configurations
ERROR: cannot import name 'sha1_re' from 'bb.fetch2.git' (/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/bitbake/lib/bb/fetch2/git.py)

https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/4067
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/3917

Can you fix this?

Thanks,
Mathieu

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com