Skip to contentSkip to conversation

bitbake: fetch2/git: add SRC_URI option versiontag

PATCH Superseded [email protected] 2 participants 4 messages Dec 4, 2019 – Jan 6, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 bitbake: fetch2/git: add SRC_URI option versiontag

From: Pascal Huerst <[email protected]>

Use latest_versionstring() to find the latest tagged release for use with AUTOREV. This can be useful, if you want to build the latest tagged release instead of a branch. UPSTREAM_CHECK_GITTAGREGEX can be used to specify a version format.

---
 bitbake/lib/bb/fetch2/git.py  | 24 ++++++++++++++++++++----
 bitbake/lib/bb/tests/fetch.py | 18 ++++++++++++++++++
 2 files changed, 38 insertions(+), 4 deletions(-)
bitbake/lib/bb/fetch2/git.py +20 -4
index fa41b078f1..3091619475 100644
@@ -47,6 +47,11 @@ Supported SRC_URI options are:
    referring to commit which is valid in tag instead of branch.
    The default is "0", set nobranch=1 if needed.
 
+- versiontag
+   Use latest_versionstring() to find the latest tagged release for use with
+   AUTOREV. Implies nobranch. Check UPSTREAM_CHECK_GITTAGREGEX on how to specify
+   your version format.
+
 - usehead
    For local git:// urls to use the current branch HEAD as the revision for use with
    AUTOREV. Implies nobranch.
@@ -150,6 +155,8 @@ class Git(FetchMethod):
 
         ud.nobranch = ud.parm.get("nobranch","0") == "1"
 
+        ud.versiontag = ud.parm.get("versiontag","0") == "1"
+
         # usehead implies nobranch
         ud.usehead = ud.parm.get("usehead","0") == "1"
         if ud.usehead:
@@ -161,9 +168,20 @@ class Git(FetchMethod):
         ud.bareclone = ud.parm.get("bareclone","0") == "1"
         if ud.bareclone:
             ud.nocheckout = 1
-  
+
+        # versiontag implies nobranch
+        if ud.versiontag:
+            ud.nobranch = 1
+
+        ud.basecmd = d.getVar("FETCHCMD_git") or "git -c core.fsyncobjectfiles=0"
+
         ud.unresolvedrev = {}
         branches = ud.parm.get("branch", "master").split(',')
+
+        if ud.versiontag:
+            upversion, revision = self.latest_versionstring(ud, d)
+            d.setVar("SRCREV", revision)
+
         if len(branches) != len(ud.names):
             raise bb.fetch2.ParameterError("The number of name and branch parameters is not balanced", ud.url)
 
@@ -221,8 +239,6 @@ class Git(FetchMethod):
         if ud.usehead:
             ud.unresolvedrev['default'] = 'HEAD'
 
-        ud.basecmd = d.getVar("FETCHCMD_git") or "git -c core.fsyncobjectfiles=0"
-
         write_tarballs = d.getVar("BB_GENERATE_MIRROR_TARBALLS") or "0"
         ud.write_tarballs = write_tarballs != "0" or ud.rebaseable
         ud.write_shallow_tarballs = (d.getVar("BB_GENERATE_SHALLOW_TARBALLS") or write_tarballs) != "0"
@@ -674,7 +690,7 @@ class Git(FetchMethod):
             if tag == None:
                 continue
 
-            tag = tag.group('pver')
+            tag = tag.group(0)
             tag = tag.replace("_", ".")
 
             if verstring and bb.utils.vercmp(("0", tag, ""), ("0", verstring, "")) < 0:
bitbake/lib/bb/tests/fetch.py +18 -0
index 83fad3ff0d..175f0abd61 100644
@@ -996,6 +996,24 @@ class FetcherNetworkTest(FetcherTest):
         self.assertTrue(os.path.exists(os.path.join(repo_path, 'edgelet/hsm-sys/azure-iot-hsm-c/deps/utpm/deps/c-utility/testtools/umock-c/deps/ctest/README.md')), msg='Missing submodule checkout')
         self.assertTrue(os.path.exists(os.path.join(repo_path, 'edgelet/hsm-sys/azure-iot-hsm-c/deps/utpm/deps/c-utility/testtools/umock-c/deps/testrunner/readme.md')), msg='Missing submodule checkout')
 
+    @skipIfNoNetwork()
+    def test_git_versiontag(self):
+        """ If versiontag is set in SRC_URI, it should fetch the latest release tag, using UPSTREAM_CHECK_GITTAGREGEX. If unset, standard format vX.X.X is used"""
+
+        url = "git://git.yoctoproject.org/git/poky;protocol=https;versiontag=1"
+
+        self.d.setVar('SRCREV', '${AUTOREV}')
+        self.d.setVar('AUTOREV', '${@bb.fetch2.get_autorev(d)}')
+        self.d.setVar('UPSTREAM_CHECK_GITTAGREGEX', "([a-z]+)-([0-9]+).([0-9]+).([0-9]+)")
+
+        fetcher = bb.fetch.Fetch([url], self.d)
+        fetcher.download()
+
+        os.chdir(os.path.dirname(self.unpackdir))
+        fetcher.unpack(self.unpackdir)
+
+        repo_path = os.path.join(self.tempdir, 'unpacked', 'git')
+
 class SVNTest(FetcherTest):
     def skipIfNoSvn():
         import shutil
Signature · 1 line
2.23.0
#1 lore raw
Quoted 7 lines · On Dec 4, 2019, at 04:16, [email protected] wrote:

From: Pascal Huerst <[email protected]>

Use latest_versionstring() to find the latest tagged release for use with AUTOREV. This can be useful, if you want to build the latest tagged release instead of a branch. UPSTREAM_CHECK_GITTAGREGEX can be used to specify a version format.

Have you seen this approach?
https://code.ossystems.com.br/gitweb?p=meta-ossystems-base.git;a=blob;f=classes/ossystems-srcrev-handler.bbclass

For development workflows, it would be useful to override a recipe-specific branch/tag via bitbake CLI. For production/stable release workflows, it would be useful to override a layer-specific branch/tag via bitbake CLI, which could be propagated to all recipes within the layer. This would help when maintaining multiple versions of a layer, corresponding to multiple OE versions.

Rich

#2 lore raw

Hey Rich,

Quoted 4 lines · Rich Persaud
Quoted 3 lines · On Dec 4, 2019, at 04:16, [email protected] wrote:

From: Pascal Huerst <[email protected]>

Quoted 8 lines
Quoted 5 lines

Use latest_versionstring() to find the latest tagged release for use with AUTOREV. This can be useful, if you want to build the latest tagged release
instead of a branch. UPSTREAM_CHECK_GITTAGREGEX can be used to specify a version format.

Have you seen this approach?
https://code.ossystems.com.br/gitweb?p=meta-ossystems-base.git;a=blob;f=classes/ossystems-srcrev-handler.bbclass

Thanks for the hint, this goes into the right direction, but does not solve my problem.

Quoted 6 lines

For development workflows, it would be useful to override a recipe-specific branch/tag via bitbake CLI.  For production/stable release workflows, it would be useful to override a layer-specific branch/tag via bitbake CLI, which could be propagated to all recipes within the layer.   This would help when maintaining multiple versions of a layer, corresponding to multiple OE versions.

That's exactly how we manage our development releases, but what I wanted to achieve with this patch is to just tag releases in our repositories and let bitbake figure out the latest releases to create an image from it. That way we don't have to fiddle around with bitbake CLI or yocto at all, but can just tag releases in our repositories.

I find that quite useful, but the interest has been near zero for that patch and I'm not sure, whether this is such an exotic use-case or if there are quality issues with my patch...?

Thanks,
Pascal

#3 lore raw
Quoted 31 lines · Pascal Huerst

Hey Rich,

Quoted 10 lines · Rich Persaud
Quoted 7 lines · On Dec 4, 2019, at 04:16, [email protected] wrote:

From: Pascal Huerst <[email protected]>
Use latest_versionstring() to find the latest tagged release for use with AUTOREV. This can be useful, if you want to build the latest tagged release
instead of a branch. UPSTREAM_CHECK_GITTAGREGEX can be used to specify a version format.

Have you seen this approach?
https://code.ossystems.com.br/gitweb?p=meta-ossystems-base.git;a=blob;f=classes/ossystems-srcrev-handler.bbclass

Thanks for the hint, this goes into the right direction, but does not solve my problem.

Quoted 6 lines

For development workflows, it would be useful to override a recipe-specific branch/tag via bitbake CLI. For production/stable release workflows, it would be useful to override a layer-specific branch/tag via bitbake CLI, which could be propagated to all recipes
within the layer. This would help when maintaining multiple versions
of a layer, corresponding to multiple OE versions.

That's exactly how we manage our development releases, but what I wanted to achieve with this patch is to just tag releases in our repositories and let bitbake figure out the latest releases to create an image from it. That way we don't have to fiddle around with bitbake CLI or yocto at all, but can just tag releases in our repositories.

Quoted 3 lines

I find that quite useful, but the interest has been near zero for that patch and I'm not sure, whether this is such an exotic use-case or if there are quality issues with my patch...?

Maybe start a thread on another mailing list (oe-arch? oe-core?) to collect requirements for related workflows, so your patch can be generalized to address more use cases, before extending the syntax for bitbake invocation? There may be other bitbake improvements that OE developers have done downstream, which could be consolidated.

Rich