Skip to contentSkip to conversation

lib/bb/fetch2/git.py: add optional major_version parameter to latest_versionstring

RFC PATCH In review [email protected] 2 participants 5 messages Apr 29 – Apr 30

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/3 lib/bb/fetch2/git.py: add optional major_version parameter to latest_versionstring

From: Chen Qi <[email protected]>

We need to handle the situation of LTS upgrades. This means for version such as 1.0.0, we need to find the upgradable version of 1.0.1 instead of something like 1.1.0.

---
 lib/bb/fetch2/git.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
lib/bb/fetch2/git.py +5 -1
index 10eb63b20..3109a7256 100644
@@ -976,7 +976,7 @@ class Git(FetchMethod):
         raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream git repository in git ls-remote output for %s" % \
             (ud.unresolvedrev, ud.host+ud.path))
 
-    def latest_versionstring(self, ud, d):
+    def latest_versionstring(self, ud, d, major_version=None):
         """
         Compute the latest release name like "x.y.x" in "x.y.x+gitHASH"
         by searching through the tags output of ls-remote, comparing
@@ -1016,6 +1016,10 @@ class Git(FetchMethod):
 
             pver = m.group('pver').replace("_", ".")
 
+            if major_version:
+                if not pver.startswith(major_version+'.'):
+                    continue
+
             if verstring and bb.utils.vercmp(("0", pver, ""), ("0", verstring, "")) < 0:
                 continue
 
Signature · 1 line
2.34.1
#1 lore raw

RFC PATCH 2/3 lib/bb/fetch2/wget.py: add optional major_version parameter to latest_versionstring

From: Chen Qi <[email protected]>

We need to handle the situation of LTS upgrades. This means for version such as 1.0.0, we need to find the upgradable version of 1.0.1 instead of something like 1.1.0.

Add optional major_version parameter to wget fetcher to support such case.

---
 lib/bb/fetch2/wget.py | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
lib/bb/fetch2/wget.py +9 -6
index 6ac4306c0..86d0a604a 100644
@@ -492,7 +492,7 @@ class Wget(FetchMethod):
 
         return fetchresult
 
-    def _check_latest_version(self, url, package, package_regex, current_version, ud, d):
+    def _check_latest_version(self, url, package, package_regex, current_version, ud, d, major_version=None):
         """
         Return the latest version of a package inside a given directory path
         If error or no version, return ""
@@ -515,6 +515,9 @@ class Wget(FetchMethod):
                 newver = self._parse_path(package_regex, str(line))
 
             if newver:
+                if major_version:
+                    if not re.sub('_', '.', newver[1]).startswith(major_version+'.'):
+                        continue
                 bb.debug(3, "Upstream version found: %s" % newver[1])
                 if valid == 0:
                     version = newver
@@ -532,7 +535,7 @@ class Wget(FetchMethod):
 
         return ""
 
-    def _check_latest_version_by_dir(self, dirver, package, package_regex, current_version, ud, d):
+    def _check_latest_version_by_dir(self, dirver, package, package_regex, current_version, ud, d, major_version=None):
         """
         Scan every directory in order to get upstream version.
         """
@@ -576,7 +579,7 @@ class Wget(FetchMethod):
                         ud.user, ud.pswd, {}])
 
                     pupver = self._check_latest_version(uri,
-                            package, package_regex, current_version, ud, d)
+                            package, package_regex, current_version, ud, d, major_version=major_version)
                     if pupver:
                         version[1] = pupver
 
@@ -637,7 +640,7 @@ class Wget(FetchMethod):
 
         return package_custom_regex_comp
 
-    def latest_versionstring(self, ud, d):
+    def latest_versionstring(self, ud, d, major_version=None):
         """
         Manipulate the URL and try to obtain the latest package version
 
@@ -678,11 +681,11 @@ class Wget(FetchMethod):
                 dirver_pn_regex = re.compile(r"%s\d?" % (re.escape(pn)))
                 if not dirver_pn_regex.search(dirver):
                     return (self._check_latest_version_by_dir(dirver,
-                        package, package_regex, current_version, ud, d), '')
+                        package, package_regex, current_version, ud, d, major_version=major_version), '')
 
             uri = bb.fetch.encodeurl([ud.type, ud.host, path, ud.user, ud.pswd, {}])
         else:
             uri = regex_uri
 
         return (self._check_latest_version(uri, package, package_regex,
-                current_version, ud, d), '')
+                current_version, ud, d, major_version=major_version), '')
Signature · 1 line
2.34.1
#2 lore raw

RFC PATCH 3/3 lib/bb/fetch2/crate.py: add optional major_version parameter to latest_versionstring

In reply to [email protected]

From: Chen Qi <[email protected]>

We need to handle the situation of LTS upgrades. This means for version such as 1.0.0, we need to find the upgradable version of 1.0.1 instead of something like 1.1.0.

Add optional major_version parameter to crate fetcher to support such case.

---
 lib/bb/fetch2/crate.py | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
lib/bb/fetch2/crate.py +7 -1
index 2d3078899..4ac71946d 100644
@@ -144,10 +144,16 @@ class Crate(Wget):
             with open(mdpath, "w") as f:
                 json.dump(metadata, f)
 
-    def latest_versionstring(self, ud, d):
+    def latest_versionstring(self, ud, d, major_version=None):
         from functools import cmp_to_key
         json_data = json.loads(self._fetch_index(ud.versionsurl, ud, d))
         versions = [(0, i["num"], "") for i in json_data["versions"]]
+        if major_version:
+            filtered_versions = []
+            for v in versions:
+                if v[1].startswith(major_version+'.'):
+                    filtered_versions.append(v)
+            versions = filtered_versions
         versions = sorted(versions, key=cmp_to_key(bb.utils.vercmp))
 
         return (versions[-1][1], "")
Signature · 1 line
2.34.1
#3 lore raw
In reply to [email protected]
Quoted 1 line · On Wed, 29 Apr 2026 at 05:24, <[email protected]> wrote:

+ def latest_versionstring(self, ud, d, major_version=None):

...

lib/bb/fetch2/git.py class Git(FetchMethod): · line 1021

Quoted 3 lines
lib/bb/fetch2/git.py +3 -0
+            if major_version:
+                if not pver.startswith(major_version+'.'):
+                    continue

This handles one way of versioning bugfix releases (x.y.z -> x.y.z+1), but upstreams might invent all kinds of different ways. E.g. systemd is using x -> x.1 -> x.2 -> ...,
or openssh is using x.y->x.yp1.

Let's make it more general. I'd suggest passing in an optional filter_regex argument that would be matched with a 'candidate' version. Then the most common use case (point releases) could be supported like this:

upstream-stable-releases-point.bbclass:
UPSTREAM_MAJMIN_VERSION_REGEX = ... (set from ${PV}, e.g. 3.4.5 becomes 3\.4) UPSTREAM_STABLE_RELEASES_REGEX = "${MAJMIN_VERSION}\.\d+" (this in turn becomes 3\.4\.\d+)

systemd and openssh on the other hand could set UPSTREAM_STABLE_RELEASES_REGEX directly according to their custom versioning schemas.

Then recipes which are known to do release maintenance (in a way that's compatible with Yocto stable policy) via point releases could inherit that class or set regex directly. And recipeutils in oe-core would pick up the regex from UPSTREAM_STABLE_RELEASES_REGEX and pass it in to the fetcher's latest version function.

Alex

author
#4 lore raw

Hi Alex,

Thanks for your suggestion. I agree with this solution. It will give us better meta data maintenance in the long run. I'll get to it after 5/1 holiday.

Regards,
Qi

-----Original Message-----
From: Alexander Kanavin <[email protected]>
Sent: Wednesday, April 29, 2026 7:15 PM
To: Chen, Qi <[email protected]>
Cc: [email protected]; [email protected]; MacLeod, Randy <[email protected]>
Subject: Re: [bitbake-devel][RFC][PATCH 1/3] lib/bb/fetch2/git.py: add optional major_version parameter to latest_versionstring

Quoted 1 line · On Wed, 29 Apr 2026 at 05:24, <[email protected]> wrote:

+ def latest_versionstring(self, ud, d, major_version=None):

...

lib/bb/fetch2/git.py class Git(FetchMethod): · line 1021

Quoted 3 lines
lib/bb/fetch2/git.py +3 -0
+            if major_version:
+                if not pver.startswith(major_version+'.'):
+                    continue

This handles one way of versioning bugfix releases (x.y.z -> x.y.z+1), but upstreams might invent all kinds of different ways. E.g. systemd is using x -> x.1 -> x.2 -> ..., or openssh is using x.y->x.yp1.

Let's make it more general. I'd suggest passing in an optional filter_regex argument that would be matched with a 'candidate' version. Then the most common use case (point releases) could be supported like this:

upstream-stable-releases-point.bbclass:
UPSTREAM_MAJMIN_VERSION_REGEX = ... (set from ${PV}, e.g. 3.4.5 becomes 3\.4) UPSTREAM_STABLE_RELEASES_REGEX = "${MAJMIN_VERSION}\.\d+" (this in turn becomes 3\.4\.\d+)

systemd and openssh on the other hand could set UPSTREAM_STABLE_RELEASES_REGEX directly according to their custom versioning schemas.

Then recipes which are known to do release maintenance (in a way that's compatible with Yocto stable policy) via point releases could inherit that class or set regex directly. And recipeutils in oe-core would pick up the regex from UPSTREAM_STABLE_RELEASES_REGEX and pass it in to the fetcher's latest version function.

Alex