Skip to contentSkip to conversation

devtool: standard: guess srcrev update mode for gitsm:// recipes too

PATCH Superseded Babanpreet Singh 3 participants 8 messages Jul 23 – Jul 24

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 devtool: standard: guess srcrev update mode for gitsm:// recipes too

'devtool update-recipe' and 'devtool finish' default to guessing the recipe update mode: 'srcrev' when the source tree HEAD sits on the upstream branch (the "checked out another upstream revision" workflow), 'patch' otherwise. The guesser matches SRC_URI entries against a literal 'git://' prefix, so a gitsm:// recipe never has any git URIs from its point of view and always falls into patch mode.

In that case there are no local commits to export either, so checking out a different upstream revision and running update-recipe reports "No patches or files need updating" and silently leaves the recipe's SRCREV untouched, while the same operation on a git:// recipe updates SRCREV. Forcing -m srcrev works, since _update_recipe_srcrev() is scheme-agnostic; only the guess is broken.

Accept gitsm:// URIs the same way e7076f1742 ("devtool: gitsm:// should be handled same as git:// in upgrades") did for the upgrade path, where the same omission was fixed; the guesser itself dates from 9b9733b7d7 (2015), before gitsm handling was a consideration.

Note this changes the guessed mode for existing gitsm:// recipes from 'patch' to 'srcrev' when HEAD is on the upstream branch — the behavior
git:// recipes have had since 2015. Trees carrying local commits still guess 'patch', because their HEAD is not reachable from the upstream branch.

AI-Generated: Uses Claude (claude-sonnet-5)

---
 scripts/lib/devtool/standard.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/lib/devtool/standard.py +1 -1
index b2b27c7ced..2dff62a7ca 100644
@@ -1816,7 +1816,7 @@ def _guess_recipe_update_mode(srctree, rdata):
     """Guess the recipe update mode to use"""
     import bb.process
     src_uri = (rdata.getVar('SRC_URI') or '').split()
-    git_uris = [uri for uri in src_uri if uri.startswith('git://')]
+    git_uris = [uri for uri in src_uri if uri.startswith('git://') or uri.startswith('gitsm://')]
     if not git_uris:
         return 'patch'
     # Just use the first URI for now
Signature · 1 line
2.43.0
#1 lore raw

PATCH 2/2 oeqa/selftest/devtool: cover srcrev update mode guessing for gitsm://

Add test_devtool_update_recipe_gitsm: devtool modify the git-submodule-test recipe, move the source tree to the parent of the pinned revision (an upstream commit, no local changes) while staying on the devtool branch, run 'devtool update-recipe' in the default auto mode, and assert the recipe's SRCREV is updated to the checked-out revision.

Without the preceding fix the mode guesser cannot see gitsm:// URIs, falls into patch mode, reports "No patches or files need updating" and leaves the recipe untouched, and this test fails with:

AssertionError: Missing file changes: [(' M', '.*/git-submodule-test.bb$')]

No existing test exercises the auto->srcrev guess for any URI scheme:
the auto-mode arm of test_devtool_update_recipe_git covers the local-commits->patch direction only.

AI-Generated: Uses Claude (claude-sonnet-5)

---
 meta/lib/oeqa/selftest/cases/devtool.py | 33 +++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
meta/lib/oeqa/selftest/cases/devtool.py +33 -0
index a10eb0c784..6df07f8c5e 100644
@@ -1378,6 +1378,39 @@ class DevtoolUpdateTests(DevtoolBase):
                            ('??', '%s/0002-Add-a-new-file.patch' % relpatchpath)]
         self._check_repo_status(os.path.dirname(recipefile), expected_status)
 
+    def test_devtool_update_recipe_gitsm(self):
+        # Check that auto mode guesses srcrev update mode for a gitsm:// recipe
+        # when HEAD is on the upstream branch, same as it does for git://
+        testrecipe = 'git-submodule-test'
+        bb_vars = get_bb_vars(['FILE', 'SRC_URI', 'SRCREV'], testrecipe)
+        recipefile = bb_vars['FILE']
+        src_uri = bb_vars['SRC_URI']
+        self.assertIn('gitsm://', src_uri, 'This test expects the %s recipe to be a gitsm recipe' % testrecipe)
+        self._check_repo_status(os.path.dirname(recipefile), [])
+        # First, modify a recipe
+        tempdir = tempfile.mkdtemp(prefix='devtoolqa')
+        self.track_for_cleanup(tempdir)
+        self.track_for_cleanup(self.workspacedir)
+        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
+        # Check git repo
+        self._check_src_repo(tempdir)
+        # Move the source tree to the parent of the pinned revision, staying
+        # on the devtool branch: a revision reachable from the upstream branch
+        # with no local commits, i.e. the "check out another upstream revision
+        # to move the recipe" workflow the srcrev guess exists to detect
+        result = runCmd('git rev-parse HEAD~1', cwd=tempdir)
+        prevrev = result.output.strip()
+        runCmd('git reset --hard %s' % prevrev, cwd=tempdir)
+        self.add_command_to_tearDown('cd %s; git checkout %s' % (os.path.dirname(recipefile), os.path.basename(recipefile)))
+        result = runCmd('devtool update-recipe %s' % testrecipe)
+        expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile))]
+        self._check_repo_status(os.path.dirname(recipefile), expected_status)
+        result = runCmd('git diff %s' % os.path.basename(recipefile), cwd=os.path.dirname(recipefile))
+        addlines = ['SRCREV = "%s"' % prevrev]
+        removelines = ['SRCREV = "%s"' % bb_vars['SRCREV']]
+        self._check_diff(result.output, addlines, removelines)
+
     def test_devtool_update_recipe_append(self):
         # Check preconditions
         testrecipe = 'minicom'
Signature · 1 line
2.43.0
#2 lore raw
In reply to Babanpreet Singh
Quoted 36 lines · Baban via lists.openembedded.org

'devtool update-recipe' and 'devtool finish' default to guessing the recipe update mode: 'srcrev' when the source tree HEAD sits on the upstream branch (the "checked out another upstream revision" workflow), 'patch' otherwise. The guesser matches SRC_URI entries against a literal 'git://' prefix, so a gitsm:// recipe never has any git URIs from its point of view and always falls into patch mode.

In that case there are no local commits to export either, so checking out a different upstream revision and running update-recipe reports "No patches or files need updating" and silently leaves the recipe's SRCREV untouched, while the same operation on a git:// recipe updates SRCREV. Forcing -m srcrev works, since _update_recipe_srcrev() is scheme-agnostic; only the guess is broken.

Accept gitsm:// URIs the same way e7076f1742 ("devtool: gitsm:// should be handled same as git:// in upgrades") did for the upgrade path, where the same omission was fixed; the guesser itself dates from 9b9733b7d7 (2015), before gitsm handling was a consideration.

Note this changes the guessed mode for existing gitsm:// recipes from 'patch' to 'srcrev' when HEAD is on the upstream branch — the behavior
git:// recipes have had since 2015. Trees carrying local commits still guess 'patch', because their HEAD is not reachable from the upstream branch.

AI-Generated: Uses Claude (claude-sonnet-5)

---
 scripts/lib/devtool/standard.py | 2 +-

 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/lib/devtool/standard.py +0 -0
index b2b27c7ced..2dff62a7ca 100644
@@ -1816,7 +1816,7 @@ def _guess_recipe_update_mode(srctree, rdata):
Quoted 5 lines

     """Guess the recipe update mode to use"""
     import bb.process
     src_uri = (rdata.getVar('SRC_URI') or '').split()
-    git_uris = [uri for uri in src_uri if uri.startswith('git://')]
+    git_uris = [uri for uri in src_uri if uri.startswith('git://') or uri.startswith('gitsm://')]

There is a slightly neater python syntax for that:

    git_uris = [uri for uri in src_uri if uri.startswith(('git://', 'gitsm://'))]

as startswith can take a tuple.

Cheers,

Richard

#3 lore raw
In reply to Babanpreet Singh
Quoted 25 lines · Baban via lists.openembedded.org

'devtool update-recipe' and 'devtool finish' default to guessing the recipe update mode: 'srcrev' when the source tree HEAD sits on the upstream branch (the "checked out another upstream revision" workflow), 'patch' otherwise. The guesser matches SRC_URI entries against a literal 'git://' prefix, so a gitsm:// recipe never has any git URIs from its point of view and always falls into patch mode.

In that case there are no local commits to export either, so checking out a different upstream revision and running update-recipe reports "No patches or files need updating" and silently leaves the recipe's SRCREV untouched, while the same operation on a git:// recipe updates SRCREV. Forcing -m srcrev works, since _update_recipe_srcrev() is scheme-agnostic; only the guess is broken.

Accept gitsm:// URIs the same way e7076f1742 ("devtool: gitsm:// should be handled same as git:// in upgrades") did for the upgrade path, where the same omission was fixed; the guesser itself dates from 9b9733b7d7 (2015), before gitsm handling was a consideration.

Note this changes the guessed mode for existing gitsm:// recipes from 'patch' to 'srcrev' when HEAD is on the upstream branch — the behavior
git:// recipes have had since 2015. Trees carrying local commits still guess 'patch', because their HEAD is not reachable from the upstream branch.

Quoted 3 lines

AI-Generated: Uses Claude (claude-sonnet-5)

---

Hi Babanpreet,

Thanks for your patch.

It looks like this is breaking some selftests:

2026-07-23 11:11:54,785 - oe-selftest - INFO - devtool.DevtoolUpdateTests.test_devtool_git_submodules (subunit.RemotedTestCase)
2026-07-23 11:11:54,786 - oe-selftest - INFO -  ... FAIL
...
2026-07-23 11:11:54,786 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/devtool.py", line 1960, in test_devtool_git_submodules
    self._check_repo_status(recipedir, expected_status)
  File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/devtool.py", line 140, in _check_repo_status
    self.fail('Missing file changes: %s' % expected_status)
  File "/usr/lib/python3.11/unittest/case.py", line 703, in fail
    raise self.failureException(msg)
AssertionError: Missing file changes: [(' M', '.*/vulkan-samples_git.bb$'), ('??', '.*/vulkan/vulkan-samples/$')]
...
2026-07-23 11:53:42,632 - oe-selftest - INFO - devtool.DevtoolUpgradeTests.test_devtool_finish_update_patch (subunit.RemotedTestCase)
2026-07-23 11:53:42,632 - oe-selftest - INFO -  ... FAIL
...
2026-07-23 12:43:54,062 - oe-selftest - INFO - devtool.DevtoolModifyTests.test_devtool_modify_nested_gitsm (subunit.RemotedTestCase)
2026-07-23 12:43:54,062 - oe-selftest - INFO -  ... FAIL

https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/4355
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/4355

Can you have a look at the issue?

Thanks,
Mathieu

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#6 lore raw

PATCH v2 1/2 devtool: standard: guess srcrev update mode for gitsm:// recipes too

In reply to Babanpreet Singh

'devtool update-recipe' and 'devtool finish' default to guessing the recipe update mode: 'srcrev' when the source tree HEAD sits on the upstream branch (the "checked out another upstream revision" workflow), 'patch' otherwise. The guesser matches SRC_URI entries against a literal 'git://' prefix, so a gitsm:// recipe never has any git URIs from its point of view and always falls into patch mode.

In that case there are no local commits to export either, so checking out a different upstream revision and running update-recipe reports "No patches or files need updating" and silently leaves the recipe's SRCREV untouched, while the same operation on a git:// recipe updates SRCREV. Forcing -m srcrev works, since _update_recipe_srcrev() is scheme-agnostic; only the guess is broken.

Accept gitsm:// URIs the same way e7076f1742 ("devtool: gitsm:// should be handled same as git:// in upgrades") did for the upgrade path, where the same omission was fixed; the guesser itself dates from 9b9733b7d7 (2015), before gitsm handling was a consideration.

Unlike git://, a gitsm:// tree can carry exportable changes that do not move the parent HEAD off the upstream branch: local commits in a submodule only show up as an out-of-sync gitlink in the parent. Guessing srcrev there would silently drop them, because _update_recipe_srcrev() only rewrites the parent SRCREV — this is exactly the scenario test_devtool_git_submodules exercises (commit inside a submodule, then devtool finish), which caught v1 of this patch on the autobuilder. So only guess srcrev when every submodule checkout matches the revision its parent records: any '+' (checkout differs) or 'U' (merge conflicts) entry in 'git submodule status --recursive' output keeps the current patch mode. This also covers a tree where another parent revision was checked out without a following 'git submodule update': srcrev must never be guessed when it could lose submodule content, and a fully synced tree is the only state where it provably cannot.

Note this changes the guessed mode for existing gitsm:// recipes from 'patch' to 'srcrev' when HEAD is on the upstream branch and the submodules are in sync — the behavior git:// recipes have had since 2015. Trees carrying local commits still guess 'patch', because their HEAD is not reachable from the upstream branch.

AI-Generated: Uses Claude (claude-sonnet-5)

---
v1 -> v2:
- Use the startswith(('git://', 'gitsm://')) tuple form (Richard Purdie)
- Only guess srcrev when 'git submodule status --recursive' reports every submodule in sync: local commits in a submodule do not move the parent HEAD off the upstream branch, and v1 silently dropped them by guessing srcrev — caught by test_devtool_git_submodules on the autobuilder (reported by Mathieu Dubois-Briand). Of the three selftest failures reported against v1, this one is the only one this series causes: test_devtool_modify_nested_gitsm fails during 'devtool modify' source extraction before the guesser can run, and test_devtool_finish_update_patch exercises a recipe with only git://
URIs, for which this change is inert; both pass on master with only this series applied (analysis in the reply on the v1 thread)

 scripts/lib/devtool/standard.py | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
scripts/lib/devtool/standard.py +9 -2
index b2b27c7ced..9b2643604d 100644
@@ -1816,7 +1816,7 @@ def _guess_recipe_update_mode(srctree, rdata):
     """Guess the recipe update mode to use"""
     import bb.process
     src_uri = (rdata.getVar('SRC_URI') or '').split()
-    git_uris = [uri for uri in src_uri if uri.startswith('git://')]
+    git_uris = [uri for uri in src_uri if uri.startswith(('git://', 'gitsm://'))]
     if not git_uris:
         return 'patch'
     # Just use the first URI for now
@@ -1831,7 +1831,14 @@ def _guess_recipe_update_mode(srctree, rdata):
                                cwd=srctree)
     remote_brs = [branch.strip() for branch in stdout.splitlines()]
     if 'origin/' + upstr_branch in remote_brs:
-        return 'srcrev'
+        # Local commits in a submodule need exporting as patches, but do not
+        # move the parent HEAD off the upstream branch, so only guess srcrev
+        # if every submodule checkout matches the revision its parent records
+        # ('+') and none has merge conflicts ('U')
+        stdout, _ = bb.process.run('git submodule status --recursive',
+                                   cwd=srctree)
+        if not any(line.startswith(('+', 'U')) for line in stdout.splitlines()):
+            return 'srcrev'
 
     return 'patch'
 
Signature · 1 line
2.43.0
#7 lore raw

PATCH v2 2/2 oeqa/selftest/devtool: cover srcrev update mode guessing for gitsm://

Add test_devtool_update_recipe_gitsm: devtool modify the git-submodule-test recipe, move the source tree to the parent of the pinned revision (an upstream commit, no local changes) while staying on the devtool branch, sync the submodules to it with 'git submodule update', run 'devtool update-recipe' in the default auto mode, and assert the recipe's SRCREV is updated to the checked-out revision.

Without the preceding fix the mode guesser cannot see gitsm:// URIs, falls into patch mode, reports "No patches or files need updating" and leaves the recipe untouched, and this test fails with:

AssertionError: Missing file changes: [(' M', '.*/git-submodule-test.bb$')]

The 'git submodule update' step matters: the pinned revision of git-submodule-test happens to bump a gitlink, so after checking out its parent the submodule no longer matches the revision the parent records, and the guesser deliberately stays in patch mode for such trees.

The other direction of the guess — local commits inside a submodule must keep patch mode, because they leave the parent HEAD on the upstream branch while a srcrev update would silently drop them — is already covered by the vulkan-samples based test_devtool_git_submodules, which is what caught exactly that regression in v1 of this series on the autobuilder, so no test is added for it.

No existing test exercises the auto->srcrev guess for any URI scheme:
the auto-mode arm of test_devtool_update_recipe_git covers the local-commits->patch direction only.

AI-Generated: Uses Claude (claude-sonnet-5)

---
v1 -> v2:
- Run 'git submodule update --recursive' after moving the source tree:
the pinned git-submodule-test revision bumps a gitlink, so without it the tree now legitimately stays in patch mode under the v2 guesser
- No new test for the submodule-local-commit direction: it is already covered by test_devtool_git_submodules, which caught the v1 regression on the autobuilder and passes with this series

 meta/lib/oeqa/selftest/cases/devtool.py | 38 +++++++++++++++++++++++++
 1 file changed, 38 insertions(+)
meta/lib/oeqa/selftest/cases/devtool.py +38 -0
index a10eb0c784..b138a5ef6f 100644
@@ -1378,6 +1378,44 @@ class DevtoolUpdateTests(DevtoolBase):
                            ('??', '%s/0002-Add-a-new-file.patch' % relpatchpath)]
         self._check_repo_status(os.path.dirname(recipefile), expected_status)
 
+    def test_devtool_update_recipe_gitsm(self):
+        # Check that auto mode guesses srcrev update mode for a gitsm:// recipe
+        # when HEAD is on the upstream branch, same as it does for git://
+        testrecipe = 'git-submodule-test'
+        bb_vars = get_bb_vars(['FILE', 'SRC_URI', 'SRCREV'], testrecipe)
+        recipefile = bb_vars['FILE']
+        src_uri = bb_vars['SRC_URI']
+        self.assertIn('gitsm://', src_uri, 'This test expects the %s recipe to be a gitsm recipe' % testrecipe)
+        self._check_repo_status(os.path.dirname(recipefile), [])
+        # First, modify a recipe
+        tempdir = tempfile.mkdtemp(prefix='devtoolqa')
+        self.track_for_cleanup(tempdir)
+        self.track_for_cleanup(self.workspacedir)
+        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
+        # Check git repo
+        self._check_src_repo(tempdir)
+        # Move the source tree to the parent of the pinned revision, staying
+        # on the devtool branch: a revision reachable from the upstream branch
+        # with no local commits, i.e. the "check out another upstream revision
+        # to move the recipe" workflow the srcrev guess exists to detect
+        result = runCmd('git rev-parse HEAD~1', cwd=tempdir)
+        prevrev = result.output.strip()
+        runCmd('git reset --hard %s' % prevrev, cwd=tempdir)
+        # Sync the submodules to the newly checked out revision: with any
+        # submodule checkout not matching the revision the parent records,
+        # there is potential submodule content to export and the guesser
+        # legitimately stays in patch mode
+        runCmd('git submodule update --recursive', cwd=tempdir)
+        self.add_command_to_tearDown('cd %s; git checkout %s' % (os.path.dirname(recipefile), os.path.basename(recipefile)))
+        result = runCmd('devtool update-recipe %s' % testrecipe)
+        expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile))]
+        self._check_repo_status(os.path.dirname(recipefile), expected_status)
+        result = runCmd('git diff %s' % os.path.basename(recipefile), cwd=os.path.dirname(recipefile))
+        addlines = ['SRCREV = "%s"' % prevrev]
+        removelines = ['SRCREV = "%s"' % bb_vars['SRCREV']]
+        self._check_diff(result.output, addlines, removelines)
+
     def test_devtool_update_recipe_append(self):
         # Check preconditions
         testrecipe = 'minicom'
Signature · 1 line
2.43.0
#4 lore raw
In reply to Mathieu Dubois-Briand

Hi Mathieu,

Thanks for the report. I dug into the three failures on build 4355:
one is a real regression in my patch (fixed in v2), the other two don't look caused by this series.

1) test_devtool_git_submodules: my fault. The test commits inside a submodule, so the parent HEAD stays on the upstream branch and v1 guessed srcrev, which only rewrites the parent SRCREV and so silently dropped the submodule commit. v2 only guesses srcrev when 'git submodule status --recursive' shows every submodule in sync, keeping such trees in patch mode. The test fails here with v1 and passes with v2, matching the autobuilder.

2) test_devtool_modify_nested_gitsm fails during 'devtool modify' source extraction ("No url found for submodule path 'nested/repo-gitsm' in .gitmodules"), before the update-mode guesser (the only thing this series touches) can run.

3) test_devtool_finish_update_patch: sysdig-selftest has no gitsm:// URIs, so the change is inert for it.

Both 2) and 3) pass here on master with only this series applied. Build 4355 tested mathieu/master-next, and the nested-git destsuffix rework in that queue touches exactly the machinery both failures point at, so I suspect an interaction with that series. Happy to be corrected if they still fail without other devtool changes in the queue.

v2 also adopts Richard's startswith-tuple suggestion.

Thanks,
Baban

#5 lore raw
Quoted 29 lines · Babanpreet Singh

Hi Mathieu,

Thanks for the report. I dug into the three failures on build 4355:
one is a real regression in my patch (fixed in v2), the other two don't look caused by this series.

1) test_devtool_git_submodules: my fault. The test commits inside a submodule, so the parent HEAD stays on the upstream branch and v1 guessed srcrev, which only rewrites the parent SRCREV and so silently dropped the submodule commit. v2 only guesses srcrev when 'git submodule status --recursive' shows every submodule in sync, keeping such trees in patch mode. The test fails here with v1 and passes with v2, matching the autobuilder.

2) test_devtool_modify_nested_gitsm fails during 'devtool modify' source extraction ("No url found for submodule path 'nested/repo-gitsm' in .gitmodules"), before the update-mode guesser (the only thing this series touches) can run.

3) test_devtool_finish_update_patch: sysdig-selftest has no gitsm:// URIs, so the change is inert for it.

Both 2) and 3) pass here on master with only this series applied. Build 4355 tested mathieu/master-next, and the nested-git destsuffix rework in that queue touches exactly the machinery both failures point at, so I suspect an interaction with that series. Happy to be corrected if they still fail without other devtool changes in the queue.

Quoted 4 lines

v2 also adopts Richard's startswith-tuple suggestion.

Thanks,
Baban

Sorry, I came to the same conclusion this morning: I wrongfully assumed that because this commit was responsible of the first fail, it was also responsible of the two others. This is confirmed by tonight build, it failed even without your commit.

Thanks for taking the time to investigate.

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com