Skip to contentSkip to conversation

devtool: upgrade: extract changelog between versions

PATCH Superseded [email protected] 2 participants 3 messages May 13

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 devtool: upgrade: extract changelog between versions

From: Daniel Turull <[email protected]>

Automatically extract changelog information when upgrading a recipe. Uses the devtool-base tags created during upgrade to diff known changelog files (NEWS, ChangeLog, CHANGES, etc.) between the old and new versions. For git-based sources, falls back to git log --oneline if no changelog file changed.

Also match per-version release notes files like git's Documentation/RelNotes/2.53.0.adoc or mesa's docs/relnotes/26.0.3.rst.

These are detected by matching filenames that contain a version number with a documentation extension (.txt, .md, .rst, .adoc). Their full content is read via git show since they are new files rather than modifications to an existing changelog.

Collapse runs of multiple blank lines into a single blank line and strip leading/trailing whitespace from the extracted changelog. This produces cleaner output that is suitable for inclusion in commit messages by tools like AUH.

Output is written to workspace/changelogs/<pn>.txt and cleaned up on devtool reset. This allows AUH and other tools to pick up the changelog without implementing their own extraction logic.

Add test cases for existings test recipes

Assisted-by kiro:claude-opus-4.6

---

v2:
- squash v1 patches into one commit and keep commit messages
- Integrate test cases with exiting devtool selftests

---
 .../python3-guessing-game_git.bb.changelog    |  3 +
 .../devtool-upgrade-test1_1.5.3.bb.changelog  | 24 +++++
 .../devtool-upgrade-test2_git.bb.changelog    |  3 +
 .../devtool-upgrade-test3_1.5.3.bb.changelog  | 24 +++++
 .../devtool-upgrade-test4_1.5.3.bb.changelog  | 24 +++++
 .../devtool-upgrade-test5_git.bb.changelog    |  3 +
 meta/lib/oeqa/selftest/cases/devtool.py       | 28 ++++++
 scripts/lib/devtool/standard.py               |  8 ++
 scripts/lib/devtool/upgrade.py                | 95 +++++++++++++++++++
 9 files changed, 212 insertions(+)
 create mode 100644 meta-selftest/recipes-devtools/python/python3-guessing-game_git.bb.changelog
 create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test1_1.5.3.bb.changelog
 create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test2_git.bb.changelog
 create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.changelog
 create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.changelog
 create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test5_git.bb.changelog
meta-selftest/recipes-devtools/python/python3-guessing-game_git.bb.changelog new file +3 -0
new file mode 100644
index 0000000000..571220b2a4
@@ -0,0 +1,3 @@
+Changelog for python3-guessing-game: 0.1.0 -> 0.2.0
+
+40cf004 Sync with maturin tutorial source
meta-selftest/recipes-test/devtool/devtool-upgrade-test1_1.5.3.bb.changelog new file +24 -0
new file mode 100644
index 0000000000..04d9be5103
@@ -0,0 +1,24 @@
+Changelog for devtool-upgrade-test1: 1.5.3 -> 1.6.0
+
+1.6.0 - 15 March 2015
+  - fix lstat64 support when unavailable - separate patches supplied by
+    Ganael Laplanche and Peter Korsgaard
+  - (#1506) new option "-D" / "--delay-start" to only show bar after N
+    seconds (Damon Harper)
+  - new option "--fineta" / "-I" to show ETA as time of day rather than time
+    remaining - patch supplied by Erkki Seppälä (r147)
+  - (#1509) change ETA (--eta / -e) so that days are given if the hours
+    remaining are 24 or more (Jacek Wielemborek)
+  - (#1499) repeat read and write attempts on partial buffer fill/empty to
+    work around post-signal transfer rate drop reported by Ralf Ramsauer
+  - (#1507) do not try to calculate total size in line mode, due to bug
+    reported by Jacek Wielemborek and Michiel Van Herwegen
+  - cleanup: removed defunct RATS comments and unnecessary copyright notices
+  - clean up displayed lines when using --watchfd PID, when PID exits
+  - output errors on a new line to avoid overwriting transfer bar
+
+1.5.7 - 26 August 2014
+  - show KiB instead of incorrect kiB (Debian bug #706175)
+  - (#1284) do not gzip man page, for non-Linux OSes (Bob Friesenhahn)
+  - work around "awk" bug in tests/016-numeric-timer in decimal "," locales
+  - fix "make rpm" and "make srpm", extend "make release" to sign releases
meta-selftest/recipes-test/devtool/devtool-upgrade-test2_git.bb.changelog new file +3 -0
new file mode 100644
index 0000000000..b7f0019d56
@@ -0,0 +1,3 @@
+Changelog for devtool-upgrade-test2: 0.1+git -> 0.1+git
+
+6cc6077 dbus-wait.c: Fix typo
meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.changelog new file +24 -0
new file mode 100644
index 0000000000..d9994fe063
@@ -0,0 +1,24 @@
+Changelog for devtool-upgrade-test3: 1.5.3 -> 1.6.0
+
+1.6.0 - 15 March 2015
+  - fix lstat64 support when unavailable - separate patches supplied by
+    Ganael Laplanche and Peter Korsgaard
+  - (#1506) new option "-D" / "--delay-start" to only show bar after N
+    seconds (Damon Harper)
+  - new option "--fineta" / "-I" to show ETA as time of day rather than time
+    remaining - patch supplied by Erkki Seppälä (r147)
+  - (#1509) change ETA (--eta / -e) so that days are given if the hours
+    remaining are 24 or more (Jacek Wielemborek)
+  - (#1499) repeat read and write attempts on partial buffer fill/empty to
+    work around post-signal transfer rate drop reported by Ralf Ramsauer
+  - (#1507) do not try to calculate total size in line mode, due to bug
+    reported by Jacek Wielemborek and Michiel Van Herwegen
+  - cleanup: removed defunct RATS comments and unnecessary copyright notices
+  - clean up displayed lines when using --watchfd PID, when PID exits
+  - output errors on a new line to avoid overwriting transfer bar
+
+1.5.7 - 26 August 2014
+  - show KiB instead of incorrect kiB (Debian bug #706175)
+  - (#1284) do not gzip man page, for non-Linux OSes (Bob Friesenhahn)
+  - work around "awk" bug in tests/016-numeric-timer in decimal "," locales
+  - fix "make rpm" and "make srpm", extend "make release" to sign releases
meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.changelog new file +24 -0
new file mode 100644
index 0000000000..3a37271eb7
@@ -0,0 +1,24 @@
+Changelog for devtool-upgrade-test4: 1.5.3 -> 1.6.0
+
+1.6.0 - 15 March 2015
+  - fix lstat64 support when unavailable - separate patches supplied by
+    Ganael Laplanche and Peter Korsgaard
+  - (#1506) new option "-D" / "--delay-start" to only show bar after N
+    seconds (Damon Harper)
+  - new option "--fineta" / "-I" to show ETA as time of day rather than time
+    remaining - patch supplied by Erkki Seppälä (r147)
+  - (#1509) change ETA (--eta / -e) so that days are given if the hours
+    remaining are 24 or more (Jacek Wielemborek)
+  - (#1499) repeat read and write attempts on partial buffer fill/empty to
+    work around post-signal transfer rate drop reported by Ralf Ramsauer
+  - (#1507) do not try to calculate total size in line mode, due to bug
+    reported by Jacek Wielemborek and Michiel Van Herwegen
+  - cleanup: removed defunct RATS comments and unnecessary copyright notices
+  - clean up displayed lines when using --watchfd PID, when PID exits
+  - output errors on a new line to avoid overwriting transfer bar
+
+1.5.7 - 26 August 2014
+  - show KiB instead of incorrect kiB (Debian bug #706175)
+  - (#1284) do not gzip man page, for non-Linux OSes (Bob Friesenhahn)
+  - work around "awk" bug in tests/016-numeric-timer in decimal "," locales
+  - fix "make rpm" and "make srpm", extend "make release" to sign releases
meta-selftest/recipes-test/devtool/devtool-upgrade-test5_git.bb.changelog new file +3 -0
new file mode 100644
index 0000000000..7c5ecf3505
@@ -0,0 +1,3 @@
+Changelog for devtool-upgrade-test5: 0.1+git -> 0.1+git
+
+0a60d6a Add dummy commit on tip for testing
meta/lib/oeqa/selftest/cases/devtool.py +28 -0
index 5ed69aee1b..40c5d619dd 100644
@@ -1944,6 +1944,23 @@ class DevtoolUpgradeTests(DevtoolBase):
         except:
             self.skip("Git user.name and user.email must be set")
 
+    def _check_changelog(self, recipe, oldrecipefile):
+        """Compare extracted changelog against reference data if available."""
+        changelog_ref = oldrecipefile + '.changelog'
+        if not os.path.exists(changelog_ref):
+            return
+        changelog_file = os.path.join(self.workspacedir, 'changelogs', '%s.txt' % recipe)
+        with open(changelog_ref, 'r') as f:
+            expected = f.read()
+        if not expected:
+            self.assertNotExists(changelog_file,
+                'Changelog file should not exist when reference is empty')
+        else:
+            self.assertExists(changelog_file, 'Changelog file should exist after upgrade')
+            with open(changelog_file, 'r') as f:
+                actual = f.read()
+            self.assertEqual(expected, actual)
+
     def test_devtool_upgrade(self):
         # Check preconditions
         self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory')
@@ -1982,6 +1999,8 @@ class DevtoolUpgradeTests(DevtoolBase):
         with open(newrecipefile, 'r') as f:
             newlines = f.readlines()
         self.assertEqual(desiredlines, newlines)
+        # Check changelog
+        self._check_changelog(recipe, oldrecipefile)
         # Check devtool reset recipe
         result = runCmd('devtool reset %s -n' % recipe)
         result = runCmd('devtool status')
@@ -2016,11 +2035,14 @@ class DevtoolUpgradeTests(DevtoolBase):
         with open(newrecipefile, 'r') as f:
             newlines = f.readlines()
         self.assertEqual(desiredlines, newlines)
+        # Check changelog
+        self._check_changelog(recipe, oldrecipefile)
         # Check devtool reset recipe
         result = runCmd('devtool reset %s -n' % recipe)
         result = runCmd('devtool status')
         self.assertNotIn(recipe, result.output)
         self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after resetting')
+        self.assertNotExists(os.path.join(self.workspacedir, 'changelogs', '%s.txt' % recipe), 'Changelog file should be removed after reset')
 
     def test_devtool_upgrade_git(self):
         self._test_devtool_upgrade_git_by_recipe('devtool-upgrade-test2', '6cc6077a36fe2648a5f993fe7c16c9632f946517')
@@ -2051,6 +2073,8 @@ class DevtoolUpgradeTests(DevtoolBase):
         with open(newrecipefile, 'r') as f:
             newlines = f.readlines()
         self.assertEqual(desiredlines, newlines)
+        # Check changelog
+        self._check_changelog(recipe, oldrecipefile)
 
     def test_devtool_upgrade_all_checksums(self):
         # Check preconditions
@@ -2075,6 +2099,8 @@ class DevtoolUpgradeTests(DevtoolBase):
         with open(newrecipefile, 'r') as f:
             newlines = f.readlines()
         self.assertEqual(desiredlines, newlines)
+        # Check changelog
+        self._check_changelog(recipe, oldrecipefile)
 
     def test_devtool_upgrade_recipe_upgrade_extra_tasks(self):
         # Check preconditions
@@ -2116,6 +2142,8 @@ class DevtoolUpgradeTests(DevtoolBase):
         with open(newcratesincfile, 'r') as f:
             newlines = f.readlines()
         self.assertEqual(desiredlines, newlines)
+        # Check changelog
+        self._check_changelog(recipe, oldrecipefile)
         # Check devtool reset recipe
         result = runCmd('devtool reset %s -n' % recipe)
         result = runCmd('devtool status')
scripts/lib/devtool/standard.py +8 -0
index 42fb13872d..2a3a62d081 100644
@@ -2046,6 +2046,14 @@ def _reset(recipes, no_clean, remove_work, config, basepath, workspace):
 
         clean_preferred_provider(pn, config.workspace_path)
 
+        # Clean up changelog if present
+        changelog_file = os.path.join(config.workspace_path, 'changelogs', '%s.txt' % pn)
+        if os.path.exists(changelog_file):
+            os.remove(changelog_file)
+            changelog_dir = os.path.dirname(changelog_file)
+            if not os.listdir(changelog_dir):
+                os.rmdir(changelog_dir)
+
 def reset(args, config, basepath, workspace):
     """Entry point for the devtool 'reset' subcommand"""
 
scripts/lib/devtool/upgrade.py +95 -0

119 lines not on this page · raw message

Signature · 1 line
2.51.0
#1 lore raw

meta/lib/oeqa/selftest/cases/devtool.py class DevtoolUpgradeTests(DevtoolBase): · line 1951

Quoted 5 lines · On Wed, 13 May 2026 at 14:20, <[email protected]> wrote:
meta/lib/oeqa/selftest/cases/devtool.py +5 -0
+    def _check_changelog(self, recipe, oldrecipefile):
+        """Compare extracted changelog against reference data if available."""
+        changelog_ref = oldrecipefile + '.changelog'
+        if not os.path.exists(changelog_ref):
+            return

I think these last two lines should be removed, so that the changelog check is enforced. Otherwise it's way too easy to regress the test, e.g. by replacing the test recipes, or adding new recipes without associated changelogs, or with changelog files that are slightly misnamed.

Other than this, lgtm, thanks!

Alex

#2 lore raw
Quoted 19 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Alexander Kanavin via lists.openembedded.org
Sent: Wednesday, 13 May 2026 14:32
To: Daniel Turull <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH v2] devtool: upgrade: extract changelog between versions

meta/lib/oeqa/selftest/cases/devtool.py class DevtoolUpgradeTests(DevtoolBase): · line 1951

Quoted 5 lines · On Wed, 13 May 2026 at 14:20, <[email protected]> wrote:
meta/lib/oeqa/selftest/cases/devtool.py +5 -0
+    def _check_changelog(self, recipe, oldrecipefile):
+        """Compare extracted changelog against reference data if available."""
+        changelog_ref = oldrecipefile + '.changelog'
+        if not os.path.exists(changelog_ref):
+            return
Quoted 5 lines

I think these last two lines should be removed, so that the changelog check is enforced. Otherwise it's way too easy to regress the test, e.g. by replacing the test recipes, or adding new recipes without associated changelogs, or with changelog files that are slightly misnamed.

Ok. I'll drop them and send a v3 with it. Thanks for the quick feedback Alex.

Quoted 3 lines

Other than this, lgtm, thanks!

Alex