devtool: upgrade: extract changelog between versions
PATCH v2 1/1 devtool: upgrade: extract changelog between versions
From: Daniel Turull <[email protected]>
Automatically extract changelog information when upgrading a recipe. Uses the devtool-base tags created during upgrade to diff known changelog files (NEWS, ChangeLog, CHANGES, etc.) between the old and new versions. For git-based sources, falls back to git log --oneline if no changelog file changed.
Also match per-version release notes files like git's Documentation/RelNotes/2.53.0.adoc or mesa's docs/relnotes/26.0.3.rst.
These are detected by matching filenames that contain a version number with a documentation extension (.txt, .md, .rst, .adoc). Their full content is read via git show since they are new files rather than modifications to an existing changelog.
Collapse runs of multiple blank lines into a single blank line and strip leading/trailing whitespace from the extracted changelog. This produces cleaner output that is suitable for inclusion in commit messages by tools like AUH.
Output is written to workspace/changelogs/<pn>.txt and cleaned up on devtool reset. This allows AUH and other tools to pick up the changelog without implementing their own extraction logic.
Add test cases for existings test recipes
---
v2:
- squash v1 patches into one commit and keep commit messages
- Integrate test cases with exiting devtool selftests
--- .../python3-guessing-game_git.bb.changelog | 3 + .../devtool-upgrade-test1_1.5.3.bb.changelog | 24 +++++ .../devtool-upgrade-test2_git.bb.changelog | 3 + .../devtool-upgrade-test3_1.5.3.bb.changelog | 24 +++++ .../devtool-upgrade-test4_1.5.3.bb.changelog | 24 +++++ .../devtool-upgrade-test5_git.bb.changelog | 3 + meta/lib/oeqa/selftest/cases/devtool.py | 28 ++++++ scripts/lib/devtool/standard.py | 8 ++ scripts/lib/devtool/upgrade.py | 95 +++++++++++++++++++ 9 files changed, 212 insertions(+) create mode 100644 meta-selftest/recipes-devtools/python/python3-guessing-game_git.bb.changelog create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test1_1.5.3.bb.changelog create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test2_git.bb.changelog create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.changelog create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.changelog create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test5_git.bb.changelog
meta-selftest/recipes-devtools/python/python3-guessing-game_git.bb.changelog new file +3 -0
@@ -0,0 +1,3 @@ +Changelog for python3-guessing-game: 0.1.0 -> 0.2.0 + +40cf004 Sync with maturin tutorial source
meta-selftest/recipes-test/devtool/devtool-upgrade-test1_1.5.3.bb.changelog new file +24 -0
@@ -0,0 +1,24 @@ +Changelog for devtool-upgrade-test1: 1.5.3 -> 1.6.0 + +1.6.0 - 15 March 2015 + - fix lstat64 support when unavailable - separate patches supplied by + Ganael Laplanche and Peter Korsgaard + - (#1506) new option "-D" / "--delay-start" to only show bar after N + seconds (Damon Harper) + - new option "--fineta" / "-I" to show ETA as time of day rather than time + remaining - patch supplied by Erkki Seppälä (r147) + - (#1509) change ETA (--eta / -e) so that days are given if the hours + remaining are 24 or more (Jacek Wielemborek) + - (#1499) repeat read and write attempts on partial buffer fill/empty to + work around post-signal transfer rate drop reported by Ralf Ramsauer + - (#1507) do not try to calculate total size in line mode, due to bug + reported by Jacek Wielemborek and Michiel Van Herwegen + - cleanup: removed defunct RATS comments and unnecessary copyright notices + - clean up displayed lines when using --watchfd PID, when PID exits + - output errors on a new line to avoid overwriting transfer bar + +1.5.7 - 26 August 2014 + - show KiB instead of incorrect kiB (Debian bug #706175) + - (#1284) do not gzip man page, for non-Linux OSes (Bob Friesenhahn) + - work around "awk" bug in tests/016-numeric-timer in decimal "," locales + - fix "make rpm" and "make srpm", extend "make release" to sign releases
meta-selftest/recipes-test/devtool/devtool-upgrade-test2_git.bb.changelog new file +3 -0
@@ -0,0 +1,3 @@ +Changelog for devtool-upgrade-test2: 0.1+git -> 0.1+git + +6cc6077 dbus-wait.c: Fix typo
meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.changelog new file +24 -0
@@ -0,0 +1,24 @@ +Changelog for devtool-upgrade-test3: 1.5.3 -> 1.6.0 + +1.6.0 - 15 March 2015 + - fix lstat64 support when unavailable - separate patches supplied by + Ganael Laplanche and Peter Korsgaard + - (#1506) new option "-D" / "--delay-start" to only show bar after N + seconds (Damon Harper) + - new option "--fineta" / "-I" to show ETA as time of day rather than time + remaining - patch supplied by Erkki Seppälä (r147) + - (#1509) change ETA (--eta / -e) so that days are given if the hours + remaining are 24 or more (Jacek Wielemborek) + - (#1499) repeat read and write attempts on partial buffer fill/empty to + work around post-signal transfer rate drop reported by Ralf Ramsauer + - (#1507) do not try to calculate total size in line mode, due to bug + reported by Jacek Wielemborek and Michiel Van Herwegen + - cleanup: removed defunct RATS comments and unnecessary copyright notices + - clean up displayed lines when using --watchfd PID, when PID exits + - output errors on a new line to avoid overwriting transfer bar + +1.5.7 - 26 August 2014 + - show KiB instead of incorrect kiB (Debian bug #706175) + - (#1284) do not gzip man page, for non-Linux OSes (Bob Friesenhahn) + - work around "awk" bug in tests/016-numeric-timer in decimal "," locales + - fix "make rpm" and "make srpm", extend "make release" to sign releases
meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.changelog new file +24 -0
@@ -0,0 +1,24 @@ +Changelog for devtool-upgrade-test4: 1.5.3 -> 1.6.0 + +1.6.0 - 15 March 2015 + - fix lstat64 support when unavailable - separate patches supplied by + Ganael Laplanche and Peter Korsgaard + - (#1506) new option "-D" / "--delay-start" to only show bar after N + seconds (Damon Harper) + - new option "--fineta" / "-I" to show ETA as time of day rather than time + remaining - patch supplied by Erkki Seppälä (r147) + - (#1509) change ETA (--eta / -e) so that days are given if the hours + remaining are 24 or more (Jacek Wielemborek) + - (#1499) repeat read and write attempts on partial buffer fill/empty to + work around post-signal transfer rate drop reported by Ralf Ramsauer + - (#1507) do not try to calculate total size in line mode, due to bug + reported by Jacek Wielemborek and Michiel Van Herwegen + - cleanup: removed defunct RATS comments and unnecessary copyright notices + - clean up displayed lines when using --watchfd PID, when PID exits + - output errors on a new line to avoid overwriting transfer bar + +1.5.7 - 26 August 2014 + - show KiB instead of incorrect kiB (Debian bug #706175) + - (#1284) do not gzip man page, for non-Linux OSes (Bob Friesenhahn) + - work around "awk" bug in tests/016-numeric-timer in decimal "," locales + - fix "make rpm" and "make srpm", extend "make release" to sign releases
meta-selftest/recipes-test/devtool/devtool-upgrade-test5_git.bb.changelog new file +3 -0
@@ -0,0 +1,3 @@ +Changelog for devtool-upgrade-test5: 0.1+git -> 0.1+git + +0a60d6a Add dummy commit on tip for testing
meta/lib/oeqa/selftest/cases/devtool.py +28 -0
@@ -1944,6 +1944,23 @@ class DevtoolUpgradeTests(DevtoolBase): except: self.skip("Git user.name and user.email must be set") + def _check_changelog(self, recipe, oldrecipefile): + """Compare extracted changelog against reference data if available.""" + changelog_ref = oldrecipefile + '.changelog' + if not os.path.exists(changelog_ref): + return + changelog_file = os.path.join(self.workspacedir, 'changelogs', '%s.txt' % recipe) + with open(changelog_ref, 'r') as f: + expected = f.read() + if not expected: + self.assertNotExists(changelog_file, + 'Changelog file should not exist when reference is empty') + else: + self.assertExists(changelog_file, 'Changelog file should exist after upgrade') + with open(changelog_file, 'r') as f: + actual = f.read() + self.assertEqual(expected, actual) + def test_devtool_upgrade(self): # Check preconditions self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory')
@@ -1982,6 +1999,8 @@ class DevtoolUpgradeTests(DevtoolBase): with open(newrecipefile, 'r') as f: newlines = f.readlines() self.assertEqual(desiredlines, newlines) + # Check changelog + self._check_changelog(recipe, oldrecipefile) # Check devtool reset recipe result = runCmd('devtool reset %s -n' % recipe) result = runCmd('devtool status')
@@ -2016,11 +2035,14 @@ class DevtoolUpgradeTests(DevtoolBase): with open(newrecipefile, 'r') as f: newlines = f.readlines() self.assertEqual(desiredlines, newlines) + # Check changelog + self._check_changelog(recipe, oldrecipefile) # Check devtool reset recipe result = runCmd('devtool reset %s -n' % recipe) result = runCmd('devtool status') self.assertNotIn(recipe, result.output) self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after resetting') + self.assertNotExists(os.path.join(self.workspacedir, 'changelogs', '%s.txt' % recipe), 'Changelog file should be removed after reset') def test_devtool_upgrade_git(self): self._test_devtool_upgrade_git_by_recipe('devtool-upgrade-test2', '6cc6077a36fe2648a5f993fe7c16c9632f946517')
@@ -2051,6 +2073,8 @@ class DevtoolUpgradeTests(DevtoolBase): with open(newrecipefile, 'r') as f: newlines = f.readlines() self.assertEqual(desiredlines, newlines) + # Check changelog + self._check_changelog(recipe, oldrecipefile) def test_devtool_upgrade_all_checksums(self): # Check preconditions
@@ -2075,6 +2099,8 @@ class DevtoolUpgradeTests(DevtoolBase): with open(newrecipefile, 'r') as f: newlines = f.readlines() self.assertEqual(desiredlines, newlines) + # Check changelog + self._check_changelog(recipe, oldrecipefile) def test_devtool_upgrade_recipe_upgrade_extra_tasks(self): # Check preconditions
@@ -2116,6 +2142,8 @@ class DevtoolUpgradeTests(DevtoolBase): with open(newcratesincfile, 'r') as f: newlines = f.readlines() self.assertEqual(desiredlines, newlines) + # Check changelog + self._check_changelog(recipe, oldrecipefile) # Check devtool reset recipe result = runCmd('devtool reset %s -n' % recipe) result = runCmd('devtool status')
scripts/lib/devtool/standard.py +8 -0
@@ -2046,6 +2046,14 @@ def _reset(recipes, no_clean, remove_work, config, basepath, workspace): clean_preferred_provider(pn, config.workspace_path) + # Clean up changelog if present + changelog_file = os.path.join(config.workspace_path, 'changelogs', '%s.txt' % pn) + if os.path.exists(changelog_file): + os.remove(changelog_file) + changelog_dir = os.path.dirname(changelog_file) + if not os.listdir(changelog_dir): + os.rmdir(changelog_dir) + def reset(args, config, basepath, workspace): """Entry point for the devtool 'reset' subcommand"""
scripts/lib/devtool/upgrade.py +95 -0
119 lines not on this page · raw message
Signature · 1 line
2.51.0
meta/lib/oeqa/selftest/cases/devtool.py class DevtoolUpgradeTests(DevtoolBase): · line 1951
Quoted 5 lines · On Wed, 13 May 2026 at 14:20, <[email protected]> wrote:
meta/lib/oeqa/selftest/cases/devtool.py +5
-0+ def _check_changelog(self, recipe, oldrecipefile): + """Compare extracted changelog against reference data if available.""" + changelog_ref = oldrecipefile + '.changelog' + if not os.path.exists(changelog_ref): + return
I think these last two lines should be removed, so that the changelog check is enforced. Otherwise it's way too easy to regress the test, e.g. by replacing the test recipes, or adding new recipes without associated changelogs, or with changelog files that are slightly misnamed.
Other than this, lgtm, thanks!
Alex
Quoted 19 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Alexander Kanavin via lists.openembedded.org
Sent: Wednesday, 13 May 2026 14:32
To: Daniel Turull <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH v2] devtool: upgrade: extract changelog between versionsmeta/lib/oeqa/selftest/cases/devtool.py class DevtoolUpgradeTests(DevtoolBase): · line 1951
Quoted 5 lines · On Wed, 13 May 2026 at 14:20, <[email protected]> wrote:
meta/lib/oeqa/selftest/cases/devtool.py +5
-0+ def _check_changelog(self, recipe, oldrecipefile): + """Compare extracted changelog against reference data if available.""" + changelog_ref = oldrecipefile + '.changelog' + if not os.path.exists(changelog_ref): + return
Quoted 5 lines
I think these last two lines should be removed, so that the changelog check is enforced. Otherwise it's way too easy to regress the test, e.g. by replacing the test recipes, or adding new recipes without associated changelogs, or with changelog files that are slightly misnamed.
Ok. I'll drop them and send a v3 with it. Thanks for the quick feedback Alex.
Quoted 3 lines
Other than this, lgtm, thanks!
Alex