Improvements for devtool/recipetool
cover Improvements for devtool/recipetool
The recent Yocto Project Summit and the devtool hands-on class triggered a couple of questions for me, and when I investigated them, I stumbled upon a number of problems. This series of patches solves them.
The first four patches are needed for me to be able to execute the devtool and recipetool selftests in our environment. The next four patches improve the identification of the license(s) used by a recipe, and makes the SRC_URI checksums used by devtool and recipetool consistent. Finally, there is a patch to make devtool --no-extract work again (it regressed with the recent devtool changes that improved git submodule support).
//Peter
Peter Kjellerstedt (9):
oeqa/selftest/devtool: Correct git clone of local repository oeqa/selftest/devtool: Avoid global Git hooks when amending a patch oeqa/selftest/devtool: Make test_devtool_load_plugin more resilient oeqa/selftest/recipetool: Make test_recipetool_load_plugin more
resilient
lib/oe/recipeutils: Avoid wrapping any SRC_URI[sha*sum] variables
recipetool: create: Improve identification of licenses
recipetool: create: Only include the expected SRC_URI checksums
devtool: upgrade: Update all existing checksums for the SRC_URI
devtool: modify: Make --no-extract work again
.../devtool/devtool-upgrade-test3_1.5.3.bb | 16 ++++ .../devtool-upgrade-test3_1.5.3.bb.upgraded | 15 +++ .../devtool/devtool-upgrade-test4_1.5.3.bb | 22 +++++ .../devtool-upgrade-test4_1.5.3.bb.upgraded | 19 ++++ meta/lib/oe/recipeutils.py | 2 +- meta/lib/oeqa/selftest/cases/devtool.py | 77 +++++++++++++-- meta/lib/oeqa/selftest/cases/recipetool.py | 44 ++++----- scripts/lib/devtool/standard.py | 3 +- scripts/lib/devtool/upgrade.py | 51 +++++----- scripts/lib/recipetool/create.py | 95 ++++++++++--------- 10 files changed, 241 insertions(+), 103 deletions(-) create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.upgraded create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.upgraded
PATCH 1/9 oeqa/selftest/devtool: Correct git clone of local repository
If the build environment is setup using repo, then poky/.git/object is a symbolic link rather than a directory. To clone such repositories, the source path must be prefixed with "file://". This avoids the following error:
fatal: failed to start iterator over '.../poky/.git/objects': Not a directory
--- meta/lib/oeqa/selftest/cases/devtool.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oeqa/selftest/cases/devtool.py +1 -1
@@ -54,7 +54,7 @@ def setUpModule(): result = runCmd('git rev-parse --show-toplevel', cwd=canonical_layerpath) oldreporoot = result.output.rstrip() newmetapath = os.path.join(corecopydir, os.path.relpath(oldmetapath, oldreporoot)) - runCmd('git clone %s %s' % (oldreporoot, corecopydir), cwd=templayerdir) + runCmd('git clone file://%s %s' % (oldreporoot, corecopydir), cwd=templayerdir) # Now we need to copy any modified files # You might ask "why not just copy the entire tree instead of # cloning and doing this?" - well, the problem with that is
PATCH 2/9 oeqa/selftest/devtool: Avoid global Git hooks when amending a patch
In reply to Peter KjellerstedtTo avoid potential problems due to global Git hooks, add --no-verify to a git commit --amend command.
--- meta/lib/oeqa/selftest/cases/devtool.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oeqa/selftest/cases/devtool.py +1 -1
@@ -1495,7 +1495,7 @@ class DevtoolUpdateTests(DevtoolBase): # Modify one file srctree = os.path.join(self.workspacedir, 'sources', testrecipe) runCmd('echo "Another line" >> README', cwd=srctree) - runCmd('git commit -a --amend --no-edit', cwd=srctree) + runCmd('git commit -a --amend --no-edit --no-verify', cwd=srctree) self.add_command_to_tearDown('cd %s; rm %s/*; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, testrecipe, os.path.basename(recipefile))) result = runCmd('devtool update-recipe %s' % testrecipe) expected_status = [(' M', '.*/%s/readme.patch.gz$' % testrecipe)]
PATCH 3/9 oeqa/selftest/devtool: Make test_devtool_load_plugin more resilient
In reply to Peter Kjellerstedt* Avoid trying to write to read-only directories and file systems.
* Support symbolic links in BBPATH.
--- meta/lib/oeqa/selftest/cases/devtool.py | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-)
meta/lib/oeqa/selftest/cases/devtool.py +20 -5
@@ -4,6 +4,7 @@ # SPDX-License-Identifier: MIT # +import errno import os import re import shutil
@@ -1900,7 +1901,15 @@ class DevtoolUpgradeTests(DevtoolBase): for p in paths: dstdir = os.path.join(dstdir, p) if not os.path.exists(dstdir): - os.makedirs(dstdir) + try: + os.makedirs(dstdir) + except PermissionError: + return False + except OSError as e: + if e.errno == errno.EROFS: + return False + else: + raise e if p == "lib": # Can race with other tests self.add_command_to_tearDown('rmdir --ignore-fail-on-non-empty %s' % dstdir)
@@ -1908,8 +1917,12 @@ class DevtoolUpgradeTests(DevtoolBase): self.track_for_cleanup(dstdir) dstfile = os.path.join(dstdir, os.path.basename(srcfile)) if srcfile != dstfile: - shutil.copy(srcfile, dstfile) + try: + shutil.copy(srcfile, dstfile) + except PermissionError: + return False self.track_for_cleanup(dstfile) + return True def test_devtool_load_plugin(self): """Test that devtool loads only the first found plugin in BBPATH."""
@@ -1927,15 +1940,17 @@ class DevtoolUpgradeTests(DevtoolBase): plugincontent = fh.readlines() try: self.assertIn('meta-selftest', srcfile, 'wrong bbpath plugin found') - for path in searchpath: - self._copy_file_with_cleanup(srcfile, path, 'lib', 'devtool') + searchpath = [ + path for path in searchpath + if self._copy_file_with_cleanup(srcfile, path, 'lib', 'devtool') + ] result = runCmd("devtool --quiet count") self.assertEqual(result.output, '1') result = runCmd("devtool --quiet multiloaded") self.assertEqual(result.output, "no") for path in searchpath: result = runCmd("devtool --quiet bbdir") - self.assertEqual(result.output, path) + self.assertEqual(os.path.realpath(result.output), os.path.realpath(path)) os.unlink(os.path.join(result.output, 'lib', 'devtool', 'bbpath.py')) finally: with open(srcfile, 'w') as fh:
PATCH 4/9 oeqa/selftest/recipetool: Make test_recipetool_load_plugin more resilient
In reply to Peter Kjellerstedt* Avoid trying to write to read-only directories and file systems.
* Support symbolic links in BBPATH.
--- meta/lib/oeqa/selftest/cases/recipetool.py | 25 +++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-)
meta/lib/oeqa/selftest/cases/recipetool.py +20 -5
@@ -4,6 +4,7 @@ # SPDX-License-Identifier: MIT # +import errno import os import shutil import tempfile
@@ -860,7 +861,15 @@ class RecipetoolTests(RecipetoolBase): for p in paths: dstdir = os.path.join(dstdir, p) if not os.path.exists(dstdir): - os.makedirs(dstdir) + try: + os.makedirs(dstdir) + except PermissionError: + return False + except OSError as e: + if e.errno == errno.EROFS: + return False + else: + raise e if p == "lib": # Can race with other tests self.add_command_to_tearDown('rmdir --ignore-fail-on-non-empty %s' % dstdir)
@@ -868,8 +877,12 @@ class RecipetoolTests(RecipetoolBase): self.track_for_cleanup(dstdir) dstfile = os.path.join(dstdir, os.path.basename(srcfile)) if srcfile != dstfile: - shutil.copy(srcfile, dstfile) + try: + shutil.copy(srcfile, dstfile) + except PermissionError: + return False self.track_for_cleanup(dstfile) + return True def test_recipetool_load_plugin(self): """Test that recipetool loads only the first found plugin in BBPATH."""
@@ -883,15 +896,17 @@ class RecipetoolTests(RecipetoolBase): plugincontent = fh.readlines() try: self.assertIn('meta-selftest', srcfile, 'wrong bbpath plugin found') - for path in searchpath: - self._copy_file_with_cleanup(srcfile, path, 'lib', 'recipetool') + searchpath = [ + path for path in searchpath + if self._copy_file_with_cleanup(srcfile, path, 'lib', 'recipetool') + ] result = runCmd("recipetool --quiet count") self.assertEqual(result.output, '1') result = runCmd("recipetool --quiet multiloaded") self.assertEqual(result.output, "no") for path in searchpath: result = runCmd("recipetool --quiet bbdir") - self.assertEqual(result.output, path) + self.assertEqual(os.path.realpath(result.output), os.path.realpath(path)) os.unlink(os.path.join(result.output, 'lib', 'recipetool', 'bbpath.py')) finally: with open(srcfile, 'w') as fh:
PATCH 5/9 lib/oe/recipeutils: Avoid wrapping any SRC_URI[sha*sum] variables
In reply to Peter KjellerstedtBefore, a variable such as SRC_URI[sha512sum] would end up as:
SRC_URI[sha512sum] = "45ff3abce4dab24a8090409e6d7bb26afa7fa7812a51e067 \
28c2aa47d5b4de610d97ba4609cf13d9173087bd909fdf377235eee988a6fdcf52abb7 \0341c40b5b"
when updated by patch_recipe_lines().
--- meta/lib/oe/recipeutils.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oe/recipeutils.py +1 -1
@@ -26,7 +26,7 @@ from bb.utils import vercmp_string # Help us to find places to insert values recipe_progression = ['SUMMARY', 'DESCRIPTION', 'HOMEPAGE', 'BUGTRACKER', 'SECTION', 'LICENSE', 'LICENSE_FLAGS', 'LIC_FILES_CHKSUM', 'PROVIDES', 'DEPENDS', 'PR', 'PV', 'SRCREV', 'SRC_URI', 'S', 'do_fetch()', 'do_unpack()', 'do_patch()', 'EXTRA_OECONF', 'EXTRA_OECMAKE', 'EXTRA_OESCONS', 'do_configure()', 'EXTRA_OEMAKE', 'do_compile()', 'do_install()', 'do_populate_sysroot()', 'INITSCRIPT', 'USERADD', 'GROUPADD', 'PACKAGES', 'FILES', 'RDEPENDS', 'RRECOMMENDS', 'RSUGGESTS', 'RPROVIDES', 'RREPLACES', 'RCONFLICTS', 'ALLOW_EMPTY', 'populate_packages()', 'do_package()', 'do_deploy()', 'BBCLASSEXTEND'] # Variables that sometimes are a bit long but shouldn't be wrapped -nowrap_vars = ['SUMMARY', 'HOMEPAGE', 'BUGTRACKER', r'SRC_URI\[(.+\.)?md5sum\]', r'SRC_URI\[(.+\.)?sha256sum\]'] +nowrap_vars = ['SUMMARY', 'HOMEPAGE', 'BUGTRACKER', r'SRC_URI\[(.+\.)?md5sum\]', r'SRC_URI\[(.+\.)?sha[0-9]+sum\]'] list_vars = ['SRC_URI', 'LIC_FILES_CHKSUM'] meta_vars = ['SUMMARY', 'DESCRIPTION', 'HOMEPAGE', 'BUGTRACKER', 'SECTION']
PATCH 6/9 recipetool: create: Improve identification of licenses
In reply to Peter KjellerstedtRather than having a static list of crunched MD5 checksums for some of the most common licenses, calculate it for all common licenses. This should improve the identification of license text variantions.
--- scripts/lib/recipetool/create.py | 91 ++++++++++++++++---------------- 1 file changed, 45 insertions(+), 46 deletions(-)
scripts/lib/recipetool/create.py +45 -46
@@ -1059,54 +1059,18 @@ def get_license_md5sums(d, static_only=False, linenumbers=False): return md5sums -def crunch_license(licfile): +def crunch_known_licenses(d): ''' - Remove non-material text from a license file and then check - its md5sum against a known list. This works well for licenses - which contain a copyright statement, but is also a useful way - to handle people's insistence upon reformatting the license text - slightly (with no material difference to the text of the - license). + Calculate the MD5 checksums for the crunched versions of all common + licenses. Also add additional known checksums. ''' - - import oe.utils - - # Note: these are carefully constructed! - license_title_re = re.compile(r'^#*\(? *(This is )?([Tt]he )?.{0,15} ?[Ll]icen[sc]e( \(.{1,10}\))?\)?[:\.]? ?#*$') - license_statement_re = re.compile(r'^((This (project|software)|.{1,10}) is( free software)? (released|licen[sc]ed)|(Released|Licen[cs]ed)) under the .{1,10} [Ll]icen[sc]e:?$') - copyright_re = re.compile('^ *[#\*]* *(Modified work |MIT LICENSED )?Copyright ?(\([cC]\))? .*$') - disclaimer_re = re.compile('^ *\*? ?All [Rr]ights [Rr]eserved\.$') - email_re = re.compile('^.*<[\w\.-]*@[\w\.\-]*>$') - header_re = re.compile('^(\/\**!?)? ?[\-=\*]* ?(\*\/)?$') - tag_re = re.compile('^ *@?\(?([Ll]icense|MIT)\)?$') - url_re = re.compile('^ *[#\*]* *https?:\/\/[\w\.\/\-]+$') - + crunched_md5sums = {} # common licenses - crunched_md5sums['89f3bf322f30a1dcfe952e09945842f0'] = 'Apache-2.0' - crunched_md5sums['13b6fe3075f8f42f2270a748965bf3a1'] = '0BSD' - crunched_md5sums['ba87a7d7c20719c8df4b8beed9b78c43'] = 'BSD-2-Clause' - crunched_md5sums['7f8892c03b72de419c27be4ebfa253f8'] = 'BSD-3-Clause' - crunched_md5sums['21128c0790b23a8a9f9e260d5f6b3619'] = 'BSL-1.0' - crunched_md5sums['975742a59ae1b8abdea63a97121f49f4'] = 'EDL-1.0' - crunched_md5sums['5322cee4433d84fb3aafc9e253116447'] = 'EPL-1.0' - crunched_md5sums['6922352e87de080f42419bed93063754'] = 'EPL-2.0' - crunched_md5sums['793475baa22295cae1d3d4046a3a0ceb'] = 'GPL-2.0-only' - crunched_md5sums['ff9047f969b02c20f0559470df5cb433'] = 'GPL-2.0-or-later' - crunched_md5sums['ea6de5453fcadf534df246e6cdafadcd'] = 'GPL-3.0-only' - crunched_md5sums['b419257d4d153a6fde92ddf96acf5b67'] = 'GPL-3.0-or-later' - crunched_md5sums['228737f4c49d3ee75b8fb3706b090b84'] = 'ISC' - crunched_md5sums['c6a782e826ca4e85bf7f8b89435a677d'] = 'LGPL-2.0-only' - crunched_md5sums['32d8f758a066752f0db09bd7624b8090'] = 'LGPL-2.0-or-later' - crunched_md5sums['4820937eb198b4f84c52217ed230be33'] = 'LGPL-2.1-only' - crunched_md5sums['db13fe9f3a13af7adab2dc7a76f9e44a'] = 'LGPL-2.1-or-later' - crunched_md5sums['d7a0f2e4e0950e837ac3eabf5bd1d246'] = 'LGPL-3.0-only' - crunched_md5sums['abbf328e2b434f9153351f06b9f79d02'] = 'LGPL-3.0-or-later' - crunched_md5sums['eecf6429523cbc9693547cf2db790b5c'] = 'MIT' - crunched_md5sums['b218b0e94290b9b818c4be67c8e1cc82'] = 'MIT-0' - crunched_md5sums['ddc18131d6748374f0f35a621c245b49'] = 'Unlicense' - crunched_md5sums['51f9570ff32571fc0a443102285c5e33'] = 'WTFPL' + crunched_md5sums['ad4e9d34a2e966dfe9837f18de03266d'] = 'GFDL-1.1-only' + crunched_md5sums['d014fb11a34eb67dc717fdcfc97e60ed'] = 'GFDL-1.2-only' + crunched_md5sums['e020ca655b06c112def28e597ab844f1'] = 'GFDL-1.3-only' # The following two were gleaned from the "forever" npm package crunched_md5sums['0a97f8e4cbaf889d6fa51f84b89a79f6'] = 'ISC'
@@ -1162,6 +1126,39 @@ def crunch_license(licfile): # https://raw.githubusercontent.com/stackgl/gl-mat3/v2.0.0/LICENSE.md crunched_md5sums['75512892d6f59dddb6d1c7e191957e9c'] = 'Zlib' + commonlicdir = d.getVar('COMMON_LICENSE_DIR') + for fn in sorted(os.listdir(commonlicdir)): + md5value, lictext = crunch_license(os.path.join(commonlicdir, fn)) + if md5value not in crunched_md5sums: + crunched_md5sums[md5value] = fn + elif fn != crunched_md5sums[md5value]: + bb.debug(2, "crunched_md5sums['%s'] is already set to '%s' rather than '%s'" % (md5value, crunched_md5sums[md5value], fn)) + else: + bb.debug(2, "crunched_md5sums['%s'] is already set to '%s'" % (md5value, crunched_md5sums[md5value])) + + return crunched_md5sums + +def crunch_license(licfile): + ''' + Remove non-material text from a license file and then calculate its + md5sum. This works well for licenses that contain a copyright statement, + but is also a useful way to handle people's insistence upon reformatting + the license text slightly (with no material difference to the text of the + license). + ''' + + import oe.utils + + # Note: these are carefully constructed! + license_title_re = re.compile(r'^#*\(? *(This is )?([Tt]he )?.{0,15} ?[Ll]icen[sc]e( \(.{1,10}\))?\)?[:\.]? ?#*$') + license_statement_re = re.compile(r'^((This (project|software)|.{1,10}) is( free software)? (released|licen[sc]ed)|(Released|Licen[cs]ed)) under the .{1,10} [Ll]icen[sc]e:?$') + copyright_re = re.compile('^ *[#\*]* *(Modified work |MIT LICENSED )?Copyright ?(\([cC]\))? .*$') + disclaimer_re = re.compile('^ *\*? ?All [Rr]ights [Rr]eserved\.$') + email_re = re.compile('^.*<[\w\.-]*@[\w\.\-]*>$') + header_re = re.compile('^(\/\**!?)? ?[\-=\*]* ?(\*\/)?$') + tag_re = re.compile('^ *@?\(?([Ll]icense|MIT)\)?$') + url_re = re.compile('^ *[#\*]* *https?:\/\/[\w\.\/\-]+$') + lictext = [] with open(licfile, 'r', errors='surrogateescape') as f: for line in f:
@@ -1203,13 +1200,14 @@ def crunch_license(licfile): except UnicodeEncodeError: md5val = None lictext = '' - license = crunched_md5sums.get(md5val, None) - return license, md5val, lictext + return md5val, lictext def guess_license(srctree, d): import bb md5sums = get_license_md5sums(d) + crunched_md5sums = crunch_known_licenses(d) + licenses = [] licspecs = ['*LICEN[CS]E*', 'COPYING*', '*[Ll]icense*', 'LEGAL*', '[Ll]egal*', '*GPL*', 'README.lic*', 'COPYRIGHT*', '[Cc]opyright*', 'e[dp]l-v10'] skip_extensions = (".html", ".js", ".json", ".svg", ".ts", ".go")
@@ -1227,7 +1225,8 @@ def guess_license(srctree, d): md5value = bb.utils.md5_file(licfile) license = md5sums.get(md5value, None) if not license: - license, crunched_md5, lictext = crunch_license(licfile) + crunched_md5, lictext = crunch_license(licfile) + license = crunched_md5sums.get(crunched_md5, None) if lictext and not license: license = 'Unknown' logger.info("Please add the following line for '%s' to a 'lib/recipetool/licenses.csv' " \
PATCH 7/9 recipetool: create: Only include the expected SRC_URI checksums
In reply to Peter KjellerstedtRather than including all SRC_URI checksums, include the ones that are expected. These are the same as are output if no checksums are included when building the recipe.
--- meta/lib/oeqa/selftest/cases/recipetool.py | 19 ------------------- scripts/lib/recipetool/create.py | 4 +++- 2 files changed, 3 insertions(+), 20 deletions(-)
meta/lib/oeqa/selftest/cases/recipetool.py +0 -19
@@ -349,7 +349,6 @@ class RecipetoolCreateTests(RecipetoolBase): checkvars['LICENSE'] = 'GPL-2.0-only' checkvars['LIC_FILES_CHKSUM'] = 'file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263' checkvars['SRC_URI'] = 'https://github.com/logrotate/logrotate/releases/download/${PV}/logrotate-${PV}.tar.xz' - checkvars['SRC_URI[md5sum]'] = 'a560c57fac87c45b2fc17406cdf79288' checkvars['SRC_URI[sha256sum]'] = '2e6a401cac9024db2288297e3be1a8ab60e7401ba8e91225218aaf4a27e82a07' self._test_recipe_contents(recipefile, checkvars, [])
@@ -407,7 +406,6 @@ class RecipetoolCreateTests(RecipetoolBase): checkvars = {} checkvars['LICENSE'] = set(['LGPL-2.1-only', 'MPL-1.1-only']) checkvars['SRC_URI'] = 'http://taglib.github.io/releases/taglib-${PV}.tar.gz' - checkvars['SRC_URI[md5sum]'] = 'cee7be0ccfc892fa433d6c837df9522a' checkvars['SRC_URI[sha256sum]'] = 'b6d1a5a610aae6ff39d93de5efd0fdc787aa9e9dc1e7026fa4c961b26563526b' checkvars['DEPENDS'] = set(['boost', 'zlib']) inherits = ['cmake']
@@ -470,7 +468,6 @@ class RecipetoolCreateTests(RecipetoolBase): checkvars['LICENSE'] = set(['MIT']) checkvars['LIC_FILES_CHKSUM'] = 'file://LICENSE;md5=16a934f165e8c3245f241e77d401bb88' checkvars['SRC_URI'] = 'https://files.pythonhosted.org/packages/84/30/80932401906eaf787f2e9bd86dc458f1d2e75b064b4c187341f29516945c/python-magic-${PV}.tar.gz' - checkvars['SRC_URI[md5sum]'] = 'e384c95a47218f66c6501cd6dd45ff59' checkvars['SRC_URI[sha256sum]'] = 'f3765c0f582d2dfc72c15f3b5a82aecfae9498bd29ca840d72f37d7bd38bfcd5' inherits = ['setuptools3'] self._test_recipe_contents(recipefile, checkvars, inherits)
@@ -500,11 +497,7 @@ class RecipetoolCreateTests(RecipetoolBase): checkvars['LICENSE'] = set(['BSD-3-Clause']) checkvars['LIC_FILES_CHKSUM'] = 'file://LICENSE;md5=702b1ef12cf66832a88f24c8f2ee9c19' checkvars['SRC_URI'] = 'https://files.pythonhosted.org/packages/a1/fb/f95560c6a5d4469d9c49e24cf1b5d4d21ffab5608251c6020a965fb7791c/webcolors-${PV}.tar.gz' - checkvars['SRC_URI[md5sum]'] = 'c9be30c5b0cf1cad32e4cbacbb2229e9' - checkvars['SRC_URI[sha1sum]'] = 'c90b84fb65eed9b4c9dea7f08c657bfac0e820a5' checkvars['SRC_URI[sha256sum]'] = 'c225b674c83fa923be93d235330ce0300373d02885cef23238813b0d5668304a' - checkvars['SRC_URI[sha384sum]'] = '45652af349660f19f68d01361dd5bda287789e5ea63608f52a8cea526ac04465614db2ea236103fb8456b1fcaea96ed7' - checkvars['SRC_URI[sha512sum]'] = '074aaf135ac6b0025b88b731d1d6dfa4c539b4fff7195658cc58a4326bb9f0449a231685d312b4a1ec48ca535a838bfa5c680787fe0e61473a2a092c448937d0' inherits = ['python_setuptools_build_meta'] self._test_recipe_contents(recipefile, checkvars, inherits)
@@ -534,11 +527,7 @@ class RecipetoolCreateTests(RecipetoolBase): checkvars['LICENSE'] = set(['MIT']) checkvars['LIC_FILES_CHKSUM'] = 'file://LICENSE;md5=aab31f2ef7ba214a5a341eaa47a7f367' checkvars['SRC_URI'] = 'https://files.pythonhosted.org/packages/b9/f3/ef59cee614d5e0accf6fd0cbba025b93b272e626ca89fb70a3e9187c5d15/iso8601-${PV}.tar.gz' - checkvars['SRC_URI[md5sum]'] = '6e33910eba87066b3be7fcf3d59d16b5' - checkvars['SRC_URI[sha1sum]'] = 'efd225b2c9fa7d9e4a1ec6ad94f3295cee982e61' checkvars['SRC_URI[sha256sum]'] = '6b1d3829ee8921c4301998c909f7829fa9ed3cbdac0d3b16af2d743aed1ba8df' - checkvars['SRC_URI[sha384sum]'] = '255002433fe65c19adfd6b91494271b613cb25ef6a35ac77436de1e03d60cc07bf89fd716451b917f1435e4384860ef6' - checkvars['SRC_URI[sha512sum]'] = 'db57ab2a25ef91e3bc479c8539d27e853cf1fbf60986820b8999ae15d7e566425a1e0cfba47d0f3b23aa703db0576db368e6c110ba2a2f46c9a34e8ee3611fb7' inherits = ['python_poetry_core'] self._test_recipe_contents(recipefile, checkvars, inherits)
@@ -568,11 +557,7 @@ class RecipetoolCreateTests(RecipetoolBase): checkvars['LICENSE'] = set(['PSF-2.0']) checkvars['LIC_FILES_CHKSUM'] = 'file://LICENSE;md5=fcf6b249c2641540219a727f35d8d2c2' checkvars['SRC_URI'] = 'https://files.pythonhosted.org/packages/1f/7a/8b94bb016069caa12fc9f587b28080ac33b4fbb8ca369b98bc0a4828543e/typing_extensions-${PV}.tar.gz' - checkvars['SRC_URI[md5sum]'] = '74bafe841fbd1c27324afdeb099babdf' - checkvars['SRC_URI[sha1sum]'] = 'f8bed69cbad4a57a1a67bf8a31b62b657b47f7a3' checkvars['SRC_URI[sha256sum]'] = 'df8e4339e9cb77357558cbdbceca33c303714cf861d1eef15e1070055ae8b7ef' - checkvars['SRC_URI[sha384sum]'] = '0bd0112234134d965c6884f3c1f95d27b6ae49cfb08108101158e31dff33c2dce729331628b69818850f1acb68f6c8d0' - checkvars['SRC_URI[sha512sum]'] = '5fbff10e085fbf3ac2e35d08d913608d8c8bca66903435ede91cdc7776d775689a53d64f5f0615fe687c6c228ac854c8651d99eb1cb96ec61c56b7ca01fdd440' inherits = ['python_flit_core'] self._test_recipe_contents(recipefile, checkvars, inherits)
@@ -603,11 +588,7 @@ class RecipetoolCreateTests(RecipetoolBase): checkvars['LICENSE'] = set(['MIT']) checkvars['LIC_FILES_CHKSUM'] = 'file://COPYING;md5=7a60a81c146ec25599a3e1dabb8610a8 file://json/LICENSE;md5=9d4de43111d33570c8fe49b4cb0e01af' checkvars['SRC_URI'] = 'https://files.pythonhosted.org/packages/e4/43/087b24516db11722c8687e0caf0f66c7785c0b1c51b0ab951dfde924e3f5/jsonschema-${PV}.tar.gz' - checkvars['SRC_URI[md5sum]'] = '4d6667ce76f820c35082c2d60a4896ab' - checkvars['SRC_URI[sha1sum]'] = '9173714cb88964d07f3a3f4fcaaef638b8ceac0c' checkvars['SRC_URI[sha256sum]'] = 'ec84cc37cfa703ef7cd4928db24f9cb31428a5d0fa77747b8b51a847458e0bbf' - checkvars['SRC_URI[sha384sum]'] = '7a53181f0e679aa3dc3eb4d05a420877b7b9bff2d02e81f5c289a37ed1127d6c0cca1f5a5f9e4e166f089ab36bcc2be9' - checkvars['SRC_URI[sha512sum]'] = '60fa769faf6e3fc2c14eb9acd189c86e9d366b157230a5681d36552af0c159cb1ad33fd920668a36afdab98bc97253f91501704c5c07b5009fdaf9d29b52060d' inherits = ['python_hatchling'] self._test_recipe_contents(recipefile, checkvars, inherits)
scripts/lib/recipetool/create.py +3 -1
@@ -677,8 +677,10 @@ def create_recipe(args): if not srcuri: lines_before.append('# No information for SRC_URI yet (only an external source tree was specified)') lines_before.append('SRC_URI = "%s"' % srcuri) + shown_checksums = ["%ssum" % s for s in bb.fetch2.SHOWN_CHECKSUM_LIST] for key, value in sorted(checksums.items()): - lines_before.append('SRC_URI[%s] = "%s"' % (key, value)) + if key in shown_checksums: + lines_before.append('SRC_URI[%s] = "%s"' % (key, value)) if srcuri and supports_srcrev(srcuri): lines_before.append('') lines_before.append('# Modify these as desired')
PATCH 8/9 devtool: upgrade: Update all existing checksums for the SRC_URI
In reply to Peter KjellerstedtRather than only updating the sha256sum and removing the md5sum, update all existing checksums. If the only existing checksum is md5sum, then replace it with the default expected checksums.
--- .../devtool/devtool-upgrade-test3_1.5.3.bb | 16 ++++++ .../devtool-upgrade-test3_1.5.3.bb.upgraded | 15 ++++++ .../devtool/devtool-upgrade-test4_1.5.3.bb | 22 ++++++++ .../devtool-upgrade-test4_1.5.3.bb.upgraded | 19 +++++++ meta/lib/oeqa/selftest/cases/devtool.py | 48 +++++++++++++++++ scripts/lib/devtool/upgrade.py | 51 ++++++++++--------- 6 files changed, 148 insertions(+), 23 deletions(-) create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.upgraded create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.upgraded
meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb new file +16 -0
@@ -0,0 +1,16 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[md5sum] = "9365d86bd884222b4bf1039b5a9ed1bd" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +
meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.upgraded new file +15 -0
@@ -0,0 +1,15 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI[sha256sum] = "9dd45391806b0ed215abee4c5ac1597d018c386fe9c1f5afd2f6bc3b07fd82c3" +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +
meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb new file +22 -0
@@ -0,0 +1,22 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[md5sum] = "9365d86bd884222b4bf1039b5a9ed1bd" +SRC_URI[sha1sum] = "63a0801350e812541c7f8e9ad74e0d6b629d0b39" +SRC_URI[sha256sum] = "681bcca9784bf3cb2207e68236d1f68e2aa7b80f999b5750dc77dcd756e81fbc" +SRC_URI[sha384sum] = "5fff6390465ff23dbf573fcf39dfad3aed2f92074a35e6c02abe58b7678858d90fa6572ff4cb56df8b3e217c739cdbe3" +SRC_URI[sha512sum] = "32efe7071a363f547afc74e96774f711795edda1d2702823a347d0f9953e859b7d8c45b3e63e18ffb9e0d5ed5910be652d7d727c8676e81b6cb3aed0b13aec00" + +PR = "r5" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +
meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.upgraded new file +19 -0
@@ -0,0 +1,19 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[sha1sum] = "395ce62f4f3e035b86c77038f04b96c5aa233595" +SRC_URI[sha256sum] = "9dd45391806b0ed215abee4c5ac1597d018c386fe9c1f5afd2f6bc3b07fd82c3" +SRC_URI[sha384sum] = "218c8d2d097aeba5310be759bc20573f18ffa0b11701eac6dd2e7e14ddf13c6e0e094ca7ca026eaa05ef92a056402e36" +SRC_URI[sha512sum] = "1cf9d7376fceefcd594d0a8b591afc8e11ce89f7210d10ad74438974ecebe9cc5d9ec4db9cc79e0566bfd2b0278c0cc263c07547803e7536432cd1ffd32d8a45" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +
meta/lib/oeqa/selftest/cases/devtool.py +48 -0
54 lines not on this page · raw message
scripts/lib/devtool/upgrade.py +28 -23
101 lines not on this page · raw message
Re: [OE-core] [PATCH 8/9] devtool: upgrade: Update all existing checksums for the SRC_URI
We've been relying on this to actually remove md5sums on version updates, so please do not regress that.
Alex
Quoted 294 lines · Peter Kjellerstedt
Rather than only updating the sha256sum and removing the md5sum, update all existing checksums. If the only existing checksum is md5sum, then replace it with the default expected checksums.
Signed-off-by Peter Kjellerstedt <[email protected]>--- .../devtool/devtool-upgrade-test3_1.5.3.bb | 16 ++++++ .../devtool-upgrade-test3_1.5.3.bb.upgraded | 15 ++++++ .../devtool/devtool-upgrade-test4_1.5.3.bb | 22 ++++++++ .../devtool-upgrade-test4_1.5.3.bb.upgraded | 19 +++++++ meta/lib/oeqa/selftest/cases/devtool.py | 48 +++++++++++++++++ scripts/lib/devtool/upgrade.py | 51 ++++++++++--------- 6 files changed, 148 insertions(+), 23 deletions(-) create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.upgraded create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.upgradedmeta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb new file +16
-0@@ -0,0 +1,16 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[md5sum] = "9365d86bd884222b4bf1039b5a9ed1bd" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.upgraded new file +15
-0@@ -0,0 +1,15 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI[sha256sum] = "9dd45391806b0ed215abee4c5ac1597d018c386fe9c1f5afd2f6bc3b07fd82c3" +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb new file +22
-0@@ -0,0 +1,22 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[md5sum] = "9365d86bd884222b4bf1039b5a9ed1bd" +SRC_URI[sha1sum] = "63a0801350e812541c7f8e9ad74e0d6b629d0b39" +SRC_URI[sha256sum] = "681bcca9784bf3cb2207e68236d1f68e2aa7b80f999b5750dc77dcd756e81fbc" +SRC_URI[sha384sum] = "5fff6390465ff23dbf573fcf39dfad3aed2f92074a35e6c02abe58b7678858d90fa6572ff4cb56df8b3e217c739cdbe3" +SRC_URI[sha512sum] = "32efe7071a363f547afc74e96774f711795edda1d2702823a347d0f9953e859b7d8c45b3e63e18ffb9e0d5ed5910be652d7d727c8676e81b6cb3aed0b13aec00" + +PR = "r5" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.upgraded new file +19
-0@@ -0,0 +1,19 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[sha1sum] = "395ce62f4f3e035b86c77038f04b96c5aa233595" +SRC_URI[sha256sum] = "9dd45391806b0ed215abee4c5ac1597d018c386fe9c1f5afd2f6bc3b07fd82c3" +SRC_URI[sha384sum] = "218c8d2d097aeba5310be759bc20573f18ffa0b11701eac6dd2e7e14ddf13c6e0e094ca7ca026eaa05ef92a056402e36" +SRC_URI[sha512sum] = "1cf9d7376fceefcd594d0a8b591afc8e11ce89f7210d10ad74438974ecebe9cc5d9ec4db9cc79e0566bfd2b0278c0cc263c07547803e7536432cd1ffd32d8a45" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta/lib/oeqa/selftest/cases/devtool.py +48
-0@@ -1883,6 +1883,54 @@ class DevtoolUpgradeTests(DevtoolBase): self.assertNotIn(recipe, result.output) self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after resetting') + def test_devtool_upgrade_drop_md5sum(self): + # Check preconditions + self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory') + self.track_for_cleanup(self.workspacedir) + self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') + # For the moment, we are using a real recipe. + recipe = 'devtool-upgrade-test3' + version = '1.6.0' + oldrecipefile = get_bb_var('FILE', recipe) + tempdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir) + # Check upgrade. Code does not check if new PV is older or newer that current PV, so, it may be that + # we are downgrading instead of upgrading. + result = runCmd('devtool upgrade %s %s -V %s' % (recipe, tempdir, version)) + # Check new recipe file is present + newrecipefile = os.path.join(self.workspacedir, 'recipes', recipe, '%s_%s.bb' % (recipe, version)) + self.assertExists(newrecipefile, 'Recipe file should exist after upgrade') + # Check recipe got changed as expected + with open(oldrecipefile + '.upgraded', 'r') as f: + desiredlines = f.readlines() + with open(newrecipefile, 'r') as f: + newlines = f.readlines() + self.assertEqual(desiredlines, newlines) + + def test_devtool_upgrade_all_checksums(self): + # Check preconditions + self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory') + self.track_for_cleanup(self.workspacedir) + self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') + # For the moment, we are using a real recipe. + recipe = 'devtool-upgrade-test4' + version = '1.6.0' + oldrecipefile = get_bb_var('FILE', recipe) + tempdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir) + # Check upgrade. Code does not check if new PV is older or newer that current PV, so, it may be that + # we are downgrading instead of upgrading. + result = runCmd('devtool upgrade %s %s -V %s' % (recipe, tempdir, version)) + # Check new recipe file is present + newrecipefile = os.path.join(self.workspacedir, 'recipes', recipe, '%s_%s.bb' % (recipe, version)) + self.assertExists(newrecipefile, 'Recipe file should exist after upgrade') + # Check recipe got changed as expected + with open(oldrecipefile + '.upgraded', 'r') as f: + desiredlines = f.readlines() + with open(newrecipefile, 'r') as f: + newlines = f.readlines() + self.assertEqual(desiredlines, newlines) + def test_devtool_layer_plugins(self): """Test that devtool can use plugins from other layers.scripts/lib/devtool/upgrade.py +28
-24@@ -192,8 +192,7 @@ def _extract_new_source(newpv, srctree, no_patch, srcrev, srcbranch, branch, kee __run('git submodule foreach \'git tag -f devtool-base-new\'') (stdout, _) = __run('git submodule --quiet foreach \'echo $sm_path\'') paths += [os.path.join(srctree, p) for p in stdout.splitlines()] - md5 = None - sha256 = None + checksums = {} _, _, _, _, _, params = bb.fetch2.decodeurl(uri) srcsubdir_rel = params.get('destsuffix', 'git') if not srcbranch:@@ -226,9 +225,6 @@ def _extract_new_source(newpv, srctree, no_patch, srcrev, srcbranch, branch, kee if ftmpdir and keep_temp: logger.info('Fetch temp directory is %s' % ftmpdir) - md5 = checksums['md5sum'] - sha256 = checksums['sha256sum'] - tmpsrctree = _get_srctree(tmpdir) srctree = os.path.abspath(srctree) srcsubdir_rel = os.path.relpath(tmpsrctree, tmpdir)@@ -297,7 +293,7 @@ def _extract_new_source(newpv, srctree, no_patch, srcrev, srcbranch, branch, kee if tmpdir != tmpsrctree: shutil.rmtree(tmpdir) - return (revs, md5, sha256, srcbranch, srcsubdir_rel) + return (revs, checksums, srcbranch, srcsubdir_rel) def _add_license_diff_to_recipe(path, diff): notice_text = """# FIXME: the LIC_FILES_CHKSUM values have been updated by 'devtool upgrade'.@@ -318,7 +314,7 @@ def _add_license_diff_to_recipe(path, diff): f.write("\n#\n\n".encode()) f.write(orig_content) -def _create_new_recipe(newpv, md5, sha256, srcrev, srcbranch, srcsubdir_old, srcsubdir_new, workspace, tinfoil, rd, license_diff, new_licenses, srctree, keep_failure): +def _create_new_recipe(newpv, checksums, srcrev, srcbranch, srcsubdir_old, srcsubdir_new, workspace, tinfoil, rd, license_diff, new_licenses, srctree, keep_failure): """Creates the new recipe under workspace""" bpn = rd.getVar('BPN')@@ -390,30 +386,39 @@ def _create_new_recipe(newpv, md5, sha256, srcrev, srcbranch, srcsubdir_old, src addnames.append(params['name']) # Find what's been set in the original recipe oldnames = [] + oldsums = [] noname = False for varflag in rd.getVarFlags('SRC_URI'): - if varflag.endswith(('.md5sum', '.sha256sum')): - name = varflag.rsplit('.', 1)[0] - if name not in oldnames: - oldnames.append(name) - elif varflag in ['md5sum', 'sha256sum']: - noname = True + for checksum in checksums: + if varflag.endswith('.' + checksum): + name = varflag.rsplit('.', 1)[0] + if name not in oldnames: + oldnames.append(name) + oldsums.append(checksum) + elif varflag == checksum: + noname = True + oldsums.append(checksum) # Even if SRC_URI has named entries it doesn't have to actually use the name if noname and addnames and addnames[0] not in oldnames: addnames = [] # Drop any old names (the name actually might include ${PV}) for name in oldnames: if name not in newnames: - newvalues['SRC_URI[%s.md5sum]' % name] = None - newvalues['SRC_URI[%s.sha256sum]' % name] = None + for checksum in oldsums: + newvalues['SRC_URI[%s.%s]' % (name, checksum)] = None - if sha256: - if addnames: - nameprefix = '%s.' % addnames[0] - else: - nameprefix = '' + nameprefix = '%s.' % addnames[0] if addnames else '' + + # md5sum is deprecated, remove any traces of it. If it was the only old + # checksum, then replace it with the default checksums. + if 'md5sum' in oldsums: newvalues['SRC_URI[%smd5sum]' % nameprefix] = None - newvalues['SRC_URI[%ssha256sum]' % nameprefix] = sha256 + oldsums.remove('md5sum') + if not oldsums: + oldsums = ["%ssum" % s for s in bb.fetch2.SHOWN_CHECKSUM_LIST] + + for checksum in oldsums: + newvalues['SRC_URI[%s%s]' % (nameprefix, checksum)] = checksums[checksum] if srcsubdir_new != srcsubdir_old: s_subdir_old = os.path.relpath(os.path.abspath(rd.getVar('S')), rd.getVar('WORKDIR'))@@ -571,12 +576,12 @@ def upgrade(args, config, basepath, workspace): rev1, srcsubdir1 = standard._extract_source(srctree, False, 'devtool-orig', False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides) old_licenses = _extract_licenses(srctree_s, (rd.getVar('LIC_FILES_CHKSUM') or "")) logger.info('Extracting upgraded version source...') - rev2, md5, sha256, srcbranch, srcsubdir2 = _extract_new_source(args.version, srctree, args.no_patch, + rev2, checksums, srcbranch, srcsubdir2 = _extract_new_source(args.version, srctree, args.no_patch, args.srcrev, args.srcbranch, args.branch, args.keep_temp, tinfoil, rd) new_licenses = _extract_licenses(srctree_s, (rd.getVar('LIC_FILES_CHKSUM') or "")) license_diff = _generate_license_diff(old_licenses, new_licenses) - rf, copied = _create_new_recipe(args.version, md5, sha256, args.srcrev, srcbranch, srcsubdir1, srcsubdir2, config.workspace_path, tinfoil, rd, license_diff, new_licenses, srctree, args.keep_failure) + rf, copied = _create_new_recipe(args.version, checksums, args.srcrev, srcbranch, srcsubdir1, srcsubdir2, config.workspace_path, tinfoil, rd, license_diff, new_licenses, srctree, args.keep_failure) except (bb.process.CmdError, DevtoolError) as e: recipedir = os.path.join(config.workspace_path, 'recipes', rd.getVar('BPN')) _upgrade_error(e, recipedir, srctree, args.keep_failure) -=-=-=-=-=-=-=-=-=-=-=-Links: You receive all messages sent to this group.
View/Reply Online (#191913): https://lists.openembedded.org/g/openembedded-core/message/191913
Mute This Topic: https://lists.openembedded.org/mt/103019944/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 8/9] devtool: upgrade: Update all existing checksums for the SRC_URI
I mean, what possible reason there could be to keep them? None as far as I can see.
Alex
Quoted 302 lines · Alexander Kanavin
We've been relying on this to actually remove md5sums on version updates, so please do not regress that.
Alex
Quoted 294 lines · Peter Kjellerstedt
Rather than only updating the sha256sum and removing the md5sum, update all existing checksums. If the only existing checksum is md5sum, then replace it with the default expected checksums.
Signed-off-by Peter Kjellerstedt <[email protected]>--- .../devtool/devtool-upgrade-test3_1.5.3.bb | 16 ++++++ .../devtool-upgrade-test3_1.5.3.bb.upgraded | 15 ++++++ .../devtool/devtool-upgrade-test4_1.5.3.bb | 22 ++++++++ .../devtool-upgrade-test4_1.5.3.bb.upgraded | 19 +++++++ meta/lib/oeqa/selftest/cases/devtool.py | 48 +++++++++++++++++ scripts/lib/devtool/upgrade.py | 51 ++++++++++--------- 6 files changed, 148 insertions(+), 23 deletions(-) create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.upgraded create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.upgradedmeta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb new file +16
-0@@ -0,0 +1,16 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[md5sum] = "9365d86bd884222b4bf1039b5a9ed1bd" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.upgraded new file +15
-0@@ -0,0 +1,15 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI[sha256sum] = "9dd45391806b0ed215abee4c5ac1597d018c386fe9c1f5afd2f6bc3b07fd82c3" +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb new file +22
-0@@ -0,0 +1,22 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[md5sum] = "9365d86bd884222b4bf1039b5a9ed1bd" +SRC_URI[sha1sum] = "63a0801350e812541c7f8e9ad74e0d6b629d0b39" +SRC_URI[sha256sum] = "681bcca9784bf3cb2207e68236d1f68e2aa7b80f999b5750dc77dcd756e81fbc" +SRC_URI[sha384sum] = "5fff6390465ff23dbf573fcf39dfad3aed2f92074a35e6c02abe58b7678858d90fa6572ff4cb56df8b3e217c739cdbe3" +SRC_URI[sha512sum] = "32efe7071a363f547afc74e96774f711795edda1d2702823a347d0f9953e859b7d8c45b3e63e18ffb9e0d5ed5910be652d7d727c8676e81b6cb3aed0b13aec00" + +PR = "r5" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.upgraded new file +19
-0@@ -0,0 +1,19 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[sha1sum] = "395ce62f4f3e035b86c77038f04b96c5aa233595" +SRC_URI[sha256sum] = "9dd45391806b0ed215abee4c5ac1597d018c386fe9c1f5afd2f6bc3b07fd82c3" +SRC_URI[sha384sum] = "218c8d2d097aeba5310be759bc20573f18ffa0b11701eac6dd2e7e14ddf13c6e0e094ca7ca026eaa05ef92a056402e36" +SRC_URI[sha512sum] = "1cf9d7376fceefcd594d0a8b591afc8e11ce89f7210d10ad74438974ecebe9cc5d9ec4db9cc79e0566bfd2b0278c0cc263c07547803e7536432cd1ffd32d8a45" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta/lib/oeqa/selftest/cases/devtool.py +48
-0@@ -1883,6 +1883,54 @@ class DevtoolUpgradeTests(DevtoolBase): self.assertNotIn(recipe, result.output) self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after resetting') + def test_devtool_upgrade_drop_md5sum(self): + # Check preconditions + self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory') + self.track_for_cleanup(self.workspacedir) + self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') + # For the moment, we are using a real recipe. + recipe = 'devtool-upgrade-test3' + version = '1.6.0' + oldrecipefile = get_bb_var('FILE', recipe) + tempdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir) + # Check upgrade. Code does not check if new PV is older or newer that current PV, so, it may be that + # we are downgrading instead of upgrading. + result = runCmd('devtool upgrade %s %s -V %s' % (recipe, tempdir, version)) + # Check new recipe file is present + newrecipefile = os.path.join(self.workspacedir, 'recipes', recipe, '%s_%s.bb' % (recipe, version)) + self.assertExists(newrecipefile, 'Recipe file should exist after upgrade') + # Check recipe got changed as expected + with open(oldrecipefile + '.upgraded', 'r') as f: + desiredlines = f.readlines() + with open(newrecipefile, 'r') as f: + newlines = f.readlines() + self.assertEqual(desiredlines, newlines) + + def test_devtool_upgrade_all_checksums(self): + # Check preconditions + self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory') + self.track_for_cleanup(self.workspacedir) + self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') + # For the moment, we are using a real recipe. + recipe = 'devtool-upgrade-test4' + version = '1.6.0' + oldrecipefile = get_bb_var('FILE', recipe) + tempdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir) + # Check upgrade. Code does not check if new PV is older or newer that current PV, so, it may be that + # we are downgrading instead of upgrading. + result = runCmd('devtool upgrade %s %s -V %s' % (recipe, tempdir, version)) + # Check new recipe file is present + newrecipefile = os.path.join(self.workspacedir, 'recipes', recipe, '%s_%s.bb' % (recipe, version)) + self.assertExists(newrecipefile, 'Recipe file should exist after upgrade') + # Check recipe got changed as expected + with open(oldrecipefile + '.upgraded', 'r') as f: + desiredlines = f.readlines() + with open(newrecipefile, 'r') as f: + newlines = f.readlines() + self.assertEqual(desiredlines, newlines) + def test_devtool_layer_plugins(self): """Test that devtool can use plugins from other layers.scripts/lib/devtool/upgrade.py +28
-24@@ -192,8 +192,7 @@ def _extract_new_source(newpv, srctree, no_patch, srcrev, srcbranch, branch, kee __run('git submodule foreach \'git tag -f devtool-base-new\'') (stdout, _) = __run('git submodule --quiet foreach \'echo $sm_path\'') paths += [os.path.join(srctree, p) for p in stdout.splitlines()] - md5 = None - sha256 = None + checksums = {} _, _, _, _, _, params = bb.fetch2.decodeurl(uri) srcsubdir_rel = params.get('destsuffix', 'git') if not srcbranch:@@ -226,9 +225,6 @@ def _extract_new_source(newpv, srctree, no_patch, srcrev, srcbranch, branch, kee if ftmpdir and keep_temp: logger.info('Fetch temp directory is %s' % ftmpdir) - md5 = checksums['md5sum'] - sha256 = checksums['sha256sum'] - tmpsrctree = _get_srctree(tmpdir) srctree = os.path.abspath(srctree) srcsubdir_rel = os.path.relpath(tmpsrctree, tmpdir)@@ -297,7 +293,7 @@ def _extract_new_source(newpv, srctree, no_patch, srcrev, srcbranch, branch, kee if tmpdir != tmpsrctree: shutil.rmtree(tmpdir) - return (revs, md5, sha256, srcbranch, srcsubdir_rel) + return (revs, checksums, srcbranch, srcsubdir_rel) def _add_license_diff_to_recipe(path, diff): notice_text = """# FIXME: the LIC_FILES_CHKSUM values have been updated by 'devtool upgrade'.@@ -318,7 +314,7 @@ def _add_license_diff_to_recipe(path, diff): f.write("\n#\n\n".encode()) f.write(orig_content) -def _create_new_recipe(newpv, md5, sha256, srcrev, srcbranch, srcsubdir_old, srcsubdir_new, workspace, tinfoil, rd, license_diff, new_licenses, srctree, keep_failure): +def _create_new_recipe(newpv, checksums, srcrev, srcbranch, srcsubdir_old, srcsubdir_new, workspace, tinfoil, rd, license_diff, new_licenses, srctree, keep_failure): """Creates the new recipe under workspace""" bpn = rd.getVar('BPN')@@ -390,30 +386,39 @@ def _create_new_recipe(newpv, md5, sha256, srcrev, srcbranch, srcsubdir_old, src addnames.append(params['name']) # Find what's been set in the original recipe oldnames = [] + oldsums = [] noname = False for varflag in rd.getVarFlags('SRC_URI'): - if varflag.endswith(('.md5sum', '.sha256sum')): - name = varflag.rsplit('.', 1)[0] - if name not in oldnames: - oldnames.append(name) - elif varflag in ['md5sum', 'sha256sum']: - noname = True + for checksum in checksums: + if varflag.endswith('.' + checksum): + name = varflag.rsplit('.', 1)[0] + if name not in oldnames: + oldnames.append(name) + oldsums.append(checksum) + elif varflag == checksum: + noname = True + oldsums.append(checksum) # Even if SRC_URI has named entries it doesn't have to actually use the name if noname and addnames and addnames[0] not in oldnames: addnames = [] # Drop any old names (the name actually might include ${PV}) for name in oldnames: if name not in newnames: - newvalues['SRC_URI[%s.md5sum]' % name] = None - newvalues['SRC_URI[%s.sha256sum]' % name] = None + for checksum in oldsums: + newvalues['SRC_URI[%s.%s]' % (name, checksum)] = None - if sha256: - if addnames: - nameprefix = '%s.' % addnames[0] - else: - nameprefix = '' + nameprefix = '%s.' % addnames[0] if addnames else '' + + # md5sum is deprecated, remove any traces of it. If it was the only old + # checksum, then replace it with the default checksums. + if 'md5sum' in oldsums: newvalues['SRC_URI[%smd5sum]' % nameprefix] = None - newvalues['SRC_URI[%ssha256sum]' % nameprefix] = sha256 + oldsums.remove('md5sum') + if not oldsums: + oldsums = ["%ssum" % s for s in bb.fetch2.SHOWN_CHECKSUM_LIST] + + for checksum in oldsums: + newvalues['SRC_URI[%s%s]' % (nameprefix, checksum)] = checksums[checksum] if srcsubdir_new != srcsubdir_old: s_subdir_old = os.path.relpath(os.path.abspath(rd.getVar('S')), rd.getVar('WORKDIR'))@@ -571,12 +576,12 @@ def upgrade(args, config, basepath, workspace): rev1, srcsubdir1 = standard._extract_source(srctree, False, 'devtool-orig', False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides) old_licenses = _extract_licenses(srctree_s, (rd.getVar('LIC_FILES_CHKSUM') or "")) logger.info('Extracting upgraded version source...') - rev2, md5, sha256, srcbranch, srcsubdir2 = _extract_new_source(args.version, srctree, args.no_patch, + rev2, checksums, srcbranch, srcsubdir2 = _extract_new_source(args.version, srctree, args.no_patch, args.srcrev, args.srcbranch, args.branch, args.keep_temp, tinfoil, rd) new_licenses = _extract_licenses(srctree_s, (rd.getVar('LIC_FILES_CHKSUM') or "")) license_diff = _generate_license_diff(old_licenses, new_licenses) - rf, copied = _create_new_recipe(args.version, md5, sha256, args.srcrev, srcbranch, srcsubdir1, srcsubdir2, config.workspace_path, tinfoil, rd, license_diff, new_licenses, srctree, args.keep_failure) + rf, copied = _create_new_recipe(args.version, checksums, args.srcrev, srcbranch, srcsubdir1, srcsubdir2, config.workspace_path, tinfoil, rd, license_diff, new_licenses, srctree, args.keep_failure) except (bb.process.CmdError, DevtoolError) as e: recipedir = os.path.join(config.workspace_path, 'recipes', rd.getVar('BPN')) _upgrade_error(e, recipedir, srctree, args.keep_failure) -=-=-=-=-=-=-=-=-=-=-=-Links: You receive all messages sent to this group.
View/Reply Online (#191913): https://lists.openembedded.org/g/openembedded-core/message/191913
Mute This Topic: https://lists.openembedded.org/mt/103019944/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
RE: [OE-core] [PATCH 8/9] devtool: upgrade: Update all existing checksums for the SRC_URI
Quoted 7 lines
-----Original Message-----
From: Alexander Kanavin <[email protected]>
Sent: den 6 december 2023 20:26
To: Peter Kjellerstedt <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH 8/9] devtool: upgrade: Update all existing checksums for theSRC_URI
Quoted 10 lines
I mean, what possible reason there could be to keep them? None as far as I can see.
Alex
Quoted 3 lines · Alexander Kanavin
We've been relying on this to actually remove md5sums on version updates, so please do not regress that.
Sorry, if it was unclear in the commit message, but the code still of course removes md5sum.
Quoted 8 lines
Quoted 8 lines
Alex
Quoted 4 lines · Peter Kjellerstedt
Rather than only updating the sha256sum and removing the md5sum, update all existing checksums. If the only existing checksum is md5sum, then replace it with the default expected checksums.
Will changing that to:
In addition to updating the sha256sum and removing the md5sum, update all other existing checksums. If the only existing checksum is md5sum, then replace it with the default expected checksums (currently only sha256sum).
make it clearer?
//Peter
Quoted 291 lines
Quoted 9 lines
Quoted 9 lines
Signed-off-by Peter Kjellerstedt <[email protected]>--- .../devtool/devtool-upgrade-test3_1.5.3.bb | 16 ++++++ .../devtool-upgrade-test3_1.5.3.bb.upgraded | 15 ++++++ .../devtool/devtool-upgrade-test4_1.5.3.bb | 22 ++++++++ .../devtool-upgrade-test4_1.5.3.bb.upgraded | 19 +++++++ meta/lib/oeqa/selftest/cases/devtool.py | 48 +++++++++++++++++ scripts/lib/devtool/upgrade.py | 51 ++++++++++---------
Quoted 281 lines
Quoted 281 lines
6 files changed, 148 insertions(+), 23 deletions(-) create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.upgraded create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.upgradedmeta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb new file +16
-0@@ -0,0 +1,16 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[md5sum] = "9365d86bd884222b4bf1039b5a9ed1bd" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.upgraded new file +15
-0@@ -0,0 +1,15 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI[sha256sum] = "9dd45391806b0ed215abee4c5ac1597d018c386fe9c1f5afd2f6bc3b07fd82c3" +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb new file +22
-0@@ -0,0 +1,22 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[md5sum] = "9365d86bd884222b4bf1039b5a9ed1bd" +SRC_URI[sha1sum] = "63a0801350e812541c7f8e9ad74e0d6b629d0b39" +SRC_URI[sha256sum] = "681bcca9784bf3cb2207e68236d1f68e2aa7b80f999b5750dc77dcd756e81fbc" +SRC_URI[sha384sum] = "5fff6390465ff23dbf573fcf39dfad3aed2f92074a35e6c02abe58b7678858d90fa6572ff4cb56df8b3e217c739cdbe3" +SRC_URI[sha512sum] = "32efe7071a363f547afc74e96774f711795edda1d2702823a347d0f9953e859b7d8c45b3e63e18ffb9e0d5ed5910be652d7d727c8676e81b6cb3aed0b13aec00" + +PR = "r5" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.upgraded new file +19
-0@@ -0,0 +1,19 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[sha1sum] = "395ce62f4f3e035b86c77038f04b96c5aa233595" +SRC_URI[sha256sum] = "9dd45391806b0ed215abee4c5ac1597d018c386fe9c1f5afd2f6bc3b07fd82c3" +SRC_URI[sha384sum] = "218c8d2d097aeba5310be759bc20573f18ffa0b11701eac6dd2e7e14ddf13c6e0e094ca7ca026eaa05ef92a056402e36" +SRC_URI[sha512sum] = "1cf9d7376fceefcd594d0a8b591afc8e11ce89f7210d10ad74438974ecebe9cc5d9ec4db9cc79e0566bfd2b0278c0cc263c07547803e7536432cd1ffd32d8a45" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta/lib/oeqa/selftest/cases/devtool.py +48
-0@@ -1883,6 +1883,54 @@ class DevtoolUpgradeTests(DevtoolBase): self.assertNotIn(recipe, result.output) self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after resetting') + def test_devtool_upgrade_drop_md5sum(self): + # Check preconditions + self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory') + self.track_for_cleanup(self.workspacedir) + self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') + # For the moment, we are using a real recipe. + recipe = 'devtool-upgrade-test3' + version = '1.6.0' + oldrecipefile = get_bb_var('FILE', recipe) + tempdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir) + # Check upgrade. Code does not check if new PV is older or newer that current PV, so, it may be that + # we are downgrading instead of upgrading. + result = runCmd('devtool upgrade %s %s -V %s' % (recipe, tempdir, version)) + # Check new recipe file is present + newrecipefile = os.path.join(self.workspacedir, 'recipes', recipe, '%s_%s.bb' % (recipe, version)) + self.assertExists(newrecipefile, 'Recipe file should exist after upgrade') + # Check recipe got changed as expected + with open(oldrecipefile + '.upgraded', 'r') as f: + desiredlines = f.readlines() + with open(newrecipefile, 'r') as f: + newlines = f.readlines() + self.assertEqual(desiredlines, newlines) + + def test_devtool_upgrade_all_checksums(self): + # Check preconditions + self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory') + self.track_for_cleanup(self.workspacedir) + self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') + # For the moment, we are using a real recipe. + recipe = 'devtool-upgrade-test4' + version = '1.6.0' + oldrecipefile = get_bb_var('FILE', recipe) + tempdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir) + # Check upgrade. Code does not check if new PV is older or newer that current PV, so, it may be that + # we are downgrading instead of upgrading. + result = runCmd('devtool upgrade %s %s -V %s' % (recipe, tempdir, version)) + # Check new recipe file is present + newrecipefile = os.path.join(self.workspacedir, 'recipes', recipe, '%s_%s.bb' % (recipe, version)) + self.assertExists(newrecipefile, 'Recipe file should exist after upgrade') + # Check recipe got changed as expected + with open(oldrecipefile + '.upgraded', 'r') as f: + desiredlines = f.readlines() + with open(newrecipefile, 'r') as f: + newlines = f.readlines() + self.assertEqual(desiredlines, newlines) + def test_devtool_layer_plugins(self): """Test that devtool can use plugins from other layers.scripts/lib/devtool/upgrade.py +28
-24@@ -192,8 +192,7 @@ def _extract_new_source(newpv, srctree, no_patch, srcrev, srcbranch, branch, kee __run('git submodule foreach \'git tag -f devtool-base-new\'') (stdout, _) = __run('git submodule --quiet foreach \'echo $sm_path\'') paths += [os.path.join(srctree, p) for p in stdout.splitlines()] - md5 = None - sha256 = None + checksums = {} _, _, _, _, _, params = bb.fetch2.decodeurl(uri) srcsubdir_rel = params.get('destsuffix', 'git') if not srcbranch:@@ -226,9 +225,6 @@ def _extract_new_source(newpv, srctree, no_patch, srcrev, srcbranch, branch, kee if ftmpdir and keep_temp: logger.info('Fetch temp directory is %s' % ftmpdir) - md5 = checksums['md5sum'] - sha256 = checksums['sha256sum'] - tmpsrctree = _get_srctree(tmpdir) srctree = os.path.abspath(srctree) srcsubdir_rel = os.path.relpath(tmpsrctree, tmpdir)@@ -297,7 +293,7 @@ def _extract_new_source(newpv, srctree, no_patch, srcrev, srcbranch, branch, kee if tmpdir != tmpsrctree: shutil.rmtree(tmpdir) - return (revs, md5, sha256, srcbranch, srcsubdir_rel) + return (revs, checksums, srcbranch, srcsubdir_rel) def _add_license_diff_to_recipe(path, diff): notice_text = """# FIXME: the LIC_FILES_CHKSUM values have been updated by 'devtool upgrade'.@@ -318,7 +314,7 @@ def _add_license_diff_to_recipe(path, diff): f.write("\n#\n\n".encode()) f.write(orig_content) -def _create_new_recipe(newpv, md5, sha256, srcrev, srcbranch, srcsubdir_old, srcsubdir_new, workspace, tinfoil, rd, license_diff, new_licenses, srctree, keep_failure): +def _create_new_recipe(newpv, checksums, srcrev, srcbranch, srcsubdir_old, srcsubdir_new, workspace, tinfoil, rd, license_diff, new_licenses, srctree, keep_failure): """Creates the new recipe under workspace""" bpn = rd.getVar('BPN')@@ -390,30 +386,39 @@ def _create_new_recipe(newpv, md5, sha256, srcrev, srcbranch, srcsubdir_old, src addnames.append(params['name']) # Find what's been set in the original recipe oldnames = [] + oldsums = [] noname = False for varflag in rd.getVarFlags('SRC_URI'): - if varflag.endswith(('.md5sum', '.sha256sum')): - name = varflag.rsplit('.', 1)[0] - if name not in oldnames: - oldnames.append(name) - elif varflag in ['md5sum', 'sha256sum']: - noname = True + for checksum in checksums: + if varflag.endswith('.' + checksum): + name = varflag.rsplit('.', 1)[0] + if name not in oldnames: + oldnames.append(name) + oldsums.append(checksum) + elif varflag == checksum: + noname = True + oldsums.append(checksum) # Even if SRC_URI has named entries it doesn't have to actually use the name if noname and addnames and addnames[0] not in oldnames: addnames = [] # Drop any old names (the name actually might include ${PV}) for name in oldnames: if name not in newnames: - newvalues['SRC_URI[%s.md5sum]' % name] = None - newvalues['SRC_URI[%s.sha256sum]' % name] = None + for checksum in oldsums: + newvalues['SRC_URI[%s.%s]' % (name, checksum)] = None - if sha256: - if addnames: - nameprefix = '%s.' % addnames[0] - else: - nameprefix = '' + nameprefix = '%s.' % addnames[0] if addnames else '' + + # md5sum is deprecated, remove any traces of it. If it was the only old + # checksum, then replace it with the default checksums. + if 'md5sum' in oldsums: newvalues['SRC_URI[%smd5sum]' % nameprefix] = None - newvalues['SRC_URI[%ssha256sum]' % nameprefix] = sha256 + oldsums.remove('md5sum') + if not oldsums: + oldsums = ["%ssum" % s for s in bb.fetch2.SHOWN_CHECKSUM_LIST] + + for checksum in oldsums: + newvalues['SRC_URI[%s%s]' % (nameprefix, checksum)] = checksums[checksum] if srcsubdir_new != srcsubdir_old: s_subdir_old = os.path.relpath(os.path.abspath(rd.getVar('S')), rd.getVar('WORKDIR'))@@ -571,12 +576,12 @@ def upgrade(args, config, basepath, workspace): rev1, srcsubdir1 = standard._extract_source(srctree, False, 'devtool-orig', False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides) old_licenses = _extract_licenses(srctree_s, (rd.getVar('LIC_FILES_CHKSUM') or "")) logger.info('Extracting upgraded version source...') - rev2, md5, sha256, srcbranch, srcsubdir2 = _extract_new_source(args.version, srctree, args.no_patch, + rev2, checksums, srcbranch, srcsubdir2 = _extract_new_source(args.version, srctree, args.no_patch, args.srcrev, args.srcbranch, args.branch, args.keep_temp, tinfoil, rd) new_licenses = _extract_licenses(srctree_s, (rd.getVar('LIC_FILES_CHKSUM') or "")) license_diff = _generate_license_diff(old_licenses, new_licenses) - rf, copied = _create_new_recipe(args.version, md5, sha256, args.srcrev, srcbranch, srcsubdir1, srcsubdir2, config.workspace_path, tinfoil, rd, license_diff, new_licenses, srctree, args.keep_failure) + rf, copied = _create_new_recipe(args.version, checksums, args.srcrev, srcbranch, srcsubdir1, srcsubdir2, config.workspace_path, tinfoil, rd, license_diff, new_licenses, srctree, args.keep_failure) except (bb.process.CmdError, DevtoolError) as e: recipedir = os.path.join(config.workspace_path, 'recipes', rd.getVar('BPN')) _upgrade_error(e, recipedir, srctree, args.keep_failure) -=-=-=-=-=-=-=-=-=-=-=-Links: You receive all messages sent to this group.
View/Reply Online (#191913): https://lists.openembedded.org/g/openembedded-core/message/191913
Mute This Topic: https://lists.openembedded.org/mt/103019944/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 8/9] devtool: upgrade: Update all existing checksums for the SRC_URI
Quoted 7 lines · Peter Kjellerstedt
Will changing that to:
In addition to updating the sha256sum and removing the md5sum, update all other existing checksums. If the only existing checksum is md5sum, then replace it with the default expected checksums (currently only sha256sum).
make it clearer?
Thanks, yes that is fine.
Alex
Re: [OE-core] [PATCH 8/9] devtool: upgrade: Update all existing checksums for the SRC_URI
In reply to Peter KjellerstedtI wonder if we should start using sha512sum along with sha256sum as default with devtool upgrade and skip others.
Quoted 294 lines · Peter Kjellerstedt
Rather than only updating the sha256sum and removing the md5sum, update all existing checksums. If the only existing checksum is md5sum, then replace it with the default expected checksums.
Signed-off-by Peter Kjellerstedt <[email protected]>--- .../devtool/devtool-upgrade-test3_1.5.3.bb | 16 ++++++ .../devtool-upgrade-test3_1.5.3.bb.upgraded | 15 ++++++ .../devtool/devtool-upgrade-test4_1.5.3.bb | 22 ++++++++ .../devtool-upgrade-test4_1.5.3.bb.upgraded | 19 +++++++ meta/lib/oeqa/selftest/cases/devtool.py | 48 +++++++++++++++++ scripts/lib/devtool/upgrade.py | 51 ++++++++++--------- 6 files changed, 148 insertions(+), 23 deletions(-) create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.upgraded create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb create mode 100644 meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.upgradedmeta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb new file +16
-0@@ -0,0 +1,16 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[md5sum] = "9365d86bd884222b4bf1039b5a9ed1bd" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta-selftest/recipes-test/devtool/devtool-upgrade-test3_1.5.3.bb.upgraded new file +15
-0@@ -0,0 +1,15 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI[sha256sum] = "9dd45391806b0ed215abee4c5ac1597d018c386fe9c1f5afd2f6bc3b07fd82c3" +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb new file +22
-0@@ -0,0 +1,22 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[md5sum] = "9365d86bd884222b4bf1039b5a9ed1bd" +SRC_URI[sha1sum] = "63a0801350e812541c7f8e9ad74e0d6b629d0b39" +SRC_URI[sha256sum] = "681bcca9784bf3cb2207e68236d1f68e2aa7b80f999b5750dc77dcd756e81fbc" +SRC_URI[sha384sum] = "5fff6390465ff23dbf573fcf39dfad3aed2f92074a35e6c02abe58b7678858d90fa6572ff4cb56df8b3e217c739cdbe3" +SRC_URI[sha512sum] = "32efe7071a363f547afc74e96774f711795edda1d2702823a347d0f9953e859b7d8c45b3e63e18ffb9e0d5ed5910be652d7d727c8676e81b6cb3aed0b13aec00" + +PR = "r5" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta-selftest/recipes-test/devtool/devtool-upgrade-test4_1.5.3.bb.upgraded new file +19
-0@@ -0,0 +1,19 @@ +SUMMARY = "Pipe viewer test recipe for devtool upgrade test" +LICENSE = "Artistic-2.0" +LIC_FILES_CHKSUM = "file://doc/COPYING;md5=9c50db2589ee3ef10a9b7b2e50ce1d02" + +SRC_URI = "http://www.ivarch.com/programs/sources/pv-${PV}.tar.gz" +UPSTREAM_CHECK_URI = "http://www.ivarch.com/programs/pv.shtml" +RECIPE_NO_UPDATE_REASON = "This recipe is used to test devtool upgrade feature" + +SRC_URI[sha1sum] = "395ce62f4f3e035b86c77038f04b96c5aa233595" +SRC_URI[sha256sum] = "9dd45391806b0ed215abee4c5ac1597d018c386fe9c1f5afd2f6bc3b07fd82c3" +SRC_URI[sha384sum] = "218c8d2d097aeba5310be759bc20573f18ffa0b11701eac6dd2e7e14ddf13c6e0e094ca7ca026eaa05ef92a056402e36" +SRC_URI[sha512sum] = "1cf9d7376fceefcd594d0a8b591afc8e11ce89f7210d10ad74438974ecebe9cc5d9ec4db9cc79e0566bfd2b0278c0cc263c07547803e7536432cd1ffd32d8a45" + +S = "${WORKDIR}/pv-${PV}" + +EXCLUDE_FROM_WORLD = "1" + +inherit autotools +meta/lib/oeqa/selftest/cases/devtool.py +48
-0@@ -1883,6 +1883,54 @@ class DevtoolUpgradeTests(DevtoolBase): self.assertNotIn(recipe, result.output) self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after resetting') + def test_devtool_upgrade_drop_md5sum(self): + # Check preconditions + self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory') + self.track_for_cleanup(self.workspacedir) + self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') + # For the moment, we are using a real recipe. + recipe = 'devtool-upgrade-test3' + version = '1.6.0' + oldrecipefile = get_bb_var('FILE', recipe) + tempdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir) + # Check upgrade. Code does not check if new PV is older or newer that current PV, so, it may be that + # we are downgrading instead of upgrading. + result = runCmd('devtool upgrade %s %s -V %s' % (recipe, tempdir, version)) + # Check new recipe file is present + newrecipefile = os.path.join(self.workspacedir, 'recipes', recipe, '%s_%s.bb' % (recipe, version)) + self.assertExists(newrecipefile, 'Recipe file should exist after upgrade') + # Check recipe got changed as expected + with open(oldrecipefile + '.upgraded', 'r') as f: + desiredlines = f.readlines() + with open(newrecipefile, 'r') as f: + newlines = f.readlines() + self.assertEqual(desiredlines, newlines) + + def test_devtool_upgrade_all_checksums(self): + # Check preconditions + self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory') + self.track_for_cleanup(self.workspacedir) + self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') + # For the moment, we are using a real recipe. + recipe = 'devtool-upgrade-test4' + version = '1.6.0' + oldrecipefile = get_bb_var('FILE', recipe) + tempdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir) + # Check upgrade. Code does not check if new PV is older or newer that current PV, so, it may be that + # we are downgrading instead of upgrading. + result = runCmd('devtool upgrade %s %s -V %s' % (recipe, tempdir, version)) + # Check new recipe file is present + newrecipefile = os.path.join(self.workspacedir, 'recipes', recipe, '%s_%s.bb' % (recipe, version)) + self.assertExists(newrecipefile, 'Recipe file should exist after upgrade') + # Check recipe got changed as expected + with open(oldrecipefile + '.upgraded', 'r') as f: + desiredlines = f.readlines() + with open(newrecipefile, 'r') as f: + newlines = f.readlines() + self.assertEqual(desiredlines, newlines) + def test_devtool_layer_plugins(self): """Test that devtool can use plugins from other layers.scripts/lib/devtool/upgrade.py +28
-24@@ -192,8 +192,7 @@ def _extract_new_source(newpv, srctree, no_patch, srcrev, srcbranch, branch, kee __run('git submodule foreach \'git tag -f devtool-base-new\'') (stdout, _) = __run('git submodule --quiet foreach \'echo $sm_path\'') paths += [os.path.join(srctree, p) for p in stdout.splitlines()] - md5 = None - sha256 = None + checksums = {} _, _, _, _, _, params = bb.fetch2.decodeurl(uri) srcsubdir_rel = params.get('destsuffix', 'git') if not srcbranch:@@ -226,9 +225,6 @@ def _extract_new_source(newpv, srctree, no_patch, srcrev, srcbranch, branch, kee if ftmpdir and keep_temp: logger.info('Fetch temp directory is %s' % ftmpdir) - md5 = checksums['md5sum'] - sha256 = checksums['sha256sum'] - tmpsrctree = _get_srctree(tmpdir) srctree = os.path.abspath(srctree) srcsubdir_rel = os.path.relpath(tmpsrctree, tmpdir)@@ -297,7 +293,7 @@ def _extract_new_source(newpv, srctree, no_patch, srcrev, srcbranch, branch, kee if tmpdir != tmpsrctree: shutil.rmtree(tmpdir) - return (revs, md5, sha256, srcbranch, srcsubdir_rel) + return (revs, checksums, srcbranch, srcsubdir_rel) def _add_license_diff_to_recipe(path, diff): notice_text = """# FIXME: the LIC_FILES_CHKSUM values have been updated by 'devtool upgrade'.@@ -318,7 +314,7 @@ def _add_license_diff_to_recipe(path, diff): f.write("\n#\n\n".encode()) f.write(orig_content) -def _create_new_recipe(newpv, md5, sha256, srcrev, srcbranch, srcsubdir_old, srcsubdir_new, workspace, tinfoil, rd, license_diff, new_licenses, srctree, keep_failure): +def _create_new_recipe(newpv, checksums, srcrev, srcbranch, srcsubdir_old, srcsubdir_new, workspace, tinfoil, rd, license_diff, new_licenses, srctree, keep_failure): """Creates the new recipe under workspace""" bpn = rd.getVar('BPN')@@ -390,30 +386,39 @@ def _create_new_recipe(newpv, md5, sha256, srcrev, srcbranch, srcsubdir_old, src addnames.append(params['name']) # Find what's been set in the original recipe oldnames = [] + oldsums = [] noname = False for varflag in rd.getVarFlags('SRC_URI'): - if varflag.endswith(('.md5sum', '.sha256sum')): - name = varflag.rsplit('.', 1)[0] - if name not in oldnames: - oldnames.append(name) - elif varflag in ['md5sum', 'sha256sum']: - noname = True + for checksum in checksums: + if varflag.endswith('.' + checksum): + name = varflag.rsplit('.', 1)[0] + if name not in oldnames: + oldnames.append(name) + oldsums.append(checksum) + elif varflag == checksum: + noname = True + oldsums.append(checksum) # Even if SRC_URI has named entries it doesn't have to actually use the name if noname and addnames and addnames[0] not in oldnames: addnames = [] # Drop any old names (the name actually might include ${PV}) for name in oldnames: if name not in newnames: - newvalues['SRC_URI[%s.md5sum]' % name] = None - newvalues['SRC_URI[%s.sha256sum]' % name] = None + for checksum in oldsums: + newvalues['SRC_URI[%s.%s]' % (name, checksum)] = None - if sha256: - if addnames: - nameprefix = '%s.' % addnames[0] - else: - nameprefix = '' + nameprefix = '%s.' % addnames[0] if addnames else '' + + # md5sum is deprecated, remove any traces of it. If it was the only old + # checksum, then replace it with the default checksums. + if 'md5sum' in oldsums: newvalues['SRC_URI[%smd5sum]' % nameprefix] = None - newvalues['SRC_URI[%ssha256sum]' % nameprefix] = sha256 + oldsums.remove('md5sum') + if not oldsums: + oldsums = ["%ssum" % s for s in bb.fetch2.SHOWN_CHECKSUM_LIST] + + for checksum in oldsums: + newvalues['SRC_URI[%s%s]' % (nameprefix, checksum)] = checksums[checksum] if srcsubdir_new != srcsubdir_old: s_subdir_old = os.path.relpath(os.path.abspath(rd.getVar('S')), rd.getVar('WORKDIR'))@@ -571,12 +576,12 @@ def upgrade(args, config, basepath, workspace): rev1, srcsubdir1 = standard._extract_source(srctree, False, 'devtool-orig', False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides) old_licenses = _extract_licenses(srctree_s, (rd.getVar('LIC_FILES_CHKSUM') or "")) logger.info('Extracting upgraded version source...') - rev2, md5, sha256, srcbranch, srcsubdir2 = _extract_new_source(args.version, srctree, args.no_patch, + rev2, checksums, srcbranch, srcsubdir2 = _extract_new_source(args.version, srctree, args.no_patch, args.srcrev, args.srcbranch, args.branch, args.keep_temp, tinfoil, rd) new_licenses = _extract_licenses(srctree_s, (rd.getVar('LIC_FILES_CHKSUM') or "")) license_diff = _generate_license_diff(old_licenses, new_licenses) - rf, copied = _create_new_recipe(args.version, md5, sha256, args.srcrev, srcbranch, srcsubdir1, srcsubdir2, config.workspace_path, tinfoil, rd, license_diff, new_licenses, srctree, args.keep_failure) + rf, copied = _create_new_recipe(args.version, checksums, args.srcrev, srcbranch, srcsubdir1, srcsubdir2, config.workspace_path, tinfoil, rd, license_diff, new_licenses, srctree, args.keep_failure) except (bb.process.CmdError, DevtoolError) as e: recipedir = os.path.join(config.workspace_path, 'recipes', rd.getVar('BPN')) _upgrade_error(e, recipedir, srctree, args.keep_failure) -=-=-=-=-=-=-=-=-=-=-=-Links: You receive all messages sent to this group.
View/Reply Online (#191913): https://lists.openembedded.org/g/openembedded-core/message/191913
Mute This Topic: https://lists.openembedded.org/mt/103019944/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
PATCH 9/9 devtool: modify: Make --no-extract work again
In reply to Peter KjellerstedtThis avoids the following error when using --no-extract, introduced in commit 900129cbdf (devtool: add support for git submodules):
Traceback (most recent call last):
File ".../scripts/devtool", line 349, in <module>
ret = main()
File ".../scripts/devtool", line 336, in main
ret = args.func(args, config, basepath, workspace)
File ".../scripts/lib/devtool/standard.py", line 995, in modify
for commit in commits[name]:
KeyError: '.'
--- scripts/lib/devtool/standard.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
scripts/lib/devtool/standard.py +2 -1
@@ -991,7 +991,8 @@ def modify(args, config, basepath, workspace): '}\n') if initial_revs: for name, rev in initial_revs.items(): - f.write('\n# initial_rev %s: %s\n' % (name, rev)) + f.write('\n# initial_rev %s: %s\n' % (name, rev)) + if name in commits: for commit in commits[name]: f.write('# commit %s: %s\n' % (name, commit)) if branch_patches: