Fixes for updates
cover Fixes for updates
This series includes fixes to allow update.py to run on 'master', by providing a check for the BB_ENV_PASSTHROUGHS_ADDITIONS variable renaming in bitbake 2.0+.
To do so, an 'is_commit_ancestor' check is added to allow for checking if a particular commit hash is present in a checked out repo (e.g. bitbake).
Also included is an improvement on the prior patch to ignore 'core' in BBFILES_COLLECTIONS (logging a debug message rather than a warning for the case where 'core' is ignored).
The following changes since commit 4dd7f0ee8f31df927c3862677dc59636988d8dfd:
layerindex/models.py: add Inactive-Upstream (2022-04-18 12:56:51 -0700)
are available in the Git repository at:
git://git.yoctoproject.org/layerindex-web timo/fixes2
http://git.yoctoproject.org/cgit.cgi/layerindex-web/log/?h=timo/fixes2
Tim Orling (3):
layerindex/utils.py: add is_commit_ancestor check recipe{desc,parse}.py: BB_ENV_PASSTHROUGH_ADDITIONS layerindex/utils.py: ignore 'core' in BBFILES_COLLECTIONS
layerindex/recipedesc.py | 8 +++++++- layerindex/recipeparse.py | 7 ++++++- layerindex/utils.py | 29 ++++++++++++++++++++++++++++- 3 files changed, 41 insertions(+), 3 deletions(-)
Signature · 1 line
2.32.0
PATCH v2 2/3 recipe{desc,parse}.py: BB_ENV_PASSTHROUGH_ADDITIONS
ERROR: Variable BB_ENV_EXTRAWHITE has been renamed to BB_ENV_PASSTHROUGH_ADDITIONS
ERROR: Variable BB_ENV_EXTRAWHITE from the shell environment has been renamed to BB_ENV_PASSTHROUGH_ADDITIONS
ERROR: Exiting to allow enviroment variables to be corrected
Replace BB_ENV_EXTRAWHITE with new variable BB_ENV_PASSTHROUGH_ADDITIONS
In order to be backward compatible with older branches, we must first check for the presence of the bitbake commit which implemented the variable name change, using layerindex.utils.is_commit_ancestor().
---
Changes in v2:
* Use new is_commit_ancestor check to check for the specific bitbake commit that introduced the variable renaming. This provides for backward compatibility in older branches (before bitbake 2.0).
layerindex/recipedesc.py | 8 +++++++- layerindex/recipeparse.py | 7 ++++++- 2 files changed, 13 insertions(+), 2 deletions(-)
layerindex/recipedesc.py +7 -1
@@ -39,6 +39,7 @@ def main(): from layerindex.models import LayerItem, Recipe from django.db import transaction import settings + from layerindex.utils import is_commit_ancestor setup_environ(settings)
@@ -62,8 +63,13 @@ def main(): print("Unable to find bitbake by searching BITBAKEDIR, specified path '%s' or its parent, or PATH" % basepath) sys.exit(1) + # Commit "bitbake: Rename environment filtering variables" + bb_var_rename_commit = "87104b6a167188921da157c7dba45938849fb22a" # Skip sanity checks - os.environ['BB_ENV_EXTRAWHITE'] = 'DISABLE_SANITY_CHECKS' + if is_commit_ancestor(bitbakepath, bb_var_rename_commit, logger=logger): + os.environ['BB_ENV_PASSTHROUGH_ADDITIONS'] = 'DISABLE_SANITY_CHECKS' + else: + os.environ['BB_ENV_EXTRAWHITE'] = 'DISABLE_SANITY_CHECKS' os.environ['DISABLE_SANITY_CHECKS'] = '1' sys.path.extend([bitbakepath + '/lib'])
layerindex/recipeparse.py +6 -1
@@ -35,8 +35,13 @@ def init_parser(settings, branch, bitbakepath, enable_tracking=False, nocheckout bitbake_ref = 'origin/%s' % branch.bitbake_branch utils.checkout_repo(bitbakepath, bitbake_ref, logger=logger) + # Commit "bitbake: Rename environment filtering variables" + bb_var_rename_commit = "87104b6a167188921da157c7dba45938849fb22a" # Skip sanity checks - os.environ['BB_ENV_EXTRAWHITE'] = 'DISABLE_SANITY_CHECKS' + if utils.is_commit_ancestor(bitbakepath, bb_var_rename_commit, logger=logger): + os.environ['BB_ENV_PASSTHROUGH_ADDITIONS'] = 'DISABLE_SANITY_CHECKS' + else: + os.environ['BB_ENV_EXTRAWHITE'] = 'DISABLE_SANITY_CHECKS' os.environ['DISABLE_SANITY_CHECKS'] = '1' fetchdir = settings.LAYER_FETCH_DIR
Signature · 1 line
2.32.0
PATCH v2 3/3 layerindex/utils.py: ignore 'core' in BBFILES_COLLECTIONS
In reply to Tim OrlingMany layers append BBFILE_COLLECTIONS and therefore have 'core <layer>'
During update.py, this means we are likely not handling the collection we expect:
WARNING: /opt/workdir/git___git_openembedded_org_meta-openembedded/meta-oe: multiple collections found, handling first one (core) only BBFILE_COLLECTIONS = "core"
---
Changes in v2:
* Use logger.debug instead of warning for the case where 'core' is ignored
layerindex/utils.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
layerindex/utils.py +6 -1
@@ -44,7 +44,12 @@ def get_layer_var(config_data, var, logger): collection = collection_list[0] layerdir = config_data.getVar('LAYERDIR', True) if len(collection_list) > 1: - logger.warn('%s: multiple collections found, handling first one (%s) only' % (layerdir, collection)) + if collection_list[0] == 'core': + # Many layers append BBFILE_COLLECTIONS and therefore have 'core <layer>' + collection = collection_list[1] + logger.debug('%s: multiple collections found, ignoring the first one (\'core\') and handling (%s) only' % (layerdir, collection)) + else: + logger.warn('%s: multiple collections found, handling first one (%s) only' % (layerdir, collection)) if var == 'BBFILE_COLLECTIONS': return collection value = config_data.getVar('%s_%s' % (var, collection), True)
Signature · 1 line
2.32.0
PATCH 1/3 layerindex/utils.py: add is_commit_ancestor check
In reply to Tim OrlingAdd a helper function to check if a given SHA1 hash is an ancestor in the currently checkout out branch, using:
git merge-branch --is-ancestor <commit> HEAD
NOTE: This will not match commits which have been cherry-picked.
--- layerindex/utils.py | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
layerindex/utils.py +22 -0
@@ -229,6 +229,28 @@ def explode_dep_versions2(bitbakepath, deps): import bb.utils return bb.utils.explode_dep_versions2(deps) +def is_commit_ancestor(repodir, commit, logger): + """ + Check if the given SHA1 hash is an ancestor in the currently checked out branch. + NOTE: This will not match commits which have been cherry-picked. + """ + try: + # check if commit is a sha1 hash + if re.match('[0-9a-f]{40}', commit): + # check if the commit is an ancestor + contained = runcmd(['git', 'merge-base', '--is-ancestor', '%s' % commit, 'HEAD'], repodir, logger=logger) + return True + else: + raise Exception('is_commit_ancestor: "commit" must be a SHA1 hash') + except CalledProcessError as e: + if e.returncode == 1: + # commit is not an ancestor + return False + else: + raise e + except Exception as esc: + logger.warn(esc) + def checkout_repo(repodir, commit, logger, force=False): """ Check out a revision in a repository, ensuring that untracked/uncommitted
Signature · 1 line
2.32.0
PATCH v2 1/3 layerindex/utils.py: add is_commit_ancestor check
Add a helper function to check if a given SHA1 hash is an ancestor in the currently checked out branch, using:
git merge-branch --is-ancestor <commit> HEAD
NOTE: This will not match commits which have been cherry-picked.
---
Changes in v2:
* Fix import of (subprocess.)CalledProcessError
layerindex/utils.py | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
layerindex/utils.py +22 -0
@@ -229,6 +229,28 @@ def explode_dep_versions2(bitbakepath, deps): import bb.utils return bb.utils.explode_dep_versions2(deps) +def is_commit_ancestor(repodir, commit, logger): + """ + Check if the given SHA1 hash is an ancestor in the currently checked out branch. + NOTE: This will not match commits which have been cherry-picked. + """ + try: + # check if commit is a sha1 hash + if re.match('[0-9a-f]{40}', commit): + # check if the commit is an ancestor + contained = runcmd(['git', 'merge-base', '--is-ancestor', '%s' % commit, 'HEAD'], repodir, logger=logger) + return True + else: + raise Exception('is_commit_ancestor: "commit" must be a SHA1 hash') + except subprocess.CalledProcessError as e: + if e.returncode == 1: + # commit is not an ancestor + return False + else: + raise e + except Exception as esc: + logger.warn(esc) + def checkout_repo(repodir, commit, logger, force=False): """ Check out a revision in a repository, ensuring that untracked/uncommitted
Signature · 1 line
2.32.0