Patches from mailing list and fixes
cover Patches from mailing list and fixes
This series catches up on some patches that had already been sent to the mailing list, just re-iterating them here exactly as they will be merged, as well as a couple new patches.
A big "Thank You!" to Robert Yang for sharing fixes that Wind River was carrying internally.
One new patch, sent on Michael Halstead's behalf, represents the current production instance changes in order to meet the Python 3.8 minimum version requirement for current bitbake.
One additional new patch replaces deprecated distutils usage (which was only used for LooseVersion), with pkg_resources.parse_version instead.
The following changes since commit d8ab1beebeb5b49693b522d8618393620495bd74:
layerindex/utils.py: ignore 'core' in BBFILES_COLLECTIONS (2022-04-28 08:15:22 -0700)
are available in the Git repository at:
https://git.yoctoproject.org/layerindex-web timo/mailing-list-patches
http://git.yoctoproject.org/cgit.cgi/layerindex-web/log/?h=timo/mailing-list-patches
Michael Halstead (2):
Dockerfile: add captcha dependencies
Dockerfile: switch to ubuntu:jammy for >python3.8
Robert Yang (3):
utils.py: Use get_layer_var() to get BBFILE_COLLECTIONS admin.py: Remove has_delete_permission() utils.py: Fix for is_commit_ancestor()
Tim Orling (1):
global: replace deprecated distutils
Dockerfile | 23 ++++++++++++----------- layerindex/admin.py | 17 ----------------- layerindex/recipedesc.py | 1 - layerindex/tools/import_classic.py | 1 - layerindex/tools/import_otherdistro.py | 1 - layerindex/update.py | 4 ++-- layerindex/update_layer.py | 6 +++--- layerindex/utils.py | 20 +++++++++++--------- layerindex/views.py | 6 ++---- rrs/tools/upgrade_history_internal.py | 4 ++-- 10 files changed, 32 insertions(+), 51 deletions(-)
Signature · 1 line
2.34.1
PATCH 1/6 Dockerfile: add captcha dependencies
From: Michael Halstead <[email protected]>
The current version of django-simple-captcha requires libz and freetype development libraries.
Refactor to apply. Fix typo. Fix previous mixed use of space vs tab.
--- Dockerfile | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
Dockerfile +3 -1
@@ -31,7 +31,9 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update \ python3-mysqldb \ python3-dev \ python3-pil \ - python3-wheel \ + python3-wheel \ + libz-dev \ + libfreetype6-dev \ libjpeg-dev \ libmariadbclient-dev \ locales \
Signature · 1 line
2.34.1
PATCH 2/6 utils.py: Use get_layer_var() to get BBFILE_COLLECTIONS
In reply to Tim OrlingFrom: Robert Yang <[email protected]>
The commit 38e6288c7dad579518b0de2b6664f23be58889b6 has added oe-core to BBLAYERS by default, and then there will be multiple BBFILE_COLLECTIONS which causes update.py failure to update layers (most of the dependencies or recommends are missing), use get_layer_var() to get BBFILE_COLLECTIONS to fix the problem since it can ignore the extra oe-core from BBFILE_COLLECTIONS.
Fix minor typos. Minor rewording.
--- layerindex/utils.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
layerindex/utils.py +1 -1
@@ -188,7 +188,7 @@ def _add_dependency(var, name, layerbranch, config_data, logger=None, required=T logger.warn('Either set REMOVE_LAYER_DEPENDENCIES to remove them from the database, or fix conf/layer.conf') def set_layerbranch_collection_version(layerbranch, config_data, logger=None): - layerbranch.collection = config_data.getVar('BBFILE_COLLECTIONS', True) + layerbranch.collection = get_layer_var(config_data, 'BBFILE_COLLECTIONS', logger) ver_str = "LAYERVERSION_" if layerbranch.collection: layerbranch.collection = layerbranch.collection.strip()
Signature · 1 line
2.34.1
PATCH 3/6 admin.py: Remove has_delete_permission()
In reply to Tim OrlingFrom: Robert Yang <[email protected]>
The following commit has added on_delete=models.CASCADE to the ForeignKey:
===
commit 2d526f9b0d363f3e442abc97ef9bd9fab37ee5e4
Author: Amber Elliot <[email protected]> Date: Thu Jun 25 15:57:34 2020 -0700
Updating models and imports for Django 2.2 upgrade.
===
The on_delete=models.CASCADE will remove the objects which reference the removed objects, so there will be errors when removing a LayerBranch with admin permissions:
===
Cannot delete layer branch
Deleting the selected layer branch would result in deleting related objects, but your account doesn't have permission to delete the following types of objects:
Append
distro
===
This is because has_delete_permission() always return False, remove it to use Django's implementation to fix the problem.
Minor rewording.
--- layerindex/admin.py | 17 ----------------- 1 file changed, 17 deletions(-)
layerindex/admin.py +0 -17
@@ -102,8 +102,6 @@ class RecipeAdmin(admin.ModelAdmin): return rofields def has_add_permission(self, request, obj=None): return False - def has_delete_permission(self, request, obj=None): - return False class PackageConfigAdmin(admin.ModelAdmin): search_fields = ['feature', 'recipe__pn']
@@ -132,8 +130,6 @@ class ClassicRecipeAdmin(admin.ModelAdmin): return rofields def has_add_permission(self, request, obj=None): return False - def has_delete_permission(self, request, obj=None): - return False class MachineAdmin(admin.ModelAdmin): search_fields = ['name']
@@ -141,8 +137,6 @@ class MachineAdmin(admin.ModelAdmin): readonly_fields = [f.name for f in Machine._meta.get_fields()] def has_add_permission(self, request, obj=None): return False - def has_delete_permission(self, request, obj=None): - return False class DistroAdmin(admin.ModelAdmin): search_fields = ['name']
@@ -150,9 +144,6 @@ class DistroAdmin(admin.ModelAdmin): readonly_fields = [f.name for f in Distro._meta.get_fields()] def has_add_permission(self, request, obj=None): return False - def has_delete_permission(self, request, obj=None): - return False - class BBAppendAdmin(admin.ModelAdmin): search_fields = ['filename']
@@ -160,8 +151,6 @@ class BBAppendAdmin(admin.ModelAdmin): readonly_fields = [f.name for f in BBAppend._meta.get_fields()] def has_add_permission(self, request, obj=None): return False - def has_delete_permission(self, request, obj=None): - return False class BBClassAdmin(admin.ModelAdmin): search_fields = ['name']
@@ -169,8 +158,6 @@ class BBClassAdmin(admin.ModelAdmin): readonly_fields = [f.name for f in BBClass._meta.get_fields()] def has_add_permission(self, request, obj=None): return False - def has_delete_permission(self, request, obj=None): - return False class PatchAdmin(admin.ModelAdmin): search_fields = ['path']
@@ -178,8 +165,6 @@ class PatchAdmin(admin.ModelAdmin): readonly_fields = [f.name for f in Patch._meta.get_fields()] def has_add_permission(self, request, obj=None): return False - def has_delete_permission(self, request, obj=None): - return False class PatchDispositionAdmin(admin.ModelAdmin): fields = ['patch', 'user', 'disposition', 'comment']
@@ -199,8 +184,6 @@ class IncFileAdmin(admin.ModelAdmin): readonly_fields = [f.name for f in IncFile._meta.get_fields()] def has_add_permission(self, request, obj=None): return False - def has_delete_permission(self, request, obj=None): - return False class RecipeChangeInline(admin.StackedInline): model = RecipeChange
Signature · 1 line
2.34.1
PATCH 4/6 utils.py: Fix for is_commit_ancestor()
In reply to Tim OrlingFrom: Robert Yang <[email protected]>
The runcmd() would print an "ERROR" on failure which causes confusion since the failure is expected on old branches, so subprocess.getstatusoutput to fix the problem.
Minor rewording.
--- layerindex/utils.py | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
layerindex/utils.py +10 -8
@@ -243,16 +243,18 @@ def is_commit_ancestor(repodir, commit, logger): # check if commit is a sha1 hash if re.match('[0-9a-f]{40}', commit): # check if the commit is an ancestor - contained = runcmd(['git', 'merge-base', '--is-ancestor', '%s' % commit, 'HEAD'], repodir, logger=logger) - return True - else: - raise Exception('is_commit_ancestor: "commit" must be a SHA1 hash') - except subprocess.CalledProcessError as e: - if e.returncode == 1: - # commit is not an ancestor + cmd = "GIT_DIR=%s/.git git merge-base --is-ancestor %s HEAD" % (repodir, commit) + logger.debug('Running "%s"' % cmd) + ret, output = subprocess.getstatusoutput(cmd) + if ret == 0: + return True + elif ret == 1: + logger.debug('output: %s' % output) return False else: - raise e + raise Exception('Failed to run command: %s: ret: %s, output: %s' % (cmd, ret, output)) + else: + raise Exception('is_commit_ancestor: "commit" must be a SHA1 hash') except Exception as esc: logger.warn(esc)
Signature · 1 line
2.34.1
PATCH 5/6 Dockerfile: switch to ubuntu:jammy for >python3.8
In reply to Tim OrlingFrom: Michael Halstead <[email protected]>
ubuntu:jammy provides python 3.10.6 which supports current bitbake. Dropped unneeded python2 packages.
Moved locale generation high to reduce error messages during image building.
--- Dockerfile | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-)
Dockerfile +10 -11
@@ -1,6 +1,6 @@ # See README for how to use this. -FROM debian:buster +FROM ubuntu:jammy LABEL maintainer="Michael Halstead <[email protected]>" ENV PYTHONUNBUFFERED=1 \
@@ -17,30 +17,29 @@ ENV PYTHONUNBUFFERED=1 \ COPY requirements.txt / RUN DEBIAN_FRONTEND=noninteractive apt-get update \ + && apt-get install -y locales \ + && echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen \ + && locale-gen en_US.UTF-8 \ + && update-locale \ && apt-get install -y --no-install-recommends \ autoconf \ g++ \ gcc \ make \ - python-pip \ - python-mysqldb \ - python-dev \ - python-pil \ - python-wheel \ + python2 \ python3-pip \ python3-mysqldb \ python3-dev \ python3-pil \ python3-wheel \ - libz-dev \ + zlib1g-dev \ libfreetype6-dev \ libjpeg-dev \ - libmariadbclient-dev \ - locales \ + libmariadb-dev-compat \ netcat-openbsd \ curl \ wget \ - git-core \ + git \ vim \ rpm2cpio \ rpm \
@@ -51,7 +50,7 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update \ && pip3 install gunicorn \ && pip3 install setuptools \ && pip3 install -r /requirements.txt \ - && apt-get purge -y autoconf g++ make python3-dev libjpeg-dev libmariadbclient-dev \ + && apt-get purge -y autoconf g++ make python3-dev libjpeg-dev \ && apt-get autoremove -y \ && rm -rf /var/lib/apt/lists/* \ && apt-get clean
Signature · 1 line
2.34.1
PATCH 6/6 global: replace deprecated distutils
In reply to Tim OrlingReplace distutils.version.LooseVersion with pkg_resource.parse_version
https://majornetwork.net/2021/05/comparing-version-numbers-in-python/
[YOCTO #14990]
--- layerindex/recipedesc.py | 1 - layerindex/tools/import_classic.py | 1 - layerindex/tools/import_otherdistro.py | 1 - layerindex/update.py | 4 ++-- layerindex/update_layer.py | 6 +++--- layerindex/views.py | 6 ++---- rrs/tools/upgrade_history_internal.py | 4 ++-- 7 files changed, 9 insertions(+), 14 deletions(-)
layerindex/recipedesc.py +0 -1
@@ -13,7 +13,6 @@ import logging import subprocess from datetime import datetime import fnmatch -from distutils.version import LooseVersion logger = None
layerindex/tools/import_classic.py +0 -1
@@ -22,7 +22,6 @@ import fnmatch import re import tempfile import shutil -from distutils.version import LooseVersion import utils import recipeparse
layerindex/tools/import_otherdistro.py +0 -1
@@ -23,7 +23,6 @@ import subprocess import string import shlex import codecs -from distutils.version import LooseVersion sys.path.insert(0, os.path.realpath(os.path.join(os.path.dirname(__file__), '..'))) sys.path.insert(0, os.path.realpath(os.path.join(os.path.dirname(__file__), 'lib')))
layerindex/update.py +2 -2
@@ -17,7 +17,7 @@ import codecs import logging import subprocess from datetime import datetime, timedelta -from distutils.version import LooseVersion +from pkg_resources import parse_version import utils import operator import re
@@ -130,7 +130,7 @@ def extract_value(valuename, output): return '' def main(): - if LooseVersion(git.__version__) < '0.3.1': + if parse_version(git.__version__) < parse_version('0.3.1'): logger.error("Version of GitPython is too old, please install GitPython (python-git) 0.3.1 or later in order to use this script") sys.exit(1)
layerindex/update_layer.py +3 -3
@@ -19,7 +19,7 @@ import re import tempfile import shutil import errno -from distutils.version import LooseVersion +from pkg_resources import parse_version import itertools import utils import recipeparse
@@ -239,7 +239,7 @@ def update_distro_conf_file(path, distro, d): distro.description = desc def main(): - if LooseVersion(git.__version__) < '0.3.1': + if parse_version(git.__version__) < parse_version('0.3.1'): logger.error("Version of GitPython is too old, please install GitPython (python-git) 0.3.1 or later in order to use this script") sys.exit(1)
@@ -808,7 +808,7 @@ def main(): logger.error(traceback.format_exc().rstrip()) sys.exit(1) finally: - if tinfoil and (LooseVersion(bb.__version__) > LooseVersion("1.27")): + if tinfoil and (parse_version(bb.__version__) > parse_version("1.27")): tinfoil.shutdown() if tempdir:
layerindex/views.py +2 -4
@@ -41,7 +41,6 @@ from django.views.generic.base import RedirectView from django.views.generic.edit import (CreateView, DeleteView, FormView, UpdateView) from django_registration.backends.activation.views import RegistrationView -from pkg_resources import parse_version from reversion.models import Revision import settings
@@ -1736,7 +1735,6 @@ class BranchCompareView(FormView): form_class = BranchComparisonForm def get_recipes(self, from_branch, to_branch, layer_ids): - from distutils.version import LooseVersion class BranchComparisonResult: def __init__(self, pn, short_desc): self.pn = pn
@@ -1791,9 +1789,9 @@ class BranchCompareView(FormView): changed = [] removed = [] for _, recipe in sorted(recipes.items(), key=lambda item: item[0]): - recipe.from_versions = sorted(recipe.from_versions, key=lambda item: LooseVersion(item.pv)) + recipe.from_versions = sorted(recipe.from_versions, key=lambda item: parse_version(item.pv)) from_version_exprs = [x.version_expr() for x in recipe.from_versions] - recipe.to_versions = sorted(recipe.to_versions, key=lambda item: LooseVersion(item.pv)) + recipe.to_versions = sorted(recipe.to_versions, key=lambda item: parser_version(item.pv)) to_version_exprs = [x.version_expr() for x in recipe.to_versions] if not from_version_exprs: added.append(recipe)
rrs/tools/upgrade_history_internal.py +2 -2
@@ -16,7 +16,7 @@ import os import optparse import logging import re -from distutils.version import LooseVersion +from pkg_resources import parse_version import git from datetime import datetime import calendar
@@ -561,7 +561,7 @@ def generate_history(options, layerbranch_id, commit, logger): pass finally: - if tinfoil and hasattr(tinfoil, 'shutdown') and (LooseVersion(bb.__version__) > LooseVersion("1.27")): + if tinfoil and hasattr(tinfoil, 'shutdown') and (parse_version(bb.__version__) > parse_version("1.27")): tinfoil.shutdown() utils.rmtree_force(tempdir)
Signature · 1 line
2.34.1