Add a mechanism to check runtime and recipe version mismatch
cover Add a mechanism to check runtime and recipe version mismatch
From: Chen Qi <[email protected]>
This RFC add a mechanims to check runtime and recipe version mismatch.
For example, if some tool says its runtime version is 1.0, but its recipe version is 1.1, we can now detect it at build time.
Let me explain the most important thing in this cover letter: why.
Briefly speaking, there are two major benefits:
1) Make things look better.
2) Detect potential problems.
See more detailed explanations below.
When people use some software they care about, they would type "--version" or something similar to check its version. And if the output does not match the package version that is installed, or there's some "bad" words in output such as "unknown", "dirty", "dev", they would raise questions in their minds: Does the vendor really know what it's building? Am I using stable versions? If they can't even get versions right, how can I trust them on other things?
So here comes the first reason on adding such a mechanim: we can detect version mismatch, we can then fix them. This makes things look better. Patch 3/3 is an example fix.
Another reason, which might be a more important reason, is that it helps us defect potential problems. First of all, version information sometimes matters more than just showing user some info. It affects runtime behavior. A typical example is k8s. If k8s is built with correct version info, it will try to pull major.minor version compatible images and do its work. But if it's built with a wrong version, either init fails or it pulls some incompatible images. Secondly, a version mismatch indicates some potential error. Use sassc an an example. It got a version mismatch because it's not using the header file generated under build directory. Instead, it uses the default one with the same name in source directory. Luckily, that header file only contains version info. But this case demonstrates that a version mismatch actually indicates some error.
During development, I found the errors that version mismatch indicates fall into the following categories:
1) The upstream project made a mistake.
e.g.,
libuser got a mismatch because upstream project forgets to update configure.ac/configure files.
2) The recipe made a mistake.
e.g.,
python3-pyyaml-include_2.2.bb got a mismatch because it's srcrev actually points to v2.2a1 tag instead of the v2.2 tag.
3) Some corner case will raise a mismatch that normal build does not see.
e.g.,
If we build libsass from git shallow tarball, we got a mismatch. The "Version" field in .pc file is 7073 which is wrong. In a normal build, the "Version" field in the .pc file is 3.6.6 which is correct.
With this mechanism added, we can detect such problems and fix them in time.
Hope to hear your opinions.
Regards,
Qi
--- Result for oe-core world build (11 mismatches) ---
WARNING: python3-unittest-automake-output-0.2-r0 do_package_check_version_mismatch: Possible runtime versions ['0.1'] do not match recipe version 0.2
WARNING: time-1.9-r0 do_package_check_version_mismatch: Possible runtime versions ['UNKNOWN'] do not match recipe version 1.9
WARNING: pinentry-1.3.1-r0 do_package_check_version_mismatch: Possible runtime versions ['1.3.1-unknown'] do not match recipe version 1.3.1
WARNING: gnupg-2.5.5-r0 do_package_check_version_mismatch: Possible runtime versions ['1.11.0-unknown', '1.6.7-unknown', '2.5.5-unknown'] do not match recipe version 2.5.5
WARNING: dpkg-1.22.11-r0 do_package_check_version_mismatch: Possible runtime versions ['1.22.11-dirty'] do not match recipe version 1.22.11
WARNING: libgpg-error-1.51-r0 do_package_check_version_mismatch: Possible runtime versions ['1.51-unknown'] do not match recipe version 1.51
WARNING: sassc-3.6.2-r0 do_package_check_version_mismatch: Possible runtime versions ['1.1.1', '3.5'] do not match recipe version 3.6.2
WARNING: libassuan-3.0.2-r0 do_package_check_version_mismatch: Possible runtime versions ['3.0.2-unknown'] do not match recipe version 3.0.2
WARNING: libksba-1.6.7-r0 do_package_check_version_mismatch: Possible runtime versions ['1.6.7-unknown'] do not match recipe version 1.6.7
WARNING: libgcrypt-1.11.0-r0 do_package_check_version_mismatch: Possible runtime versions ['1.11.0-unknown'] do not match recipe version 1.11.0
WARNING: glslang-1_1.3.296.0-r0 do_package_check_version_mismatch: Possible runtime versions ['0.97', '15.0.0', '3.20', '4.60'] do not match recipe version 1.3.296.0
----------------------------------------
The following changes since commit 6610cad12a062592956257961a790ec6a3012b8b:
bitbake: data_smart: Ensure module dependency changes invalidate the base config cache (2025-03-13 21:21:04 +0000)
are available in the Git repository at:
https://github.com/ChenQi1989/poky version-check
https://github.com/ChenQi1989/poky/tree/version-check
Chen Qi (3):
version-check.conf: add mechanism for checking version mismatch
mc: fix buildpath QA regarding unzip
time: fix runtime version from UNKNOWN to 1.9
meta/classes/check-version-mismatch.bbclass | 399 ++++++++++++++++++
meta/{classes-recipe => classes}/qemu.bbclass | 0
meta/conf/version-check.conf | 14 +
meta/recipes-extended/mc/mc_4.8.33.bb | 1 +
meta/recipes-extended/time/time_1.9.bb | 6 +
5 files changed, 420 insertions(+)
create mode 100644 meta/classes/check-version-mismatch.bbclass
rename meta/{classes-recipe => classes}/qemu.bbclass (100%)
create mode 100644 meta/conf/version-check.confSignature · 1 line
2.34.1
RFC PATCH 1/3 version-check.conf: add mechanism for checking version mismatch
From: Chen Qi <[email protected]>
Add a mechanism to check mismatch between runtime version and build time version.
To use, add the following line to local.conf:
include conf/version-check.conf
The basic idea is to use qemu to run executables at build time, extract possible versions, and check if there's a mismatch found.
Python meta data and .pc files are also checked for quick match. This is because such info are also easy to be checked by users.
check-version-mismatch.bbclass is the class that does the actual work. A new variable, CHECK_VERSION_PV, is introduced. It defaults to ${PKGV}, but also allows override. This allows us to handle special cases in each layer.
version-check.conf is the configuration file that makes this functionality easier to use and draws some baseline. It contains some override settings for some recipes. With these overrides, all recipes in oe-core are handled well. All warnings are valid warnings.
Note that 'ps' is added to HOSTTOOLS in version-check.conf. This is because we need 'ps' to find stale processes and then clean them.
The warnings are like below:
WARNING: time-1.9-r0 do_package_check_version_mismatch: Possible runtime versions ['UNKNOWN'] do not match recipe version 1.9
WARNING: python3-unittest-automake-output-0.2-r0 do_package_check_version_mismatch: Possible runtime versions ['0.1'] do not match recipe version 0.2
WARNING: pinentry-1.3.1-r0 do_package_check_version_mismatch: Possible runtime versions ['1.3.1-unknown'] do not match recipe version 1.3.1
...
There will be a data directory containing all details: tmp/check-version-mismatch. This directory contains detailed data for each recipe that is built. If users don't want it, they can set DEBUG_VERSION_MISMATCH_CHECK to 0.
---
meta/classes/check-version-mismatch.bbclass | 399 ++++++++++++++++++
meta/{classes-recipe => classes}/qemu.bbclass | 0
meta/conf/version-check.conf | 14 +
3 files changed, 413 insertions(+)
create mode 100644 meta/classes/check-version-mismatch.bbclass
rename meta/{classes-recipe => classes}/qemu.bbclass (100%)
create mode 100644 meta/conf/version-check.confmeta/classes/check-version-mismatch.bbclass new file +399 -0
@@ -0,0 +1,399 @@ +inherit qemu + +ENABLE_VERSION_MISMATCH_CHECK ?= "${@'1' if bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', True, False, d) else '0'}" +DEBUG_VERSION_MISMATCH_CHECK ?= "1" +CHECK_VERSION_PV ?= "" + +DEPENDS:append:class-target = "${@' qemu-native' if bb.utils.to_boolean(d.getVar('ENABLE_VERSION_MISMATCH_CHECK')) else ''}" + +QEMU_EXEC ?= "${@qemu_wrapper_cmdline(d, '${STAGING_DIR_HOST}', ['${STAGING_DIR_HOST}${libdir}','${STAGING_DIR_HOST}${base_libdir}', '${PKGD}${libdir}', '${PKGD}${base_libdir}'])}" + +python do_package_check_version_mismatch() { + import re + import subprocess + import shutil + import signal + + classes_skip = ["nopackage", "image", "native", "cross", "crosssdk", "cross-canadian"] + for cs in classes_skip: + if bb.data.inherits_class(cs, d): + bb.note(f"Skip do_package_check_version_mismatch as {cs} is inherited.") + return + + if not bb.utils.to_boolean(d.getVar('ENABLE_VERSION_MISMATCH_CHECK')): + bb.note("Skip do_package_check_version_mismatch as ENABLE_VERSION_MISMATCH_CHECK is disabled.") + return + + __regexp_version_broad_match__ = re.compile(r"(?:\s|^|-|_|/|=| go|\()" + + r"(?P<version>v?[0-9][0-9.][0-9+.\-_~\(\)]*?|UNKNOWN)" + + r"(?:-release|-stable|)" + + r"(?P<extra>[+\-]unknown|[+\-]dirty|[+\-]rc?\d{1,3}|\+cargo-[0-9.]+|" + + r"[a-z]|-?[pP][0-9]{1,3}|-beta[^\s]*|-alpha[^\s]*|)" + + r"(?P<extra2>[+\-]dev|)" + + r"(?:,|:|\.|\)|)" + + r"(?=\s|$)" + ) + __regexp_exclude_year__ = re.compile(r"^(19|20)[0-9]{2}$") + __regexp_single_number_ending_with_dot__ = re.compile(r"^\d\.$") + + def is_shared_library(filepath): + return re.match(r'.*\.so(\.\d+)*$', filepath) is not None + + def get_possible_versions(output_contents, full_cmd=None, max_lines=None): + # + # Algorithm: + # 1. Check version line by line. + # 2. Skip some lines which we know that do not contain version information, e.g., License, Copyright. + # 3. Do broad match, finding all possible versions. + # 4. If there's a version found by any match, do exclude match (e.g., exclude years) + # 5. If there's a valid version, do stripping and converting and then add to possible_versions. + # 6. Return possible_versions + # + possible_versions = [] + content_lines = output_contents.split("\n") + if max_lines: + content_lines = content_lines[0:max_lines] + if full_cmd: + base_cmd = os.path.basename(full_cmd) + __regex_help_format__ = re.compile(r"-[^\s].*") + for line in content_lines: + line = line.strip() + # skip help lines + if __regex_help_format__.match(line): + continue + # avoid command itself affecting output + if full_cmd: + if line.startswith(base_cmd): + line = line[len(base_cmd):] + elif line.startswith(full_cmd): + line = line[len(full_cmd):] + # skip specific lines + skip_keywords_start = ["Copyright", "License"] + skip_line = False + for sks in skip_keywords_start: + if line.startswith(sks): + skip_line = True + break + if skip_line: + continue + + # try broad match + for match in __regexp_version_broad_match__.finditer(line): + version = match.group("version") + #print(f"version = {version}") + # do exclude match + exclude_match = __regexp_exclude_year__.match(version) + if exclude_match: + continue + exclude_match = __regexp_single_number_ending_with_dot__.match(version) + if exclude_match: + continue + # do some stripping and converting + if version.startswith("("): + version = version[1:-1] + if version.startswith("v"): + version = version[1:] + if version.endswith(")") and "(" not in version: + version = version[:-1] + # handle extra version info + version = version + match.group("extra") + match.group("extra2") + possible_versions.append(version) + return possible_versions + + def is_version_mismatch(rvs, pv): + got_match = False + if pv.startswith("git"): + return False + if "-pre" in pv: + pv = pv.split("-pre")[0] + if pv.startswith("v"): + pv = pv[1:] + for rv in rvs: + if rv == pv: + got_match = True + break + pv = pv.split("+git")[0] + # handle % character in pv which means matching any chars + if '%' in pv: + escaped_pv = re.escape(pv) + regex_pattern = escaped_pv.replace('%', '.*') + regex_pattern = f'^{regex_pattern}$' + if re.fullmatch(regex_pattern, rv): + got_match = True + break + else: + continue + # handle cases such as 2.36.0-r0 v.s. 2.36.0 + if "-r" in rv: + rv = rv.split("-r")[0] + chars_to_replace = ["-", "+", "_", "~"] + # convert to use "." as the version seperator + for cr in chars_to_replace: + rv = rv.replace(cr, ".") + pv = pv.replace(cr, ".") + if rv == pv: + got_match = True + break + # handle case such as 5.2.37(1) v.s. 5.2.37 + if "(" in rv: + rv = rv.split("(")[0] + if rv == pv: + got_match = True + break + # handle case such as 4.4.3p1 + if "p" in pv and "p" in rv.lower(): + pv = pv.lower().replace(".p", "p") + rv = rv.lower().replace(".p", "p") + if pv == rv: + got_match = True + break + # handle cases such as 6.00 v.s. 6.0 + if rv.startswith(pv): + if rv == pv + "0" or rv == pv + ".0": + got_match = True + break + elif pv.startswith(rv): + if pv == rv + "0" or pv == rv + ".0": + got_match = True + break + # handle cases such as 21306 v.s. 2.13.6 + if "." in pv and not "." in rv: + pv_components = pv.split(".") + if rv.startswith(pv_components[0]): + pv_num = 0 + for i in range(0, len(pv_components)): + pv_num = pv_num * 100 + int(pv_components[i]) + if pv_num == int(rv): + got_match = True + break + if got_match: + return False + else: + return True + + # helper function to get PKGV, useful for recipes such as perf + def get_pkgv(pn): + pkgdestwork = d.getVar("PKGDESTWORK") + recipe_data_fn = pkgdestwork + "/" + pn + pn_data = oe.packagedata.read_pkgdatafile(recipe_data_fn) + if not "PACKAGES" in pn_data: + return d.getVar("PV") + packages = pn_data["PACKAGES"].split() + for pkg in packages: + pkg_fn = pkgdestwork + "/runtime/" + pkg + pkg_data = oe.packagedata.read_pkgdatafile(pkg_fn) + if "PKGV" in pkg_data: + return pkg_data["PKGV"] + + # + # traverse PKGD, find executables and run them to get runtime version information and compare it with recipe version information + # + enable_debug = bb.utils.to_boolean(d.getVar("DEBUG_VERSION_MISMATCH_CHECK")) + pkgd = d.getVar("PKGD") + pn = d.getVar("PN") + pv = d.getVar("CHECK_VERSION_PV") + if not pv: + pv = get_pkgv(pn) + qemu_exec = d.getVar("QEMU_EXEC").strip() + executables = [] + possible_versions_all = [] + data_lines = [] + + if enable_debug: + debug_directory = d.getVar("TMPDIR") + "/check-version-mismatch" + debug_data_file = debug_directory + "/" + pn + os.makedirs(debug_directory, exist_ok=True) + data_lines.append("pv: %s\n" % pv) + + got_quick_match_result = False + # handle python3-xxx recipes quickly + __regex_python_module_version__ = re.compile(r"(?:^|.*:)Version: (?P<version>.*)$") + if "python3-" in pn: + version_check_cmd = "find %s -name 'METADATA' | xargs grep '^Version: '" % pkgd + try: + output = subprocess.check_output(version_check_cmd, shell=True).decode("utf-8") + data_lines.append("version_check_cmd: %s\n" % version_check_cmd) + data_lines.append("output:\n'''\n%s'''\n" % output) + possible_versions = [] + for line in output.split("\n"): + match = __regex_python_module_version__.match(line) + if match: + possible_versions.append(match.group("version")) + possible_versions = sorted(set(possible_versions)) + data_lines.append("possible versions: %s\n" % possible_versions) + if is_version_mismatch(possible_versions, pv): + data_lines.append("FINAL RESULT: MISMATCH (%s v.s. %s)\n\n" % (possible_versions, pv)) + bb.warn("Possible runtime versions %s do not match recipe version %s" % (possible_versions, pv)) + else: + data_lines.append("FINAL RESULT: MATCH (%s v.s. %s)\n\n" % (possible_versions, pv)) + got_quick_match_result = True + except: + data_lines.append("version_check_cmd: %s\n" % version_check_cmd) + data_lines.append("result: RUN_FAILED\n\n") + if got_quick_match_result: + if enable_debug: + with open(debug_data_file, "w") as f: + f.writelines(data_lines) + return + + # handle .pc files + version_check_cmd = "find %s -name '*.pc' | xargs grep -i version" % pkgd + try: + output = subprocess.check_output(version_check_cmd, shell=True).decode("utf-8") + data_lines.append("version_check_cmd: %s\n" % version_check_cmd) + data_lines.append("output:\n'''\n%s'''\n" % output) + possible_versions = get_possible_versions(output) + possible_versions = sorted(set(possible_versions)) + data_lines.append("possible versions: %s\n" % possible_versions) + if is_version_mismatch(possible_versions, pv): + if pn.startswith("lib"): + data_lines.append("FINAL RESULT: MISMATCH (%s v.s. %s)\n\n" % (possible_versions, pv)) + bb.warn("Possible runtime versions %s do not match recipe version %s" % (possible_versions, pv)) + got_quick_match_result = True + else: + data_lines.append("result: MISMATCH (%s v.s. %s)\n\n" % (possible_versions, pv)) + else: + data_lines.append("FINAL RESULT: MATCH (%s v.s. %s)\n\n" % (possible_versions, pv)) + got_quick_match_result = True + except: + data_lines.append("version_check_cmd: %s\n" % version_check_cmd) + data_lines.append("result: RUN_FAILED\n\n") + if got_quick_match_result: + if enable_debug: + with open(debug_data_file, "w") as f: + f.writelines(data_lines) + return + + skipped_directories = [".debug", "ptest", "installed-tests", "tests", "test", "__pycache__", "testcases"] + pkgd_libdir = pkgd + d.getVar("libdir") + pkgd_base_libdir = pkgd + d.getVar("base_libdir") + extra_exec_libdirs = [] + for root, dirs, files in os.walk(pkgd): + for dname in dirs: + fdir = os.path.join(root, dname) + if os.path.isdir(fdir) and fdir != pkgd_libdir and fdir != pkgd_base_libdir: + if fdir.startswith(pkgd_libdir) or fdir.startswith(pkgd_base_libdir): + for sd in skipped_directories: + if fdir.endswith("/" + sd) or ("/" + sd + "/") in fdir: + break + else: + extra_exec_libdirs.append(fdir) + for fname in files: + fpath = os.path.join(root, fname) + if os.path.isfile(fpath) and os.access(fpath, os.X_OK): + for sd in skipped_directories: + if ("/" + sd + "/") in fpath: + break + else: + if is_shared_library(fpath): + # we don't check shared libraries + continue + else: + executables.append(fpath) + if enable_debug: + data_lines.append("executables: %s\n" % executables) + + found_match = False + some_cmd_succeed = False + if not executables: + bb.debug(1, "No executable found for %s" % pn) + data_lines.append("FINAL RESULT: NO_EXECUTABLE_FOUND\n\n") + else: + # first we extend qemu_exec to include library path if needed + if extra_exec_libdirs: + qemu_exec += ":" + ":".join(extra_exec_libdirs) + for fexec in executables: + for version_option in ["--version", "-V", "-v", "--help"]: + version_check_cmd_full = "%s %s %s" % (qemu_exec, fexec, version_option) + version_check_cmd = version_check_cmd_full + #version_check_cmd = "%s %s" % (os.path.relpath(fexec, pkgd), version_option) + + try: + cwd_temp = d.getVar("TMPDIR") + "/check-version-mismatch/cwd-temp/" + pn + os.makedirs(cwd_temp, exist_ok=True) + # avoid pseudo to manage any file we create + sp_env = os.environ.copy() + sp_env["PSEUDO_UNLOAD"] = "1" + output = subprocess.check_output(version_check_cmd_full, + shell=True, + stderr=subprocess.STDOUT, + cwd=cwd_temp, + timeout=10, + env=sp_env).decode("utf-8") + some_cmd_succeed = True + data_lines.append("version_check_cmd: %s\n" % version_check_cmd) + data_lines.append("output:\n'''\n%s'''\n" % output) + if version_option == "--help": + max_lines = 5 + else: + max_lines = None + possible_versions = get_possible_versions(output, full_cmd=fexec, max_lines=max_lines) + if "." in pv: + possible_versions = [item for item in possible_versions if "." in item or item == "UNKNOWN"] + data_lines.append("possible versions: %s\n" % possible_versions) + if not possible_versions: + data_lines.append("result: NO_RUNTIME_VERSION_FOUND\n\n") + continue + possible_versions_all.extend(possible_versions) + possible_versions_all = sorted(set(possible_versions_all)) + if is_version_mismatch(possible_versions, pv): + data_lines.append("result: MISMATCH (%s v.s. %s)\n\n" % (possible_versions, pv)) + else: + found_match = True + data_lines.append("result: MATCH (%s v.s. %s)\n\n" % (possible_versions, pv)) + break + except: + data_lines.append("version_check_cmd: %s\n" % version_check_cmd) + data_lines.append("result: RUN_FAILED\n\n") + finally: + shutil.rmtree(cwd_temp) + if found_match: + break + if executables: + if found_match: + data_lines.append("FINAL RESULT: MATCH (%s v.s. %s)\n" % (possible_versions_all, pv)) + elif len(possible_versions_all) == 0: + if some_cmd_succeed: + bb.debug(1, "No valid runtime version found") + data_lines.append("FINAL RESULT: NO_VALID_RUNTIME_VERSION_FOUND\n") + else: + bb.debug(1, "All version check command failed") + data_lines.append("FINAL RESULT: RUN_FAILED\n") + else: + bb.warn("Possible runtime versions %s do not match recipe version %s" % (possible_versions_all, pv)) + data_lines.append("FINAL RESULT: MISMATCH (%s v.s. %s)\n" % (possible_versions_all, pv)) + + if enable_debug: + with open(debug_data_file, "w") as f: + f.writelines(data_lines) + + # clean up stale processes + process_name_common_prefix = "%s %s" % (' '.join(qemu_exec.split()[1:]), pkgd) + find_stale_process_cmd = "ps -e -o pid,args | grep -v grep | grep -F '%s'" % process_name_common_prefix + try: + stale_process_output = subprocess.check_output(find_stale_process_cmd, shell=True).decode("utf-8") + stale_process_pids = [] + for line in stale_process_output.split("\n"): + line = line.strip() + if not line: + continue + pid = line.split()[0] + stale_process_pids.append(pid) + for pid in stale_process_pids: + os.kill(int(pid), signal.SIGKILL) + except Exception as e: + bb.debug(1, "No stale process") +} + +addtask do_package_check_version_mismatch after do_package before do_build + +do_build[rdeptask] += "do_package_check_version_mismatch" +do_rootfs[recrdeptask] += "do_package_check_version_mismatch" + +SSTATETASKS += "do_package_check_version_mismatch" +do_package_check_version_mismatch[sstate-inputdirs] = "" +do_package_check_version_mismatch[sstate-outputdirs] = "" +python do_package_check_version_mismatch_setscene () { + sstate_setscene(d) +} +addtask do_package_check_version_mismatch_setscene
meta/classes/qemu.bbclass renamed from meta/classes-recipe/qemu.bbclass +0 -0
meta/conf/version-check.conf new file +14 -0
@@ -0,0 +1,14 @@ +INHERIT += "check-version-mismatch" +# we need ps command to clean stale processes +HOSTTOOLS += "ps" + +# Special cases that need to be handled. +# % has the same meaning as in bbappend files, that is, match any chars. +CHECK_VERSION_PV:pn-rust-llvm = "${LLVM_RELEASE}" +CHECK_VERSION_PV:pn-igt-gpu-tools = "${PV}-${PV}" +CHECK_VERSION_PV:pn-vim = "${@'.'.join(d.getVar('PV').split('.')[:-1])}" +CHECK_VERSION_PV:pn-vim-tiny = "${@'.'.join(d.getVar('PV').split('.')[:-1])}" +CHECK_VERSION_PV:pn-ncurses = "${PV}.%" +CHECK_VERSION_PV:pn-alsa-tools = "%" +CHECK_VERSION_PV:pn-gst-examples = "%" +CHECK_VERSION_PV:pn-libedit = "${@d.getVar('PV').split('-')[1]}"
Signature · 1 line
2.34.1
RFC PATCH 2/3 mc: fix buildpath QA regarding unzip
In reply to [email protected]From: Chen Qi <[email protected]>
When unzip-native is somehow introduced indirectly into the dependency chain, then we get buildpath QA issue. Pass a similar value as the ZIP to fix this issue.
--- meta/recipes-extended/mc/mc_4.8.33.bb | 1 + 1 file changed, 1 insertion(+)
meta/recipes-extended/mc/mc_4.8.33.bb +1 -0
@@ -32,6 +32,7 @@ CACHED_CONFIGUREVARS += "ac_cv_path_PERL_FOR_BUILD='/usr/bin/env perl'" CACHED_CONFIGUREVARS += "ac_cv_path_PYTHON='/usr/bin/env python'" CACHED_CONFIGUREVARS += "ac_cv_path_GREP='/usr/bin/env grep'" CACHED_CONFIGUREVARS += "ac_cv_path_ZIP='/usr/bin/zip'" +CACHED_CONFIGUREVARS += "ac_cv_path_UNZIP='/usr/bin/unzip'" CACHED_CONFIGUREVARS += "mc_cv_have_zipinfo=yes" do_install:append () {
Signature · 1 line
2.34.1
RFC PATCH 3/3 time: fix runtime version from UNKNOWN to 1.9
In reply to [email protected]From: Chen Qi <[email protected]>
The runtime version of time is determined by:
build-aux/git-version-gen .tarball-version
But there's no .tarball-version in the tarball. And time's last commit dates back to 20210105, it's very unlikely for them to generate a new tarball. So we add this file manually with ${PV} as the content.
Before the fix, time --version:
time (GNU Time) UNKNOWN
After the fix, time --version:
time (GNU Time) 1.9
--- meta/recipes-extended/time/time_1.9.bb | 6 ++++++ 1 file changed, 6 insertions(+)
meta/recipes-extended/time/time_1.9.bb +6 -0
@@ -21,3 +21,9 @@ SRC_URI[md5sum] = "d2356e0fe1c0b85285d83c6b2ad51b5f" SRC_URI[sha256sum] = "fbacf0c81e62429df3e33bda4cee38756604f18e01d977338e23306a3e3b521e" inherit autotools + +# We don't want our version to be "UNKNOWN", and time repo's latest commits dates back +# to 20210105. It's very unlikely it'll generate a new tarball with a .tarball-version included. +do_configure:prepend () { + echo ${PV} > ${S}/.tarball-version +}
Signature · 1 line
2.34.1
Re: [OE-core] [RFC PATCH 3/3] time: fix runtime version from UNKNOWN to 1.9
Quoted 10 lines · Chen Qi via lists.openembedded.org
From: Chen Qi <[email protected]>
The runtime version of time is determined by:
build-aux/git-version-gen .tarball-versionBut there's no .tarball-version in the tarball. And time's last commit dates back to 20210105, it's very unlikely for them to generate a new tarball. So we add this file manually with ${PV} as the content.
The upstream git repo has had activity since the 1.9 release so there _might_ be a new release in the future.
Can you link to the submitted fix for this (https://lists.gnu.org/archive/html/bug-time/2021-01/msg00000.html) and only write to .tarball-version if it doesn’t exist, so we don’t overwrite a file if it appears in a future release.
Ross
Re: [OE-core] [RFC PATCH 1/3] version-check.conf: add mechanism for checking version mismatch
In reply to [email protected]Quoted 2 lines · Chen Qi via lists.openembedded.org
The basic idea is to use qemu to run executables at build time, extract possible versions, and check if there's a mismatch found.
Should this check be in its own layer for now? If it's in core, there should be evidence that it's broadly useful and used (and no one's asked for it before as far as I'm aware), and it should be tested for correct behaviour and against regressions. I don't feel it meets these criteria at the moment.
Git fetcher has recently gained the ability to match up source revisions with tags specified in recipes, and will error out if SRCREV is set wrongly. We just need to update recipes to specify tags (with ${PV} in them somewhere, so they remain valid on version updates). So sending in those updates would be most welcome.
Alex
Re: [OE-core] [RFC PATCH 1/3] version-check.conf: add mechanism for checking version mismatch
Quoted 13 lines · Alexander Kanavin via lists.openembedded.org
Quoted 3 lines · Chen Qi via lists.openembedded.org
The basic idea is to use qemu to run executables at build time, extract
possible versions, and check if there's a mismatch found.Should this check be in its own layer for now? If it's in core, there should be evidence that it's broadly useful and used (and no one's asked for it before as far as I'm aware), and it should be tested for correct behaviour and against regressions. I don't feel it meets these
criteria at the moment.
Quoted 6 lines
Git fetcher has recently gained the ability to match up source revisions with tags specified in recipes, and will error out if SRCREV
is set wrongly. We just need to update recipes to specify tags (with ${PV} in them somewhere, so they remain valid on version updates). So sending in those updates would be most welcome.
The new SRCREV vs tag code was my first thought when seeing this patch too.
It is a nice idea to check the version information and having something somewhere to do it is nice but I'm not sure maintaining this in OE-Core is the right thing to do for it...
Cheers,
Richard
Re: [OE-core] [RFC PATCH 1/3] version-check.conf: add mechanism for checking version mismatch
Quoted 23 lines · Richard Purdie
Quoted 12 lines · Alexander Kanavin via lists.openembedded.org
Quoted 3 lines · Chen Qi via lists.openembedded.org
The basic idea is to use qemu to run executables at build time, extract
possible versions, and check if there's a mismatch found.Should this check be in its own layer for now? If it's in core, there should be evidence that it's broadly useful and used (and no one's asked for it before as far as I'm aware), and it should be tested for correct behaviour and against regressions. I don't feel it meets these
criteria at the moment.Quoted 6 lines
Git fetcher has recently gained the ability to match up source revisions with tags specified in recipes, and will error out if SRCREV
is set wrongly. We just need to update recipes to specify tags (with ${PV} in them somewhere, so they remain valid on version updates). So sending in those updates would be most welcome.The new SRCREV vs tag code was my first thought when seeing this patch too.
Quoted 7 lines
It is a nice idea to check the version information and having something somewhere to do it is nice but I'm not sure maintaining this in OE-Core is the right thing to do for it...
Cheers,
Richard
Got it. I'll maintain it in its own layer.
Patch 2/3 and patch 3/3 are fixes. Please consider merging them. I'll send out V2 according to Ross's advice.
I personally think this version check feature is useful for layer maintainers because it will detect a few problems I mentioned in the cover letter. So if people want this feature in the future and the decision is changed at that time, I can send out a new RFC with oeqa test cases added.
Regards,
Qi
Re: [OE-core] [RFC PATCH 1/3] version-check.conf: add mechanism for checking version mismatch
Hi Chen,
Quoted 30 lines · Chen Qi via lists.openembedded.org
Quoted 29 lines · Richard Purdie
Quoted 12 lines · Alexander Kanavin via lists.openembedded.org
Quoted 3 lines · Chen Qi via lists.openembedded.org
The basic idea is to use qemu to run executables at build time, extract
possible versions, and check if there's a mismatch found.Should this check be in its own layer for now? If it's in core, there should be evidence that it's broadly useful and used (and no one's asked for it before as far as I'm aware), and it should be tested for correct behaviour and against regressions. I don't feel it meets these
criteria at the moment.Quoted 6 lines
Git fetcher has recently gained the ability to match up source revisions with tags specified in recipes, and will error out if SRCREV
is set wrongly. We just need to update recipes to specify tags (with ${PV} in them somewhere, so they remain valid on version updates). So sending in those updates would be most welcome.The new SRCREV vs tag code was my first thought when seeing this patch too.
It is a nice idea to check the version information and having something somewhere to do it is nice but I'm not sure maintaining this in OE-Core is the right thing to do for it...
Cheers,
Quoted 8 lines
Quoted 1 line
Richard
Got it. I'll maintain it in its own layer.
Patch 2/3 and patch 3/3 are fixes. Please consider merging them. I'll send out V2 according to Ross's advice.
I personally think this version check feature is useful for layer maintainers because it will detect a few problems I mentioned in the cover letter. So if people want this feature in the future and the decision is changed at that time, I can send out a new RFC with oeqa test cases added.
If the CVE reporting tool is using PV, having a PV different from the actual version we're building is an issue. So running this tool regularly would be nice to make sure we're building what we say we are building?
Maybe we would want to run this on the autobuilder every now and then to catch issues before releases for examples? If that was the case, in which layer would we want to store this?
Cheers,
Quentin