oeqa: Run diffoscop on saved output
cover oeqa: Run diffoscop on saved output
Adds recipes to build the diffoscope tool and run it when the OEQA reproducible build test saves output to a local path. This should make it much easier to debug reproducible build issues from the autobuilder, since the published output can be easily viewed on the website.
Joshua Watt (4):
python: Add libarchive-c recipe
python: Add magic recipe
recipes-support: Add diffoscope recipe
oeqa: reproducible: Run diffoscope on saved output
meta/conf/distro/include/maintainers.inc | 3 +++ meta/lib/oeqa/selftest/cases/reproducible.py | 24 +++++++++++++------ .../python/python3-libarchive-c_2.9.bb | 17 +++++++++++++ .../python/python3-magic_0.4.15.bb | 19 +++++++++++++++ .../diffoscope/diffoscope_136.bb | 15 ++++++++++++ 5 files changed, 71 insertions(+), 7 deletions(-) create mode 100644 meta/recipes-devtools/python/python3-libarchive-c_2.9.bb create mode 100644 meta/recipes-devtools/python/python3-magic_0.4.15.bb create mode 100644 meta/recipes-support/diffoscope/diffoscope_136.bb
Signature · 1 line
2.23.0
PATCH 1/4 python: Add libarchive-c recipe
The libarchive python module is used by diffoscope tool to make build comparisons.
--- meta/conf/distro/include/maintainers.inc | 1 + .../python/python3-libarchive-c_2.9.bb | 17 +++++++++++++++++ 2 files changed, 18 insertions(+) create mode 100644 meta/recipes-devtools/python/python3-libarchive-c_2.9.bb
meta/conf/distro/include/maintainers.inc +1 -0
@@ -579,6 +579,7 @@ RECIPE_MAINTAINER_pn-python3-extras = "Oleksandr Kravchuk <open.source@oleksandr RECIPE_MAINTAINER_pn-python3-git = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER_pn-python3-gitdb = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER_pn-python3-iniparse = "Oleksandr Kravchuk <[email protected]>" +RECIPE_MAINTAINER_pn-python3-libarchive-c = "Joshua Watt <[email protected]>" RECIPE_MAINTAINER_pn-python3-mako = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER_pn-python3-nose = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER_pn-python3-numpy = "Oleksandr Kravchuk <[email protected]>"
meta/recipes-devtools/python/python3-libarchive-c_2.9.bb new file +17 -0
@@ -0,0 +1,17 @@ +SUMMARY = "Python interface to libarchive" +DESCRIPTION = "A Python interface to libarchive. It uses the standard ctypes module to \ + dynamically load and access the C library." +HOMEPAGE = "https://github.com/Changaco/python-libarchive-c" +LICENSE = "CC0-1.0" +LIC_FILES_CHKSUM = "file://LICENSE.md;md5=bcab380227a83bc147350b40a81e6ffc" + +PYPI_PACKAGE = "libarchive-c" + +inherit pypi setuptools3 + +SRC_URI[md5sum] = "083bd2cb0043c1e22a52cb9a05e31532" +SRC_URI[sha256sum] = "9919344cec203f5db6596a29b5bc26b07ba9662925a05e24980b84709232ef60" + +RDEPENDS_${PN} += "libarchive" + +BBCLASSEXTEND = "native"
Signature · 1 line
2.23.0
PATCH 2/4 python: Add magic recipe
In reply to Joshua WattThe python-magic module is used by diffoscope tool to make build comparisons.
--- meta/conf/distro/include/maintainers.inc | 1 + .../python/python3-magic_0.4.15.bb | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+) create mode 100644 meta/recipes-devtools/python/python3-magic_0.4.15.bb
meta/conf/distro/include/maintainers.inc +1 -0
@@ -580,6 +580,7 @@ RECIPE_MAINTAINER_pn-python3-git = "Oleksandr Kravchuk <open.source@oleksandr-kr RECIPE_MAINTAINER_pn-python3-gitdb = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER_pn-python3-iniparse = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER_pn-python3-libarchive-c = "Joshua Watt <[email protected]>" +RECIPE_MAINTAINER_pn-python3-magic = "Joshua Watt <[email protected]>" RECIPE_MAINTAINER_pn-python3-mako = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER_pn-python3-nose = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER_pn-python3-numpy = "Oleksandr Kravchuk <[email protected]>"
meta/recipes-devtools/python/python3-magic_0.4.15.bb new file +19 -0
@@ -0,0 +1,19 @@ +SUMMARY = "File type identification using libmagic" +DESCRIPTION = "This module uses ctypes to access the libmagic file type \ + identification library. It makes use of the local magic database and supports \ + both textual and MIME-type output." +HOMEPAGE = "http://github.com/ahupp/python-magic" + +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://LICENSE;md5=16a934f165e8c3245f241e77d401bb88" + +PYPI_PACKAGE = "python-magic" + +inherit pypi setuptools3 + +SRC_URI[md5sum] = "e384c95a47218f66c6501cd6dd45ff59" +SRC_URI[sha256sum] = "f3765c0f582d2dfc72c15f3b5a82aecfae9498bd29ca840d72f37d7bd38bfcd5" + +RDEPENDS_${PN} += "file" + +BBCLASSEXTEND = "native"
Signature · 1 line
2.23.0
PATCH 3/4 recipes-support: Add diffoscope recipe
In reply to Joshua WattDiffoscope is the universal diff tool, capable of comparing many different formats.
--- meta/conf/distro/include/maintainers.inc | 1 + meta/recipes-support/diffoscope/diffoscope_136.bb | 15 +++++++++++++++ 2 files changed, 16 insertions(+) create mode 100644 meta/recipes-support/diffoscope/diffoscope_136.bb
meta/conf/distro/include/maintainers.inc +1 -0
@@ -146,6 +146,7 @@ RECIPE_MAINTAINER_pn-dejagnu = "Nathan Rossi <[email protected]>" RECIPE_MAINTAINER_pn-depmodwrapper-cross = "Mark Hatle <[email protected]>" RECIPE_MAINTAINER_pn-desktop-file-utils = "Alexander Kanavin <[email protected]>" RECIPE_MAINTAINER_pn-dhcp = "Hongxu Jia <[email protected]>" +RECIPE_MAINTAINER_pn-diffoscope = "Joshua Watt <[email protected]>" RECIPE_MAINTAINER_pn-diffstat = "Chen Qi <[email protected]>" RECIPE_MAINTAINER_pn-diffutils = "Chen Qi <[email protected]>" RECIPE_MAINTAINER_pn-distcc = "Hongxu Jia <[email protected]>"
meta/recipes-support/diffoscope/diffoscope_136.bb new file +15 -0
@@ -0,0 +1,15 @@ +SUMMARY = "in-depth comparison of files, archives, and directories" +HOMEPAGE = "https://diffoscope.org/" +LICENSE = "GPL-3.0+" +LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504" + +PYPI_PACKAGE = "diffoscope" + +inherit pypi setuptools3 + +SRC_URI[md5sum] = "c84d8d308a40176ba2f5dc4abdbf6f73" +SRC_URI[sha256sum] = "0d6486d6eb6e0445ba21fee2e8bdd3a366ce786bfac98e00e5a95038b7815f15" + +RDEPENDS_${PN} += "binutils vim squashfs-tools python3-libarchive-c python3-magic" + +BBCLASSEXTEND = "native"
Signature · 1 line
2.23.0
PATCH 4/4 oeqa: reproducible: Run diffoscope on saved output
In reply to Joshua WattIf there are differing packages and they are being saved for review, automatically run diffoscope on them and include the output in the saved output. The output is currently done in HTML format since these are typically published on a webpage by the autobuilder.
--- meta/lib/oeqa/selftest/cases/reproducible.py | 24 ++++++++++++++------ 1 file changed, 17 insertions(+), 7 deletions(-)
meta/lib/oeqa/selftest/cases/reproducible.py +17 -7
@@ -1,7 +1,7 @@ # # SPDX-License-Identifier: MIT # -# Copyright 2019 by Garmin Ltd. or its subsidiaries +# Copyright 2019-2020 by Garmin Ltd. or its subsidiaries from oeqa.selftest.case import OESelftestTestCase from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars
@@ -167,10 +167,16 @@ class ReproducibleTests(OESelftestTestCase): return d def test_reproducible_builds(self): + def strip_topdir(s): + if s.startswith(self.topdir): + return s[len(self.topdir):] + return s + # Build native utilities self.write_config('') - bitbake("diffutils-native -c addto_recipe_sysroot") + bitbake("diffoscope-native diffutils-native -c addto_recipe_sysroot") diffutils_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "diffutils-native") + diffoscope_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "diffoscope-native") if self.save_results: os.makedirs(self.save_results, exist_ok=True)
@@ -206,18 +212,22 @@ class ReproducibleTests(OESelftestTestCase): if self.save_results: for d in result.different: - self.copy_file(d.reference, '/'.join([save_dir, d.reference])) - self.copy_file(d.test, '/'.join([save_dir, d.test])) + self.copy_file(d.reference, '/'.join([save_dir, 'packages', strip_topdir(d.reference)])) + self.copy_file(d.test, '/'.join([save_dir, 'packages', strip_topdir(d.test)])) if result.missing or result.different: fails.append("The following %s packages are missing or different: %s" % (c, '\n'.join(r.test for r in (result.missing + result.different)))) - if fails: - self.fail('\n'.join(fails)) - # Clean up empty directories if self.save_results: if not os.listdir(save_dir): os.rmdir(save_dir) + else: + self.logger.info('Running diffoscope') + runCmd(['diffoscope', '--no-default-limits', '--exclude-directory-metadata', '--html-dir', 'diff-html', 'reproducibleA', 'reproducibleB'], + native_sysroot=diffoscope_sysroot, ignore_status=True, cwd=os.path.join(save_dir, 'packages')) + + if fails: + self.fail('\n'.join(fails))
Signature · 1 line
2.23.0
Quoted 4 lines · Joshua Watt
Adds recipes to build the diffoscope tool and run it when the OEQA reproducible build test saves output to a local path. This should make it much easier to debug reproducible build issues from the autobuilder, since the published output can be easily viewed on the website.\
Thanks
Quoted 17 lines
Joshua Watt (4):
python: Add libarchive-c recipe
python: Add magic recipe
recipes-support: Add diffoscope recipe
oeqa: reproducible: Run diffoscope on saved outputmeta/conf/distro/include/maintainers.inc | 3 +++ meta/lib/oeqa/selftest/cases/reproducible.py | 24 +++++++++++++------ .../python/python3-libarchive-c_2.9.bb | 17 +++++++++++++ .../python/python3-magic_0.4.15.bb | 19 +++++++++++++++ .../diffoscope/diffoscope_136.bb | 15 ++++++++++++ 5 files changed, 71 insertions(+), 7 deletions(-) create mode 100644 meta/recipes-devtools/python/python3-libarchive-c_2.9.bb create mode 100644 meta/recipes-devtools/python/python3-magic_0.4.15.bb create mode 100644 meta/recipes-support/diffoscope/diffoscope_136.bb
Re: [PATCH 3/4] recipes-support: Add diffoscope recipe
In reply to Joshua WattHi Joshua
I think it needs to be ignored on musl
15:37:23 ERROR: Nothing RPROVIDES 'squashfs-tools' (but /mnt/jenkins/workspace/Yocto-world-musl/sources/openembedded-core/meta/recipes-support/diffoscope/diffoscope_136.bb RDEPENDS on or otherwise requires it)
15:37:23 squashfs-tools was skipped: incompatible with host
aarch64-yoe-linux-musl (not in COMPATIBLE_HOST)
Quoted 52 lines · Joshua Watt
Diffoscope is the universal diff tool, capable of comparing many different formats.
Signed-off-by Joshua Watt <[email protected]>--- meta/conf/distro/include/maintainers.inc | 1 + meta/recipes-support/diffoscope/diffoscope_136.bb | 15 +++++++++++++++ 2 files changed, 16 insertions(+) create mode 100644 meta/recipes-support/diffoscope/diffoscope_136.bbmeta/conf/distro/include/maintainers.inc +1
-0@@ -146,6 +146,7 @@ RECIPE_MAINTAINER_pn-dejagnu = "Nathan Rossi <[email protected]>" RECIPE_MAINTAINER_pn-depmodwrapper-cross = "Mark Hatle <[email protected]>" RECIPE_MAINTAINER_pn-desktop-file-utils = "Alexander Kanavin <[email protected]>" RECIPE_MAINTAINER_pn-dhcp = "Hongxu Jia <[email protected]>" +RECIPE_MAINTAINER_pn-diffoscope = "Joshua Watt <[email protected]>" RECIPE_MAINTAINER_pn-diffstat = "Chen Qi <[email protected]>" RECIPE_MAINTAINER_pn-diffutils = "Chen Qi <[email protected]>" RECIPE_MAINTAINER_pn-distcc = "Hongxu Jia <[email protected]>"meta/recipes-support/diffoscope/diffoscope_136.bb new file +15
-1@@ -0,0 +1,15 @@ +SUMMARY = "in-depth comparison of files, archives, and directories" +HOMEPAGE = "https://diffoscope.org/" +LICENSE = "GPL-3.0+" +LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504" + +PYPI_PACKAGE = "diffoscope" + +inherit pypi setuptools3 + +SRC_URI[md5sum] = "c84d8d308a40176ba2f5dc4abdbf6f73" +SRC_URI[sha256sum] = "0d6486d6eb6e0445ba21fee2e8bdd3a366ce786bfac98e00e5a95038b7815f15" + +RDEPENDS_${PN} += "binutils vim squashfs-tools python3-libarchive-c python3-magic" + +BBCLASSEXTEND = "native" --2.23.0
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [PATCH 3/4] recipes-support: Add diffoscope recipe
Quoted 4 lines · Khem Raj
Hi Joshua
I think it needs to be ignored on musl
Quoted 8 lines
15:37:23 ERROR: Nothing RPROVIDES 'squashfs-tools' (but
/mnt/jenkins/workspace/Yocto-world-musl/sources/openembedded-core/meta/recipes-support/diffoscope/diffoscope_136.bb
RDEPENDS on or otherwise requires it)
15:37:23 squashfs-tools was skipped: incompatible with host
aarch64-yoe-linux-musl (not inCOMPATIBLE_HOST
Would an RRECOMMENDS fix it?
Quoted 61 lines · Joshua Watt
Quoted 4 lines
Diffoscope is the universal diff tool, capable of comparing many different formats.
Review
Quoted 8 lines
Signed-off-by Joshua Watt <[email protected]>--- meta/conf/distro/include/maintainers.inc | 1 + meta/recipes-support/diffoscope/diffoscope_136.bb | 15 +++++++++++++++ 2 files changed, 16 insertions(+) create mode 100644 meta/recipes-support/diffoscope/diffoscope_136.bb(quoted diff) +0
-0b/meta/conf/distro/include/maintainers.inc
meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER_pn-dejagnu = "Nathan Rossi <
Quoted 4 lines
index
171a43615e3..4a267faabce100644meta/conf/distro/include/maintainers.inc +0
-0@@ -146,6 +146,7 @@ RECIPE_MAINTAINER_pn-dejagnu = "Nathan Rossi <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER_pn-dejagnu = "Nathan Rossi < · line 146
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER_pn-depmodwrapper-cross = "Mark Hatle <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER_pn-dejagnu = "Nathan Rossi < · line 147
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER_pn-desktop-file-utils = "Alexander Kanavin <Review
Quoted 6 lines
meta/conf/distro/include/maintainers.inc +1
-0RECIPE_MAINTAINER_pn-dhcp = "Hongxu Jia <[email protected]>" +RECIPE_MAINTAINER_pn-diffoscope = "Joshua Watt <[email protected]>" RECIPE_MAINTAINER_pn-diffstat = "Chen Qi <[email protected]>" RECIPE_MAINTAINER_pn-diffutils = "Chen Qi <[email protected]>" RECIPE_MAINTAINER_pn-distcc = "Hongxu Jia <[email protected]>"(quoted diff) +0
-0b/meta/recipes-support/diffoscope/diffoscope_136.bb
Quoted 6 lines
new file mode 100644
index 00000000000..a00f281deb0meta/recipes-support/diffoscope/diffoscope_136.bb new file +1
-0@@ -0,0 +1,15 @@ +SUMMARY = "in-depth comparison of files, archives, and directories"meta/recipes-support/diffoscope/diffoscope_136.bb line 11
Quoted 10 lines
meta/recipes-support/diffoscope/diffoscope_136.bb new file +10
-0+HOMEPAGE = "https://diffoscope.org/" +LICENSE = "GPL-3.0+" +LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504" + +PYPI_PACKAGE = "diffoscope" + +inherit pypi setuptools3 + +SRC_URI[md5sum] = "c84d8d308a40176ba2f5dc4abdbf6f73" +SRC_URI[sha256sum] ="
0d6486d6eb6e0445ba21fee2e8bdd3a366ce786bfac98e00e5a95038b7815f15"meta/recipes-support/diffoscope/diffoscope_136.bb line 13
Quoted 2 lines
meta/recipes-support/diffoscope/diffoscope_136.bb new file +2
-0+ +RDEPENDS_${PN} += "binutils vim squashfs-tools python3-libarchive-cpython3-magic"
Quoted 10 lines
meta/recipes-support/diffoscope/diffoscope_136.bb new file +2
-1+ +BBCLASSEXTEND = "native" --2.23.0
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Richard Purdie
AppliedRe: [PATCH 3/4] recipes-support: Add diffoscope recipe
Quoted 7 lines · Joshua Watt
Quoted 4 lines · Khem Raj
Hi Joshua
I think it needs to be ignored on musl
Quoted 8 lines
Quoted 6 lines
15:37:23 ERROR: Nothing RPROVIDES 'squashfs-tools' (but
/mnt/jenkins/workspace/Yocto-world-musl/sources/openembedded-core/meta/recipes-support/diffoscope/diffoscope_136.bb RDEPENDS on or otherwise requires it)
15:37:23 squashfs-tools was skipped: incompatible with host aarch64-yoe-linux-musl (not inCOMPATIBLE_HOSTWould an RRECOMMENDS fix it?
I merged your patch and one of mine to follow it adding a COMPATIBLE_HOST entry.
I see Khem is fixing some other software so we may end up not needing it but we can remove later.
Cheers,
Richard
Re: [PATCH 3/4] recipes-support: Add diffoscope recipe
Quoted 16 lines · Richard Purdie
Quoted 14 lines · Joshua Watt
Quoted 10 lines · Khem Raj
Hi Joshua
I think it needs to be ignored on musl
15:37:23 ERROR: Nothing RPROVIDES 'squashfs-tools' (but
/mnt/jenkins/workspace/Yocto-world-musl/sources/openembedded-core/meta/recipes-support/diffoscope/diffoscope_136.bb RDEPENDS on or otherwise requires it)
15:37:23 squashfs-tools was skipped: incompatible with host aarch64-yoe-linux-musl (not inCOMPATIBLE_HOST
Quoted 8 lines
Quoted 1 line
Would an RRECOMMENDS fix it?
I merged your patch and one of mine to follow it adding a
COMPATIBLE_HOSTentry.I see Khem is fixing some other software so we may end up not needing it but we can remove later.
right, we were unnecessarily masking squashfs-tools for musl
Quoted 4 lines
Cheers,
Richard
Quoted 7 lines · Joshua Watt
Adds recipes to build the diffoscope tool and run it when the OEQA reproducible build test saves output to a local path. This should make
it much easier to debug reproducible build issues from the autobuilder,
since the published output can be easily viewed on the website.
Quoted 5 lines
Joshua Watt (4):
python: Add libarchive-c recipe
python: Add magic recipe
recipes-support: Add diffoscope recipe
oeqa: reproducible: Run diffoscope on saved output
Thanks!
The first production use:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20200215-t1s21q9r/packages/diff-html/
:)
I am a bit puzzled/concerned about how this hasn't been removed from the system yet as it should have been, unless its the hashequiv problem with timestamps continuing to cause problems. Need to fix my patch...
Cheers,
Richard
Quoted 14 lines · Richard Purdie
Quoted 7 lines · Joshua Watt
Adds recipes to build the diffoscope tool and run it when the OEQA reproducible build test saves output to a local path. This should make
it much easier to debug reproducible build issues from the autobuilder,
since the published output can be easily viewed on the website.Quoted 5 lines
Joshua Watt (4):
python: Add libarchive-c recipe
python: Add magic recipe
recipes-support: Add diffoscope recipe
oeqa: reproducible: Run diffoscope on saved output
Quoted 9 lines
Thanks!
The first production use:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20200215-t1s21q9r/packages/diff-html/
:)
That's great!
Quoted 4 lines
I am a bit puzzled/concerned about how this hasn't been removed from the system yet as it should have been, unless its the hashequiv problem with timestamps continuing to cause problems. Need to fix my patch...
It looks like a directory ordering issue to me, since the timestamps are the same. One way to really flush these out would be to use disorderfs (
https://salsa.debian.org/reproducible-builds/disorderfs) which permutates the order in which entries are listed in a directory, but I think we should try to get "wider" coverage (more recipes) before we try that.
Quoted 5 lines
Cheers,
Richard
Quoted 33 lines · Joshua Watt
Quoted 14 lines · Richard Purdie
Quoted 7 lines · Joshua Watt
Adds recipes to build the diffoscope tool and run it when the OEQA reproducible build test saves output to a local path. This should make
it much easier to debug reproducible build issues from the autobuilder,
since the published output can be easily viewed on the website.Quoted 5 lines
Joshua Watt (4):
python: Add libarchive-c recipe
python: Add magic recipe
recipes-support: Add diffoscope recipe
oeqa: reproducible: Run diffoscope on saved outputQuoted 7 lines
Thanks!
The first production use:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20200215-t1s21q9r/packages/diff-html/
:)
That's great!
Quoted 6 lines
I am a bit puzzled/concerned about how this hasn't been removed from
the system yet as it should have been, unless its the hashequiv problem
with timestamps continuing to cause problems. Need to fix my patch...
Quoted 6 lines
It looks like a directory ordering issue to me, since the timestamps are the same. One way to really flush these out would be to use disorderfs (https://salsa.debian.org/reproducible-builds/disorderfs)
which permutates the order in which entries are listed in a directory, but I think we should try to get "wider" coverage (more recipes) before we try that.
Its more complicated unfortunately. We should have fixed the directory order issue with:
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=09c6a41b751d080f0c12fc4172a31d1dbe760b0b
However issues which shows signs of that bug still persisted in the reproducibility test results.
I theorised that since opkg-utils is in ABISAFE, the change wasn't accounted for. I therefore merged:
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=5fd3eb4bda651bddf4e4dc76d456333c70df4ddc
However with current master, we're seeing the above reproducible build failure. I checked in a build and its using this sstate object for comparison:
pub/sstate/b0/ca/sstate\:matchbox-config-gtk\:core2-64-poky-linux\:0.2\:r0\:core2-64\:3\:b0ca3b3559d37ba8de8ade8b282baa5082e612ef6590d3ff94fcebe38e9f17b9_package_write_ipk.tgz.siginfo
which is interesting in that its from 2nd Feb (predating both commits above) and doesn't show the above change to package_ipk.bbclass.
The change to package_ipk means the task's own basehash has to have changed which means it *has* to have rerun. Somehow hashequiv has then marked it as equivalent.
I guess the issue may be that in some builds, this did generate output that was equivalent and things have then raced so this "broken" sstate object has persisted.
The question then becomes, how to we get this out the cache?! I'm not entirely sure how we can :(
(obviously I can cleansstate a few things which I'll do now but its not good this can't recover even through metadata changes)
Cheers,
Richard
Re: [OE-core] [PATCH 1/4] python: Add libarchive-c recipe
In reply to Joshua WattHi Joshua,
I encounter below error when use the libarchive module, could you help to take a look?
# python3 Python 3.8.2 (default, Jun 8 2020, 06:35:32) [GCC 10.1.0] on linux Type "help", "copyright", "credits" or "license" for more information.
Quoted 1 line
Quoted 1 line
Quoted 1 line
import libarchive
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.8/site-packages/libarchive/__init__.py",
line 1, in <module>
from .entry import ArchiveEntry
File "/usr/lib64/python3.8/site-packages/libarchive/entry.py", line
6, in <module>
from . import ffi
File "/usr/lib64/python3.8/site-packages/libarchive/ffi.py", line
109, in <module>
errno = ffi('errno', [c_archive_p], c_int) File "/usr/lib64/python3.8/site-packages/libarchive/ffi.py", line 96,
in ffi
f = getattr(libarchive, 'archive_'+name)
File "/usr/lib64/python3.8/ctypes/__init__.py", line 386, in __getattr__
func = self.__getitem__(name)
File "/usr/lib64/python3.8/ctypes/__init__.py", line 391, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: python3: undefined symbol: archive_errno
Quoted 1 line
Quoted 1 line
Quoted 1 line
Thanks,
Quoted 46 lines · Joshua Watt
The libarchive python module is used by diffoscope tool to make build comparisons.
Signed-off-by Joshua Watt <[email protected]>--- meta/conf/distro/include/maintainers.inc | 1 + .../python/python3-libarchive-c_2.9.bb | 17 +++++++++++++++++ 2 files changed, 18 insertions(+)create mode 100644 meta/recipes-devtools/python/python3-libarchive-c_2.9.bb
meta/conf/distro/include/maintainers.inc +1
-0@@ -579,6 +579,7 @@ RECIPE_MAINTAINER_pn-python3-extras = "Oleksandr Kravchuk <open.source@oleksandr RECIPE_MAINTAINER_pn-python3-git = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER_pn-python3-gitdb = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER_pn-python3-iniparse = "Oleksandr Kravchuk <[email protected]>" +RECIPE_MAINTAINER_pn-python3-libarchive-c = "Joshua Watt <[email protected]>" RECIPE_MAINTAINER_pn-python3-mako = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER_pn-python3-nose = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER_pn-python3-numpy = "Oleksandr Kravchuk <[email protected]>"meta/recipes-devtools/python/python3-libarchive-c_2.9.bb new file +17
-0@@ -0,0 +1,17 @@ +SUMMARY = "Python interface to libarchive" +DESCRIPTION = "A Python interface to libarchive. It uses the standard ctypes module to \ + dynamically load and access the C library." +HOMEPAGE = "https://github.com/Changaco/python-libarchive-c" +LICENSE = "CC0-1.0" +LIC_FILES_CHKSUM = "file://LICENSE.md;md5=bcab380227a83bc147350b40a81e6ffc" + +PYPI_PACKAGE = "libarchive-c" + +inherit pypi setuptools3 + +SRC_URI[md5sum] = "083bd2cb0043c1e22a52cb9a05e31532" +SRC_URI[sha256sum] = "9919344cec203f5db6596a29b5bc26b07ba9662925a05e24980b84709232ef60" + +RDEPENDS_${PN} += "libarchive" + +BBCLASSEXTEND = "native"