Create selftest suite for go-vendor.bbclass
v2 cover Create selftest suite for go-vendor.bbclass
This patch series adds a selftest suit for the go-vendor.bbclass and fixes a couple of bugs in the go-vendor.bbclass.
John Ripple (3):
go-vendor.bbclass: Fix test file exclusion and license propagation oeqa/selftest: Add go-vendor.bbclass selftest suite go-vendor.bbclass: Run vendor_unlink task under fakeroot
.../recipes-test/govendor/files/modules.txt | 6 + .../govendor/go-vendor-src-uri_1.0.bb | 13 ++ .../govendor/govendor-real-test_1.0.bb | 20 +++ meta/classes/go-vendor.bbclass | 9 +- meta/lib/oeqa/selftest/cases/govendor.py | 169 ++++++++++++++++++ 5 files changed, 213 insertions(+), 4 deletions(-) create mode 100644 meta-selftest/recipes-test/govendor/files/modules.txt create mode 100644 meta-selftest/recipes-test/govendor/go-vendor-src-uri_1.0.bb create mode 100644 meta-selftest/recipes-test/govendor/govendor-real-test_1.0.bb create mode 100644 meta/lib/oeqa/selftest/cases/govendor.py
PATCH v2 1/3 go-vendor.bbclass: Fix test file exclusion and license propagation
Two bugs in do_go_vendor:
1. The shutil.ignore_patterns glob was "*._test.go" (with an extra dot), which never matches standard Go test files (*_test.go). Remove the extra dot.
2. The license-propagation block used os.path.join(src, "LICENSE") as the copy destination, writing into the already-processed vendor.fetch source tree rather than into the vendor destination directory. Switch to os.path.join(dst, "LICENSE") so the LICENSE lands in vendor/ where go tools and licence scanners expect it. Also fix the guard condition from not os.path.exists(subdir) (checking an unrelated relative name) to not os.path.exists(subdirLicense) (checking the actual target).
--- meta/classes/go-vendor.bbclass | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
meta/classes/go-vendor.bbclass +3 -3
@@ -141,7 +141,7 @@ python do_go_vendor() { shutil.copytree(src, dst, symlinks=True, dirs_exist_ok=True, \ ignore=shutil.ignore_patterns(".git", \ "vendor", \ - "*._test.go")) + "*_test.go")) # If the root directory has a LICENSE file but not the subdir # we copy the root license to the sub module since the license
@@ -149,9 +149,9 @@ python do_go_vendor() { # see https://go.dev/ref/mod#vcs-license if subdir: rootdirLicese = os.path.join(rootdir, "LICENSE") - subdirLicense = os.path.join(src, "LICENSE") + subdirLicense = os.path.join(dst, "LICENSE") - if not os.path.exists(subdir) and \ + if not os.path.exists(subdirLicense) and \ os.path.exists(rootdirLicese): shutil.copy2(rootdirLicese, subdirLicense)
PATCH v2 2/3 oeqa/selftest: Add go-vendor.bbclass selftest suite
In reply to John RippleAdd govendor.py with two test classes that cover the behaviors fixed and defined in go-vendor.bbclass:
GoVendorSrcUriTests (offline, ~10 s): unit tests for go_src_uri() parsed via bb.tinfoil; verifies the basic URI shape, path/pathmajor/subdir/replaces encoding, and that non-git VCS omits the git-specific nobranch/protocol flags.
GoVendorTaskTests: runs do_go_vendor once against the real recipetool-go-test project and checks that external deps land in vendor/, *_test.go files are excluded, pathmajor=/v5 falls back to the repo root when no v5/ subdir exists, modules.txt is reproduced in vendor/, a local replacement (./is) appears as a symlink, and the S/src/GO_IMPORT/vendor -> S/src/import/vendor symlink is created.
--- .../recipes-test/govendor/files/modules.txt | 6 + .../govendor/go-vendor-src-uri_1.0.bb | 13 ++ .../govendor/govendor-real-test_1.0.bb | 20 +++ meta/lib/oeqa/selftest/cases/govendor.py | 169 ++++++++++++++++++ 4 files changed, 208 insertions(+) create mode 100644 meta-selftest/recipes-test/govendor/files/modules.txt create mode 100644 meta-selftest/recipes-test/govendor/go-vendor-src-uri_1.0.bb create mode 100644 meta-selftest/recipes-test/govendor/govendor-real-test_1.0.bb create mode 100644 meta/lib/oeqa/selftest/cases/govendor.py
meta-selftest/recipes-test/govendor/files/modules.txt new file +6 -0
@@ -0,0 +1,6 @@ +# github.com/godbus/dbus/v5 v5.1.0 +## explicit; go 1.18 +github.com/godbus/dbus/v5 +# github.com/matryer/is v1.4.1 => ./is +## explicit; go 1.14 +github.com/matryer/is
meta-selftest/recipes-test/govendor/go-vendor-src-uri_1.0.bb new file +13 -0
@@ -0,0 +1,13 @@ +SUMMARY = "Selftest fixture: go_src_uri parameter variants" +LICENSE = "CLOSED" + +inherit go-vendor + +GO_IMPORT = "example.com/testpkg" + +GO_SRC_URI_BASIC = "${@go_src_uri('github.com/foo/bar', 'v1.0.0')}" +GO_SRC_URI_PATH = "${@go_src_uri('github.com/foo/bar', 'v2.1.0', path='github.com/foo/bar/v2')}" +GO_SRC_URI_PATHMAJOR = "${@go_src_uri('github.com/foo/bar', 'v2.0.0', pathmajor='/v2')}" +GO_SRC_URI_SUBDIR = "${@go_src_uri('github.com/foo/bar', 'v1.0.0', subdir='sub/pkg')}" +GO_SRC_URI_REPLACES = "${@go_src_uri('github.com/baz/qux', 'v1.0.0', replaces='../local/baz')}" +GO_SRC_URI_SVN = "${@go_src_uri('example.com/svnmod', 'r42', vcs='svn')}"
meta-selftest/recipes-test/govendor/govendor-real-test_1.0.bb new file +20 -0
@@ -0,0 +1,20 @@ +SUMMARY = "go-vendor integration test: recipetool-go-test" +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://src/${GO_IMPORT}/LICENSE;md5=4e3933dd47afbf115e484d11385fb3bd" + +inherit go-vendor + +GO_IMPORT = "git.yoctoproject.org/recipetool-go-test" + +# modules.txt for recipetool-go-test at SRCREV: +# - github.com/godbus/dbus/v5 v5.1.0 (external, fetched via go_src_uri) +# - github.com/matryer/is v1.4.1 (local replacement => ./is) +SRC_URI = "\ + git://${GO_IMPORT};name=recipetool-go-test;branch=main;protocol=https;nobranch=1;destsuffix=${GO_SRCURI_DESTSUFFIX} \ + ${@go_src_uri('github.com/godbus/dbus', 'v5.1.0', path='github.com/godbus/dbus/v5', pathmajor='/v5')} \ + file://modules.txt \ + " +SRCREV_FORMAT = "recipetool-go-test" +# git.yoctoproject.org/recipetool-go-test main branch HEAD +SRCREV_recipetool-go-test = "c3e213c01b6c1406b430df03ef0d1ae77de5d2f7" +SRCREV_github.com.godbus.dbus.v5 = "e523abc905595cf17fb0001a7d77eaaddfaa216d"
meta/lib/oeqa/selftest/cases/govendor.py new file +169 -0
169 lines not on this page · raw message
Re: [OE-core] [meta][meta-selftest][PATCH v2 2/3] oeqa/selftest: Add go-vendor.bbclass selftest suite
Quoted 14 lines · John Ripple via lists.openembedded.org
Add
govendor.pywith two test classes that cover the behaviors fixed and defined in go-vendor.bbclass:GoVendorSrcUriTests (offline, ~10 s): unit tests for
go_src_uri()parsed via bb.tinfoil; verifies the basic URI shape, path/pathmajor/subdir/replaces encoding, and that non-git VCS omits the git-specific nobranch/protocol flags.GoVendorTaskTests: runs
do_go_vendoronce against the real recipetool-go-test project and checks that external deps land in vendor/, *_test.go files are excluded, pathmajor=/v5 falls back to the repo root when no v5/ subdir exists, modules.txt is reproduced in vendor/, a local replacement (./is) appears as a symlink, and the S/src/GO_IMPORT/vendor -> S/src/import/vendor symlink is created.
Quoted 2 lines
Signed-off-by John Ripple <[email protected]>---
Hi John,
Thanks for your patch.
We get a single failure with this patch, during the reproducible.ReproducibleTests.test_reproducible_builds selftest:
https://autobuilder.yoctoproject.org/valkyrie/#/builders/37/builds/4392
Can you have a look at the issue?
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v2 3/3 go-vendor.bbclass: Run vendor_unlink task under fakeroot
In reply to John Rippledo_install and do_package both run under fakeroot (pseudo), which keeps pseudo's ownership/inode database in sync with what is actually on disk under D. do_vendor_unlink runs between them but was never marked fakeroot, so its os.unlink() of the vendor symlink happened outside of pseudo's view: the file disappeared from disk but pseudo's database still held a stale record for that path/inode.
The next fakeroot task to touch that path (do_package) then finds the on-disk state and pseudo's database disagreeing, and pseudo aborts with "path mismatch" / "inode mismatch" errors, failing do_package.
Mark do_vendor_unlink fakeroot, matching do_install and do_package, so its filesystem operations are tracked by pseudo consistently.
--- meta/classes/go-vendor.bbclass | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
meta/classes/go-vendor.bbclass +2 -1
@@ -40,13 +40,14 @@ def go_src_uri(repo, version, path=None, subdir=None, \ return src_uri -python do_vendor_unlink() { +fakeroot python do_vendor_unlink() { go_import = d.getVar('GO_IMPORT') linkname = os.path.join(d.getVar('D') + d.getVar('libdir'), 'go', 'src', go_import, 'vendor') if os.path.islink(linkname): os.unlink(linkname) } +do_vendor_unlink[depends] += "virtual/fakeroot-native:do_populate_sysroot" addtask vendor_unlink before do_package after do_install python do_go_vendor() {