Skip to contentSkip to conversation

Create selftest suite for go-vendor.bbclass

PATCH New John Ripple 1 participant 4 messages Jul 22

lore
Share & tools
lore
#0 lore raw

v4 cover Create selftest suite for go-vendor.bbclass

This patch series adds a selftest suit for the go-vendor.bbclass and fixes a couple of bugs in the go-vendor.bbclass.

v1: Initial patch series
v2:
- Remove patch that requires no git history for a layer.
- Create go vendor recipes in meta-selftest instead of creating a dummy layer.
- Use fakeroot for the do_vendor_unlink task.
v3:
- Exclude go vendor test recipes from the world build.
v4:
- Use common MIT license for the go-vendor-src-uri recipe.

John Ripple (3):
go-vendor.bbclass: Fix test file exclusion and license propagation oeqa/selftest: Add go-vendor.bbclass selftest suite go-vendor.bbclass: Run vendor_unlink task under fakeroot

 .../recipes-test/govendor/files/modules.txt   |   7 +
 .../govendor/go-vendor-src-uri_1.0.bb         |  18 ++
 .../govendor/govendor-real-test_1.0.bb        |  29 +++
 meta/classes/go-vendor.bbclass                |   9 +-
 meta/lib/oeqa/selftest/cases/govendor.py      | 179 ++++++++++++++++++
 5 files changed, 238 insertions(+), 4 deletions(-)
 create mode 100644 meta-selftest/recipes-test/govendor/files/modules.txt
 create mode 100644 meta-selftest/recipes-test/govendor/go-vendor-src-uri_1.0.bb
 create mode 100644 meta-selftest/recipes-test/govendor/govendor-real-test_1.0.bb
 create mode 100644 meta/lib/oeqa/selftest/cases/govendor.py
#1 lore raw

PATCH v4 1/3 go-vendor.bbclass: Fix test file exclusion and license propagation

Two bugs in do_go_vendor:

1. The shutil.ignore_patterns glob was "*._test.go" (with an extra dot), which never matches standard Go test files (*_test.go). Remove the extra dot.

2. The license-propagation block used os.path.join(src, "LICENSE") as the copy destination, writing into the already-processed vendor.fetch source tree rather than into the vendor destination directory. Switch to os.path.join(dst, "LICENSE") so the LICENSE lands in vendor/ where go tools and licence scanners expect it. Also fix the guard condition from not os.path.exists(subdir) (checking an unrelated relative name) to not os.path.exists(subdirLicense) (checking the actual target).

---
 meta/classes/go-vendor.bbclass | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
meta/classes/go-vendor.bbclass +3 -3
index e879d629a8..85a3c1b586 100644
@@ -141,7 +141,7 @@ python do_go_vendor() {
         shutil.copytree(src, dst, symlinks=True, dirs_exist_ok=True, \
             ignore=shutil.ignore_patterns(".git", \
                                             "vendor", \
-                                            "*._test.go"))
+                                            "*_test.go"))
 
         # If the root directory has a LICENSE file but not the subdir
         # we copy the root license to the sub module since the license
@@ -149,9 +149,9 @@ python do_go_vendor() {
         # see https://go.dev/ref/mod#vcs-license
         if subdir:
             rootdirLicese = os.path.join(rootdir, "LICENSE")
-            subdirLicense = os.path.join(src, "LICENSE")
+            subdirLicense = os.path.join(dst, "LICENSE")
 
-            if not os.path.exists(subdir) and \
+            if not os.path.exists(subdirLicense) and \
                 os.path.exists(rootdirLicese):
                 shutil.copy2(rootdirLicese, subdirLicense)
 
#2 lore raw

PATCH v4 2/3 oeqa/selftest: Add go-vendor.bbclass selftest suite

In reply to John Ripple

Add govendor.py with two test classes that cover the behaviors fixed and defined in go-vendor.bbclass:

GoVendorSrcUriTests (offline, ~10 s): unit tests for go_src_uri() parsed via bb.tinfoil; verifies the basic URI shape, path/pathmajor/subdir/replaces encoding, and that non-git VCS omits the git-specific nobranch/protocol flags.

GoVendorTaskTests: runs do_go_vendor once against the real recipetool-go-test project and checks that external deps land in vendor/, *_test.go files are excluded, pathmajor=/v5 falls back to the repo root when no v5/ subdir exists, modules.txt is reproduced in vendor/, a local replacement (./is) appears as a symlink, and the S/src/GO_IMPORT/vendor -> S/src/import/vendor symlink is created.

Exclude the govendor test recipes from world builds since they are only meant to be built directly by the govendor test suite.

---
 .../recipes-test/govendor/files/modules.txt   |   7 +
 .../govendor/go-vendor-src-uri_1.0.bb         |  18 ++
 .../govendor/govendor-real-test_1.0.bb        |  29 +++
 meta/lib/oeqa/selftest/cases/govendor.py      | 179 ++++++++++++++++++
 4 files changed, 233 insertions(+)
 create mode 100644 meta-selftest/recipes-test/govendor/files/modules.txt
 create mode 100644 meta-selftest/recipes-test/govendor/go-vendor-src-uri_1.0.bb
 create mode 100644 meta-selftest/recipes-test/govendor/govendor-real-test_1.0.bb
 create mode 100644 meta/lib/oeqa/selftest/cases/govendor.py
meta-selftest/recipes-test/govendor/files/modules.txt new file +7 -0
new file mode 100644
index 0000000000..f64f5245b5
@@ -0,0 +1,7 @@
+# github.com/godbus/dbus/v5 v5.1.0
+## explicit; go 1.12
+github.com/godbus/dbus/v5
+# github.com/matryer/is v1.4.1 => ./is
+## explicit; go 1.14
+github.com/matryer/is
+# github.com/matryer/is => ./is
\ No newline at end of file
meta-selftest/recipes-test/govendor/go-vendor-src-uri_1.0.bb new file +18 -0
new file mode 100644
index 0000000000..6f0364b7da
@@ -0,0 +1,18 @@
+SUMMARY = "Selftest fixture: go_src_uri parameter variants"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" 
+
+# Parsed via tinfoil for its GO_SRC_URI_* variables only; never actually built
+# (it has no real SRC_URI, so do_go_vendor would fail).
+EXCLUDE_FROM_WORLD = "1"
+
+inherit go-vendor
+
+GO_IMPORT = "example.com/testpkg"
+
+GO_SRC_URI_BASIC     = "${@go_src_uri('github.com/foo/bar', 'v1.0.0')}"
+GO_SRC_URI_PATH      = "${@go_src_uri('github.com/foo/bar', 'v2.1.0', path='github.com/foo/bar/v2')}"
+GO_SRC_URI_PATHMAJOR = "${@go_src_uri('github.com/foo/bar', 'v2.0.0', pathmajor='/v2')}"
+GO_SRC_URI_SUBDIR    = "${@go_src_uri('github.com/foo/bar', 'v1.0.0', subdir='sub/pkg')}"
+GO_SRC_URI_REPLACES  = "${@go_src_uri('github.com/baz/qux', 'v1.0.0', replaces='../local/baz')}"
+GO_SRC_URI_SVN       = "${@go_src_uri('example.com/svnmod', 'r42', vcs='svn')}"
meta-selftest/recipes-test/govendor/govendor-real-test_1.0.bb new file +29 -0
new file mode 100644
index 0000000000..8086922e10
@@ -0,0 +1,29 @@
+SUMMARY = "go-vendor integration test: recipetool-go-test"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://src/${GO_IMPORT}/LICENSE;md5=4e3933dd47afbf115e484d11385fb3bd"
+
+# Only meant to be built directly by GoVendorTaskTests (bitbake('%s -c go_vendor')),
+# not as part of a general world build.
+EXCLUDE_FROM_WORLD = "1"
+
+# Native variant lets GoVendorTaskTests.test_do_compile_succeeds exercise the
+# real go toolchain without requiring a full target Go cross-toolchain.
+BBCLASSEXTEND = "native"
+
+inherit go-vendor
+
+GO_IMPORT = "git.yoctoproject.org/recipetool-go-test"
+
+# modules.txt for recipetool-go-test at SRCREV:
+#   - github.com/godbus/dbus/v5 v5.1.0  (external, fetched via go_src_uri)
+#   - github.com/matryer/is v1.4.1       (local replacement => ./is)
+# Use go mod vendor to generate the modules.txt
+SRC_URI = "\
+    git://${GO_IMPORT};name=recipetool-go-test;branch=main;protocol=https;nobranch=1;destsuffix=${GO_SRCURI_DESTSUFFIX} \
+    ${@go_src_uri('github.com/godbus/dbus', 'v5.1.0', path='github.com/godbus/dbus/v5', pathmajor='/v5')} \
+    file://modules.txt \
+    "
+SRCREV_FORMAT = "recipetool-go-test"
+# git.yoctoproject.org/recipetool-go-test main branch HEAD
+SRCREV_recipetool-go-test = "c3e213c01b6c1406b430df03ef0d1ae77de5d2f7"
+SRCREV_github.com.godbus.dbus.v5 = "e523abc905595cf17fb0001a7d77eaaddfaa216d"
meta/lib/oeqa/selftest/cases/govendor.py new file +179 -0

179 lines not on this page · raw message

#3 lore raw

PATCH v4 3/3 go-vendor.bbclass: Run vendor_unlink task under fakeroot

In reply to John Ripple

do_install and do_package both run under fakeroot (pseudo), which keeps pseudo's ownership/inode database in sync with what is actually on disk under D. do_vendor_unlink runs between them but was never marked fakeroot, so its os.unlink() of the vendor symlink happened outside of pseudo's view: the file disappeared from disk but pseudo's database still held a stale record for that path/inode.

The next fakeroot task to touch that path (do_package) then finds the on-disk state and pseudo's database disagreeing, and pseudo aborts with "path mismatch" / "inode mismatch" errors, failing do_package.

Mark do_vendor_unlink fakeroot, matching do_install and do_package, so its filesystem operations are tracked by pseudo consistently.

---
 meta/classes/go-vendor.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
meta/classes/go-vendor.bbclass +2 -1
index 85a3c1b586..5bb4e8e876 100644
@@ -40,13 +40,14 @@ def go_src_uri(repo, version, path=None, subdir=None, \
 
     return src_uri
 
-python do_vendor_unlink() {
+fakeroot python do_vendor_unlink() {
     go_import = d.getVar('GO_IMPORT')
     linkname = os.path.join(d.getVar('D') + d.getVar('libdir'), 'go', 'src', go_import, 'vendor')
     if os.path.islink(linkname):
         os.unlink(linkname)
 }
 
+do_vendor_unlink[depends] += "virtual/fakeroot-native:do_populate_sysroot"
 addtask vendor_unlink before do_package after do_install
 
 python do_go_vendor() {