Add support for using a pre-compiled Go compiler
cover Add support for using a pre-compiled Go compiler
Improvement from the buildroot TODO list. This patch add supports for pre-built version of the Go compiler.
Right now this patch introduce a new virtual package called 'goc' to do the selection between two provider:
- go
- go-bin
Right now there is some variable from the 'go' package are used in multiple go provider packages.
In an upcoming patch all the go related packages will move to a 'go' subfolder that will hold all the common variables.
Thomas Perale (4):
package/goc: new virtual package package/pkg-golang: set dependencies to host-goc package/go-bin: new package support/testing: add tests for Go
fs/oci/Config.in | 1 + package/Config.in.host | 2 ++ package/balena-engine/Config.in | 1 + package/containerd/Config.in | 1 + package/crucible/Config.in | 1 + package/delve/Config.in | 1 + package/docker-cli/Config.in | 1 + package/docker-compose/Config.in | 1 + package/docker-engine/Config.in | 1 + package/docker-proxy/Config.in | 1 + package/embiggen-disk/Config.in | 1 + package/flannel/Config.in | 1 + package/gitlab-runner/Config.in | 1 + package/go-bin/Config.in.host | 5 +++ package/go-bin/go-bin.hash | 4 +++ package/go-bin/go-bin.mk | 20 ++++++++++++ package/go/go.mk | 2 ++ package/goc/Config.in.host | 39 +++++++++++++++++++++++ package/goc/goc.mk | 7 +++++ package/gocryptfs/Config.in | 1 + package/mender-artifact/Config.in.host | 1 + package/mender-connect/Config.in | 1 + package/mender/Config.in | 1 + package/moby-buildkit/Config.in | 1 + package/moby-buildkit/Config.in.host | 1 + package/nerdctl/Config.in | 1 + package/pkg-golang.mk | 4 +-- package/runc/Config.in | 1 + package/tinifier/Config.in | 1 + package/wtfutil/Config.in | 1 + support/testing/tests/package/test_go.py | 40 ++++++++++++++++++++++++ 31 files changed, 143 insertions(+), 2 deletions(-) create mode 100644 package/go-bin/Config.in.host create mode 100644 package/go-bin/go-bin.hash create mode 100644 package/go-bin/go-bin.mk create mode 100644 package/goc/Config.in.host create mode 100644 package/goc/goc.mk create mode 100644 support/testing/tests/package/test_go.py
Signature · 6 lines
2.37.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH 1/4 package/goc: new virtual package
This virtual package implement the already existing go compiler compiled from source and shows it on the host package menu.
The usage of a virtual package on top of the go compiler enable the supports of for multiple providers for the Go compiler. Similarly to the rust compiler, a pre-built go compiler package will be available in parallel to the current compiled go compiler package.
--- package/Config.in.host | 1 + package/go/go.mk | 2 ++ package/goc/Config.in.host | 15 +++++++++++++++ package/goc/goc.mk | 7 +++++++ 4 files changed, 25 insertions(+) create mode 100644 package/goc/Config.in.host create mode 100644 package/goc/goc.mk
package/Config.in.host +1 -0
@@ -38,6 +38,7 @@ menu "Host utilities" source "package/gnupg/Config.in.host" source "package/go/Config.in.host" source "package/go-bootstrap/Config.in.host" + source "package/goc/Config.in.host" source "package/google-breakpad/Config.in.host" source "package/gptfdisk/Config.in.host" source "package/imagemagick/Config.in.host"
package/go/go.mk +2 -0
@@ -12,6 +12,8 @@ GO_LICENSE = BSD-3-Clause GO_LICENSE_FILES = LICENSE GO_CPE_ID_VENDOR = golang +HOST_GO_PROVIDES = host-goc + HOST_GO_DEPENDENCIES = host-go-bootstrap HOST_GO_GOPATH = $(HOST_DIR)/share/go-path HOST_GO_HOST_CACHE = $(HOST_DIR)/share/host-go-cache
package/goc/Config.in.host new file +15 -0
@@ -0,0 +1,15 @@ +config BR2_PACKAGE_HOST_GOC + bool "host go compiler" + depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS + help + Compiler for the Go language + + https://go.dev + +if BR2_PACKAGE_HOST_GOC + +config BR2_PACKAGE_PROVIDES_HOST_GOC + string + default "host-go" + +endif
package/goc/goc.mk new file +7 -0
@@ -0,0 +1,7 @@ +################################################################################ +# +# goc +# +################################################################################ + +$(eval $(host-virtual-package))
Signature · 6 lines
2.37.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH 2/4 package/pkg-golang: set dependencies to host-goc
In reply to Thomas Perale via buildrootUse 'host-goc' instead of 'host-go' to build one of the implementation of the 'goc' package instead of building the 'go' package by default.
Also select 'goc' in every package that are built by the go compiler to keep the dependencies in the 'menuconfig' corrects.
--- fs/oci/Config.in | 1 + package/balena-engine/Config.in | 1 + package/containerd/Config.in | 1 + package/crucible/Config.in | 1 + package/delve/Config.in | 1 + package/docker-cli/Config.in | 1 + package/docker-compose/Config.in | 1 + package/docker-engine/Config.in | 1 + package/docker-proxy/Config.in | 1 + package/embiggen-disk/Config.in | 1 + package/flannel/Config.in | 1 + package/gitlab-runner/Config.in | 1 + package/gocryptfs/Config.in | 1 + package/mender-artifact/Config.in.host | 1 + package/mender-connect/Config.in | 1 + package/mender/Config.in | 1 + package/moby-buildkit/Config.in | 1 + package/moby-buildkit/Config.in.host | 1 + package/nerdctl/Config.in | 1 + package/pkg-golang.mk | 4 ++-- package/runc/Config.in | 1 + package/tinifier/Config.in | 1 + package/wtfutil/Config.in | 1 + 23 files changed, 24 insertions(+), 2 deletions(-)
fs/oci/Config.in +1 -0
@@ -1,6 +1,7 @@ config BR2_TARGET_ROOTFS_OCI bool "oci image" depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS + select BR2_PACKAGE_HOST_GOC help Build an OCI (Open Container Initiative) image.
package/balena-engine/Config.in +1 -0
@@ -11,6 +11,7 @@ config BR2_PACKAGE_BALENA_ENGINE select BR2_PACKAGE_UTIL_LINUX_MOUNT # runtime select BR2_PACKAGE_UTIL_LINUX_MOUNTPOINT # runtime select BR2_PACKAGE_UTIL_LINUX_PIVOT_ROOT # runtime + select BR2_PACKAGE_HOST_GOC help balenaEngine is a new container engine purpose-built for embedded and IoT use cases and compatible with Docker
package/containerd/Config.in +1 -0
@@ -11,6 +11,7 @@ config BR2_PACKAGE_CONTAINERD select BR2_PACKAGE_UTIL_LINUX_PIVOT_ROOT select BR2_PACKAGE_UTIL_LINUX_MOUNTPOINT select BR2_PACKAGE_UTIL_LINUX_MOUNT + select BR2_PACKAGE_HOST_GOC help containerd is a daemon to control runC.
package/crucible/Config.in +1 -0
@@ -1,6 +1,7 @@ config BR2_PACKAGE_CRUCIBLE bool "crucible" depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS + select BR2_PACKAGE_HOST_GOC help Crucible is a One-Time-Programmable (OTP) fusing tool for the i.MX family.
package/delve/Config.in +1 -0
@@ -7,6 +7,7 @@ config BR2_PACKAGE_DELVE_ARCH_SUPPORTS default y if BR2_x86_64 depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS depends on BR2_PACKAGE_HOST_GO_TARGET_CGO_LINKING_SUPPORTS + select BR2_PACKAGE_HOST_GOC config BR2_PACKAGE_DELVE bool "delve"
package/docker-cli/Config.in +1 -0
@@ -3,6 +3,7 @@ config BR2_PACKAGE_DOCKER_CLI depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS depends on BR2_PACKAGE_HOST_GO_TARGET_CGO_LINKING_SUPPORTS depends on BR2_TOOLCHAIN_HAS_THREADS + select BR2_PACKAGE_HOST_GOC help Docker is a platform to build, ship, and run applications as lightweight containers.
package/docker-compose/Config.in +1 -0
@@ -4,6 +4,7 @@ config BR2_PACKAGE_DOCKER_COMPOSE depends on BR2_PACKAGE_HOST_GO_TARGET_CGO_LINKING_SUPPORTS depends on BR2_TOOLCHAIN_HAS_THREADS depends on BR2_PACKAGE_DOCKER_CLI + select BR2_PACKAGE_HOST_GOC help Multi-container applications with the Docker CLI.
package/docker-engine/Config.in +1 -0
@@ -9,6 +9,7 @@ config BR2_PACKAGE_DOCKER_ENGINE select BR2_PACKAGE_CONTAINERD # runtime dependency select BR2_PACKAGE_DOCKER_PROXY # runtime dependency select BR2_PACKAGE_IPTABLES # runtime dependency + select BR2_PACKAGE_HOST_GOC help Docker is a platform to build, ship, and run applications as lightweight containers.
package/docker-proxy/Config.in +1 -0
@@ -3,6 +3,7 @@ config BR2_PACKAGE_DOCKER_PROXY depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS depends on BR2_PACKAGE_HOST_GO_TARGET_CGO_LINKING_SUPPORTS depends on BR2_TOOLCHAIN_HAS_THREADS + select BR2_PACKAGE_HOST_GOC help Libnetwork is a Container Network Model that provides a consistent programming interface and the required network
package/embiggen-disk/Config.in +1 -0
@@ -7,6 +7,7 @@ config BR2_PACKAGE_EMBIGGEN_DISK depends on BR2_USE_MMU # util-linux select BR2_PACKAGE_UTIL_LINUX # sfdisk select BR2_PACKAGE_UTIL_LINUX_BINARIES # sfdisk + select BR2_PACKAGE_HOST_GOC help embiggen-disk is a tool to resize disk partitions at runtime.
package/flannel/Config.in +1 -0
@@ -8,6 +8,7 @@ config BR2_PACKAGE_FLANNEL depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS depends on BR2_PACKAGE_HOST_GO_TARGET_CGO_LINKING_SUPPORTS depends on BR2_TOOLCHAIN_HAS_THREADS + select BR2_PACKAGE_HOST_GOC help Flannel is a virtual network that gives a subnet to each host for use with container runtimes.
package/gitlab-runner/Config.in +1 -0
@@ -16,6 +16,7 @@ config BR2_PACKAGE_GITLAB_RUNNER select BR2_PACKAGE_OPENSSL_FORCE_LIBOPENSSL # runtime select BR2_PACKAGE_LIBOPENSSL_BIN # runtime select BR2_PACKAGE_TAR # runtime + select BR2_PACKAGE_HOST_GOC help GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in
package/gocryptfs/Config.in +1 -0
@@ -2,6 +2,7 @@ config BR2_PACKAGE_GOCRYPTFS bool "gocryptfs" depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS depends on BR2_TOOLCHAIN_HAS_THREADS + select BR2_PACKAGE_HOST_GOC help gocryptfs is an encrypted FUSE overlay filesystem.
package/mender-artifact/Config.in.host +1 -0
@@ -1,6 +1,7 @@ config BR2_PACKAGE_HOST_MENDER_ARTIFACT bool "host mender-artifact" depends on BR2_PACKAGE_HOST_GO_HOST_ARCH_SUPPORTS + select BR2_PACKAGE_HOST_GOC help The mender-artifact tool is a CLI implementation of the Mender artifacts library.
package/mender-connect/Config.in +1 -0
@@ -9,6 +9,7 @@ config BR2_PACKAGE_MENDER_CONNECT select BR2_PACKAGE_LIBGLIB2 select BR2_PACKAGE_MENDER # runtime select BR2_PACKAGE_OPENSSL + select BR2_PACKAGE_HOST_GOC help mender-connect is a daemon responsible for handling bidirectional (websocket) communication with the Mender
package/mender/Config.in +1 -0
@@ -6,6 +6,7 @@ config BR2_PACKAGE_MENDER select BR2_PACKAGE_OPENSSL select BR2_PACKAGE_LIBOPENSSL_ENABLE_MD4 if BR2_PACKAGE_LIBOPENSSL select BR2_PACKAGE_LIBOPENSSL_ENABLE_RMD160 if BR2_PACKAGE_LIBOPENSSL + select BR2_PACKAGE_HOST_GOC help Mender is an open source over-the-air (OTA) software updater for embedded Linux devices. Mender comprises a client
package/moby-buildkit/Config.in +1 -0
@@ -5,6 +5,7 @@ config BR2_PACKAGE_MOBY_BUILDKIT depends on BR2_TOOLCHAIN_HAS_THREADS depends on !BR2_TOOLCHAIN_USES_UCLIBC depends on BR2_USE_MMU + select BR2_PACKAGE_HOST_GOC select BR2_PACKAGE_CGROUPFS_MOUNT if !BR2_PACKAGE_SYSTEMD # runtime dependency help BuildKit is a toolkit for converting source code to build
package/moby-buildkit/Config.in.host +1 -0
@@ -1,6 +1,7 @@ config BR2_PACKAGE_HOST_MOBY_BUILDKIT bool "host moby-buildkit" depends on BR2_PACKAGE_HOST_GO_HOST_ARCH_SUPPORTS + select BR2_PACKAGE_HOST_GOC help BuildKit is a toolkit for converting source code to build artifacts in an efficient expressive and repeatable manner.
package/nerdctl/Config.in +1 -0
@@ -6,6 +6,7 @@ config BR2_PACKAGE_NERDCTL depends on !BR2_TOOLCHAIN_USES_UCLIBC # containerd depends on BR2_USE_MMU # util-linux select BR2_PACKAGE_CONTAINERD # runtime + select BR2_PACKAGE_HOST_GOC help Docker-compatible CLI for containerd, controlling runc.
package/pkg-golang.mk +2 -2
@@ -49,8 +49,8 @@ $(2)_BUILD_OPTS += \ # Target packages need the Go compiler on the host at download time (for # vendoring), and at build and install time. -$(2)_DOWNLOAD_DEPENDENCIES += host-go -$(2)_DEPENDENCIES += host-go +$(2)_DOWNLOAD_DEPENDENCIES += host-goc +$(2)_DEPENDENCIES += host-goc $(2)_BUILD_TARGETS ?= .
package/runc/Config.in +1 -0
@@ -4,6 +4,7 @@ config BR2_PACKAGE_RUNC depends on BR2_PACKAGE_HOST_GO_TARGET_CGO_LINKING_SUPPORTS depends on BR2_TOOLCHAIN_HAS_THREADS depends on !BR2_TOOLCHAIN_USES_UCLIBC # no fexecve + select BR2_PACKAGE_HOST_GOC help runC is a CLI tool for spawning and running containers according to the OCI specification.
package/tinifier/Config.in +1 -0
@@ -1,6 +1,7 @@ config BR2_PACKAGE_TINIFIER bool "tinifier" depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS + select BR2_PACKAGE_HOST_GOC help CLI tool for images compressing
package/wtfutil/Config.in +1 -0
@@ -2,6 +2,7 @@ config BR2_PACKAGE_WTFUTIL bool "wtfutil" depends on BR2_USE_MMU # fork() depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS + select BR2_PACKAGE_HOST_GOC help WTF is the personal information dashboard for your terminal.
Signature · 6 lines
2.37.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH 3/4 package/go-bin: new package
In reply to Thomas Perale via buildrootThis package provides a pre-built version of the go compiler for the host target.
This enable the user to select between the pre-built and the compiled version of the go compiler. The pre-built version will be used by default when a go compiler is required.
The virtual package 'goc' creates a menu that let the user select which variant of the go compiler to use.
The supported host platform for the pre-built compiler are actually the same as the one for 'go-bootstrap' no further verification are required than the one already in place.
For now GO_BIN_VERSION and GO_VERSION are set the same by hand but in next version this variable should be in common. Because the order of the '.mk' file is not guaranteed this will be investigated in a next patch.
--- package/Config.in.host | 1 + package/go-bin/Config.in.host | 5 +++++ package/go-bin/go-bin.hash | 4 ++++ package/go-bin/go-bin.mk | 20 ++++++++++++++++++++ package/goc/Config.in.host | 26 +++++++++++++++++++++++++- 5 files changed, 55 insertions(+), 1 deletion(-) create mode 100644 package/go-bin/Config.in.host create mode 100644 package/go-bin/go-bin.hash create mode 100644 package/go-bin/go-bin.mk
package/Config.in.host +1 -0
@@ -37,6 +37,7 @@ menu "Host utilities" source "package/genpart/Config.in.host" source "package/gnupg/Config.in.host" source "package/go/Config.in.host" + source "package/go-bin/Config.in.host" source "package/go-bootstrap/Config.in.host" source "package/goc/Config.in.host" source "package/google-breakpad/Config.in.host"
package/go-bin/Config.in.host new file +5 -0
@@ -0,0 +1,5 @@ +config BR2_PACKAGE_HOST_GO_BIN_HOST_ARCH + string + default 386 if BR2_HOSTARCH = "x86" + default amd64 if BR2_HOSTARCH = "x86_64" + default arm64 if BR2_HOSTARCH = "arm"
package/go-bin/go-bin.hash new file +4 -0
@@ -0,0 +1,4 @@ +# sha256 checksum from https://go.dev/dl/ +sha256 9acc57342400c5b0c2da07b5b01b50da239dd4a7fad41a1fb56af8363ef4133f go1.19.1.linux-386.tar.gz +sha256 acc512fbab4f716a8f97a8b3fbaa9ddd39606a28be6c2515ef7c6c6311acffde go1.19.1.linux-amd64.tar.gz +sha256 49960821948b9c6b14041430890eccee58c76b52e2dbaafce971c3c38d43df9f go1.19.1.linux-arm64.tar.gz
package/go-bin/go-bin.mk new file +20 -0
@@ -0,0 +1,20 @@ +################################################################################ +# +# go-bin +# +################################################################################ + +GO_BIN_VERSION = 1.19.1 +GO_BIN_SITE = https://go.dev/dl +GO_BIN_LICENSE = BSD-3-Clause +GO_BIN_LICENSE_FILES = LICENSE + +HOST_GO_BIN_PROVIDES = host-goc + +HOST_GO_BIN_SOURCE = go$(GO_BIN_VERSION).linux-$(call qstrip, $(BR2_PACKAGE_HOST_GO_BIN_HOST_ARCH)).tar.gz + +define HOST_GO_BIN_INSTALL_CMDS + $(HOST_GO_INSTALL_CMDS) +endef + +$(eval $(host-generic-package))
package/goc/Config.in.host +25 -1
@@ -8,8 +8,32 @@ config BR2_PACKAGE_HOST_GOC if BR2_PACKAGE_HOST_GOC +choice + prompt "Go compiler variant" + default BR2_PACKAGE_HOST_GO_BIN + help + Select a Go compiler variant. + + Default to 'host-go-bin'. + +config BR2_PACKAGE_HOST_GO + bool "host go" + depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS + help + This package will build the go compiler for the host. + +config BR2_PACKAGE_HOST_GO_BIN + bool "host go (pre-built)" + help + This package will install pre-built versions of the compiler + for the host. + +endchoice + config BR2_PACKAGE_PROVIDES_HOST_GOC string - default "host-go" + # Default to host-go-bin + default "host-go-bin" if BR2_PACKAGE_HOST_GO_BIN + default "host-go" if BR2_PACKAGE_HOST_GO endif
Signature · 6 lines
2.37.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH 4/4 support/testing: add tests for Go
In reply to Thomas Perale via buildrootTest the support of both the pre-built Go compiler and the Go compiler built from source.
The test consists of compiling and installing in the root file system of a ARM vexpress QEMU system. The tests pass if the program can be ran correctly on the target.
--- support/testing/tests/package/test_go.py | 40 ++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 support/testing/tests/package/test_go.py
support/testing/tests/package/test_go.py new file +40 -0
@@ -0,0 +1,40 @@ +import os + +import infra.basetest + + +class TestGoBase(infra.basetest.BRTest): + + def login(self): + cpio_file = os.path.join(self.builddir, "images", "rootfs.cpio") + self.emulator.boot(arch="armv5", + kernel="builtin", + options=["-initrd", cpio_file]) + self.emulator.login() + + +class TestGoBin(TestGoBase): + config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \ + """ + BR2_TARGET_ROOTFS_CPIO=y + BR2_PACKAGE_HOST_GOC=y + BR2_PACKAGE_TINIFIER=y + """ + + def test_run(self): + self.login() + self.assertRunOk("tinifier -h") + + +class TestGo(TestGoBase): + config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \ + """ + BR2_TARGET_ROOTFS_CPIO=y + BR2_PACKAGE_HOST_GOC=y + BR2_PACKAGE_HOST_GO=y + BR2_PACKAGE_TINIFIER=y + """ + + def test_run(self): + self.login() + self.assertRunOk("tinifier -h")
Signature · 6 lines
2.37.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot