Skip to contentSkip to conversation

package/go-native: add a new download only package for go compilers

PATCH In review Charles Hardin 2 participants 5 messages Jul 7, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 package/go-native: add a new download only package for go compilers

The go-native is mainly for the aarch64 support since that was added after the go-bootstrap written in C was dropped. So, to get a cross compiler for go a host/native compiler needs to be available to generate the setup for the needed target. This is using the same go version as the expected cross-compiler to keep the packages in sync. A specific version could be used.

---
 package/Config.in.host           |  1 +
 package/go-native/Config.in.host | 12 ++++++++
 package/go-native/go-native.hash |  5 ++++
 package/go-native/go-native.mk   | 48 ++++++++++++++++++++++++++++++++
 package/go/Config.in.host        |  6 ++--
 package/go/go.mk                 | 12 +++++++-
 6 files changed, 81 insertions(+), 3 deletions(-)
 create mode 100644 package/go-native/Config.in.host
 create mode 100644 package/go-native/go-native.hash
 create mode 100644 package/go-native/go-native.mk
package/Config.in.host +1 -0
index dcadbfdfc1..d7932f048b 100644
@@ -41,6 +41,7 @@ menu "Host utilities"
 	source "package/gnupg/Config.in.host"
 	source "package/go/Config.in.host"
 	source "package/go-bootstrap/Config.in.host"
+	source "package/go-native/Config.in.host"
 	source "package/google-breakpad/Config.in.host"
 	source "package/gptfdisk/Config.in.host"
 	source "package/imagemagick/Config.in.host"
package/go-native/Config.in.host new file +12 -0
new file mode 100644
index 0000000000..7e978e02b5
@@ -0,0 +1,12 @@
+config BR2_PACKAGE_HOST_GO_NATIVE_ARCH_SUPPORTS
+	bool
+	# See https://go.dev/dl/ for supported architecture downloads
+	default y if BR2_HOSTARCH = "x86"
+	default y if BR2_HOSTARCH = "x86_64"
+	default y if BR2_HOSTARCH = "arm"
+	default y if BR2_HOSTARCH = "aarch64"
+
+config BR2_PACKAGE_HOST_GO_NATIVE
+	bool "host go-native"
+	depends on BR2_PACKAGE_HOST_GO_NATIVE_ARCH_SUPPORTS
+	default y if !BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS
package/go-native/go-native.hash new file +5 -0
new file mode 100644
index 0000000000..3af2f51abc
@@ -0,0 +1,5 @@
+# From https://go.dev/dl/
+sha256  45a34313664d66087d49b46ca63a3ecf292f56e9e63bd439829b5eaf0270586d  go1.19.10.linux-386.tar.gz
+sha256  8b045a483d3895c6edba2e90a9189262876190dbbd21756870cdd63821810677  go1.19.10.linux-amd64.tar.gz
+sha256  df98698821211c819e8b2420c77a0f802d989e377718578a31b1f91f6be2c5b4  go1.19.10.linux-arm64.tar.gz
+sha256  937f885c35f191be4c6a618868ae0dbbb15b0213925a1a104d122eba02622356  go1.19.10.linux-armv6l.tar.gz
package/go-native/go-native.mk new file +48 -0

48 lines not on this page · raw message

package/go/Config.in.host +4 -2
index ded02d3b3a..4dbbe9a221 100644
@@ -2,7 +2,8 @@
 config BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS
 	bool
 	default y
-	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS \
+		|| BR2_PACKAGE_HOST_GO_NATIVE_ARCH_SUPPORTS
 	depends on (BR2_arm && BR2_TOOLCHAIN_SUPPORTS_PIE) || BR2_aarch64 \
 		|| BR2_i386 || BR2_x86_64 || BR2_powerpc64le \
 		|| BR2_mips64 || BR2_mips64el || BR2_riscv || BR2_s390x
@@ -28,4 +29,5 @@ config BR2_PACKAGE_HOST_GO_TARGET_CGO_LINKING_SUPPORTS
 config BR2_PACKAGE_HOST_GO_HOST_ARCH_SUPPORTS
 	bool
 	default y
-	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS \
+		|| BR2_PACKAGE_HOST_GO_NATIVE_ARCH_SUPPORTS
package/go/go.mk +11 -1
index 545d2117b7..af2e3b2f74 100644
@@ -12,7 +12,11 @@ GO_LICENSE = BSD-3-Clause
 GO_LICENSE_FILES = LICENSE
 GO_CPE_ID_VENDOR = golang
 
+ifeq ($(BR2_PACKAGE_HOST_GO_NATIVE),y)
+HOST_GO_DEPENDENCIES = host-go-native
+else
 HOST_GO_DEPENDENCIES = host-go-bootstrap
+endif
 HOST_GO_GOPATH = $(HOST_DIR)/share/go-path
 HOST_GO_HOST_CACHE = $(HOST_DIR)/share/host-go-cache
 HOST_GO_ROOT = $(HOST_DIR)/lib/go
@@ -123,10 +127,16 @@ HOST_GO_HOST_ENV = \
 
 # The go build system is not compatible with ccache, so use
 # HOSTCC_NOCCACHE.  See https://github.com/golang/go/issues/11685.
+ifeq ($(BR2_PACKAGE_HOST_GO_NATIVE),y)
 HOST_GO_MAKE_ENV = \
+	GOROOT_BOOTSTRAP=$(HOST_GO_NATIVE_ROOT)
+else
+HOST_GO_MAKE_ENV = \
+	GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_ROOT)
+endif
+HOST_GO_MAKE_ENV += \
 	GO111MODULE=off \
 	GOCACHE=$(HOST_GO_HOST_CACHE) \
-	GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_ROOT) \
 	GOROOT_FINAL=$(HOST_GO_ROOT) \
 	GOROOT="$(@D)" \
 	GOBIN="$(@D)/bin" \
Signature · 6 lines
2.39.2 (Apple Git-143)

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw

Hi Charles,

This is not necessary with this patch series:

https://patchwork.ozlabs.org/project/buildroot/list/?series=358451

We can correctly bootstrap on arm64.

Quoted 7 lines · Charles Hardin

The go-native is mainly for the aarch64 support since that was added after the go-bootstrap written in C was dropped. So, to get a cross compiler for go a host/native compiler needs to be available to generate the setup for the needed target. This is using the same go version as the expected cross-compiler to keep the packages in sync. A specific version could be used.

Thanks,
Christian Stewart
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#2 lore raw

Hi Charles,

Quoted 5 lines · Christian Stewart

This is not necessary with this patch series:

https://patchwork.ozlabs.org/project/buildroot/list/?series=358451

We can correctly bootstrap on arm64.

I should clarify: actually go bootstrap stage1 cannot work on arm64 (you're correct on this).

I had submitted this patch as well to use the host "go" in this case:

https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/

Similar to how we use host gcc.

However, I understand that your series intends to add downloading the binaries for the Go compiler to Buildroot as well.

This is potentially useful but I still submit that we need to merge the above series updating to Go 1.20 and splitting the bootstrap into two phases before merging the go-native series.

I'll start testing your patch as well.

Thanks,
Christian

Quoted 8 lines · Charles Hardin
Quoted 7 lines

The go-native is mainly for the aarch64 support since that was added after the go-bootstrap written in C was dropped. So, to get a cross compiler for go a host/native compiler needs to be available to generate the setup for the needed target. This is using the same go version as the expected cross-compiler to keep the packages in sync. A specific version could be used.

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#3 lore raw

Christian

Quoted 14 lines · Christian Stewart

Hi Charles,

Quoted 5 lines · Christian Stewart

This is not necessary with this patch series:

https://patchwork.ozlabs.org/project/buildroot/list/?series=358451

We can correctly bootstrap on arm64.

I should clarify: actually go bootstrap stage1 cannot work on arm64 (you're correct on this).

Quoted 8 lines

I had submitted this patch as well to use the host "go" in this case:

https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/

Similar to how we use host gcc.

Weird - i was looking for a fix and never saw - perhaps i didn't have the magic google search keywords. So, this was "our fix" to just get something working specifically on a aarch64 builds. This is our annual timeline of let's take our patches we have locally and send them upstream to see what happens and see if we can drop our local patches in favor of upstream fixes.

Quoted 8 lines

However, I understand that your series intends to add downloading the binaries for the Go compiler to Buildroot as well.

This is potentially useful but I still submit that we need to merge the above series updating to Go 1.20 and splitting the bootstrap into two phases before merging the go-native series.

I just went for a quick and dirty to just use a native instead of doing the bootstrap - we have a major issue in that we don't have many x86_64 build systems anymore and we are having to move to aarch64 builds almost exclusively. So, not having a way to get the compiler going was becoming a headache.

Quoted 3 lines

I'll start testing your patch as well.

This is not meant to replace - more of a - here is a way that worked for us, if there is a better way that fits into buildroot then please just merge that. Right now we don't have anything in the proper mainline so everything is a deviation at this time.

Quoted 13 lines

Thanks,
Christian

Quoted 1 line

On Fri, Jul 7, 2023 at 11:51 AM Charles Hardin <[email protected]>

Quoted 7 lines · wrote:
Quoted 7 lines

The go-native is mainly for the aarch64 support since that was added after the go-bootstrap written in C was dropped. So, to get a cross compiler for go a host/native compiler needs to be available to generate the setup for the needed target. This is using the same go version as the expected cross-compiler to keep the packages in sync. A specific version could be used.

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#4 lore raw

Hi Charles,

Quoted 42 lines · Charles Hardin

Christian

Quoted 14 lines · Christian Stewart

Hi Charles,

Quoted 5 lines · Christian Stewart

This is not necessary with this patch series:

https://patchwork.ozlabs.org/project/buildroot/list/?series=358451

We can correctly bootstrap on arm64.

I should clarify: actually go bootstrap stage1 cannot work on arm64 (you're correct on this).

Quoted 8 lines

I had submitted this patch as well to use the host "go" in this case:

https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/

Similar to how we use host gcc.

Weird - i was looking for a fix and never saw - perhaps i didn't have the magic google search keywords. So, this was "our fix" to just get something working specifically on a aarch64 builds. This is our annual timeline of let's take our patches we have locally and send them upstream to see what happens and see if we can drop our local patches in favor of upstream fixes.

Quoted 8 lines

However, I understand that your series intends to add downloading the binaries for the Go compiler to Buildroot as well.

This is potentially useful but I still submit that we need to merge the above series updating to Go 1.20 and splitting the bootstrap into two phases before merging the go-native series.

Quoted 6 lines

I just went for a quick and dirty to just use a native instead of doing the bootstrap - we have a major issue in that we don't have many x86_64 build systems anymore and we are having to move to aarch64 builds almost exclusively. So, not having a way to get the compiler going was becoming a headache.

Ah - okay this makes sense. In this case I recommend using my patch to use the host Go compiler to bootstrap the Go compiler.

This still compiles whatever version of Go buildroot has in the tree from source using the host Go compiler to bootstrap.

Thomas, all; I do request that we merge the go 1.20 series and the "use host compiler to bootstrap on unsupported architectures" patch asap as these are significant blockers for some users (as seen here)

Thanks!

Christian

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot