Skip to contentSkip to conversation

go package fixes

PATCH Superseded Anisse Astier 2 participants 8 messages May 11, 2018

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Anisse Astier

author
#0 lore raw

cover go package fixes

Hi,

In this patch series, you'll find a fix to the build issues in the go package, as well as a debug feature and a minor version bump.

Regards,

Anisse

--

Anisse Astier (3):
go: patch to fix cross-compilation support
go: honor building with V=1 for debugging purposes
go: bump to version 1.10.2

 DEVELOPERS                                         |  3 ++
 ...dist-explicit-option-for-crosscompilation.patch | 52 ++++++++++++++++++++++
 package/go/go.hash                                 |  2 +-
 package/go/go.mk                                   | 36 +++++----------
 4 files changed, 68 insertions(+), 25 deletions(-)
 create mode 100644 package/go/0001-cmd-dist-explicit-option-for-crosscompilation.patch
Signature · 1 line
2.13.6
author
#1 lore raw

PATCH 1/3 go: patch to fix cross-compilation support

This patches the go toolchain with upstream-proposed patch by Angelo in order to fix cross-compilation when building a go toolchain for a target that has the same os and arch as host, but a different libc. This should fix the buildroot autobuild failures on x86_64 musl/uclibc targets.

This patch should go as soon as it's accepted or a similar upstream feature is merged.

I'm adding myself to the DEVELOPERS file in order to get CC-ed to future build failures.

Reference: https://github.com/golang/go/issues/25177
Reference: https://golang.org/cl/112156

Cc Angelo Compagnucci <[email protected]>
Signed-off-by Anisse Astier <[email protected]>
---
 DEVELOPERS                                         |  3 ++
 ...dist-explicit-option-for-crosscompilation.patch | 52 ++++++++++++++++++++++
 package/go/go.mk                                   | 29 +++---------
 3 files changed, 62 insertions(+), 22 deletions(-)
 create mode 100644 package/go/0001-cmd-dist-explicit-option-for-crosscompilation.patch
DEVELOPERS +3 -0
index b910a86a8c..dabea65b03 100644
@@ -153,6 +153,9 @@ F:	package/python-pydal/
 F:	package/python-web2py/
 F:	package/sysdig/
 
+N:	Anisse Astier <[email protected]>
+F:	package/go/
+
 N:	Antony Pavlov <[email protected]>
 F:	package/lsscsi/
package/go/0001-cmd-dist-explicit-option-for-crosscompilation.patch new file +52 -0

52 lines not on this page · raw message

package/go/go.mk +7 -22
index 4d55e16cc3..37bb9801c2 100644
@@ -58,13 +58,7 @@ else
 HOST_GO_CGO_ENABLED = 0
 endif
 
-# The go build system doesn't have the notion of cross compiling, but just the
-# notion of architecture.  When the host and target architectures are different
-# it expects to be given a target cross compiler in CC_FOR_TARGET.  When the
-# architectures are the same it will use CC_FOR_TARGET for both host and target
-# compilation.  To work around this limitation build and install a set of
-# compiler and tool binaries built with CC_FOR_TARGET set to the host compiler.
-# Also, the go build system is not compatible with ccache, so use
+# The go build system is not compatible with ccache, so use
 # HOSTCC_NOCCACHE.  See https://github.com/golang/go/issues/11685.
 HOST_GO_MAKE_ENV = \
 	GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_ROOT) \

@@ -75,31 +69,22 @@ HOST_GO_MAKE_ENV = \
 	$(if $(GO_GOARM),GOARM=$(GO_GOARM)) \
 	GOOS=linux \
 	CC=$(HOSTCC_NOCCACHE) \
-	CXX=$(HOSTCXX_NOCCACHE)
+	CXX=$(HOSTCXX_NOCCACHE) \
+	GO_ASSUME_CROSSCOMPILING=1
 
 HOST_GO_TARGET_CC = \
 	CC_FOR_TARGET="$(TARGET_CC)" \
 	CXX_FOR_TARGET="$(TARGET_CXX)"
 
-HOST_GO_HOST_CC = \
-	CC_FOR_TARGET=$(HOSTCC_NOCCACHE) \
-	CXX_FOR_TARGET=$(HOSTCXX_NOCCACHE)
-
-HOST_GO_TMP = $(@D)/host-go-tmp
 
 define HOST_GO_BUILD_CMDS
 	cd $(@D)/src && \
-		$(HOST_GO_MAKE_ENV) $(HOST_GO_HOST_CC) CGO_ENABLED=0 ./make.bash
-	mkdir -p $(HOST_GO_TMP)
-	mv $(@D)/pkg/tool $(HOST_GO_TMP)/
-	mv $(@D)/bin/ $(HOST_GO_TMP)/
-	cd $(@D)/src && \
 		$(HOST_GO_MAKE_ENV) $(HOST_GO_TARGET_CC) CGO_ENABLED=$(HOST_GO_CGO_ENABLED) ./make.bash
 endef
 
 define HOST_GO_INSTALL_CMDS
-	$(INSTALL) -D -m 0755 $(HOST_GO_TMP)/bin/go $(HOST_GO_ROOT)/bin/go
-	$(INSTALL) -D -m 0755 $(HOST_GO_TMP)/bin/gofmt $(HOST_GO_ROOT)/bin/gofmt
+	$(INSTALL) -D -m 0755 $(@D)/bin/go $(HOST_GO_ROOT)/bin/go
+	$(INSTALL) -D -m 0755 $(@D)/bin/gofmt $(HOST_GO_ROOT)/bin/gofmt
 
 	ln -sf ../lib/go/bin/go $(HOST_DIR)/bin/
 	ln -sf ../lib/go/bin/gofmt $(HOST_DIR)/bin/

@@ -108,7 +93,7 @@ define HOST_GO_INSTALL_CMDS
 
 	mkdir -p $(HOST_GO_ROOT)/pkg
 	cp -a $(@D)/pkg/include $(@D)/pkg/linux_* $(HOST_GO_ROOT)/pkg/
-	cp -a $(HOST_GO_TMP)/tool $(HOST_GO_ROOT)/pkg/
+	cp -a $(@D)/pkg/tool $(HOST_GO_ROOT)/pkg/
 
 	# There is a known issue which requires the go sources to be installed
 	# https://golang.org/issue/2775

@@ -116,7 +101,7 @@ define HOST_GO_INSTALL_CMDS
 
 	# Set all file timestamps to prevent the go compiler from rebuilding any
 	# built in packages when programs are built.
-	find $(HOST_GO_ROOT) -type f -exec touch -r $(HOST_GO_TMP)/bin/go {} \;
+	find $(HOST_GO_ROOT) -type f -exec touch -r $(@D)/bin/go {} \;
 endef
 
 $(eval $(host-generic-package))
Signature · 1 line
2.13.6
#2 lore raw

PATCH 1/3 go: patch to fix cross-compilation support

Hello,

Thanks a lot Anisse for those patches!

Quoted 7 lines · Anisse Astier

This patches the go toolchain with upstream-proposed patch by Angelo in order to fix cross-compilation when building a go toolchain for a target that has the same os and arch as host, but a different libc. This should fix the buildroot autobuild failures on x86_64 musl/uclibc targets.

This patch should go as soon as it's accepted or a similar upstream feature is merged.

The commit log of the patch that fixes the build issue should have a reference to the autobuilder failure being fixed, such as:

Fixes:

http://autobuild.buildroot.org/...

Quoted 2 lines

I'm adding myself to the DEVELOPERS file in order to get CC-ed to future build failures.

This should be done as part of a separate patch.

Quoted 22 lines
package/go/0001-cmd-dist-explicit-option-for-crosscompilation.patch new file +16 -0
new file mode 100644
index 0000000000..a779179609
@@ -0,0 +1,52 @@
+From acb08e41dad7f0e642149449564f7276bca3ada4 Mon Sep 17 00:00:00 2001
+From: Angelo Compagnucci <[email protected]>
+Date: Tue, 8 May 2018 16:08:44 +0200
+Subject: [PATCH] cmd/dist: explicit option for crosscompilation
+
+If GOHOSTOS == GOOS || GOHOSTARCH == GOARCH the go build system assume
+it's not cross compiling and uses the same compiler for both
+CC_FOR_TARGET and CC even if they are declared different. During go
+compilation, this produces the error:
+
+fork/exec
+/accts/mlweber1/rclinux/rc-buildroot-test/scripts/instance-2/output/build/host-go-1.10/bin/go:
+no such file or directory
+
+cause the go binary produced is linked against a different libc and
+cannot be run on the host machine.

package/go/0001-cmd-dist-explicit-option-for-crosscompilation.patch line 26

Quoted 10 lines
package/go/0001-cmd-dist-explicit-option-for-crosscompilation.patch new file +10 -0
+
+This is a problem in case the cross compilation mandates a different
+toolchain for host and target like happens in cross compilation
+environments like buildroot. This patch adds GO_ASSUME_CROSSCOMPILING
+varible to assure that in case of cross compilation CC_FOR_TARGET can be
+different from CC.
+
+Fixes #25177
+
+Change-Id: I4833c6d522407e29b039ca5660fd79df81e4b7ed

This should have your Signed-off-by as well.

Other than that, looks good to me. Could you respin a new version ?

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
author
#3 lore raw

PATCH 1/3 go: patch to fix cross-compilation support

Hi Thomas,

Thanks for taking the time to review this series.

Quoted 13 lines · Thomas Petazzoni

Hello,

Thanks a lot Anisse for those patches!

Quoted 7 lines · Anisse Astier

This patches the go toolchain with upstream-proposed patch by Angelo in order to fix cross-compilation when building a go toolchain for a target that has the same os and arch as host, but a different libc. This should fix the buildroot autobuild failures on x86_64 musl/uclibc targets.

This patch should go as soon as it's accepted or a similar upstream feature is merged.

Quoted 6 lines

The commit log of the patch that fixes the build issue should have a reference to the autobuilder failure being fixed, such as:

Fixes:

http://autobuild.buildroot.org/...

Will do.

Quoted 5 lines
Quoted 3 lines

I'm adding myself to the DEVELOPERS file in order to get CC-ed to future build failures.

This should be done as part of a separate patch.

That's what I thought, but the documentation stated otherwise:
"[...] please add yourself to the DEVELOPERS file. This should be done in the same patch creating or modifying the package. [...]"

https://buildroot.org/downloads/manual/manual.html#submitting-patches

Quoted 28 lines
Quoted 28 lines
package/go/0001-cmd-dist-explicit-option-for-crosscompilation.patch new file +20 -0
new file mode 100644
index 0000000000..a779179609
@@ -0,0 +1,52 @@
+From acb08e41dad7f0e642149449564f7276bca3ada4 Mon Sep 17 00:00:00 2001
+From: Angelo Compagnucci <[email protected]>
+Date: Tue, 8 May 2018 16:08:44 +0200
+Subject: [PATCH] cmd/dist: explicit option for crosscompilation
+
+If GOHOSTOS == GOOS || GOHOSTARCH == GOARCH the go build system assume
+it's not cross compiling and uses the same compiler for both
+CC_FOR_TARGET and CC even if they are declared different. During go
+compilation, this produces the error:
+
+fork/exec
+/accts/mlweber1/rclinux/rc-buildroot-test/scripts/instance-2/output/build/host-go-1.10/bin/go:
+no such file or directory
+
+cause the go binary produced is linked against a different libc and
+cannot be run on the host machine.
+
+This is a problem in case the cross compilation mandates a different
+toolchain for host and target like happens in cross compilation
+environments like buildroot. This patch adds GO_ASSUME_CROSSCOMPILING
Quoted 8 lines

package/go/0001-cmd-dist-explicit-option-for-crosscompilation.patch line 26

Quoted 6 lines
package/go/0001-cmd-dist-explicit-option-for-crosscompilation.patch new file +6 -0
+varible to assure that in case of cross compilation CC_FOR_TARGET can be
+different from CC.
+
+Fixes #25177
+
+Change-Id: I4833c6d522407e29b039ca5660fd79df81e4b7ed

This should have your Signed-off-by as well.

Will be in the next version.

Regards,

Anisse

#4 lore raw

PATCH 1/3 go: patch to fix cross-compilation support

Hello,

Quoted 10 lines · anisse at astier.eu
Quoted 4 lines
Quoted 2 lines

I'm adding myself to the DEVELOPERS file in order to get CC-ed to future build failures.

This should be done as part of a separate patch.

That's what I thought, but the documentation stated otherwise:
"[...] please add yourself to the DEVELOPERS file. This should be done in the same patch creating or modifying the package. [...]"

https://buildroot.org/downloads/manual/manual.html#submitting-patches

I think our documentation is wrong. It should be in the same patch when creating the package, but not when modifying the package.

Quoted 1 line

Will be in the next version.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
author
#5 lore raw

PATCH 2/3 go: honor building with V=1 for debugging purposes

In reply to Anisse Astier
Signed-off-by Anisse Astier <[email protected]>
---
 package/go/go.mk | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
package/go/go.mk +4 -1
index 37bb9801c2..bc5aca2bb7 100644
@@ -76,10 +76,13 @@ HOST_GO_TARGET_CC = \
 	CC_FOR_TARGET="$(TARGET_CC)" \
 	CXX_FOR_TARGET="$(TARGET_CXX)"
 
+ifneq ($(V),)
+VERBOSE=-v
+endif
 
 define HOST_GO_BUILD_CMDS
 	cd $(@D)/src && \
-		$(HOST_GO_MAKE_ENV) $(HOST_GO_TARGET_CC) CGO_ENABLED=$(HOST_GO_CGO_ENABLED) ./make.bash
+		$(HOST_GO_MAKE_ENV) $(HOST_GO_TARGET_CC) CGO_ENABLED=$(HOST_GO_CGO_ENABLED) ./make.bash $(VERBOSE)
 endef
 
 define HOST_GO_INSTALL_CMDS
Signature · 1 line
2.13.6
#6 lore raw

PATCH 2/3 go: honor building with V=1 for debugging purposes

Hello Anisse,

Quoted 10 lines · Anisse Astier
Signed-off-by Anisse Astier <[email protected]>
---
 package/go/go.mk | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
package/go/go.mk +0 -0
index 37bb9801c2..bc5aca2bb7 100644

package/go/go.mk HOST_GO_TARGET_CC = \ · line 80

Quoted 5 lines
package/go/go.mk +2 -0
@@ -76,10 +76,13 @@ HOST_GO_TARGET_CC = \
 	CC_FOR_TARGET="$(TARGET_CC)" \
 	CXX_FOR_TARGET="$(TARGET_CXX)"
 
+ifneq ($(V),)
+VERBOSE=-v

Please name the variable GO_VERBOSE and not just VERBOSE. Remember that Buildroot has a global namespace of make variables, so if you define VERBOSE here, you override the value of the VERBOSE variable defined in the main Makefile.

Also, please use VERBOSE instead of testing $(V).

Perhaps do just like cargo.mk is doing:

HOST_CARGO_SNAP_OPTS = \
        --release \
        $(if $(VERBOSE),--verbose)

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
author
#7 lore raw

PATCH 3/3 go: bump to version 1.10.2

In reply to Anisse Astier

This bump contains many bug fixes, as well as the following security issue, patched in Go 1.10.1:

CVE-2018-7187: The "go get" implementation in Go 1.9.4, when the -insecure command-line option is used, does not validate the import path (get/vcs.go only checks for "://" anywhere in the string), which allows remote attackers to execute arbitrary OS commands via a crafted web site.

Signed-off-by Anisse Astier <[email protected]>
---
 package/go/go.hash | 2 +-
 package/go/go.mk   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
package/go/go.hash +1 -1
index 73c1578d0b..9f5b80e9f5 100644
@@ -1,2 +1,2 @@
 # From https://golang.org/dl/
-sha256	f3de49289405fda5fd1483a8fe6bd2fa5469e005fd567df64485c4fa000c7f24	go1.10.src.tar.gz
+sha256	6264609c6b9cd8ed8e02ca84605d727ce1898d74efa79841660b2e3e985a98bd	go1.10.2.src.tar.gz
package/go/go.mk +1 -1
index bc5aca2bb7..8d899141f2 100644
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-GO_VERSION = 1.10
+GO_VERSION = 1.10.2
 GO_SITE = https://storage.googleapis.com/golang
 GO_SOURCE = go$(GO_VERSION).src.tar.gz
Signature · 1 line
2.13.6