package/golang: new package
PATCH v6 1/1 package/golang: new package
From: Christian Stewart <[email protected]>
This patch adds golang, the Go compiler-and-interpreter.
We're adding a host package to get the golang cross-compiler, and a target package to get the golang interpreter on the target.
It is to be noted that golang's buildsystem is going at great length to not do anything like any other buildsystem, and is full of little tricks.
First-off, even though it generates code for the target, the host-variant does not need the toolchain; it has its own built-in code generator. Yet, building the target variant, the toolchain is indeed required. Weird...
Second, it misdetects cross-compilation: it considers that if the target CPU and the build CPU are the same, then this is not cross-compilation. Of course, this is completely stupid and wrong. So, we need to patch it to override its insane heuristic.
Third, even so, when the target CPU is the same as the build CPU, the resulting binaries will be dynamically linked, otherwise, they are statically linked. We can't even force it one way or the other... Instead, we mark it as requiring dynamic libraries, and let the golang ill buildsystem decide whether to do static or dynamic link; we can do so because any sane toolchain will always have the static variants of the libc and libpthread, which is all that golang requires anyway.
[yann.morin.1998 at free.fr:
- add architecture-specific options
- needs dynamic linking
- fix licensing info
- fix tab-space indentation in patch 0003
- fix offset in patch 0001
- misc cleanups
]
---
Changes v4 -> v6;
- origin of hash (Thomas)
- better indent commands (Thomas)
- don't "deinstall" prior to installing (Thomas)
- fix offset in patch 0001
Changes v3 -> v4;
- adopted by Yann, worked on in concert with Christian
- lots of refactoring and cleanups
--- package/Config.in | 1 + package/golang/0001-add-no-march-option-gccp.patch | 21 ++++++ .../0002-remove-unnecessary-march-ld-flag.patch | 29 ++++++++ package/golang/0003-fix-make-bash-script.patch | 41 +++++++++++ package/golang/Config.in | 22 ++++++ package/golang/golang.hash | 2 + package/golang/golang.mk | 84 ++++++++++++++++++++++ 7 files changed, 200 insertions(+) create mode 100644 package/golang/0001-add-no-march-option-gccp.patch create mode 100644 package/golang/0002-remove-unnecessary-march-ld-flag.patch create mode 100644 package/golang/0003-fix-make-bash-script.patch create mode 100644 package/golang/Config.in create mode 100644 package/golang/golang.hash create mode 100644 package/golang/golang.mk
package/Config.in +1 -0
@@ -460,6 +460,7 @@ menu "Erlang libraries/modules" source "package/erlang-p1-zlib/Config.in" endmenu endif + source "package/golang/Config.in" source "package/guile/Config.in" source "package/haserl/Config.in" source "package/jamvm/Config.in"
package/golang/0001-add-no-march-option-gccp.patch new file +21 -0
@@ -0,0 +1,21 @@ +Adds an option to not add the problematic -m parameter to GCC. + +Signed-off-by: Christian Stewart <[email protected]> +[yann.morin.1998 at free.fr: fix offset] +Signed-off-by: "Yann E. MORIN" <[email protected]> + +diff -Nau golang.orig/src/cmd/cgo/gcc.go golang/src/cmd/cgo/gcc.go +--- golang.orig/src/cmd/cgo/gcc.go.orig 2015-07-09 15:53:55.720794139 -0700 ++++ golang/src/cmd/cgo/gcc.go 2015-07-09 17:46:43.664496374 -0700 +@@ -739,6 +739,11 @@ + + // gccMachine returns the gcc -m flag to use, either "-m32", "-m64" or "-marm". + func (p *Package) gccMachine() []string { ++ ++ if os.Getenv("CGO_NO_EMULATION") == "1" { ++ return nil ++ } ++ + switch goarch { + case "amd64": + return []string{"-m64"}
package/golang/0002-remove-unnecessary-march-ld-flag.patch new file +29 -0
@@ -0,0 +1,29 @@ +Removes defining -m parameter to LD, which is unnecessary in buildroot. + +Signed-off-by: Christian Stewart <[email protected]> + +diff -Nau golang.orig/src/cmd/ld/lib.c golang/src/cmd/ld/lib.c +--- golang.orig/src/cmd/ld/lib.c.orig 2015-07-09 18:38:44.192359082 -0700 ++++ golang/src/cmd/ld/lib.c 2015-07-09 18:39:02.108358294 -0700 +@@ -589,20 +589,7 @@ + if(extld == nil) + extld = "gcc"; + argv[argc++] = extld; +- switch(thechar){ +- case '8': +- argv[argc++] = "-m32"; +- break; +- case '6': +- argv[argc++] = "-m64"; +- break; +- case '5': +- argv[argc++] = "-marm"; +- break; +- } +- if(!debug['s'] && !debug_s) { +- argv[argc++] = "-gdwarf-2"; +- } else { ++ if(debug['s'] || debug_s) { + argv[argc++] = "-s"; + } + if(HEADTYPE == Hdarwin)
package/golang/0003-fix-make-bash-script.patch new file +41 -0
41 lines not on this page · raw message
package/golang/Config.in new file +22 -0
@@ -0,0 +1,22 @@ +comment "golang needs a toolchain w/ threads, dynamic library" + depends on BR2_PACKAGE_GOLANG_ARCH_SUPPORTS + depends on !BR2_TOOLCHAIN_HAS_THREADS || BR2_STATIC_LIBS + +# golang only supports x86, x86_64 and ARM (LE) targets. +# For ARM, armv5 or above is required. +config BR2_PACKAGE_GOLANG_ARCH_SUPPORTS + bool + default y + depends on BR2_USE_MMU + depends on BR2_i386 || BR2_x86_64 || BR2_arm + depends on !BR2_ARM_CPU_ARMV4 + +config BR2_PACKAGE_GOLANG + bool "golang" + depends on BR2_PACKAGE_GOLANG_ARCH_SUPPORTS + depends on !BR2_STATIC_LIBS + depends on BR2_TOOLCHAIN_HAS_THREADS + help + Go compiler and cli tool. + + http://golang.org/
package/golang/golang.hash new file +2 -0
@@ -0,0 +1,2 @@ +# Hash from: https://golang.org/dl/ +sha1 460caac03379f746c473814a65223397e9c9a2f6 go1.4.2.src.tar.gz
package/golang/golang.mk new file +84 -0
84 lines not on this page · raw message
Signature · 1 line
1.9.1
PATCH v6 1/1 package/golang: new package
Hi Yann,
Quoted 65 lines
From: Christian Stewart <[email protected]>
This patch adds golang, the Go compiler-and-interpreter.
We're adding a host package to get the golang cross-compiler, and a target package to get the golang interpreter on the target.
It is to be noted that golang's buildsystem is going at great length to not do anything like any other buildsystem, and is full of little tricks.
First-off, even though it generates code for the target, the host-variant does not need the toolchain; it has its own built-in code generator. Yet, building the target variant, the toolchain is indeed required. Weird...
Second, it misdetects cross-compilation: it considers that if the target CPU and the build CPU are the same, then this is not cross-compilation. Of course, this is completely stupid and wrong. So, we need to patch it to override its insane heuristic.
Third, even so, when the target CPU is the same as the build CPU, the resulting binaries will be dynamically linked, otherwise, they are statically linked. We can't even force it one way or the other... Instead, we mark it as requiring dynamic libraries, and let the golang ill buildsystem decide whether to do static or dynamic link; we can do so because any sane toolchain will always have the static variants of the libc and libpthread, which is all that golang requires anyway.
Signed-off-by Christian Stewart <[email protected]>[yann.morin.1998 at free.fr:
- add architecture-specific options
- needs dynamic linking
- fix licensing info
- fix tab-space indentation in patch 0003
- fix offset in patch 0001
- misc cleanups
]Signed-off-by Yann E. MORIN <[email protected]>---
Changes v4 -> v6;
- origin of hash (Thomas)
- better indent commands (Thomas)
- don't "deinstall" prior to installing (Thomas)
- fix offset in patch 0001Changes v3 -> v4;
- adopted by Yann, worked on in concert with Christian
- lots of refactoring and cleanups--- package/Config.in | 1 + package/golang/0001-add-no-march-option-gccp.patch | 21 ++++++ .../0002-remove-unnecessary-march-ld-flag.patch | 29 ++++++++ package/golang/0003-fix-make-bash-script.patch | 41 +++++++++++ package/golang/Config.in | 22 ++++++ package/golang/golang.hash | 2 + package/golang/golang.mk | 84 ++++++++++++++++++++++ 7 files changed, 200 insertions(+)create mode 100644
package/golang/0001-add-no-march-option-gccp.patchcreate mode 100644package/golang/0002-remove-unnecessary-march-ld-flag.patchcreate mode 100644package/golang/0003-fix-make-bash-script.patchcreate mode 100644 package/golang/Config.in
create mode 100644 package/golang/golang.hash
create mode 100644package/golang/golang.mk
[...]
package/golang/golang.mk line 3
Quoted 9 lines
package/golang/golang.mk new file +3
-0@@ -0,0 +1,84 @@ +################################################################################ +# +# GOLANG
This should be in lowercase.
Regards,
Jerzy
Quoted 81 lines
package/golang/golang.mk new file +81
-0+# +################################################################################ + +GOLANG_VERSION = 1.4.2 +GOLANG_SOURCE = go$(GOLANG_VERSION).src.tar.gz +GOLANG_SITE = https://storage.googleapis.com/golang +GOLANG_LICENSE = BSD-3c +GOLANG_LICENSE_FILES = LICENSE + +GOLANG = $(HOST_DIR)/usr/bin/go +GOLANGFMT = $(HOST_DIR)/usr/bin/gofmt + +GOLANG_ENV = \ + CC_FOR_TARGET="$(TARGET_CC)" \ + LD_FOR_TARGET="$(TARGET_LD)" \ + GOOS=linux + +ifeq ($(BR2_i386),y) +GOLANG_ARCH = 386 +ifeq ($(BR2_X86_CPU_HAS_SSE2),y) +GOLANG_ENV += GO386=sse2 +else +GOLANG_ENV += GO386=387 +endif +endif # i386 + +ifeq ($(BR2_x86_64),y) +GOLANG_ARCH = amd64 +endif # x86_64 + +# For ARM, the selection of the instruciton set is a bit unusual, +# and is decided on whether the CPU has an FPU, and which one. +ifeq ($(BR2_arm),y) +GOLANG_ARCH = arm +ifeq ($(BR2_ARM_CPU_HAS_VFPV3),y) +GOLANG_ENV += GOARM=7 +else ifeq ($(BR2_ARM_CPU_HAS_VFPV2),y) +GOLANG_ENV += GOARM=6 +else +GOLANG_ENV += GOARM=5 +endif +endif # arm + +GOLANG_ENV += GOARCH=$(GOLANG_ARCH) + +define GOLANG_BUILD_CMDS + cd $(@D)/src/; \ + $(GOLANG_ENV) GOROOT_FINAL="/usr/lib/go" GO_NO_HOST=1 \ + ./make.bash --no-banner +endef + +# We must install both the src/ and include/ subdirs because they +# contain the go "runtime". +define GOLANG_INSTALL_TARGET_CMDS + $(INSTALL) -D -m 0755 $(@D)/bin/linux_$(GOLANG_ARCH)/go $(TARGET_DIR)/usr/bin/go + $(INSTALL) -D -m 0755 $(@D)/bin/linux_$(GOLANG_ARCH)/gofmt $(TARGET_DIR)/usr/bin/gofmt + mkdir -p $(TARGET_DIR)/usr/lib/go/ + cp -a $(@D)/src $(TARGET_DIR)/usr/lib/go/ + cp -a $(@D)/include $(TARGET_DIR)/usr/lib/go/ + cp -a $(@D)/pkg $(TARGET_DIR)/usr/lib/go/ +endef + +define HOST_GOLANG_BUILD_CMDS + cd $(@D)/src/ ; \ + $(GOLANG_ENV) GOROOT_FINAL="$(HOST_DIR)/usr/lib/go" GO_NO_TARGET=1 \ + ./make.bash --no-banner +endef + +# We must install both the src/ and include/ subdirs because they +# contain the go "runtime" +define HOST_GOLANG_INSTALL_CMDS + $(INSTALL) -D -m 0755 $(@D)/bin/host/go $(HOST_DIR)/usr/bin/go + $(INSTALL) -D -m 0755 $(@D)/bin/host/gofmt $(HOST_DIR)/usr/bin/gofmt + mkdir -p $(HOST_DIR)/usr/lib/go/ + cp -a $(@D)/src $(HOST_DIR)/usr/lib/go/ + cp -a $(@D)/include $(HOST_DIR)/usr/lib/go/ + cp -a $(@D)/pkg $(HOST_DIR)/usr/lib/go/ +endef + +$(eval $(generic-package)) +$(eval $(host-generic-package))
PATCH v6 1/1 package/golang: new package
In reply to Yann E. MORINYann, Christian,
Quoted 2 lines · Yann E. MORIN
We're adding a host package to get the golang cross-compiler, and a target package to get the golang interpreter on the target.
As discussed on IRC, I tested the host package, and it is not a cross-compiler, but a regular compiler, which makes it pretty useless:
$ ./output/host/usr/bin/go build toto.go $ file toto toto: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped
Quoted 16 lines
package/golang/Config.in new file +10
-0@@ -0,0 +1,22 @@ +comment "golang needs a toolchain w/ threads, dynamic library" + depends on BR2_PACKAGE_GOLANG_ARCH_SUPPORTS + depends on !BR2_TOOLCHAIN_HAS_THREADS || BR2_STATIC_LIBS + +# golang only supports x86, x86_64 and ARM (LE) targets. +# For ARM, armv5 or above is required. +config BR2_PACKAGE_GOLANG_ARCH_SUPPORTS + bool + default y + depends on BR2_USE_MMU
package/golang/Config.in line 20
Quoted 10 lines
package/golang/Config.in new file +10
-0+ depends on BR2_i386 || BR2_x86_64 || BR2_arm + depends on !BR2_ARM_CPU_ARMV4 + +config BR2_PACKAGE_GOLANG + bool "golang" + depends on BR2_PACKAGE_GOLANG_ARCH_SUPPORTS + depends on !BR2_STATIC_LIBS + depends on BR2_TOOLCHAIN_HAS_THREADS + help + Go compiler and cli tool.
I think this is fairly confusing. We should instead say:
Go interpreter.
and indicate that if people want to run Go applications that have been built on their host machine, then they don't need the golang target package.
package/golang/golang.mk line 3
Quoted 9 lines
package/golang/golang.mk new file +3
-0@@ -0,0 +1,84 @@ +################################################################################ +# +# GOLANG
should be lower case, as already noted by Jerzy.
package/golang/golang.mk line 59
Quoted 5 lines
package/golang/golang.mk new file +5
-0+# We must install both the src/ and include/ subdirs because they +# contain the go "runtime". +define GOLANG_INSTALL_TARGET_CMDS + $(INSTALL) -D -m 0755 $(@D)/bin/linux_$(GOLANG_ARCH)/go $(TARGET_DIR)/usr/bin/go + $(INSTALL) -D -m 0755 $(@D)/bin/linux_$(GOLANG_ARCH)/gofmt $(TARGET_DIR)/usr/bin/gofmt
Why do we install gofmt on the target? Buildroot doesn't install tools to do development on the target, so installing gofmt seems weird.
package/golang/golang.mk line 63
Quoted 4 lines
package/golang/golang.mk new file +4
-0+ mkdir -p $(TARGET_DIR)/usr/lib/go/ + cp -a $(@D)/src $(TARGET_DIR)/usr/lib/go/ + cp -a $(@D)/include $(TARGET_DIR)/usr/lib/go/ + cp -a $(@D)/pkg $(TARGET_DIR)/usr/lib/go/
If I understand correctly, all of this "runtime" is actually only needed if you use "go" as an interpreter on the target. If you use "go" on the host as a cross-compiler to build your application, you don't need any of this "runtime". This stuff is actually quite huge (around 70 MB!).
Note that we more commonly use "cp -dprf" in Buildroot rather than "cp -a", but I guess it's not a super-strict rule.
I've marked the patch as Changes Requested in patchwork.
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
A message that is not in the archive (sent off-list or not yet received): CA+h8R2p5xgF7rJJW5z0AzB7b359F=S9yWrXpiTmCVK-D0Sk8pw@mail.gmail.com
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH v6 1/1 package/golang: new package
Thomas,
... to everyone this time...
Quoted 3 lines
Maybe the host-golang compiler needs to be passed some magic arguments or environment variables to build?
Yes, this is the case. You have to specify GOARCH and similar. It's done in the docker package pull request, see
These things are already not copied if only host-golang is built.
Quoted 1 line
Then I don't see the issue. If you're wanting to use Go as an interpreter, Go needs to be able to access all of the source code required to compile the code and the dependencies. So, the runtime is required to be copied.
Thanks!
Christian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20151021/3f990a66/attachment.html>
PATCH v6 1/1 package/golang: new package
In reply to a message not in the archive---------- Forwarded message ---------
From: Christian Stewart <[email protected]>
Date: Wed, Oct 21, 2015 at 5:16 PM
Subject: Re: [Buildroot] [PATCHv6] package/golang: new package
To: Thomas Petazzoni <[email protected]>
Thomas,
Quoted 3 lines · Thomas Petazzoni
As discussed on IRC, I tested the host package, and it is not a cross-compiler, but a regular compiler, which makes it pretty useless:
Odd. Yann, can you look into this? I think it works to cross-compile Docker.
I think this is fairly confusing. We should instead say:
Quoted 7 lines
Go interpreter.
and indicate that if people want to run Go applications that have been built on their host machine, then they don't need the golang target package.
Sounds Ok to me.
Quoted 2 lines
should be lower case, as already noted by Jerzy.
OK.
Quoted 11 lines
package/golang/golang.mk
Quoted 4 lines
package/golang/golang.mk new file +4
-0+# We must install both the src/ and include/ subdirs because they +# contain the go "runtime". +define GOLANG_INSTALL_TARGET_CMDS + $(INSTALL) -D -m 0755 $(@D)/bin/linux_$(GOLANG_ARCH)/go$(TARGET_DIR)/usr/bin/go
Quoted 1 line
+ $(INSTALL) -D -m 0755 $(@D)/bin/linux_$(GOLANG_ARCH)/gofmt
$(TARGET_DIR)/usr/bin/gofmt
Why do we install gofmt on the target? Buildroot doesn't install tools to do development on the target, so installing gofmt seems weird.
I believe as an "interpreter" go uses gofmt a lot.
Quoted 6 lines
package/golang/golang.mk line 63
Quoted 5 lines
package/golang/golang.mk new file +4
-0+ mkdir -p $(TARGET_DIR)/usr/lib/go/ + cp -a $(@D)/src $(TARGET_DIR)/usr/lib/go/ + cp -a $(@D)/include $(TARGET_DIR)/usr/lib/go/ + cp -a $(@D)/pkg $(TARGET_DIR)/usr/lib/go/
Quoted 6 lines
If I understand correctly, all of this "runtime" is actually only needed if you use "go" as an interpreter on the target. If you use "go" on the host as a cross-compiler to build your application, you don't need any of this "runtime". This stuff is actually quite huge (around 70 MB!).
Yes you don't need this as long as you've pre-compiled everything on the target and don't want to use it as an interpreter. it shouldn't be copied if the host is selected only.
Quoted 2 lines
Note that we more commonly use "cp -dprf" in Buildroot rather than "cp -a", but I guess it's not a super-strict rule.
~ Christian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20151021/4b7a00f5/attachment.html>
PATCH v6 1/1 package/golang: new package
Thomas, Yann,
Is there anything that needs to be done regarding golang for it to be merged? After golang we can push towards docker support.
Best,
Christian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20151104/8cac96e9/attachment.html>
PATCH v6 1/1 package/golang: new package
Christian, All,
On 2015-11-04 15:39 +0000, Christian Stewart spake thusly:
Quoted 2 lines
Is there anything that needs to be done regarding golang for it to be merged? After golang we can push towards docker support.
IIRC, we need to provide an $(TUPLE)-go to act as a cross-compiler. That would be a shell script that exports the appropriate environement variables (you provided the list) and then calls the real go executable.
Otherwise, just calling the go program, even when set up as a cross-compiler, does not produce target binaries as we've observed.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH v6 1/1 package/golang: new package
Hello,
Quoted 10 lines · Yann E. MORIN
On 2015-11-04 15:39 +0000, Christian Stewart spake thusly:
Quoted 2 lines
Is there anything that needs to be done regarding golang for it to be merged? After golang we can push towards docker support.
IIRC, we need to provide an $(TUPLE)-go to act as a cross-compiler. That would be a shell script that exports the appropriate environement variables (you provided the list) and then calls the real go executable.
Otherwise, just calling the go program, even when set up as a cross-compiler, does not produce target binaries as we've observed.
Correct. Also, I think I raised a few comments (about gofmt, about the situations where go in the target is useful). Those comments were answered, but it would be nicer if the answers were turned into actual comments into the code, or in the Config.in help text.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
PATCH v6 1/1 package/golang: new package
Yann,
Would you like to address these or should I? If I add the command with env variables variable as well as the docs, should I respin it or should I send it to you? What's the best way to do this?
Correct. Also, I think I raised a few comments (about gofmt, about the
Quoted 4 lines
situations where go in the target is useful). Those comments were answered, but it would be nicer if the answers were turned into actual comments into the code, or in the Config.in help text.
Thanks!
Christian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20151104/bef5c71b/attachment.html>
PATCH v6 1/1 package/golang: new package
Christian, All,
On 2015-11-04 21:21 +0000, Christian Stewart spake thusly:
Quoted 3 lines
Would you like to address these or should I? If I add the command with env variables variable as well as the docs, should I respin it or should I send it to you? What's the best way to do this?
Please do the change and respin a new patch to the list.
Thanks! :-)
Regards,
Yann E. MORIN.
Quoted 8 lines
Quoted 5 lines
Correct. Also, I think I raised a few comments (about gofmt, about the situations where go in the target is useful). Those comments were answered, but it would be nicer if the answers were turned into actual comments into the code, or in the Config.in help text.
Thanks!
Christian
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'