Skip to contentSkip to conversation

package/go: fixing crosscompilation settings

RFC Superseded Angelo Compagnucci 2 participants 6 messages Oct 5, 2017 – Oct 16, 2017

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

RFC 1/4 package/go: fixing crosscompilation settings

This patch fixes a bug with the BR2_TOOLCHAIN_HAS_THREADS variable handling which causes CGO_ENABLED to be always 0.

Furthermore, it fixes the cross compilation options for the go compiler:
setting CGO_ENABLED should be done only for the target compiler not the host one.

---
 package/go/go.mk | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)
package/go/go.mk +8 -6
index 0781dff..8e510e7 100644
@@ -52,7 +52,7 @@ HOST_GO_TARGET_ENV = \
 # set, build in cgo support for any go programs that may need it.  Note that
 # any target package needing cgo support must include
 # 'depends on BR2_TOOLCHAIN_HAS_THREADS' in its config file.
-ifeq (BR2_TOOLCHAIN_HAS_THREADS,y)
+ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)
 HOST_GO_CGO_ENABLED = 1
 else
 HOST_GO_CGO_ENABLED = 0
@@ -74,8 +74,8 @@ HOST_GO_MAKE_ENV = \
 	GOARCH=$(GO_GOARCH) \
 	$(if $(GO_GOARM),GOARM=$(GO_GOARM)) \
 	GOOS=linux \
-	CGO_ENABLED=$(HOST_GO_CGO_ENABLED) \
-	CC=$(HOSTCC_NOCCACHE)
+	CC=$(HOSTCC_NOCCACHE) \
+	CXX=$(HOSTCXX_NOCCACHE)
 
 HOST_GO_TARGET_CC = \
 	CC_FOR_TARGET="$(TARGET_CC)" \
@@ -83,16 +83,18 @@ HOST_GO_TARGET_CC = \
 
 HOST_GO_HOST_CC = \
 	CC_FOR_TARGET=$(HOSTCC_NOCCACHE) \
-	CXX_FOR_TARGET=$(HOSTCC_NOCCACHE)
+	CXX_FOR_TARGET=$(HOSTCXX_NOCCACHE)
 
 HOST_GO_TMP = $(@D)/host-go-tmp
 
 define HOST_GO_BUILD_CMDS
-	cd $(@D)/src && $(HOST_GO_MAKE_ENV) $(HOST_GO_HOST_CC) ./make.bash
+	cd $(@D)/src && $(HOST_GO_MAKE_ENV) $(HOST_GO_HOST_CC) \
+		CGO_ENABLED=0 ./make.bash
 	mkdir -p $(HOST_GO_TMP)
 	mv $(@D)/pkg/tool $(HOST_GO_TMP)/
 	mv $(@D)/bin/ $(HOST_GO_TMP)/
-	cd $(@D)/src && $(HOST_GO_MAKE_ENV) $(HOST_GO_TARGET_CC) ./make.bash
+	cd $(@D)/src && $(HOST_GO_MAKE_ENV) $(HOST_GO_TARGET_CC) \
+		CGO_ENABLED=$(HOST_GO_CGO_ENABLED) ./make.bash
 endef
 
 define HOST_GO_INSTALL_CMDS
Signature · 1 line
2.7.4
#1 lore raw

RFC 2/4 package/pkg-golang: new package infrastructure

This patch adds a new infrastructure for golang based packages.

---
 package/Makefile.in   |   1 +
 package/pkg-golang.mk | 121 ++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 122 insertions(+)
 create mode 100644 package/pkg-golang.mk
package/Makefile.in +1 -0
index a1a5316..60d98d0 100644
@@ -429,3 +429,4 @@ include package/pkg-kconfig.mk
 include package/pkg-rebar.mk
 include package/pkg-kernel-module.mk
 include package/pkg-waf.mk
+include package/pkg-golang.mk
package/pkg-golang.mk new file +121 -0

121 lines not on this page · raw message

Signature · 1 line
2.7.4
#2 lore raw

RFC 3/4 docs/manual: adding documentation for the golang infrastructure

In reply to Angelo Compagnucci

This patch adds the documentation for the golang infrastructure.

---
 docs/manual/adding-packages-golang.txt | 111 +++++++++++++++++++++++++++++++++
 docs/manual/adding-packages.txt        |   2 +
 2 files changed, 113 insertions(+)
 create mode 100644 docs/manual/adding-packages-golang.txt
docs/manual/adding-packages-golang.txt new file +111 -0
new file mode 100644
index 0000000..2ba45f0
@@ -0,0 +1,111 @@
+// -*- mode:doc; -*-
+// vim: set syntax=asciidoc:
+
+=== Infrastructure for Go packages
+
+This infrastructure applies to Go packages that use the standard
+build system and use bundled dependencies.
+
+[[golang-package-tutorial]]
+
+==== +golang-package+ tutorial
+
+First, let's see how to write a +.mk+ file for a go package,
+with an example :
+
+------------------------
+01: ################################################################################
+02: #
+03: # go-foo
+04: #
+05: ################################################################################
+06:
+07: GO_FOO_VERSION = 1.0
+08: GO_FOO_SOURCE = go-foo-$(GO_FOO_VERSION).tar.xz
+09: GO_FOO_SITE = http://www.foosoftware.org/download
+10: GO_FOO_LICENSE = BSD-3-Clause
+11: GO_FOO_LICENSE_FILES = LICENSE
+12:
+13: $(eval $(golang-package))
+------------------------
+
+On line 7, we declare the version of the package.
+
+On line 8 and 9, we declare the name of the tarball (xz-ed tarball
+recommended) and the location of the tarball on the Web. Buildroot
+will automatically download the tarball from this location.
+
+On line 10 and 11, we give licensing details about the package (its
+license on line 10, and the file containing the license text on line
+11).
+
+Finally, on line 13, we invoke the +golang-package+ macro that
+generates all the Makefile rules that actually allow the package to be
+built.
+
+[[golang-package-reference]]
+
+==== +golang-package+ reference
+
+As a policy packages can freely choose their name (existing example in 
+Buildroot is +flannel+).
+
+In their +Config.in+ file, they should depend on 
++BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS+ and
++BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS+ cause host-go will compile when Buildroot will add the
+dependency automatically.
+
+The main macro of the Go package infrastructure is
++golang-package+. It is similar to the +generic-package+ macro. It is
+also possible to create Go host packages with the
++host-golang-package+ macro.
+
+Just like the generic infrastructure, the Go infrastructure works
+by defining a number of variables before calling the +golang-package+
+or +host-golang-package+ macros.
+
+All the package metadata information variables that exist in the
+xref:generic-package-reference[generic package infrastructure] also
+exist in the Go infrastructure: +GO_FOO_VERSION+,
++GO_FOO_SOURCE+, +GO_FOO_PATCH+, +GO_FOO_SITE+,
++GO_FOO_SUBDIR+, +GO_FOO_DEPENDENCIES+, +GO_FOO_LICENSE+,
++GO_FOO_LICENSE_FILES+, +GO_FOO_INSTALL_STAGING+, etc.
+
+Note that:
+
+ * It is not necessary to add +go+ or +host-go+ in the
+   +GO_FOO_DEPENDENCIES+ variable of a package, since these basic
+   dependencies are automatically added as needed by the Go
+   package infrastructure.
+
+A few additional variables, specific to the Go infrastructure, can
+optionally be defined, depending on the package's needs. Many of them
+are only useful in very specific cases, typical packages will
+therefore only use a few of them, or none.
+
+* +GO_FOO_GO_SRC_PATH+: go sources should be compiled inside GOPATH.
+  The golang package infrastructure tries to guess the correct 
+  GOPATH subfolder to compile in. If guessing is not correct or your
+  package behaves differently, you can use this variable to
+  adjust the path.
+  
+
+* +GO_FOO_BUILD_OPTS+, to specify additional options to pass to the
+  Go +setup.py+ script during the build step. For target distutils
+  packages, the +PKG_GO_DISTUTILS_BUILD_OPTS+ options are already
+  passed automatically by the infrastructure.
+
+With the Go infrastructure, all the steps required to build and
+install the packages are already defined, and they generally work well
+for most Go-based packages. However, when required, it is still
+possible to customize what is done in any particular step:
+
+* By adding a post-operation hook (after extract, patch, configure,
+  build or install). See xref:hooks[] for details.
+
+* By overriding one of the steps. For example, even if the Go
+  infrastructure is used, if the package +.mk+ file defines its own
+  +GO_FOO_BUILD_CMDS+ variable, it will be used instead of the
+  default Go one. However, using this method should be restricted
+  to very specific cases. Do not use it in the general case.
+
docs/manual/adding-packages.txt +2 -0
index d577ff0..be7468b 100644
@@ -34,6 +34,8 @@ include::adding-packages-rebar.txt[]
 
 include::adding-packages-waf.txt[]
 
+include::adding-packages-golang.txt[]
+
 include::adding-packages-kernel-module.txt[]
 
 include::adding-packages-asciidoc.txt[]
Signature · 1 line
2.7.4
#3 lore raw

RFC 4/4 package/flannel: converting to golang infrastructure

In reply to Angelo Compagnucci

This patch converts the flannel package to the new golang infrastructure.

---
 package/flannel/flannel.mk | 25 ++++---------------------
 1 file changed, 4 insertions(+), 21 deletions(-)
package/flannel/flannel.mk +4 -21
index bbb2c72..641bb97 100644
@@ -11,14 +11,6 @@ FLANNEL_SOURCE = $(FLANNEL_VERSION).tar.gz
 FLANNEL_LICENSE = Apache-2.0
 FLANNEL_LICENSE_FILES = LICENSE
 
-FLANNEL_DEPENDENCIES = host-go
-
-FLANNEL_MAKE_ENV = \
-	$(HOST_GO_TARGET_ENV) \
-	GOBIN="$(@D)/bin" \
-	GOPATH="$(@D)/gopath" \
-	CGO_ENABLED=1
-
 FLANNEL_GLDFLAGS = \
 	-X github.com/coreos/flannel/version.Version=$(FLANNEL_VERSION)
 
@@ -26,21 +18,12 @@ ifeq ($(BR2_STATIC_LIBS),y)
 FLANNEL_GLDFLAGS += -extldflags '-static'
 endif
 
-define FLANNEL_CONFIGURE_CMDS
-	# Put sources at prescribed GOPATH location.
-	mkdir -p $(@D)/gopath/src/github.com/coreos
-	ln -s $(@D) $(@D)/gopath/src/github.com/coreos/flannel
-endef
-
-define FLANNEL_BUILD_CMDS
-	cd $(@D) && $(FLANNEL_MAKE_ENV) $(HOST_DIR)/bin/go \
-		build -v -o $(@D)/bin/flanneld -ldflags "$(FLANNEL_GLDFLAGS)" .
-endef
+FLANNEL_BUILD_OPTS = -ldflags "$(FLANNEL_GLDFLAGS)"
 
+# Install flannel to its well known location.
 define FLANNEL_INSTALL_TARGET_CMDS
-	# Install flannel to its well known location.
-	$(INSTALL) -D -m 0755 $(@D)/bin/flanneld $(TARGET_DIR)/opt/bin/flanneld
+	$(INSTALL) -D -m 0755 $(FLANNEL_EXECUTABLE) $(TARGET_DIR)/opt/bin/flanneld
 	$(INSTALL) -D -m 0755 $(@D)/dist/mk-docker-opts.sh $(TARGET_DIR)/opt/bin/mk-docker-opts.sh
 endef
 
-$(eval $(generic-package))
+$(eval $(golang-package))
Signature · 1 line
2.7.4
#4 lore raw

RFC 1/4 package/go: fixing crosscompilation settings

In reply to Angelo Compagnucci

Angelo, All,

On 2017-10-05 14:40 +0200, Angelo Compagnucci spake thusly:

Quoted 17 lines

This patch fixes a bug with the BR2_TOOLCHAIN_HAS_THREADS variable handling which causes CGO_ENABLED to be always 0.

Furthermore, it fixes the cross compilation options for the go compiler:
setting CGO_ENABLED should be done only for the target compiler not the host one.

---
 package/go/go.mk | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)
package/go/go.mk +0 -0
index 0781dff..8e510e7 100644

package/go/go.mk HOST_GO_TARGET_ENV = \ · line 55

Quoted 5 lines
package/go/go.mk +1 -1
@@ -52,7 +52,7 @@ HOST_GO_TARGET_ENV = \
 # set, build in cgo support for any go programs that may need it.  Note that
 # any target package needing cgo support must include
 # 'depends on BR2_TOOLCHAIN_HAS_THREADS' in its config file.
-ifeq (BR2_TOOLCHAIN_HAS_THREADS,y)
+ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)

Indeed... Wonderfull that it passed through the cracks and stayed unnoticed so far...

Quoted 17 lines
package/go/go.mk +2 -2
 HOST_GO_CGO_ENABLED = 1
 else
 HOST_GO_CGO_ENABLED = 0
@@ -74,8 +74,8 @@ HOST_GO_MAKE_ENV = \
 	GOARCH=$(GO_GOARCH) \
 	$(if $(GO_GOARM),GOARM=$(GO_GOARM)) \
 	GOOS=linux \
-	CGO_ENABLED=$(HOST_GO_CGO_ENABLED) \
-	CC=$(HOSTCC_NOCCACHE)
+	CC=$(HOSTCC_NOCCACHE) \
+	CXX=$(HOSTCXX_NOCCACHE)
 
 HOST_GO_TARGET_CC = \
 	CC_FOR_TARGET="$(TARGET_CC)" \
@@ -83,16 +83,18 @@ HOST_GO_TARGET_CC = \
 
 HOST_GO_HOST_CC = \

package/go/go.mk HOST_GO_TARGET_CC = \ · line 92

Quoted 10 lines
package/go/go.mk +3 -2
 	CC_FOR_TARGET=$(HOSTCC_NOCCACHE) \
-	CXX_FOR_TARGET=$(HOSTCC_NOCCACHE)
+	CXX_FOR_TARGET=$(HOSTCXX_NOCCACHE)
 
 HOST_GO_TMP = $(@D)/host-go-tmp
 
 define HOST_GO_BUILD_CMDS
-	cd $(@D)/src && $(HOST_GO_MAKE_ENV) $(HOST_GO_HOST_CC) ./make.bash
+	cd $(@D)/src && $(HOST_GO_MAKE_ENV) $(HOST_GO_HOST_CC) \
+		CGO_ENABLED=0 ./make.bash

Can we use a more logical location to split the line (I don't care if the second line is indented or not, but usually we do indent it):

    cd $(@D)/src && \
    $(HOST_GO_MAKE_ENV) $(HOST_GO_HOST_CC) CGO_ENABLED=0 ./make.bash

package/go/go.mk HOST_GO_TARGET_CC = \ · line 97

Quoted 6 lines
package/go/go.mk +2 -1
 	mkdir -p $(HOST_GO_TMP)
 	mv $(@D)/pkg/tool $(HOST_GO_TMP)/
 	mv $(@D)/bin/ $(HOST_GO_TMP)/
-	cd $(@D)/src && $(HOST_GO_MAKE_ENV) $(HOST_GO_TARGET_CC) ./make.bash
+	cd $(@D)/src && $(HOST_GO_MAKE_ENV) $(HOST_GO_TARGET_CC) \
+		CGO_ENABLED=$(HOST_GO_CGO_ENABLED) ./make.bash

Ditto?

Otherwise looks good:

Can go in, even without the rest of the series, I think, no?

Regards,
Yann E. MORIN.

Quoted 10 lines
package/go/go.mk +0 -0
 endef
 
 define HOST_GO_INSTALL_CMDS
Signature · 6 lines
2.7.4

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#5 lore raw

RFC 1/4 package/go: fixing crosscompilation settings

Dear Yann,

2017-10-16 18:56 GMT+02:00 Yann E. MORIN <[email protected]>:

Quoted 15 lines

Angelo, All,

On 2017-10-05 14:40 +0200, Angelo Compagnucci spake thusly:

Quoted 12 lines

This patch fixes a bug with the BR2_TOOLCHAIN_HAS_THREADS variable handling which causes CGO_ENABLED to be always 0.

Furthermore, it fixes the cross compilation options for the go compiler:
setting CGO_ENABLED should be done only for the target compiler not the host one.

---
 package/go/go.mk | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)
Quoted 13 lines

package/go/go.mk HOST_GO_TARGET_ENV = \ · line 55

Quoted 10 lines
package/go/go.mk +1 -1
index 0781dff..8e510e7 100644
@@ -52,7 +52,7 @@ HOST_GO_TARGET_ENV = \
 # set, build in cgo support for any go programs that may need it.  Note that
 # any target package needing cgo support must include
 # 'depends on BR2_TOOLCHAIN_HAS_THREADS' in its config file.
-ifeq (BR2_TOOLCHAIN_HAS_THREADS,y)
+ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)

Indeed... Wonderfull that it passed through the cracks and stayed unnoticed so far...

We haven't right now in buildroot a software that really needs cgo. I'm working with mender and noticed this bug straight.

Quoted 24 lines
Quoted 24 lines
package/go/go.mk +3 -3
 HOST_GO_CGO_ENABLED = 1
 else
 HOST_GO_CGO_ENABLED = 0
@@ -74,8 +74,8 @@ HOST_GO_MAKE_ENV = \
      GOARCH=$(GO_GOARCH) \
      $(if $(GO_GOARM),GOARM=$(GO_GOARM)) \
      GOOS=linux \
-     CGO_ENABLED=$(HOST_GO_CGO_ENABLED) \
-     CC=$(HOSTCC_NOCCACHE)
+     CC=$(HOSTCC_NOCCACHE) \
+     CXX=$(HOSTCXX_NOCCACHE)
 
 HOST_GO_TARGET_CC = \
      CC_FOR_TARGET="$(TARGET_CC)" \
@@ -83,16 +83,18 @@ HOST_GO_TARGET_CC = \
 
 HOST_GO_HOST_CC = \
      CC_FOR_TARGET=$(HOSTCC_NOCCACHE) \
-     CXX_FOR_TARGET=$(HOSTCC_NOCCACHE)
+     CXX_FOR_TARGET=$(HOSTCXX_NOCCACHE)
 
 HOST_GO_TMP = $(@D)/host-go-tmp
 
Quoted 7 lines

package/go/go.mk HOST_GO_TARGET_CC = \ · line 92

Quoted 4 lines
package/go/go.mk +2 -1
 define HOST_GO_BUILD_CMDS
-     cd $(@D)/src && $(HOST_GO_MAKE_ENV) $(HOST_GO_HOST_CC) ./make.bash
+     cd $(@D)/src && $(HOST_GO_MAKE_ENV) $(HOST_GO_HOST_CC) \
+             CGO_ENABLED=0 ./make.bash

Can we use a more logical location to split the line (I don't care if the second line is indented or not, but usually we do indent it):

Will do

Quoted 4 lines
    cd $(@D)/src && \
    $(HOST_GO_MAKE_ENV) $(HOST_GO_HOST_CC) CGO_ENABLED=0 ./make.bash
Quoted 8 lines

package/go/go.mk HOST_GO_TARGET_CC = \ · line 97

Quoted 6 lines
package/go/go.mk +2 -1
      mkdir -p $(HOST_GO_TMP)
      mv $(@D)/pkg/tool $(HOST_GO_TMP)/
      mv $(@D)/bin/ $(HOST_GO_TMP)/
-     cd $(@D)/src && $(HOST_GO_MAKE_ENV) $(HOST_GO_TARGET_CC) ./make.bash
+     cd $(@D)/src && $(HOST_GO_MAKE_ENV) $(HOST_GO_TARGET_CC) \
+             CGO_ENABLED=$(HOST_GO_CGO_ENABLED) ./make.bash

Ditto?

Will do.

Quoted 6 lines

Otherwise looks good:

Can go in, even without the rest of the series, I think, no?

Yes, no problem. For packages without cgo enabled don't change anything.

Quoted 22 lines

Regards,
Yann E. MORIN.

Quoted 10 lines
package/go/go.mk +0 -1
 endef
 
 define HOST_GO_INSTALL_CMDS
--

2.7.4

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

--
.-----------------.--------------------.------------------.--------------------.

|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |

'------------------------------^-------^------------------^--------------------'

Signature · 1 line
Profile: http://it.linkedin.com/in/compagnucciangelo