Skip to contentSkip to conversation

package/pkg-golang: enable pkg specific go env also for download step

PATCH In review Patrick Gerber via buildroot 2 participants 4 messages Dec 4, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 package/pkg-golang: enable pkg specific go env also for download step

Currently package secific go env is used only during package build step.

Go vendering is done during the download step and it's sometimes required to specify package secific go env also for this step.

For example, when importing custom go modules who are hosted on a private host, it’s required to set GOPRIVATE to avoid public sum checking.

---
 package/pkg-golang.mk | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
package/pkg-golang.mk +2 -1
index 0b3dc3d32f..cdc67b32c0 100644
@@ -87,7 +87,8 @@ $(2)_POST_PATCH_HOOKS += $(2)_GEN_GOMOD
 $(2)_DOWNLOAD_POST_PROCESS = go
 $(2)_DL_ENV += \
 	$$(HOST_GO_COMMON_ENV) \
-	GOPROXY=direct
+	GOPROXY=direct \
+	$$($(2)_GO_ENV)
 
 # Due to vendoring, it is pretty likely that not all licenses are
 # listed in <pkg>_LICENSE.
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw

PATCH 2/2 package/mender-artifact: remove unnecessary GO_ENV

All go packages are built with:
GOFLAGS="-mod=vendor"
(see HOST_GO_COMMON_ENV in go/go.mk)

---
 package/mender-artifact/mender-artifact.mk | 8 --------
 1 file changed, 8 deletions(-)
package/mender-artifact/mender-artifact.mk +0 -8
index 1031166f34..6ade624a23 100644
@@ -86,14 +86,6 @@ HOST_MENDER_ARTIFACT_LICENSE_FILES = \
 
 HOST_MENDER_ARTIFACT_DEPENDENCIES = host-xz
 
-# By default, go will attempt to download needed modules before building, which
-# is not desirable. This behavior also causes permission issues when cleaning,
-# as go downloads modules as read-only by default. Because mender-artifact
-# includes the modules in the vendor directory, mod=vendor prevents the package
-# from downloading the go modules during the build process and prevents
-# permission issues when cleaning.
-HOST_MENDER_ARTIFACT_GO_ENV = GOFLAGS="-mod=vendor"
-
 HOST_MENDER_ARTIFACT_LDFLAGS = -X github.com/mendersoftware/mender-artifact/cli.Version=$(HOST_MENDER_ARTIFACT_VERSION)
 
 HOST_MENDER_ARTIFACT_BIN_NAME = mender-artifact
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#2 lore raw

Re: [Buildroot] [PATCH 2/2] package/mender-artifact: remove unnecessary GO_ENV

Patrick, All,

On 2022-12-04 15:06 +0100, Patrick Gerber via buildroot spake thusly:

Quoted 5 lines

All go packages are built with:
GOFLAGS="-mod=vendor"
(see HOST_GO_COMMON_ENV in go/go.mk)

I slightly extended the commit log to explain why mender-artifacts had that settings to begin with and applied to next, thanks.

Regards,
Yann E. MORIN.

Quoted 30 lines
---
 package/mender-artifact/mender-artifact.mk | 8 --------
 1 file changed, 8 deletions(-)
package/mender-artifact/mender-artifact.mk +0 -8
index 1031166f34..6ade624a23 100644
@@ -86,14 +86,6 @@ HOST_MENDER_ARTIFACT_LICENSE_FILES = \
 
 HOST_MENDER_ARTIFACT_DEPENDENCIES = host-xz
 
-# By default, go will attempt to download needed modules before building, which
-# is not desirable. This behavior also causes permission issues when cleaning,
-# as go downloads modules as read-only by default. Because mender-artifact
-# includes the modules in the vendor directory, mod=vendor prevents the package
-# from downloading the go modules during the build process and prevents
-# permission issues when cleaning.
-HOST_MENDER_ARTIFACT_GO_ENV = GOFLAGS="-mod=vendor"
-
 HOST_MENDER_ARTIFACT_LDFLAGS = -X github.com/mendersoftware/mender-artifact/cli.Version=$(HOST_MENDER_ARTIFACT_VERSION)
 
 HOST_MENDER_ARTIFACT_BIN_NAME = mender-artifact
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#3 lore raw
In reply to Patrick Gerber via buildroot

Patrick, All,

Thanks for this respin. :-)

On 2022-12-04 15:06 +0100, Patrick Gerber spake thusly:

Quoted 18 lines

Currently package secific go env is used only during package build step.

Go vendering is done during the download step and it's sometimes required to specify package secific go env also for this step.

For example, when importing custom go modules who are hosted on a private host, it’s required to set GOPRIVATE to avoid public sum checking.

---
 package/pkg-golang.mk | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
package/pkg-golang.mk +0 -0
index 0b3dc3d32f..cdc67b32c0 100644

package/pkg-golang.mk $(2)_POST_PATCH_HOOKS += $(2)_GEN_GOMOD · line 91

Quoted 6 lines
package/pkg-golang.mk +2 -1
@@ -87,7 +87,8 @@ $(2)_POST_PATCH_HOOKS += $(2)_GEN_GOMOD
 $(2)_DOWNLOAD_POST_PROCESS = go
 $(2)_DL_ENV += \
 	$$(HOST_GO_COMMON_ENV) \
-	GOPROXY=direct
+	GOPROXY=direct \
+	$$($(2)_GO_ENV)

As you can see, the pkg-golang infra extends $(2)_DL_ENV, so you can already achieve what you want by adding the following to your package:
FOO_DL_ENV = GOPRIVATE=blablabla

or even:
FOO_GO_ENV = GOPRIVATE=blablabla
FOO_DL_ENV = $(FOO_GO_ENV)

Also, what was probably not very clear in my previous review, is the concern whether we can have some package that set FOO_GO_ENV to something that would break the download step.

So I had a look at the official documentation for generic environment variables that apply to the go command [0]; I could not spot a generic variable that would obviously have an impact on downloads, unless also listed as those that just apply to the go-mod command [1].

It thus looks like the correct behaviour to use the build environment variables during hte download. Still, I was not sure this was material for the master branch, so I applied to next. Thanks!

[0] https://pkg.go.dev/cmd/go#hdr-Environment_variables
[1] https://go.dev/ref/mod#environment-variables

Regards,
Yann E. MORIN.

Quoted 5 lines
package/pkg-golang.mk +0 -0
 # Due to vendoring, it is pretty likely that not all licenses are
 # listed in <pkg>_LICENSE.
Signature · 1 line
2.25.1
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot