Skip to contentSkip to conversation

go-mod.bbclass: Set GO_MOD_CACHE_DIR

PATCH Superseded Christian Lindeberg 5 participants 6 messages Sep 12, 2024 – Sep 13, 2024

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 go-mod.bbclass: Set GO_MOD_CACHE_DIR

From: Christian Lindeberg <[email protected]>

Set the GO_MOD_CACHE_DIR variable and move the location of the module cache to enable the use of the go module fetchers for module dependencies.
Also, clean out the module cache before unpacking.

---
 meta/classes-recipe/go-mod.bbclass | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
meta/classes-recipe/go-mod.bbclass +4 -4
index ca3a690d05..71a6712c47 100644
@@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw"
 
 inherit go
 
+GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), 'pkg/mod')}"
+export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), d.getVar('GO_MOD_CACHE_DIR'))}"
+do_unpack[cleandirs] += "${GOMODCACHE}"
+
 GO_WORKDIR ?= "${GO_IMPORT}"
 do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
-
-export GOMODCACHE = "${B}/.mod"
-
-do_compile[cleandirs] += "${B}/.mod"
Signature · 1 line
2.39.2
#1 lore raw

Hi Christian,

Christian Lindeberg via lists.openembedded.org <christian.lindeberg= [email protected]> escreveu (quinta, 12/09/2024 à(s) 15:30):

Quoted 22 lines

From: Christian Lindeberg <[email protected]>

Set the GO_MOD_CACHE_DIR variable and move the location of the module cache to enable the use of the go module fetchers for module dependencies.
Also, clean out the module cache before unpacking.

---
 meta/classes-recipe/go-mod.bbclass | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
meta/classes-recipe/go-mod.bbclass +1 -0
index ca3a690d05..71a6712c47 100644
@@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw"
 
 inherit go
 
+GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')),
Quoted 4 lines

'pkg/mod')}"
+export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), d.getVar('GO_MOD_CACHE_DIR'))}"

Using the default [1] bitbake S = "${WORKDIR}/${BP}" config, I think this will be equal to:

GO_MOD_CACHE_DIR ?= "${BP}/pkg/mod"
export GOMODCACHE = "${WORKDIR}/${BP}/pkg/mod"

in the end this can be written as:

export GOMODCACHE ?= "${S}/pkg/mod"

Is it correct or am I seeing something wrong?

[1]
https://github.com/openembedded/openembedded-core/blob/40d2fbece1c2f6ecf62bffa44ad37850e90268cb/meta/conf/bitbake.conf#L405C6-L405C16

Jose

+do_unpack[cleandirs] += "${GOMODCACHE}"

Quoted 22 lines
meta/classes-recipe/go-mod.bbclass +1 -5
+
 GO_WORKDIR ?= "${GO_IMPORT}"
 do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
-
-export GOMODCACHE = "${B}/.mod"
-
-do_compile[cleandirs] += "${B}/.mod"
--

2.39.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#204444):
https://lists.openembedded.org/g/openembedded-core/message/204444
Mute This Topic: https://lists.openembedded.org/mt/108413937/5052612
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 3 lines
Best regards,

José Quaresma
#2 lore raw

[I can’t reply inline due to the HTML-formatting.]

I believe the goal is to have pkg/mod as a subdirectory of ${S}. At the same time, ${GO_MOD_CACHE_DIR} needs to be a relative path (as it is passed to the fetcher via the subdir= parameter). However, the suggested code only works for cases where ${S} matches ${WORKDIR}/subdirectory. In case it is ${WORKDIR}/sub/directory, it will fail. I would suggest to change the code like this:

export GOMODCACHE = "${S}/pkg/mod"
GO_MOD_CACHE_DIR = "${@os.path.relpath(d.getVar('GOMODCACHE'), d.getVar('WORKDIR'))}" do_unpack[cleandirs] += "${GOMODCACHE}"

I have verified that the above fetches a Go module into the correct directory with S set to ${WORKDIR}/git/${BPN}. I have also verified that crucible (one of the recipes in meta-oe that uses the go-mod bbclass) still builds with the above. And as a bonus I now also have a patch that modifies the crucible recipe to use the gomod fetcher (which requires the above change and also the patch I just sent to the bitbake list for the gomod fetcher). Unfortunately I will not have time to finish it until I’m back from OSS in Vienna next week.

//Peter

From: [email protected] <[email protected]> On Behalf Of Jose Quaresma
Sent: den 12 september 2024 19:01
To: Christian Lindeberg <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH] go-mod.bbclass: Set GO_MOD_CACHE_DIR

Hi Christian,

Christian Lindeberg via lists.openembedded.org<http://lists.openembedded.org> <[email protected]<mailto:[email protected]>> escreveu (quinta, 12/09/2024 à(s) 15:30):
From: Christian Lindeberg <[email protected]<mailto:[email protected]>>

Set the GO_MOD_CACHE_DIR variable and move the location of the module cache to enable the use of the go module fetchers for module dependencies.
Also, clean out the module cache before unpacking.

---
 meta/classes-recipe/go-mod.bbclass | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
meta/classes-recipe/go-mod.bbclass +2 -0
index ca3a690d05..71a6712c47 100644
@@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw"
 
 inherit go
 
+GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), 'pkg/mod')}<mailto:$%[email protected](os.path.basename(d.getVar('S')),%20'pkg/mod')%7d>"
+export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), d.getVar('GO_MOD_CACHE_DIR'))}<mailto:$%[email protected](os.path.dirname(d.getVar('S')),%20d.getVar('GO_MOD_CACHE_DIR'))%7d>"
 

Using the default [1] bitbake S = "${WORKDIR}/${BP}" config, I think this will be equal to:

GO_MOD_CACHE_DIR ?= "${BP}/pkg/mod"
export GOMODCACHE = "${WORKDIR}/${BP}/pkg/mod"

in the end this can be written as:

export GOMODCACHE ?= "${S}/pkg/mod"

Is it correct or am I seeing something wrong?
[1] https://github.com/openembedded/openembedded-core/blob/40d2fbece1c2f6ecf62bffa44ad37850e90268cb/meta/conf/bitbake.conf#L405C6-L405C16

Jose

+do_unpack[cleandirs] += "${GOMODCACHE}"
+
GO_WORKDIR ?= "${GO_IMPORT}"
do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
-
-export GOMODCACHE = "${B}/.mod"
-
-do_compile[cleandirs] += "${B}/.mod"
--
2.39.2

--
Best regards,

José Quaresma

#3 lore raw
In reply to Jose Quaresma
Quoted 37 lines · Jose Quaresma

Hi Christian,

Christian Lindeberg via lists.openembedded.org (
http://lists.openembedded.org ) <christian.lindeberg= [email protected]

Quoted 23 lines

escreveu (quinta, 12/09/2024 à(s) 15:30):

From: Christian Lindeberg < [email protected] >

Set the GO_MOD_CACHE_DIR variable and move the location of the module cache to enable the use of the go module fetchers for module dependencies.
Also, clean out the module cache before unpacking.

---
meta/classes-recipe/go-mod.bbclass | 8 ++++----

1 file changed, 4 insertions(+), 4 deletions(-)
meta/classes-recipe/go-mod.bbclass +0 -0
index ca3a690d05..71a6712c47 100644
@@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw"
 

inherit go

Quoted 4 lines

+GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), 'pkg/mod')}"
+export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), d.getVar('GO_MOD_CACHE_DIR'))}"

Using the default [1] bitbake S = "${WORKDIR}/${BP}" config, I think this will be equal to:

Quoted 9 lines

GO_MOD_CACHE_DIR ?= "${BP}/pkg/mod"
export GOMODCACHE = "${WORKDIR}/${BP}/pkg/mod"

in the end this can be written as:

export GOMODCACHE ?= "${S}/pkg/mod"

Is it correct or am I seeing something wrong?

Correct, I sent a new version with this and the suggestion from Peter Kjellerstedt.

Thanks,
Christian

Quoted 31 lines

[1] https://github.com/openembedded/openembedded-core/blob/40d2fbece1c2f6ecf62bffa44ad37850e90268cb/meta/conf/bitbake.conf#L405C6-L405C16

Jose

Quoted 9 lines
meta/classes-recipe/go-mod.bbclass +2 -0
+do_unpack[cleandirs] += "${GOMODCACHE}"
+

GO_WORKDIR ?= "${GO_IMPORT}"
do_compile[dirs] += "${B}/src/${GO_WORKDIR}"

meta/classes-recipe/go-mod.bbclass +0 -5
-
-export GOMODCACHE = "${B}/.mod"
-
-do_compile[cleandirs] += "${B}/.mod"
--
Quoted 5 lines

2.39.2

--
Best regards,

José Quaresma

#4 lore raw
In reply to Christian Lindeberg
Quoted 21 lines · Christian Lindeberg

From: Christian Lindeberg <[email protected]>

Set the GO_MOD_CACHE_DIR variable and move the location of the module cache to enable the use of the go module fetchers for module dependencies.
Also, clean out the module cache before unpacking.

---
  meta/classes-recipe/go-mod.bbclass | 8 ++++----
  1 file changed, 4 insertions(+), 4 deletions(-)
meta/classes-recipe/go-mod.bbclass +0 -0
index ca3a690d05..71a6712c47 100644
@@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw"
 
  inherit go
 

meta/classes-recipe/go-mod.bbclass GOBUILDFLAGS:append = " -modcacherw" · line 30

Quoted 10 lines
meta/classes-recipe/go-mod.bbclass +4 -4
+GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), 'pkg/mod')}"
+export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), d.getVar('GO_MOD_CACHE_DIR'))}"
+do_unpack[cleandirs] += "${GOMODCACHE}"
+
  GO_WORKDIR ?= "${GO_IMPORT}"
  do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
-
-export GOMODCACHE = "${B}/.mod"
-
-do_compile[cleandirs] += "${B}/.mod"

What is the reasoning behind moving the cache dir out of the build dir ${B} and into the bitbake managed sources dir ${S} ? There might be a good reason for doing it, but it was not explained in the commit message.

Quoted 9 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#204444): https://lists.openembedded.org/g/openembedded-core/message/204444
Mute This Topic: https://lists.openembedded.org/mt/108413937/6551054
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 3 lines
Ryan Eatmon                [email protected]
-----------------------------------------
Texas Instruments, Inc.  -  LCPD  -  MGTS
#5 lore raw
Quoted 33 lines · Ryan Eatmon
Quoted 30 lines · Christian Lindeberg

From: Christian Lindeberg <[email protected]>

Set the GO_MOD_CACHE_DIR variable and move the location of the module cache to enable the use of the go module fetchers for module dependencies.
Also, clean out the module cache before unpacking.

---
meta/classes-recipe/go-mod.bbclass | 8 ++++----

1 file changed, 4 insertions(+), 4 deletions(-)
meta/classes-recipe/go-mod.bbclass +0 -0
index ca3a690d05..71a6712c47 100644
@@ -22,9 +22,9 @@ GOBUILDFLAGS:append = " -modcacherw"
 

inherit go

+GO_MOD_CACHE_DIR ?= "${@os.path.join(os.path.basename(d.getVar('S')), 'pkg/mod')}"
+export GOMODCACHE = "${@os.path.join(os.path.dirname(d.getVar('S')), d.getVar('GO_MOD_CACHE_DIR'))}"

meta/classes-recipe/go-mod.bbclass +2 -0
+do_unpack[cleandirs] += "${GOMODCACHE}"
+

GO_WORKDIR ?= "${GO_IMPORT}"
do_compile[dirs] += "${B}/src/${GO_WORKDIR}"
-

Quoted 8 lines

meta/classes-recipe/go-mod.bbclass GOBUILDFLAGS:append = " -modcacherw" · line 30

Quoted 3 lines
meta/classes-recipe/go-mod.bbclass +0 -3
-export GOMODCACHE = "${B}/.mod"
-
-do_compile[cleandirs] += "${B}/.mod"

What is the reasoning behind moving the cache dir out of the build dir ${B} and into the bitbake managed sources dir ${S} ? There might be a good reason for doing it, but it was not explained in the commit message.

In the review of the gomod and gomodgit fetchers concerns were raised about letting the fetchers unpack under ${GOMODCACHE} instead of relative to the unpack root directory.

With the fetchers there is now a way to have the module cache part of the bitbake managed sources instead of having to resort to do_compile[network] = "1".

Thanks,
Christian

Quoted 8 lines

--
Ryan Eatmon [email protected]
-----------------------------------------
Texas Instruments, Inc. - LCPD - MGTS