Skip to contentSkip to conversation

go.bbclass: change GOTMPDIR to improve reproducibility

PATCH Superseded [email protected] 3 participants 10 messages Dec 8, 2025 – Dec 15, 2025

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 go.bbclass: change GOTMPDIR to improve reproducibility

From: Changqing Li <[email protected]>

The Go toolchain writes temporary source files under GOTMPDIR and compiles them there. To support reproducibility, Go passes options such as -ffile-prefix-map=$WORK/b387=/tmp/go-build to the GCC instance it invokes. The variable WORK is a temporary directory created under GOTMPDIR, refer this example:

WORK=/build/tmp/work/x86-64-v3-wrs-linux/buildah/1.41.5/build-tmp/go-build377321751
cd $WORK/b387
TERM='dumb' x86_64-wrs-linux-gcc -m64 -march=x86-64-v3 -fstack-protector-strong -O2 -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/tmp/work/x86-64-v3-wrs-linux/buildah/1.41.5/recipe-sysroot -I /tmp/work/x86-64-v3-wrs-linux/buildah/1.41.5/sources/buildah-1.41.5/src/github.com/containers/buildah/vendor/github.com/proglottis/gpgme -fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=$WORK/b387=/tmp/go-build -gno-record-gcc-switches -v -D_FILE_OFFSET_BITS=64 -I $WORK/b387/ -O2 -g -ffile-prefix-map=/tmp/work/x86-64-v3-wrs-linux/buildah/1.41.5/sources/buildah-1.41.5=/usr/src/debug/buildah/1.41.5 -ffile-prefix-map=/tmp/work/x86-64-v3-wrs-linux/buildah/1.41.5/build=/usr/src/debug/buildah/1.41.5 -ffile-prefix-map=/tmp/work/x86-64-v3-wrs-linux/buildah/1.41.5/recipe-sysroot= -ffile-prefix-map=/tmp/work/x86-64-v3-wrs-linux/buildah/1.41.5/recipe-sysroot-native= -pipe -v -ffile-prefix-map=/tmp/work/x86-64-v3-wrs-linux/buildah/1.41.5/sources/buildah-1.41.5/src/github.com/containers/buildah/vendor=/_/vendor -frandom-seed=TZkSPVSBUvDMjg4wKjWS -o $WORK/b387/_x004.o -c unset_agent_info.cgo2.c

OE also passes its own DEBUG_PREFIX_MAP to GCC(finally by CGO_CFLAGS), including -ffile-prefix-map=${B}=${TARGET_DBGSRC_DIR}, where B is ${WORKDIR}/build. Because GOTMPDIR defaults to ${WORKDIR}/build-tmp, the Go temporary directory looks like ${WORKDIR}/build-tmp/go-buildXYZ. Its prefix therefore begins with ${WORKDIR}/build, so GCC matches the DEBUG_PREFIX_MAP entry for ${B} first.

As a result, a path such as ${WORKDIR}/build-tmp/go-buildXYZ is rewritten to ${TARGET_DBGSRC_DIR}-tmp/go-buildXYZ. This breaks the -ffile-prefix-map option that Go itself adds, because the original WORK path no longer matches the value Go expects. Since Go creates go-buildXYZ directories randomly and internally, this causes the build non-reproducible.

This patch changes GOTMPDIR from ${WORKDIR}/build-tmp to ${WORKDIR}/go-build-tmp so that the path no longer matches ${B}. This prevents unintended replacements by OE's DEBUG_PREFIX_MAP and restores reproducibility.

---
 meta/classes-recipe/go.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-recipe/go.bbclass +1 -1
index e0f667373e..ed986ff5d4 100644
@@ -79,7 +79,7 @@ B = "${WORKDIR}/build"
 export GOPATH = "${B}"
 export GOENV = "off"
 export GOPROXY ??= "https://proxy.golang.org,direct"
-export GOTMPDIR ?= "${WORKDIR}/build-tmp"
+export GOTMPDIR ?= "${WORKDIR}/go-build-tmp"
 GOTMPDIR[vardepvalue] = ""
 
 GO_SRCURI_DESTSUFFIX = "${@os.path.join(os.path.basename(d.getVar('S')), 'src', d.getVar('GO_IMPORT')) + '/'}"
Signature · 1 line
2.34.1
#1 lore raw

PATCH 2/2 go.bbclass: set buildid to empty string to improve reproducibility

From: Changqing Li <[email protected]>

Seems there are other things making Go BuildID not reproducible, set buildid to empty string to improve reproducibility, open a new issue [1], and refer an old issue [2].

Refer:
[1] https://github.com/golang/go/issues/76743
[2] https://github.com/golang/go/issues/34186

---
 meta/classes-recipe/go.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-recipe/go.bbclass +1 -1
index ed986ff5d4..2593368af7 100644
@@ -50,7 +50,7 @@ GO_RPATH:class-native = "${@'-r ${STAGING_LIBDIR_NATIVE}/go/pkg/${TARGET_GOTUPLE
 GO_RPATH_LINK:class-native = "${@'-Wl,-rpath-link=${STAGING_LIBDIR_NATIVE}/go/pkg/${TARGET_GOTUPLE}_dynlink' if d.getVar('GO_DYNLINK') else ''}"
 GO_EXTLDFLAGS ?= "${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS} ${GO_RPATH_LINK} ${LDFLAGS}"
 GO_LINKMODE ?= ""
-GO_EXTRA_LDFLAGS ?= ""
+GO_EXTRA_LDFLAGS ?= "-buildid="
 GO_LINUXLOADER ?= "-I ${@get_linuxloader(d)}"
 # Use system loader. If uninative is used, the uninative loader will be patched automatically
 GO_LINUXLOADER:class-native = ""
Signature · 1 line
2.34.1
#2 lore raw

RE: [OE-core] [PATCH 2/2] go.bbclass: set buildid to empty string to improve reproducibility

Quoted 30 lines

-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Changqing Li via lists.openembedded.org
Sent: den 8 december 2025 10:12
To: [email protected]
Subject: [OE-core] [PATCH 2/2] go.bbclass: set buildid to empty string to improve reproducibility

From: Changqing Li <[email protected]>

Seems there are other things making Go BuildID not reproducible, set buildid to empty string to improve reproducibility, open a new issue [1], and refer an old issue [2].

Refer:
[1] https://github.com/golang/go/issues/76743
[2] https://github.com/golang/go/issues/34186

---
 meta/classes-recipe/go.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-recipe/go.bbclass +0 -0
index ed986ff5d4..2593368af7 100644
@@ -50,7 +50,7 @@ GO_RPATH:class-native = "${@'-r

${STAGING_LIBDIR_NATIVE}/go/pkg/${TARGET_GOTUPLE
GO_RPATH_LINK:class-native = "${@'-Wl,-rpath-link=${STAGING_LIBDIR_NATIVE}/go/pkg/${TARGET_GOTUPLE}_dynlink' if d.getVar('GO_DYNLINK') else ''}" GO_EXTLDFLAGS ?= "${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS} ${GO_RPATH_LINK} ${LDFLAGS}"
GO_LINKMODE ?= ""

meta/classes-recipe/go.bbclass GO_RPATH:class-native = "${@'-r ${STAGING_LIBDIR_NATIVE}/go/pkg/${TARGET_GOTUPLE · line 53

Quoted 2 lines
meta/classes-recipe/go.bbclass +1 -1
-GO_EXTRA_LDFLAGS ?= ""
+GO_EXTRA_LDFLAGS ?= "-buildid="

Adding it to GO_EXTRA_LDFLAGS will probably not have the intended effect as some recipes already set that variable and thus will not receive this option. You should probably add a new variable for it instead, like the other options that are set in GO_LDFLAGS.

Quoted 5 lines

GO_LINUXLOADER ?= "-I ${@get_linuxloader(d)}"
# Use system loader. If uninative is used, the uninative loader will be patched automatically GO_LINUXLOADER:class-native = ""
--
2.34.1

//Peter

#3 lore raw

Re: [OE-core] [PATCH 2/2] go.bbclass: set buildid to empty string to improve reproducibility

Quoted 24 lines · Peter Kjellerstedt

CAUTION: This email comes from a non Wind River email account!
Do not click links or open attachments unless you recognize the sender and know the content is safe.

Quoted 21 lines

-----Original Message-----
From:[email protected] <[email protected]> On Behalf Of Changqing Li via lists.openembedded.org
Sent: den 8 december 2025 10:12
To:[email protected]
Subject: [OE-core] [PATCH 2/2] go.bbclass: set buildid to empty string to improve reproducibility

From: Changqing Li<[email protected]>

Seems there are other things making Go BuildID not reproducible, set buildid to empty string to improve reproducibility, open a new issue [1], and refer an old issue [2].

Refer:
[1]https://github.com/golang/go/issues/76743
[2]https://github.com/golang/go/issues/34186

---
  meta/classes-recipe/go.bbclass | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 16 lines

meta/classes-recipe/go.bbclass GO_RPATH:class-native = "${@'-r ${STAGING_LIBDIR_NATIVE}/go/pkg/${TARGET_GOTUPLE · line 53

Quoted 12 lines
meta/classes-recipe/go.bbclass +0 -0
index ed986ff5d4..2593368af7 100644
@@ -50,7 +50,7 @@ GO_RPATH:class-native = "${@'-r

${STAGING_LIBDIR_NATIVE}/go/pkg/${TARGET_GOTUPLE
GO_RPATH_LINK:class-native ="${@'-Wl,-rpath-link=${STAGING_LIBDIR_NATIVE}/go/pkg/${TARGET_GOTUPLE}_dynlink'
if d.getVar('GO_DYNLINK') else ''}"
GO_EXTLDFLAGS ?= "${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS} ${GO_RPATH_LINK} ${LDFLAGS}" GO_LINKMODE ?= ""

meta/classes-recipe/go.bbclass +1 -1
-GO_EXTRA_LDFLAGS ?= ""
+GO_EXTRA_LDFLAGS ?= "-buildid="

Adding it to GO_EXTRA_LDFLAGS will probably not have the intended effect as some recipes already set that variable and thus will not receive this option. You should probably add a new variable for it instead, like the other options that are set in GO_LDFLAGS.

Good suggestion.  Maybe I can add a varibale GO_BUILDID ?= "-buildid="

//Changqing

Quoted 7 lines
Quoted 5 lines

GO_LINUXLOADER ?= "-I ${@get_linuxloader(d)}"
# Use system loader. If uninative is used, the uninative loader will be patched automatically GO_LINUXLOADER:class-native = ""
--
2.34.1

//Peter

#4 lore raw
In reply to [email protected]
Quoted 4 lines · Changqing Li via lists.openembedded.org

This patch changes GOTMPDIR from ${WORKDIR}/build-tmp to ${WORKDIR}/go-build-tmp so that the path no longer matches ${B}. This prevents unintended replacements by OE's DEBUG_PREFIX_MAP and restores reproducibility.

Wait. Go does not have reproducibility problems right now: it is a part of reproducibility test, and not in the exclusion list. So how can the issues that this patch set aims to fix be observed?

Alex

#5 lore raw
Quoted 6 lines · Alexander Kanavin

CAUTION: This email comes from a non Wind River email account!
Do not click links or open attachments unless you recognize the sender and know the content is safe.

On Mon, 8 Dec 2025 at 10:11, Changqing Li via lists.openembedded.org <[email protected]> wrote:

Quoted 7 lines
Quoted 4 lines

This patch changes GOTMPDIR from ${WORKDIR}/build-tmp to ${WORKDIR}/go-build-tmp so that the path no longer matches ${B}. This prevents unintended replacements by OE's DEBUG_PREFIX_MAP and restores reproducibility.

Wait. Go does not have reproducibility problems right now: it is a part of reproducibility test, and not in the exclusion list. So how can the issues that this patch set aims to fix be observed?

I checked autobuilder configs,  although we have reproducible test, but only test with openembeded-core

and layers under meta-openembeded,   and there is no recipe inherit go.bbclass(except etcd under meta-oe,  but already skipped)

So actually,  we don't have reproducible test for recipes that compile with go.

And I tested with this step:

on a host,  with build path ra,  and another build path rb-extend, bitbake buildah under both project,

the rpm of ra, and rb-extend is different,   and with these 2 patches,  the rpm will be the same.

//Changqing

Quoted 2 lines

Alex

#6 lore raw
Quoted 9 lines · Changqing Li

Wait. Go does not have reproducibility problems right now: it is a part of reproducibility test, and not in the exclusion list. So how can the issues that this patch set aims to fix be observed?

I checked autobuilder configs, although we have reproducible test, but only test with openembeded-core

and layers under meta-openembeded, and there is no recipe inherit go.bbclass(except etcd under meta-oe, but already skipped)

So actually, we don't have reproducible test for recipes that compile with go.

I see. I was confused and thought this fixes the go toolchain somehow.

There is one recipe on oe-core that does inherit go class (go-helloworld), can you check why reproducibility issues don't happen there? It would be good to have a reproducibility test for things written in go in core, so this doesn't again regress quietly.

Alex

#7 lore raw
Quoted 12 lines · Alexander Kanavin

CAUTION: This email comes from a non Wind River email account!
Do not click links or open attachments unless you recognize the sender and know the content is safe.

Quoted 8 lines · Changqing Li

Wait. Go does not have reproducibility problems right now: it is a part of reproducibility test, and not in the exclusion list. So how can the issues that this patch set aims to fix be observed?

I checked autobuilder configs, although we have reproducible test, but only test with openembeded-core

and layers under meta-openembeded, and there is no recipe inherit go.bbclass(except etcd under meta-oe, but already skipped)

Quoted 7 lines
Quoted 1 line

So actually, we don't have reproducible test for recipes that compile with go.

I see. I was confused and thought this fixes the go toolchain somehow.

There is one recipe on oe-core that does inherit go class (go-helloworld), can you check why reproducibility issues don't happen there? It would be good to have a reproducibility test for things written in go in core, so this doesn't again regress quietly.

This issue reproduced when cgo is enabled,  and go internally add --ffile-prefix-map="WorkDir"=/tmp/go-build" to gcc

to tell gcc not include workdir in object file. for go-helloworld, it should be pure go,  does not include the "C", so it will not met this issue.

//Changqing

Quoted 1 line

Alex

#8 lore raw
Quoted 3 lines · Changqing Li

This issue reproduced when cgo is enabled, and go internally add --ffile-prefix-map="WorkDir"=/tmp/go-build" to gcc

to tell gcc not include workdir in object file. for go-helloworld, it should be pure go, does not include the "C", so it will not met this issue.

Alright. The patches are ok then, please summarize these points in v2 commit messages.

Alex

#9 lore raw
In reply to Changqing Li
Quoted 8 lines · Changqing Li

There is one recipe on oe-core that does inherit go class (go-helloworld), can you check why reproducibility issues don't happen there? It would be good to have a reproducibility test for things written in go in core, so this doesn't again regress quietly.

This issue reproduced when cgo is enabled, and go internally add --ffile-prefix-map="WorkDir"=/tmp/go-build" to gcc

to tell gcc not include workdir in object file. for go-helloworld, it should be pure go, does not include the "C", so it will not met this issue.

I have filed a bugzilla ticket for the missong cgo reproducibility
testing: https://bugzilla.yoctoproject.org/show_bug.cgi?id=16100

If you have ideas how to address it (with something minimal) in oe-core, that would be welcome.

Alex