Skip to contentSkip to conversation

rootfs: Make BUILDNAME a weak default in reproducible_build_simple

PATCH In review Alex Kiernan 2 participants 4 messages Jan 18, 2020 – Feb 9, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 rootfs: Make BUILDNAME a weak default in reproducible_build_simple

11e45082ad00 ("rootfs-postcommands.bbclass: improve binary reproducibility") fixed binary reproducibility of /etc/version, but with the move to reproducibilty in all builds, setting /etc/version to anything other than the default fixed timestamp is tricky because rootfs_reproducible() runs very late.

rootfs.py uses BUILDNAME if set for /etc/version, so introduce a weak default for BUILDNAME of "REPRODUCIBLE_TIMESTAMP_ROOTFS", when enabling reproducible builds hence allowing BUILDNAME to be overridden elsewhere.

---

 meta/classes/reproducible_build_simple.bbclass | 1 +
 meta/classes/rootfs-postcommands.bbclass       | 5 -----
 2 files changed, 1 insertion(+), 5 deletions(-)
meta/classes/reproducible_build_simple.bbclass +1 -0
index d0842f0f902a..153a07607219 100644
@@ -7,5 +7,6 @@ export PERL_HASH_SEED = "0"
 export SOURCE_DATE_EPOCH ??= "1520598896"
 
 REPRODUCIBLE_TIMESTAMP_ROOTFS ??= "1520598896"
+BUILDNAME ??= "${REPRODUCIBLE_TIMESTAMP_ROOTFS}"
 
 inherit podfix
meta/classes/rootfs-postcommands.bbclass +0 -5
index fdc63bb64d55..ebf96be7e85f 100644
@@ -356,11 +356,6 @@ python rootfs_log_check_recommends() {
 # Perform any additional adjustments needed to make rootf binary reproducible
 rootfs_reproducible () {
 	if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
-		# Convert UTC into %4Y%2m%2d%2H%2M%2S
-		sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M%2S`
-		echo $sformatted > ${IMAGE_ROOTFS}/etc/version
-		bbnote "rootfs_reproducible: set /etc/version to $sformatted"
-
 		if [ -d ${IMAGE_ROOTFS}${sysconfdir}/gconf ]; then
 			find ${IMAGE_ROOTFS}${sysconfdir}/gconf -name '%gconf.xml' -print0 | xargs -0r \
 			sed -i -e 's@\bmtime="[0-9][0-9]*"@mtime="'${REPRODUCIBLE_TIMESTAMP_ROOTFS}'"@g'
Signature · 1 line
2.17.1
#1 lore raw
Quoted 30 lines

-----Original Message-----
From: [email protected] <openembedded-core- [email protected]> On Behalf Of Alex Kiernan
Sent: den 18 januari 2020 06:41
To: [email protected]
Subject: [OE-core] [OE-Core][PATCH] rootfs: Make BUILDNAME a weak default in reproducible_build_simple

11e45082ad00 ("rootfs-postcommands.bbclass: improve binary reproducibility") fixed binary reproducibility of /etc/version, but with the move to reproducibilty in all builds, setting /etc/version to anything other than the default fixed timestamp is tricky because rootfs_reproducible() runs very late.

rootfs.py uses BUILDNAME if set for /etc/version, so introduce a weak default for BUILDNAME of "REPRODUCIBLE_TIMESTAMP_ROOTFS", when enabling reproducible builds hence allowing BUILDNAME to be overridden elsewhere.

---

 meta/classes/reproducible_build_simple.bbclass | 1 +
 meta/classes/rootfs-postcommands.bbclass       | 5 -----
 2 files changed, 1 insertion(+), 5 deletions(-)
meta/classes/reproducible_build_simple.bbclass +0 -0
index d0842f0f902a..153a07607219 100644

meta/classes/reproducible_build_simple.bbclass export PERL_HASH_SEED = "0" · line 10

Quoted 4 lines
meta/classes/reproducible_build_simple.bbclass +1 -0
@@ -7,5 +7,6 @@ export PERL_HASH_SEED = "0"
 export SOURCE_DATE_EPOCH ??= "1520598896"
 
 REPRODUCIBLE_TIMESTAMP_ROOTFS ??= "1520598896"
+BUILDNAME ??= "${REPRODUCIBLE_TIMESTAMP_ROOTFS}"

Can we please revert this change? This breaks (all?) other usages of ${BUILDNAME}. E.g., the information in tmp/buildstats is totally useless now since all builds end up in the same directory "tmp/buildstats/1520598896", overwriting each other...

Quoted 19 lines
meta/classes/reproducible_build_simple.bbclass +0 -0
 inherit podfix
meta/classes/rootfs-postcommands.bbclass +0 -6
index fdc63bb64d55..ebf96be7e85f 100644
@@ -356,11 +356,6 @@ python rootfs_log_check_recommends() {
 # Perform any additional adjustments needed to make rootf binary reproducible
 rootfs_reproducible () {
 	if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
-		# Convert UTC into %4Y%2m%2d%2H%2M%2S
-		sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M%2S`
-		echo $sformatted > ${IMAGE_ROOTFS}/etc/version
-		bbnote "rootfs_reproducible: set /etc/version to $sformatted"
-
 		if [ -d ${IMAGE_ROOTFS}${sysconfdir}/gconf ]; then
 			find ${IMAGE_ROOTFS}${sysconfdir}/gconf -name '%gconf.xml' -print0 | xargs -0r \
 			sed -i -e 's@\bmtime="[0-9][0-9]*"@mtime="'${REPRODUCIBLE_TIMESTAMP_ROOTFS}'"@g'
--
Quoted 1 line

2.17.1

//Peter

#2 lore raw
Quoted 33 lines · Peter Kjellerstedt
Quoted 33 lines

-----Original Message-----
From: [email protected] <openembedded-core- [email protected]> On Behalf Of Alex Kiernan
Sent: den 18 januari 2020 06:41
To: [email protected]
Subject: [OE-core] [OE-Core][PATCH] rootfs: Make BUILDNAME a weak default in reproducible_build_simple

11e45082ad00 ("rootfs-postcommands.bbclass: improve binary reproducibility") fixed binary reproducibility of /etc/version, but with the move to reproducibilty in all builds, setting /etc/version to anything other than the default fixed timestamp is tricky because rootfs_reproducible() runs very late.

rootfs.py uses BUILDNAME if set for /etc/version, so introduce a weak default for BUILDNAME of "REPRODUCIBLE_TIMESTAMP_ROOTFS", when enabling reproducible builds hence allowing BUILDNAME to be overridden elsewhere.

---

 meta/classes/reproducible_build_simple.bbclass | 1 +
 meta/classes/rootfs-postcommands.bbclass       | 5 -----
 2 files changed, 1 insertion(+), 5 deletions(-)
meta/classes/reproducible_build_simple.bbclass +0 -0
index d0842f0f902a..153a07607219 100644
@@ -7,5 +7,6 @@ export PERL_HASH_SEED = "0"
 export SOURCE_DATE_EPOCH ??= "1520598896"
 
Quoted 8 lines

meta/classes/reproducible_build_simple.bbclass export PERL_HASH_SEED = "0" · line 10

Quoted 2 lines
meta/classes/reproducible_build_simple.bbclass +1 -0
 REPRODUCIBLE_TIMESTAMP_ROOTFS ??= "1520598896"
+BUILDNAME ??= "${REPRODUCIBLE_TIMESTAMP_ROOTFS}"

Can we please revert this change? This breaks (all?) other usages of ${BUILDNAME}. E.g., the information in tmp/buildstats is totally useless now since all builds end up in the same directory "tmp/buildstats/1520598896", overwriting each other...

+1; I'd agree that seems rather more broken than the problem I was trying to fix here.

Signature · 1 line
Alex Kiernan
#3 lore raw
Quoted 58 lines

-----Original Message-----
From: Alex Kiernan <[email protected]>
Sent: den 7 februari 2020 18:04
To: Peter Kjellerstedt <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [OE-Core][PATCH] rootfs: Make BUILDNAME a weak default in reproducible_build_simple

Quoted 3 lines · Peter Kjellerstedt
Quoted 3 lines

-----Original Message-----
From: [email protected] <openembedded-

core-

Quoted 4 lines
Quoted 4 lines

[email protected]> On Behalf Of Alex Kiernan
Sent: den 18 januari 2020 06:41
To: [email protected]
Subject: [OE-core] [OE-Core][PATCH] rootfs: Make BUILDNAME a weak

default

Quoted 4 lines
Quoted 4 lines

in reproducible_build_simple

11e45082ad00 ("rootfs-postcommands.bbclass: improve binary reproducibility") fixed binary reproducibility of /etc/version, but

with

Quoted 6 lines
Quoted 6 lines

the move to reproducibilty in all builds, setting /etc/version to anything other than the default fixed timestamp is tricky because rootfs_reproducible() runs very late.

rootfs.py uses BUILDNAME if set for /etc/version, so introduce a weak default for BUILDNAME of "REPRODUCIBLE_TIMESTAMP_ROOTFS", when

enabling

Quoted 1 line
Quoted 1 line

reproducible builds hence allowing BUILDNAME to be overridden

elsewhere.

Quoted 9 lines

Review

Quoted 9 lines

---

 meta/classes/reproducible_build_simple.bbclass | 1 +
 meta/classes/rootfs-postcommands.bbclass       | 5 -----
 2 files changed, 1 insertion(+), 5 deletions(-)
(quoted diff) +0 -0

b/meta/classes/reproducible_build_simple.bbclass

Quoted 6 lines
Quoted 6 lines

index d0842f0f902a..153a07607219 100644

meta/classes/reproducible_build_simple.bbclass +0 -0
@@ -7,5 +7,6 @@ export PERL_HASH_SEED = "0"
 export SOURCE_DATE_EPOCH ??= "1520598896"
 
Quoted 8 lines

meta/classes/reproducible_build_simple.bbclass export PERL_HASH_SEED = "0" · line 10

Quoted 2 lines
meta/classes/reproducible_build_simple.bbclass +1 -0
 REPRODUCIBLE_TIMESTAMP_ROOTFS ??= "1520598896"
+BUILDNAME ??= "${REPRODUCIBLE_TIMESTAMP_ROOTFS}"

Can we please revert this change? This breaks (all?) other usages of ${BUILDNAME}. E.g., the information in tmp/buildstats is totally useless now since all builds end up in the same directory "tmp/buildstats/1520598896", overwriting each other...

Quoted 5 lines

+1; I'd agree that seems rather more broken than the problem I was trying to fix here.

--
Alex Kiernan

I guess you can achieve what you originally wanted by introducing some new variable, but I will send a patch to revert the above change for now.

//Peter