Skip to contentSkip to conversation

Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

PATCH In review Paul Kocialkowski 11 participants 43 messages Jul 26, 2015 – Oct 2, 2015 Kbuild

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

In order to achieve reproducible builds in U-Boot, timestamps that are defined at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment variable allows setting a fixed value for those timestamps.

Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be built reproducibly. This is the case for e.g. sunxi devices.

However, some other devices might need some more tweaks, especially regarding the image generation tools.

---
 Makefile              |  7 ++++---
 README                | 12 ++++++++++++
 tools/default_image.c | 21 ++++++++++++++++++++-
 3 files changed, 36 insertions(+), 4 deletions(-)
Makefile +4 -3
index 54ef2cd..596974c 100644
@@ -1232,9 +1232,10 @@ define filechk_version.h
 endef
 
 define filechk_timestamp.h
-	(LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
-	LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
-	LC_ALL=C date +'#define U_BOOT_TZ "%z"')
+	(SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
+	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
+	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
 endef
 
 $(version_h): include/config/uboot.release FORCE
README +12 -0
index 4e0ff9f..1bcb63c 100644
@@ -5081,6 +5081,18 @@ This firmware often needs to be loaded during U-Boot booting.
 - CONFIG_SYS_MEM_TOP_HIDE_MIN
 	Define minimum DDR size to be hided from top of the DDR memory
 
+Reproducible builds
+-------------------
+
+In order to achieve reproducible builds, timestamps used in the U-Boot build
+process have to be set to a fixed value.
+
+This is done using the SOURCE_DATE_EPOCH environment variable.
+SOURCE_DATE_EPOCH is to be set on the build host's shell, not as a configuration
+option for U-Boot or an environment variable in U-Boot.
+
+SOURCE_DATE_EPOCH should be set to a number of seconds since the epoch, in UTC.
+
 Building the Software:
 ======================
 
tools/default_image.c +20 -1
index cf5c0d4..18940af 100644
@@ -88,6 +88,9 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd,
 				struct image_tool_params *params)
 {
 	uint32_t checksum;
+	char *source_date_epoch;
+	struct tm *time_universal;
+	time_t time;
 
 	image_header_t * hdr = (image_header_t *)ptr;
 
@@ -96,9 +99,25 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd,
 				sizeof(image_header_t)),
 			sbuf->st_size - sizeof(image_header_t));
 
+	source_date_epoch = getenv("SOURCE_DATE_EPOCH");
+	if (source_date_epoch != NULL) {
+		time = (time_t) strtol(source_date_epoch, NULL, 10);
+
+		time_universal = gmtime(&time);
+		if (time_universal == NULL) {
+			fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n",
+				__func__);
+			time = 0;
+		} else {
+			time = mktime(time_universal);
+		}
+	} else {
+		time = sbuf->st_mtime;
+	}
+
 	/* Build new header */
 	image_set_magic(hdr, IH_MAGIC);
-	image_set_time(hdr, sbuf->st_mtime);
+	image_set_time(hdr, time);
 	image_set_size(hdr, sbuf->st_size - sizeof(image_header_t));
 	image_set_load(hdr, params->addr);
 	image_set_ep(hdr, params->ep);
Signature · 1 line
1.9.1
#1 lore raw

PATCH v2 1/1 Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

Quoted 11 lines · Paul Kocialkowski

In order to achieve reproducible builds in U-Boot, timestamps that are defined at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment variable allows setting a fixed value for those timestamps.

Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be built reproducibly. This is the case for e.g. sunxi devices.

However, some other devices might need some more tweaks, especially regarding the image generation tools.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150728/784dbdb0/attachment.sig>
#2 lore raw

PATCH v2 1/1 Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

Hi,

Quoted 9 lines · Tom Rini
Quoted 7 lines · Paul Kocialkowski

In order to achieve reproducible builds in U-Boot, timestamps that are defined at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment variable allows setting a fixed value for those timestamps.

Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be built reproducibly. This is the case for e.g. sunxi devices.

Quoted 8 lines
Quoted 4 lines

However, some other devices might need some more tweaks, especially regarding the image generation tools.

Applied to u-boot/master, thanks!

--

This commit breaks the following commit:

commit f3f431a712729a1af94d01bd1bfde17a252ff02c

Author: Chris Packham <[email protected]>
Date:   Sun May 10 21:02:09 2015 +1200

Makefile: Add U_BOOT_TZ and include in version

Define U_BOOT_TZ alongside U_BOOT_TIME and U_BOOT_DATE and use it to include the timezone in the version output.

Before this commit I have:
U-Boot 2015.07-00345-g9c57487 (Jul 31 2015 - 10:49:31 +0800)

After this commit I have:
U-Boot 2015.07-00346-gf3f431a (Jul 31 2015 - 02:50:54 +0000)

As you see: the timezone information is missing, and U-Boot's timestamp is now GMT+0 (the correct one should be GMT+8)

Is this intended behavior? Or if not, please fix it.

Regards,
Bin

#3 lore raw

PATCH v2 1/1 Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

Quoted 41 lines · Bin Meng

Hi,

Quoted 9 lines · Tom Rini
Quoted 7 lines · Paul Kocialkowski

In order to achieve reproducible builds in U-Boot, timestamps that are defined at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment variable allows setting a fixed value for those timestamps.

Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be built reproducibly. This is the case for e.g. sunxi devices.

Quoted 8 lines
Quoted 4 lines

However, some other devices might need some more tweaks, especially regarding the image generation tools.

Applied to u-boot/master, thanks!

--

This commit breaks the following commit:

commit f3f431a712729a1af94d01bd1bfde17a252ff02c

Author: Chris Packham <[email protected]>
Date:   Sun May 10 21:02:09 2015 +1200

Makefile: Add U_BOOT_TZ and include in version

Define U_BOOT_TZ alongside U_BOOT_TIME and U_BOOT_DATE and use it to include the timezone in the version output.

Before this commit I have:
U-Boot 2015.07-00345-g9c57487 (Jul 31 2015 - 10:49:31 +0800)

After this commit I have:
U-Boot 2015.07-00346-gf3f431a (Jul 31 2015 - 02:50:54 +0000)

Quoted 7 lines

As you see: the timezone information is missing, and U-Boot's timestamp is now GMT+0 (the correct one should be GMT+8)

Is this intended behavior? Or if not, please fix it.

Regards,
Bin

The problem is that date -u implies UTC. So +0000 is right if you do want the time to be displayed in UTC (for me living at +12/+13 UTC makes my head hurt because all the dates are yesterday).

The intent of f3f431a7 was to reflect the timezone that the build machine was set to. Dropping the -u would probably be sufficient but perhaps it would be better to make whole lot conditional on SOURCE_DATE_EPOCH being set. Something like this (untested, apologies gmail web interface)

ifneq ($(SOURCE_DATE_EPOCH),)
define filechk_timestamp.h

       (SOURCE_DATE="$(SOURCE_DATE_EPOCH)"; \
       LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b
%d %C%y"'; \
       LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
       LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"' )
endef
else
define filechk_timestamp.h
       (LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
       LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
       LC_ALL=C date +'#define U_BOOT_TZ "%z"')

endef
endif

#4 lore raw

PATCH 1/1 Makefile: Use correct timezone for U_BOOT_TZ

When building with SOURCE_DATE_EPOCH the timezone is in UTC. When building normally the timezone is taken from the build machine's locale setting.

Fixes f3f431a71272 ("Reproducible U-Boot build support, using

SOURCE_DATE_EPOCH")

---
Hopefully this will suit everyone. I've done a quick sanity test with sandbox and the date shown in the version output looks to be what I expect with/without SOURCE_DATE_EPOCH.

 Makefile | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
Makefile +10 -4
index 394ed09..f75c730 100644
@@ -1230,10 +1230,16 @@ define filechk_version.h
 endef
 
 define filechk_timestamp.h
-	(SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
+	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
+		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+	else \
+		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
+	fi)
 endef
 
 $(version_h): include/config/uboot.release FORCE
Signature · 1 line
2.5.0.rc0
#5 lore raw

PATCH 1/1 Makefile: Use correct timezone for U_BOOT_TZ

Quoted 31 lines · Chris Packham

When building with SOURCE_DATE_EPOCH the timezone is in UTC. When building normally the timezone is taken from the build machine's locale setting.

Fixes f3f431a71272 ("Reproducible U-Boot build support, using

SOURCE_DATE_EPOCH")

---
Hopefully this will suit everyone. I've done a quick sanity test with sandbox and the date shown in the version output looks to be what I expect with/without SOURCE_DATE_EPOCH.

 Makefile | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
Makefile +4 -4
index 394ed09..f75c730 100644
@@ -1230,10 +1230,16 @@ define filechk_version.h
 endef
 
 define filechk_timestamp.h
-       (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
-       LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-       LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
-       LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
+       (if test -n "$${SOURCE_DATE_EPOCH}"; then \
+               SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
+               LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+               LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \

Makefile define filechk_version.h

Quoted 10 lines
Makefile +6 -1
+               LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+       else \
+               LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
+               LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
+               LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
+       fi)
 endef
 
 $(version_h): include/config/uboot.release FORCE
--
#6 lore raw

PATCH 1/1 Makefile: Use correct timezone for U_BOOT_TZ

In reply to Chris Packham

Le vendredi 31 juillet 2015 ? 22:04 +1200, Chris Packham a ?crit :

Quoted 3 lines

When building with SOURCE_DATE_EPOCH the timezone is in UTC. When building normally the timezone is taken from the build machine's locale setting.

Thanks a lot for introducing this, I thought it might disturb a few people.

Quoted 2 lines
Fixes f3f431a71272 ("Reproducible U-Boot build support, using

SOURCE_DATE_EPOCH")

It is incorrect to say that your patch fixes my commit, because my commit didn't introduce anything unexpected or faulty.

If anything, it was merged before people gave their opinion about whether UTC should be default. I was also going to introduce a patch like yours later. For the record, UTC is default for Coreboot's timestamps.

Thus, please remove that line.

Otherwise, this is:

Quoted 24 lines

---
Hopefully this will suit everyone. I've done a quick sanity test with sandbox and the date shown in the version output looks to be what I expect with/without SOURCE_DATE_EPOCH.

 Makefile | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
Makefile +3 -4
index 394ed09..f75c730 100644
@@ -1230,10 +1230,16 @@ define filechk_version.h
 endef
 
 define filechk_timestamp.h
-	(SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
+	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
+		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \

Makefile define filechk_version.h · line 1245

Quoted 10 lines
Makefile +7 -0
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+	else \
+		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
+	fi)
 endef
 
 $(version_h): include/config/uboot.release FORCE

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150731/e7d7218c/attachment-0001.sig>

#7 lore raw

PATCH 1/1 Makefile: Use correct timezone for U_BOOT_TZ

(note I had some patman config that erroneously included extra people on the original Cc list, I've now dropped them)

Quoted 14 lines · Paul Kocialkowski

Le vendredi 31 juillet 2015 ? 22:04 +1200, Chris Packham a ?crit :

Quoted 3 lines

When building with SOURCE_DATE_EPOCH the timezone is in UTC. When building normally the timezone is taken from the build machine's locale setting.

Thanks a lot for introducing this, I thought it might disturb a few people.

Quoted 2 lines
Fixes f3f431a71272 ("Reproducible U-Boot build support, using

SOURCE_DATE_EPOCH")

It is incorrect to say that your patch fixes my commit, because my commit didn't introduce anything unexpected or faulty.

Quoted 6 lines

If anything, it was merged before people gave their opinion about whether UTC should be default. I was also going to introduce a patch like yours later. For the record, UTC is default for Coreboot's timestamps.

Thus, please remove that line.

Done. v2 on it's way.

The other mailing list thread also talked about the possibility of having SOURCE_DATE_TZOFFSET I'll look at a proof of concept for that.

Quoted 38 lines

Otherwise, this is:

Quoted 34 lines

---
Hopefully this will suit everyone. I've done a quick sanity test with sandbox and the date shown in the version output looks to be what I expect with/without SOURCE_DATE_EPOCH.

 Makefile | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
Makefile +10 -4
index 394ed09..f75c730 100644
@@ -1230,10 +1230,16 @@ define filechk_version.h
 endef
 
 define filechk_timestamp.h
-     (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
-     LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-     LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
-     LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
+     (if test -n "$${SOURCE_DATE_EPOCH}"; then \
+             SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
+             LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+             LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+             LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+     else \
+             LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
+             LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
+             LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
+     fi)
 endef
 
 $(version_h): include/config/uboot.release FORCE
#8 lore raw

PATCH v2 1/1 Makefile: Use correct timezone for U_BOOT_TZ

When building with SOURCE_DATE_EPOCH the timezone is in UTC. When building normally the timezone is taken from the build machine's locale setting.

---

Changes in v2:
- Collect some tested-by tags
- Remove reference to f3f431a71272 in the commit message
- Drop Ccs that were erroneously added when submitting v1, remaining Ccs are from the original mailing list thread

 Makefile | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
Makefile +10 -4
index 394ed09..f75c730 100644
@@ -1230,10 +1230,16 @@ define filechk_version.h
 endef
 
 define filechk_timestamp.h
-	(SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
-	LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
+	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
+		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+	else \
+		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
+	fi)
 endef
 
 $(version_h): include/config/uboot.release FORCE
Signature · 1 line
2.5.0.rc0
#9 lore raw

PATCH v2 1/1 Makefile: Use correct timezone for U_BOOT_TZ

Hi Tom,

Quoted 44 lines · Chris Packham

When building with SOURCE_DATE_EPOCH the timezone is in UTC. When building normally the timezone is taken from the build machine's locale setting.

---

Changes in v2:
- Collect some tested-by tags
- Remove reference to f3f431a71272 in the commit message
- Drop Ccs that were erroneously added when submitting v1, remaining Ccs are from the original mailing list thread

 Makefile | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
Makefile +10 -5
index 394ed09..f75c730 100644
@@ -1230,10 +1230,16 @@ define filechk_version.h
 endef
 
 define filechk_timestamp.h
-       (SOURCE_DATE="$${SOURCE_DATE_EPOCH:+@$$SOURCE_DATE_EPOCH}"; \
-       LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-       LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TIME "%T"'; \
-       LC_ALL=C date -u -d "$${SOURCE_DATE:-now}" +'#define U_BOOT_TZ "%z"' )
+       (if test -n "$${SOURCE_DATE_EPOCH}"; then \
+               SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
+               LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+               LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+               LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+       else \
+               LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
+               LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
+               LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
+       fi)
 endef
 
 $(version_h): include/config/uboot.release FORCE
--
Quoted 2 lines

2.5.0.rc0

Ping? I think this may have been missed due to an excessive Cc list (which I've now culled).

#10 lore raw

PATCH v2 1/1 Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

In reply to Chris Packham
Quoted 4 lines · Chris Packham

The problem is that date -u implies UTC. So +0000 is right if you do want the time to be displayed in UTC (for me living at +12/+13 UTC makes my head hurt because all the dates are yesterday).

One of the main problems with human-readable date/time is that it implies a calendar, with all its exceptions like leap years, changing DST, etc. Many RFCs, and the POSIX date functions, are highly flawed and conflate calendars with absolute time, e.g. don't specify they mean the Gregorian calendar, and don't specify what to do with dates before when the Gregorian calendar was introduced.

If you want to be precise about these things (so as to not cause problems in the future), then you need to specify all of that when dealing with human-readable dates. This is why for now, the Reproducible Builds timestamps proposal only defines SOURCE_DATE_EPOCH. We might extend it later, though.

Quoted 19 lines

The intent of f3f431a7 was to reflect the timezone that the build machine was set to. Dropping the -u would probably be sufficient but perhaps it would be better to make whole lot conditional on SOURCE_DATE_EPOCH being set. Something like this (untested, apologies gmail web interface)

ifneq ($(SOURCE_DATE_EPOCH),)
define filechk_timestamp.h

       (SOURCE_DATE="$(SOURCE_DATE_EPOCH)"; \
       LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b
%d %C%y"'; \
       LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
       LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"' )
endef
else
define filechk_timestamp.h
       (LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
       LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
       LC_ALL=C date +'#define U_BOOT_TZ "%z"')
Quoted 3 lines

endef
endif

Yes, it is best to force the TZ as UTC for now, if SOURCE_DATE_EPOCH is set.

At some point later, we might add SOURCE_DATE_TZOFFSET, or other formats that include a timezone offset, or something. (That would match Git's internal timestamp format, which is "$epochts $tzoffset", and can be translated unambiguously into ISO8601 and RFC2822 formats). For now though, we decided to keep things simple. But please say so, if you think that is something you (or others who use timestamps in build systems) would really want.

BTW, you need to do "@$${SOURCE_DATE_EPOCH}" when giving timestamps to date(1):

$ date -d @1438330000 +%s
1438330000
$ date -d 1438330000 +%s
date: invalid date ?1438330000?
1

X

Signature · 10 lines
GPG: 4096R/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150731/79985a4a/attachment.sig>
#11 lore raw

RFC PATCH 1/1 Makefile: Add SOURCE_DATE_TZ

Along with SOURCE_DATE_EPOCH SOURCE_DATE_TZ can be used to recreate a build with a specific date timestamp. This allows the verification of source supplied with a pre-compiled binary.

If SOURCE_DATE_EPOCH is supplied SOURCE_DATE_TZ can be used to specify what will appear in the output of the version command. If SOURCE_DATE_TZ is not specified UTC will be used. SOURCE_DATE_TZ on it's own will not have an affect.

---
This is a quick proof of concept that allows some control of how the timezone is displayed.

 Makefile | 7 ++++---
 README   | 9 ++++++---
 2 files changed, 10 insertions(+), 6 deletions(-)
Makefile +4 -3
index f75c730..5102a9c 100644
@@ -1232,9 +1232,10 @@ endef
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		SOURCE_TZ="$${SOURCE_DATE_TZ:-UTC}"; \
+		TZ="$${SOURCE_TZ}" LC_ALL=C date -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		TZ="$${SOURCE_TZ}" LC_ALL=C date -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		TZ="$${SOURCE_TZ}" LC_ALL=C date -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
README +6 -3
index 1bcb63c..d2e3e94 100644
@@ -5087,11 +5087,14 @@ Reproducible builds
 In order to achieve reproducible builds, timestamps used in the U-Boot build
 process have to be set to a fixed value.
 
-This is done using the SOURCE_DATE_EPOCH environment variable.
-SOURCE_DATE_EPOCH is to be set on the build host's shell, not as a configuration
-option for U-Boot or an environment variable in U-Boot.
+This is done using the SOURCE_DATE_EPOCH and SOURCE_DATE_TZ environment
+variables. These are to be set on the build host's shell, not as a
+configuration option for U-Boot or an environment variable in U-Boot.
 
 SOURCE_DATE_EPOCH should be set to a number of seconds since the epoch, in UTC.
+SOURCE_DATE_TZ will default to UTC but can be set to another timezone and is
+used to determine how the date is displayed (tzselect(1) can be used to
+determine an appropriate value).
 
 Building the Software:
 ======================
Signature · 1 line
2.5.0.rc0
#12 lore raw

RFC PATCH 1/1 Makefile: Add SOURCE_DATE_TZ

Le samedi 01 ao?t 2015 ? 22:32 +1200, Chris Packham a ?crit :

Quoted 8 lines

Along with SOURCE_DATE_EPOCH SOURCE_DATE_TZ can be used to recreate a build with a specific date timestamp. This allows the verification of source supplied with a pre-compiled binary.

If SOURCE_DATE_EPOCH is supplied SOURCE_DATE_TZ can be used to specify what will appear in the output of the version command. If SOURCE_DATE_TZ is not specified UTC will be used. SOURCE_DATE_TZ on it's own will not have an affect.

Well, I worked with the assumption that SOURCE_DATE_EPOCH would always be provided in UTC, but I see no harm in providing SOURCE_DATE_TZ as well, provided that it falls back to UTC when not set, as is the current behaviour of tour patch.

Looks good to me!

Quoted 39 lines

---
This is a quick proof of concept that allows some control of how the timezone is displayed.

 Makefile | 7 ++++---
 README   | 9 ++++++---
 2 files changed, 10 insertions(+), 6 deletions(-)
Makefile +4 -3
index f75c730..5102a9c 100644
@@ -1232,9 +1232,10 @@ endef
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		SOURCE_TZ="$${SOURCE_DATE_TZ:-UTC}"; \
+		TZ="$${SOURCE_TZ}" LC_ALL=C date -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		TZ="$${SOURCE_TZ}" LC_ALL=C date -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		TZ="$${SOURCE_TZ}" LC_ALL=C date -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
README +1 -3
index 1bcb63c..d2e3e94 100644
@@ -5087,11 +5087,14 @@ Reproducible builds
 In order to achieve reproducible builds, timestamps used in the U-Boot build
 process have to be set to a fixed value.
 
-This is done using the SOURCE_DATE_EPOCH environment variable.
-SOURCE_DATE_EPOCH is to be set on the build host's shell, not as a configuration
-option for U-Boot or an environment variable in U-Boot.
+This is done using the SOURCE_DATE_EPOCH and SOURCE_DATE_TZ environment

README Reproducible builds · line 5100

Quoted 10 lines
README +5 -0
+variables. These are to be set on the build host's shell, not as a
+configuration option for U-Boot or an environment variable in U-Boot.
 
 SOURCE_DATE_EPOCH should be set to a number of seconds since the epoch, in UTC.
+SOURCE_DATE_TZ will default to UTC but can be set to another timezone and is
+used to determine how the date is displayed (tzselect(1) can be used to
+determine an appropriate value).
 
 Building the Software:
 ======================

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150801/ae46b64a/attachment.sig>

#13 lore raw

RFC PATCH 1/1 Makefile: Add SOURCE_DATE_TZ

Quoted 10 lines · Paul Kocialkowski

Le samedi 01 ao?t 2015 ? 22:32 +1200, Chris Packham a ?crit :

Quoted 8 lines

Along with SOURCE_DATE_EPOCH SOURCE_DATE_TZ can be used to recreate a build with a specific date timestamp. This allows the verification of source supplied with a pre-compiled binary.

If SOURCE_DATE_EPOCH is supplied SOURCE_DATE_TZ can be used to specify what will appear in the output of the version command. If SOURCE_DATE_TZ is not specified UTC will be used. SOURCE_DATE_TZ on it's own will not have an affect.

Quoted 5 lines

Well, I worked with the assumption that SOURCE_DATE_EPOCH would always be provided in UTC, but I see no harm in providing SOURCE_DATE_TZ as well, provided that it falls back to UTC when not set, as is the current behaviour of tour patch.

To clarify, SOURCE_DATE_EPOCH is a unix timestamp which is defined as the number of seconds (excluding leap seconds) since Jan 1 1970 UTC. There is no way to specify this "in another timezone"; there is no ambiguity here.

However, I am not sure that us at Reproducible Builds will actually adopt this variable. We haven't talked about it beyond my previous email [1], it was just me quickly skimming off my thoughts and firing off quick ideas. My personal concern about SOURCE_DATE_TZ is that it implies that it could take actual named time zones, like "EST" or "Europe/London"; it is **extremely unlikely** that we will do anything like this soon, because this would involve timezone databases and complex things like that. This is why in my previous email I suggested the SOURCE_DATE_TZOFFSET variable. Also, the TZ variable as specified by POSIX specifies the offset in the *opposite* direction to what ISO8601 and RFC2822 displays.

The git internal timestamp format only supports timezone offsets in the common direction, opposite to TZ, i.e. positive for east of Greenwich and negative for west.

I'd suggest to leave out SOURCE_DATE_TZ for now, until we come up with a more precisely defined *meaning* for this variable. It is meant to be a standard across all tools, so some time to think through the issues is necessary. Otherwise we risk leaving a clusterfuck of a legacy, like the POSIX time functions.

X

[1] https://lists.alioth.debian.org/pipermail/reproducible-builds/Week-of-Mon-20150727/002562.html or http://lists.denx.de/pipermail/u-boot/2015-July/221301.html

Signature · 10 lines
GPG: 4096R/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150802/4bc9a798/attachment.sig>
#14 lore raw

RFC PATCH 1/1 Makefile: Add SOURCE_DATE_TZ

Quoted 1 line · Ximin Luo

However, I am not sure that us at Reproducible Builds will actually adopt this variable. We haven't talked about it beyond my previous email [1], it was just me quickly skimming off my thoughts and firing off quick ideas.

Whoops, I mean the _TZ variable. The _EPOCH variable has been decided and we're trying to get tools to support this.

Signature · 10 lines
GPG: 4096R/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150802/3e352a9f/attachment.sig>
#15 lore raw

[U-Boot] [U-Boot,RFC] Makefile: Add SOURCE_DATE_TZ

In reply to Chris Packham
Quoted 9 lines · Chris Packham

Along with SOURCE_DATE_EPOCH SOURCE_DATE_TZ can be used to recreate a build with a specific date timestamp. This allows the verification of source supplied with a pre-compiled binary.

If SOURCE_DATE_EPOCH is supplied SOURCE_DATE_TZ can be used to specify what will appear in the output of the version command. If SOURCE_DATE_TZ is not specified UTC will be used. SOURCE_DATE_TZ on it's own will not have an affect.

Quoted 4 lines

---
This is a quick proof of concept that allows some control of how the timezone is displayed.

Can you please re-submit this, and the related EPOC patches as a series together? Thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150812/2f249189/attachment.sig>
#16 lore raw

[U-Boot] [U-Boot,RFC] Makefile: Add SOURCE_DATE_TZ

Quoted 11 lines · Tom Rini
Quoted 9 lines · Chris Packham

Along with SOURCE_DATE_EPOCH SOURCE_DATE_TZ can be used to recreate a build with a specific date timestamp. This allows the verification of source supplied with a pre-compiled binary.

If SOURCE_DATE_EPOCH is supplied SOURCE_DATE_TZ can be used to specify what will appear in the output of the version command. If SOURCE_DATE_TZ is not specified UTC will be used. SOURCE_DATE_TZ on it's own will not have an affect.

Quoted 8 lines
Quoted 4 lines

---
This is a quick proof of concept that allows some control of how the timezone is displayed.

Can you please re-submit this, and the related EPOC patches as a series together? Thanks!

Hi Tom,

Will do although Ximin Luo had some concerns[1]. The code works but there may (will?) be some churn in the naming/concept for the TZ part.

--
[1] - http://marc.info/?l=u-boot&m=143851366020937&w=2

#17 lore raw

PATCH v2 1/1 Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

In reply to Bin Meng
Quoted 17 lines · Bin Meng

This commit breaks the following commit:

commit f3f431a712729a1af94d01bd1bfde17a252ff02c

Author: Chris Packham <[email protected]>
Date:   Sun May 10 21:02:09 2015 +1200

Makefile: Add U_BOOT_TZ and include in version

Define U_BOOT_TZ alongside U_BOOT_TIME and U_BOOT_DATE and use it to include the timezone in the version output.

Before this commit I have:
U-Boot 2015.07-00345-g9c57487 (Jul 31 2015 - 10:49:31 +0800)

Quoted 7 lines

After this commit I have:
U-Boot 2015.07-00346-gf3f431a (Jul 31 2015 - 02:50:54 +0000)

As you see: the timezone information is missing, and U-Boot's timestamp is now GMT+0 (the correct one should be GMT+8)

Is this intended behavior? Or if not, please fix it.

I notice the same behavior here and I agree it would be nice to fix this.

Thanks

#18 lore raw

PATCH v2 1/1 Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

In reply to Tom Rini
Quoted 9 lines · Tom Rini
Quoted 7 lines · Paul Kocialkowski

In order to achieve reproducible builds in U-Boot, timestamps that are defined at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment variable allows setting a fixed value for those timestamps.

Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be built reproducibly. This is the case for e.g. sunxi devices.

Quoted 6 lines
Quoted 4 lines

However, some other devices might need some more tweaks, especially regarding the image generation tools.

Applied to u-boot/master, thanks!

This commit breaks build on non GNU hosts (like OS X and persumably other *BSD hosts). Before, those hosts where supported, so for me this has to be fixed for 2015.10

We need a) some mechanism to search for the GNU date variant or b) some wrapper to provide the correct output on those host machines.

I vote for a), it is acceptable to have the GNU date available but we should error on 'no GNU date available'. Furthermore we need to have the date command exchangeable by e.g. gdate, gnudate, ... maybe with full path.

Andreas

#19 lore raw

PATCH v2 1/1 Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

Quoted 20 lines · Andreas Bie?mann
Quoted 8 lines · Tom Rini
Quoted 7 lines · Paul Kocialkowski

In order to achieve reproducible builds in U-Boot, timestamps that are defined at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment variable allows setting a fixed value for those timestamps.

Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be built reproducibly. This is the case for e.g. sunxi devices.

Quoted 6 lines
Quoted 4 lines

However, some other devices might need some more tweaks, especially regarding the image generation tools.

Applied to u-boot/master, thanks!

This commit breaks build on non GNU hosts (like OS X and persumably other *BSD hosts). Before, those hosts where supported, so for me this has to be fixed for 2015.10

Quoted 6 lines

We need a) some mechanism to search for the GNU date variant or b) some wrapper to provide the correct output on those host machines.

I vote for a), it is acceptable to have the GNU date available but we should error on 'no GNU date available'. Furthermore we need to have the date command exchangeable by e.g. gdate, gnudate, ... maybe with full path.

There was a proposed patch which only uses the GNU date extensions if SOURCE_DATE_EPOCH environment variable is set, would this sufficiently address your concerns, at least for the short term?

Message-Id: <[email protected]>
http://lists.denx.de/pipermail/u-boot/2015-August/221429.html

live well,
vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150825/928dc33c/attachment.sig>

#20 lore raw

PATCH v2 1/1 Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

Quoted 28 lines · Vagrant Cascadian
Quoted 20 lines · Andreas Bie?mann
Quoted 8 lines · Tom Rini
Quoted 7 lines · Paul Kocialkowski

In order to achieve reproducible builds in U-Boot, timestamps that are defined at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment variable allows setting a fixed value for those timestamps.

Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be built reproducibly. This is the case for e.g. sunxi devices.

Quoted 6 lines
Quoted 4 lines

However, some other devices might need some more tweaks, especially regarding the image generation tools.

Applied to u-boot/master, thanks!

This commit breaks build on non GNU hosts (like OS X and persumably other *BSD hosts). Before, those hosts where supported, so for me this has to be fixed for 2015.10

Quoted 6 lines

We need a) some mechanism to search for the GNU date variant or b) some wrapper to provide the correct output on those host machines.

I vote for a), it is acceptable to have the GNU date available but we should error on 'no GNU date available'. Furthermore we need to have the date command exchangeable by e.g. gdate, gnudate, ... maybe with full path.

Quoted 6 lines

There was a proposed patch which only uses the GNU date extensions if SOURCE_DATE_EPOCH environment variable is set, would this sufficiently address your concerns, at least for the short term?

Message-Id: <[email protected]>
http://lists.denx.de/pipermail/u-boot/2015-August/221429.html

thanks for the pointer, normal builds work with that change.

Andreas

#21 lore raw

PATCH v2 1/1 Reproducible U-Boot build support, using SOURCE_DATE_EPOCH

Le mardi 25 ao?t 2015 ? 12:20 +0200, Andreas Bie?mann a ?crit :

Quoted 29 lines · Vagrant Cascadian
Quoted 28 lines · Andreas Bie?mann
Quoted 26 lines · Tom Rini
Quoted 8 lines · Paul Kocialkowski
Quoted 7 lines

In order to achieve reproducible builds in U-Boot, timestamps that are defined at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment variable allows setting a fixed value for those timestamps.

Simply by setting SOURCE_DATE_EPOCH to a fixed value, a number of targets can be built reproducibly. This is the case for e.g. sunxi devices.

Quoted 6 lines
Quoted 4 lines

However, some other devices might need some more tweaks, especially regarding the image generation tools.

Applied to u-boot/master, thanks!

This commit breaks build on non GNU hosts (like OS X and persumably other *BSD hosts). Before, those hosts where supported, so for me this has to be fixed for 2015.10

We need a) some mechanism to search for the GNU date variant or b) some wrapper to provide the correct output on those host machines.

I vote for a), it is acceptable to have the GNU date available but we should error on 'no GNU date available'. Furthermore we need to have the date command exchangeable by e.g. gdate, gnudate, ... maybe with full path.

Quoted 8 lines
Quoted 6 lines

There was a proposed patch which only uses the GNU date extensions if SOURCE_DATE_EPOCH environment variable is set, would this sufficiently address your concerns, at least for the short term?

Message-Id: <[email protected]>
http://lists.denx.de/pipermail/u-boot/2015-August/221429.html

thanks for the pointer, normal builds work with that change.

I think we should get that patch merged so that it doesn't affect regular builds on non-GNU hosts. It is also relevant for the purpose it serves initially, too.

Signature · 16 lines
Paul Kocialkowski, Replicant developer

Replicant is a fully free Android distribution running on several
devices, a free software mobile operating system putting the emphasis on
freedom and privacy/security.

Website: http://www.replicant.us/
Blog: http://blog.replicant.us/
Wiki/tracker/forums: http://redmine.replicant.us/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150825/2d93b03d/attachment.sig>
#22 lore raw

[U-Boot] SOURCE_DATE_EPOCH must not be linux only... (was Re: [Reproducible-builds] [U-Boot, v2] Reproducible U-Boot build support, using SOURCE_DATE_EPOCH)

In reply to Vagrant Cascadian

Hi,

(just stating the (IMHO) obvious here?)

Quoted 3 lines · Vagrant Cascadian

There was a proposed patch which only uses the GNU date extensions if SOURCE_DATE_EPOCH environment variable is set, would this sufficiently address your concerns, at least for the short term?

there also should be a solution which works on BSD in the not so long term if we want the SOURCE_DATE_EPOCH specification to take off..!

(and this probably also affects other software where we want to (or did) introduce SOURCE_DATE_EPOCH!)

cheers,
Holger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150825/225583c0/attachment.sig>

#23 lore raw

RFC PATCH 1/1 Makefile: search for GNU date

In reply to Andreas Bießmann

The SOURCE_DATE_EPOCH mechanism for reproducible builds requires the GNU variant of date. Respect this and search it, error on missing GNU date.

---
This commit tries to figure out if we have a GNU date variant available. It errors on missing GNU date when it is required (for SOURCE_DATE_EPOCH set). The result is:

---8<---
abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make SOURCE_DATE_EPOCH="date -R" O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h make[1]: Entering directory '/tmp/picosam'
/home/abiessmann/src/u-boot/Makefile:1303: *** "No GNU date found". Stop.
make[1]: Leaving directory '/tmp/picosam'
Makefile:146: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2
abiessmann@punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h make[1]: Entering directory '/tmp/picosam'
CHK include/generated/timestamp_autogenerated.h
UPD include/generated/timestamp_autogenerated.h
make[1]: Leaving directory '/tmp/picosam'
--->8---

It applies on top of http://patchwork.ozlabs.org/patch/506856/ (Makefile: Use correct timezone for U_BOOT_TZ).

 Makefile | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
Makefile +11 -3
index b9b2375..95eae64 100644
@@ -346,6 +346,9 @@ PERL		= perl
 PYTHON		= python
 DTC		= dtc
 CHECK		= sparse
+GNUDATE	       := $(foreach date,gdate date.gnu date, \
+		    $(shell _date=`which $(date)`; \
+		      $${_date} --version 2> /dev/null | $(AWK) "/GNU coreutils/ { print \"$${_date}\"; }"))
 
 CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
 		  -Wbitwise -Wno-return-void -D__CHECK_ENDIAN__ $(CF)
@@ -1281,9 +1284,9 @@ endef
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		LC_ALL=C $(GNUDATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C $(GNUDATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C $(GNUDATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
@@ -1295,6 +1298,11 @@ $(version_h): include/config/uboot.release FORCE
 	$(call filechk,version.h)
 
 $(timestamp_h): $(srctree)/Makefile FORCE
+ifneq ($(strip $(SOURCE_DATE_EPOCH)),)
+ifeq ($(strip $(GNUDATE)),)
+	$(error "No GNU date found")
+endif
+endif
 	$(call filechk,timestamp.h)
 
 # ---------------------------------------------------------------------------
Signature · 1 line
2.1.4
#24 lore raw

RFC PATCH 1/1 Makefile: search for GNU date

Quoted 2 lines · Andreas Bie?mann

The SOURCE_DATE_EPOCH mechanism for reproducible builds requires the GNU variant of date.

Why does it require the GNU date ?

Quoted 1 line

Respect this and search it, error on missing GNU date.

Wouldn't it make more sense to fix the code to NOT depend on GNU extensions ?

Quoted 1 line

[...]

Best regards,
Marek Vasut

#25 lore raw

RFC PATCH 1/1 Makefile: search for GNU date

Hi Marek,

Quoted 5 lines · Marek Vasut
Quoted 2 lines · Andreas Bie?mann

The SOURCE_DATE_EPOCH mechanism for reproducible builds requires the GNU variant of date.

Why does it require the GNU date ?

Cause of the -d switch. It is used in GNU date variants to set the time to print the string from. On the other hand *BSD variants use it to set the DST flag.

Quoted 3 lines
Quoted 1 line

Respect this and search it, error on missing GNU date.

Wouldn't it make more sense to fix the code to NOT depend on GNU extensions ?

That would be great, but so we need to check for the tool and use the right switches depending on that.
I think forcing the user to provide the right tool here is Ok.

Best regards

Andreas

#26 lore raw

RFC PATCH 1/1 Makefile: search for GNU date

Quoted 1 line · Andreas Bie?mann

Hi Marek,

Hi!

Quoted 10 lines · Marek Vasut
Quoted 5 lines · Andreas Bie?mann
Quoted 2 lines

The SOURCE_DATE_EPOCH mechanism for reproducible builds requires the GNU variant of date.

Why does it require the GNU date ?

Cause of the -d switch. It is used in GNU date variants to set the time to print the string from. On the other hand *BSD variants use it to set the DST flag.

This sort of stuff should be in the commit message :)

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Respect this and search it, error on missing GNU date.

Wouldn't it make more sense to fix the code to NOT depend on GNU extensions ?

That would be great, but so we need to check for the tool and use the right switches depending on that.
I think forcing the user to provide the right tool here is Ok.

I think we don't have a better option here anyway, right ? Maybe try checking uname ? But you can have linux with bsd userspace ... and probably BSD with GNU userspace, meh.

Best regards,
Marek Vasut

#27 lore raw

RFC PATCH 1/1 Makefile: search for GNU date

In reply to Andreas Bießmann

Le jeudi 27 ao?t 2015 ? 10:13 +0200, Andreas Bie?mann a ?crit :

Quoted 2 lines

The SOURCE_DATE_EPOCH mechanism for reproducible builds requires the GNU variant of date. Respect this and search it, error on missing GNU date.

Well, IMHO we shouldn't check for GNU date but for the extensions it implements. Those could be included in other date implementations. For instance, it looks like busybox's date implements the -u and -d options too.

Quoted 5 lines

---
This commit tries to figure out if we have a GNU date variant available. It errors on missing GNU date when it is required (for SOURCE_DATE_EPOCH set). The result is:

Also, I don't think it should be up to the Makefile to figure out which date binary to use. It should be up to the user to put the right one as "date", e.g. with an alias. That is, unless the name for GNU date is something really standardized on non-GNU systems.

Quoted 24 lines

---8<---
abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make SOURCE_DATE_EPOCH="date -R" O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h make[1]: Entering directory '/tmp/picosam'
/home/abiessmann/src/u-boot/Makefile:1303: *** "No GNU date found". Stop.
make[1]: Leaving directory '/tmp/picosam'
Makefile:146: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2
abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h make[1]: Entering directory '/tmp/picosam'
CHK include/generated/timestamp_autogenerated.h
UPD include/generated/timestamp_autogenerated.h
make[1]: Leaving directory '/tmp/picosam'
--->8---

It applies on top of http://patchwork.ozlabs.org/patch/506856/ (Makefile: Use correct timezone for U_BOOT_TZ).

 Makefile | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
Makefile +0 -0
index b9b2375..95eae64 100644

Makefile PERL = perl · line 351

Quoted 6 lines
Makefile +3 -0
@@ -346,6 +346,9 @@ PERL		= perl
 PYTHON		= python
 DTC		= dtc
 CHECK		= sparse
+GNUDATE	       := $(foreach date,gdate date.gnu date, \
+		    $(shell _date=`which $(date)`; \
+		      $${_date} --version 2> /dev/null | $(AWK) "/GNU coreutils/ { print \"$${_date}\"; }"))

I advise running date with the needed options (-u -d) and checked the error code instead of this. And of course, the variable names shouldn't mention GNU date.

Quoted 17 lines
Makefile +3 -3
 CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
 		  -Wbitwise -Wno-return-void -D__CHECK_ENDIAN__ $(CF)
@@ -1281,9 +1284,9 @@ endef
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		LC_ALL=C $(GNUDATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C $(GNUDATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C $(GNUDATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
@@ -1295,6 +1298,11 @@ $(version_h): include/config/uboot.release FORCE
 	$(call filechk,version.h)

Makefile $(version_h): include/config/uboot.release FORCE · line 1308

Quoted 10 lines
Makefile +5 -0
 
 $(timestamp_h): $(srctree)/Makefile FORCE
+ifneq ($(strip $(SOURCE_DATE_EPOCH)),)
+ifeq ($(strip $(GNUDATE)),)
+	$(error "No GNU date found")
+endif
+endif
 	$(call filechk,timestamp.h)
 
 # ---------------------------------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150827/c052cc01/attachment.sig>

#28 lore raw

RFC PATCH 1/1 Makefile: search for GNU date

Quoted 8 lines · Paul Kocialkowski

Le jeudi 27 ao?t 2015 ? 10:13 +0200, Andreas Bie?mann a ?crit :

Quoted 2 lines

The SOURCE_DATE_EPOCH mechanism for reproducible builds requires the GNU variant of date. Respect this and search it, error on missing GNU date.

Well, IMHO we shouldn't check for GNU date but for the extensions it implements. Those could be included in other date implementations. For instance, it looks like busybox's date implements the -u and -d options too.

But BSD date used the -d switch to set the timezone [1] ...

---8<---
-d dst Set the kernel's value for daylight saving time. If dst
is non-zero, future calls to gettimeofday(2) will return
a non-zero for tz_dsttime.
--->8---

And it is the same for OS X [2].

Quoted 10 lines
Quoted 5 lines

---
This commit tries to figure out if we have a GNU date variant available. It errors on missing GNU date when it is required (for SOURCE_DATE_EPOCH set). The result is:

Also, I don't think it should be up to the Makefile to figure out which date binary to use. It should be up to the user to put the right one as "date", e.g. with an alias. That is, unless the name for GNU date is something really standardized on non-GNU systems.

Most *BSD systems use gdate or date.gnu for the GNU variant of date, if available. The date tool available is the one of the *BSD distributiuon. HAving a shell alias for date covering gdate is for some users not an option. So it has to be some make variable. For me it would be neat to have the make find some possible solution (e.g. search for the GNU variant on systems which have it).

Quoted 7 lines
Quoted 7 lines
Makefile +0 -0
index b9b2375..95eae64 100644
@@ -346,6 +346,9 @@ PERL		= perl
 PYTHON		= python
 DTC		= dtc
Quoted 8 lines

Makefile PERL = perl · line 351

Quoted 4 lines
Makefile +3 -0
 CHECK		= sparse
+GNUDATE	       := $(foreach date,gdate date.gnu date, \
+		    $(shell _date=`which $(date)`; \
+		      $${_date} --version 2> /dev/null | $(AWK) "/GNU coreutils/ { print \"$${_date}\"; }"))

I advise running date with the needed options (-u -d) and checked the error code instead of this. And of course, the variable names shouldn't mention GNU date.

Good point, let's see how my OS X date behaves with a set '-d' switch.

Andreas

[1] http://www.freebsd.org/cgi/man.cgi?query=date
[2]
https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/date.1.html

#29 lore raw

RFC PATCH v2 1/1 Makefile: search for GNU date

In reply to Andreas Bießmann

The SOURCE_DATE_EPOCH mechanism for reproducible builds require some date with -u and -d switch to print the date string of another time. In other words it requires some date that behaves like the GNU date.

Respect this and search a working date, error on no working version.

---
This commit tries to figure out if we have a date variant available that supports the '-u' and '-d "@0"' switches. It errors on non-working variants of date.

To respect *BSD host systems search for gdate and date.gnu. Those pre- and suffixes are widespread used for the GNU variant of a tool also avialable on *BSD systems.

The result is:

---8<---
abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h make[1]: Entering directory '/tmp/picosam'
CHK include/generated/timestamp_autogenerated.h
UPD include/generated/timestamp_autogenerated.h
make[1]: Leaving directory '/tmp/picosam'
abiessmann@punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam SOURCE_DATE_EPOCH="0" ARCH=arm include/generated/timestamp_autogenerated.h make[1]: Entering directory '/tmp/picosam'
/home/abiessmann/src/u-boot/Makefile:1304: *** "Your gdate/date.gnu/date does not support the '-u' and '-d' switches like GNU date does!". Stop. make[1]: Leaving directory '/tmp/picosam'
Makefile:146: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2
--->8---

It applies on top of http://patchwork.ozlabs.org/patch/506856/ (Makefile: Use correct timezone for U_BOOT_TZ).

Changes in v2:
* check for '-u' and '-d "@0"' switch rather than for the GNU variant

 Makefile | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
Makefile +12 -3
index b9b2375..b797e38 100644
@@ -346,6 +346,10 @@ PERL		= perl
 PYTHON		= python
 DTC		= dtc
 CHECK		= sparse
+DATE	       := $(foreach date,gdate date.gnu date, \
+		    $(shell _date=`which $(date)`; \
+		      $${_date} -u -d "@0" >/dev/null 2>&1; \
+		      test $$? -eq 0 && echo $${_date}))
 
 CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
 		  -Wbitwise -Wno-return-void -D__CHECK_ENDIAN__ $(CF)
@@ -1281,9 +1285,9 @@ endef
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
@@ -1295,6 +1299,11 @@ $(version_h): include/config/uboot.release FORCE
 	$(call filechk,version.h)
 
 $(timestamp_h): $(srctree)/Makefile FORCE
+ifneq ($(strip $(SOURCE_DATE_EPOCH)),)
+ifeq ($(strip $(DATE)),)
+	$(error "Your gdate/date.gnu/date does not support the '-u' and '-d' switches like GNU date does!")
+endif
+endif
 	$(call filechk,timestamp.h)
 
 # ---------------------------------------------------------------------------
Signature · 1 line
2.1.4
#30 lore raw

RFC PATCH v2 1/1 Makefile: search for GNU date

Le jeudi 27 ao?t 2015 ? 11:30 +0200, Andreas Bie?mann a ?crit :

Quoted 5 lines

The SOURCE_DATE_EPOCH mechanism for reproducible builds require some date with -u and -d switch to print the date string of another time. In other words it requires some date that behaves like the GNU date.

Respect this and search a working date, error on no working version.

Looks good to me, except one nitpick, see below.

Quoted 9 lines

---
This commit tries to figure out if we have a date variant available that supports the '-u' and '-d "@0"' switches. It errors on non-working variants of date.

To respect *BSD host systems search for gdate and date.gnu. Those pre- and suffixes are widespread used for the GNU variant of a tool also avialable on *BSD systems.

Fair enough then, that's fine with me.

Quoted 29 lines

The result is:

---8<---
abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h make[1]: Entering directory '/tmp/picosam'
CHK include/generated/timestamp_autogenerated.h
UPD include/generated/timestamp_autogenerated.h
make[1]: Leaving directory '/tmp/picosam'
abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam SOURCE_DATE_EPOCH="0" ARCH=arm include/generated/timestamp_autogenerated.h make[1]: Entering directory '/tmp/picosam'
/home/abiessmann/src/u-boot/Makefile:1304: *** "Your gdate/date.gnu/date does not support the '-u' and '-d' switches like GNU date does!". Stop. make[1]: Leaving directory '/tmp/picosam'
Makefile:146: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2
--->8---

It applies on top of http://patchwork.ozlabs.org/patch/506856/ (Makefile: Use correct timezone for U_BOOT_TZ).

Changes in v2:
* check for '-u' and '-d "@0"' switch rather than for the GNU variant

 Makefile | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
Makefile +0 -0
index b9b2375..b797e38 100644

Makefile PERL = perl · line 352

Quoted 7 lines
Makefile +4 -0
@@ -346,6 +346,10 @@ PERL		= perl
 PYTHON		= python
 DTC		= dtc
 CHECK		= sparse
+DATE	       := $(foreach date,gdate date.gnu date, \
+		    $(shell _date=`which $(date)`; \
+		      $${_date} -u -d "@0" >/dev/null 2>&1; \
+		      test $$? -eq 0 && echo $${_date}))

First, I don't understand why you need to call date with the full path:
if which can find it, then calling the binary without its full path should do just as well, right?

Then, correct me if I'm wrong, but calling test and using && is overkill, you could simply do: $${_date} -u -d "@0" >/dev/null 2>&1 && echo $${_date}

So in the end, the whole line would look like:

DATE	       := $(foreach date,gdate date.gnu date, \
		    $($${date} -u -d "@0" >/dev/null 2>&1 \
		      && echo $${date}))

Let me know what you think (and please test it as well).

Quoted 17 lines
Makefile +3 -3
 CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
 		  -Wbitwise -Wno-return-void -D__CHECK_ENDIAN__ $(CF)
@@ -1281,9 +1285,9 @@ endef
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
@@ -1295,6 +1299,11 @@ $(version_h): include/config/uboot.release FORCE
 	$(call filechk,version.h)

Makefile $(version_h): include/config/uboot.release FORCE · line 1309

Quoted 10 lines
Makefile +5 -0
 
 $(timestamp_h): $(srctree)/Makefile FORCE
+ifneq ($(strip $(SOURCE_DATE_EPOCH)),)
+ifeq ($(strip $(DATE)),)
+	$(error "Your gdate/date.gnu/date does not support the '-u' and '-d' switches like GNU date does!")
+endif
+endif
 	$(call filechk,timestamp.h)
 
 # ---------------------------------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150827/06986ace/attachment.sig>

#31 lore raw

RFC PATCH v2 1/1 Makefile: search for GNU date

Quoted 9 lines · Paul Kocialkowski

Le jeudi 27 ao?t 2015 ? 11:30 +0200, Andreas Bie?mann a ?crit :

Quoted 6 lines

Changes in v2:
* check for '-u' and '-d "@0"' switch rather than for the GNU variant

 Makefile | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
Quoted 16 lines

Makefile PERL = perl · line 352

Quoted 12 lines
Makefile +4 -0
index b9b2375..b797e38 100644
@@ -346,6 +346,10 @@ PERL		= perl
 PYTHON		= python
 DTC		= dtc
 CHECK		= sparse
+DATE	       := $(foreach date,gdate date.gnu date, \
+		    $(shell _date=`which $(date)`; \
+		      $${_date} -u -d "@0" >/dev/null 2>&1; \
+		      test $$? -eq 0 && echo $${_date}))

First, I don't understand why you need to call date with the full path:
if which can find it, then calling the binary without its full path should do just as well, right?

You'r right.

Quoted 3 lines

Then, correct me if I'm wrong, but calling test and using && is overkill, you could simply do: $${_date} -u -d "@0" >/dev/null 2>&1 && echo $${_date}

Also true.

Quoted 7 lines

So in the end, the whole line would look like:

DATE	       := $(foreach date,gdate date.gnu date, \
		    $($${date} -u -d "@0" >/dev/null 2>&1 \
		      && echo $${date}))

Let me know what you think (and please test it as well).

That should work. I wonder however why we don't include this snippet in the filechk_timestamp.h script directly. In fact the $(DATE) (a date that support -d '@0' switch) is just used there, so why clobber the Makefile with it?

---8<---
define filechk_timestamp.h
(if test -n "$${SOURCE_DATE_EPOCH}"; then \
date=""; \
for _date in gdate date.gnu date; do \

   $${_date} -u -d "$${SOURCE_DATE}" >/dev/null 2>&1 && \
      date="$${_date}"; \
   done; \
   if test -n "$${date}"; then \
    SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
    LC_ALL=C $${date} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b
%d %C%y"'; \
    LC_ALL=C $${date} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
    LC_ALL=C $${date} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
   else \
    return 42; \

fi; \
else \
LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
fi)
endef
--->8---

Quoted 28 lines
Quoted 27 lines
Makefile +8 -3
 CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
 		  -Wbitwise -Wno-return-void -D__CHECK_ENDIAN__ $(CF)
@@ -1281,9 +1285,9 @@ endef
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
@@ -1295,6 +1299,11 @@ $(version_h): include/config/uboot.release FORCE
 	$(call filechk,version.h)
 
 $(timestamp_h): $(srctree)/Makefile FORCE
+ifneq ($(strip $(SOURCE_DATE_EPOCH)),)
+ifeq ($(strip $(DATE)),)
+	$(error "Your gdate/date.gnu/date does not support the '-u' and '-d' switches like GNU date does!")
+endif
+endif
 	$(call filechk,timestamp.h)
 
 # ---------------------------------------------------------------------------
#32 lore raw

RFC PATCH v2 1/1 Makefile: search for GNU date

Le jeudi 27 ao?t 2015 ? 15:52 +0200, Andreas Bie?mann a ?crit :

Quoted 41 lines · Paul Kocialkowski
Quoted 9 lines

Le jeudi 27 ao?t 2015 ? 11:30 +0200, Andreas Bie?mann a ?crit :

Quoted 6 lines

Changes in v2:
* check for '-u' and '-d "@0"' switch rather than for the GNU variant

 Makefile | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
Quoted 16 lines

Makefile PERL = perl · line 352

Quoted 12 lines
Makefile +4 -0
index b9b2375..b797e38 100644
@@ -346,6 +346,10 @@ PERL		= perl
 PYTHON		= python
 DTC		= dtc
 CHECK		= sparse
+DATE	       := $(foreach date,gdate date.gnu date, \
+		    $(shell _date=`which $(date)`; \
+		      $${_date} -u -d "@0" >/dev/null 2>&1; \
+		      test $$? -eq 0 && echo $${_date}))

First, I don't understand why you need to call date with the full path:
if which can find it, then calling the binary without its full path should do just as well, right?

You'r right.

Quoted 3 lines

Then, correct me if I'm wrong, but calling test and using && is overkill, you could simply do: $${_date} -u -d "@0" >/dev/null 2>&1 && echo $${_date}

Also true.

Quoted 6 lines

So in the end, the whole line would look like:

DATE	       := $(foreach date,gdate date.gnu date, \
		    $($${date} -u -d "@0" >/dev/null 2>&1 \
		      && echo $${date}))
Quoted 6 lines
Quoted 1 line

Let me know what you think (and please test it as well).

That should work. I wonder however why we don't include this snippet in the filechk_timestamp.h script directly. In fact the $(DATE) (a date that support -d '@0' switch) is just used there, so why clobber the Makefile with it?

I'm fine with both options, but your suggestion also makes it more obvious why we're doing that check at all.

Quoted 21 lines

---8<---
define filechk_timestamp.h

 (if test -n "$${SOURCE_DATE_EPOCH}"; then \
   date=""; \
   for _date in gdate date.gnu date; do \
   $${_date} -u -d "$${SOURCE_DATE}" >/dev/null 2>&1 && \
      date="$${_date}"; \
   done; \
   if test -n "$${date}"; then \
    SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
    LC_ALL=C $${date} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b
%d %C%y"'; \
    LC_ALL=C $${date} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
    LC_ALL=C $${date} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
   else \
    return 42; \
   fi; \
  else \
   LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
   LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
   LC_ALL=C date +'#define U_BOOT_TZ "%z"'; \
Quoted 34 lines

fi)
endef
--->8---

Quoted 28 lines
Quoted 27 lines
Makefile +8 -3
 CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
 		  -Wbitwise -Wno-return-void -D__CHECK_ENDIAN__ $(CF)
@@ -1281,9 +1285,9 @@ endef
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+		LC_ALL=C $(DATE) -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
@@ -1295,6 +1299,11 @@ $(version_h): include/config/uboot.release FORCE
 	$(call filechk,version.h)
 
 $(timestamp_h): $(srctree)/Makefile FORCE
+ifneq ($(strip $(SOURCE_DATE_EPOCH)),)
+ifeq ($(strip $(DATE)),)
+	$(error "Your gdate/date.gnu/date does not support the '-u' and '-d' switches like GNU date does!")
+endif
+endif
 	$(call filechk,timestamp.h)
 
 # ---------------------------------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150827/bc4b8558/attachment.sig>

#33 lore raw

PATCH v3 1/1 Makefile: fix SOURCE_DATE_EPOCH for *BSD host

In reply to Andreas Bießmann

The SOURCE_DATE_EPOCH mechanism for reproducible builds require some date(1) with -d switch to print the relevant date and time strings of another point of time.

In other words it requires some date(1) that behaves like the GNU date(1) [1]. The BSD date(1) [2] on the other hand has the same switch but with a different meaning.

Respect this and check the date(1) abilities before usage, error on non working version. Use the well known pre- and suffixes for the GNU variant of a tool on *BSD hosts to search for a working date(1) version.

[1] http://man7.org/linux/man-pages/man1/date.1.html [2]
http://www.freebsd.org/cgi/man.cgi?query=date

---
This commit tries to figure out if we have a date variant available that supports the '-u' and '-d "@0"' switches. It errors on non-working variants of date.

To respect *BSD host systems search for gdate and date.gnu. Those pre- and suffixes are widespread used for the GNU variant of a tool also avialable on *BSD systems.

The result is:

---8<---
abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h make[1]: Entering directory '/tmp/picosam'
CHK include/generated/timestamp_autogenerated.h
UPD include/generated/timestamp_autogenerated.h
make[1]: Leaving directory '/tmp/picosam'
abiessmann@punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam SOURCE_DATE_EPOCH="$(date +'%s')" ARCH=arm include/generated/timestamp_autogenerated.h make[1]: Entering directory '/tmp/picosam'
CHK include/generated/timestamp_autogenerated.h
/home/abiessmann/src/u-boot/Makefile:1311: recipe for target 'include/generated/timestamp_autogenerated.h' failed
make[1]: *** [include/generated/timestamp_autogenerated.h] Error 42
make[1]: Leaving directory '/tmp/picosam'
Makefile:146: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2
--->8---

It applies on top of http://patchwork.ozlabs.org/patch/506856/ (Makefile: Use correct timezone for U_BOOT_TZ).

Changes in v3:
* move the check in the timestamp.h generation script
* remove RFC
* reword commit message

Changes in v2:
* check for '-u' and '-d "@0"' switch rather than for the GNU variant

 Makefile | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
Makefile +15 -3
index e8ea71c..81f12b8 100644
@@ -1279,12 +1279,24 @@ define filechk_version.h
 	echo \#define LD_VERSION_STRING \"$$($(LD) --version | head -n 1)\"; )
 endef
 
+# The SOURCE_DATE_EPOCH mechanism requires a date that behaves like GNU date.
+# The BSD date on the other hand behaves different and would produce errors
+# with the misused '-d' switch.  Respect that and search a working date with
+# well known pre- and suffixes for the GNU variant of date.
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		DATE=""; \
+		for date in gdate date.gnu date; do \
+			$${date} -u -d "$${SOURCE_DATE}" >/dev/null 2>&1 && DATE="$${date}"; \
+		done; \
+		if test -n "$${DATE}"; then \
+			LC_ALL=C $${DATE} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+			LC_ALL=C $${DATE} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+			LC_ALL=C $${DATE} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		else \
+			return 42; \
+		fi; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \
Signature · 1 line
2.1.4
#34 lore raw

PATCH v3 1/1 Makefile: fix SOURCE_DATE_EPOCH for *BSD host

Quoted 8 lines · Andreas Bie?mann

The SOURCE_DATE_EPOCH mechanism for reproducible builds require some date(1) with -d switch to print the relevant date and time strings of another point of time.

In other words it requires some date(1) that behaves like the GNU date(1) [1]. The BSD date(1) [2] on the other hand has the same switch but with a different meaning.

Quoted 8 lines

Respect this and check the date(1) abilities before usage, error on non working version. Use the well known pre- and suffixes for the GNU variant of a tool on *BSD hosts to search for a working date(1) version.

[1] http://man7.org/linux/man-pages/man1/date.1.html [2]
http://www.freebsd.org/cgi/man.cgi?query=date

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150828/4758b9aa/attachment.sig>
#35 lore raw

PATCH v3 1/1 Makefile: fix SOURCE_DATE_EPOCH for *BSD host

Hi,

Quoted 1 line · Tom Rini

Applied to u-boot/master, thanks!

yay, glad to see this thread come to a happy ending, both nicely and quickly! ;-) Thanks to everyone involved!

cheers,
Holger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150828/1aa06476/attachment.sig>

#36 lore raw

PATCH v3 1/1 Makefile: fix SOURCE_DATE_EPOCH for *BSD host

In reply to Andreas Bießmann

Le vendredi 28 ao?t 2015 ? 10:29 +0200, Andreas Bie?mann a ?crit :

Quoted 11 lines

The SOURCE_DATE_EPOCH mechanism for reproducible builds require some date(1) with -d switch to print the relevant date and time strings of another point of time.

In other words it requires some date(1) that behaves like the GNU date(1) [1]. The BSD date(1) [2] on the other hand has the same switch but with a different meaning.

Respect this and check the date(1) abilities before usage, error on non working version. Use the well known pre- and suffixes for the GNU variant of a tool on *BSD hosts to search for a working date(1) version.

Looks good to me! Have you actually tried setting SOURCE_DATE_EPOCH to a fixed value, building U-Boot twice and checking that the produced file is the very same?

Quoted 66 lines

[1] http://man7.org/linux/man-pages/man1/date.1.html [2]
http://www.freebsd.org/cgi/man.cgi?query=date

---
This commit tries to figure out if we have a date variant available that supports the '-u' and '-d "@0"' switches. It errors on non-working variants of date.

To respect *BSD host systems search for gdate and date.gnu. Those pre- and suffixes are widespread used for the GNU variant of a tool also avialable on *BSD systems.

The result is:

---8<---
abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam ARCH=arm include/generated/timestamp_autogenerated.h make[1]: Entering directory '/tmp/picosam'
CHK include/generated/timestamp_autogenerated.h
UPD include/generated/timestamp_autogenerated.h
make[1]: Leaving directory '/tmp/picosam'
abiessmann at punisher % PATH=$ARMv5_PATH:$PATH CROSS_COMPILE=arm-v5te-linux-gnueabi- make O=/tmp/picosam SOURCE_DATE_EPOCH="$(date +'%s')" ARCH=arm include/generated/timestamp_autogenerated.h make[1]: Entering directory '/tmp/picosam'
CHK include/generated/timestamp_autogenerated.h
/home/abiessmann/src/u-boot/Makefile:1311: recipe for target 'include/generated/timestamp_autogenerated.h' failed
make[1]: *** [include/generated/timestamp_autogenerated.h] Error 42
make[1]: Leaving directory '/tmp/picosam'
Makefile:146: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2
--->8---

It applies on top of http://patchwork.ozlabs.org/patch/506856/ (Makefile: Use correct timezone for U_BOOT_TZ).

Changes in v3:
* move the check in the timestamp.h generation script
* remove RFC
* reword commit message

Changes in v2:
* check for '-u' and '-d "@0"' switch rather than for the GNU variant

 Makefile | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
Makefile +8 -3
index e8ea71c..81f12b8 100644
@@ -1279,12 +1279,24 @@ define filechk_version.h
 	echo \#define LD_VERSION_STRING \"$$($(LD) --version | head -n 1)\"; )
 endef
 
+# The SOURCE_DATE_EPOCH mechanism requires a date that behaves like GNU date.
+# The BSD date on the other hand behaves different and would produce errors
+# with the misused '-d' switch.  Respect that and search a working date with
+# well known pre- and suffixes for the GNU variant of date.
 define filechk_timestamp.h
 	(if test -n "$${SOURCE_DATE_EPOCH}"; then \
 		SOURCE_DATE="@$${SOURCE_DATE_EPOCH}"; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
-		LC_ALL=C date -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		DATE=""; \
+		for date in gdate date.gnu date; do \
+			$${date} -u -d "$${SOURCE_DATE}" >/dev/null 2>&1 && DATE="$${date}"; \
+		done; \

Makefile define filechk_version.h · line 1302

Quoted 10 lines
Makefile +7 -0
+		if test -n "$${DATE}"; then \
+			LC_ALL=C $${DATE} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_DATE "%b %d %C%y"'; \
+			LC_ALL=C $${DATE} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TIME "%T"'; \
+			LC_ALL=C $${DATE} -u -d "$${SOURCE_DATE}" +'#define U_BOOT_TZ "%z"'; \
+		else \
+			return 42; \
+		fi; \
 	else \
 		LC_ALL=C date +'#define U_BOOT_DATE "%b %d %C%y"'; \
 		LC_ALL=C date +'#define U_BOOT_TIME "%T"'; \

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150901/f5fbd352/attachment.sig>

#37 lore raw

PATCH v3 1/1 Makefile: fix SOURCE_DATE_EPOCH for *BSD host

Quoted 9 lines · Paul Kocialkowski

Le vendredi 28 ao?t 2015 ? 10:29 +0200, Andreas Bie?mann a ?crit :

Quoted 8 lines

The SOURCE_DATE_EPOCH mechanism for reproducible builds require some date(1) with -d switch to print the relevant date and time strings of another point of time.

In other words it requires some date(1) that behaves like the GNU date(1) [1]. The BSD date(1) [2] on the other hand has the same switch but with a different meaning.

Quoted 7 lines
Quoted 3 lines

Respect this and check the date(1) abilities before usage, error on non working version. Use the well known pre- and suffixes for the GNU variant of a tool on *BSD hosts to search for a working date(1) version.

Looks good to me! Have you actually tried setting SOURCE_DATE_EPOCH to a fixed value, building U-Boot twice and checking that the produced file is the very same?

It works:

---8<---
andreas at imac % shasum /tmp/u-boot.img.1441178617 u-boot.img /tmp/u-boot.bin.1441178617 u-boot.bin /tmp/boot.bin.1441178617 boot.bin

1a779ba79efa1874a5b307650392737d861005bb  /tmp/u-boot.img.1441178617
1a779ba79efa1874a5b307650392737d861005bb  u-boot.img
dbe56f02e510ee251a2e5f9f8b6ce430884557bb  /tmp/u-boot.bin.1441178617
dbe56f02e510ee251a2e5f9f8b6ce430884557bb  u-boot.bin
4aa9b3ba9641febae0bfdbb374c54c287b463ac0  /tmp/boot.bin.1441178617
4aa9b3ba9641febae0bfdbb374c54c287b463ac0  boot.bin

andreas at imac % uname -a
Darwin imac 14.5.0 Darwin Kernel Version 14.5.0: Wed Jul 29 02:26:53 PDT 2015; root:xnu-2782.40.9~1/RELEASE_X86_64 x86_64 andreas@imac % git describe
v2015.10-rc2-304-gb7e84c9
--->8---

Both are fresh builds with the same SOURCE_DATE_EPOCH set. However using the time stamp from timestamp_autogenerated.h does _not_ work due to the wrong TZ (+0200 for me, UTC with SOURCE_DATE_EPOCH). But I guess this is a known problem.

Andreas

#38 lore raw
In reply to Paul Kocialkowski
Quoted 3 lines · Paul Kocialkowski

In order to achieve reproducible builds in U-Boot, timestamps that are defined at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment variable allows setting a fixed value for those timestamps.

...

Quoted 2 lines

However, some other devices might need some more tweaks, especially regarding the image generation tools.

With this patch, there is still variation based on timezone in any of the u-boot.img and u-boot-sunxi-with-spl.bin produced in the Debian packages:

https://reproducible.debian.net/rb-pkg/unstable/armhf/u-boot.html

The good news is that all the u-boot.bin targets are produced reproducibly, so here's to progress!

I think the use of "time = mktime(time_universal);" is where the problem lies:

Quoted 33 lines
tools/default_image.c +17 -1
index cf5c0d4..18940af 100644
@@ -96,9 +99,25 @@ static void image_set_header(void *ptr, struct stat *sbuf, int ifd,
 				sizeof(image_header_t)),
 			sbuf->st_size - sizeof(image_header_t));
 
+	source_date_epoch = getenv("SOURCE_DATE_EPOCH");
+	if (source_date_epoch != NULL) {
+		time = (time_t) strtol(source_date_epoch, NULL, 10);
+
+		time_universal = gmtime(&time);
+		if (time_universal == NULL) {
+			fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n",
+				__func__);
+			time = 0;
+		} else {
+			time = mktime(time_universal);
+		}
+	} else {
+		time = sbuf->st_mtime;
+	}
+
 	/* Build new header */
 	image_set_magic(hdr, IH_MAGIC);
-	image_set_time(hdr, sbuf->st_mtime);
+	image_set_time(hdr, time);
 	image_set_size(hdr, sbuf->st_size - sizeof(image_header_t));
 	image_set_load(hdr, params->addr);
 	image_set_ep(hdr, params->ep);
Signature · 1 line
1.9.1

According to the mktime manpage:

The mktime() function converts a broken-down time structure, expressed as local time, to calendar time representation.

So my interpetation is that it's taking the UTC time and converts it into local time using the configured timezone... not sure what would be a viable alternative to mktime.

Running with the TZ=UTC environment variable exported works around the problem; not sure if it would be appropriate to always run with TZ=UTC when SOURCE_DATE_EPOCH is set...

live well,
vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150924/4e92e27e/attachment.sig>

#39 lore raw

Le jeudi 24 septembre 2015 ? 09:05 -0700, Vagrant Cascadian a ?crit :

Quoted 5 lines · Paul Kocialkowski
Quoted 3 lines

In order to achieve reproducible builds in U-Boot, timestamps that are defined at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment variable allows setting a fixed value for those timestamps.

...

Quoted 8 lines
Quoted 2 lines

However, some other devices might need some more tweaks, especially regarding the image generation tools.

With this patch, there is still variation based on timezone in any of the u-boot.img and u-boot-sunxi-with-spl.bin produced in the Debian packages:

https://reproducible.debian.net/rb-pkg/unstable/armhf/u-boot.html

Thanks for reporting this!

Quoted 2 lines

The good news is that all the u-boot.bin targets are produced reproducibly, so here's to progress!

Good, that's a nice first step forward.

Quoted 2 lines

I think the use of "time = mktime(time_universal);" is where the problem lies:

[?]

Quoted 8 lines

According to the mktime manpage:

The mktime() function converts a broken-down time structure, expressed as local time, to calendar time representation.

So my interpetation is that it's taking the UTC time and converts it into local time using the configured timezone... not sure what would be a viable alternative to mktime.

That seems to make sense. Come to think of it, it probably was not necessary to call gmtime in the first place: if SOURCE_DATE_EPOCH is always in UTC, we should be able to stick that as-is in the time variable. At best, gmtime + mktime (assuming mktime working in UTC) would give us back the same timestamp.

What do you think? Please let me know if I'm wrong.

Quoted 3 lines

Running with the TZ=UTC environment variable exported works around the problem; not sure if it would be appropriate to always run with TZ=UTC when SOURCE_DATE_EPOCH is set...

Well that's too much of a workaround to be a reliable solution for the long term, IMHO.

Signature · 17 lines
Paul Kocialkowski, Replicant developer

Replicant is a fully free Android distribution running on several
devices, a free software mobile operating system putting the emphasis on
freedom and privacy/security.

Website: https://www.replicant.us/
Blog: https://blog.replicant.us/
Wiki/tracker/forums: https://redmine.replicant.us/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150928/b0842d2c/attachment.sig>
#40 lore raw
Quoted 15 lines · Paul Kocialkowski

Le jeudi 24 septembre 2015 ? 09:05 -0700, Vagrant Cascadian a ?crit :

Quoted 2 lines

I think the use of "time = mktime(time_universal);" is where the problem lies:

[?]

Quoted 8 lines

According to the mktime manpage:

The mktime() function converts a broken-down time structure, expressed as local time, to calendar time representation.

So my interpetation is that it's taking the UTC time and converts it into local time using the configured timezone... not sure what would be a viable alternative to mktime.

Quoted 7 lines

That seems to make sense. Come to think of it, it probably was not necessary to call gmtime in the first place: if SOURCE_DATE_EPOCH is always in UTC, we should be able to stick that as-is in the time variable. At best, gmtime + mktime (assuming mktime working in UTC) would give us back the same timestamp.

What do you think? Please let me know if I'm wrong.

This patch on top of 2015.10-rc4 seems to resolve the issue for me:

tools/default_image.c +0 -2
Index: u-boot/tools/default_image.c
===================================================================
@@ -108,8 +108,6 @@ static void image_set_header(void *ptr,
 			fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n",
 				__func__);
 			time = 0;
-		} else {
-			time = mktime(time_universal);
 		}
 	} else {
 		time = sbuf->st_mtime;

It still checks for the validity of SOURCE_DATE_EPOCH using gmtime, but doesn't call mktime at all, just re-uses the value set from SOURCE_DATE_EPOCH.

live well,
vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150930/a5900ac9/attachment.sig>

#41 lore raw

Le mercredi 30 septembre 2015 ? 08:50 -0700, Vagrant Cascadian a ?crit :

Quoted 20 lines · Paul Kocialkowski
Quoted 1 line

What do you think? Please let me know if I'm wrong.

This patch on top of 2015.10-rc4 seems to resolve the issue for me:

Index: u-boot/tools/default_image.c
===================================================================

tools/default_image.c +0 -2
@@ -108,8 +108,6 @@ static void image_set_header(void *ptr,
 			fprintf(stderr, "%s: SOURCE_DATE_EPOCH is not valid\n",
 				__func__);
 			time = 0;
-		} else {
-			time = mktime(time_universal);
 		}
 	} else {
 		time = sbuf->st_mtime;
Quoted 3 lines

It still checks for the validity of SOURCE_DATE_EPOCH using gmtime, but doesn't call mktime at all, just re-uses the value set from SOURCE_DATE_EPOCH.

That's a good plan! I guess we should also fully get rid of the time_universal variable and make the check inline:

if (gmtime(&time) == NULL)

and of course drop the else statement.

Would you like to craft that patch for upstream U-Boot? If not, I'd be happy to do it.

Thanks for your work!

Signature · 17 lines
Paul Kocialkowski, Replicant developer

Replicant is a fully free Android distribution running on several
devices, a free software mobile operating system putting the emphasis on
freedom and privacy/security.

Website: https://www.replicant.us/
Blog: https://blog.replicant.us/
Wiki/tracker/forums: https://redmine.replicant.us/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20151002/e45fb4e0/attachment.sig>
#42 lore raw
In reply to Vagrant Cascadian
Quoted 8 lines · On Thu, 24 Sep 2015 09:05:10 -0700 Vagrant Cascadian <[email protected]> wrote:
Quoted 3 lines · Paul Kocialkowski

In order to achieve reproducible builds in U-Boot, timestamps that are defined at build-time have to be somewhat eliminated. The SOURCE_DATE_EPOCH environment variable allows setting a fixed value for those timestamps.

...

Quoted 2 lines

However, some other devices might need some more tweaks, especially regarding the image generation tools.

Quoted 8 lines

With this patch, there is still variation based on timezone in any of the u-boot.img and u-boot-sunxi-with-spl.bin produced in the Debian packages:

https://reproducible.debian.net/rb-pkg/unstable/armhf/u-boot.html

The good news is that all the u-boot.bin targets are produced reproducibly, so here's to progress!

Hello,

I see that Debian is building U-Boot 2015.07 according to the information from that link. It may be a good idea to ensure that the following simple patch is also applied:

    http://git.denx.de/?p=u-boot.git;a=commitdiff;h=bfb05d0187d70274c77d02dc0de5e728e1f8be05

It had been submitted at the day when somebody (probably you) was troubleshooting reproducible build issues and asking questions in the U-Boot IRC channel.

Signature · 2 lines
Best regards,
Siarhei Siamashka