package/cpio: add host version
PATCH 1/2 package/cpio: add host version
From: "Yann E. MORIN" <[email protected]>
The latest cpio has a --reproducible option, which may come handy when we try to, well, be reproducible...
---
Changes v1 -> v2:
- Add patch from Yann to my series
--- package/cpio/cpio.mk | 1 + 1 file changed, 1 insertion(+)
package/cpio/cpio.mk +1 -0
@@ -17,3 +17,4 @@ CPIO_DEPENDENCIES += argp-standalone endif $(eval $(autotools-package)) +$(eval $(host-autotools-package))
Signature · 1 line
2.22.0
PATCH 2/2 fs/cpio: make cpio rootfs reproducible
Pass the recommended argument in the CPIO manual to make cpio archives reproducible.
Reference: https://www.gnu.org/software/cpio/manual/cpio.html#Copy_002dpass-mode
Pre-patch diffoscope output: https://gitlab.com/snippets/1874745
Post-patch: https://gitlab.com/snippets/1874746
We can see that post-patch, the archive related differences are removed. The differences are arising from utils/bin/getconf. This will have to be investigated further. However, that is unrelated to cpio.
---
Changes v1 -> v2:
- Added comment
- Build host-cpio if we are reproducible, reason is --reproducible option does not exist prior to cpio 2.12. (suggested by Yann, Arnout)
--- fs/cpio/cpio.mk | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
fs/cpio/cpio.mk +9 -1
@@ -29,8 +29,16 @@ endif # BR2_ROOTFS_DEVICE_CREATION_STATIC ROOTFS_CPIO_PRE_GEN_HOOKS += ROOTFS_CPIO_ADD_INIT +# --repoducible option was not introduced until cpio v2.12. Some distros +# like Ubuntu 16.04 or Debian oldstable which package v2.11. Hence build +# host-cpio. +ifeq ($(BR2_REPRODUCIBLE), y) +ROOTFS_CPIO_DEPENDENCIES += host-cpio +CPIO_OPTS = --reproducible +endif + define ROOTFS_CPIO_CMD - cd $(TARGET_DIR) && find . | cpio --quiet -o -H newc > $@ + cd $(TARGET_DIR) && find . | cpio $(CPIO_OPTS) --quiet -o -H newc > $@ endef ifeq ($(BR2_TARGET_ROOTFS_CPIO_UIMAGE),y)
Signature · 1 line
2.22.0
PATCH 2/2 fs/cpio: make cpio rootfs reproducible
Atharva, All,
On 2019-07-14 21:53 +0530, Atharva Lele spake thusly:
Quoted 27 lines
Pass the recommended argument in the CPIO manual to make cpio archives reproducible.
Reference: https://www.gnu.org/software/cpio/manual/cpio.html#Copy_002dpass-mode
Pre-patch diffoscope output: https://gitlab.com/snippets/1874745
Post-patch: https://gitlab.com/snippets/1874746We can see that post-patch, the archive related differences are removed. The differences are arising from utils/bin/getconf. This will have to be investigated further. However, that is unrelated to cpio.
Signed-off-by Atharva Lele <[email protected]>---
Changes v1 -> v2:
- Added comment
- Build host-cpio if we are reproducible, reason is --reproducible option does not exist prior to cpio 2.12. (suggested by Yann, Arnout)--- fs/cpio/cpio.mk | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)fs/cpio/cpio.mk +0
-0
fs/cpio/cpio.mk endif # BR2_ROOTFS_DEVICE_CREATION_STATIC · line 37
Quoted 9 lines
fs/cpio/cpio.mk +6
-0@@ -29,8 +29,16 @@ endif # BR2_ROOTFS_DEVICE_CREATION_STATIC ROOTFS_CPIO_PRE_GEN_HOOKS += ROOTFS_CPIO_ADD_INIT +# --repoducible option was not introduced until cpio v2.12. Some distros +# like Ubuntu 16.04 or Debian oldstable which package v2.11. Hence build +# host-cpio. +ifeq ($(BR2_REPRODUCIBLE), y) +ROOTFS_CPIO_DEPENDENCIES += host-cpio +CPIO_OPTS = --reproducible
Having a pure assignment in a condition is risky for the future. If we were to later add new unconditional options, we would add them before this part (as is customary to do: uncoditional stuff goes before conditional stuff). In this case, the assignment here would override the uncoditional options, which is not nice.
Instead, I think we prefer an append-assignment in conditional blocks, even if the variable is not already defined.
Arnout, Thomas, can you confirm/infirm my opinion? ;-)
Otherwise:
Regards,
Yann E. MORIN.
Quoted 15 lines
fs/cpio/cpio.mk +3
-1+endif + define ROOTFS_CPIO_CMD - cd $(TARGET_DIR) && find . | cpio --quiet -o -H newc > $@ + cd $(TARGET_DIR) && find . | cpio $(CPIO_OPTS) --quiet -o -H newc > $@ endef ifeq ($(BR2_TARGET_ROOTFS_CPIO_UIMAGE),y)Signature · 6 lines
2.22.0 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 2/2 fs/cpio: make cpio rootfs reproducible
Atharva, All,
On 2019-07-14 18:47 +0200, Yann E. MORIN spake thusly:
Quoted 3 lines
On 2019-07-14 21:53 +0530, Atharva Lele spake thusly:
Quoted 2 lines
Pass the recommended argument in the CPIO manual to make cpio archives reproducible.
[--SNIP--]
Quoted 6 lines
fs/cpio/cpio.mk endif # BR2_ROOTFS_DEVICE_CREATION_STATIC · line 37
Quoted 6 lines
fs/cpio/cpio.mk +6
-0+# --repoducible option was not introduced until cpio v2.12. Some distros +# like Ubuntu 16.04 or Debian oldstable which package v2.11. Hence build +# host-cpio. +ifeq ($(BR2_REPRODUCIBLE), y) +ROOTFS_CPIO_DEPENDENCIES += host-cpio +CPIO_OPTS = --reproducible
In addition to my previous comments: the variable shall be prefixed with ROOTFS_CPIO_, not just with CPIO_, otherwise it could clash with variables from the cpio package itself.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 2/2 fs/cpio: make cpio rootfs reproducible
Yann, all:
Replying to both of Yann's emails in this one:
Quoted 8 lines · IST Yann E. MORIN
Having a pure assignment in a condition is risky for the future. If we were to later add new unconditional options, we would add them before this part (as is customary to do: uncoditional stuff goes before conditional stuff). In this case, the assignment here would override the uncoditional options, which is not nice.
Instead, I think we prefer an append-assignment in conditional blocks, even if the variable is not already defined.
Ah yes that makes sense. I'll change it to an append-assignment in v2. Thanks!
Quoted 9 lines · IST Yann E. MORIN
Atharva, All,
On 2019-07-14 18:47 +0200, Yann E. MORIN spake thusly:
Quoted 3 lines
On 2019-07-14 21:53 +0530, Atharva Lele spake thusly:
Quoted 2 lines
Pass the recommended argument in the CPIO manual to make cpio archives reproducible.
[--SNIP--]
Quoted 12 lines
Quoted 8 lines
fs/cpio/cpio.mk endif # BR2_ROOTFS_DEVICE_CREATION_STATIC · line 37
Quoted 8 lines
+# --repoducible option was not introduced until cpio v2.12. Some distros
+# like Ubuntu 16.04 or Debian oldstable which package v2.11. Hence buildfs/cpio/cpio.mk +4
-0+# host-cpio. +ifeq ($(BR2_REPRODUCIBLE), y) +ROOTFS_CPIO_DEPENDENCIES += host-cpio +CPIO_OPTS = --reproducibleIn addition to my previous comments: the variable shall be prefixed with
ROOTFS_CPIO_, not just withCPIO_, otherwise it could clash with variables from the cpio package itself.
Agreed, fixing it in v2.
Quoted 2 lines
Regards,
Yann E. MORIN.
Thanks for the review!
Regards,
Atharva Lele
PATCH 1/2 package/cpio: add host version
In reply to Atharva LeleAtharva, All,
On 2019-07-14 21:53 +0530, Atharva Lele spake thusly:
Quoted 8 lines
From: "Yann E. MORIN" <[email protected]>
The latest cpio has a --reproducible option, which may come handy when we try to, well, be reproducible...
Reported-by Arnout Vandecappelle <[email protected]>Signed-off-by Yann E. MORIN <[email protected]>[Atharva: Made change pointed out by Arnout: remove HOST_CPIO_CONF_OPTS]
Actually, this is not very intereesting, because it does not say what was in HOST_CPIO_CONF_OPTS. What we need to know is that we do not need to force --bindir. So, I'd have written:
[Atharva: don't force --bindir, as noticed by Arnout]
Quoted 3 lines
Signed-off-by Atharva Lele <[email protected]>Cc Arnout Vandecappelle <[email protected]>Cc Atharva Lele <[email protected]>
Except for the minor nit above:
Thanks! :-)
Regards,
Yann E. MORIN.
Quoted 23 lines
---
Changes v1 -> v2:
- Add patch from Yann to my series--- package/cpio/cpio.mk | 1 + 1 file changed, 1 insertion(+)package/cpio/cpio.mk +1
-0@@ -17,3 +17,4 @@ CPIO_DEPENDENCIES += argp-standalone endif $(eval $(autotools-package)) +$(eval $(host-autotools-package))Signature · 6 lines
2.22.0 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'