fs/cpio: make cpio rootfs reproducible
PATCH 1/1 fs/cpio: make cpio rootfs reproducible
Pass the recommended argument in the CPIO manual to make cpio archives reproducible.
Reference: https://www.gnu.org/software/cpio/manual/cpio.html#Copy_002dpass-mode
Pre-patch diffoscope output: https://gitlab.com/snippets/1874745
Post-patch: https://gitlab.com/snippets/1874746
We can see that post-patch, the archive related differences are removed. The differences are arising from utils/bin/getconf. This will have to be investigated further. However, that is unrelated to cpio.
--- fs/cpio/cpio.mk | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
fs/cpio/cpio.mk +5 -1
@@ -29,8 +29,12 @@ endif # BR2_ROOTFS_DEVICE_CREATION_STATIC ROOTFS_CPIO_PRE_GEN_HOOKS += ROOTFS_CPIO_ADD_INIT +ifeq ($(BR2_REPRODUCIBLE), y) +CPIO_OPTS = --reproducible +endif + define ROOTFS_CPIO_CMD - cd $(TARGET_DIR) && find . | cpio --quiet -o -H newc > $@ + cd $(TARGET_DIR) && find . | cpio $(CPIO_OPTS) --quiet -o -H newc > $@ endef ifeq ($(BR2_TARGET_ROOTFS_CPIO_UIMAGE),y)
Signature · 1 line
2.22.0
PATCH 1/1 fs/cpio: make cpio rootfs reproducible
Hi Atharva,
Quoted 22 lines · Atharva Lele
Pass the recommended argument in the CPIO manual to make cpio archives reproducible.
Reference: https://www.gnu.org/software/cpio/manual/cpio.html#Copy_002dpass-mode
Pre-patch diffoscope output: https://gitlab.com/snippets/1874745
Post-patch: https://gitlab.com/snippets/1874746We can see that post-patch, the archive related differences are removed. The differences are arising from utils/bin/getconf. This will have to be investigated further. However, that is unrelated to cpio.
Signed-off-by Atharva Lele <[email protected]>--- fs/cpio/cpio.mk | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)fs/cpio/cpio.mk +0
-0
fs/cpio/cpio.mk endif # BR2_ROOTFS_DEVICE_CREATION_STATIC · line 33
Quoted 5 lines
fs/cpio/cpio.mk +2
-0@@ -29,8 +29,12 @@ endif # BR2_ROOTFS_DEVICE_CREATION_STATIC ROOTFS_CPIO_PRE_GEN_HOOKS += ROOTFS_CPIO_ADD_INIT +ifeq ($(BR2_REPRODUCIBLE), y) +CPIO_OPTS = --reproducible
Unfortunately, this option was only introduced in cpio 2.12 (2015). Older
distros, including Debian oldstable and Ubuntu 2016.04, both of which are still being maintained, have an earlier cpio version (2.11) so don't have this option.
Therefore, I think we have to check for a suitable system cpio and build
host-cpio if it is not suitable. Or maybe, for reproducible, always build host-cpio?
Yann, what do you think?
Otherwise, the patch looks good. But I've marked it as Changes Requested in
pathchwork.
Regards,
Arnout
Quoted 9 lines
fs/cpio/cpio.mk +3
-1+endif + define ROOTFS_CPIO_CMD - cd $(TARGET_DIR) && find . | cpio --quiet -o -H newc > $@ + cd $(TARGET_DIR) && find . | cpio $(CPIO_OPTS) --quiet -o -H newc > $@ endef ifeq ($(BR2_TARGET_ROOTFS_CPIO_UIMAGE),y)
PATCH 1/1 fs/cpio: make cpio rootfs reproducible
Arnout, Atharva, All,
On 2019-07-14 00:34 +0200, Arnout Vandecappelle spake thusly:
Quoted 3 lines · Atharva Lele
Quoted 2 lines
Pass the recommended argument in the CPIO manual to make cpio archives reproducible.
[--SNIP--]
Quoted 10 lines
fs/cpio/cpio.mk endif # BR2_ROOTFS_DEVICE_CREATION_STATIC · line 33
Quoted 2 lines
fs/cpio/cpio.mk +2
-0+ifeq ($(BR2_REPRODUCIBLE), y) +CPIO_OPTS = --reproducibleUnfortunately, this option was only introduced in cpio 2.12 (2015). Older
distros, including Debian oldstable and Ubuntu 2016.04, both of which are still being maintained, have an earlier cpio version (2.11) so don't have this option.Therefore, I think we have to check for a suitable system cpio and build
host-cpio if it is not suitable. Or maybe, for reproducible, always build host-cpio?
If we are reproducible, then always build our cpio version.
We currently do not have a host-cpio package, but it is relatively trivial to add (patch pending). Yet, I tried and it adds about 25s to the build, somewhat comparable to the overhead for host-tar...
If someone gets sufficiently annoyed by the overhead, they'll provide a patch to switch to checking for a suitable cpio.
But in any case, I think that people who do care about reproducibility, do care more about it than they do about time.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'