check-host-tar.sh: bump minimum tar version to 1.27 for reproducible tar files with long paths

PATCH New Peter Korsgaard 2 participants 4 messages Apr 2, 2018 – Apr 8, 2018

lore

Share & tools

lore

Peter Korsgaard

author

2018-04-02 09:48 UTC #0 lore raw

PATCH 1/1 check-host-tar.sh: bump minimum tar version to 1.27 for reproducible tar files with long paths

Fixes:
http://autobuild.buildroot.net/results/b18/b187e64a61918f17f69588e2355a03286bc5808e

tar 1.27 subtly changed the tar format when a GNU long link entry is added (which is done for path elements > 100 characters). The code used to set the permission mode of the link entry to 0:

  header = start_private_header ("././@LongLink", size, time (NULL));
  FILL (header->header.mtime, '0');
  FILL (header->header.mode, '0');
  FILL (header->header.uid, '0');
  FILL (header->header.gid, '0');
  FILL (header->header.devmajor, 0);
  FILL (header->header.devminor, 0);

This got dropped in 1.27 by commit df7b55a8f6354e3 (Fix some problems with negative and out-of-range integers), so the settings from start_private_header() are used directly - Which are:

TIME_TO_CHARS (t < 0 ? 0 : min (t, MAX_OCTAL_VAL (header->header.mtime)),

		 header->header.mtime);

MODE_TO_CHARS (S_IFREG|S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH, header->header.mode); UID_TO_CHARS (0, header->header.uid);
GID_TO_CHARS (0, header->header.gid);

The end result is that tar >= 1.27 sets mode to 644.

The consequence of this is that we create different tar files when long path names are encountered (which often happens when a package downloads a specific sha1 from a git repo) depending on the host tar version used, causing hash mismatches.

As a workaround, bump our minimum tar version to 1.27. It would be nicer to only do this if we have packages from bzr/git/hg enabled, but that is an exercise for later.

Signed-off-by Peter Korsgaard <[email protected]>

---
 support/dependencies/check-host-tar.sh | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

support/dependencies/check-host-tar.sh +4 -3

index 2143877524..0857307396 100755
@@ -26,10 +26,11 @@ if [ ! -z "${version_bsd}" ] ; then
   minor=0
 fi
 
-# Minimal version = 1.17 (previous versions do not correctly unpack archives
-# containing hard-links if the --strip-components option is used).
+# Minimal version = 1.27 (previous versions do not correctly unpack archives
+# containing hard-links if the --strip-components option is used or create
+# different gnu long link headers for path elements > 100 characters).
 major_min=1
-minor_min=17
+minor_min=27
 
 # Maximal version = 1.29 (1.30 changed --numeric-owner output for
 # filenames > 100 characters). This is really a fix for a bug in

Signature · 1 line

2.11.0

Yann E. MORIN

2018-04-02 12:35 UTC #1 lore raw

PATCH 1/1 check-host-tar.sh: bump minimum tar version to 1.27 for reproducible tar files with long paths

Peter, All,

On 2018-04-02 11:48 +0200, Peter Korsgaard spake thusly:

Quoted 32 lines

Fixes:
http://autobuild.buildroot.net/results/b18/b187e64a61918f17f69588e2355a03286bc5808e
tar 1.27 subtly changed the tar format when a GNU long link entry is added (which is done for path elements > 100 characters). The code used to set the permission mode of the link entry to 0:
  header = start_private_header ("././@LongLink", size, time (NULL));
  FILL (header->header.mtime, '0');
  FILL (header->header.mode, '0');
  FILL (header->header.uid, '0');
  FILL (header->header.gid, '0');
  FILL (header->header.devmajor, 0);
  FILL (header->header.devminor, 0);
This got dropped in 1.27 by commit df7b55a8f6354e3 (Fix some problems with negative and out-of-range integers), so the settings from start_private_header() are used directly - Which are:
TIME_TO_CHARS (t < 0 ? 0 : min (t, MAX_OCTAL_VAL (header->header.mtime)),
		 header->header.mtime);
MODE_TO_CHARS (S_IFREG|S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH, header->header.mode); UID_TO_CHARS (0, header->header.uid);
GID_TO_CHARS (0, header->header.gid);
The end result is that tar >= 1.27 sets mode to 644.
The consequence of this is that we create different tar files when long path names are encountered (which often happens when a package downloads a specific sha1 from a git repo) depending on the host tar version used, causing hash mismatches.

Quoted 5 lines

As a workaround, bump our minimum tar version to 1.27. It would be nicer to only do this if we have packages from bzr/git/hg enabled, but that is an exercise for later.
Signed-off-by Peter Korsgaard <[email protected]>

Acked-by "Yann E. MORIN" <[email protected]>

Regards,
Yann E. MORIN.

Quoted 30 lines

---
 support/dependencies/check-host-tar.sh | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

support/dependencies/check-host-tar.sh +4 -3

index 2143877524..0857307396 100755
@@ -26,10 +26,11 @@ if [ ! -z "${version_bsd}" ] ; then
   minor=0
 fi
 
-# Minimal version = 1.17 (previous versions do not correctly unpack archives
-# containing hard-links if the --strip-components option is used).
+# Minimal version = 1.27 (previous versions do not correctly unpack archives
+# containing hard-links if the --strip-components option is used or create
+# different gnu long link headers for path elements > 100 characters).
 major_min=1
-minor_min=17
+minor_min=27
 
 # Maximal version = 1.29 (1.30 changed --numeric-owner output for
 # filenames > 100 characters). This is really a fix for a bug in

Signature · 6 lines

2.11.0

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Signature · 6 lines

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Peter Korsgaard

author

2018-04-02 12:47 UTC #2 lore raw

PATCH 1/1 check-host-tar.sh: bump minimum tar version to 1.27 for reproducible tar files with long paths

In reply to Peter Korsgaard

Quoted 17 lines

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Peter" == Peter Korsgaard <[email protected]> writes:
Fixes:
http://autobuild.buildroot.net/results/b18/b187e64a61918f17f69588e2355a03286bc5808e
tar 1.27 subtly changed the tar format when a GNU long link entry is added (which is done for path elements > 100 characters). The code used to set the permission mode of the link entry to 0:
  header = start_private_header ("././@LongLink", size, time (NULL));
  FILL (header->header.mtime, '0');
  FILL (header->header.mode, '0');
  FILL (header->header.uid, '0');
  FILL (header->header.gid, '0');
  FILL (header->header.devmajor, 0);
  FILL (header->header.devminor, 0);

Quoted 5 lines

This got dropped in 1.27 by commit df7b55a8f6354e3 (Fix some problems with negative and out-of-range integers), so the settings from start_private_header() are used directly - Which are:
TIME_TO_CHARS (t < 0 ? 0 : min (t, MAX_OCTAL_VAL (header->header.mtime)),

header-> header.mtime);

Quoted 11 lines

  MODE_TO_CHARS (S_IFREG|S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH, header->header.mode);
  UID_TO_CHARS (0, header->header.uid);
  GID_TO_CHARS (0, header->header.gid);
The end result is that tar >= 1.27 sets mode to 644.
The consequence of this is that we create different tar files when long path names are encountered (which often happens when a package downloads a specific sha1 from a git repo) depending on the host tar version used, causing hash mismatches.

Quoted 5 lines

As a workaround, bump our minimum tar version to 1.27. It would be nicer to only do this if we have packages from bzr/git/hg enabled, but that is an exercise for later.
Signed-off-by Peter Korsgaard <[email protected]>

Committed, thanks.

Signature · 1 line

Bye, Peter Korsgaard

Peter Korsgaard

author

2018-04-08 19:40 UTC #3 lore raw

PATCH 1/1 check-host-tar.sh: bump minimum tar version to 1.27 for reproducible tar files with long paths

In reply to Peter Korsgaard

Quoted 17 lines

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Peter" == Peter Korsgaard <[email protected]> writes:
Fixes:
http://autobuild.buildroot.net/results/b18/b187e64a61918f17f69588e2355a03286bc5808e
tar 1.27 subtly changed the tar format when a GNU long link entry is added (which is done for path elements > 100 characters). The code used to set the permission mode of the link entry to 0:
  header = start_private_header ("././@LongLink", size, time (NULL));
  FILL (header->header.mtime, '0');
  FILL (header->header.mode, '0');
  FILL (header->header.uid, '0');
  FILL (header->header.gid, '0');
  FILL (header->header.devmajor, 0);
  FILL (header->header.devminor, 0);

Quoted 5 lines

This got dropped in 1.27 by commit df7b55a8f6354e3 (Fix some problems with negative and out-of-range integers), so the settings from start_private_header() are used directly - Which are:
TIME_TO_CHARS (t < 0 ? 0 : min (t, MAX_OCTAL_VAL (header->header.mtime)),

header-> header.mtime);

Quoted 11 lines

  MODE_TO_CHARS (S_IFREG|S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH, header->header.mode);
  UID_TO_CHARS (0, header->header.uid);
  GID_TO_CHARS (0, header->header.gid);
The end result is that tar >= 1.27 sets mode to 644.
The consequence of this is that we create different tar files when long path names are encountered (which often happens when a package downloads a specific sha1 from a git repo) depending on the host tar version used, causing hash mismatches.

Quoted 5 lines

As a workaround, bump our minimum tar version to 1.27. It would be nicer to only do this if we have packages from bzr/git/hg enabled, but that is an exercise for later.
Signed-off-by Peter Korsgaard <[email protected]>

Committed to 2018.02.x, thanks.

Signature · 1 line

Bye, Peter Korsgaard