Skip to contentSkip to conversation

dependencies.mk: check for valid host-tar before other host dependencies

PATCH New Peter Korsgaard 2 participants 7 messages Feb 27, 2018 – Apr 10, 2018

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/2 dependencies.mk: check for valid host-tar before other host dependencies

host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ before these in case they are both needed, otherwise the tools will fail to extract.

With the upcoming change to blacklist modern tar versions this situation is likely to trigger more often.

The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework, but that series is a bit too intrusive to add this close to 2018.02, so therefore this hack.

---
Changes since v1:
- Issue not specific to lzip, so put tar in front of everything else

 support/dependencies/dependencies.mk | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
support/dependencies/dependencies.mk +4 -1
index 4f606f849d..3fc235863c 100644
@@ -12,7 +12,10 @@
 define suitable-host-package
 $(shell support/dependencies/check-host-$(1).sh $(2))
 endef
--include $(sort $(wildcard support/dependencies/check-host-*.mk))
+# host utilities needs host-tar to extract the source code tarballs, so
+# ensure check-host-tar.mk is included before the rest
+include support/dependencies/check-host-tar.mk
+-include $(sort $(filter-out %-tar.mk,$(wildcard support/dependencies/check-host-*.mk)))
 
 ifeq ($(BR2_CCACHE),y)
 DEPENDENCIES_HOST_PREREQ += host-ccache
Signature · 1 line
2.11.0
#1 lore raw

PATCH v2 1/2 dependencies.mk: check for valid host-tar before other host dependencies

Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Peter" == Peter Korsgaard <[email protected]> writes:

host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ before these in case they are both needed, otherwise the tools will fail to extract.

With the upcoming change to blacklist modern tar versions this situation is likely to trigger more often.

Quoted 8 lines

The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework, but that series is a bit too intrusive to add this close to 2018.02, so therefore this hack.

---
Changes since v1:
- Issue not specific to lzip, so put tar in front of everything else

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#2 lore raw

PATCH v2 1/2 dependencies.mk: check for valid host-tar before other host dependencies

In reply to Peter Korsgaard

All,

Quoted 5 lines · Peter Korsgaard

host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ before these in case they are both needed, otherwise the tools will fail to extract.

Quoted 8 lines

With the upcoming change to blacklist modern tar versions this situation is likely to trigger more often.

The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework, but that series is a bit too intrusive to add this close to 2018.02, so therefore this hack.

#3 lore raw

PATCH v2 1/2 dependencies.mk: check for valid host-tar before other host dependencies

In reply to Peter Korsgaard
Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Peter" == Peter Korsgaard <[email protected]> writes:

host-{cmake,lzip,xz} needs host-tar to extract their source code tarball, so we need to ensure that host-tar gets added to DEPENDENCIES_HOST_PREREQ before these in case they are both needed, otherwise the tools will fail to extract.

With the upcoming change to blacklist modern tar versions this situation is likely to trigger more often.

Quoted 8 lines

The real solution to this issue is the <foo>_EXTRACT_DEPENDENCIES rework, but that series is a bit too intrusive to add this close to 2018.02, so therefore this hack.

---
Changes since v1:
- Issue not specific to lzip, so put tar in front of everything else

Committed to 2017.02.x, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#4 lore raw

PATCH v2 2/2 check-host-tar.sh: blacklist tar 1.30+

In reply to Peter Korsgaard

Tar 1.30 changed the --numeric-owner output for filenames > 100 characters, leading to hash mismatches for the tar archives we create ourselves from git. This is really a fix for a bug in earlier tar versions regarding deterministic output, so it is unlikely to be reverted in later versions.

To work around this issue, blacklist tar 1.30+ similar to how we do it for pre-1.17 versions so Buildroot falls back to building host-tar.

---
Changes since v1:
- None

 support/dependencies/check-host-tar.sh | 33 ++++++++++++++++++++++++---------
 1 file changed, 24 insertions(+), 9 deletions(-)
support/dependencies/check-host-tar.sh +24 -9
index 932d3c4fb2..2143877524 100755
@@ -30,13 +30,28 @@ fi
 # containing hard-links if the --strip-components option is used).
 major_min=1
 minor_min=17
-if [ $major -gt $major_min ]; then
-	echo $tar
-else
-	if [ $major -eq $major_min -a $minor -ge $minor_min ]; then
-		echo $tar
-	else
-		# echo nothing: no suitable tar found
-		exit 1
-	fi
+
+# Maximal version = 1.29 (1.30 changed --numeric-owner output for
+# filenames > 100 characters). This is really a fix for a bug in
+# earlier tar versions regarding deterministic output so it is
+# unlikely to be reverted in later versions.
+major_max=1
+minor_max=29
+
+if [ $major -lt $major_min -o $major -gt $major_max ]; then
+	# echo nothing: no suitable tar found
+	exit 1
 fi
+
+if [ $major -eq $major_min -a $minor -lt $minor_min ]; then
+	# echo nothing: no suitable tar found
+	exit 1
+fi
+
+if [ $major -eq $major_max -a $minor -gt $minor_max ]; then
+	# echo nothing: no suitable tar found
+	exit 1
+fi
+
+# valid
+echo $tar
Signature · 1 line
2.11.0
#5 lore raw

PATCH v2 2/2 check-host-tar.sh: blacklist tar 1.30+

Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Peter" == Peter Korsgaard <[email protected]> writes:

Tar 1.30 changed the --numeric-owner output for filenames > 100 characters, leading to hash mismatches for the tar archives we create ourselves from git. This is really a fix for a bug in earlier tar versions regarding deterministic output, so it is unlikely to be reverted in later versions.

Quoted 7 lines

To work around this issue, blacklist tar 1.30+ similar to how we do it for pre-1.17 versions so Buildroot falls back to building host-tar.

---
Changes since v1:
- None

Committed after adding a link to
http://lists.busybox.net/pipermail/buildroot/2018-January/211222.html
which explains the issue in more details, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#6 lore raw

PATCH v2 2/2 check-host-tar.sh: blacklist tar 1.30+

In reply to Peter Korsgaard
Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Peter" == Peter Korsgaard <[email protected]> writes:

Tar 1.30 changed the --numeric-owner output for filenames > 100 characters, leading to hash mismatches for the tar archives we create ourselves from git. This is really a fix for a bug in earlier tar versions regarding deterministic output, so it is unlikely to be reverted in later versions.

To work around this issue, blacklist tar 1.30+ similar to how we do it for pre-1.17 versions so Buildroot falls back to building host-tar.

Quoted 7 lines

---
Changes since v1:
- None

 support/dependencies/check-host-tar.sh | 33 ++++++++++++++++++++++++---------
 1 file changed, 24 insertions(+), 9 deletions(-)

Committed to 2017.02.x, thanks.

Signature · 1 line
Bye, Peter Korsgaard