everywhere: use $(TAR) not tar (branch yem/tarTAR)
v2 cover everywhere: use $(TAR) not tar (branch yem/tarTAR)
Hello All!
To allow for reproducible archives, and to ensure we can extract all archives, we require that we use tar 1.35 or above.
There are then two cases:
1. the system has a suitable tar: we use that;
2. the system has no suitable tar: we build our own host-tar, and use that (it ends up in HOST_DIR/bin so is in the PATH).
However, to avoid building host-tar again and again, people can set the TAR environment variable, to point to a tar version they believe is suitable (if it is not, we fallback to case 2, above). If the user provides a suitable tar, then we do not build our own host-tar and use the one provided by the user.
However, there are a few places where we do hard-code calls to 'tar' without considering the one provided by the user. This breaks cases where an unsuitable tar is available on the system, as that unsuitable tar is then used to create and extract tarballs. This breaks generatign tarballs from VCS (git, svn) or from vendoring (cargo, go).
Fix all those cases.
How to find calls to tar:
$ git grep -E '\<tar '
There are a lot of false positive:
- CHANGES
- comments
- docs
- patches
- misc (e.g. a script with a $tar variable)
After this series, we still have a very few locations which directly call to a plain tar:
- Makefile: the 'release' target
- scripts not called as part of the build
- boards readmes
- patches
The first will be addressed in a separate patch, as it is not critical (users do not nromally cut Buildroot releases!). The others can't be changed, as they execute outside Buildroot's control.
Changes v1 -> v2:
- fix apply-pathes (Brandon)
- drop useless tarball-patch for liblockfile
Regards,
Yann E. MORIN.
----------------------------------------------------------------
Yann E. MORIN (10):
package/liblockfile: drop empty patch tarball
fs/tar: use appropriate TAR
package/cvs: use appropriate TAR package/flutter-engine: use appropriate TAR package/pkg-download: export TAR for download backends support/download/helpers: use appropriate TAR support/download/cvs: fix shellcheck support/download/cvs: use appropriate TAR support/scripts/apply-patches: use appropriate TAR package/sunxi-mali-utgard-driver: handle patching the Buildroot way
.checkpackageignore | 1 - fs/tar/tar.mk | 2 +- package/Makefile.in | 2 +- package/cvs/cvs.mk | 2 +- package/flutter-engine/flutter-engine.mk | 1 + package/flutter-engine/gen-tarball | 2 +- package/liblockfile/liblockfile.mk | 1 - package/pkg-download.mk | 4 ++- .../0001-build.sh-do-not-apply-patches.patch | 39 ++++++++++++++++++++++ .../sunxi-mali-utgard-driver.mk | 3 +- support/download/cvs | 5 +-- support/download/helpers | 4 +-- support/scripts/apply-patches.sh | 2 +- 13 files changed, 54 insertions(+), 14 deletions(-) create mode 100644 package/sunxi-mali-utgard-driver/0001-build.sh-do-not-apply-patches.patch
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 1/10 package/liblockfile: drop empty patch tarball
The Debian control aarchive does not contain any patch for liblockfile 1.17; it has had no patch since Debian packaged version 1.16-1.1.
Drop the path tarball now.
--- package/liblockfile/liblockfile.mk | 1 - 1 file changed, 1 deletion(-)
package/liblockfile/liblockfile.mk +0 -1
@@ -7,7 +7,6 @@ LIBLOCKFILE_VERSION = 1.17 LIBLOCKFILE_SOURCE = liblockfile_$(LIBLOCKFILE_VERSION).orig.tar.gz LIBLOCKFILE_SITE = http://snapshot.debian.org/archive/debian/20210128T210947Z/pool/main/libl/liblockfile -LIBLOCKFILE_PATCH = liblockfile_$(LIBLOCKFILE_VERSION)-1.debian.tar.bz2 LIBLOCKFILE_LICENSE = LGPL-2.0+, GPL-2.0+ (dotlockfile) LIBLOCKFILE_LICENSE_FILES = COPYRIGHT licenses/GPL-2 licenses/LGPL-2
Signature · 6 lines
2.45.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 01/10 v2] package/liblockfile: drop empty patch tarball
Quoted 15 lines · Yann E. MORIN
The Debian control aarchive does not contain any patch for liblockfile 1.17; it has had no patch since Debian packaged version 1.16-1.1.
Drop the path tarball now.
Signed-off-by Yann E. MORIN <[email protected]>--- package/liblockfile/liblockfile.mk | 1 - 1 file changed, 1 deletion(-)package/liblockfile/liblockfile.mk +0
-0
package/liblockfile/liblockfile.mk line 10
Quoted 4 lines
package/liblockfile/liblockfile.mk +0
-1@@ -7,7 +7,6 @@ LIBLOCKFILE_VERSION = 1.17 LIBLOCKFILE_SOURCE = liblockfile_$(LIBLOCKFILE_VERSION).orig.tar.gz LIBLOCKFILE_SITE = http://snapshot.debian.org/archive/debian/20210128T210947Z/pool/main/libl/liblockfile -LIBLOCKFILE_PATCH = liblockfile_$(LIBLOCKFILE_VERSION)-1.debian.tar.bz2
I took the liberty of also dropping it from the .hash file :-)
Applied to master, thanks.
Regards,
Arnout
package/liblockfile/liblockfile.mk line 12
Quoted 3 lines
package/liblockfile/liblockfile.mk +0
-0LIBLOCKFILE_LICENSE = LGPL-2.0+, GPL-2.0+ (dotlockfile) LIBLOCKFILE_LICENSE_FILES = COPYRIGHT licenses/GPL-2 licenses/LGPL-2
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 01/10 v2] package/liblockfile: drop empty patch tarball
In reply to Yann E. MORINQuoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
The Debian control aarchive does not contain any patch for liblockfile 1.17; it has had no patch since Debian packaged version 1.16-1.1.
Drop the path tarball now.
Signed-off-by Yann E. MORIN <[email protected]>
Committed to 2024.02.x, thanks.
Signature · 5 lines
Bye, Peter Korsgaard _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 2/10 fs/tar: use appropriate TAR
In reply to Yann E. MORIN--- fs/tar/tar.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/tar/tar.mk +1 -1
@@ -13,7 +13,7 @@ TAR_OPTS += --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0 define ROOTFS_TAR_CMD (cd $(TARGET_DIR); find -print0 | LC_ALL=C sort -z | \ - tar $(TAR_OPTS) -cf $@ --null --xattrs-include='*' --no-recursion -T - --numeric-owner) + $(TAR) $(TAR_OPTS) -cf $@ --null --xattrs-include='*' --no-recursion -T - --numeric-owner) endef $(eval $(rootfs))
Signature · 6 lines
2.45.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 02/10 v2] fs/tar: use appropriate TAR
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by Yann E. MORIN <[email protected]>
Committed to 2024.02.x, thanks.
Quoted 25 lines
--- fs/tar/tar.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/tar/tar.mk +1
-1@@ -13,7 +13,7 @@ TAR_OPTS += --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0 define ROOTFS_TAR_CMD (cd $(TARGET_DIR); find -print0 | LC_ALL=C sort -z | \ - tar $(TAR_OPTS) -cf $@ --null --xattrs-include='*' --no-recursion -T - --numeric-owner) + $(TAR) $(TAR_OPTS) -cf $@ --null --xattrs-include='*' --no-recursion -T - --numeric-owner) endef $(eval $(rootfs))Signature · 6 lines
2.45.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 5 lines
Bye, Peter Korsgaard _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 3/10 package/cvs: use appropriate TAR
In reply to Yann E. MORIN--- package/cvs/cvs.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
package/cvs/cvs.mk +1 -1
@@ -31,7 +31,7 @@ CVS_CONF_OPTS += --with-external-zlib endif define CVS_BZIP_UNPACK - $(BZCAT) $(@D)/cvs-$(CVS_VERSION).tar.bz2 | tar -C $(BUILD_DIR) $(TAR_OPTIONS) - + $(BZCAT) $(@D)/cvs-$(CVS_VERSION).tar.bz2 | $(TAR) -C $(BUILD_DIR) $(TAR_OPTIONS) - rm -f $(@D)/cvs-$(CVS_VERSION).tar.bz2 endef
Signature · 6 lines
2.45.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 03/10 v2] package/cvs: use appropriate TAR
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by Yann E. MORIN <[email protected]>
Committed, thanks.
Signature · 5 lines
Bye, Peter Korsgaard _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 4/10 package/flutter-engine: use appropriate TAR
In reply to Yann E. MORIN--- package/flutter-engine/flutter-engine.mk | 1 + package/flutter-engine/gen-tarball | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
package/flutter-engine/flutter-engine.mk +1 -0
@@ -176,6 +176,7 @@ endif # Generate a tarball if one does not already exist. define FLUTTER_ENGINE_GENERATE_TARBALL + TAR=$(TAR) \ PATH=$(HOST_DIR)/share/depot_tools:$(BR_PATH) \ PYTHONPATH=$(HOST_DIR)/lib/python$(PYTHON3_VERSION_MAJOR) \ PUB_CACHE=$(FLUTTER_SDK_BIN_PUB_CACHE) \
package/flutter-engine/gen-tarball +1 -1
@@ -70,7 +70,7 @@ gen_tarball() { # - it insists on having a full git repositoy, with .git et al., which means # we can't generate a reproducible archive anyway. # So we jsut create a plain tarball. - tar -C "${SCRATCH_DIR}"/src -czf "${TARBALL_NAME}" . + ${TAR} -C "${SCRATCH_DIR}"/src -czf "${TARBALL_NAME}" . mv "${TARBALL_NAME}" "${TARBALL_DL_PATH}" }
Signature · 6 lines
2.45.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 04/10 v2] package/flutter-engine: use appropriate TAR
Quoted 12 lines · Yann E. MORIN
Signed-off-by Yann E. MORIN <[email protected]>Cc Adam Duskett <[email protected]>--- package/flutter-engine/flutter-engine.mk | 1 + package/flutter-engine/gen-tarball | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)package/flutter-engine/flutter-engine.mk +0
-0
package/flutter-engine/flutter-engine.mk endif · line 179
Quoted 4 lines
package/flutter-engine/flutter-engine.mk +1
-0@@ -176,6 +176,7 @@ endif # Generate a tarball if one does not already exist. define FLUTTER_ENGINE_GENERATE_TARBALL + TAR=$(TAR) \
I've taken the liberty to change this to TAR="$(TAR)". I don't know if
make TAR="tar --some-extra-option" is supported everywhere, but it doesn't hurt to not break it here :-)
Applied to master with that fixed, thanks.
Regards,
Arnout
Quoted 8 lines
package/flutter-engine/flutter-engine.mk +0
-0PATH=$(HOST_DIR)/share/depot_tools:$(BR_PATH) \ PYTHONPATH=$(HOST_DIR)/lib/python$(PYTHON3_VERSION_MAJOR) \ PUB_CACHE=$(FLUTTER_SDK_BIN_PUB_CACHE) \package/flutter-engine/gen-tarball +0
-0
package/flutter-engine/gen-tarball gen_tarball() { · line 76
Quoted 8 lines
package/flutter-engine/gen-tarball +1
-1@@ -70,7 +70,7 @@ gen_tarball() { # - it insists on having a full git repositoy, with .git et al., which means # we can't generate a reproducible archive anyway. # So we jsut create a plain tarball. - tar -C "${SCRATCH_DIR}"/src -czf "${TARBALL_NAME}" . + ${TAR} -C "${SCRATCH_DIR}"/src -czf "${TARBALL_NAME}" . mv "${TARBALL_NAME}" "${TARBALL_DL_PATH}" }
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 04/10 v2] package/flutter-engine: use appropriate TAR
In reply to Yann E. MORINQuoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by Yann E. MORIN <[email protected]>Cc Adam Duskett <[email protected]>
Committed to 2024.02.x, thanks.
Signature · 5 lines
Bye, Peter Korsgaard _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 5/10 package/pkg-download: export TAR for download backends
In reply to Yann E. MORIN--- package/pkg-download.mk | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
package/pkg-download.mk +3 -1
@@ -109,7 +109,9 @@ endif define DOWNLOAD $(Q)mkdir -p $($(2)_DL_DIR) - $(Q)$(EXTRA_ENV) $($(2)_DL_ENV) \ + $(Q)$(EXTRA_ENV) \ + $($(2)_DL_ENV) \ + TAR=$(TAR) \ BR_NO_CHECK_HASH_FOR="$(if $(BR2_DOWNLOAD_FORCE_CHECK_HASHES),,$(BR_NO_CHECK_HASH_FOR))" \ flock $($(2)_DL_DIR)/.lock $(DL_WRAPPER) \ -c '$($(2)_DL_VERSION)' \
Signature · 6 lines
2.45.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 05/10 v2] package/pkg-download: export TAR for download backends
Quoted 10 lines · Yann E. MORIN
Signed-off-by Yann E. MORIN <[email protected]>--- package/pkg-download.mk | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)package/pkg-download.mk +0
-0
package/pkg-download.mk endif · line 114
Quoted 7 lines
package/pkg-download.mk +3
-1@@ -109,7 +109,9 @@ endif define DOWNLOAD $(Q)mkdir -p $($(2)_DL_DIR) - $(Q)$(EXTRA_ENV) $($(2)_DL_ENV) \ + $(Q)$(EXTRA_ENV) \ + $($(2)_DL_ENV) \ + TAR=$(TAR) \
Same here, added quotes.
Regards,
Arnout
package/pkg-download.mk endif · line 117
Quoted 3 lines
package/pkg-download.mk +0
-0BR_NO_CHECK_HASH_FOR="$(if $(BR2_DOWNLOAD_FORCE_CHECK_HASHES),,$(BR_NO_CHECK_HASH_FOR))" \ flock $($(2)_DL_DIR)/.lock $(DL_WRAPPER) \ -c '$($(2)_DL_VERSION)' \
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 05/10 v2] package/pkg-download: export TAR for download backends
In reply to Yann E. MORINQuoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by Yann E. MORIN <[email protected]>
Committed to 2024.02.x, thanks.
Signature · 5 lines
Bye, Peter Korsgaard _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 6/10 support/download/helpers: use appropriate TAR
In reply to Yann E. MORIN--- support/download/helpers | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
support/download/helpers +2 -2
@@ -59,7 +59,7 @@ mk_tar_gz() { LC_ALL=C sort <"${tmp}.list" >"${tmp}.sorted" # Create POSIX tarballs, since that's the format the most reproducible - tar cf - --transform="s#^\./#${base_dir}/#S" \ + ${TAR} cf - --transform="s#^\./#${base_dir}/#S" \ --numeric-owner --owner=0 --group=0 --mtime="${date}" \ --format=posix --pax-option="${pax_options}" --mode='go=u,go-w' \ -T "${tmp}.sorted" >"${tmp}.tar"
@@ -78,7 +78,7 @@ post_process_unpack() { local one_file mkdir "${dest}" - tar -C "${dest}" --strip-components=1 -xzf "${tarball}" + ${TAR} -C "${dest}" --strip-components=1 -xzf "${tarball}" one_file="$(find "${dest}" -type f -print0 |LC_ALL=C sort -z |sed 's/\x0.*//')" touch -r "${one_file}" "${dest}.timestamp" }
Signature · 6 lines
2.45.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 06/10 v2] support/download/helpers: use appropriate TAR
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by Yann E. MORIN <[email protected]>
Committed to 2024.02.x, thanks.
Signature · 5 lines
Bye, Peter Korsgaard _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 7/10 support/download/cvs: fix shellcheck
In reply to Yann E. MORIN--- .checkpackageignore | 1 - support/download/cvs | 3 ++- 2 files changed, 2 insertions(+), 2 deletions(-)
.checkpackageignore +0 -1
@@ -1653,7 +1653,6 @@ support/dependencies/check-host-xzcat.sh Shellcheck support/dependencies/dependencies.sh Shellcheck support/download/bzr Shellcheck lib_shellscript.ConsecutiveEmptyLines support/download/cargo-post-process Shellcheck -support/download/cvs Shellcheck support/download/dl-wrapper Shellcheck support/download/file Shellcheck support/download/go-post-process Shellcheck
support/download/cvs +2 -1
@@ -40,12 +40,13 @@ shift $((OPTIND-1)) # Get rid of our options # 10 minutes should do the trick. _cvs() { if [ -z "${quiet}" ]; then - printf '%s ' timeout 10m ${CVS} "${@}"; printf '\n' + printf '%s ' timeout 10m "${CVS}" "${@}"; printf '\n' fi _plain_cvs "$@" } # Note: please keep command below aligned with what is printed above _plain_cvs() { + # shellcheck disable=SC2086 # We really want word splitting in CVS eval timeout 10m ${CVS} "${@}" }
Signature · 6 lines
2.45.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 07/10 v2] support/download/cvs: fix shellcheck
Quoted 25 lines · Yann E. MORIN
Signed-off-by Yann E. MORIN <[email protected]>--- .checkpackageignore | 1 - support/download/cvs | 3 ++- 2 files changed, 2 insertions(+), 2 deletions(-).checkpackageignore +0
-1@@ -1653,7 +1653,6 @@ support/dependencies/check-host-xzcat.sh Shellcheck support/dependencies/dependencies.sh Shellcheck support/download/bzr Shellcheck lib_shellscript.ConsecutiveEmptyLines support/download/cargo-post-process Shellcheck -support/download/cvs Shellcheck support/download/dl-wrapper Shellcheck support/download/file Shellcheck support/download/go-post-process Shellchecksupport/download/cvs +0
-0@@ -40,12 +40,13 @@ shift $((OPTIND-1)) # Get rid of our options # 10 minutes should do the trick. _cvs() {
support/download/cvs shift $((OPTIND-1)) # Get rid of our options · line 50
Quoted 10 lines
support/download/cvs +2
-1if [ -z "${quiet}" ]; then - printf '%s ' timeout 10m ${CVS} "${@}"; printf '\n' + printf '%s ' timeout 10m "${CVS}" "${@}"; printf '\n' fi _plain_cvs "$@" } # Note: please keep command below aligned with what is printed above _plain_cvs() { + # shellcheck disable=SC2086 # We really want word splitting in CVS eval timeout 10m ${CVS} "${@}"
My shellcheck 0.9.0 (compared to the 0.7.1 in our docker image) complains here
that "${@}" is pointless here because of the eval. That seems correct. Shouldn't this be '"${@}"', which also removes the need from quoting in the caller?
Shell is icky...
Anyway, applied (unchanged) to master.
Regards,
Arnout
support/download/cvs shift $((OPTIND-1)) # Get rid of our options · line 52
Quoted 2 lines
support/download/cvs +0
-0}
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 07/10 v2] support/download/cvs: fix shellcheck
In reply to Yann E. MORINQuoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by Yann E. MORIN <[email protected]>
Committed to 2024.02.x, thanks.
Quoted 44 lines
--- .checkpackageignore | 1 - support/download/cvs | 3 ++- 2 files changed, 2 insertions(+), 2 deletions(-).checkpackageignore +0
-1@@ -1653,7 +1653,6 @@ support/dependencies/check-host-xzcat.sh Shellcheck support/dependencies/dependencies.sh Shellcheck support/download/bzr Shellcheck lib_shellscript.ConsecutiveEmptyLines support/download/cargo-post-process Shellcheck -support/download/cvs Shellcheck support/download/dl-wrapper Shellcheck support/download/file Shellcheck support/download/go-post-process Shellchecksupport/download/cvs +2
-1@@ -40,12 +40,13 @@ shift $((OPTIND-1)) # Get rid of our options # 10 minutes should do the trick. _cvs() { if [ -z "${quiet}" ]; then - printf '%s ' timeout 10m ${CVS} "${@}"; printf '\n' + printf '%s ' timeout 10m "${CVS}" "${@}"; printf '\n' fi _plain_cvs "$@" } # Note: please keep command below aligned with what is printed above _plain_cvs() { + # shellcheck disable=SC2086 # We really want word splitting in CVS eval timeout 10m ${CVS} "${@}" }Signature · 6 lines
2.45.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 5 lines
Bye, Peter Korsgaard _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 8/10 support/download/cvs: use appropriate TAR
In reply to Yann E. MORIN--- support/download/cvs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
support/download/cvs +1 -1
@@ -68,4 +68,4 @@ export TZ=UTC _cvs ${quiet} -z3 -d"'${uri}'" \ co "${@}" -d "'${basename}'" ${select} "'${rev}'" -P "'${rawname}'" -tar czf "${output}" "${basename}" +${TAR} czf "${output}" "${basename}"
Signature · 6 lines
2.45.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 08/10 v2] support/download/cvs: use appropriate TAR
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by Yann E. MORIN <[email protected]>
Committed to 2024.02.x, thanks.
Signature · 5 lines
Bye, Peter Korsgaard _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 9/10 support/scripts/apply-patches: use appropriate TAR
In reply to Yann E. MORIN---
Changes v1 -> v2:
- use ${TAR}, not $(TAR) (Brandon)
--- package/Makefile.in | 2 +- support/scripts/apply-patches.sh | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
package/Makefile.in +1 -1
@@ -231,7 +231,7 @@ endif INSTALL := $(shell which install || type -p install) UNZIP := $(shell which unzip || type -p unzip) -q -APPLY_PATCHES = PATH=$(HOST_DIR)/bin:$$PATH support/scripts/apply-patches.sh $(if $(QUIET),-s) +APPLY_PATCHES = TAR=$(TAR) PATH=$(HOST_DIR)/bin:$$PATH support/scripts/apply-patches.sh $(if $(QUIET),-s) HOST_CPPFLAGS = -I$(HOST_DIR)/include HOST_CFLAGS ?= -O2
support/scripts/apply-patches.sh +1 -1
@@ -146,7 +146,7 @@ function scan_patchdir { unpackedarchivedir="$builddir/.patches-$(basename $i)-unpacked" rm -rf "$unpackedarchivedir" 2> /dev/null mkdir "$unpackedarchivedir" - tar -C "$unpackedarchivedir" -xaf "${path}/$i" + ${TAR} -C "$unpackedarchivedir" -xaf "${path}/$i" scan_patchdir "$unpackedarchivedir" else apply_patch "$path" "$i"
Signature · 6 lines
2.45.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [External] [PATCH 09/10 v2] support/scripts/apply-patches: use appropriate TAR
Quoted 12 lines
-----Original Message-----
From: Yann E. MORIN <[email protected]> On Behalf Of Yann E. MORIN
Sent: Wednesday, June 5, 2024 12:53 PM
To: [email protected]
Cc: Yann E. MORIN <[email protected]>; Maier, Brandon L Collins <[email protected]>
Subject: [External] [PATCH 09/10 v2] support/scripts/apply-patches: use appropriate TAR
Quoted 2 lines
Signed-off-by Yann E. MORIN <[email protected]>Cc "Maier, Brandon L Collins" <[email protected]>
LGTM
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 09/10 v2] support/scripts/apply-patches: use appropriate TAR
In reply to Yann E. MORINQuoted 29 lines · Yann E. MORIN
Signed-off-by Yann E. MORIN <[email protected]>Cc "Maier, Brandon L Collins" <[email protected]>---
Changes v1 -> v2:
- use ${TAR}, not $(TAR) (Brandon)--- package/Makefile.in | 2 +- support/scripts/apply-patches.sh | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)package/Makefile.in +1
-1@@ -231,7 +231,7 @@ endif INSTALL := $(shell which install || type -p install) UNZIP := $(shell which unzip || type -p unzip) -q -APPLY_PATCHES = PATH=$(HOST_DIR)/bin:$$PATH support/scripts/apply-patches.sh $(if $(QUIET),-s) +APPLY_PATCHES = TAR=$(TAR) PATH=$(HOST_DIR)/bin:$$PATH support/scripts/apply-patches.sh $(if $(QUIET),-s) HOST_CPPFLAGS = -I$(HOST_DIR)/include HOST_CFLAGS ?= -O2support/scripts/apply-patches.sh +0
-0
support/scripts/apply-patches.sh function scan_patchdir { · line 149
Quoted 5 lines
support/scripts/apply-patches.sh +1
-1@@ -146,7 +146,7 @@ function scan_patchdir { unpackedarchivedir="$builddir/.patches-$(basename $i)-unpacked" rm -rf "$unpackedarchivedir" 2> /dev/null mkdir "$unpackedarchivedir" - tar -C "$unpackedarchivedir" -xaf "${path}/$i" + ${TAR} -C "$unpackedarchivedir" -xaf "${path}/$i"
For this one, there's actually completely no need to use ${TAR}. Still, it's
better for consistency, so I just added a note to the commit message.
Regards,
Arnout
support/scripts/apply-patches.sh function scan_patchdir { · line 152
Quoted 3 lines
support/scripts/apply-patches.sh +0
-0scan_patchdir "$unpackedarchivedir" else apply_patch "$path" "$i"
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 09/10 v2] support/scripts/apply-patches: use appropriate TAR
In reply to Yann E. MORINQuoted 31 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by Yann E. MORIN <[email protected]>Cc "Maier, Brandon L Collins" <[email protected]>---
Changes v1 -> v2:
- use ${TAR}, not $(TAR) (Brandon)--- package/Makefile.in | 2 +- support/scripts/apply-patches.sh | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)package/Makefile.in +1
-1@@ -231,7 +231,7 @@ endif INSTALL := $(shell which install || type -p install) UNZIP := $(shell which unzip || type -p unzip) -q -APPLY_PATCHES = PATH=$(HOST_DIR)/bin:$$PATH support/scripts/apply-patches.sh $(if $(QUIET),-s) +APPLY_PATCHES = TAR=$(TAR) PATH=$(HOST_DIR)/bin:$$PATH support/scripts/apply-patches.sh $(if $(QUIET),-s) HOST_CPPFLAGS = -I$(HOST_DIR)/include HOST_CFLAGS ?= -O2support/scripts/apply-patches.sh +0
-0
support/scripts/apply-patches.sh function scan_patchdir { · line 149
Quoted 5 lines
support/scripts/apply-patches.sh +1
-1@@ -146,7 +146,7 @@ function scan_patchdir { unpackedarchivedir="$builddir/.patches-$(basename $i)-unpacked" rm -rf "$unpackedarchivedir" 2> /dev/null mkdir "$unpackedarchivedir" - tar -C "$unpackedarchivedir" -xaf "${path}/$i" + ${TAR} -C "$unpackedarchivedir" -xaf "${path}/$i"
This means that you can not directly/manually execute apply-patches.sh. I am not sure if there is any use case for doing so, but as it is not really fixing anything I have NOT backported it to 2024.02.x.
Signature · 5 lines
Bye, Peter Korsgaard _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 10/10 package/sunxi-mali-utgard-driver: handle patching the Buildroot way
In reply to Yann E. MORIN--- .../0001-build.sh-do-not-apply-patches.patch | 39 +++++++++++++++++++ .../sunxi-mali-utgard-driver.mk | 3 +- 2 files changed, 40 insertions(+), 2 deletions(-) create mode 100644 package/sunxi-mali-utgard-driver/0001-build.sh-do-not-apply-patches.patch
package/sunxi-mali-utgard-driver/0001-build.sh-do-not-apply-patches.patch new file +39 -0
@@ -0,0 +1,39 @@ +From 079ed8aa57ba6629b992c3d0d01904d3e5170d3b Mon Sep 17 00:00:00 2001 +From: "Yann E. MORIN" <[email protected]> +Date: Wed, 5 Jun 2024 13:06:16 +0200 +Subject: [PATCH] build.sh: do not apply patches + +In Buildroot, we handle applying the patches in a generic way, with +appropriate tools (e.g. with a proper tar). + +Upstream: not applicable +Signed-off-by: Yann E. MORIN <[email protected]> +--- + build.sh | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/build.sh b/build.sh +index d1fedff..26efc53 100755 +--- a/build.sh ++++ b/build.sh +@@ -10,6 +10,8 @@ BUILD_OPTS="USING_UMP=0 + USING_DEVFREQ=1" + + apply_patches() { ++ return 0 # Patching handled by Buildroot ++ + pushd $2 + + quilt push -a +@@ -22,6 +24,8 @@ apply_patches() { + } + + unapply_patches() { ++ return 0 # Patching handled by Buildroot ++ + pushd $2 + + quilt pop -a +-- +2.45.1 +
package/sunxi-mali-utgard-driver/sunxi-mali-utgard-driver.mk +1 -2
@@ -16,8 +16,7 @@ SUNXI_MALI_UTGARD_DRIVER_MAKE_OPTS = \ define SUNXI_MALI_UTGARD_DRIVER_USE_APPLY_PATCHES ln -sf $(SUNXI_MALI_UTGARD_REV)/series $(@D)/patches - $(SED) 's|quilt push -a|$(TOPDIR)/support/scripts/apply-patches.sh . ../patches|' \ - $(@D)/build.sh + $(APPLY_PATCHES) $(@D)/$(SUNXI_MALI_UTGARD_REV) $(@D)/patches endef SUNXI_MALI_UTGARD_DRIVER_POST_PATCH_HOOKS += SUNXI_MALI_UTGARD_DRIVER_USE_APPLY_PATCHES
Signature · 6 lines
2.45.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 10/10 v2] package/sunxi-mali-utgard-driver: handle patching the Buildroot way
Hi Yann,
Quoted 2 lines · Yann E. MORIN
Signed-off-by Yann E. MORIN <[email protected]>Cc Giulio Benetti <[email protected]>
Thank you for the cleaner approach.
Best regards
Signature · 2 lines
Giulio Benetti CEO&CTO@Benetti Engineering
Quoted 56 lines
--- .../0001-build.sh-do-not-apply-patches.patch | 39 +++++++++++++++++++ .../sunxi-mali-utgard-driver.mk | 3 +- 2 files changed, 40 insertions(+), 2 deletions(-)create mode 100644
package/sunxi-mali-utgard-driver/0001-build.sh-do-not-apply-patches.patchpackage/sunxi-mali-utgard-driver/0001-build.sh-do-not-apply-patches.patch new file +39
-0@@ -0,0 +1,39 @@ +From 079ed8aa57ba6629b992c3d0d01904d3e5170d3b Mon Sep 17 00:00:00 2001 +From: "Yann E. MORIN" <[email protected]> +Date: Wed, 5 Jun 2024 13:06:16 +0200 +Subject: [PATCH] build.sh: do not apply patches + +In Buildroot, we handle applying the patches in a generic way, with +appropriate tools (e.g. with a proper tar). + +Upstream: not applicable +Signed-off-by: Yann E. MORIN <[email protected]> +--- + build.sh | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/build.sh b/build.sh +index d1fedff..26efc53 100755 +--- a/build.sh ++++ b/build.sh +@@ -10,6 +10,8 @@ BUILD_OPTS="USING_UMP=0 + USING_DEVFREQ=1" + + apply_patches() { ++ return 0 # Patching handled by Buildroot ++ + pushd $2 + + quilt push -a +@@ -22,6 +24,8 @@ apply_patches() { + } + + unapply_patches() { ++ return 0 # Patching handled by Buildroot ++ + pushd $2 + + quilt pop -a +-- +2.45.1 +package/sunxi-mali-utgard-driver/sunxi-mali-utgard-driver.mk +0
-0
package/sunxi-mali-utgard-driver/sunxi-mali-utgard-driver.mk SUNXI_MALI_UTGARD_DRIVER_MAKE_OPTS = \ · line 22
Quoted 9 lines
package/sunxi-mali-utgard-driver/sunxi-mali-utgard-driver.mk +1
-2@@ -16,8 +16,7 @@ SUNXI_MALI_UTGARD_DRIVER_MAKE_OPTS = \ define SUNXI_MALI_UTGARD_DRIVER_USE_APPLY_PATCHES ln -sf $(SUNXI_MALI_UTGARD_REV)/series $(@D)/patches - $(SED) 's|quilt push -a|$(TOPDIR)/support/scripts/apply-patches.sh . ../patches|' \ - $(@D)/build.sh + $(APPLY_PATCHES) $(@D)/$(SUNXI_MALI_UTGARD_REV) $(@D)/patches endef SUNXI_MALI_UTGARD_DRIVER_POST_PATCH_HOOKS += SUNXI_MALI_UTGARD_DRIVER_USE_APPLY_PATCHES
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH 10/10 v2] package/sunxi-mali-utgard-driver: handle patching the Buildroot way
In reply to Yann E. MORINQuoted 58 lines · Yann E. MORIN
Signed-off-by Yann E. MORIN <[email protected]>Cc Giulio Benetti <[email protected]>--- .../0001-build.sh-do-not-apply-patches.patch | 39 +++++++++++++++++++ .../sunxi-mali-utgard-driver.mk | 3 +- 2 files changed, 40 insertions(+), 2 deletions(-)create mode 100644
package/sunxi-mali-utgard-driver/0001-build.sh-do-not-apply-patches.patchpackage/sunxi-mali-utgard-driver/0001-build.sh-do-not-apply-patches.patch new file +39
-0@@ -0,0 +1,39 @@ +From 079ed8aa57ba6629b992c3d0d01904d3e5170d3b Mon Sep 17 00:00:00 2001 +From: "Yann E. MORIN" <[email protected]> +Date: Wed, 5 Jun 2024 13:06:16 +0200 +Subject: [PATCH] build.sh: do not apply patches + +In Buildroot, we handle applying the patches in a generic way, with +appropriate tools (e.g. with a proper tar). + +Upstream: not applicable +Signed-off-by: Yann E. MORIN <[email protected]> +--- + build.sh | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/build.sh b/build.sh +index d1fedff..26efc53 100755 +--- a/build.sh ++++ b/build.sh +@@ -10,6 +10,8 @@ BUILD_OPTS="USING_UMP=0 + USING_DEVFREQ=1" + + apply_patches() { ++ return 0 # Patching handled by Buildroot ++ + pushd $2 + + quilt push -a +@@ -22,6 +24,8 @@ apply_patches() { + } + + unapply_patches() { ++ return 0 # Patching handled by Buildroot ++ + pushd $2 + + quilt pop -a +-- +2.45.1 +package/sunxi-mali-utgard-driver/sunxi-mali-utgard-driver.mk +0
-0
package/sunxi-mali-utgard-driver/sunxi-mali-utgard-driver.mk SUNXI_MALI_UTGARD_DRIVER_MAKE_OPTS = \ · line 19
Quoted 6 lines
package/sunxi-mali-utgard-driver/sunxi-mali-utgard-driver.mk +1
-2@@ -16,8 +16,7 @@ SUNXI_MALI_UTGARD_DRIVER_MAKE_OPTS = \ define SUNXI_MALI_UTGARD_DRIVER_USE_APPLY_PATCHES ln -sf $(SUNXI_MALI_UTGARD_REV)/series $(@D)/patches - $(SED) 's|quilt push -a|$(TOPDIR)/support/scripts/apply-patches.sh . ../patches|' \ - $(@D)/build.sh + $(APPLY_PATCHES) $(@D)/$(SUNXI_MALI_UTGARD_REV) $(@D)/patches
This looks like just cleanup to me, not fixing anything. Except that it makes
sure that the proper tar is used, but that's not even needed for apply-patches.sh.
So I applied this to next instead of master.
Regards,
Arnout
package/sunxi-mali-utgard-driver/sunxi-mali-utgard-driver.mk SUNXI_MALI_UTGARD_DRIVER_MAKE_OPTS = \ · line 22
Quoted 3 lines
package/sunxi-mali-utgard-driver/sunxi-mali-utgard-driver.mk +0
-0endef SUNXI_MALI_UTGARD_DRIVER_POST_PATCH_HOOKS += SUNXI_MALI_UTGARD_DRIVER_USE_APPLY_PATCHES
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 42 lines · Yann E. MORIN
Hello All!
To allow for reproducible archives, and to ensure we can extract all archives, we require that we use tar 1.35 or above.
There are then two cases:
1. the system has a suitable tar: we use that;
2. the system has no suitable tar: we build our own host-tar, and use that (it ends up in HOST_DIR/bin so is in the PATH).However, to avoid building host-tar again and again, people can set the TAR environment variable, to point to a tar version they believe is suitable (if it is not, we fallback to case 2, above). If the user provides a suitable tar, then we do not build our own host-tar and use the one provided by the user.
However, there are a few places where we do hard-code calls to 'tar' without considering the one provided by the user. This breaks cases where an unsuitable tar is available on the system, as that unsuitable tar is then used to create and extract tarballs. This breaks generatign tarballs from VCS (git, svn) or from vendoring (cargo, go).
Fix all those cases.
How to find calls to tar:
$ git grep -E '\<tar 'There are a lot of false positive:
- CHANGES
- comments
- docs
- patches
- misc (e.g. a script with a $tar variable)After this series, we still have a very few locations which directly call to a plain tar:
-Makefile: the 'release' target
- scripts not called as part of the build
- boards readmes
- patches
Quoted 3 lines
The first will be addressed in a separate patch, as it is not critical (users do not nromally cut Buildroot releases!). The others can't be changed, as they execute outside Buildroot's control.
In case it wasn't obvious: series applied to master, except for the last one
which went to next.
Regards,
Arnout
Quoted 44 lines
Changes v1 -> v2:
- fix apply-pathes (Brandon)
- drop useless tarball-patch for liblockfileRegards,
Yann E. MORIN.----------------------------------------------------------------
Yann E. MORIN (10):
package/liblockfile: drop empty patch tarball
fs/tar: use appropriate TAR
package/cvs: use appropriate TAR package/flutter-engine: use appropriate TAR package/pkg-download: export TAR for download backends support/download/helpers: use appropriate TAR support/download/cvs: fix shellcheck support/download/cvs: use appropriate TAR support/scripts/apply-patches: use appropriate TAR package/sunxi-mali-utgard-driver: handle patching the Buildroot way.checkpackageignore | 1 - fs/tar/tar.mk | 2 +- package/Makefile.in | 2 +- package/cvs/cvs.mk | 2 +- package/flutter-engine/flutter-engine.mk | 1 + package/flutter-engine/gen-tarball | 2 +- package/liblockfile/liblockfile.mk | 1 - package/pkg-download.mk | 4 ++- .../0001-build.sh-do-not-apply-patches.patch | 39 ++++++++++++++++++++++ .../sunxi-mali-utgard-driver.mk | 3 +- support/download/cvs | 5 +-- support/download/helpers | 4 +-- support/scripts/apply-patches.sh | 2 +- 13 files changed, 54 insertions(+), 14 deletions(-)create mode 100644
package/sunxi-mali-utgard-driver/0001-build.sh-do-not-apply-patches.patch--
.-----------------.--------------------.------------------.--------------------.| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
Quoted 5 lines
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot