Skip to contentSkip to conversation

package/ccache: bump version

PATCH Applied Heiko Thiery 2 participants 7 messages Jul 13, 2024 Zstd

lore
Share & tools
lore
#0 lore raw

cover package/ccache: bump version

---- 8< ----
/tmp/ccmQFw5y.s: Assembler messages:
/tmp/ccmQFw5y.s:89386: Error: unsupported instruction `vmovdqu'
/tmp/ccmQFw5y.s:89400: Error: unsupported instruction `vmovdqu'
/tmp/ccmQFw5y.s:89464: Error: unsupported instruction `vmovdqu'
/tmp/ccmQFw5y.s:89518: Error: unsupported instruction `vmovdqu'
/tmp/ccmQFw5y.s:89525: Error: unsupported instruction `vmovdqu'
/tmp/ccmQFw5y.s:123938: Error: unsupported instruction `vmovdqu'
/tmp/ccmQFw5y.s:123953: Error: unsupported instruction `vmovdqu'
make[4]: *** [src/third_party/blake3/CMakeFiles/blake3.dir/build.make:160: src/third_party/blake3/CMakeFiles/blake3.dir/blake3_avx512.c.o] Error 1 make[3]: *** [CMakeFiles/Makefile2:406: src/third_party/blake3/CMakeFiles/blake3.dir/all] Error 2
---- 8< ----

To solve the problem above I started bumping the ccache package to 4.10.1. While doing this I observed the problem that ccache tries to download xxhash by its own and fails.

---- 8< ----
-- Downloading...
dst='/srv/storage/hthiery/buildroot-2/output/build/host-ccache-4.10.1/_deps/xxhash-subbuild/xxhash-populate-prefix/src/v0.8.2.tar.gz' timeout='none'
inactivity timeout='none'
-- Using src='https://github.com/Cyan4973/xxhash/archive/refs/tags/v0.8.2.tar.gz'
CMake Error at xxhash-subbuild/xxhash-populate-prefix/src/xxhash-populate-stamp/download-xxhash-populate.cmake:170 (message):
Each download failed!

error: downloading 'https://github.com/Cyan4973/xxhash/archive/refs/tags/v0.8.2.tar.gz' failed
status_code: 1
status_string: "Unsupported protocol"
log:
--- LOG BEGIN ---
Protocol "https" not supported or disabled in libcurl
Closing connection

          --- LOG END ---

---- 8< ----

So I added support for the host package of xxhash and added this to the ccache dependencies.

In addition the bundled blake3 version seems not to build correctly on all machines. See the first error about 'unsupported instruction'. Therefore the new host package for blake3 is introduced.

v2:
- change subject (fixed naming from ccmake to ccache)

v3:
- fix legal-info for ccache and blake3

v4
- requested changes by Arnout, see history of patches for details

Heiko Thiery (4):
package/blake3: add new host package
package/xxhash: add host support
package/ccache: bump version to 4.10.1 package/pkg-generic.mk: add variable to skip ccache dependency

 DEVELOPERS                   |  1 +
 package/blake3/blake3.hash   |  3 +++
 package/blake3/blake3.mk     | 22 ++++++++++++++++++++++
 package/ccache/ccache.hash   |  4 ++--
 package/ccache/ccache.mk     | 31 +++++++++++++++++++++----------
 package/cmake/cmake.mk       |  3 +++
 package/fakedate/fakedate.mk |  3 +++
 package/hiredis/hiredis.mk   |  3 +++
 package/lzip/lzip.mk         |  3 +++
 package/pkg-generic.mk       | 13 +++++++------
 package/pkgconf/pkgconf.mk   |  3 +++
 package/skeleton/skeleton.mk |  3 +++
 package/tar/tar.mk           |  3 +++
 package/xxhash/xxhash.mk     | 16 ++++++++++++++++
 package/xz/xz.mk             |  3 +++
 package/zstd/zstd.mk         |  3 +++
 16 files changed, 99 insertions(+), 18 deletions(-)
 create mode 100644 package/blake3/blake3.hash
 create mode 100644 package/blake3/blake3.mk
Signature · 6 lines
2.39.2

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw

PATCH v4 1/4 package/blake3: add new host package

This package is used and required for ccache update.

---

v2:
- on change

v3:
- fix BLAKE3_LICENSE_FILES and corresponding hash values

v4:
- use github macro to get tagged version instead of refs/tags
- remove override of CMAKE_ASM_COMPILER (requires series https://patchwork.ozlabs.org/project/buildroot/list/?series=415079)

 DEVELOPERS                 |  1 +
 package/blake3/blake3.hash |  3 +++
 package/blake3/blake3.mk   | 19 +++++++++++++++++++
 3 files changed, 23 insertions(+)
 create mode 100644 package/blake3/blake3.hash
 create mode 100644 package/blake3/blake3.mk
DEVELOPERS +1 -0
index 1fa425f3ee..0222b37efd 100644
@@ -1369,6 +1369,7 @@ F:	configs/kontron_bl_imx8mm_defconfig
 F:	configs/kontron_smarc_sal28_defconfig
 F:	configs/kontron_pitx_imx8m_defconfig
 F:	package/altera-stapl/
+F:	package/blake3/
 F:	package/ipmitool/
 F:	package/libnetconf2/
 F:	package/libyang/
package/blake3/blake3.hash new file +3 -0
new file mode 100644
index 0000000000..2a00d19b02
@@ -0,0 +1,3 @@
+# sha256 computed locally
+sha256  822cd37f70152e5985433d2c50c8f6b2ec83aaf11aa31be9fe71486a91744f37  blake3-1.5.1.tar.gz
+sha256  6a94bedb8b707ed97f6e310d0d015ab14e0683ffa0a612b02958581b9cc9fc0e  LICENSE
package/blake3/blake3.mk new file +19 -0
new file mode 100644
index 0000000000..5625046db6
@@ -0,0 +1,19 @@
+################################################################################
+#
+# blake3
+#
+################################################################################
+
+BLAKE3_VERSION = 1.5.1
+BLAKE3_SITE = $(call github,BLAKE3-team,BLAKE3,$(BLAKE3_VERSION))
+BLAKE3_SUBDIR = c
+BLAKE3_LICENSE = Apache-2.0, CC0-1.0
+BLAKE3_LICENSE_FILES = LICENSE
+
+# We may be a ccache dependency, so we can't use ccache; reset the
+# options set by the cmake infra.
+HOST_BLAKE3_CONF_OPTS += \
+	-UCMAKE_C_COMPILER_LAUNCHER \
+	-UCMAKE_CXX_COMPILER_LAUNCHER
+
+$(eval $(host-cmake-package))
Signature · 6 lines
2.39.2

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#2 lore raw

PATCH v4 2/4 package/xxhash: add host support

In reply to Heiko Thiery

The host support is required for ccache.

---
v4:
- use standard install target for host variant
- fix typo in commit message (ccmake -> ccache)

 package/xxhash/xxhash.mk | 13 +++++++++++++
 1 file changed, 13 insertions(+)
package/xxhash/xxhash.mk +13 -0
index 7708ba1e19..fde8e464f9 100644
@@ -41,4 +41,17 @@ define XXHASH_INSTALL_TARGET_CMDS
 		PREFIX=/usr DESTDIR=$(TARGET_DIR) $(XXHASH_INSTALL_TARGETS)
 endef
 
+# we are a ccache dependency, so we can't use ccache
+HOST_XXHASH_ENV = $(HOST_MAKE_ENV) $(HOST_CONFIGURE_OPTS) CC="$(HOSTCC_NOCCACHE)" CXX="$(HOSTCXX_NOCCACHE)"
+HOST_XXHASH_OPTS += DESTDIR=$(HOST_DIR) PREFIX=/usr
+
+define HOST_XXHASH_BUILD_CMDS
+	$(HOST_XXHASH_ENV) $(MAKE) $(HOST_XXHASH_OPTS) -C $(@D)
+endef
+
+define HOST_XXHASH_INSTALL_CMDS
+	$(HOST_XXHASH_ENV) $(MAKE) $(HOST_XXHASH_OPTS) -C $(@D) install
+endef
+
 $(eval $(generic-package))
+$(eval $(host-generic-package))
Signature · 6 lines
2.39.2

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#3 lore raw

PATCH v4 3/4 package/ccache: bump version to 4.10.1

In reply to Heiko Thiery

https://ccache.dev/releasenotes.html#_ccache_4_10_1

This version requires use the system dependencies of host-blake3 and host-xxhash. The bundled version is not able to build/download.

The ZSTD_FROM_INTERNET and HIREDIS_FROM_INTERNET variables are replaced in the ccache build system with DEP_ZSTD and DEP_HIREDIS in version 4.10. For the sake of completeness, all DEP_<xxx> variables were set for external dependencies.

The generation of documentation has also been switched off.

The LICENSE.adoc was updated regarding the bundled libraries version used like cpp-httplib, BLAKE3, fmt, nonstd-span, cxxurl.

---
v4:
- add note for change reason in LICENCE.adoc

 package/ccache/ccache.hash |  4 ++--
 package/ccache/ccache.mk   | 27 +++++++++++++++++----------
 package/pkg-generic.mk     |  2 +-
 3 files changed, 20 insertions(+), 13 deletions(-)
package/ccache/ccache.hash +2 -2
index a2537077b6..744e167489 100644
@@ -1,4 +1,4 @@
 # sha256 computed locally
-sha256  4c03bc840699127d16c3f0e6112e3f40ce6a230d5873daa78c60a59c7ef59d25  ccache-4.9.1.tar.xz
+sha256  3a43442ce3916ea48bb6ccf6f850891cbff01d1feddff7cd4bbd49c5cf1188f6  ccache-4.10.1.tar.xz
 sha256  80b5112739a423dfac7bed1ca8a1df3cccda3d794425441997d4462b83db4dd5  GPL-3.0.txt
-sha256  a6aea7358a30ded3f8abbff979219bdbe7b7a9aa489a8c70030afd96a83caf84  LICENSE.adoc
+sha256  de4684f5251a0a2c58f3a7ab94d6a8d3443822cbd01abcdadd9250d3d1e0bfe1  LICENSE.adoc
package/ccache/ccache.mk +17 -10
index 0b2c02e8a7..d92608512c 100644
@@ -4,20 +4,27 @@
 #
 ################################################################################
 
-CCACHE_VERSION = 4.9.1
+CCACHE_VERSION = 4.10.1
 CCACHE_SITE = https://github.com/ccache/ccache/releases/download/v$(CCACHE_VERSION)
 CCACHE_SOURCE = ccache-$(CCACHE_VERSION).tar.xz
 CCACHE_LICENSE = GPL-3.0+, others
 CCACHE_LICENSE_FILES = LICENSE.adoc GPL-3.0.txt
-HOST_CCACHE_DEPENDENCIES = host-hiredis host-zstd
+HOST_CCACHE_DEPENDENCIES = host-hiredis host-zstd host-xxhash host-blake3
 
 # We are ccache, so we can't use ccache
 HOST_CCACHE_CONF_OPTS += \
 	-UCMAKE_C_COMPILER_LAUNCHER \
 	-UCMAKE_CXX_COMPILER_LAUNCHER \
 	-DCCACHE_DEV_MODE=OFF \
-	-DZSTD_FROM_INTERNET=OFF \
-	-DHIREDIS_FROM_INTERNET=OFF \
+	-DDEP_BLAKE3=SYSTEM \
+	-DDEP_FMT=BUNDLED \
+	-DDEP_CPPHTTPLIB=BUNDLED \
+	-DDEP_HIREDIS=SYSTEM \
+	-DDEP_NONSTDSPAN=BUNDLED \
+	-DDEP_TLEXPECTED=BUNDLED \
+	-DDEP_XXHASH=SYSTEM \
+	-DDEP_ZSTD=SYSTEM \
+	-DENABLE_DOCUMENTATION=OFF \
 	-DENABLE_TESTING=OFF
 
 # Patch host-ccache as follows:
@@ -32,12 +39,12 @@ HOST_CCACHE_CONF_OPTS += \
 HOST_CCACHE_DEFAULT_CCACHE_DIR = $(patsubst $(HOME)/%,%,$(BR_CACHE_DIR))
 
 define HOST_CCACHE_PATCH_CONFIGURATION
-	sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/src/Config.cpp
-	sed -i 's,".ccache","$(HOST_CCACHE_DEFAULT_CCACHE_DIR)",' $(@D)/src/Config.cpp
-	sed -i 's,"/.cache/ccache","/$(HOST_CCACHE_DEFAULT_CCACHE_DIR)",' $(@D)/src/Config.cpp
-	sed -i 's,"/.config/ccache","/$(HOST_CCACHE_DEFAULT_CCACHE_DIR)",' $(@D)/src/Config.cpp
-	sed -i 's,getenv("XDG_CACHE_HOME"),nullptr,' $(@D)/src/Config.cpp
-	sed -i 's,getenv("XDG_CONFIG_HOME"),nullptr,' $(@D)/src/Config.cpp
+	sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/src/ccache/Config.cpp
+	sed -i 's,".ccache","$(HOST_CCACHE_DEFAULT_CCACHE_DIR)",' $(@D)/src/ccache/Config.cpp
+	sed -i 's,"/.cache/ccache","/$(HOST_CCACHE_DEFAULT_CCACHE_DIR)",' $(@D)/src/ccache/Config.cpp
+	sed -i 's,"/.config/ccache","/$(HOST_CCACHE_DEFAULT_CCACHE_DIR)",' $(@D)/src/ccache/Config.cpp
+	sed -i 's,getenv("XDG_CACHE_HOME"),nullptr,' $(@D)/src/ccache/Config.cpp
+	sed -i 's,getenv("XDG_CONFIG_HOME"),nullptr,' $(@D)/src/ccache/Config.cpp
 endef
 
 HOST_CCACHE_POST_PATCH_HOOKS += HOST_CCACHE_PATCH_CONFIGURATION
package/pkg-generic.mk +1 -1
index e1c16b7343..eeb7531dbb 100644
@@ -793,7 +793,7 @@ $(2)_EXTRACT_DEPENDENCIES += \
 endif
 
 ifeq ($$(BR2_CCACHE),y)
-ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate host-ccache host-cmake host-hiredis host-pkgconf host-zstd,$(1)),)
+ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate host-ccache host-cmake host-hiredis host-pkgconf host-zstd host-xxhash host-blake3,$(1)),)
 $(2)_DEPENDENCIES += host-ccache
 endif
 endif
Signature · 6 lines
2.39.2

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#4 lore raw

PATCH v4 4/4 package/pkg-generic.mk: add variable to skip ccache dependency

In reply to Heiko Thiery

As more and more packages are required to build the host ccache package and therefore they has to be filtered out for ccache dependency the list growed up.

By adding the variable <PKG>_ADD_CCACHE_DEPENDENCY with the default value to 'yes' we can remove that dependency for each package that is needed by ccache by setting the value to 'no'. This can be done in the package makefile instead of tracking the list in the core pkg-generic.mk file.

This behavior is similar to the skeleton and toolchain dependency.

---
v4:
- new commit

 package/blake3/blake3.mk     |  3 +++
 package/ccache/ccache.mk     |  4 ++++
 package/cmake/cmake.mk       |  3 +++
 package/fakedate/fakedate.mk |  3 +++
 package/hiredis/hiredis.mk   |  3 +++
 package/lzip/lzip.mk         |  3 +++
 package/pkg-generic.mk       | 13 +++++++------
 package/pkgconf/pkgconf.mk   |  3 +++
 package/skeleton/skeleton.mk |  3 +++
 package/tar/tar.mk           |  3 +++
 package/xxhash/xxhash.mk     |  3 +++
 package/xz/xz.mk             |  3 +++
 package/zstd/zstd.mk         |  3 +++
 13 files changed, 44 insertions(+), 6 deletions(-)
package/blake3/blake3.mk +3 -0
index 5625046db6..3b1ccb44cc 100644
@@ -10,6 +10,9 @@ BLAKE3_SUBDIR = c
 BLAKE3_LICENSE = Apache-2.0, CC0-1.0
 BLAKE3_LICENSE_FILES = LICENSE
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_BLAKE3_ADD_CCACHE_DEPENDENCY = NO
+
 # We may be a ccache dependency, so we can't use ccache; reset the
 # options set by the cmake infra.
 HOST_BLAKE3_CONF_OPTS += \
package/ccache/ccache.mk +4 -0
index d92608512c..f727ca04bd 100644
@@ -9,6 +9,10 @@ CCACHE_SITE = https://github.com/ccache/ccache/releases/download/v$(CCACHE_VERSI
 CCACHE_SOURCE = ccache-$(CCACHE_VERSION).tar.xz
 CCACHE_LICENSE = GPL-3.0+, others
 CCACHE_LICENSE_FILES = LICENSE.adoc GPL-3.0.txt
+
+# Do not add a recursive dependency to its own
+HOST_CCACHE_ADD_CCACHE_DEPENDENCY = NO
+
 HOST_CCACHE_DEPENDENCIES = host-hiredis host-zstd host-xxhash host-blake3
 
 # We are ccache, so we can't use ccache
package/cmake/cmake.mk +3 -0
index eacecdc0ee..842457eb34 100644
@@ -14,6 +14,9 @@ CMAKE_CPE_ID_VALID = YES
 # Tool download MITM attack warning if using npm package to install cmake
 CMAKE_IGNORE_CVES = CVE-2016-10642
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_CMAKE_ADD_CCACHE_DEPENDENCY = NO
+
 # CMake is a particular package:
 # * CMake can be built using the generic infrastructure or the cmake one.
 #   Since Buildroot has no requirement regarding the host system cmake
package/fakedate/fakedate.mk +3 -0
index 99ea748e07..3db9145e63 100644
@@ -7,6 +7,9 @@
 # source included in buildroot
 HOST_FAKEDATE_LICENSE = GPL-2.0+
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_FAKEDATE_ADD_CCACHE_DEPENDENCY = NO
+
 define HOST_FAKEDATE_INSTALL_CMDS
 	$(INSTALL) -D -m 755 package/fakedate/fakedate $(HOST_DIR)/bin/date
 endef
package/hiredis/hiredis.mk +3 -0
index 476b079336..86f35bd4cd 100644
@@ -12,6 +12,9 @@ HIREDIS_LICENSE_FILES = COPYING
 HIREDIS_CPE_ID_VENDOR = redislabs
 HIREDIS_INSTALL_STAGING = YES
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_HIREDIS_ADD_CCACHE_DEPENDENCY = NO
+
 HIREDIS_CONF_OPTS = -DENABLE_NUGET=OFF -DDISABLE_TESTS=ON
 HOST_HIREDIS_CONF_OPTS = -DENABLE_NUGET=OFF -DDISABLE_TESTS=ON -DENABLE_SSL=OFF
 
package/lzip/lzip.mk +3 -0
index 31b5750767..9e09e578db 100644
@@ -9,6 +9,9 @@ LZIP_SITE = http://download.savannah.gnu.org/releases/lzip
 LZIP_LICENSE = GPL-2.0+
 LZIP_LICENSE_FILES = COPYING
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_LZIP_ADD_CCACHE_DEPENDENCY = NO
+
 define LZIP_CONFIGURE_CMDS
 	(cd $(@D); $(TARGET_MAKE_ENV) ./configure --prefix=/usr \
 		$(TARGET_CONFIGURE_OPTS) )
package/pkg-generic.mk +7 -6
index eeb7531dbb..0dd9f6b26d 100644
@@ -761,6 +761,9 @@ endif # ifeq ($$($(2)_CPE_ID_VALID),YES)
 # Similarly for the skeleton.
 $(2)_ADD_TOOLCHAIN_DEPENDENCY	?= YES
 $(2)_ADD_SKELETON_DEPENDENCY	?= YES
+ifeq ($$(BR2_CCACHE),y)
+$(2)_ADD_CCACHE_DEPENDENCY	?= YES
+endif
 
 
 ifeq ($(4),target)
@@ -772,6 +775,10 @@ $(2)_DEPENDENCIES += toolchain
 endif
 endif
 
+ifeq ($$($(2)_ADD_CCACHE_DEPENDENCY),YES)
+$(2)_DEPENDENCIES += host-ccache
+endif
+
 ifneq ($(1),host-skeleton)
 $(2)_DEPENDENCIES += host-skeleton
 endif
@@ -792,12 +799,6 @@ $(2)_EXTRACT_DEPENDENCIES += \
 		$$(call extractor-pkg-dependency,$$(notdir $$(dl))))
 endif
 
-ifeq ($$(BR2_CCACHE),y)
-ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate host-ccache host-cmake host-hiredis host-pkgconf host-zstd host-xxhash host-blake3,$(1)),)
-$(2)_DEPENDENCIES += host-ccache
-endif
-endif
-
 ifeq ($$(BR2_REPRODUCIBLE),y)
 ifeq ($$(filter host-skeleton host-fakedate,$(1)),)
 $(2)_DEPENDENCIES += host-fakedate
package/pkgconf/pkgconf.mk +3 -0
index 1d73b691e1..a8c7b48287 100644
@@ -11,6 +11,9 @@ PKGCONF_LICENSE = pkgconf license
 PKGCONF_LICENSE_FILES = COPYING
 PKGCONF_CPE_ID_VENDOR = pkgconf
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_PKGCONF_ADD_CCACHE_DEPENDENCY = NO
+
 # We are a ccache dependency, so we can't use ccache
 HOST_PKGCONF_CONF_ENV = \
 	CC="$(HOSTCC_NOCCACHE)" \
package/skeleton/skeleton.mk +3 -0
index 4eaaf32ead..1bb3971191 100644
@@ -11,6 +11,9 @@
 SKELETON_ADD_TOOLCHAIN_DEPENDENCY = NO
 SKELETON_ADD_SKELETON_DEPENDENCY = NO
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_SKELETON_ADD_CCACHE_DEPENDENCY = NO
+
 # Although the $(HOST_DIR)/usr symlink is mostly for backward compatibility,
 # there are weird packages that still require it (see host-systemd, and
 # commit 35c11a027c88).
package/tar/tar.mk +3 -0
index a7e85afb72..b9de082603 100644
@@ -13,6 +13,9 @@ TAR_CPE_ID_VENDOR = gnu
 TAR_DEPENDENCIES = $(TARGET_NLS_DEPENDENCIES)
 TAR_CONF_ENV = LIBS=$(TARGET_NLS_LIBS)
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_TAR_ADD_CCACHE_DEPENDENCY = NO
+
 # busybox installs in /bin, so we need tar to install as well in /bin
 # so that we don't end up with two different tar
 #
package/xxhash/xxhash.mk +3 -0
index fde8e464f9..f16a784966 100644
@@ -10,6 +10,9 @@ XXHASH_LICENSE = BSD-2-Clause (library), GPL-2.0+ (xxhsum)
 XXHASH_LICENSE_FILES = LICENSE cli/COPYING
 XXHASH_INSTALL_STAGING = YES
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_XXHASH_ADD_CCACHE_DEPENDENCY = NO
+
 XXHASH_TARGETS = xxhsum libxxhash.pc
 XXHASH_INSTALL_TARGETS = \
 	install_libxxhash.includes \
package/xz/xz.mk +3 -0
index fa4f6dbda9..e295144ed1 100644
@@ -13,6 +13,9 @@ XZ_LICENSE = Public Domain, BSD-0-Clause, GPL-2.0+, GPL-3.0+, LGPL-2.1+
 XZ_LICENSE_FILES = COPYING COPYING.0BSD COPYING.GPLv2 COPYING.GPLv3 COPYING.LGPLv2.1
 XZ_CPE_ID_VENDOR = tukaani
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_XZ_ADD_CCACHE_DEPENDENCY = NO
+
 XZ_CONF_OPTS = \
 	--enable-encoders=lzma1,lzma2,delta,x86,powerpc,ia64,arm,armthumb,arm64,sparc,riscv \
 	--enable-decoders=lzma1,lzma2,delta,x86,powerpc,ia64,arm,armthumb,arm64,sparc,riscv \
package/zstd/zstd.mk +3 -0
index a2da0dd479..575bb2c838 100644
@@ -12,6 +12,9 @@ ZSTD_LICENSE_FILES = LICENSE COPYING
 ZSTD_CPE_ID_VENDOR = facebook
 ZSTD_CPE_ID_PRODUCT = zstandard
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_ZSTD_ADD_CCACHE_DEPENDENCY = NO
+
 ZSTD_OPTS += PREFIX=/usr
 ZSTD_OPTS += ZSTD_LEGACY_SUPPORT=0
 ifeq ($(BR2_PACKAGE_ZLIB),y)
Signature · 6 lines
2.39.2

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#5 lore raw

Re: [Buildroot] [PATCH v4 4/4] package/pkg-generic.mk: add variable to skip ccache dependency

Quoted 12 lines · Heiko Thiery

As more and more packages are required to build the host ccache package and therefore they has to be filtered out for ccache dependency the list growed up.

By adding the variable <PKG>_ADD_CCACHE_DEPENDENCY with the default value to 'yes' we can remove that dependency for each package that is needed by ccache by setting the value to 'no'. This can be done in the package makefile instead of tracking the list in the core pkg-generic.mk file.

This behavior is similar to the skeleton and toolchain dependency.

[snip]

package/pkg-generic.mk endif # ifeq ($$($(2)_CPE_ID_VALID),YES) · line 764

Quoted 9 lines
package/pkg-generic.mk +1 -0
index eeb7531dbb..0dd9f6b26d 100644
@@ -761,6 +761,9 @@ endif # ifeq ($$($(2)_CPE_ID_VALID),YES)
  # Similarly for the skeleton.
  $(2)_ADD_TOOLCHAIN_DEPENDENCY	?= YES
  $(2)_ADD_SKELETON_DEPENDENCY	?= YES
+ifeq ($$(BR2_CCACHE),y)

I've removed this condition and instead added it ...

package/pkg-generic.mk $(2)_DEPENDENCIES += toolchain · line 778

Quoted 10 lines
package/pkg-generic.mk +3 -0
+$(2)_ADD_CCACHE_DEPENDENCY	?= YES
+endif
 
 
  ifeq ($(4),target)
@@ -772,6 +775,10 @@ $(2)_DEPENDENCIES += toolchain
  endif
  endif
 
+ifeq ($$($(2)_ADD_CCACHE_DEPENDENCY),YES)

... here

ifeq ($$(BR2_CCACHE):$$($(2)_ADD_CCACHE_DEPENDENCY),y:YES)

Regards,
Arnout

Quoted 94 lines
package/pkg-generic.mk +3 -6
+$(2)_DEPENDENCIES += host-ccache
+endif
+
  ifneq ($(1),host-skeleton)
  $(2)_DEPENDENCIES += host-skeleton
  endif
@@ -792,12 +799,6 @@ $(2)_EXTRACT_DEPENDENCIES += \
  		$$(call extractor-pkg-dependency,$$(notdir $$(dl))))
  endif
 
-ifeq ($$(BR2_CCACHE),y)
-ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate host-ccache host-cmake host-hiredis host-pkgconf host-zstd host-xxhash host-blake3,$(1)),)
-$(2)_DEPENDENCIES += host-ccache
-endif
-endif
-
  ifeq ($$(BR2_REPRODUCIBLE),y)
  ifeq ($$(filter host-skeleton host-fakedate,$(1)),)
  $(2)_DEPENDENCIES += host-fakedate
package/pkgconf/pkgconf.mk +3 -0
index 1d73b691e1..a8c7b48287 100644
@@ -11,6 +11,9 @@ PKGCONF_LICENSE = pkgconf license
  PKGCONF_LICENSE_FILES = COPYING
  PKGCONF_CPE_ID_VENDOR = pkgconf
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_PKGCONF_ADD_CCACHE_DEPENDENCY = NO
+
  # We are a ccache dependency, so we can't use ccache
  HOST_PKGCONF_CONF_ENV = \
  	CC="$(HOSTCC_NOCCACHE)" \
package/skeleton/skeleton.mk +3 -0
index 4eaaf32ead..1bb3971191 100644
@@ -11,6 +11,9 @@
  SKELETON_ADD_TOOLCHAIN_DEPENDENCY = NO
  SKELETON_ADD_SKELETON_DEPENDENCY = NO
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_SKELETON_ADD_CCACHE_DEPENDENCY = NO
+
  # Although the $(HOST_DIR)/usr symlink is mostly for backward compatibility,
  # there are weird packages that still require it (see host-systemd, and
  # commit 35c11a027c88).
package/tar/tar.mk +3 -0
index a7e85afb72..b9de082603 100644
@@ -13,6 +13,9 @@ TAR_CPE_ID_VENDOR = gnu
  TAR_DEPENDENCIES = $(TARGET_NLS_DEPENDENCIES)
  TAR_CONF_ENV = LIBS=$(TARGET_NLS_LIBS)
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_TAR_ADD_CCACHE_DEPENDENCY = NO
+
  # busybox installs in /bin, so we need tar to install as well in /bin
  # so that we don't end up with two different tar
  #
package/xxhash/xxhash.mk +3 -0
index fde8e464f9..f16a784966 100644
@@ -10,6 +10,9 @@ XXHASH_LICENSE = BSD-2-Clause (library), GPL-2.0+ (xxhsum)
  XXHASH_LICENSE_FILES = LICENSE cli/COPYING
  XXHASH_INSTALL_STAGING = YES
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_XXHASH_ADD_CCACHE_DEPENDENCY = NO
+
  XXHASH_TARGETS = xxhsum libxxhash.pc
  XXHASH_INSTALL_TARGETS = \
  	install_libxxhash.includes \
package/xz/xz.mk +3 -0
index fa4f6dbda9..e295144ed1 100644
@@ -13,6 +13,9 @@ XZ_LICENSE = Public Domain, BSD-0-Clause, GPL-2.0+, GPL-3.0+, LGPL-2.1+
  XZ_LICENSE_FILES = COPYING COPYING.0BSD COPYING.GPLv2 COPYING.GPLv3 COPYING.LGPLv2.1
  XZ_CPE_ID_VENDOR = tukaani
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_XZ_ADD_CCACHE_DEPENDENCY = NO
+
  XZ_CONF_OPTS = \
  	--enable-encoders=lzma1,lzma2,delta,x86,powerpc,ia64,arm,armthumb,arm64,sparc,riscv \
  	--enable-decoders=lzma1,lzma2,delta,x86,powerpc,ia64,arm,armthumb,arm64,sparc,riscv \
package/zstd/zstd.mk +0 -0
index a2da0dd479..575bb2c838 100644

package/zstd/zstd.mk ZSTD_LICENSE_FILES = LICENSE COPYING · line 20

Quoted 9 lines
package/zstd/zstd.mk +3 -0
@@ -12,6 +12,9 @@ ZSTD_LICENSE_FILES = LICENSE COPYING
  ZSTD_CPE_ID_VENDOR = facebook
  ZSTD_CPE_ID_PRODUCT = zstandard
 
+# The package is a dependency to ccache so ccache cannot be a dependency
+HOST_ZSTD_ADD_CCACHE_DEPENDENCY = NO
+
  ZSTD_OPTS += PREFIX=/usr
  ZSTD_OPTS += ZSTD_LEGACY_SUPPORT=0
  ifeq ($(BR2_PACKAGE_ZLIB),y)

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#6 lore raw
In reply to Heiko Thiery
Quoted 33 lines · Heiko Thiery

---- 8< ----
/tmp/ccmQFw5y.s: Assembler messages:
/tmp/ccmQFw5y.s:89386: Error: unsupported instruction `vmovdqu'
/tmp/ccmQFw5y.s:89400: Error: unsupported instruction `vmovdqu'
/tmp/ccmQFw5y.s:89464: Error: unsupported instruction `vmovdqu'
/tmp/ccmQFw5y.s:89518: Error: unsupported instruction `vmovdqu'
/tmp/ccmQFw5y.s:89525: Error: unsupported instruction `vmovdqu'
/tmp/ccmQFw5y.s:123938: Error: unsupported instruction `vmovdqu'
/tmp/ccmQFw5y.s:123953: Error: unsupported instruction `vmovdqu'
make[4]: *** [src/third_party/blake3/CMakeFiles/blake3.dir/build.make:160: src/third_party/blake3/CMakeFiles/blake3.dir/blake3_avx512.c.o] Error 1 make[3]: *** [CMakeFiles/Makefile2:406: src/third_party/blake3/CMakeFiles/blake3.dir/all] Error 2
---- 8< ----

To solve the problem above I started bumping the ccache package to 4.10.1. While doing this I observed the problem that ccache tries to download xxhash by its own and fails.

---- 8< ----
-- Downloading...

    dst='/srv/storage/hthiery/buildroot-2/output/build/host-ccache-4.10.1/_deps/xxhash-subbuild/xxhash-populate-prefix/src/v0.8.2.tar.gz'
    timeout='none'
    inactivity timeout='none'

-- Using src='https://github.com/Cyan4973/xxhash/archive/refs/tags/v0.8.2.tar.gz'
CMake Error at xxhash-subbuild/xxhash-populate-prefix/src/xxhash-populate-stamp/download-xxhash-populate.cmake:170 (message):
Each download failed!

error: downloading 'https://github.com/Cyan4973/xxhash/archive/refs/tags/v0.8.2.tar.gz' failed
status_code: 1
status_string: "Unsupported protocol"
log:
--- LOG BEGIN ---
Protocol "https" not supported or disabled in libcurl
Closing connection

Quoted 8 lines
           --- LOG END ---

---- 8< ----

So I added support for the host package of xxhash and added this to the ccache dependencies.

In addition the bundled blake3 version seems not to build correctly on all machines. See the first error about 'unsupported instruction'.

Maybe that was also due to the CMAKE_ASM_COMPILER mistake? BTW you should have
included that fix as part of this series, otherwise the blake3 package wouldn't have built. Fortunately I remembered (this time) :-)

Applied to master, thanks. One change that I'll comment separately.

Regards,
Arnout

Quoted 35 lines

Therefore the new host package for blake3 is introduced.

v2:
- change subject (fixed naming from ccmake to ccache)

v3:
- fix legal-info for ccache and blake3

v4
- requested changes by Arnout, see history of patches for details

Heiko Thiery (4):
package/blake3: add new host package
package/xxhash: add host support
package/ccache: bump version to 4.10.1 package/pkg-generic.mk: add variable to skip ccache dependency

  DEVELOPERS                   |  1 +
  package/blake3/blake3.hash   |  3 +++
  package/blake3/blake3.mk     | 22 ++++++++++++++++++++++
  package/ccache/ccache.hash   |  4 ++--
  package/ccache/ccache.mk     | 31 +++++++++++++++++++++----------
  package/cmake/cmake.mk       |  3 +++
  package/fakedate/fakedate.mk |  3 +++
  package/hiredis/hiredis.mk   |  3 +++
  package/lzip/lzip.mk         |  3 +++
  package/pkg-generic.mk       | 13 +++++++------
  package/pkgconf/pkgconf.mk   |  3 +++
  package/skeleton/skeleton.mk |  3 +++
  package/tar/tar.mk           |  3 +++
  package/xxhash/xxhash.mk     | 16 ++++++++++++++++
  package/xz/xz.mk             |  3 +++
  package/zstd/zstd.mk         |  3 +++
  16 files changed, 99 insertions(+), 18 deletions(-)
Quoted 3 lines

create mode 100644 package/blake3/blake3.hash
create mode 100644 package/blake3/blake3.mk

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot