Skip to contentSkip to conversation

ChaCha and BLAKE2s cleanups

PATCH Applied Eric Biggers 3 participants 19 messages Aug 27, 2025 – Sep 6, 2025

Subsystems (8)
lore
Share & tools
lore

Eric Biggers

author maintainer
#0 lore raw

cover ChaCha and BLAKE2s cleanups

This series is targeting libcrypto-next. It can also be retrieved from:

git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git chacha-blake2s-v1

This series consolidates how the ChaCha and BLAKE2s code is organized. This is essentially the same change that I made to the other algorithms, so this should be fairly boring by now.

These algorithms were the last two users of lib/crypto/$(SRCARCH)/{Makefile,Kconfig}. So this series removes all those files, finishing the transition to the centralized build process (at least for the algorithms supported by lib/crypto/ so far).

This series also makes the arch-optimized BLAKE2s code start being enabled by default, again following the pattern of the other algorithms.

Finally, it adds a KUnit test suite for BLAKE2s and deletes the older blake2s-selftest.

Eric Biggers (12):
arm: configs: Remove obsolete assignments to CRYPTO_CHACHA20_NEON
crypto: chacha - register only "-lib" drivers
lib/crypto: chacha: Remove unused function chacha_is_arch_optimized() lib/crypto: chacha: Rename chacha.c to chacha-block-generic.c lib/crypto: chacha: Rename libchacha.c to chacha.c lib/crypto: chacha: Consolidate into single module lib/crypto: x86/blake2s: Reduce size of BLAKE2S_SIGMA2 lib/crypto: blake2s: Remove obsolete self-test
lib/crypto: blake2s: Always enable arch-optimized BLAKE2s code lib/crypto: blake2s: Move generic code into blake2s.c lib/crypto: blake2s: Consolidate into single C translation unit lib/crypto: tests: Add KUnit tests for BLAKE2s

 arch/arm/configs/exynos_defconfig             |   1 -
 arch/arm/configs/milbeaut_m10v_defconfig      |   1 -
 arch/arm/configs/multi_v7_defconfig           |   1 -
 arch/arm/configs/omap2plus_defconfig          |   1 -
 crypto/Kconfig                                |   1 -
 crypto/chacha.c                               | 129 +---
 crypto/testmgr.c                              |   9 +-
 include/crypto/chacha.h                       |  37 +-
 include/crypto/internal/blake2s.h             |  21 -
 lib/crypto/Kconfig                            |  74 +-
 lib/crypto/Makefile                           |  57 +-
 lib/crypto/arm/Kconfig                        |  19 -
 lib/crypto/arm/Makefile                       |   8 -
 lib/crypto/arm/blake2s-core.S                 |   5 +-
 lib/crypto/arm/blake2s-glue.c                 |   7 -
 lib/crypto/arm/blake2s.h                      |   5 +
 lib/crypto/arm/{chacha-glue.c => chacha.h}    |  35 +-
 lib/crypto/arm64/Kconfig                      |   8 -
 lib/crypto/arm64/Makefile                     |   4 -
 .../arm64/{chacha-neon-glue.c => chacha.h}    |  32 +-
 lib/crypto/blake2s-generic.c                  | 111 ---
 lib/crypto/blake2s-selftest.c                 | 651 ------------------
 lib/crypto/blake2s.c                          | 105 ++-
 lib/crypto/chacha-block-generic.c             | 114 +++
 lib/crypto/chacha.c                           | 142 ++--
 lib/crypto/libchacha.c                        |  35 -
 lib/crypto/mips/Kconfig                       |   7 -
 lib/crypto/mips/Makefile                      |   5 -
 lib/crypto/mips/chacha-glue.c                 |  29 -
 lib/crypto/mips/chacha.h                      |  14 +
 lib/crypto/powerpc/Kconfig                    |   8 -
 lib/crypto/powerpc/Makefile                   |   4 -
 .../powerpc/{chacha-p10-glue.c => chacha.h}   |  36 +-
 lib/crypto/riscv/Kconfig                      |   8 -
 lib/crypto/riscv/Makefile                     |   4 -
 .../riscv/{chacha-riscv64-glue.c => chacha.h} |  36 +-
 lib/crypto/s390/Kconfig                       |   7 -
 lib/crypto/s390/Makefile                      |   4 -
 lib/crypto/s390/{chacha-glue.c => chacha.h}   |  29 +-
 lib/crypto/tests/Kconfig                      |  10 +
 lib/crypto/tests/Makefile                     |   1 +
 lib/crypto/tests/blake2s-testvecs.h           | 238 +++++++
 lib/crypto/tests/blake2s_kunit.c              | 134 ++++
 lib/crypto/x86/Kconfig                        |  20 -
 lib/crypto/x86/Makefile                       |   7 -
 lib/crypto/x86/blake2s-core.S                 |  28 +-
 lib/crypto/x86/{blake2s-glue.c => blake2s.h}  |  16 +-
 lib/crypto/x86/{chacha_glue.c => chacha.h}    |  36 +-
 scripts/crypto/gen-hash-testvecs.py           |  27 +-
 49 files changed, 840 insertions(+), 1481 deletions(-)
 delete mode 100644 include/crypto/internal/blake2s.h
 delete mode 100644 lib/crypto/arm/Kconfig
 delete mode 100644 lib/crypto/arm/Makefile
 delete mode 100644 lib/crypto/arm/blake2s-glue.c
 create mode 100644 lib/crypto/arm/blake2s.h
 rename lib/crypto/arm/{chacha-glue.c => chacha.h} (76%)
 delete mode 100644 lib/crypto/arm64/Kconfig
 delete mode 100644 lib/crypto/arm64/Makefile
 rename lib/crypto/arm64/{chacha-neon-glue.c => chacha.h} (75%)
 delete mode 100644 lib/crypto/blake2s-generic.c
 delete mode 100644 lib/crypto/blake2s-selftest.c
 create mode 100644 lib/crypto/chacha-block-generic.c
 delete mode 100644 lib/crypto/libchacha.c
 delete mode 100644 lib/crypto/mips/Kconfig
 delete mode 100644 lib/crypto/mips/Makefile
 delete mode 100644 lib/crypto/mips/chacha-glue.c
 create mode 100644 lib/crypto/mips/chacha.h
 delete mode 100644 lib/crypto/powerpc/Kconfig
 delete mode 100644 lib/crypto/powerpc/Makefile
 rename lib/crypto/powerpc/{chacha-p10-glue.c => chacha.h} (62%)
 delete mode 100644 lib/crypto/riscv/Kconfig
 delete mode 100644 lib/crypto/riscv/Makefile
 rename lib/crypto/riscv/{chacha-riscv64-glue.c => chacha.h} (57%)
 delete mode 100644 lib/crypto/s390/Kconfig
 delete mode 100644 lib/crypto/s390/Makefile
 rename lib/crypto/s390/{chacha-glue.c => chacha.h} (51%)
 create mode 100644 lib/crypto/tests/blake2s-testvecs.h
 create mode 100644 lib/crypto/tests/blake2s_kunit.c
 delete mode 100644 lib/crypto/x86/Kconfig
 delete mode 100644 lib/crypto/x86/Makefile
 rename lib/crypto/x86/{blake2s-glue.c => blake2s.h} (83%)
 rename lib/crypto/x86/{chacha_glue.c => chacha.h} (85%)

base-commit: 44781c45f26623c3b92b28e933bf349144c10fe6

Signature · 1 line
2.50.1
author maintainer
#1 lore raw

PATCH 1/12 arm: configs: Remove obsolete assignments to CRYPTO_CHACHA20_NEON

Since v6.15, CRYPTO_CHACHA20_NEON is a hidden option and is selected automatically. Therefore, assigning a value to it in a defconfig no longer has any effect. Remove it from all files that did this.

---
 arch/arm/configs/exynos_defconfig        | 1 -
 arch/arm/configs/milbeaut_m10v_defconfig | 1 -
 arch/arm/configs/multi_v7_defconfig      | 1 -
 arch/arm/configs/omap2plus_defconfig     | 1 -
 4 files changed, 4 deletions(-)
arch/arm/configs/exynos_defconfig +0 -1
index 6915c766923a2..84070e9698e8c 100644
@@ -362,11 +362,10 @@ CONFIG_CRYPTO_LZ4=m
 CONFIG_CRYPTO_USER_API_HASH=m
 CONFIG_CRYPTO_USER_API_SKCIPHER=m
 CONFIG_CRYPTO_USER_API_RNG=m
 CONFIG_CRYPTO_USER_API_AEAD=m
 CONFIG_CRYPTO_AES_ARM_BS=m
-CONFIG_CRYPTO_CHACHA20_NEON=m
 CONFIG_CRYPTO_DEV_EXYNOS_RNG=y
 CONFIG_CRYPTO_DEV_S5P=y
 CONFIG_DMA_CMA=y
 CONFIG_CMA_SIZE_MBYTES=96
 CONFIG_FONTS=y
arch/arm/configs/milbeaut_m10v_defconfig +0 -1
index a3be0b2ede09c..a2995eb390c60 100644
@@ -99,11 +99,10 @@ CONFIG_CRYPTO_AES=y
 CONFIG_CRYPTO_SEQIV=m
 CONFIG_CRYPTO_GHASH_ARM_CE=m
 CONFIG_CRYPTO_AES_ARM=m
 CONFIG_CRYPTO_AES_ARM_BS=m
 CONFIG_CRYPTO_AES_ARM_CE=m
-CONFIG_CRYPTO_CHACHA20_NEON=m
 # CONFIG_CRYPTO_HW is not set
 CONFIG_DMA_CMA=y
 CONFIG_CMA_SIZE_MBYTES=64
 CONFIG_PRINTK_TIME=y
 CONFIG_MAGIC_SYSRQ=y
arch/arm/configs/multi_v7_defconfig +0 -1
index f2822eeefb957..cc0e0e4a879cb 100644
@@ -1289,11 +1289,10 @@ CONFIG_CRYPTO_USER_API_RNG=m
 CONFIG_CRYPTO_USER_API_AEAD=m
 CONFIG_CRYPTO_GHASH_ARM_CE=m
 CONFIG_CRYPTO_AES_ARM=m
 CONFIG_CRYPTO_AES_ARM_BS=m
 CONFIG_CRYPTO_AES_ARM_CE=m
-CONFIG_CRYPTO_CHACHA20_NEON=m
 CONFIG_CRYPTO_DEV_SUN4I_SS=m
 CONFIG_CRYPTO_DEV_FSL_CAAM=m
 CONFIG_CRYPTO_DEV_EXYNOS_RNG=m
 CONFIG_CRYPTO_DEV_S5P=m
 CONFIG_CRYPTO_DEV_ATMEL_AES=m
arch/arm/configs/omap2plus_defconfig +0 -1
index 939913ed9a73b..1d5f752417398 100644
@@ -706,11 +706,10 @@ CONFIG_NLS_ISO8859_1=y
 CONFIG_SECURITY=y
 CONFIG_CRYPTO_MICHAEL_MIC=y
 CONFIG_CRYPTO_GHASH_ARM_CE=m
 CONFIG_CRYPTO_AES_ARM=m
 CONFIG_CRYPTO_AES_ARM_BS=m
-CONFIG_CRYPTO_CHACHA20_NEON=m
 CONFIG_CRYPTO_DEV_OMAP=m
 CONFIG_CRYPTO_DEV_OMAP_SHAM=m
 CONFIG_CRYPTO_DEV_OMAP_AES=m
 CONFIG_CRYPTO_DEV_OMAP_DES=m
 CONFIG_DMA_CMA=y
Signature · 1 line
2.50.1
author maintainer
#2 lore raw

PATCH 2/12 crypto: chacha - register only "-lib" drivers

In reply to Eric Biggers

For the "chacha20", "xchacha20", and "xchacha12" skcipher algorithms, instead of registering "*-generic" drivers as well as conditionally registering "*-$(ARCH)" drivers, instead just register "*-lib" drivers. These just use the regular library functions, so they just do the right thing and are fully accelerated when supported by the CPU.

This eliminates the need for the ChaCha library to support chacha_crypt_generic() and hchacha_block_generic() as part of its external interface. A later commit will make chacha_crypt_generic() a static function.

Since this commit removes several "*-generic" driver names which crypto/testmgr.c expects to exist, update testmgr.c accordingly.

---
 crypto/Kconfig   |   1 -
 crypto/chacha.c  | 129 ++++++++---------------------------------------
 crypto/testmgr.c |   9 +++-
 3 files changed, 29 insertions(+), 110 deletions(-)
crypto/Kconfig +0 -1
index e8ccf5f51b855..09e8fb6ee0813 100644
@@ -646,11 +646,10 @@ config CRYPTO_ARC4
 	  weakness of the algorithm.
 
 config CRYPTO_CHACHA20
 	tristate "ChaCha"
 	select CRYPTO_LIB_CHACHA
-	select CRYPTO_LIB_CHACHA_GENERIC
 	select CRYPTO_SKCIPHER
 	help
 	  The ChaCha20, XChaCha20, and XChaCha12 stream cipher algorithms
 
 	  ChaCha20 is a 256-bit high-speed stream cipher designed by Daniel J.
crypto/chacha.c +22 -107

233 lines not on this page · raw message

crypto/testmgr.c +7 -2
index beab926ba102e..781445f5f56a6 100644
@@ -4150,18 +4150,18 @@ static int alg_test_null(const struct alg_test_desc *desc,
 
 /* Please keep this list sorted by algorithm name. */
 static const struct alg_test_desc alg_test_descs[] = {
 	{
 		.alg = "adiantum(xchacha12,aes)",
-		.generic_driver = "adiantum(xchacha12-generic,aes-generic,nhpoly1305-generic)",
+		.generic_driver = "adiantum(xchacha12-lib,aes-generic,nhpoly1305-generic)",
 		.test = alg_test_skcipher,
 		.suite = {
 			.cipher = __VECS(adiantum_xchacha12_aes_tv_template)
 		},
 	}, {
 		.alg = "adiantum(xchacha20,aes)",
-		.generic_driver = "adiantum(xchacha20-generic,aes-generic,nhpoly1305-generic)",
+		.generic_driver = "adiantum(xchacha20-lib,aes-generic,nhpoly1305-generic)",
 		.test = alg_test_skcipher,
 		.suite = {
 			.cipher = __VECS(adiantum_xchacha20_aes_tv_template)
 		},
 	}, {
@@ -4483,10 +4483,11 @@ static const struct alg_test_desc alg_test_descs[] = {
 				.einval_allowed = 1,
 			}
 		}
 	}, {
 		.alg = "chacha20",
+		.generic_driver = "chacha20-lib",
 		.test = alg_test_skcipher,
 		.suite = {
 			.cipher = __VECS(chacha20_tv_template)
 		},
 	}, {
@@ -5418,16 +5419,18 @@ static const struct alg_test_desc alg_test_descs[] = {
 				.aad_iv = 1,
 			}
 		}
 	}, {
 		.alg = "rfc7539(chacha20,poly1305)",
+		.generic_driver = "rfc7539(chacha20-lib,poly1305-generic)",
 		.test = alg_test_aead,
 		.suite = {
 			.aead = __VECS(rfc7539_tv_template)
 		}
 	}, {
 		.alg = "rfc7539esp(chacha20,poly1305)",
+		.generic_driver = "rfc7539esp(chacha20-lib,poly1305-generic)",
 		.test = alg_test_aead,
 		.suite = {
 			.aead = {
 				____VECS(rfc7539esp_tv_template),
 				.einval_allowed = 1,
@@ -5589,16 +5592,18 @@ static const struct alg_test_desc alg_test_descs[] = {
 		.suite = {
 			.hash = __VECS(sm4_xcbc128_tv_template)
 		}
 	}, {
 		.alg = "xchacha12",
+		.generic_driver = "xchacha12-lib",
 		.test = alg_test_skcipher,
 		.suite = {
 			.cipher = __VECS(xchacha12_tv_template)
 		},
 	}, {
 		.alg = "xchacha20",
+		.generic_driver = "xchacha20-lib",
 		.test = alg_test_skcipher,
 		.suite = {
 			.cipher = __VECS(xchacha20_tv_template)
 		},
 	}, {
Signature · 1 line
2.50.1
author maintainer
#3 lore raw

PATCH 3/12 lib/crypto: chacha: Remove unused function chacha_is_arch_optimized()

In reply to Eric Biggers

chacha_is_arch_optimized() is no longer used, so remove it.

---
 include/crypto/chacha.h                | 9 ---------
 lib/crypto/arm/chacha-glue.c           | 7 -------
 lib/crypto/arm64/chacha-neon-glue.c    | 6 ------
 lib/crypto/mips/chacha-glue.c          | 6 ------
 lib/crypto/powerpc/chacha-p10-glue.c   | 6 ------
 lib/crypto/riscv/chacha-riscv64-glue.c | 6 ------
 lib/crypto/s390/chacha-glue.c          | 6 ------
 lib/crypto/x86/chacha_glue.c           | 6 ------
 8 files changed, 52 deletions(-)
include/crypto/chacha.h +0 -9
index 91f6b4cf561c7..be25a0b65a05f 100644
@@ -117,15 +117,6 @@ static inline void chacha20_crypt(struct chacha_state *state,
 static inline void chacha_zeroize_state(struct chacha_state *state)
 {
 	memzero_explicit(state, sizeof(*state));
 }
 
-#if IS_ENABLED(CONFIG_CRYPTO_ARCH_HAVE_LIB_CHACHA)
-bool chacha_is_arch_optimized(void);
-#else
-static inline bool chacha_is_arch_optimized(void)
-{
-	return false;
-}
-#endif
-
 #endif /* _CRYPTO_CHACHA_H */
lib/crypto/arm/chacha-glue.c +0 -7
index 88ec964152831..67ba045cae359 100644
@@ -99,17 +99,10 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
 		dst += todo;
 	} while (bytes);
 }
 EXPORT_SYMBOL(chacha_crypt_arch);
 
-bool chacha_is_arch_optimized(void)
-{
-	/* We always can use at least the ARM scalar implementation. */
-	return true;
-}
-EXPORT_SYMBOL(chacha_is_arch_optimized);
-
 static int __init chacha_arm_mod_init(void)
 {
 	if (IS_ENABLED(CONFIG_KERNEL_MODE_NEON) && (elf_hwcap & HWCAP_NEON)) {
 		switch (read_cpuid_part()) {
 		case ARM_CPU_PART_CORTEX_A7:
lib/crypto/arm64/chacha-neon-glue.c +0 -6
index d0188f974ca5c..48097aa34af7c 100644
@@ -93,16 +93,10 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
 		dst += todo;
 	} while (bytes);
 }
 EXPORT_SYMBOL(chacha_crypt_arch);
 
-bool chacha_is_arch_optimized(void)
-{
-	return static_key_enabled(&have_neon);
-}
-EXPORT_SYMBOL(chacha_is_arch_optimized);
-
 static int __init chacha_simd_mod_init(void)
 {
 	if (cpu_have_named_feature(ASIMD))
 		static_branch_enable(&have_neon);
 	return 0;
lib/crypto/mips/chacha-glue.c +0 -6
index 88c097594eb0f..f8390af21dc92 100644
@@ -16,14 +16,8 @@ EXPORT_SYMBOL(chacha_crypt_arch);
 
 asmlinkage void hchacha_block_arch(const struct chacha_state *state,
 				   u32 out[HCHACHA_OUT_WORDS], int nrounds);
 EXPORT_SYMBOL(hchacha_block_arch);
 
-bool chacha_is_arch_optimized(void)
-{
-	return true;
-}
-EXPORT_SYMBOL(chacha_is_arch_optimized);
-
 MODULE_DESCRIPTION("ChaCha and HChaCha functions (MIPS optimized)");
 MODULE_AUTHOR("Ard Biesheuvel <[email protected]>");
 MODULE_LICENSE("GPL v2");
lib/crypto/powerpc/chacha-p10-glue.c +0 -6
index fcd23c6f1590b..5d3d5506d7f94 100644
@@ -74,16 +74,10 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
 		dst += todo;
 	} while (bytes);
 }
 EXPORT_SYMBOL(chacha_crypt_arch);
 
-bool chacha_is_arch_optimized(void)
-{
-	return static_key_enabled(&have_p10);
-}
-EXPORT_SYMBOL(chacha_is_arch_optimized);
-
 static int __init chacha_p10_init(void)
 {
 	if (cpu_has_feature(CPU_FTR_ARCH_31))
 		static_branch_enable(&have_p10);
 	return 0;
lib/crypto/riscv/chacha-riscv64-glue.c +0 -6
index 8c3f11d79be31..a15f0aca3fc43 100644
@@ -48,16 +48,10 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
 	}
 	kernel_vector_end();
 }
 EXPORT_SYMBOL(chacha_crypt_arch);
 
-bool chacha_is_arch_optimized(void)
-{
-	return static_key_enabled(&use_zvkb);
-}
-EXPORT_SYMBOL(chacha_is_arch_optimized);
-
 static int __init riscv64_chacha_mod_init(void)
 {
 	if (riscv_isa_extension_available(NULL, ZVKB) &&
 	    riscv_vector_vlen() >= 128)
 		static_branch_enable(&use_zvkb);
lib/crypto/s390/chacha-glue.c +0 -6
index c57dc851214fa..d8137387fe28f 100644
@@ -45,13 +45,7 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
 				CHACHA_BLOCK_SIZE;
 	}
 }
 EXPORT_SYMBOL(chacha_crypt_arch);
 
-bool chacha_is_arch_optimized(void)
-{
-	return cpu_has_vx();
-}
-EXPORT_SYMBOL(chacha_is_arch_optimized);
-
 MODULE_DESCRIPTION("ChaCha stream cipher (s390 optimized)");
 MODULE_LICENSE("GPL v2");
lib/crypto/x86/chacha_glue.c +0 -6
index 10b2c945f5412..de7da9d512af2 100644
@@ -158,16 +158,10 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
 		dst += todo;
 	} while (bytes);
 }
 EXPORT_SYMBOL(chacha_crypt_arch);
 
-bool chacha_is_arch_optimized(void)
-{
-	return static_key_enabled(&chacha_use_simd);
-}
-EXPORT_SYMBOL(chacha_is_arch_optimized);
-
 static int __init chacha_simd_mod_init(void)
 {
 	if (!boot_cpu_has(X86_FEATURE_SSSE3))
 		return 0;
 
Signature · 1 line
2.50.1
author maintainer
#4 lore raw

PATCH 4/12 lib/crypto: chacha: Rename chacha.c to chacha-block-generic.c

In reply to Eric Biggers

Rename chacha.c to chacha-block-generic.c to free up the name chacha.c for the high-level API entry points (chacha_crypt() and hchacha_block()), similar to the other algorithms.

---
 lib/crypto/Makefile                             | 4 ++--
 lib/crypto/{chacha.c => chacha-block-generic.c} | 0
 2 files changed, 2 insertions(+), 2 deletions(-)
 rename lib/crypto/{chacha.c => chacha-block-generic.c} (100%)
lib/crypto/Makefile +2 -2
index 8d91572b4d5ef..ca65924e861ff 100644
@@ -13,12 +13,12 @@ obj-$(CONFIG_KUNIT)				+= tests/
 obj-$(CONFIG_CRYPTO_HASH_INFO)			+= hash_info.o
 
 obj-$(CONFIG_CRYPTO_LIB_UTILS)			+= libcryptoutils.o
 libcryptoutils-y				:= memneq.o utils.o
 
-# chacha is used by the /dev/random driver which is always builtin
-obj-y						+= chacha.o
+# chacha20_block() is used by the /dev/random driver which is always builtin
+obj-y						+= chacha-block-generic.o
 obj-$(CONFIG_CRYPTO_LIB_CHACHA_GENERIC)		+= libchacha.o
 
 obj-$(CONFIG_CRYPTO_LIB_AES)			+= libaes.o
 libaes-y					:= aes.o
 
lib/crypto/chacha-block-generic.c renamed from lib/crypto/chacha.c +0 -0
similarity index 100%
rename from lib/crypto/chacha.c
rename to lib/crypto/chacha-block-generic.c
Signature · 1 line
2.50.1
author maintainer
#5 lore raw

PATCH 5/12 lib/crypto: chacha: Rename libchacha.c to chacha.c

In reply to Eric Biggers

Rename libchacha.c to chacha.c to make the naming consistent with other algorithms and allow additional source files to be added to the libchacha module. This file currently contains chacha_crypt_generic(), but it will soon be updated to contain chacha_crypt().

---
 lib/crypto/Makefile                  | 1 +
 lib/crypto/{libchacha.c => chacha.c} | 0
 2 files changed, 1 insertion(+)
 rename lib/crypto/{libchacha.c => chacha.c} (100%)
lib/crypto/Makefile +1 -0
index ca65924e861ff..5749d176be1c1 100644
@@ -16,10 +16,11 @@ obj-$(CONFIG_CRYPTO_LIB_UTILS)			+= libcryptoutils.o
 libcryptoutils-y				:= memneq.o utils.o
 
 # chacha20_block() is used by the /dev/random driver which is always builtin
 obj-y						+= chacha-block-generic.o
 obj-$(CONFIG_CRYPTO_LIB_CHACHA_GENERIC)		+= libchacha.o
+libchacha-y := chacha.o
 
 obj-$(CONFIG_CRYPTO_LIB_AES)			+= libaes.o
 libaes-y					:= aes.o
 
 obj-$(CONFIG_CRYPTO_LIB_AESCFB)			+= libaescfb.o
lib/crypto/chacha.c renamed from lib/crypto/libchacha.c +0 -0
similarity index 100%
rename from lib/crypto/libchacha.c
rename to lib/crypto/chacha.c
Signature · 1 line
2.50.1
author maintainer
#6 lore raw

PATCH 6/12 lib/crypto: chacha: Consolidate into single module

In reply to Eric Biggers

Consolidate the ChaCha code into a single module (excluding chacha-block-generic.c which remains always built-in for random.c), similar to various other algorithms:

- Each arch now provides a header file lib/crypto/$(SRCARCH)/chacha.h, replacing lib/crypto/$(SRCARCH)/chacha*.c. The header defines chacha_crypt_arch() and hchacha_block_arch(). It is included by lib/crypto/chacha.c, and thus the code gets built into the single libchacha module, with improved inlining in some cases.

- Whether arch-optimized ChaCha is buildable is now controlled centrally by lib/crypto/Kconfig instead of by lib/crypto/$(SRCARCH)/Kconfig. The conditions for enabling it remain the same as before, and it remains enabled by default.

- Any additional arch-specific translation units for the optimized ChaCha code, such as assembly files, are now compiled by lib/crypto/Makefile instead of lib/crypto/$(SRCARCH)/Makefile.

This removes the last use for the Makefile and Kconfig files in the arm64, mips, powerpc, riscv, and s390 subdirectories of lib/crypto/. So also remove those files and the references to them.

---
 include/crypto/chacha.h                       | 28 ++---------
 lib/crypto/Kconfig                            | 47 +++++--------------
 lib/crypto/Makefile                           | 43 +++++++++++++----
 lib/crypto/arm/Kconfig                        |  5 --
 lib/crypto/arm/Makefile                       |  4 --
 lib/crypto/arm/{chacha-glue.c => chacha.h}    | 28 +++--------
 lib/crypto/arm64/Kconfig                      |  8 ----
 lib/crypto/arm64/Makefile                     |  4 --
 .../arm64/{chacha-neon-glue.c => chacha.h}    | 26 +++-------
 lib/crypto/chacha.c                           | 41 ++++++++++++++--
 lib/crypto/mips/Kconfig                       |  7 ---
 lib/crypto/mips/Makefile                      |  5 --
 lib/crypto/mips/{chacha-glue.c => chacha.h}   | 11 +----
 lib/crypto/powerpc/Kconfig                    |  8 ----
 lib/crypto/powerpc/Makefile                   |  4 --
 .../powerpc/{chacha-p10-glue.c => chacha.h}   | 30 +++---------
 lib/crypto/riscv/Kconfig                      |  8 ----
 lib/crypto/riscv/Makefile                     |  4 --
 .../riscv/{chacha-riscv64-glue.c => chacha.h} | 30 +++---------
 lib/crypto/s390/Kconfig                       |  7 ---
 lib/crypto/s390/Makefile                      |  4 --
 lib/crypto/s390/{chacha-glue.c => chacha.h}   | 23 ++-------
 lib/crypto/x86/Kconfig                        |  7 ---
 lib/crypto/x86/Makefile                       |  3 --
 lib/crypto/x86/{chacha_glue.c => chacha.h}    | 30 ++++--------
 25 files changed, 126 insertions(+), 289 deletions(-)
 rename lib/crypto/arm/{chacha-glue.c => chacha.h} (79%)
 delete mode 100644 lib/crypto/arm64/Kconfig
 delete mode 100644 lib/crypto/arm64/Makefile
 rename lib/crypto/arm64/{chacha-neon-glue.c => chacha.h} (78%)
 delete mode 100644 lib/crypto/mips/Kconfig
 delete mode 100644 lib/crypto/mips/Makefile
 rename lib/crypto/mips/{chacha-glue.c => chacha.h} (55%)
 delete mode 100644 lib/crypto/powerpc/Kconfig
 delete mode 100644 lib/crypto/powerpc/Makefile
 rename lib/crypto/powerpc/{chacha-p10-glue.c => chacha.h} (66%)
 delete mode 100644 lib/crypto/riscv/Kconfig
 delete mode 100644 lib/crypto/riscv/Makefile
 rename lib/crypto/riscv/{chacha-riscv64-glue.c => chacha.h} (61%)
 delete mode 100644 lib/crypto/s390/Kconfig
 delete mode 100644 lib/crypto/s390/Makefile
 rename lib/crypto/s390/{chacha-glue.c => chacha.h} (55%)
 rename lib/crypto/x86/{chacha_glue.c => chacha.h} (87%)
include/crypto/chacha.h +4 -24
index be25a0b65a05f..38e26dff27b00 100644
@@ -43,23 +43,15 @@ static inline void chacha20_block(struct chacha_state *state,
 				  u8 out[CHACHA_BLOCK_SIZE])
 {
 	chacha_block_generic(state, out, 20);
 }
 
-void hchacha_block_arch(const struct chacha_state *state,
-			u32 out[HCHACHA_OUT_WORDS], int nrounds);
 void hchacha_block_generic(const struct chacha_state *state,
 			   u32 out[HCHACHA_OUT_WORDS], int nrounds);
 
-static inline void hchacha_block(const struct chacha_state *state,
-				 u32 out[HCHACHA_OUT_WORDS], int nrounds)
-{
-	if (IS_ENABLED(CONFIG_CRYPTO_ARCH_HAVE_LIB_CHACHA))
-		hchacha_block_arch(state, out, nrounds);
-	else
-		hchacha_block_generic(state, out, nrounds);
-}
+void hchacha_block(const struct chacha_state *state,
+		   u32 out[HCHACHA_OUT_WORDS], int nrounds);
 
 enum chacha_constants { /* expand 32-byte k */
 	CHACHA_CONSTANT_EXPA = 0x61707865U,
 	CHACHA_CONSTANT_ND_3 = 0x3320646eU,
 	CHACHA_CONSTANT_2_BY = 0x79622d32U,
@@ -91,24 +83,12 @@ static inline void chacha_init(struct chacha_state *state,
 	state->x[13] = get_unaligned_le32(iv +  4);
 	state->x[14] = get_unaligned_le32(iv +  8);
 	state->x[15] = get_unaligned_le32(iv + 12);
 }
 
-void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
-		       unsigned int bytes, int nrounds);
-void chacha_crypt_generic(struct chacha_state *state, u8 *dst, const u8 *src,
-			  unsigned int bytes, int nrounds);
-
-static inline void chacha_crypt(struct chacha_state *state,
-				u8 *dst, const u8 *src,
-				unsigned int bytes, int nrounds)
-{
-	if (IS_ENABLED(CONFIG_CRYPTO_ARCH_HAVE_LIB_CHACHA))
-		chacha_crypt_arch(state, dst, src, bytes, nrounds);
-	else
-		chacha_crypt_generic(state, dst, src, bytes, nrounds);
-}
+void chacha_crypt(struct chacha_state *state, u8 *dst, const u8 *src,
+		  unsigned int bytes, int nrounds);
 
 static inline void chacha20_crypt(struct chacha_state *state,
 				  u8 *dst, const u8 *src, unsigned int bytes)
 {
 	chacha_crypt(state, dst, src, bytes, 20);
lib/crypto/Kconfig +13 -34

71 lines not on this page · raw message

lib/crypto/Makefile +33 -10

73 lines not on this page · raw message

lib/crypto/arm/Kconfig +0 -5
index 0d821e282c645..740341aa35d21 100644
@@ -10,10 +10,5 @@ config CRYPTO_BLAKE2S_ARM
 
 	  This is faster than the generic implementations of BLAKE2s and
 	  BLAKE2b, but slower than the NEON implementation of BLAKE2b.
 	  There is no NEON implementation of BLAKE2s, since NEON doesn't
 	  really help with it.
-
-config CRYPTO_CHACHA20_NEON
-	tristate
-	default CRYPTO_LIB_CHACHA
-	select CRYPTO_ARCH_HAVE_LIB_CHACHA
lib/crypto/arm/Makefile +0 -4
index 9f70e61d419e2..0574b0e9739e3 100644
@@ -1,8 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0-only
 
 obj-$(CONFIG_CRYPTO_BLAKE2S_ARM) += libblake2s-arm.o
 libblake2s-arm-y := blake2s-core.o blake2s-glue.o
-
-obj-$(CONFIG_CRYPTO_CHACHA20_NEON) += chacha-neon.o
-chacha-neon-y := chacha-scalar-core.o chacha-glue.o
-chacha-neon-$(CONFIG_KERNEL_MODE_NEON) += chacha-neon-core.o
lib/crypto/arm/chacha.h renamed from lib/crypto/arm/chacha-glue.c +7 -21
similarity index 79%
rename from lib/crypto/arm/chacha-glue.c
rename to lib/crypto/arm/chacha.h
index 67ba045cae359..0cae30f8ee5d1 100644
@@ -1,18 +1,16 @@
-// SPDX-License-Identifier: GPL-2.0
+/* SPDX-License-Identifier: GPL-2.0 */
 /*
  * ChaCha and HChaCha functions (ARM optimized)
  *
  * Copyright (C) 2016-2019 Linaro, Ltd. <[email protected]>
  * Copyright (C) 2015 Martin Willi
  */
 
-#include <crypto/chacha.h>
 #include <crypto/internal/simd.h>
 #include <linux/jump_label.h>
 #include <linux/kernel.h>
-#include <linux/module.h>
 
 #include <asm/cputype.h>
 #include <asm/hwcap.h>
 #include <asm/neon.h>
 #include <asm/simd.h>
@@ -62,25 +60,24 @@ static void chacha_doneon(struct chacha_state *state, u8 *dst, const u8 *src,
 			memcpy(dst, buf, bytes);
 		state->x[12]++;
 	}
 }
 
-void hchacha_block_arch(const struct chacha_state *state,
-			u32 out[HCHACHA_OUT_WORDS], int nrounds)
+static void hchacha_block_arch(const struct chacha_state *state,
+			       u32 out[HCHACHA_OUT_WORDS], int nrounds)
 {
 	if (!IS_ENABLED(CONFIG_KERNEL_MODE_NEON) || !neon_usable()) {
 		hchacha_block_arm(state, out, nrounds);
 	} else {
 		kernel_neon_begin();
 		hchacha_block_neon(state, out, nrounds);
 		kernel_neon_end();
 	}
 }
-EXPORT_SYMBOL(hchacha_block_arch);
 
-void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
-		       unsigned int bytes, int nrounds)
+static void chacha_crypt_arch(struct chacha_state *state, u8 *dst,
+			      const u8 *src, unsigned int bytes, int nrounds)
 {
 	if (!IS_ENABLED(CONFIG_KERNEL_MODE_NEON) || !neon_usable() ||
 	    bytes <= CHACHA_BLOCK_SIZE) {
 		chacha_doarm(dst, src, bytes, state, nrounds);
 		state->x[12] += DIV_ROUND_UP(bytes, CHACHA_BLOCK_SIZE);
@@ -97,13 +94,13 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
 		bytes -= todo;
 		src += todo;
 		dst += todo;
 	} while (bytes);
 }
-EXPORT_SYMBOL(chacha_crypt_arch);
 
-static int __init chacha_arm_mod_init(void)
+#define chacha_mod_init_arch chacha_mod_init_arch
+static void chacha_mod_init_arch(void)
 {
 	if (IS_ENABLED(CONFIG_KERNEL_MODE_NEON) && (elf_hwcap & HWCAP_NEON)) {
 		switch (read_cpuid_part()) {
 		case ARM_CPU_PART_CORTEX_A7:
 		case ARM_CPU_PART_CORTEX_A5:
@@ -115,17 +112,6 @@ static int __init chacha_arm_mod_init(void)
 			break;
 		default:
 			static_branch_enable(&use_neon);
 		}
 	}
-	return 0;
 }
-subsys_initcall(chacha_arm_mod_init);
-
-static void __exit chacha_arm_mod_exit(void)
-{
-}
-module_exit(chacha_arm_mod_exit);
-
-MODULE_DESCRIPTION("ChaCha and HChaCha functions (ARM optimized)");
-MODULE_AUTHOR("Ard Biesheuvel <[email protected]>");
-MODULE_LICENSE("GPL v2");
lib/crypto/arm64/Kconfig deleted +0 -8
deleted file mode 100644
index 07c8a4f0ab03a..0000000000000
@@ -1,8 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-config CRYPTO_CHACHA20_NEON
-	tristate
-	depends on KERNEL_MODE_NEON
-	default CRYPTO_LIB_CHACHA
-	select CRYPTO_LIB_CHACHA_GENERIC
-	select CRYPTO_ARCH_HAVE_LIB_CHACHA
lib/crypto/arm64/Makefile deleted +0 -4
deleted file mode 100644
index d49cceca3d1ca..0000000000000
@@ -1,4 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-obj-$(CONFIG_CRYPTO_CHACHA20_NEON) += chacha-neon.o
-chacha-neon-y := chacha-neon-core.o chacha-neon-glue.o
lib/crypto/arm64/chacha.h renamed from lib/crypto/arm64/chacha-neon-glue.c +6 -20
similarity index 78%
rename from lib/crypto/arm64/chacha-neon-glue.c
rename to lib/crypto/arm64/chacha.h
index 48097aa34af7c..ba6c22d460863 100644
@@ -16,15 +16,13 @@
  * it under the terms of the GNU General Public License as published by
  * the Free Software Foundation; either version 2 of the License, or
  * (at your option) any later version.
  */
 
-#include <crypto/chacha.h>
 #include <crypto/internal/simd.h>
 #include <linux/jump_label.h>
 #include <linux/kernel.h>
-#include <linux/module.h>
 
 #include <asm/hwcap.h>
 #include <asm/neon.h>
 #include <asm/simd.h>
 
@@ -59,25 +57,24 @@ static void chacha_doneon(struct chacha_state *state, u8 *dst, const u8 *src,
 		dst += l;
 		state->x[12] += DIV_ROUND_UP(l, CHACHA_BLOCK_SIZE);
 	}
 }
 
-void hchacha_block_arch(const struct chacha_state *state,
-			u32 out[HCHACHA_OUT_WORDS], int nrounds)
+static void hchacha_block_arch(const struct chacha_state *state,
+			       u32 out[HCHACHA_OUT_WORDS], int nrounds)
 {
 	if (!static_branch_likely(&have_neon) || !crypto_simd_usable()) {
 		hchacha_block_generic(state, out, nrounds);
 	} else {
 		kernel_neon_begin();
 		hchacha_block_neon(state, out, nrounds);
 		kernel_neon_end();
 	}
 }
-EXPORT_SYMBOL(hchacha_block_arch);
 
-void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
-		       unsigned int bytes, int nrounds)
+static void chacha_crypt_arch(struct chacha_state *state, u8 *dst,
+			      const u8 *src, unsigned int bytes, int nrounds)
 {
 	if (!static_branch_likely(&have_neon) || bytes <= CHACHA_BLOCK_SIZE ||
 	    !crypto_simd_usable())
 		return chacha_crypt_generic(state, dst, src, bytes, nrounds);
 
@@ -91,23 +88,12 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
 		bytes -= todo;
 		src += todo;
 		dst += todo;
 	} while (bytes);
 }
-EXPORT_SYMBOL(chacha_crypt_arch);
 
-static int __init chacha_simd_mod_init(void)
+#define chacha_mod_init_arch chacha_mod_init_arch
+static void chacha_mod_init_arch(void)
 {
 	if (cpu_have_named_feature(ASIMD))
 		static_branch_enable(&have_neon);
-	return 0;
 }
-subsys_initcall(chacha_simd_mod_init);
-
-static void __exit chacha_simd_mod_exit(void)
-{
-}
-module_exit(chacha_simd_mod_exit);
-
-MODULE_DESCRIPTION("ChaCha and HChaCha functions (ARM64 optimized)");
-MODULE_AUTHOR("Ard Biesheuvel <[email protected]>");
-MODULE_LICENSE("GPL v2");
lib/crypto/chacha.c +38 -3

59 lines not on this page · raw message

lib/crypto/mips/Kconfig deleted +0 -7
deleted file mode 100644
index 94c1a0892c203..0000000000000
@@ -1,7 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-config CRYPTO_CHACHA_MIPS
-	tristate
-	depends on CPU_MIPS32_R2
-	default CRYPTO_LIB_CHACHA
-	select CRYPTO_ARCH_HAVE_LIB_CHACHA
lib/crypto/mips/Makefile deleted +0 -5
deleted file mode 100644
index b5ea0e25c21ef..0000000000000
@@ -1,5 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-obj-$(CONFIG_CRYPTO_CHACHA_MIPS) += chacha-mips.o
-chacha-mips-y := chacha-core.o chacha-glue.o
-AFLAGS_chacha-core.o += -O2 # needed to fill branch delay slots
lib/crypto/mips/chacha.h renamed from lib/crypto/mips/chacha-glue.c +1 -10
similarity index 55%
rename from lib/crypto/mips/chacha-glue.c
rename to lib/crypto/mips/chacha.h
index f8390af21dc92..0c18c0dc2a406 100644
@@ -1,23 +1,14 @@
-// SPDX-License-Identifier: GPL-2.0
+/* SPDX-License-Identifier: GPL-2.0 */
 /*
  * ChaCha and HChaCha functions (MIPS optimized)
  *
  * Copyright (C) 2019 Linaro, Ltd. <[email protected]>
  */
 
-#include <crypto/chacha.h>
 #include <linux/kernel.h>
-#include <linux/module.h>
 
 asmlinkage void chacha_crypt_arch(struct chacha_state *state,
 				  u8 *dst, const u8 *src,
 				  unsigned int bytes, int nrounds);
-EXPORT_SYMBOL(chacha_crypt_arch);
-
 asmlinkage void hchacha_block_arch(const struct chacha_state *state,
 				   u32 out[HCHACHA_OUT_WORDS], int nrounds);
-EXPORT_SYMBOL(hchacha_block_arch);
-
-MODULE_DESCRIPTION("ChaCha and HChaCha functions (MIPS optimized)");
-MODULE_AUTHOR("Ard Biesheuvel <[email protected]>");
-MODULE_LICENSE("GPL v2");
lib/crypto/powerpc/Kconfig deleted +0 -8
deleted file mode 100644
index e41012a61876e..0000000000000
@@ -1,8 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-config CRYPTO_CHACHA20_P10
-	tristate
-	depends on PPC64 && CPU_LITTLE_ENDIAN && VSX
-	default CRYPTO_LIB_CHACHA
-	select CRYPTO_LIB_CHACHA_GENERIC
-	select CRYPTO_ARCH_HAVE_LIB_CHACHA
lib/crypto/powerpc/Makefile deleted +0 -4
deleted file mode 100644
index 778a04edd226c..0000000000000
@@ -1,4 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-obj-$(CONFIG_CRYPTO_CHACHA20_P10) += chacha-p10-crypto.o
-chacha-p10-crypto-y := chacha-p10-glue.o chacha-p10le-8x.o
lib/crypto/powerpc/chacha.h renamed from lib/crypto/powerpc/chacha-p10-glue.c +6 -24
similarity index 66%
rename from lib/crypto/powerpc/chacha-p10-glue.c
rename to lib/crypto/powerpc/chacha.h
index 5d3d5506d7f94..1df6e1ce31c46 100644
@@ -1,16 +1,14 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
+/* SPDX-License-Identifier: GPL-2.0-or-later */
 /*
  * ChaCha stream cipher (P10 accelerated)
  *
  * Copyright 2023- IBM Corp. All rights reserved.
  */
 
-#include <crypto/chacha.h>
 #include <crypto/internal/simd.h>
 #include <linux/kernel.h>
-#include <linux/module.h>
 #include <linux/cpufeature.h>
 #include <linux/sizes.h>
 #include <asm/simd.h>
 #include <asm/switch_to.h>
 
@@ -46,19 +44,14 @@ static void chacha_p10_do_8x(struct chacha_state *state, u8 *dst, const u8 *src,
 
 	if (bytes > 0)
 		chacha_crypt_generic(state, dst, src, bytes, nrounds);
 }
 
-void hchacha_block_arch(const struct chacha_state *state,
-			u32 out[HCHACHA_OUT_WORDS], int nrounds)
-{
-	hchacha_block_generic(state, out, nrounds);
-}
-EXPORT_SYMBOL(hchacha_block_arch);
+#define hchacha_block_arch hchacha_block_generic /* not implemented yet */
 
-void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
-		       unsigned int bytes, int nrounds)
+static void chacha_crypt_arch(struct chacha_state *state, u8 *dst,
+			      const u8 *src, unsigned int bytes, int nrounds)
 {
 	if (!static_branch_likely(&have_p10) || bytes <= CHACHA_BLOCK_SIZE ||
 	    !crypto_simd_usable())
 		return chacha_crypt_generic(state, dst, src, bytes, nrounds);
 
@@ -72,23 +65,12 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
 		bytes -= todo;
 		src += todo;
 		dst += todo;
 	} while (bytes);
 }
-EXPORT_SYMBOL(chacha_crypt_arch);
 
-static int __init chacha_p10_init(void)
+#define chacha_mod_init_arch chacha_mod_init_arch
+static void chacha_mod_init_arch(void)
 {
 	if (cpu_has_feature(CPU_FTR_ARCH_31))
 		static_branch_enable(&have_p10);
-	return 0;
-}
-subsys_initcall(chacha_p10_init);
-
-static void __exit chacha_p10_exit(void)
-{
 }
-module_exit(chacha_p10_exit);
-
-MODULE_DESCRIPTION("ChaCha stream cipher (P10 accelerated)");
-MODULE_AUTHOR("Danny Tsen <[email protected]>");
-MODULE_LICENSE("GPL v2");
lib/crypto/riscv/Kconfig deleted +0 -8
deleted file mode 100644
index bc7a43f33eb3a..0000000000000
@@ -1,8 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-config CRYPTO_CHACHA_RISCV64
-	tristate
-	depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO
-	default CRYPTO_LIB_CHACHA
-	select CRYPTO_ARCH_HAVE_LIB_CHACHA
-	select CRYPTO_LIB_CHACHA_GENERIC
lib/crypto/riscv/Makefile deleted +0 -4
deleted file mode 100644
index e27b78f317fc8..0000000000000
@@ -1,4 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-obj-$(CONFIG_CRYPTO_CHACHA_RISCV64) += chacha-riscv64.o
-chacha-riscv64-y := chacha-riscv64-glue.o chacha-riscv64-zvkb.o
lib/crypto/riscv/chacha.h renamed from lib/crypto/riscv/chacha-riscv64-glue.c +6 -24
similarity index 61%
rename from lib/crypto/riscv/chacha-riscv64-glue.c
rename to lib/crypto/riscv/chacha.h
index a15f0aca3fc43..5c000c6aef4be 100644
@@ -1,34 +1,27 @@
-// SPDX-License-Identifier: GPL-2.0-only
+/* SPDX-License-Identifier: GPL-2.0-only */
 /*
  * ChaCha stream cipher (RISC-V optimized)
  *
  * Copyright (C) 2023 SiFive, Inc.
  * Author: Jerry Shih <[email protected]>
  */
 
 #include <asm/simd.h>
 #include <asm/vector.h>
-#include <crypto/chacha.h>
 #include <crypto/internal/simd.h>
 #include <linux/linkage.h>
-#include <linux/module.h>
 
 static __ro_after_init DEFINE_STATIC_KEY_FALSE(use_zvkb);
 
 asmlinkage void chacha_zvkb(struct chacha_state *state, const u8 *in, u8 *out,
 			    size_t nblocks, int nrounds);
 
-void hchacha_block_arch(const struct chacha_state *state,
-			u32 out[HCHACHA_OUT_WORDS], int nrounds)
-{
-	hchacha_block_generic(state, out, nrounds);
-}
-EXPORT_SYMBOL(hchacha_block_arch);
+#define hchacha_block_arch hchacha_block_generic /* not implemented yet */
 
-void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
-		       unsigned int bytes, int nrounds)
+static void chacha_crypt_arch(struct chacha_state *state, u8 *dst,
+			      const u8 *src, unsigned int bytes, int nrounds)
 {
 	u8 block_buffer[CHACHA_BLOCK_SIZE];
 	unsigned int full_blocks = bytes / CHACHA_BLOCK_SIZE;
 	unsigned int tail_bytes = bytes % CHACHA_BLOCK_SIZE;
 
@@ -46,24 +39,13 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
 		chacha_zvkb(state, block_buffer, block_buffer, 1, nrounds);
 		memcpy(dst, block_buffer, tail_bytes);
 	}
 	kernel_vector_end();
 }
-EXPORT_SYMBOL(chacha_crypt_arch);
 
-static int __init riscv64_chacha_mod_init(void)
+#define chacha_mod_init_arch chacha_mod_init_arch
+static void chacha_mod_init_arch(void)
 {
 	if (riscv_isa_extension_available(NULL, ZVKB) &&
 	    riscv_vector_vlen() >= 128)
 		static_branch_enable(&use_zvkb);
-	return 0;
-}
-subsys_initcall(riscv64_chacha_mod_init);
-
-static void __exit riscv64_chacha_mod_exit(void)
-{
 }
-module_exit(riscv64_chacha_mod_exit);
-
-MODULE_DESCRIPTION("ChaCha stream cipher (RISC-V optimized)");
-MODULE_AUTHOR("Jerry Shih <[email protected]>");
-MODULE_LICENSE("GPL");
lib/crypto/s390/Kconfig deleted +0 -7
deleted file mode 100644
index 069b355fe51aa..0000000000000
@@ -1,7 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-config CRYPTO_CHACHA_S390
-	tristate
-	default CRYPTO_LIB_CHACHA
-	select CRYPTO_LIB_CHACHA_GENERIC
-	select CRYPTO_ARCH_HAVE_LIB_CHACHA
lib/crypto/s390/Makefile deleted +0 -4
deleted file mode 100644
index 06c2cf77178ef..0000000000000
@@ -1,4 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-obj-$(CONFIG_CRYPTO_CHACHA_S390) += chacha_s390.o
-chacha_s390-y := chacha-glue.o chacha-s390.o
lib/crypto/s390/chacha.h renamed from lib/crypto/s390/chacha-glue.c +4 -19
similarity index 55%
rename from lib/crypto/s390/chacha-glue.c
rename to lib/crypto/s390/chacha.h
index d8137387fe28f..fd9c4a4223656 100644
@@ -1,34 +1,23 @@
-// SPDX-License-Identifier: GPL-2.0
+/* SPDX-License-Identifier: GPL-2.0 */
 /*
  * ChaCha stream cipher (s390 optimized)
  *
  * Copyright IBM Corp. 2021
  */
 
-#define KMSG_COMPONENT "chacha_s390"
-#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
-
-#include <crypto/chacha.h>
 #include <linux/cpufeature.h>
 #include <linux/export.h>
 #include <linux/kernel.h>
-#include <linux/module.h>
 #include <linux/sizes.h>
 #include <asm/fpu.h>
 #include "chacha-s390.h"
 
-void hchacha_block_arch(const struct chacha_state *state,
-			u32 out[HCHACHA_OUT_WORDS], int nrounds)
-{
-	/* TODO: implement hchacha_block_arch() in assembly */
-	hchacha_block_generic(state, out, nrounds);
-}
-EXPORT_SYMBOL(hchacha_block_arch);
+#define hchacha_block_arch hchacha_block_generic /* not implemented yet */
 
-void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
-		       unsigned int bytes, int nrounds)
+static void chacha_crypt_arch(struct chacha_state *state, u8 *dst,
+			      const u8 *src, unsigned int bytes, int nrounds)
 {
 	/* s390 chacha20 implementation has 20 rounds hard-coded,
 	 * it cannot handle a block of data or less, but otherwise
 	 * it can handle data of arbitrary size
 	 */
@@ -43,9 +32,5 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
 
 		state->x[12] += round_up(bytes, CHACHA_BLOCK_SIZE) /
 				CHACHA_BLOCK_SIZE;
 	}
 }
-EXPORT_SYMBOL(chacha_crypt_arch);
-
-MODULE_DESCRIPTION("ChaCha stream cipher (s390 optimized)");
-MODULE_LICENSE("GPL v2");
lib/crypto/x86/Kconfig +0 -7
index 24dc9a59b2728..eb47da71aa6b6 100644
@@ -9,12 +9,5 @@ config CRYPTO_BLAKE2S_X86
 	  BLAKE2s cryptographic hash function (RFC 7693)
 
 	  Architecture: x86_64 using:
 	  - SSSE3 (Supplemental SSE3)
 	  - AVX-512 (Advanced Vector Extensions-512)
-
-config CRYPTO_CHACHA20_X86_64
-	tristate
-	depends on 64BIT
-	default CRYPTO_LIB_CHACHA
-	select CRYPTO_LIB_CHACHA_GENERIC
-	select CRYPTO_ARCH_HAVE_LIB_CHACHA
lib/crypto/x86/Makefile +0 -3
index 16c9d76f99472..4454556b243e7 100644
@@ -1,7 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0-only
 
 obj-$(CONFIG_CRYPTO_BLAKE2S_X86) += libblake2s-x86_64.o
 libblake2s-x86_64-y := blake2s-core.o blake2s-glue.o
-
-obj-$(CONFIG_CRYPTO_CHACHA20_X86_64) += chacha-x86_64.o
-chacha-x86_64-y := chacha-avx2-x86_64.o chacha-ssse3-x86_64.o chacha-avx512vl-x86_64.o chacha_glue.o
lib/crypto/x86/chacha.h renamed from lib/crypto/x86/chacha_glue.c +8 -22
similarity index 87%
rename from lib/crypto/x86/chacha_glue.c
rename to lib/crypto/x86/chacha.h
index de7da9d512af2..10cf8f1c569dc 100644
@@ -1,17 +1,15 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
+/* SPDX-License-Identifier: GPL-2.0-or-later */
 /*
  * ChaCha and HChaCha functions (x86_64 optimized)
  *
  * Copyright (C) 2015 Martin Willi
  */
 
 #include <asm/simd.h>
-#include <crypto/chacha.h>
 #include <linux/jump_label.h>
 #include <linux/kernel.h>
-#include <linux/module.h>
 #include <linux/sizes.h>
 
 asmlinkage void chacha_block_xor_ssse3(const struct chacha_state *state,
 				       u8 *dst, const u8 *src,
 				       unsigned int len, int nrounds);
@@ -124,25 +122,24 @@ static void chacha_dosimd(struct chacha_state *state, u8 *dst, const u8 *src,
 		chacha_block_xor_ssse3(state, dst, src, bytes, nrounds);
 		state->x[12]++;
 	}
 }
 
-void hchacha_block_arch(const struct chacha_state *state,
-			u32 out[HCHACHA_OUT_WORDS], int nrounds)
+static void hchacha_block_arch(const struct chacha_state *state,
+			       u32 out[HCHACHA_OUT_WORDS], int nrounds)
 {
 	if (!static_branch_likely(&chacha_use_simd)) {
 		hchacha_block_generic(state, out, nrounds);
 	} else {
 		kernel_fpu_begin();
 		hchacha_block_ssse3(state, out, nrounds);
 		kernel_fpu_end();
 	}
 }
-EXPORT_SYMBOL(hchacha_block_arch);
 
-void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
-		       unsigned int bytes, int nrounds)
+static void chacha_crypt_arch(struct chacha_state *state, u8 *dst,
+			      const u8 *src, unsigned int bytes, int nrounds)
 {
 	if (!static_branch_likely(&chacha_use_simd) ||
 	    bytes <= CHACHA_BLOCK_SIZE)
 		return chacha_crypt_generic(state, dst, src, bytes, nrounds);
 
@@ -156,16 +153,16 @@ void chacha_crypt_arch(struct chacha_state *state, u8 *dst, const u8 *src,
 		bytes -= todo;
 		src += todo;
 		dst += todo;
 	} while (bytes);
 }
-EXPORT_SYMBOL(chacha_crypt_arch);
 
-static int __init chacha_simd_mod_init(void)
+#define chacha_mod_init_arch chacha_mod_init_arch
+static void chacha_mod_init_arch(void)
 {
 	if (!boot_cpu_has(X86_FEATURE_SSSE3))
-		return 0;
+		return;
 
 	static_branch_enable(&chacha_use_simd);
 
 	if (boot_cpu_has(X86_FEATURE_AVX) &&
 	    boot_cpu_has(X86_FEATURE_AVX2) &&
@@ -174,17 +171,6 @@ static int __init chacha_simd_mod_init(void)
 
 		if (boot_cpu_has(X86_FEATURE_AVX512VL) &&
 		    boot_cpu_has(X86_FEATURE_AVX512BW)) /* kmovq */
 			static_branch_enable(&chacha_use_avx512vl);
 	}
-	return 0;
 }
-subsys_initcall(chacha_simd_mod_init);
-
-static void __exit chacha_simd_mod_exit(void)
-{
-}
-module_exit(chacha_simd_mod_exit);
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Martin Willi <[email protected]>");
-MODULE_DESCRIPTION("ChaCha and HChaCha functions (x86_64 optimized)");
Signature · 1 line
2.50.1
author maintainer
#7 lore raw

PATCH 7/12 lib/crypto: x86/blake2s: Reduce size of BLAKE2S_SIGMA2

In reply to Eric Biggers

Save 480 bytes of .rodata by replacing the .long constants with .bytes, and using the vpmovzxbd instruction to expand them.

Also update the code to do the loads before incrementing %rax rather than after. This avoids the need for the first load to use an offset.

---
 lib/crypto/x86/blake2s-core.S | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)
lib/crypto/x86/blake2s-core.S +14 -14
index ac1c845445a4d..ef8e9f427aab3 100644
@@ -27,23 +27,23 @@ SIGMA:
 .byte  2,  6,  0,  8, 12, 10, 11,  3,  1,  4,  7, 15,  9, 13,  5, 14
 .byte 12,  1, 14,  4,  5, 15, 13, 10,  8,  0,  6,  9, 11,  7,  3,  2
 .byte 13,  7, 12,  3, 11, 14,  1,  9,  2,  5, 15,  8, 10,  0,  4,  6
 .byte  6, 14, 11,  0, 15,  9,  3,  8, 10, 12, 13,  1,  5,  2,  7,  4
 .byte 10,  8,  7,  1,  2,  4,  6,  5, 13, 15,  9,  3,  0, 11, 14, 12
-.section .rodata.cst64.BLAKE2S_SIGMA2, "aM", @progbits, 640
+.section .rodata.cst64.BLAKE2S_SIGMA2, "aM", @progbits, 160
 .align 64
 SIGMA2:
-.long  0,  2,  4,  6,  1,  3,  5,  7, 14,  8, 10, 12, 15,  9, 11, 13
-.long  8,  2, 13, 15, 10,  9, 12,  3,  6,  4,  0, 14,  5, 11,  1,  7
-.long 11, 13,  8,  6,  5, 10, 14,  3,  2,  4, 12, 15,  1,  0,  7,  9
-.long 11, 10,  7,  0,  8, 15,  1, 13,  3,  6,  2, 12,  4, 14,  9,  5
-.long  4, 10,  9, 14, 15,  0, 11,  8,  1,  7,  3, 13,  2,  5,  6, 12
-.long  2, 11,  4, 15, 14,  3, 10,  8, 13,  6,  5,  7,  0, 12,  1,  9
-.long  4,  8, 15,  9, 14, 11, 13,  5,  3,  2,  1, 12,  6, 10,  7,  0
-.long  6, 13,  0, 14, 12,  2,  1, 11, 15,  4,  5,  8,  7,  9,  3, 10
-.long 15,  5,  4, 13, 10,  7,  3, 11, 12,  2,  0,  6,  9,  8,  1, 14
-.long  8,  7, 14, 11, 13, 15,  0, 12, 10,  4,  5,  6,  3,  2,  1,  9
+.byte  0,  2,  4,  6,  1,  3,  5,  7, 14,  8, 10, 12, 15,  9, 11, 13
+.byte  8,  2, 13, 15, 10,  9, 12,  3,  6,  4,  0, 14,  5, 11,  1,  7
+.byte 11, 13,  8,  6,  5, 10, 14,  3,  2,  4, 12, 15,  1,  0,  7,  9
+.byte 11, 10,  7,  0,  8, 15,  1, 13,  3,  6,  2, 12,  4, 14,  9,  5
+.byte  4, 10,  9, 14, 15,  0, 11,  8,  1,  7,  3, 13,  2,  5,  6, 12
+.byte  2, 11,  4, 15, 14,  3, 10,  8, 13,  6,  5,  7,  0, 12,  1,  9
+.byte  4,  8, 15,  9, 14, 11, 13,  5,  3,  2,  1, 12,  6, 10,  7,  0
+.byte  6, 13,  0, 14, 12,  2,  1, 11, 15,  4,  5,  8,  7,  9,  3, 10
+.byte 15,  5,  4, 13, 10,  7,  3, 11, 12,  2,  0,  6,  9,  8,  1, 14
+.byte  8,  7, 14, 11, 13, 15,  0, 12, 10,  4,  5,  6,  3,  2,  1,  9
 
 .text
 SYM_FUNC_START(blake2s_compress_ssse3)
 	testq		%rdx,%rdx
 	je		.Lendofloop
@@ -191,13 +191,13 @@ SYM_FUNC_START(blake2s_compress_avx512)
 	vmovdqu		0x20(%rsi),%ymm7
 	addq		$0x40,%rsi
 	leaq		SIGMA2(%rip),%rax
 	movb		$0xa,%cl
 .Lblake2s_compress_avx512_roundloop:
-	addq		$0x40,%rax
-	vmovdqa		-0x40(%rax),%ymm8
-	vmovdqa		-0x20(%rax),%ymm9
+	vpmovzxbd	(%rax),%ymm8
+	vpmovzxbd	0x8(%rax),%ymm9
+	addq		$0x10,%rax
 	vpermi2d	%ymm7,%ymm6,%ymm8
 	vpermi2d	%ymm7,%ymm6,%ymm9
 	vmovdqa		%ymm8,%ymm6
 	vmovdqa		%ymm9,%ymm7
 	vpaddd		%xmm8,%xmm0,%xmm0
Signature · 1 line
2.50.1
author maintainer
#8 lore raw

PATCH 8/12 lib/crypto: blake2s: Remove obsolete self-test

In reply to Eric Biggers

Remove the original BLAKE2s self-test, since it will be superseded by blake2s_kunit.

---
 include/crypto/internal/blake2s.h |   2 -
 lib/crypto/Makefile               |   1 -
 lib/crypto/blake2s-selftest.c     | 651 ------------------------------
 lib/crypto/blake2s.c              |  10 -
 4 files changed, 664 deletions(-)
 delete mode 100644 lib/crypto/blake2s-selftest.c
include/crypto/internal/blake2s.h +0 -2
index 506d56530ca93..3b82572c94337 100644
@@ -14,8 +14,6 @@ void blake2s_compress_generic(struct blake2s_state *state, const u8 *block,
 			      size_t nblocks, const u32 inc);
 
 void blake2s_compress(struct blake2s_state *state, const u8 *block,
 		      size_t nblocks, const u32 inc);
 
-bool blake2s_selftest(void);
-
 #endif /* _CRYPTO_INTERNAL_BLAKE2S_H */
lib/crypto/Makefile +0 -1
index 33d7288265db2..eeddb993da033 100644
@@ -31,11 +31,10 @@ obj-$(CONFIG_CRYPTO_LIB_GF128MUL)		+= gf128mul.o
 
 # blake2s is used by the /dev/random driver which is always builtin
 obj-y						+= libblake2s.o
 libblake2s-y					:= blake2s.o
 libblake2s-$(CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC)	+= blake2s-generic.o
-libblake2s-$(CONFIG_CRYPTO_SELFTESTS)		+= blake2s-selftest.o
 
 ################################################################################
 
 # chacha20_block() is used by the /dev/random driver which is always builtin
 obj-y += chacha-block-generic.o
lib/crypto/blake2s-selftest.c deleted +0 -651

651 lines not on this page · raw message

lib/crypto/blake2s.c +0 -10
index f6ec68c3dcdae..51f2dd7a38a4e 100644
@@ -9,11 +9,10 @@
  */
 
 #include <crypto/internal/blake2s.h>
 #include <linux/bug.h>
 #include <linux/export.h>
-#include <linux/init.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/string.h>
 #include <linux/types.h>
 
@@ -57,16 +56,7 @@ void blake2s_final(struct blake2s_state *state, u8 *out)
 	memcpy(out, state->h, state->outlen);
 	memzero_explicit(state, sizeof(*state));
 }
 EXPORT_SYMBOL(blake2s_final);
 
-static int __init blake2s_mod_init(void)
-{
-	if (IS_ENABLED(CONFIG_CRYPTO_SELFTESTS) &&
-	    WARN_ON(!blake2s_selftest()))
-		return -ENODEV;
-	return 0;
-}
-
-module_init(blake2s_mod_init);
 MODULE_DESCRIPTION("BLAKE2s hash function");
 MODULE_AUTHOR("Jason A. Donenfeld <[email protected]>");
Signature · 1 line
2.50.1
author maintainer
#9 lore raw

PATCH 9/12 lib/crypto: blake2s: Always enable arch-optimized BLAKE2s code

In reply to Eric Biggers

When support for a crypto algorithm is enabled, the arch-optimized implementation of that algorithm should be enabled too. We've learned this the hard way many times over the years: people regularly forget to enable the arch-optimized implementations of the crypto algorithms, resulting in significant performance being left on the table.

Currently, BLAKE2s support is always enabled ('obj-y'), since random.c uses it. Therefore, the arch-optimized BLAKE2s code, which exists for ARM and x86_64, should be always enabled too. Let's do that.

Note that the effect on kernel image size is very small and should not be a concern. On ARM, enabling CRYPTO_BLAKE2S_ARM actually *shrinks* the kernel size by about 1200 bytes, since the ARM-optimized blake2s_compress() completely replaces the generic blake2s_compress(). On x86_64, enabling CRYPTO_BLAKE2S_X86 increases the kernel size by about 1400 bytes, as the generic blake2s_compress() is still included as a fallback; however, for context, that is only about a quarter the size of the generic blake2s_compress(). The x86_64 optimized BLAKE2s code uses much less icache at runtime than the generic code.

---
 lib/crypto/arm/Kconfig | 2 +-
 lib/crypto/x86/Kconfig | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
lib/crypto/arm/Kconfig +1 -1
index 740341aa35d21..a5607ad079c4f 100644
@@ -1,9 +1,9 @@
 # SPDX-License-Identifier: GPL-2.0-only
 
 config CRYPTO_BLAKE2S_ARM
-	bool "Hash functions: BLAKE2s"
+	def_bool y
 	select CRYPTO_ARCH_HAVE_LIB_BLAKE2S
 	help
 	  BLAKE2s cryptographic hash function (RFC 7693)
 
 	  Architecture: arm
lib/crypto/x86/Kconfig +1 -1
index eb47da71aa6b6..ffa718321369f 100644
@@ -1,9 +1,9 @@
 # SPDX-License-Identifier: GPL-2.0-only
 
 config CRYPTO_BLAKE2S_X86
-	bool "Hash functions: BLAKE2s (SSSE3/AVX-512)"
+	def_bool y
 	depends on 64BIT
 	select CRYPTO_LIB_BLAKE2S_GENERIC
 	select CRYPTO_ARCH_HAVE_LIB_BLAKE2S
 	help
 	  BLAKE2s cryptographic hash function (RFC 7693)
Signature · 1 line
2.50.1
#10 lore raw

Re: [PATCH 09/12] lib/crypto: blake2s: Always enable arch-optimized BLAKE2s code

Quoted 3 lines · Eric Biggers

Currently, BLAKE2s support is always enabled ('obj-y'), since random.c uses it. Therefore, the arch-optimized BLAKE2s code, which exists for ARM and x86_64, should be always enabled too.

Maybe a stupid question: what about ARM64? The current NEON implementation in kernel arch/arm/crypto/blake2s-core.S seems to be just for ARM.

While the upstream BLAKE2s with NEON is both for ARM and Aarch64 (ARM64):

https://github.com/BLAKE2/BLAKE2/blob/master/neon

author maintainer
#11 lore raw

Re: [PATCH 09/12] lib/crypto: blake2s: Always enable arch-optimized BLAKE2s code

Quoted 6 lines · Honza Fikar
Quoted 3 lines · Eric Biggers

Currently, BLAKE2s support is always enabled ('obj-y'), since random.c uses it. Therefore, the arch-optimized BLAKE2s code, which exists for ARM and x86_64, should be always enabled too.

Quoted 7 lines

Maybe a stupid question: what about ARM64? The current NEON implementation in kernel arch/arm/crypto/blake2s-core.S seems to be just for ARM.

While the upstream BLAKE2s with NEON is both for ARM and Aarch64 (ARM64):

https://github.com/BLAKE2/BLAKE2/blob/master/neon

There's no ARM64 optimized BLAKE2s code in the Linux kernel yet. If it's useful, someone would need to contribute it.

- Eric

maintainer
#12 lore raw

Re: [PATCH 09/12] lib/crypto: blake2s: Always enable arch-optimized BLAKE2s code

Quoted 4 lines · Eric Biggers
Quoted 2 lines · Honza Fikar

On Fri, Aug 29, 2025 at 2:54 PM Eric Biggers <[email protected]> wrote:

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

Currently, BLAKE2s support is always enabled ('obj-y'), since random.c uses it. Therefore, the arch-optimized BLAKE2s code, which exists for ARM and x86_64, should be always enabled too.

Maybe a stupid question: what about ARM64? The current NEON implementation in kernel arch/arm/crypto/blake2s-core.S seems to be just for ARM.

That code is scalar not NEON, and is carefully tuned to make use of the ARM barrel shifter, which does not exist on arm64.

Quoted 7 lines
Quoted 3 lines

While the upstream BLAKE2s with NEON is both for ARM and Aarch64 (ARM64):

https://github.com/BLAKE2/BLAKE2/blob/master/neon

There's no ARM64 optimized BLAKE2s code in the Linux kernel yet. If it's useful, someone would need to contribute it.

NEON is cumbersome in the kernel so this only makes sense if it is substantially more performant, and I'm skeptical that this is the case, as you pointed out yourself in

commit 5172d322d34c30fb926b29aeb5a064e1fd8a5e13

Author: Eric Biggers <[email protected]>
Date:   Wed Dec 23 00:09:59 2020 -0800

crypto: arm/blake2s - add ARM scalar optimized BLAKE2s

Add an ARM scalar optimized implementation of BLAKE2s.

NEON isn't very useful for BLAKE2s because the BLAKE2s block size is too small for NEON to help. Each NEON instruction would depend on the previous one, resulting in poor performance.

Even if NEON code might be slightly faster on some cores, the fact that it is sensitive to micro-architectural details makes it less attractive.

author maintainer
#13 lore raw

Re: [PATCH 09/12] lib/crypto: blake2s: Always enable arch-optimized BLAKE2s code

Quoted 37 lines · Ard Biesheuvel
Quoted 4 lines · Eric Biggers
Quoted 2 lines · Honza Fikar

On Fri, Aug 29, 2025 at 2:54 PM Eric Biggers <[email protected]> wrote:

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

Currently, BLAKE2s support is always enabled ('obj-y'), since random.c uses it. Therefore, the arch-optimized BLAKE2s code, which exists for ARM and x86_64, should be always enabled too.

Maybe a stupid question: what about ARM64? The current NEON implementation in kernel arch/arm/crypto/blake2s-core.S seems to be just for ARM.

That code is scalar not NEON, and is carefully tuned to make use of the ARM barrel shifter, which does not exist on arm64.

Quoted 7 lines
Quoted 3 lines

While the upstream BLAKE2s with NEON is both for ARM and Aarch64 (ARM64):

https://github.com/BLAKE2/BLAKE2/blob/master/neon

There's no ARM64 optimized BLAKE2s code in the Linux kernel yet. If it's useful, someone would need to contribute it.

NEON is cumbersome in the kernel so this only makes sense if it is substantially more performant, and I'm skeptical that this is the case, as you pointed out yourself in

commit 5172d322d34c30fb926b29aeb5a064e1fd8a5e13

Author: Eric Biggers <[email protected]>
Date:   Wed Dec 23 00:09:59 2020 -0800

crypto: arm/blake2s - add ARM scalar optimized BLAKE2s

Add an ARM scalar optimized implementation of BLAKE2s.

Quoted 7 lines

NEON isn't very useful for BLAKE2s because the BLAKE2s block size is too small for NEON to help. Each NEON instruction would depend on the previous one, resulting in poor performance.

Even if NEON code might be slightly faster on some cores, the fact that it is sensitive to micro-architectural details makes it less attractive.

Yes, agreed: there isn't much opportunity for an ARM64 optimized BLAKE2s implementation to be faster than the generic C code.

- Eric

author maintainer
#14 lore raw

PATCH 10/12 lib/crypto: blake2s: Move generic code into blake2s.c

In reply to Eric Biggers

Move blake2s_compress_generic() from blake2s-generic.c to blake2s.c.

For now it's still guarded by CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC, but this prepares for changing it to a 'static __maybe_unused' function and just using the compiler to automatically decide its inclusion.

---
 lib/crypto/Makefile          |   1 -
 lib/crypto/blake2s-generic.c | 111 -----------------------------------
 lib/crypto/blake2s.c         |  94 +++++++++++++++++++++++++++++
 3 files changed, 94 insertions(+), 112 deletions(-)
 delete mode 100644 lib/crypto/blake2s-generic.c
lib/crypto/Makefile +0 -1
index eeddb993da033..be45dcfb8969b 100644
@@ -30,11 +30,10 @@ libarc4-y					:= arc4.o
 obj-$(CONFIG_CRYPTO_LIB_GF128MUL)		+= gf128mul.o
 
 # blake2s is used by the /dev/random driver which is always builtin
 obj-y						+= libblake2s.o
 libblake2s-y					:= blake2s.o
-libblake2s-$(CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC)	+= blake2s-generic.o
 
 ################################################################################
 
 # chacha20_block() is used by the /dev/random driver which is always builtin
 obj-y += chacha-block-generic.o
lib/crypto/blake2s-generic.c deleted +0 -111

111 lines not on this page · raw message

lib/crypto/blake2s.c +94 -0

104 lines not on this page · raw message

Signature · 1 line
2.50.1
author maintainer
#15 lore raw

PATCH 11/12 lib/crypto: blake2s: Consolidate into single C translation unit

In reply to Eric Biggers

As was done with the other algorithms, reorganize the BLAKE2s code so that the generic implementation and the arch-specific "glue" code is consolidated into a single translation unit, so that the compiler will inline the functions and automatically decide whether to include the generic code in the resulting binary or not.

Similarly, also consolidate the build rules into lib/crypto/{Makefile,Kconfig}. This removes the last uses of lib/crypto/{arm,x86}/{Makefile,Kconfig}, so remove those too.

Don't keep the !KMSAN dependency. It was needed only for other algorithms such as ChaCha that initialize memory from assembly code.

---
 include/crypto/internal/blake2s.h            | 19 -------------
 lib/crypto/Kconfig                           | 29 ++++----------------
 lib/crypto/Makefile                          | 13 +++++----
 lib/crypto/arm/Kconfig                       | 14 ----------
 lib/crypto/arm/Makefile                      |  4 ---
 lib/crypto/arm/blake2s-core.S                |  5 +++-
 lib/crypto/arm/blake2s-glue.c                |  7 -----
 lib/crypto/arm/blake2s.h                     |  5 ++++
 lib/crypto/blake2s.c                         | 29 +++++++++++++-------
 lib/crypto/x86/Kconfig                       | 13 ---------
 lib/crypto/x86/Makefile                      |  4 ---
 lib/crypto/x86/{blake2s-glue.c => blake2s.h} | 16 ++++-------
 12 files changed, 47 insertions(+), 111 deletions(-)
 delete mode 100644 include/crypto/internal/blake2s.h
 delete mode 100644 lib/crypto/arm/Kconfig
 delete mode 100644 lib/crypto/arm/Makefile
 delete mode 100644 lib/crypto/arm/blake2s-glue.c
 create mode 100644 lib/crypto/arm/blake2s.h
 delete mode 100644 lib/crypto/x86/Kconfig
 delete mode 100644 lib/crypto/x86/Makefile
 rename lib/crypto/x86/{blake2s-glue.c => blake2s.h} (83%)
include/crypto/internal/blake2s.h deleted +0 -19
deleted file mode 100644
index 3b82572c94337..0000000000000
@@ -1,19 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 OR MIT */
-/*
- * Helper functions for BLAKE2s implementations.
- * Keep this in sync with the corresponding BLAKE2b header.
- */
-
-#ifndef _CRYPTO_INTERNAL_BLAKE2S_H
-#define _CRYPTO_INTERNAL_BLAKE2S_H
-
-#include <crypto/blake2s.h>
-#include <linux/string.h>
-
-void blake2s_compress_generic(struct blake2s_state *state, const u8 *block,
-			      size_t nblocks, const u32 inc);
-
-void blake2s_compress(struct blake2s_state *state, const u8 *block,
-		      size_t nblocks, const u32 inc);
-
-#endif /* _CRYPTO_INTERNAL_BLAKE2S_H */
lib/crypto/Kconfig +6 -23
index c1db483bc2306..37d85e0c9b979 100644
@@ -26,25 +26,17 @@ config CRYPTO_LIB_ARC4
 	tristate
 
 config CRYPTO_LIB_GF128MUL
 	tristate
 
-config CRYPTO_ARCH_HAVE_LIB_BLAKE2S
-	bool
-	help
-	  Declares whether the architecture provides an arch-specific
-	  accelerated implementation of the Blake2s library interface,
-	  either builtin or as a module.
+# BLAKE2s support is always built-in, so there's no CRYPTO_LIB_BLAKE2S option.
 
-config CRYPTO_LIB_BLAKE2S_GENERIC
-	def_bool !CRYPTO_ARCH_HAVE_LIB_BLAKE2S
-	help
-	  This symbol can be depended upon by arch implementations of the
-	  Blake2s library interface that require the generic code as a
-	  fallback, e.g., for SIMD implementations. If no arch specific
-	  implementation is enabled, this implementation serves the users
-	  of CRYPTO_LIB_BLAKE2S.
+config CRYPTO_LIB_BLAKE2S_ARCH
+	bool
+	depends on !UML
+	default y if ARM
+	default y if X86_64
 
 config CRYPTO_LIB_CHACHA
 	tristate
 	select CRYPTO_LIB_UTILS
 	help
@@ -206,15 +198,6 @@ config CRYPTO_LIB_SHA512_ARCH
 config CRYPTO_LIB_SM3
 	tristate
 
 source "lib/crypto/tests/Kconfig"
 
-if !KMSAN # avoid false positives from assembly
-if ARM
-source "lib/crypto/arm/Kconfig"
-endif
-if X86
-source "lib/crypto/x86/Kconfig"
-endif
-endif
-
 endmenu
lib/crypto/Makefile +8 -5
index be45dcfb8969b..2551843694b7e 100644
@@ -27,13 +27,19 @@ libaesgcm-y					:= aesgcm.o
 obj-$(CONFIG_CRYPTO_LIB_ARC4)			+= libarc4.o
 libarc4-y					:= arc4.o
 
 obj-$(CONFIG_CRYPTO_LIB_GF128MUL)		+= gf128mul.o
 
+################################################################################
+
 # blake2s is used by the /dev/random driver which is always builtin
-obj-y						+= libblake2s.o
-libblake2s-y					:= blake2s.o
+obj-y += blake2s.o
+ifeq ($(CONFIG_CRYPTO_LIB_BLAKE2S_ARCH),y)
+CFLAGS_blake2s.o += -I$(src)/$(SRCARCH)
+obj-$(CONFIG_ARM) += arm/blake2s-core.o
+obj-$(CONFIG_X86) += x86/blake2s-core.o
+endif
 
 ################################################################################
 
 # chacha20_block() is used by the /dev/random driver which is always builtin
 obj-y += chacha-block-generic.o
@@ -252,11 +258,8 @@ obj-$(CONFIG_MPILIB) += mpi/
 obj-$(CONFIG_CRYPTO_SELFTESTS_FULL)		+= simd.o
 
 obj-$(CONFIG_CRYPTO_LIB_SM3)			+= libsm3.o
 libsm3-y					:= sm3.o
 
-obj-$(CONFIG_ARM) += arm/
-obj-$(CONFIG_X86) += x86/
-
 # clean-files must be defined unconditionally
 clean-files += arm/sha256-core.S arm/sha512-core.S
 clean-files += arm64/sha256-core.S arm64/sha512-core.S
lib/crypto/arm/Kconfig deleted +0 -14
deleted file mode 100644
index a5607ad079c4f..0000000000000
@@ -1,14 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-config CRYPTO_BLAKE2S_ARM
-	def_bool y
-	select CRYPTO_ARCH_HAVE_LIB_BLAKE2S
-	help
-	  BLAKE2s cryptographic hash function (RFC 7693)
-
-	  Architecture: arm
-
-	  This is faster than the generic implementations of BLAKE2s and
-	  BLAKE2b, but slower than the NEON implementation of BLAKE2b.
-	  There is no NEON implementation of BLAKE2s, since NEON doesn't
-	  really help with it.
lib/crypto/arm/Makefile deleted +0 -4
deleted file mode 100644
index 0574b0e9739e3..0000000000000
@@ -1,4 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-obj-$(CONFIG_CRYPTO_BLAKE2S_ARM) += libblake2s-arm.o
-libblake2s-arm-y := blake2s-core.o blake2s-glue.o
lib/crypto/arm/blake2s-core.S +4 -1
index df40e46601f10..293f44fa8f316 100644
@@ -1,8 +1,11 @@
 /* SPDX-License-Identifier: GPL-2.0-or-later */
 /*
- * BLAKE2s digest algorithm, ARM scalar implementation
+ * BLAKE2s digest algorithm, ARM scalar implementation.  This is faster
+ * than the generic implementations of BLAKE2s and BLAKE2b, but slower
+ * than the NEON implementation of BLAKE2b.  There is no NEON
+ * implementation of BLAKE2s, since NEON doesn't really help with it.
  *
  * Copyright 2020 Google LLC
  *
  * Author: Eric Biggers <[email protected]>
  */
lib/crypto/arm/blake2s-glue.c deleted +0 -7
deleted file mode 100644
index 0238a70d9581e..0000000000000
@@ -1,7 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-
-#include <crypto/internal/blake2s.h>
-#include <linux/module.h>
-
-/* defined in blake2s-core.S */
-EXPORT_SYMBOL(blake2s_compress);
lib/crypto/arm/blake2s.h new file +5 -0
new file mode 100644
index 0000000000000..aa7a97139ea74
@@ -0,0 +1,5 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+/* defined in blake2s-core.S */
+void blake2s_compress(struct blake2s_state *state, const u8 *block,
+		      size_t nblocks, u32 inc);
lib/crypto/blake2s.c +19 -10
index b5b75ade46588..5638ed9d882d8 100644
@@ -6,19 +6,18 @@
  *
  * Information: https://blake2.net/
  *
  */
 
-#include <crypto/internal/blake2s.h>
+#include <crypto/blake2s.h>
 #include <linux/bug.h>
 #include <linux/export.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/string.h>
 #include <linux/types.h>
 
-#ifdef CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC
 static const u8 blake2s_sigma[10][16] = {
 	{ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 },
 	{ 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 },
 	{ 11, 8, 12, 0, 5, 2, 15, 13, 10, 14, 3, 6, 7, 1, 9, 4 },
 	{ 7, 9, 3, 1, 13, 12, 11, 14, 2, 6, 5, 10, 4, 0, 15, 8 },
@@ -35,16 +34,13 @@ static inline void blake2s_increment_counter(struct blake2s_state *state,
 {
 	state->t[0] += inc;
 	state->t[1] += (state->t[0] < inc);
 }
 
-void blake2s_compress(struct blake2s_state *state, const u8 *block,
-		      size_t nblocks, const u32 inc)
-		      __weak __alias(blake2s_compress_generic);
-
-void blake2s_compress_generic(struct blake2s_state *state, const u8 *block,
-			      size_t nblocks, const u32 inc)
+static void __maybe_unused
+blake2s_compress_generic(struct blake2s_state *state, const u8 *block,
+			 size_t nblocks, const u32 inc)
 {
 	u32 m[16];
 	u32 v[16];
 	int i;
 
@@ -105,12 +101,16 @@ void blake2s_compress_generic(struct blake2s_state *state, const u8 *block,
 
 		block += BLAKE2S_BLOCK_SIZE;
 		--nblocks;
 	}
 }
-EXPORT_SYMBOL(blake2s_compress_generic);
-#endif /* CONFIG_CRYPTO_LIB_BLAKE2S_GENERIC */
+
+#ifdef CONFIG_CRYPTO_LIB_BLAKE2S_ARCH
+#include "blake2s.h" /* $(SRCARCH)/blake2s.h */
+#else
+#define blake2s_compress blake2s_compress_generic
+#endif
 
 static inline void blake2s_set_lastblock(struct blake2s_state *state)
 {
 	state->f[0] = -1;
 }
@@ -150,7 +150,16 @@ void blake2s_final(struct blake2s_state *state, u8 *out)
 	memcpy(out, state->h, state->outlen);
 	memzero_explicit(state, sizeof(*state));
 }
 EXPORT_SYMBOL(blake2s_final);
 
+#ifdef blake2s_mod_init_arch
+static int __init blake2s_mod_init(void)
+{
+	blake2s_mod_init_arch();
+	return 0;
+}
+subsys_initcall(blake2s_mod_init);
+#endif
+
 MODULE_DESCRIPTION("BLAKE2s hash function");
 MODULE_AUTHOR("Jason A. Donenfeld <[email protected]>");
lib/crypto/x86/Kconfig deleted +0 -13
deleted file mode 100644
index ffa718321369f..0000000000000
@@ -1,13 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-config CRYPTO_BLAKE2S_X86
-	def_bool y
-	depends on 64BIT
-	select CRYPTO_LIB_BLAKE2S_GENERIC
-	select CRYPTO_ARCH_HAVE_LIB_BLAKE2S
-	help
-	  BLAKE2s cryptographic hash function (RFC 7693)
-
-	  Architecture: x86_64 using:
-	  - SSSE3 (Supplemental SSE3)
-	  - AVX-512 (Advanced Vector Extensions-512)
lib/crypto/x86/Makefile deleted +0 -4
deleted file mode 100644
index 4454556b243e7..0000000000000
@@ -1,4 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-only
-
-obj-$(CONFIG_CRYPTO_BLAKE2S_X86) += libblake2s-x86_64.o
-libblake2s-x86_64-y := blake2s-core.o blake2s-glue.o
lib/crypto/x86/blake2s.h renamed from lib/crypto/x86/blake2s-glue.c +5 -11
similarity index 83%
rename from lib/crypto/x86/blake2s-glue.c
rename to lib/crypto/x86/blake2s.h
index adc296cd17c93..b6d30d2fa045e 100644
@@ -1,16 +1,14 @@
-// SPDX-License-Identifier: GPL-2.0 OR MIT
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
 /*
  * Copyright (C) 2015-2019 Jason A. Donenfeld <[email protected]>. All Rights Reserved.
  */
 
 #include <asm/cpufeature.h>
 #include <asm/fpu/api.h>
 #include <asm/processor.h>
 #include <asm/simd.h>
-#include <crypto/internal/blake2s.h>
-#include <linux/init.h>
 #include <linux/jump_label.h>
 #include <linux/kernel.h>
 #include <linux/sizes.h>
 
 asmlinkage void blake2s_compress_ssse3(struct blake2s_state *state,
@@ -21,12 +19,12 @@ asmlinkage void blake2s_compress_avx512(struct blake2s_state *state,
 					const u32 inc);
 
 static __ro_after_init DEFINE_STATIC_KEY_FALSE(blake2s_use_ssse3);
 static __ro_after_init DEFINE_STATIC_KEY_FALSE(blake2s_use_avx512);
 
-void blake2s_compress(struct blake2s_state *state, const u8 *block,
-		      size_t nblocks, const u32 inc)
+static void blake2s_compress(struct blake2s_state *state, const u8 *block,
+			     size_t nblocks, const u32 inc)
 {
 	/* SIMD disables preemption, so relax after processing each page. */
 	BUILD_BUG_ON(SZ_4K / BLAKE2S_BLOCK_SIZE < 8);
 
 	if (!static_branch_likely(&blake2s_use_ssse3) || !may_use_simd()) {
@@ -47,13 +45,13 @@ void blake2s_compress(struct blake2s_state *state, const u8 *block,
 
 		nblocks -= blocks;
 		block += blocks * BLAKE2S_BLOCK_SIZE;
 	} while (nblocks);
 }
-EXPORT_SYMBOL(blake2s_compress);
 
-static int __init blake2s_mod_init(void)
+#define blake2s_mod_init_arch blake2s_mod_init_arch
+static void blake2s_mod_init_arch(void)
 {
 	if (boot_cpu_has(X86_FEATURE_SSSE3))
 		static_branch_enable(&blake2s_use_ssse3);
 
 	if (boot_cpu_has(X86_FEATURE_AVX) &&
@@ -61,10 +59,6 @@ static int __init blake2s_mod_init(void)
 	    boot_cpu_has(X86_FEATURE_AVX512F) &&
 	    boot_cpu_has(X86_FEATURE_AVX512VL) &&
 	    cpu_has_xfeatures(XFEATURE_MASK_SSE | XFEATURE_MASK_YMM |
 			      XFEATURE_MASK_AVX512, NULL))
 		static_branch_enable(&blake2s_use_avx512);
-
-	return 0;
 }
-
-subsys_initcall(blake2s_mod_init);
Signature · 1 line
2.50.1
author maintainer
#16 lore raw

PATCH 12/12 lib/crypto: tests: Add KUnit tests for BLAKE2s

In reply to Eric Biggers

Add a KUnit test suite for BLAKE2s. Most of the core test logic is in the previously-added hash-test-template.h. This commit just adds the actual KUnit suite, commits the generated test vectors to the tree so that gen-hash-testvecs.py won't have to be run at build time, and adds a few BLAKE2s-specific test cases.

This is the replacement for blake2s-selftest, which an earlier commit removed. Improvements over blake2s-selftest include integration with KUnit, more comprehensive test cases, and support for benchmarking.

---
 lib/crypto/tests/Kconfig            |  10 ++
 lib/crypto/tests/Makefile           |   1 +
 lib/crypto/tests/blake2s-testvecs.h | 238 ++++++++++++++++++++++++++++
 lib/crypto/tests/blake2s_kunit.c    | 134 ++++++++++++++++
 scripts/crypto/gen-hash-testvecs.py |  27 +++-
 5 files changed, 407 insertions(+), 3 deletions(-)
 create mode 100644 lib/crypto/tests/blake2s-testvecs.h
 create mode 100644 lib/crypto/tests/blake2s_kunit.c
lib/crypto/tests/Kconfig +10 -0
index c21d53fd4b0ce..fd341aa12f157 100644
@@ -1,7 +1,17 @@
 # SPDX-License-Identifier: GPL-2.0-or-later
 
+config CRYPTO_LIB_BLAKE2S_KUNIT_TEST
+	tristate "KUnit tests for BLAKE2s" if !KUNIT_ALL_TESTS
+	depends on KUNIT
+	default KUNIT_ALL_TESTS || CRYPTO_SELFTESTS
+	select CRYPTO_LIB_BENCHMARK_VISIBLE
+	# No need to select CRYPTO_LIB_BLAKE2S here, as that option doesn't
+	# exist; the BLAKE2s code is always built-in for the /dev/random driver.
+	help
+	  KUnit tests for the BLAKE2s cryptographic hash function.
+
 config CRYPTO_LIB_MD5_KUNIT_TEST
 	tristate "KUnit tests for MD5" if !KUNIT_ALL_TESTS
 	depends on KUNIT
 	default KUNIT_ALL_TESTS || CRYPTO_SELFTESTS
 	select CRYPTO_LIB_BENCHMARK_VISIBLE
lib/crypto/tests/Makefile +1 -0
index f6f82c6f9cb5d..be7de929af2cc 100644
@@ -1,7 +1,8 @@
 # SPDX-License-Identifier: GPL-2.0-or-later
 
+obj-$(CONFIG_CRYPTO_LIB_BLAKE2S_KUNIT_TEST) += blake2s_kunit.o
 obj-$(CONFIG_CRYPTO_LIB_MD5_KUNIT_TEST) += md5_kunit.o
 obj-$(CONFIG_CRYPTO_LIB_POLY1305_KUNIT_TEST) += poly1305_kunit.o
 obj-$(CONFIG_CRYPTO_LIB_SHA1_KUNIT_TEST) += sha1_kunit.o
 obj-$(CONFIG_CRYPTO_LIB_SHA256_KUNIT_TEST) += sha224_kunit.o sha256_kunit.o
 obj-$(CONFIG_CRYPTO_LIB_SHA512_KUNIT_TEST) += sha384_kunit.o sha512_kunit.o
lib/crypto/tests/blake2s-testvecs.h new file +238 -0

238 lines not on this page · raw message

lib/crypto/tests/blake2s_kunit.c new file +134 -0

134 lines not on this page · raw message

scripts/crypto/gen-hash-testvecs.py +24 -3
index 4ac927d40cf5c..fc063f2ee95f1 100755
@@ -82,15 +82,20 @@ def print_static_u8_array_definition(name, value):
 def print_c_struct_u8_array_field(name, value):
     print(f'\t\t.{name} = {{')
     print_bytes('\t\t\t', value, 8)
     print('\t\t},')
 
+def alg_digest_size_const(alg):
+    if alg == 'blake2s':
+        return 'BLAKE2S_HASH_SIZE'
+    return f'{alg.upper()}_DIGEST_SIZE'
+
 def gen_unkeyed_testvecs(alg):
     print('')
     print('static const struct {')
     print('\tsize_t data_len;')
-    print(f'\tu8 digest[{alg.upper()}_DIGEST_SIZE];')
+    print(f'\tu8 digest[{alg_digest_size_const(alg)}];')
     print('} hash_testvecs[] = {')
     for data_len in DATA_LENS:
         data = rand_bytes(data_len)
         print('\t{')
         print(f'\t\t.data_len = {data_len},')
@@ -101,11 +106,11 @@ def gen_unkeyed_testvecs(alg):
     data = rand_bytes(4096)
     ctx = hash_init(alg)
     for data_len in range(len(data) + 1):
         hash_update(ctx, compute_hash(alg, data[:data_len]))
     print_static_u8_array_definition(
-            f'hash_testvec_consolidated[{alg.upper()}_DIGEST_SIZE]',
+            f'hash_testvec_consolidated[{alg_digest_size_const(alg)}]',
             hash_final(ctx))
 
 def gen_hmac_testvecs(alg):
     ctx = hmac.new(rand_bytes(32), digestmod=alg)
     data = rand_bytes(4096)
@@ -117,10 +122,24 @@ def gen_hmac_testvecs(alg):
         ctx.update(mac)
     print_static_u8_array_definition(
             f'hmac_testvec_consolidated[{alg.upper()}_DIGEST_SIZE]',
             ctx.digest())
 
+BLAKE2S_KEY_SIZE = 32
+BLAKE2S_HASH_SIZE = 32
+
+def gen_additional_blake2s_testvecs():
+    hashes = b''
+    for key_len in range(BLAKE2S_KEY_SIZE + 1):
+        for out_len in range(1, BLAKE2S_HASH_SIZE + 1):
+            h = hashlib.blake2s(digest_size=out_len, key=rand_bytes(key_len))
+            h.update(rand_bytes(100))
+            hashes += h.digest()
+    print_static_u8_array_definition(
+            'blake2s_keyed_testvec_consolidated[BLAKE2S_HASH_SIZE]',
+            compute_hash('blake2s', hashes))
+
 def gen_additional_poly1305_testvecs():
     key = b'\xff' * POLY1305_KEY_SIZE
     data = b''
     ctx = Poly1305(key)
     for _ in range(32):
@@ -139,9 +158,11 @@ if len(sys.argv) != 2:
 
 alg = sys.argv[1]
 print('/* SPDX-License-Identifier: GPL-2.0-or-later */')
 print(f'/* This file was generated by: {sys.argv[0]} {" ".join(sys.argv[1:])} */')
 gen_unkeyed_testvecs(alg)
-if alg == 'poly1305':
+if alg == 'blake2s':
+    gen_additional_blake2s_testvecs()
+elif alg == 'poly1305':
     gen_additional_poly1305_testvecs()
 else:
     gen_hmac_testvecs(alg)
Signature · 1 line
2.50.1
maintainer Reviewed-by on the v1 series
#17 lore raw
In reply to Eric Biggers
Quoted 20 lines · Eric Biggers

This series is targeting libcrypto-next. It can also be retrieved from:

git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git chacha-blake2s-v1

This series consolidates how the ChaCha and BLAKE2s code is organized. This is essentially the same change that I made to the other algorithms, so this should be fairly boring by now.

These algorithms were the last two users of lib/crypto/$(SRCARCH)/{Makefile,Kconfig}. So this series removes all those files, finishing the transition to the centralized build process (at least for the algorithms supported by lib/crypto/ so far).

This series also makes the arch-optimized BLAKE2s code start being enabled by default, again following the pattern of the other algorithms.

Finally, it adds a KUnit test suite for BLAKE2s and deletes the older blake2s-selftest.

Quoted 14 lines

Eric Biggers (12):
arm: configs: Remove obsolete assignments to CRYPTO_CHACHA20_NEON
crypto: chacha - register only "-lib" drivers
lib/crypto: chacha: Remove unused function chacha_is_arch_optimized() lib/crypto: chacha: Rename chacha.c to chacha-block-generic.c lib/crypto: chacha: Rename libchacha.c to chacha.c lib/crypto: chacha: Consolidate into single module lib/crypto: x86/blake2s: Reduce size of BLAKE2S_SIGMA2 lib/crypto: blake2s: Remove obsolete self-test
lib/crypto: blake2s: Always enable arch-optimized BLAKE2s code lib/crypto: blake2s: Move generic code into blake2s.c lib/crypto: blake2s: Consolidate into single C translation unit lib/crypto: tests: Add KUnit tests for BLAKE2s

author maintainer Applied
#18 lore raw
In reply to Eric Biggers
Quoted 19 lines · Eric Biggers

This series is targeting libcrypto-next. It can also be retrieved from:

git fetch https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git chacha-blake2s-v1

This series consolidates how the ChaCha and BLAKE2s code is organized. This is essentially the same change that I made to the other algorithms, so this should be fairly boring by now.

These algorithms were the last two users of lib/crypto/$(SRCARCH)/{Makefile,Kconfig}. So this series removes all those files, finishing the transition to the centralized build process (at least for the algorithms supported by lib/crypto/ so far).

This series also makes the arch-optimized BLAKE2s code start being enabled by default, again following the pattern of the other algorithms.

Finally, it adds a KUnit test suite for BLAKE2s and deletes the older blake2s-selftest.

Quoted 13 lines

Eric Biggers (12):
arm: configs: Remove obsolete assignments to CRYPTO_CHACHA20_NEON
crypto: chacha - register only "-lib" drivers
lib/crypto: chacha: Remove unused function chacha_is_arch_optimized() lib/crypto: chacha: Rename chacha.c to chacha-block-generic.c lib/crypto: chacha: Rename libchacha.c to chacha.c lib/crypto: chacha: Consolidate into single module lib/crypto: x86/blake2s: Reduce size of BLAKE2S_SIGMA2 lib/crypto: blake2s: Remove obsolete self-test
lib/crypto: blake2s: Always enable arch-optimized BLAKE2s code lib/crypto: blake2s: Move generic code into blake2s.c lib/crypto: blake2s: Consolidate into single C translation unit lib/crypto: tests: Add KUnit tests for BLAKE2s

Applied to https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git/log/?h=libcrypto-next

- Eric