Skip to contentSkip to conversation

Remove per-architecture ChaCha skcipher glue code

PATCH In review Eric Biggers 2 participants 14 messages Apr 5, 2025 – Apr 18, 2025

Subsystems (10)
Arm (32-bit) x86 PowerPC MIPS +6 more subsystems
lore
Share & tools
lore
#0 lore raw

cover Remove per-architecture ChaCha skcipher glue code

Currently each architecture exposes ChaCha not only through the library API, but also through the crypto_skcipher API. That requires each architecture to implement essentially the same skcipher glue code.

Following the example of what's been done for crc32 and crc32c, eliminate this redundancy by making crypto/chacha.c register both the generic and architecture-optimized skcipher algorithms, implemented on top of the appropriate library functions. This removes almost 800 lines of code and disentangles the library code from the skcipher API.

From what I remember, the following are the reasons why it wasn't just done this way originally. But none of these really hold water:

- The skcipher code was there first, so it may have seemed more natural to add onto it rather than replace it.

- Architectures could register multiple skcipher algorithms using different CPU features and have them all be tested in a single boot. This was convenient in theory, but it never really worked properly. It didn't apply to the library code, the x86 ChaCha code wasn't actually doing this (it used static keys instead), and this cannot catch bugs like accidentally using an AVX instruction in SSE code. Instead, a correct solution, which also doesn't require any special kernel support, is to just boot the kernel in QEMU using different -cpu arguments as needed to test all the code.

- There was a concern about changing cra_driver_names potentially breaking users. But in practice users rely on cra_name, not cra_driver_name. We already change, add, and remove cra_driver_names occasionally for various reasons. And even if someone was relying on a specific cra_driver_name, there are some more lightweight compatibility tricks that could be used.

- There was a desire for users to be able to override the kernel's choice of ChaCha implementation by blacklisting the arch-optimized ChaCha module. But that already became mostly impossible when the library functions were added to the same module. And in practice users don't do this anyway. Even if, hypothetically, someone really needed to do this and for some reason the kernel couldn't be fixed to make the right choice in their case automatically, there are other ways this could be implemented such as a module parameter.

Eric Biggers (9):
crypto: riscv/chacha - implement library instead of skcipher
crypto: chacha - centralize the skcipher wrappers for arch code
crypto: arm/chacha - remove the redundant skcipher algorithms
crypto: arm64/chacha - remove the skcipher algorithms
crypto: mips/chacha - remove the skcipher algorithms
crypto: powerpc/chacha - remove the skcipher algorithms
crypto: s390/chacha - remove the skcipher algorithms
crypto: x86/chacha - remove the skcipher algorithms
crypto: chacha - remove <crypto/internal/chacha.h>

 arch/arm/crypto/Kconfig                 |   7 -
 arch/arm/crypto/chacha-glue.c           | 243 +---------------------
 arch/arm/crypto/chacha-neon-core.S      |   2 +-
 arch/arm64/crypto/Kconfig               |   7 -
 arch/arm64/crypto/chacha-neon-core.S    |   2 +-
 arch/arm64/crypto/chacha-neon-glue.c    | 146 +------------
 arch/mips/crypto/Kconfig                |   6 -
 arch/mips/crypto/chacha-glue.c          | 131 +-----------
 arch/powerpc/crypto/Kconfig             |   8 -
 arch/powerpc/crypto/chacha-p10-glue.c   | 147 +-------------
 arch/riscv/crypto/Kconfig               |  11 +-
 arch/riscv/crypto/chacha-riscv64-glue.c | 112 ++++------
 arch/riscv/crypto/chacha-riscv64-zvkb.S |  71 +++----
 arch/s390/crypto/Kconfig                |   7 -
 arch/s390/crypto/chacha-glue.c          |  99 ++-------
 arch/x86/crypto/Kconfig                 |   9 -
 arch/x86/crypto/chacha_glue.c           | 144 +------------
 crypto/Makefile                         |   3 +-
 crypto/chacha.c                         | 260 ++++++++++++++++++++++++
 crypto/chacha_generic.c                 | 139 -------------
 include/crypto/chacha.h                 |   9 +
 include/crypto/internal/chacha.h        |  43 ----
 22 files changed, 413 insertions(+), 1193 deletions(-)
 create mode 100644 crypto/chacha.c
 delete mode 100644 crypto/chacha_generic.c
 delete mode 100644 include/crypto/internal/chacha.h

base-commit: 56f944529ec2292cbe63377a76df3759d702dd39

Signature · 1 line
2.49.0
#1 lore raw

PATCH 1/9 crypto: riscv/chacha - implement library instead of skcipher

From: Eric Biggers <[email protected]>

Currently the RISC-V optimized ChaCha20 is only wired up to the crypto_skcipher API, which makes it unavailable to users of the library API. The crypto_skcipher API for ChaCha20 is going to change to be implemented on top of the library API, so the library API needs to be supported. And of course it's needed anyway to serve the library users.

Therefore, change the RISC-V ChaCha20 code to implement the library API instead of the crypto_skcipher API.

The library functions take the ChaCha state matrix directly (instead of key and IV) and support both ChaCha20 and ChaCha12. To make the RISC-V code work properly for that, change the assembly code to take the state matrix directly and add a nrounds parameter.

---
 arch/riscv/crypto/Kconfig               |  11 +--
 arch/riscv/crypto/chacha-riscv64-glue.c | 106 ++++++++----------------
 arch/riscv/crypto/chacha-riscv64-zvkb.S |  71 ++++++++--------
 3 files changed, 75 insertions(+), 113 deletions(-)
arch/riscv/crypto/Kconfig +4 -7
index c67095a3d6690..6392e1e11bc96 100644
@@ -17,18 +17,15 @@ config CRYPTO_AES_RISCV64
 	  - Zvbb vector extension (XTS)
 	  - Zvkb vector crypto extension (CTR)
 	  - Zvkg vector crypto extension (XTS)
 
 config CRYPTO_CHACHA_RISCV64
-	tristate "Ciphers: ChaCha"
+	tristate
 	depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO
-	select CRYPTO_SKCIPHER
-	help
-	  Length-preserving ciphers: ChaCha20 stream cipher algorithm
-
-	  Architecture: riscv64 using:
-	  - Zvkb vector crypto extension
+	select CRYPTO_ARCH_HAVE_LIB_CHACHA
+	select CRYPTO_LIB_CHACHA_GENERIC
+	default CRYPTO_LIB_CHACHA_INTERNAL
 
 config CRYPTO_GHASH_RISCV64
 	tristate "Hash functions: GHASH"
 	depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO
 	select CRYPTO_GCM
arch/riscv/crypto/chacha-riscv64-glue.c +34 -72

135 lines not on this page · raw message

arch/riscv/crypto/chacha-riscv64-zvkb.S +37 -34

151 lines not on this page · raw message

Signature · 1 line
2.49.0
#2 lore raw

PATCH 2/9 crypto: chacha - centralize the skcipher wrappers for arch code

In reply to Eric Biggers

From: Eric Biggers <[email protected]>

Following the example of the crc32 and crc32c code, make the crypto subsystem register both generic and architecture-optimized chacha20, xchacha20, and xchacha12 skcipher algorithms, all implemented on top of the appropriate library functions. This eliminates the need for every architecture to implement the same skcipher glue code.

To register the architecture-optimized skciphers only when architecture-optimized code is actually being used, add a function chacha_is_arch_optimized() and make each arch implement it. Change each architecture's ChaCha module_init function to arch_initcall so that the CPU feature detection is guaranteed to run before chacha_is_arch_optimized() gets called by crypto/chacha.c. In the case of s390, remove the CPU feature based module autoloading, which is no longer needed since the module just gets pulled in via function linkage.

---
 arch/arm/crypto/chacha-glue.c           |   9 +-
 arch/arm64/crypto/chacha-neon-glue.c    |   8 +-
 arch/mips/crypto/chacha-glue.c          |   8 +-
 arch/powerpc/crypto/chacha-p10-glue.c   |   8 +-
 arch/riscv/crypto/chacha-riscv64-glue.c |   8 +-
 arch/s390/crypto/chacha-glue.c          |   8 +-
 arch/x86/crypto/chacha_glue.c           |   8 +-
 crypto/Makefile                         |   3 +-
 crypto/chacha.c                         | 227 ++++++++++++++++++++++++
 crypto/chacha_generic.c                 | 139 ---------------
 include/crypto/chacha.h                 |   9 +
 11 files changed, 288 insertions(+), 147 deletions(-)
 create mode 100644 crypto/chacha.c
 delete mode 100644 crypto/chacha_generic.c
arch/arm/crypto/chacha-glue.c +8 -1
index 50e635512046e..e1cb34d317712 100644
@@ -285,10 +285,17 @@ static struct skcipher_alg neon_algs[] = {
 		.encrypt		= xchacha_neon,
 		.decrypt		= xchacha_neon,
 	}
 };
 
+bool chacha_is_arch_optimized(void)
+{
+	/* We always can use at least the ARM scalar implementation. */
+	return true;
+}
+EXPORT_SYMBOL(chacha_is_arch_optimized);
+
 static int __init chacha_simd_mod_init(void)
 {
 	int err = 0;
 
 	if (IS_REACHABLE(CONFIG_CRYPTO_SKCIPHER)) {
@@ -331,11 +338,11 @@ static void __exit chacha_simd_mod_fini(void)
 		if (IS_ENABLED(CONFIG_KERNEL_MODE_NEON) && (elf_hwcap & HWCAP_NEON))
 			crypto_unregister_skciphers(neon_algs, ARRAY_SIZE(neon_algs));
 	}
 }
 
-module_init(chacha_simd_mod_init);
+arch_initcall(chacha_simd_mod_init);
 module_exit(chacha_simd_mod_fini);
 
 MODULE_DESCRIPTION("ChaCha and XChaCha stream ciphers (scalar and NEON accelerated)");
 MODULE_AUTHOR("Ard Biesheuvel <[email protected]>");
 MODULE_LICENSE("GPL v2");
arch/arm64/crypto/chacha-neon-glue.c +7 -1
index 229876acfc581..bb9b52321bda7 100644
@@ -204,10 +204,16 @@ static struct skcipher_alg algs[] = {
 		.encrypt		= xchacha_neon,
 		.decrypt		= xchacha_neon,
 	}
 };
 
+bool chacha_is_arch_optimized(void)
+{
+	return static_key_enabled(&have_neon);
+}
+EXPORT_SYMBOL(chacha_is_arch_optimized);
+
 static int __init chacha_simd_mod_init(void)
 {
 	if (!cpu_have_named_feature(ASIMD))
 		return 0;
 
@@ -221,11 +227,11 @@ static void __exit chacha_simd_mod_fini(void)
 {
 	if (IS_REACHABLE(CONFIG_CRYPTO_SKCIPHER) && cpu_have_named_feature(ASIMD))
 		crypto_unregister_skciphers(algs, ARRAY_SIZE(algs));
 }
 
-module_init(chacha_simd_mod_init);
+arch_initcall(chacha_simd_mod_init);
 module_exit(chacha_simd_mod_fini);
 
 MODULE_DESCRIPTION("ChaCha and XChaCha stream ciphers (NEON accelerated)");
 MODULE_AUTHOR("Ard Biesheuvel <[email protected]>");
 MODULE_LICENSE("GPL v2");
arch/mips/crypto/chacha-glue.c +7 -1
index f6fc2e1079a19..64ccaeaeaa1e1 100644
@@ -118,10 +118,16 @@ static struct skcipher_alg algs[] = {
 		.encrypt		= xchacha_mips,
 		.decrypt		= xchacha_mips,
 	}
 };
 
+bool chacha_is_arch_optimized(void)
+{
+	return true;
+}
+EXPORT_SYMBOL(chacha_is_arch_optimized);
+
 static int __init chacha_simd_mod_init(void)
 {
 	return IS_REACHABLE(CONFIG_CRYPTO_SKCIPHER) ?
 		crypto_register_skciphers(algs, ARRAY_SIZE(algs)) : 0;
 }
@@ -130,11 +136,11 @@ static void __exit chacha_simd_mod_fini(void)
 {
 	if (IS_REACHABLE(CONFIG_CRYPTO_SKCIPHER))
 		crypto_unregister_skciphers(algs, ARRAY_SIZE(algs));
 }
 
-module_init(chacha_simd_mod_init);
+arch_initcall(chacha_simd_mod_init);
 module_exit(chacha_simd_mod_fini);
 
 MODULE_DESCRIPTION("ChaCha and XChaCha stream ciphers (MIPS accelerated)");
 MODULE_AUTHOR("Ard Biesheuvel <[email protected]>");
 MODULE_LICENSE("GPL v2");
arch/powerpc/crypto/chacha-p10-glue.c +7 -1
index d8796decc1fb7..3355305b6c7f8 100644
@@ -187,10 +187,16 @@ static struct skcipher_alg algs[] = {
 		.encrypt		= xchacha_p10,
 		.decrypt		= xchacha_p10,
 	}
 };
 
+bool chacha_is_arch_optimized(void)
+{
+	return static_key_enabled(&have_p10);
+}
+EXPORT_SYMBOL(chacha_is_arch_optimized);
+
 static int __init chacha_p10_init(void)
 {
 	if (!cpu_has_feature(CPU_FTR_ARCH_31))
 		return 0;
 
@@ -205,11 +211,11 @@ static void __exit chacha_p10_exit(void)
 		return;
 
 	crypto_unregister_skciphers(algs, ARRAY_SIZE(algs));
 }
 
-module_init(chacha_p10_init);
+arch_initcall(chacha_p10_init);
 module_exit(chacha_p10_exit);
 
 MODULE_DESCRIPTION("ChaCha and XChaCha stream ciphers (P10 accelerated)");
 MODULE_AUTHOR("Danny Tsen <[email protected]>");
 MODULE_LICENSE("GPL v2");
arch/riscv/crypto/chacha-riscv64-glue.c +7 -1
index 68caef7a3d50b..ccaab0dea383f 100644
@@ -47,17 +47,23 @@ void chacha_crypt_arch(u32 *state, u8 *dst, const u8 *src, unsigned int bytes,
 	}
 	kernel_vector_end();
 }
 EXPORT_SYMBOL(chacha_crypt_arch);
 
+bool chacha_is_arch_optimized(void)
+{
+	return static_key_enabled(&use_zvkb);
+}
+EXPORT_SYMBOL(chacha_is_arch_optimized);
+
 static int __init riscv64_chacha_mod_init(void)
 {
 	if (riscv_isa_extension_available(NULL, ZVKB) &&
 	    riscv_vector_vlen() >= 128)
 		static_branch_enable(&use_zvkb);
 	return 0;
 }
-module_init(riscv64_chacha_mod_init);
+arch_initcall(riscv64_chacha_mod_init);
 
 MODULE_DESCRIPTION("ChaCha stream cipher (RISC-V optimized)");
 MODULE_AUTHOR("Jerry Shih <[email protected]>");
 MODULE_LICENSE("GPL");
arch/s390/crypto/chacha-glue.c +7 -1
index 920e9f0941e75..0c68191f2aa4c 100644
@@ -101,10 +101,16 @@ static struct skcipher_alg chacha_algs[] = {
 		.encrypt		= chacha20_s390,
 		.decrypt		= chacha20_s390,
 	}
 };
 
+bool chacha_is_arch_optimized(void)
+{
+	return cpu_has_vx();
+}
+EXPORT_SYMBOL(chacha_is_arch_optimized);
+
 static int __init chacha_mod_init(void)
 {
 	return IS_REACHABLE(CONFIG_CRYPTO_SKCIPHER) ?
 		crypto_register_skciphers(chacha_algs, ARRAY_SIZE(chacha_algs)) : 0;
 }
@@ -113,11 +119,11 @@ static void __exit chacha_mod_fini(void)
 {
 	if (IS_REACHABLE(CONFIG_CRYPTO_SKCIPHER))
 		crypto_unregister_skciphers(chacha_algs, ARRAY_SIZE(chacha_algs));
 }
 
-module_cpu_feature_match(S390_CPU_FEATURE_VXRS, chacha_mod_init);
+arch_initcall(chacha_mod_init);
 module_exit(chacha_mod_fini);
 
 MODULE_DESCRIPTION("ChaCha20 stream cipher");
 MODULE_LICENSE("GPL v2");
 
arch/x86/crypto/chacha_glue.c +7 -1
index 8bb74a2728798..83a07b31cdd3a 100644
@@ -268,10 +268,16 @@ static struct skcipher_alg algs[] = {
 		.encrypt		= xchacha_simd,
 		.decrypt		= xchacha_simd,
 	},
 };
 
+bool chacha_is_arch_optimized(void)
+{
+	return static_key_enabled(&chacha_use_simd);
+}
+EXPORT_SYMBOL(chacha_is_arch_optimized);
+
 static int __init chacha_simd_mod_init(void)
 {
 	if (!boot_cpu_has(X86_FEATURE_SSSE3))
 		return 0;
 
@@ -295,11 +301,11 @@ static void __exit chacha_simd_mod_fini(void)
 {
 	if (IS_REACHABLE(CONFIG_CRYPTO_SKCIPHER) && boot_cpu_has(X86_FEATURE_SSSE3))
 		crypto_unregister_skciphers(algs, ARRAY_SIZE(algs));
 }
 
-module_init(chacha_simd_mod_init);
+arch_initcall(chacha_simd_mod_init);
 module_exit(chacha_simd_mod_fini);
 
 MODULE_LICENSE("GPL");
 MODULE_AUTHOR("Martin Willi <[email protected]>");
 MODULE_DESCRIPTION("ChaCha and XChaCha stream ciphers (x64 SIMD accelerated)");
crypto/Makefile +2 -1
index 0e6ab5ffd3f77..98510a2aa0b1e 100644
@@ -146,11 +146,12 @@ obj-$(CONFIG_CRYPTO_ARC4) += arc4.o
 obj-$(CONFIG_CRYPTO_TEA) += tea.o
 obj-$(CONFIG_CRYPTO_KHAZAD) += khazad.o
 obj-$(CONFIG_CRYPTO_ANUBIS) += anubis.o
 obj-$(CONFIG_CRYPTO_SEED) += seed.o
 obj-$(CONFIG_CRYPTO_ARIA) += aria_generic.o
-obj-$(CONFIG_CRYPTO_CHACHA20) += chacha_generic.o
+obj-$(CONFIG_CRYPTO_CHACHA20) += chacha.o
+CFLAGS_chacha.o += -DARCH=$(ARCH)
 obj-$(CONFIG_CRYPTO_POLY1305) += poly1305_generic.o
 obj-$(CONFIG_CRYPTO_DEFLATE) += deflate.o
 obj-$(CONFIG_CRYPTO_MICHAEL_MIC) += michael_mic.o
 obj-$(CONFIG_CRYPTO_CRC32C) += crc32c_generic.o
 obj-$(CONFIG_CRYPTO_CRC32) += crc32_generic.o
crypto/chacha.c new file +227 -0

227 lines not on this page · raw message

crypto/chacha_generic.c deleted +0 -139

139 lines not on this page · raw message

include/crypto/chacha.h +9 -0
index f8cc073bba41b..58129e18cc31d 100644
@@ -97,6 +97,15 @@ static inline void chacha20_crypt(u32 *state, u8 *dst, const u8 *src,
 				  unsigned int bytes)
 {
 	chacha_crypt(state, dst, src, bytes, 20);
 }
 
+#if IS_ENABLED(CONFIG_CRYPTO_ARCH_HAVE_LIB_CHACHA)
+bool chacha_is_arch_optimized(void);
+#else
+static inline bool chacha_is_arch_optimized(void)
+{
+	return false;
+}
+#endif
+
 #endif /* _CRYPTO_CHACHA_H */
Signature · 1 line
2.49.0
#3 lore raw

PATCH 3/9 crypto: arm/chacha - remove the redundant skcipher algorithms

In reply to Eric Biggers

From: Eric Biggers <[email protected]>

Since crypto/chacha.c now registers chacha20-$(ARCH), xchacha20-$(ARCH), and xchacha12-$(ARCH) skcipher algorithms that use the architecture's ChaCha and HChaCha library functions, individual architectures no longer need to do the same. Therefore, remove the redundant skcipher algorithms and leave just the library functions.

---
 arch/arm/crypto/Kconfig            |   7 -
 arch/arm/crypto/chacha-glue.c      | 242 +----------------------------
 arch/arm/crypto/chacha-neon-core.S |   2 +-
 3 files changed, 7 insertions(+), 244 deletions(-)
arch/arm/crypto/Kconfig +0 -7
index 23e4ea067ddbb..624c4c8c5296a 100644
@@ -214,17 +214,10 @@ config CRYPTO_AES_ARM_CE
 	  Architecture: arm using:
 	  - ARMv8 Crypto Extensions
 
 config CRYPTO_CHACHA20_NEON
 	tristate
-	select CRYPTO_SKCIPHER
 	select CRYPTO_ARCH_HAVE_LIB_CHACHA
 	default CRYPTO_LIB_CHACHA_INTERNAL
-	help
-	  Length-preserving ciphers: ChaCha20, XChaCha20, and XChaCha12
-	  stream cipher algorithms
-
-	  Architecture: arm using:
-	  - NEON (Advanced SIMD) extensions
 
 endmenu
 
arch/arm/crypto/chacha-glue.c +6 -236

286 lines not on this page · raw message

arch/arm/crypto/chacha-neon-core.S +1 -1
index 13d12f672656b..ddd62b6294a57 100644
@@ -1,7 +1,7 @@
 /*
- * ChaCha/XChaCha NEON helper functions
+ * ChaCha/HChaCha NEON helper functions
  *
  * Copyright (C) 2016 Linaro, Ltd. <[email protected]>
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License version 2 as
Signature · 1 line
2.49.0
#4 lore raw

PATCH 4/9 crypto: arm64/chacha - remove the skcipher algorithms

In reply to Eric Biggers

From: Eric Biggers <[email protected]>

Since crypto/chacha.c now registers chacha20-$(ARCH), xchacha20-$(ARCH), and xchacha12-$(ARCH) skcipher algorithms that use the architecture's ChaCha and HChaCha library functions, individual architectures no longer need to do the same. Therefore, remove the redundant skcipher algorithms and leave just the library functions.

---
 arch/arm64/crypto/Kconfig            |   7 --
 arch/arm64/crypto/chacha-neon-core.S |   2 +-
 arch/arm64/crypto/chacha-neon-glue.c | 144 ++-------------------------
 3 files changed, 7 insertions(+), 146 deletions(-)
arch/arm64/crypto/Kconfig +0 -7
index 3418c8d3c78d4..ce655da0fbeea 100644
@@ -187,20 +187,13 @@ config CRYPTO_AES_ARM64_NEON_BLK
 	  - NEON (Advanced SIMD) extensions
 
 config CRYPTO_CHACHA20_NEON
 	tristate
 	depends on KERNEL_MODE_NEON
-	select CRYPTO_SKCIPHER
 	select CRYPTO_LIB_CHACHA_GENERIC
 	select CRYPTO_ARCH_HAVE_LIB_CHACHA
 	default CRYPTO_LIB_CHACHA_INTERNAL
-	help
-	  Length-preserving ciphers: ChaCha20, XChaCha20, and XChaCha12
-	  stream cipher algorithms
-
-	  Architecture: arm64 using:
-	  - NEON (Advanced SIMD) extensions
 
 config CRYPTO_AES_ARM64_BS
 	tristate "Ciphers: AES, modes: ECB/CBC/CTR/XCTR/XTS modes (bit-sliced NEON)"
 	depends on KERNEL_MODE_NEON
 	select CRYPTO_SKCIPHER
arch/arm64/crypto/chacha-neon-core.S +1 -1
index b70ac76f2610c..80079586ecc7a 100644
@@ -1,7 +1,7 @@
 /*
- * ChaCha/XChaCha NEON helper functions
+ * ChaCha/HChaCha NEON helper functions
  *
  * Copyright (C) 2016-2018 Linaro, Ltd. <[email protected]>
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License version 2 as
arch/arm64/crypto/chacha-neon-glue.c +6 -138

179 lines not on this page · raw message

Signature · 1 line
2.49.0
#5 lore raw

PATCH 5/9 crypto: mips/chacha - remove the skcipher algorithms

In reply to Eric Biggers

From: Eric Biggers <[email protected]>

Since crypto/chacha.c now registers chacha20-$(ARCH), xchacha20-$(ARCH), and xchacha12-$(ARCH) skcipher algorithms that use the architecture's ChaCha and HChaCha library functions, individual architectures no longer need to do the same. Therefore, remove the redundant skcipher algorithms and leave just the library functions.

---
 arch/mips/crypto/Kconfig       |   6 --
 arch/mips/crypto/chacha-glue.c | 131 +--------------------------------
 2 files changed, 3 insertions(+), 134 deletions(-)
arch/mips/crypto/Kconfig +0 -6
index 545fc0e12422d..0189686de3a12 100644
@@ -54,15 +54,9 @@ config CRYPTO_SHA512_OCTEON
 	  Architecture: mips OCTEON using crypto instructions, when available
 
 config CRYPTO_CHACHA_MIPS
 	tristate
 	depends on CPU_MIPS32_R2
-	select CRYPTO_SKCIPHER
 	select CRYPTO_ARCH_HAVE_LIB_CHACHA
 	default CRYPTO_LIB_CHACHA_INTERNAL
-	help
-	  Length-preserving ciphers: ChaCha20, XChaCha20, and XChaCha12
-	  stream cipher algorithms
-
-	  Architecture: MIPS32r2
 
 endmenu
arch/mips/crypto/chacha-glue.c +3 -128

155 lines not on this page · raw message

Signature · 1 line
2.49.0
#6 lore raw

PATCH 6/9 crypto: powerpc/chacha - remove the skcipher algorithms

In reply to Eric Biggers

From: Eric Biggers <[email protected]>

Since crypto/chacha.c now registers chacha20-$(ARCH), xchacha20-$(ARCH), and xchacha12-$(ARCH) skcipher algorithms that use the architecture's ChaCha and HChaCha library functions, individual architectures no longer need to do the same. Therefore, remove the redundant skcipher algorithms and leave just the library functions.

---
 arch/powerpc/crypto/Kconfig           |   8 --
 arch/powerpc/crypto/chacha-p10-glue.c | 145 ++------------------------
 2 files changed, 6 insertions(+), 147 deletions(-)
arch/powerpc/crypto/Kconfig +0 -8
index 370db8192ce62..47dccdd496374 100644
@@ -93,21 +93,13 @@ config CRYPTO_AES_GCM_P10
 	  later CPU. This module supports stitched acceleration for AES/GCM.
 
 config CRYPTO_CHACHA20_P10
 	tristate
 	depends on PPC64 && CPU_LITTLE_ENDIAN && VSX
-	select CRYPTO_SKCIPHER
 	select CRYPTO_LIB_CHACHA_GENERIC
 	select CRYPTO_ARCH_HAVE_LIB_CHACHA
 	default CRYPTO_LIB_CHACHA_INTERNAL
-	help
-	  Length-preserving ciphers: ChaCha20, XChaCha20, and XChaCha12
-	  stream cipher algorithms
-
-	  Architecture: PowerPC64
-	  - Power10 or later
-	  - Little-endian
 
 config CRYPTO_POLY1305_P10
 	tristate "Hash functions: Poly1305 (P10 or later)"
 	depends on PPC64 && CPU_LITTLE_ENDIAN && VSX
 	select CRYPTO_HASH
arch/powerpc/crypto/chacha-p10-glue.c +6 -139

175 lines not on this page · raw message

Signature · 1 line
2.49.0
#7 lore raw

PATCH 7/9 crypto: s390/chacha - remove the skcipher algorithms

In reply to Eric Biggers

From: Eric Biggers <[email protected]>

Since crypto/chacha.c now registers chacha20-$(ARCH), xchacha20-$(ARCH), and xchacha12-$(ARCH) skcipher algorithms that use the architecture's ChaCha and HChaCha library functions, individual architectures no longer need to do the same. Therefore, remove the redundant skcipher algorithms and leave just the library functions.

---
 arch/s390/crypto/Kconfig       |   7 ---
 arch/s390/crypto/chacha-glue.c | 101 +++++----------------------------
 2 files changed, 13 insertions(+), 95 deletions(-)
arch/s390/crypto/Kconfig +0 -7
index 8c4db8b64fa21..055b08f259ab2 100644
@@ -108,20 +108,13 @@ config CRYPTO_DES_S390
 	  As of z196 the CTR mode is hardware accelerated.
 
 config CRYPTO_CHACHA_S390
 	tristate
 	depends on S390
-	select CRYPTO_SKCIPHER
 	select CRYPTO_LIB_CHACHA_GENERIC
 	select CRYPTO_ARCH_HAVE_LIB_CHACHA
 	default CRYPTO_LIB_CHACHA_INTERNAL
-	help
-	  Length-preserving cipher: ChaCha20 stream cipher (RFC 7539)
-
-	  Architecture: s390
-
-	  It is available as of z13.
 
 config CRYPTO_HMAC_S390
 	tristate "Keyed-hash message authentication code: HMAC"
 	depends on S390
 	select CRYPTO_HASH
arch/s390/crypto/chacha-glue.c +13 -88

139 lines not on this page · raw message

Signature · 1 line
2.49.0
#8 lore raw

PATCH 8/9 crypto: x86/chacha - remove the skcipher algorithms

In reply to Eric Biggers

From: Eric Biggers <[email protected]>

Since crypto/chacha.c now registers chacha20-$(ARCH), xchacha20-$(ARCH), and xchacha12-$(ARCH) skcipher algorithms that use the architecture's ChaCha and HChaCha library functions, individual architectures no longer need to do the same. Therefore, remove the redundant skcipher algorithms and leave just the library functions.

---
 arch/x86/crypto/Kconfig       |   9 ---
 arch/x86/crypto/chacha_glue.c | 142 +---------------------------------
 2 files changed, 4 insertions(+), 147 deletions(-)
arch/x86/crypto/Kconfig +0 -9
index 3d948f10c94cd..8cf0a3f55576e 100644
@@ -350,22 +350,13 @@ config CRYPTO_ARIA_GFNI_AVX512_X86_64
 	  Processes 64 blocks in parallel.
 
 config CRYPTO_CHACHA20_X86_64
 	tristate
 	depends on X86 && 64BIT
-	select CRYPTO_SKCIPHER
 	select CRYPTO_LIB_CHACHA_GENERIC
 	select CRYPTO_ARCH_HAVE_LIB_CHACHA
 	default CRYPTO_LIB_CHACHA_INTERNAL
-	help
-	  Length-preserving ciphers: ChaCha20, XChaCha20, and XChaCha12
-	  stream cipher algorithms
-
-	  Architecture: x86_64 using:
-	  - SSSE3 (Supplemental SSE3)
-	  - AVX2 (Advanced Vector Extensions 2)
-	  - AVX-512VL (Advanced Vector Extensions-512VL)
 
 config CRYPTO_AEGIS128_AESNI_SSE2
 	tristate "AEAD ciphers: AEGIS-128 (AES-NI/SSE4.1)"
 	depends on X86 && 64BIT
 	select CRYPTO_AEAD
arch/x86/crypto/chacha_glue.c +4 -138

174 lines not on this page · raw message

Signature · 1 line
2.49.0
#12 lore raw

PATCH 9/9 crypto: chacha - remove <crypto/internal/chacha.h>

In reply to Eric Biggers

From: Eric Biggers <[email protected]>

<crypto/internal/chacha.h> is now included only by crypto/chacha.c, so fold it into there.

---
 crypto/chacha.c                  | 35 +++++++++++++++++++++++++-
 include/crypto/internal/chacha.h | 43 --------------------------------
 2 files changed, 34 insertions(+), 44 deletions(-)
 delete mode 100644 include/crypto/internal/chacha.h
crypto/chacha.c +34 -1
index 2009038c5e56c..5103bc0b2881f 100644
@@ -6,14 +6,47 @@
  * Copyright (C) 2018 Google LLC
  */
 
 #include <linux/unaligned.h>
 #include <crypto/algapi.h>
-#include <crypto/internal/chacha.h>
+#include <crypto/chacha.h>
 #include <crypto/internal/skcipher.h>
 #include <linux/module.h>
 
+struct chacha_ctx {
+	u32 key[8];
+	int nrounds;
+};
+
+static int chacha_setkey(struct crypto_skcipher *tfm,
+			 const u8 *key, unsigned int keysize, int nrounds)
+{
+	struct chacha_ctx *ctx = crypto_skcipher_ctx(tfm);
+	int i;
+
+	if (keysize != CHACHA_KEY_SIZE)
+		return -EINVAL;
+
+	for (i = 0; i < ARRAY_SIZE(ctx->key); i++)
+		ctx->key[i] = get_unaligned_le32(key + i * sizeof(u32));
+
+	ctx->nrounds = nrounds;
+	return 0;
+}
+
+static int chacha20_setkey(struct crypto_skcipher *tfm,
+			   const u8 *key, unsigned int keysize)
+{
+	return chacha_setkey(tfm, key, keysize, 20);
+}
+
+static int chacha12_setkey(struct crypto_skcipher *tfm,
+			   const u8 *key, unsigned int keysize)
+{
+	return chacha_setkey(tfm, key, keysize, 12);
+}
+
 static int chacha_stream_xor(struct skcipher_request *req,
 			     const struct chacha_ctx *ctx, const u8 *iv,
 			     bool arch)
 {
 	struct skcipher_walk walk;
include/crypto/internal/chacha.h deleted +0 -43

43 lines not on this page · raw message

Signature · 1 line
2.49.0
maintainer
#13 lore raw
In reply to Eric Biggers
Quoted 76 lines · Eric Biggers

Currently each architecture exposes ChaCha not only through the library API, but also through the crypto_skcipher API. That requires each architecture to implement essentially the same skcipher glue code.

Following the example of what's been done for crc32 and crc32c, eliminate this redundancy by making crypto/chacha.c register both the generic and architecture-optimized skcipher algorithms, implemented on top of the appropriate library functions. This removes almost 800 lines of code and disentangles the library code from the skcipher API.

From what I remember, the following are the reasons why it wasn't just done this way originally. But none of these really hold water:

- The skcipher code was there first, so it may have seemed more natural to add onto it rather than replace it.

- Architectures could register multiple skcipher algorithms using different CPU features and have them all be tested in a single boot. This was convenient in theory, but it never really worked properly. It didn't apply to the library code, the x86 ChaCha code wasn't actually doing this (it used static keys instead), and this cannot catch bugs like accidentally using an AVX instruction in SSE code. Instead, a correct solution, which also doesn't require any special kernel support, is to just boot the kernel in QEMU using different -cpu arguments as needed to test all the code.

- There was a concern about changing cra_driver_names potentially breaking users. But in practice users rely on cra_name, not cra_driver_name. We already change, add, and remove cra_driver_names occasionally for various reasons. And even if someone was relying on a specific cra_driver_name, there are some more lightweight compatibility tricks that could be used.

- There was a desire for users to be able to override the kernel's choice of ChaCha implementation by blacklisting the arch-optimized ChaCha module. But that already became mostly impossible when the library functions were added to the same module. And in practice users don't do this anyway. Even if, hypothetically, someone really needed to do this and for some reason the kernel couldn't be fixed to make the right choice in their case automatically, there are other ways this could be implemented such as a module parameter.

Eric Biggers (9):
crypto: riscv/chacha - implement library instead of skcipher
crypto: chacha - centralize the skcipher wrappers for arch code
crypto: arm/chacha - remove the redundant skcipher algorithms
crypto: arm64/chacha - remove the skcipher algorithms
crypto: mips/chacha - remove the skcipher algorithms
crypto: powerpc/chacha - remove the skcipher algorithms
crypto: s390/chacha - remove the skcipher algorithms
crypto: x86/chacha - remove the skcipher algorithms
crypto: chacha - remove <crypto/internal/chacha.h>

arch/arm/crypto/Kconfig                 |   7 -
arch/arm/crypto/chacha-glue.c           | 243 +---------------------
arch/arm/crypto/chacha-neon-core.S      |   2 +-
arch/arm64/crypto/Kconfig               |   7 -
arch/arm64/crypto/chacha-neon-core.S    |   2 +-
arch/arm64/crypto/chacha-neon-glue.c    | 146 +------------
arch/mips/crypto/Kconfig                |   6 -
arch/mips/crypto/chacha-glue.c          | 131 +-----------
arch/powerpc/crypto/Kconfig             |   8 -
arch/powerpc/crypto/chacha-p10-glue.c   | 147 +-------------
arch/riscv/crypto/Kconfig               |  11 +-
arch/riscv/crypto/chacha-riscv64-glue.c | 112 ++++------
arch/riscv/crypto/chacha-riscv64-zvkb.S |  71 +++----
arch/s390/crypto/Kconfig                |   7 -
arch/s390/crypto/chacha-glue.c          |  99 ++-------
arch/x86/crypto/Kconfig                 |   9 -
arch/x86/crypto/chacha_glue.c           | 144 +------------
crypto/Makefile                         |   3 +-
crypto/chacha.c                         | 260 ++++++++++++++++++++++++
crypto/chacha_generic.c                 | 139 -------------
include/crypto/chacha.h                 |   9 +
include/crypto/internal/chacha.h        |  43 ----
22 files changed, 413 insertions(+), 1193 deletions(-)
Quoted 5 lines

create mode 100644 crypto/chacha.c
delete mode 100644 crypto/chacha_generic.c delete mode 100644 include/crypto/internal/chacha.h

base-commit: 56f944529ec2292cbe63377a76df3759d702dd39

All applied. Thanks.

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
maintainer
#9 lore raw

Re: [PATCH 8/9] crypto: x86/chacha - remove the skcipher algorithms

In reply to Eric Biggers
Quoted 9 lines · Eric Biggers
arch/x86/crypto/chacha_glue.c +0 -6
-static void __exit chacha_simd_mod_fini(void)
-{
-       if (IS_REACHABLE(CONFIG_CRYPTO_SKCIPHER) && boot_cpu_has(X86_FEATURE_SSSE3))
-               crypto_unregister_skciphers(algs, ARRAY_SIZE(algs));
-}
-

arch_initcall(chacha_simd_mod_init);
-module_exit(chacha_simd_mod_fini);

Why did you remove the ability to remove the module?

Cheers,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#10 lore raw

Re: [PATCH 8/9] crypto: x86/chacha - remove the skcipher algorithms

Quoted 13 lines · Herbert Xu
Quoted 9 lines · Eric Biggers
arch/x86/crypto/chacha_glue.c +0 -6
-static void __exit chacha_simd_mod_fini(void)
-{
-       if (IS_REACHABLE(CONFIG_CRYPTO_SKCIPHER) && boot_cpu_has(X86_FEATURE_SSSE3))
-               crypto_unregister_skciphers(algs, ARRAY_SIZE(algs));
-}
-

arch_initcall(chacha_simd_mod_init);
-module_exit(chacha_simd_mod_fini);

Why did you remove the ability to remove the module?

Well, I forgot that an empty module_exit needs to be kept around for the module to be removable. I'll send a patch that adds these back in, though I'm doubtful that anyone ever removes these modules in practice.

- Eric

maintainer
#11 lore raw

Re: [PATCH 8/9] crypto: x86/chacha - remove the skcipher algorithms

Quoted 4 lines · Eric Biggers

Well, I forgot that an empty module_exit needs to be kept around for the module to be removable. I'll send a patch that adds these back in, though I'm doubtful that anyone ever removes these modules in practice.

I just tried to remove chacha_x86_64 in order to make sure that I was actually using the arch-optimised version of chacha and that's how I noticed.

I remove the algorithm modules all the time because it's much easier to rebuild one module rather than the whole kernel.

Cheers,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt