RISC-V crypto with reworked asm files
Subsystems (2)
v3 cover RISC-V crypto with reworked asm files
This patchset, which applies to v6.8-rc1, adds cryptographic algorithm implementations accelerated using the RISC-V vector crypto extensions
(https://github.com/riscv/riscv-crypto/releases/download/v1.0.0/riscv-crypto-spec-vector.pdf)
and RISC-V vector extension
(https://github.com/riscv/riscv-v-spec/releases/download/v1.0/riscv-v-spec-1.0.pdf).
The following algorithms are included: AES in ECB, CBC, CTR, and XTS modes; ChaCha20; GHASH; SHA-2; SM3; and SM4.
In general, the assembly code requires a 64-bit RISC-V CPU with VLEN >= 128, little endian byte order, and vector unaligned access support. The ECB, CTR, XTS, and ChaCha20 code is designed to naturally scale up to larger VLEN values. Building the assembly code requires tip-of-tree binutils (future 2.42) or tip-of-tree clang (future 18.x). All algorithms pass testing in QEMU, using CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y. Much of the assembly code is derived from OpenSSL code that was added by https://github.com/openssl/openssl/pull/21923.
It's been cleaned up for integration with the kernel, e.g. reducing code duplication, eliminating use of .inst and perlasm, and fixing a few bugs.
This patchset incorporates the work of multiple people, including Jerry Shih, Heiko Stuebner, Christoph Müllner, Phoebe Chen, Charalampos Mitrodimas, and myself. This patchset went through several versions from Heiko (last version
https://lore.kernel.org/linux-crypto/[email protected]),
then several versions from Jerry (last version:
https://lore.kernel.org/linux-crypto/[email protected]),
then finally several versions from me. Thanks to everyone who has contributed to this patchset or its prerequisites. Since v6.8-rc1, all prerequisite kernel patches are upstream. I think this is now ready, and I'd like for it to be applied for 6.9, either to the crypto or riscv tree (at maintainers' choice).
Below is the changelog for my versions of the patchset. For the changelog of the older versions, see the above links.
Changed in v3:
- Fixed a bug in the AES-XTS implementation where it assumed the CPU always set vl to the maximum possible value. This was okay for QEMU, but the vector spec allows CPUs to have different behavior.
- Increased the LMUL for AES-ECB to 8, as the registers are available.
- Fixed some license text that I had mistakenly changed when doing a find-and-replace of code.
- Addressed a checkpatch warning by not including filename in file.
- Rename some labels.
- Constify a variable.
Changed in v2:
- Merged the AES modules together to prevent a build error.
- Only unregister AES algorithms that were registered.
- Corrected walksize properties to match the LMUL used by asm code.
- Simplified the CTR and XTS glue code slightly.
- Minor cleanups.
Changed in v1:
- Refer to my cover letter
https://lore.kernel.org/linux-crypto/[email protected]/
Eric Biggers (1):
RISC-V: add TOOLCHAIN_HAS_VECTOR_CRYPTO
Heiko Stuebner (2):
RISC-V: add helper function to read the vector VLEN
RISC-V: hook new crypto subdir into build-system
Jerry Shih (7):
crypto: riscv - add vector crypto accelerated AES-{ECB,CBC,CTR,XTS}
crypto: riscv - add vector crypto accelerated ChaCha20
crypto: riscv - add vector crypto accelerated GHASH
crypto: riscv - add vector crypto accelerated SHA-{256,224}
crypto: riscv - add vector crypto accelerated SHA-{512,384}
crypto: riscv - add vector crypto accelerated SM3
crypto: riscv - add vector crypto accelerated SM4
arch/riscv/Kbuild | 1 + arch/riscv/Kconfig | 7 + arch/riscv/crypto/Kconfig | 93 +++ arch/riscv/crypto/Makefile | 23 + arch/riscv/crypto/aes-macros.S | 156 +++++ arch/riscv/crypto/aes-riscv64-glue.c | 550 ++++++++++++++++++ .../crypto/aes-riscv64-zvkned-zvbb-zvkg.S | 312 ++++++++++ arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S | 146 +++++ arch/riscv/crypto/aes-riscv64-zvkned.S | 180 ++++++ arch/riscv/crypto/chacha-riscv64-glue.c | 101 ++++ arch/riscv/crypto/chacha-riscv64-zvkb.S | 294 ++++++++++ arch/riscv/crypto/ghash-riscv64-glue.c | 168 ++++++ arch/riscv/crypto/ghash-riscv64-zvkg.S | 72 +++ arch/riscv/crypto/sha256-riscv64-glue.c | 137 +++++ .../sha256-riscv64-zvknha_or_zvknhb-zvkb.S | 225 +++++++ arch/riscv/crypto/sha512-riscv64-glue.c | 133 +++++ .../riscv/crypto/sha512-riscv64-zvknhb-zvkb.S | 203 +++++++ arch/riscv/crypto/sm3-riscv64-glue.c | 112 ++++ arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S | 123 ++++ arch/riscv/crypto/sm4-riscv64-glue.c | 107 ++++ arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.S | 117 ++++ arch/riscv/include/asm/vector.h | 11 + crypto/Kconfig | 3 + 23 files changed, 3274 insertions(+) create mode 100644 arch/riscv/crypto/Kconfig create mode 100644 arch/riscv/crypto/Makefile create mode 100644 arch/riscv/crypto/aes-macros.S create mode 100644 arch/riscv/crypto/aes-riscv64-glue.c create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvbb-zvkg.S create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned.S create mode 100644 arch/riscv/crypto/chacha-riscv64-glue.c create mode 100644 arch/riscv/crypto/chacha-riscv64-zvkb.S create mode 100644 arch/riscv/crypto/ghash-riscv64-glue.c create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkg.S create mode 100644 arch/riscv/crypto/sha256-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha256-riscv64-zvknha_or_zvknhb-zvkb.S create mode 100644 arch/riscv/crypto/sha512-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha512-riscv64-zvknhb-zvkb.S create mode 100644 arch/riscv/crypto/sm3-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S create mode 100644 arch/riscv/crypto/sm4-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.S
base-commit: 6613476e225e090cc9aad49be7fa504e290dd33d
Signature · 1 line
2.43.0
PATCH v3 1/10 RISC-V: add helper function to read the vector VLEN
From: Heiko Stuebner <[email protected]>
VLEN describes the length of each vector register and some instructions need specific minimal VLENs to work correctly.
The vector code already includes a variable riscv_v_vsize that contains the value of "32 vector registers with vlenb length" that gets filled during boot. vlenb is the value contained in the CSR_VLENB register and the value represents "VLEN / 8".
So add riscv_vector_vlen() to return the actual VLEN value for in-kernel users when they need to check the available VLEN.
--- arch/riscv/include/asm/vector.h | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/riscv/include/asm/vector.h +11 -0
@@ -277,11 +277,22 @@ static inline bool riscv_v_vstate_ctrl_user_allowed(void) { return false; }#define riscv_v_vstate_restore (vstate, regs) do {} while (0)#define __switch_to_vector (__prev, __next) do {} while (0)#define riscv_v_vstate_off (regs) do {} while (0)#define riscv_v_vstate_on (regs) do {} while (0)#define riscv_v_thread_free (tsk) do {} while (0)#define riscv_v_setup_ctx_cache () do {} while (0)#define riscv_v_thread_alloc (tsk) do {} while (0)#endif /* CONFIG_RISCV_ISA_V */ +/* +* Return the implementation's vlen value. +* +* riscv_v_vsize contains the value of "32 vector registers with vlenb length" +* so rebuild the vlen value in bits from it. +*/ +static inline int riscv_vector_vlen (void ) +{ +return riscv_v_vsize /32 *8 ; +} +#endif /* ! __ASM_RISCV_VECTOR_H */
Signature · 1 line
2.43.0
PATCH v3 2/10 RISC-V: add TOOLCHAIN_HAS_VECTOR_CRYPTO
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Add a kconfig symbol that indicates whether the toolchain supports the vector crypto extensions. This is needed by the RISC-V crypto code.
--- arch/riscv/Kconfig | 7 +++++++ 1 file changed, 7 insertions(+)
arch/riscv/Kconfig +7 -0
@@ -571,20 +571,27 @@ config RISCV_ISA_V_PREEMPTIVE consumption due to the allocation of per-task's kernel Vector context. config TOOLCHAIN_HAS_ZBB bool default y depends on !64BIT || $(cc-option,-mabi=lp64 -march=rv64ima_zbb) depends on !32BIT || $(cc-option,-mabi=ilp32 -march=rv32ima_zbb) depends on LLD_VERSION >= 150000 || LD_VERSION >= 23900 depends on AS_HAS_OPTION_ARCH +# This symbol indicates that the toolchain supports all v1.0 vector crypto +# extensions, including Zvk*, Zvbb, and Zvbc. LLVM added all of these at once. +# binutils added all except Zvkb, then added Zvkb. So we just check for Zvkb. +config TOOLCHAIN_HAS_VECTOR_CRYPTO + def_bool $(as-instr, .option arch$(comma) +zvkb) + depends on AS_HAS_OPTION_ARCH + config RISCV_ISA_ZBB bool "Zbb extension support for bit manipulation instructions" depends on TOOLCHAIN_HAS_ZBB depends on MMU depends on RISCV_ALTERNATIVE default y help Adds support to dynamically detect the presence of the ZBB extension (basic bit manipulation) and enable its usage.
Signature · 1 line
2.43.0
PATCH v3 3/10 RISC-V: hook new crypto subdir into build-system
In reply to Eric BiggersFrom: Heiko Stuebner <[email protected]>
Create a crypto subdirectory for added accelerated cryptography routines and hook it into the riscv Kbuild and the main crypto Kconfig.
--- arch/riscv/Kbuild | 1 + arch/riscv/crypto/Kconfig | 5 +++++ arch/riscv/crypto/Makefile | 1 + crypto/Kconfig | 3 +++ 4 files changed, 10 insertions(+) create mode 100644 arch/riscv/crypto/Kconfig create mode 100644 arch/riscv/crypto/Makefile
arch/riscv/Kbuild +1 -0
@@ -1,11 +1,12 @@ # SPDX-License-Identifier: GPL-2.0-only obj-y += kernel/ mm/ net/ obj-$(CONFIG_BUILTIN_DTB) += boot/dts/ +obj-$(CONFIG_CRYPTO) += crypto/ obj-y += errata/ obj-$(CONFIG_KVM) += kvm/ obj-$(CONFIG_ARCH_SUPPORTS_KEXEC_PURGATORY) += purgatory/ # for cleaning subdir- += boot
arch/riscv/crypto/Kconfig new file +5 -0
@@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + +menu "Accelerated Cryptographic Algorithms for CPU (riscv)" + +endmenu
arch/riscv/crypto/Makefile new file +1 -0
@@ -0,0 +1 @@ +# SPDX-License-Identifier: GPL-2.0-only
crypto/Kconfig +3 -0
@@ -1489,20 +1489,23 @@ source "arch/arm64/crypto/Kconfig" endif if LOONGARCH source "arch/loongarch/crypto/Kconfig" endif if MIPS source "arch/mips/crypto/Kconfig" endif if PPC source "arch/powerpc/crypto/Kconfig" endif +if RISCV +source "arch/riscv/crypto/Kconfig" +endif if S390 source "arch/s390/crypto/Kconfig" endif if SPARC source "arch/sparc/crypto/Kconfig" endif if X86 source "arch/x86/crypto/Kconfig" endif endif
Signature · 1 line
2.43.0
PATCH v3 4/10 crypto: riscv - add vector crypto accelerated AES-{ECB,CBC,CTR,XTS}
In reply to Eric BiggersFrom: Jerry Shih <[email protected]>
Add implementations of AES-ECB, AES-CBC, AES-CTR, and AES-XTS, as well as bare (single-block) AES, using the RISC-V vector crypto extensions. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using regular .S files instead of the so-called perlasm, using the assembler instead of bare '.inst', greatly reducing code duplication, supporting AES-192, and making the code use the same AES key structure as the C code.
--- arch/riscv/crypto/Kconfig | 16 + arch/riscv/crypto/Makefile | 4 + arch/riscv/crypto/aes-macros.S | 156 +++++ arch/riscv/crypto/aes-riscv64-glue.c | 550 ++++++++++++++++++ .../crypto/aes-riscv64-zvkned-zvbb-zvkg.S | 312 ++++++++++ arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S | 146 +++++ arch/riscv/crypto/aes-riscv64-zvkned.S | 180 ++++++ 7 files changed, 1364 insertions(+) create mode 100644 arch/riscv/crypto/aes-macros.S create mode 100644 arch/riscv/crypto/aes-riscv64-glue.c create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvbb-zvkg.S create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned.S
arch/riscv/crypto/Kconfig +16 -0
@@ -1,5 +1,21 @@ # SPDX-License-Identifier: GPL-2.0 menu "Accelerated Cryptographic Algorithms for CPU (riscv)" +config CRYPTO_AES_RISCV64 + tristate "Ciphers: AES, modes: ECB, CBC, CTR, XTS" + depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO + select CRYPTO_ALGAPI + select CRYPTO_LIB_AES + select CRYPTO_SKCIPHER + help + Block cipher: AES cipher algorithms + Length-preserving ciphers: AES with ECB, CBC, CTR, XTS + + Architecture: riscv64 using: + - Zvkned vector crypto extension + - Zvbb vector extension (XTS) + - Zvkb vector crypto extension (CTR) + - Zvkg vector crypto extension (XTS) + endmenu
arch/riscv/crypto/Makefile +4 -0
@@ -1 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only + +obj-$(CONFIG_CRYPTO_AES_RISCV64) += aes-riscv64.o +aes-riscv64-y := aes-riscv64-glue.o aes-riscv64-zvkned.o \ + aes-riscv64-zvkned-zvbb-zvkg.o aes-riscv64-zvkned-zvkb.o
arch/riscv/crypto/aes-macros.S new file +156 -0
156 lines not on this page · raw message
arch/riscv/crypto/aes-riscv64-glue.c new file +550 -0
550 lines not on this page · raw message
arch/riscv/crypto/aes-riscv64-zvkned-zvbb-zvkg.S new file +312 -0
312 lines not on this page · raw message
arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S new file +146 -0
146 lines not on this page · raw message
arch/riscv/crypto/aes-riscv64-zvkned.S new file +180 -0
180 lines not on this page · raw message
Signature · 1 line
2.43.0
PATCH v3 5/10 crypto: riscv - add vector crypto accelerated ChaCha20
In reply to Eric BiggersFrom: Jerry Shih <[email protected]>
Add an implementation of ChaCha20 using the Zvkb extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and reducing code duplication.
--- arch/riscv/crypto/Kconfig | 11 + arch/riscv/crypto/Makefile | 3 + arch/riscv/crypto/chacha-riscv64-glue.c | 101 ++++++++ arch/riscv/crypto/chacha-riscv64-zvkb.S | 294 ++++++++++++++++++++++++ 4 files changed, 409 insertions(+) create mode 100644 arch/riscv/crypto/chacha-riscv64-glue.c create mode 100644 arch/riscv/crypto/chacha-riscv64-zvkb.S
arch/riscv/crypto/Kconfig +11 -0
@@ -11,11 +11,22 @@ config CRYPTO_AES_RISCV64 help Block cipher: AES cipher algorithms Length-preserving ciphers: AES with ECB, CBC, CTR, XTS Architecture: riscv64 using: - Zvkned vector crypto extension - Zvbb vector extension (XTS) - Zvkb vector crypto extension (CTR) - Zvkg vector crypto extension (XTS) +config CRYPTO_CHACHA_RISCV64 + tristate "Ciphers: ChaCha" + depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO + select CRYPTO_SKCIPHER + select CRYPTO_LIB_CHACHA_GENERIC + help + Length-preserving ciphers: ChaCha20 stream cipher algorithm + + Architecture: riscv64 using: + - Zvkb vector crypto extension + endmenu
arch/riscv/crypto/Makefile +3 -0
@@ -1,5 +1,8 @@ # SPDX-License-Identifier: GPL-2.0-only obj-$(CONFIG_CRYPTO_AES_RISCV64) += aes-riscv64.o aes-riscv64-y := aes-riscv64-glue.o aes-riscv64-zvkned.o \ aes-riscv64-zvkned-zvbb-zvkg.o aes-riscv64-zvkned-zvkb.o + +obj-$(CONFIG_CRYPTO_CHACHA_RISCV64) += chacha-riscv64.o +chacha-riscv64-y := chacha-riscv64-glue.o chacha-riscv64-zvkb.o
arch/riscv/crypto/chacha-riscv64-glue.c new file +101 -0
101 lines not on this page · raw message
arch/riscv/crypto/chacha-riscv64-zvkb.S new file +294 -0
294 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v3 05/10] crypto: riscv - add vector crypto accelerated ChaCha20
On 2024/1/22 08:19, Eric Biggers wrote:> From: Jerry Shih <[email protected]>
Quoted 9 lines
Add an implementation of ChaCha20 using the Zvkb extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and reducing code duplication.
Quoted 4 lines
Signed-off-by Jerry Shih <[email protected]>Co-developed-by Eric Biggers <[email protected]>Signed-off-by Eric Biggers <[email protected]>---
Hi, Jerry Shih and Eric Biggers,
Since this implementation is derived from OpenSSL, I have an issue on OpenSSL about this chacha20 implementation using Zvkb [1] and see a fixes PR by Jerry Shih [2].
I wonder will kernel need to port those fixes to handle the non-multiple block length of the chacha20 cipher? If yes, we should port it immediately to catch up with the v6.9 release if we can.
[1] https://github.com/openssl/openssl/issues/24070
[2] https://github.com/openssl/openssl/pull/24097
Thanks,
Yangyu Chen
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Re: [PATCH v3 05/10] crypto: riscv - add vector crypto accelerated ChaCha20
Quoted 25 lines · Yangyu Chen
On 2024/1/22 08:19, Eric Biggers wrote:> From: Jerry Shih <[email protected]>
Quoted 9 lines
Add an implementation of ChaCha20 using the Zvkb extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and reducing code duplication.
Quoted 4 lines
Signed-off-by Jerry Shih <[email protected]>Co-developed-by Eric Biggers <[email protected]>Signed-off-by Eric Biggers <[email protected]>---
Hi, Jerry Shih and Eric Biggers,
Since this implementation is derived from OpenSSL, I have an issue on OpenSSL about this chacha20 implementation using Zvkb [1] and see a fixes PR by Jerry Shih [2].
I wonder will kernel need to port those fixes to handle the non-multiple block length of the chacha20 cipher? If yes, we should port it immediately to catch up with the v6.9 release if we can.
Quoted 6 lines
[1] https://github.com/openssl/openssl/issues/24070
[2] https://github.com/openssl/openssl/pull/24097Thanks,
Yangyu Chen
There is no non-multiple block size issue here. It's already handled by the gluing part.
-Jerry
PATCH v3 6/10 crypto: riscv - add vector crypto accelerated GHASH
In reply to Eric BiggersFrom: Jerry Shih <[email protected]>
Add an implementation of GHASH using the zvkg extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', reducing code duplication, and eliminating unnecessary endianness conversions.
--- arch/riscv/crypto/Kconfig | 10 ++ arch/riscv/crypto/Makefile | 3 + arch/riscv/crypto/ghash-riscv64-glue.c | 168 +++++++++++++++++++++++++ arch/riscv/crypto/ghash-riscv64-zvkg.S | 72 +++++++++++ 4 files changed, 253 insertions(+) create mode 100644 arch/riscv/crypto/ghash-riscv64-glue.c create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkg.S
arch/riscv/crypto/Kconfig +10 -0
@@ -22,11 +22,21 @@ config CRYPTO_CHACHA_RISCV64 tristate "Ciphers: ChaCha" depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO select CRYPTO_SKCIPHER select CRYPTO_LIB_CHACHA_GENERIC help Length-preserving ciphers: ChaCha20 stream cipher algorithm Architecture: riscv64 using: - Zvkb vector crypto extension +config CRYPTO_GHASH_RISCV64 + tristate "Hash functions: GHASH" + depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO + select CRYPTO_GCM + help + GCM GHASH function (NIST SP 800-38D) + + Architecture: riscv64 using: + - Zvkg vector crypto extension + endmenu
arch/riscv/crypto/Makefile +3 -0
@@ -1,8 +1,11 @@ # SPDX-License-Identifier: GPL-2.0-only obj-$(CONFIG_CRYPTO_AES_RISCV64) += aes-riscv64.o aes-riscv64-y := aes-riscv64-glue.o aes-riscv64-zvkned.o \ aes-riscv64-zvkned-zvbb-zvkg.o aes-riscv64-zvkned-zvkb.o obj-$(CONFIG_CRYPTO_CHACHA_RISCV64) += chacha-riscv64.o chacha-riscv64-y := chacha-riscv64-glue.o chacha-riscv64-zvkb.o + +obj-$(CONFIG_CRYPTO_GHASH_RISCV64) += ghash-riscv64.o +ghash-riscv64-y := ghash-riscv64-glue.o ghash-riscv64-zvkg.o
arch/riscv/crypto/ghash-riscv64-glue.c new file +168 -0
168 lines not on this page · raw message
arch/riscv/crypto/ghash-riscv64-zvkg.S new file +72 -0
72 lines not on this page · raw message
Signature · 1 line
2.43.0
PATCH v3 7/10 crypto: riscv - add vector crypto accelerated SHA-{256,224}
In reply to Eric BiggersFrom: Jerry Shih <[email protected]>
Add an implementation of SHA-256 and SHA-224 using the Zvknha or Zvknhb extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and greatly reducing code duplication.
--- arch/riscv/crypto/Kconfig | 11 + arch/riscv/crypto/Makefile | 3 + arch/riscv/crypto/sha256-riscv64-glue.c | 137 +++++++++++ .../sha256-riscv64-zvknha_or_zvknhb-zvkb.S | 225 ++++++++++++++++++ 4 files changed, 376 insertions(+) create mode 100644 arch/riscv/crypto/sha256-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha256-riscv64-zvknha_or_zvknhb-zvkb.S
arch/riscv/crypto/Kconfig +11 -0
@@ -32,11 +32,22 @@ config CRYPTO_CHACHA_RISCV64 config CRYPTO_GHASH_RISCV64 tristate "Hash functions: GHASH" depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO select CRYPTO_GCM help GCM GHASH function (NIST SP 800-38D) Architecture: riscv64 using: - Zvkg vector crypto extension +config CRYPTO_SHA256_RISCV64 + tristate "Hash functions: SHA-224 and SHA-256" + depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO + select CRYPTO_SHA256 + help + SHA-224 and SHA-256 secure hash algorithm (FIPS 180) + + Architecture: riscv64 using: + - Zvknha or Zvknhb vector crypto extensions + - Zvkb vector crypto extension + endmenu
arch/riscv/crypto/Makefile +3 -0
@@ -2,10 +2,13 @@ obj-$(CONFIG_CRYPTO_AES_RISCV64) += aes-riscv64.o aes-riscv64-y := aes-riscv64-glue.o aes-riscv64-zvkned.o \ aes-riscv64-zvkned-zvbb-zvkg.o aes-riscv64-zvkned-zvkb.o obj-$(CONFIG_CRYPTO_CHACHA_RISCV64) += chacha-riscv64.o chacha-riscv64-y := chacha-riscv64-glue.o chacha-riscv64-zvkb.o obj-$(CONFIG_CRYPTO_GHASH_RISCV64) += ghash-riscv64.o ghash-riscv64-y := ghash-riscv64-glue.o ghash-riscv64-zvkg.o + +obj-$(CONFIG_CRYPTO_SHA256_RISCV64) += sha256-riscv64.o +sha256-riscv64-y := sha256-riscv64-glue.o sha256-riscv64-zvknha_or_zvknhb-zvkb.o
arch/riscv/crypto/sha256-riscv64-glue.c new file +137 -0
137 lines not on this page · raw message
arch/riscv/crypto/sha256-riscv64-zvknha_or_zvknhb-zvkb.S new file +225 -0
225 lines not on this page · raw message
Signature · 1 line
2.43.0
PATCH v3 8/10 crypto: riscv - add vector crypto accelerated SHA-{512,384}
In reply to Eric BiggersFrom: Jerry Shih <[email protected]>
Add an implementation of SHA-512 and SHA-384 using the Zvknhb extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and greatly reducing code duplication.
--- arch/riscv/crypto/Kconfig | 11 + arch/riscv/crypto/Makefile | 3 + arch/riscv/crypto/sha512-riscv64-glue.c | 133 ++++++++++++ .../riscv/crypto/sha512-riscv64-zvknhb-zvkb.S | 203 ++++++++++++++++++ 4 files changed, 350 insertions(+) create mode 100644 arch/riscv/crypto/sha512-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha512-riscv64-zvknhb-zvkb.S
arch/riscv/crypto/Kconfig +11 -0
@@ -43,11 +43,22 @@ config CRYPTO_SHA256_RISCV64 tristate "Hash functions: SHA-224 and SHA-256" depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO select CRYPTO_SHA256 help SHA-224 and SHA-256 secure hash algorithm (FIPS 180) Architecture: riscv64 using: - Zvknha or Zvknhb vector crypto extensions - Zvkb vector crypto extension +config CRYPTO_SHA512_RISCV64 + tristate "Hash functions: SHA-384 and SHA-512" + depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO + select CRYPTO_SHA512 + help + SHA-384 and SHA-512 secure hash algorithm (FIPS 180) + + Architecture: riscv64 using: + - Zvknhb vector crypto extension + - Zvkb vector crypto extension + endmenu
arch/riscv/crypto/Makefile +3 -0
@@ -5,10 +5,13 @@ aes-riscv64-y := aes-riscv64-glue.o aes-riscv64-zvkned.o \ aes-riscv64-zvkned-zvbb-zvkg.o aes-riscv64-zvkned-zvkb.o obj-$(CONFIG_CRYPTO_CHACHA_RISCV64) += chacha-riscv64.o chacha-riscv64-y := chacha-riscv64-glue.o chacha-riscv64-zvkb.o obj-$(CONFIG_CRYPTO_GHASH_RISCV64) += ghash-riscv64.o ghash-riscv64-y := ghash-riscv64-glue.o ghash-riscv64-zvkg.o obj-$(CONFIG_CRYPTO_SHA256_RISCV64) += sha256-riscv64.o sha256-riscv64-y := sha256-riscv64-glue.o sha256-riscv64-zvknha_or_zvknhb-zvkb.o + +obj-$(CONFIG_CRYPTO_SHA512_RISCV64) += sha512-riscv64.o +sha512-riscv64-y := sha512-riscv64-glue.o sha512-riscv64-zvknhb-zvkb.o
arch/riscv/crypto/sha512-riscv64-glue.c new file +133 -0
133 lines not on this page · raw message
arch/riscv/crypto/sha512-riscv64-zvknhb-zvkb.S new file +203 -0
203 lines not on this page · raw message
Signature · 1 line
2.43.0
PATCH v3 9/10 crypto: riscv - add vector crypto accelerated SM3
In reply to Eric BiggersFrom: Jerry Shih <[email protected]>
Add an implementation of SM3 using the Zvksh extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and greatly reducing code duplication.
--- arch/riscv/crypto/Kconfig | 12 ++ arch/riscv/crypto/Makefile | 3 + arch/riscv/crypto/sm3-riscv64-glue.c | 112 +++++++++++++++++++ arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S | 123 +++++++++++++++++++++ 4 files changed, 250 insertions(+) create mode 100644 arch/riscv/crypto/sm3-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S
arch/riscv/crypto/Kconfig +12 -0
@@ -54,11 +54,23 @@ config CRYPTO_SHA512_RISCV64 tristate "Hash functions: SHA-384 and SHA-512" depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO select CRYPTO_SHA512 help SHA-384 and SHA-512 secure hash algorithm (FIPS 180) Architecture: riscv64 using: - Zvknhb vector crypto extension - Zvkb vector crypto extension +config CRYPTO_SM3_RISCV64 + tristate "Hash functions: SM3 (ShangMi 3)" + depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO + select CRYPTO_HASH + select CRYPTO_SM3 + help + SM3 (ShangMi 3) secure hash function (OSCCA GM/T 0004-2012) + + Architecture: riscv64 using: + - Zvksh vector crypto extension + - Zvkb vector crypto extension + endmenu
arch/riscv/crypto/Makefile +3 -0
@@ -8,10 +8,13 @@ obj-$(CONFIG_CRYPTO_CHACHA_RISCV64) += chacha-riscv64.o chacha-riscv64-y := chacha-riscv64-glue.o chacha-riscv64-zvkb.o obj-$(CONFIG_CRYPTO_GHASH_RISCV64) += ghash-riscv64.o ghash-riscv64-y := ghash-riscv64-glue.o ghash-riscv64-zvkg.o obj-$(CONFIG_CRYPTO_SHA256_RISCV64) += sha256-riscv64.o sha256-riscv64-y := sha256-riscv64-glue.o sha256-riscv64-zvknha_or_zvknhb-zvkb.o obj-$(CONFIG_CRYPTO_SHA512_RISCV64) += sha512-riscv64.o sha512-riscv64-y := sha512-riscv64-glue.o sha512-riscv64-zvknhb-zvkb.o + +obj-$(CONFIG_CRYPTO_SM3_RISCV64) += sm3-riscv64.o +sm3-riscv64-y := sm3-riscv64-glue.o sm3-riscv64-zvksh-zvkb.o
arch/riscv/crypto/sm3-riscv64-glue.c new file +112 -0
112 lines not on this page · raw message
arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S new file +123 -0
123 lines not on this page · raw message
Signature · 1 line
2.43.0
PATCH v3 10/10 crypto: riscv - add vector crypto accelerated SM4
In reply to Eric BiggersFrom: Jerry Shih <[email protected]>
Add an implementation of SM4 using the Zvksed extension. The assembly code is derived from OpenSSL code (openssl/openssl#21923) that was dual-licensed so that it could be reused in the kernel. Nevertheless, the assembly has been significantly reworked for integration with the kernel, for example by using a regular .S file instead of the so-called perlasm, using the assembler instead of bare '.inst', and greatly reducing code duplication.
--- arch/riscv/crypto/Kconfig | 17 +++ arch/riscv/crypto/Makefile | 3 + arch/riscv/crypto/sm4-riscv64-glue.c | 107 ++++++++++++++++++ arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.S | 117 ++++++++++++++++++++ 4 files changed, 244 insertions(+) create mode 100644 arch/riscv/crypto/sm4-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.S
arch/riscv/crypto/Kconfig +17 -0
@@ -66,11 +66,28 @@ config CRYPTO_SM3_RISCV64 depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO select CRYPTO_HASH select CRYPTO_SM3 help SM3 (ShangMi 3) secure hash function (OSCCA GM/T 0004-2012) Architecture: riscv64 using: - Zvksh vector crypto extension - Zvkb vector crypto extension +config CRYPTO_SM4_RISCV64 + tristate "Ciphers: SM4 (ShangMi 4)" + depends on 64BIT && RISCV_ISA_V && TOOLCHAIN_HAS_VECTOR_CRYPTO + select CRYPTO_ALGAPI + select CRYPTO_SM4 + help + SM4 block cipher algorithm (OSCCA GB/T 32907-2016, + ISO/IEC 18033-3:2010/Amd 1:2021) + + SM4 (GBT.32907-2016) is a cryptographic standard issued by the + Organization of State Commercial Administration of China (OSCCA) + as an authorized cryptographic algorithm for use within China. + + Architecture: riscv64 using: + - Zvksed vector crypto extension + - Zvkb vector crypto extension + endmenu
arch/riscv/crypto/Makefile +3 -0
@@ -11,10 +11,13 @@ obj-$(CONFIG_CRYPTO_GHASH_RISCV64) += ghash-riscv64.o ghash-riscv64-y := ghash-riscv64-glue.o ghash-riscv64-zvkg.o obj-$(CONFIG_CRYPTO_SHA256_RISCV64) += sha256-riscv64.o sha256-riscv64-y := sha256-riscv64-glue.o sha256-riscv64-zvknha_or_zvknhb-zvkb.o obj-$(CONFIG_CRYPTO_SHA512_RISCV64) += sha512-riscv64.o sha512-riscv64-y := sha512-riscv64-glue.o sha512-riscv64-zvknhb-zvkb.o obj-$(CONFIG_CRYPTO_SM3_RISCV64) += sm3-riscv64.o sm3-riscv64-y := sm3-riscv64-glue.o sm3-riscv64-zvksh-zvkb.o + +obj-$(CONFIG_CRYPTO_SM4_RISCV64) += sm4-riscv64.o +sm4-riscv64-y := sm4-riscv64-glue.o sm4-riscv64-zvksed-zvkb.o
arch/riscv/crypto/sm4-riscv64-glue.c new file +107 -0
107 lines not on this page · raw message
arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.S new file +117 -0
117 lines not on this page · raw message
Signature · 1 line
2.43.0
Palmer Dabbelt
maintainerQuoted 18 lines · On Sun, 21 Jan 2024 16:19:11 PST (-0800), [email protected] wrote:
This patchset, which applies to v6.8-rc1, adds cryptographic algorithm implementations accelerated using the RISC-V vector crypto extensions
(https://github.com/riscv/riscv-crypto/releases/download/v1.0.0/riscv-crypto-spec-vector.pdf)
and RISC-V vector extension
(https://github.com/riscv/riscv-v-spec/releases/download/v1.0/riscv-v-spec-1.0.pdf).
The following algorithms are included: AES in ECB, CBC, CTR, and XTS modes; ChaCha20; GHASH; SHA-2; SM3; and SM4.In general, the assembly code requires a 64-bit RISC-V CPU with VLEN >= 128, little endian byte order, and vector unaligned access support. The ECB, CTR, XTS, and ChaCha20 code is designed to naturally scale up to larger VLEN values. Building the assembly code requires tip-of-tree binutils (future 2.42) or tip-of-tree clang (future 18.x). All algorithms pass testing in QEMU, using
CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y. Much of the assembly code is derived from OpenSSL code that was added by https://github.com/openssl/openssl/pull/21923.
It's been cleaned up for integration with the kernel, e.g. reducing code duplication, eliminating use of .inst and perlasm, and fixing a few bugs.
Quoted 10 lines
This patchset incorporates the work of multiple people, including Jerry Shih, Heiko Stuebner, Christoph Müllner, Phoebe Chen, Charalampos Mitrodimas, and myself. This patchset went through several versions from Heiko (last version
https://lore.kernel.org/linux-crypto/[email protected]),
then several versions from Jerry (last version:
https://lore.kernel.org/linux-crypto/[email protected]),
then finally several versions from me. Thanks to everyone who has contributed to this patchset or its prerequisites. Since v6.8-rc1, all prerequisite kernel patches are upstream. I think this is now ready, and I'd like for it to be applied for 6.9, either to the crypto or riscv tree (at maintainers' choice).
I'm OK taking this through the RISC-V tree if folks want, I've gone and queued it up for my tester. Just LMK if it should go somewhere else, otherwise it'll end up on for-next in a day or so (I'm trying to get LTO builds added right now, so it might take a bit).
Quoted 88 lines
Below is the changelog for my versions of the patchset. For the changelog of the older versions, see the above links.
Changed in v3:
- Fixed a bug in the AES-XTS implementation where it assumed the CPU always set vl to the maximum possible value. This was okay for QEMU, but the vector spec allows CPUs to have different behavior.
- Increased the LMUL for AES-ECB to 8, as the registers are available.
- Fixed some license text that I had mistakenly changed when doing a find-and-replace of code.
- Addressed a checkpatch warning by not including filename in file.
- Rename some labels.
- Constify a variable.Changed in v2:
- Merged the AES modules together to prevent a build error.
- Only unregister AES algorithms that were registered.
- Corrected walksize properties to match the LMUL used by asm code.
- Simplified the CTR and XTS glue code slightly.
- Minor cleanups.Changed in v1:
- Refer to my cover letterhttps://lore.kernel.org/linux-crypto/[email protected]/Eric Biggers (1):
RISC-V: addTOOLCHAIN_HAS_VECTOR_CRYPTOHeiko Stuebner (2):
RISC-V: add helper function to read the vector VLEN
RISC-V: hook new crypto subdir into build-systemJerry Shih (7):
crypto: riscv - add vector crypto accelerated AES-{ECB,CBC,CTR,XTS}
crypto: riscv - add vector crypto accelerated ChaCha20
crypto: riscv - add vector crypto accelerated GHASH
crypto: riscv - add vector crypto accelerated SHA-{256,224}
crypto: riscv - add vector crypto accelerated SHA-{512,384}
crypto: riscv - add vector crypto accelerated SM3
crypto: riscv - add vector crypto accelerated SM4arch/riscv/Kbuild | 1 + arch/riscv/Kconfig | 7 + arch/riscv/crypto/Kconfig | 93 +++ arch/riscv/crypto/Makefile | 23 + arch/riscv/crypto/aes-macros.S | 156 +++++ arch/riscv/crypto/aes-riscv64-glue.c | 550 ++++++++++++++++++ .../crypto/aes-riscv64-zvkned-zvbb-zvkg.S | 312 ++++++++++ arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S | 146 +++++ arch/riscv/crypto/aes-riscv64-zvkned.S | 180 ++++++ arch/riscv/crypto/chacha-riscv64-glue.c | 101 ++++ arch/riscv/crypto/chacha-riscv64-zvkb.S | 294 ++++++++++ arch/riscv/crypto/ghash-riscv64-glue.c | 168 ++++++ arch/riscv/crypto/ghash-riscv64-zvkg.S | 72 +++ arch/riscv/crypto/sha256-riscv64-glue.c | 137 +++++ .../sha256-riscv64-zvknha_or_zvknhb-zvkb.S | 225 +++++++ arch/riscv/crypto/sha512-riscv64-glue.c | 133 +++++ .../riscv/crypto/sha512-riscv64-zvknhb-zvkb.S | 203 +++++++ arch/riscv/crypto/sm3-riscv64-glue.c | 112 ++++ arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S | 123 ++++ arch/riscv/crypto/sm4-riscv64-glue.c | 107 ++++ arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.S | 117 ++++ arch/riscv/include/asm/vector.h | 11 + crypto/Kconfig | 3 + 23 files changed, 3274 insertions(+) create mode 100644 arch/riscv/crypto/Kconfig create mode 100644 arch/riscv/crypto/Makefile create mode 100644 arch/riscv/crypto/aes-macros.S create mode 100644 arch/riscv/crypto/aes-riscv64-glue.c create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvbb-zvkg.S create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned.S create mode 100644 arch/riscv/crypto/chacha-riscv64-glue.c create mode 100644 arch/riscv/crypto/chacha-riscv64-zvkb.S create mode 100644 arch/riscv/crypto/ghash-riscv64-glue.c create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkg.S create mode 100644 arch/riscv/crypto/sha256-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha256-riscv64-zvknha_or_zvknhb-zvkb.S create mode 100644 arch/riscv/crypto/sha512-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha512-riscv64-zvknhb-zvkb.S create mode 100644 arch/riscv/crypto/sm3-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S create mode 100644 arch/riscv/crypto/sm4-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.Sbase-commit:
6613476e225e090cc9aad49be7fa504e290dd33d
Quoted 30 lines · Eric Biggers
This patchset, which applies to v6.8-rc1, adds cryptographic algorithm implementations accelerated using the RISC-V vector crypto extensions
(https://github.com/riscv/riscv-crypto/releases/download/v1.0.0/riscv-crypto-spec-vector.pdf)
and RISC-V vector extension
(https://github.com/riscv/riscv-v-spec/releases/download/v1.0/riscv-v-spec-1.0.pdf).
The following algorithms are included: AES in ECB, CBC, CTR, and XTS modes; ChaCha20; GHASH; SHA-2; SM3; and SM4.In general, the assembly code requires a 64-bit RISC-V CPU with VLEN >= 128, little endian byte order, and vector unaligned access support. The ECB, CTR, XTS, and ChaCha20 code is designed to naturally scale up to larger VLEN values. Building the assembly code requires tip-of-tree binutils (future 2.42) or tip-of-tree clang (future 18.x). All algorithms pass testing in QEMU, using
CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y. Much of the assembly code is derived from OpenSSL code that was added by https://github.com/openssl/openssl/pull/21923.
It's been cleaned up for integration with the kernel, e.g. reducing code duplication, eliminating use of .inst and perlasm, and fixing a few bugs.This patchset incorporates the work of multiple people, including Jerry Shih, Heiko Stuebner, Christoph Müllner, Phoebe Chen, Charalampos Mitrodimas, and myself. This patchset went through several versions from Heiko (last version
https://lore.kernel.org/linux-crypto/[email protected]),
then several versions from Jerry (last version:
https://lore.kernel.org/linux-crypto/[email protected]),
then finally several versions from me. Thanks to everyone who has contributed to this patchset or its prerequisites. Since v6.8-rc1, all prerequisite kernel patches are upstream. I think this is now ready, and I'd like for it to be applied for 6.9, either to the crypto or riscv tree (at maintainers' choice).
Quoted 2 lines
Below is the changelog for my versions of the patchset. For the changelog of the older versions, see the above links.
For all patches of this series:
Eric, thank you for working on this!
Quoted 88 lines
Changed in v3:
- Fixed a bug in the AES-XTS implementation where it assumed the CPU always set vl to the maximum possible value. This was okay for QEMU, but the vector spec allows CPUs to have different behavior.
- Increased the LMUL for AES-ECB to 8, as the registers are available.
- Fixed some license text that I had mistakenly changed when doing a find-and-replace of code.
- Addressed a checkpatch warning by not including filename in file.
- Rename some labels.
- Constify a variable.Changed in v2:
- Merged the AES modules together to prevent a build error.
- Only unregister AES algorithms that were registered.
- Corrected walksize properties to match the LMUL used by asm code.
- Simplified the CTR and XTS glue code slightly.
- Minor cleanups.Changed in v1:
- Refer to my cover letterhttps://lore.kernel.org/linux-crypto/[email protected]/Eric Biggers (1):
RISC-V: addTOOLCHAIN_HAS_VECTOR_CRYPTOHeiko Stuebner (2):
RISC-V: add helper function to read the vector VLEN
RISC-V: hook new crypto subdir into build-systemJerry Shih (7):
crypto: riscv - add vector crypto accelerated AES-{ECB,CBC,CTR,XTS}
crypto: riscv - add vector crypto accelerated ChaCha20
crypto: riscv - add vector crypto accelerated GHASH
crypto: riscv - add vector crypto accelerated SHA-{256,224}
crypto: riscv - add vector crypto accelerated SHA-{512,384}
crypto: riscv - add vector crypto accelerated SM3
crypto: riscv - add vector crypto accelerated SM4arch/riscv/Kbuild | 1 + arch/riscv/Kconfig | 7 + arch/riscv/crypto/Kconfig | 93 +++ arch/riscv/crypto/Makefile | 23 + arch/riscv/crypto/aes-macros.S | 156 +++++ arch/riscv/crypto/aes-riscv64-glue.c | 550 ++++++++++++++++++ .../crypto/aes-riscv64-zvkned-zvbb-zvkg.S | 312 ++++++++++ arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S | 146 +++++ arch/riscv/crypto/aes-riscv64-zvkned.S | 180 ++++++ arch/riscv/crypto/chacha-riscv64-glue.c | 101 ++++ arch/riscv/crypto/chacha-riscv64-zvkb.S | 294 ++++++++++ arch/riscv/crypto/ghash-riscv64-glue.c | 168 ++++++ arch/riscv/crypto/ghash-riscv64-zvkg.S | 72 +++ arch/riscv/crypto/sha256-riscv64-glue.c | 137 +++++ .../sha256-riscv64-zvknha_or_zvknhb-zvkb.S | 225 +++++++ arch/riscv/crypto/sha512-riscv64-glue.c | 133 +++++ .../riscv/crypto/sha512-riscv64-zvknhb-zvkb.S | 203 +++++++ arch/riscv/crypto/sm3-riscv64-glue.c | 112 ++++ arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S | 123 ++++ arch/riscv/crypto/sm4-riscv64-glue.c | 107 ++++ arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.S | 117 ++++ arch/riscv/include/asm/vector.h | 11 + crypto/Kconfig | 3 + 23 files changed, 3274 insertions(+) create mode 100644 arch/riscv/crypto/Kconfig create mode 100644 arch/riscv/crypto/Makefile create mode 100644 arch/riscv/crypto/aes-macros.S create mode 100644 arch/riscv/crypto/aes-riscv64-glue.c create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvbb-zvkg.S create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned-zvkb.S create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned.S create mode 100644 arch/riscv/crypto/chacha-riscv64-glue.c create mode 100644 arch/riscv/crypto/chacha-riscv64-zvkb.S create mode 100644 arch/riscv/crypto/ghash-riscv64-glue.c create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkg.S create mode 100644 arch/riscv/crypto/sha256-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha256-riscv64-zvknha_or_zvknhb-zvkb.S create mode 100644 arch/riscv/crypto/sha512-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha512-riscv64-zvknhb-zvkb.S create mode 100644 arch/riscv/crypto/sm3-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm3-riscv64-zvksh-zvkb.S create mode 100644 arch/riscv/crypto/sm4-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm4-riscv64-zvksed-zvkb.Sbase-commit:
6613476e225e090cc9aad49be7fa504e290dd33d--
2.43.0
This series was applied to riscv/linux.git (for-next) by Palmer Dabbelt <[email protected]>:
Hello:
This series was applied to riscv/linux.git (for-next) by Palmer Dabbelt <[email protected]>:
Quoted 9 lines · On Sun, 21 Jan 2024 16:19:11 -0800 you wrote:
This patchset, which applies to v6.8-rc1, adds cryptographic algorithm implementations accelerated using the RISC-V vector crypto extensions
(https://github.com/riscv/riscv-crypto/releases/download/v1.0.0/riscv-crypto-spec-vector.pdf)
and RISC-V vector extension
(https://github.com/riscv/riscv-v-spec/releases/download/v1.0/riscv-v-spec-1.0.pdf).
The following algorithms are included: AES in ECB, CBC, CTR, and XTS modes; ChaCha20; GHASH; SHA-2; SM3; and SM4.[...]
Here is the summary with links:
- [v3,01/10] RISC-V: add helper function to read the vector VLEN (no matching commit)
- [v3,02/10] RISC-V: add TOOLCHAIN_HAS_VECTOR_CRYPTO
https://git.kernel.org/riscv/c/34ca4ec628de - [v3,03/10] RISC-V: hook new crypto subdir into build-system https://git.kernel.org/riscv/c/178f3856436c - [v3,04/10] crypto: riscv - add vector crypto accelerated AES-{ECB,CBC,CTR,XTS} https://git.kernel.org/riscv/c/eb24af5d7a05 - [v3,05/10] crypto: riscv - add vector crypto accelerated ChaCha20 https://git.kernel.org/riscv/c/bb54668837a0 - [v3,06/10] crypto: riscv - add vector crypto accelerated GHASH https://git.kernel.org/riscv/c/600a3853dfa0 - [v3,07/10] crypto: riscv - add vector crypto accelerated SHA-{256,224} https://git.kernel.org/riscv/c/8c8e40470ffe - [v3,08/10] crypto: riscv - add vector crypto accelerated SHA-{512,384} https://git.kernel.org/riscv/c/b3415925a08b - [v3,09/10] crypto: riscv - add vector crypto accelerated SM3 https://git.kernel.org/riscv/c/563a5255afa2 - [v3,10/10] crypto: riscv - add vector crypto accelerated SM4 https://git.kernel.org/riscv/c/b8d06352bbf3
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html