RISC-V: support some cryptography accelerations
Subsystems (2)
v2 cover RISC-V: support some cryptography accelerations
From: Heiko Stuebner <[email protected]>
So this was my playground the last days.
The base is v13 of the vector patchset but the first patches up to doing the Zbc-based GCM GHash can also run without those. Of course the vector- crypto extensions are also not ratified yet, hence the marking as RFC.
As v13 of the vector patchset dropped the patches for in-kernel usage of vector instructions, I picked the ones from v12 over into this series for now.
My basic goal was to not re-invent cryptographic code, so the heavy lifting is done by those perl-asm scripts used in openssl and the perl code used here-in stems from code that is targetted at openssl [0] and is unmodified from there to limit needed review effort.
With a matching qemu (there are patches for vector-crypto flying around) the in-kernel crypto-selftests (also the extended ones) are very happy so far.
Things to do:
- use correct Co-developed-attribution for the code coming from openssl
- follow openSSL changes along until they get eventually merged
changes in v2:
- rebased on 6.2 + zbb series, so don't include already applied changes anymore
- refresh code picked from openssl as that side matures
- more algorithms (SHA512, AES, SM3, SM4)
[0] both still open
https://github.com/openssl/openssl/pull/20078
https://github.com/openssl/openssl/pull/20149
Greentime Hu (2):
riscv: Add support for kernel mode vector
riscv: Add vector extension XOR implementation
Heiko Stuebner (14):
RISC-V: add Zbc extension detection
RISC-V: add Zbkb extension detection
RISC-V: hook new crypto subdir into build-system
RISC-V: crypto: add accelerated GCM GHASH implementation
RISC-V: add helper function to read the vector VLEN
RISC-V: add vector crypto extension detection
RISC-V: crypto: update perl include with helpers for vector (crypto)
instructions
RISC-V: crypto: add Zvkb accelerated GCM GHASH implementation
RISC-V: crypto: add Zvkg accelerated GCM GHASH implementation
RISC-V: crypto: add a vector-crypto-accelerated SHA256 implementation
RISC-V: crypto: add a vector-crypto-accelerated SHA512 implementation
RISC-V: crypto: add Zvkned accelerated AES encryption implementation
RISC-V: crypto: add Zvksed accelerated SM4 encryption implementation
RISC-V: crypto: add Zvksh accelerated SM3 hash implementation
arch/riscv/Kbuild | 1 + arch/riscv/Kconfig | 22 + arch/riscv/crypto/Kconfig | 82 +++ arch/riscv/crypto/Makefile | 60 ++ arch/riscv/crypto/aes-riscv-glue.c | 169 ++++++ arch/riscv/crypto/aes-riscv64-zvkned.pl | 500 ++++++++++++++++ arch/riscv/crypto/ghash-riscv64-glue.c | 485 +++++++++++++++ arch/riscv/crypto/ghash-riscv64-zbc.pl | 400 +++++++++++++ arch/riscv/crypto/ghash-riscv64-zvkb.pl | 349 +++++++++++ arch/riscv/crypto/ghash-riscv64-zvkg.pl | 161 +++++ arch/riscv/crypto/riscv.pm | 659 +++++++++++++++++++++ arch/riscv/crypto/sha256-riscv64-glue.c | 114 ++++ arch/riscv/crypto/sha256-riscv64-zvknha.pl | 284 +++++++++ arch/riscv/crypto/sha512-riscv64-glue.c | 104 ++++ arch/riscv/crypto/sha512-riscv64-zvknhb.pl | 347 +++++++++++ arch/riscv/crypto/sm3-riscv64-glue.c | 112 ++++ arch/riscv/crypto/sm3-riscv64-zvksh.pl | 195 ++++++ arch/riscv/crypto/sm4-riscv64-glue.c | 163 +++++ arch/riscv/crypto/sm4-riscv64-zvksed.pl | 270 +++++++++ arch/riscv/include/asm/hwcap.h | 9 + arch/riscv/include/asm/vector.h | 25 + arch/riscv/include/asm/xor.h | 82 +++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/cpu.c | 9 + arch/riscv/kernel/cpufeature.c | 9 + arch/riscv/kernel/kernel_mode_vector.c | 132 +++++ arch/riscv/lib/Makefile | 1 + arch/riscv/lib/xor.S | 81 +++ crypto/Kconfig | 3 + 29 files changed, 4829 insertions(+) create mode 100644 arch/riscv/crypto/Kconfig create mode 100644 arch/riscv/crypto/Makefile create mode 100644 arch/riscv/crypto/aes-riscv-glue.c create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned.pl create mode 100644 arch/riscv/crypto/ghash-riscv64-glue.c create mode 100644 arch/riscv/crypto/ghash-riscv64-zbc.pl create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkb.pl create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkg.pl create mode 100644 arch/riscv/crypto/riscv.pm create mode 100644 arch/riscv/crypto/sha256-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha256-riscv64-zvknha.pl create mode 100644 arch/riscv/crypto/sha512-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha512-riscv64-zvknhb.pl create mode 100644 arch/riscv/crypto/sm3-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm3-riscv64-zvksh.pl create mode 100644 arch/riscv/crypto/sm4-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm4-riscv64-zvksed.pl create mode 100644 arch/riscv/include/asm/xor.h create mode 100644 arch/riscv/kernel/kernel_mode_vector.c create mode 100644 arch/riscv/lib/xor.S
Signature · 1 line
2.39.0
RFC PATCH v2 1/16 riscv: Add support for kernel mode vector
From: Greentime Hu <[email protected]>
Add kernel_rvv_begin() and kernel_rvv_end() function declarations and corresponding definitions in kernel_mode_vector.c
These are needed to wrap uses of vector in kernel mode.
--- arch/riscv/include/asm/vector.h | 14 +++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/kernel_mode_vector.c | 132 +++++++++++++++++++++++++ 3 files changed, 147 insertions(+) create mode 100644 arch/riscv/kernel/kernel_mode_vector.c
arch/riscv/include/asm/vector.h +14 -0
@@ -134,6 +134,20 @@ static inline void vstate_restore(struct task_struct *task, } } +static inline void vector_flush_cpu_state (void ) +{ + asmvolatile ( +"vsetvli t0, x0, e8, m8, ta, ma\n\t" +"vmv.v.i v0, 0\n\t" +"vmv.v.i v8, 0\n\t" +"vmv.v.i v16, 0\n\t" +"vmv.v.i v24, 0\n\t" + : : :"t0" ); +} + +void kernel_rvv_begin (void ); +void kernel_rvv_end (void ); +#else /* ! CONFIG_RISCV_ISA_V */ struct pt_regs ;
arch/riscv/kernel/Makefile +1 -0
@@ -56,6 +56,7 @@ obj-$(CONFIG_MMU) += vdso.o vdso/ obj-$(CONFIG_RISCV_M_MODE) += traps_misaligned.o obj-$(CONFIG_FPU) += fpu.o obj-$(CONFIG_RISCV_ISA_V) += vector.o +obj-$(CONFIG_RISCV_ISA_V) += kernel_mode_vector.o obj-$(CONFIG_SMP) += smpboot.o obj-$(CONFIG_SMP) += smp.o obj-$(CONFIG_SMP) += cpu_ops.o
arch/riscv/kernel/kernel_mode_vector.c new file +132 -0
132 lines not on this page · raw message
Signature · 1 line
2.39.0
RFC PATCH v2 2/16 riscv: Add vector extension XOR implementation
In reply to Heiko StuebnerFrom: Greentime Hu <[email protected]>
This patch adds support for vector optimized XOR and it is tested in qemu.
--- arch/riscv/include/asm/xor.h | 82 ++++++++++++++++++++++++++++++++++++ arch/riscv/lib/Makefile | 1 + arch/riscv/lib/xor.S | 81 +++++++++++++++++++++++++++++++++++ 3 files changed, 164 insertions(+) create mode 100644 arch/riscv/include/asm/xor.h create mode 100644 arch/riscv/lib/xor.S
arch/riscv/include/asm/xor.h new file +82 -0
@@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* Copyright (C) 2021 SiFive +*/ + +#include <linux/hardirq.h> +#include <asm-generic/xor.h> +#ifdef CONFIG_VECTOR +#include <asm/vector.h> +#include <asm/switch_to.h> + +void xor_regs_2_ (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2); +void xor_regs_3_ (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2, +const unsigned long *__restrict p3); +void xor_regs_4_ (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2, +const unsigned long *__restrict p3, +const unsigned long *__restrict p4); +void xor_regs_5_ (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2, +const unsigned long *__restrict p3, +const unsigned long *__restrict p4, +const unsigned long *__restrict p5); + +static void xor_rvv_2 (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2) +{ +kernel_rvv_begin (); +xor_regs_2_ (bytes, p1, p2); +kernel_rvv_end (); +} + +static void xor_rvv_3 (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2, +const unsigned long *__restrict p3) +{ +kernel_rvv_begin (); +xor_regs_3_ (bytes, p1, p2, p3); +kernel_rvv_end (); +} + +static void xor_rvv_4 (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2, +const unsigned long *__restrict p3, +const unsigned long *__restrict p4) +{ +kernel_rvv_begin (); +xor_regs_4_ (bytes, p1, p2, p3, p4); +kernel_rvv_end (); +} + +static void xor_rvv_5 (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2, +const unsigned long *__restrict p3, +const unsigned long *__restrict p4, +const unsigned long *__restrict p5) +{ +kernel_rvv_begin (); +xor_regs_5_ (bytes, p1, p2, p3, p4, p5); +kernel_rvv_end (); +} + +static struct xor_block_template xor_block_rvv = { + .name ="rvv" , + .do_2 = xor_rvv_2, + .do_3 = xor_rvv_3, + .do_4 = xor_rvv_4, + .do_5 = xor_rvv_5 +}; + +#undef XOR_TRY_TEMPLATES +#define XOR_TRY_TEMPLATES \ + do { \ + xor_speed(&xor_block_8regs); \ + xor_speed(&xor_block_32regs); \ + if (has_vector()) { \ + xor_speed(&xor_block_rvv);\ + } \ + } while (0) +#endif
arch/riscv/lib/Makefile +1 -0
@@ -10,3 +10,4 @@ lib-$(CONFIG_MMU) += uaccess.o lib-$(CONFIG_64BIT) += tishift.o obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o +lib-$(CONFIG_VECTOR) += xor.o
arch/riscv/lib/xor.S new file +81 -0
81 lines not on this page · raw message
Signature · 1 line
2.39.0
RFC PATCH v2 3/16 RISC-V: add Zbc extension detection
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
Add handling for Zbc extension.
Zbc provides instruction for carry-less multiplication.
--- arch/riscv/Kconfig | 22 ++++++++++++++++++++++ arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/kernel/cpu.c | 1 + arch/riscv/kernel/cpufeature.c | 1 + 4 files changed, 25 insertions(+)
arch/riscv/Kconfig +22 -0
@@ -450,6 +450,28 @@ config RISCV_ISA_ZBB If you don't know what to do here, say Y. +config TOOLCHAIN_HAS_ZBC + bool + default y + depends on !64BIT || $(cc-option,-mabi=lp64 -march=rv64ima_zbc) + depends on !32BIT || $(cc-option,-mabi=ilp32 -march=rv32ima_zbc) + depends on LLD_VERSION >= 150000 || LD_VERSION >= 23900 + depends on AS_IS_GNU + +config RISCV_ISA_ZBC + bool "Zbc extension support for bit manipulation instructions" + depends on TOOLCHAIN_HAS_ZBC + depends on !XIP_KERNEL && MMU + default y + help + Adds support to dynamically detect the presence of the ZBC + extension (carry-less multiplication) and enable its usage. + + The Zbc extension provides instructions clmul, clmulh and clmulr + to accelerate carry-less multiplications. + + If you don't know what to do here, say Y. + config TOOLCHAIN_HAS_ZICBOM bool default y
arch/riscv/include/asm/hwcap.h +1 -0
@@ -60,6 +60,7 @@ enum riscv_isa_ext_id {RISCV_ISA_EXT_SVINVAL ,RISCV_ISA_EXT_SVPBMT ,RISCV_ISA_EXT_ZBB , +RISCV_ISA_EXT_ZBC ,RISCV_ISA_EXT_ZICBOM ,RISCV_ISA_EXT_ZIHINTPAUSE ,RISCV_ISA_EXT_ID_MAX
arch/riscv/kernel/cpu.c +1 -0
@@ -188,6 +188,7 @@ static struct riscv_isa_ext_data isa_ext_arr[] = {__RISCV_ISA_EXT_DATA (zicbom,RISCV_ISA_EXT_ZICBOM ),__RISCV_ISA_EXT_DATA (zihintpause,RISCV_ISA_EXT_ZIHINTPAUSE ),__RISCV_ISA_EXT_DATA (zbb,RISCV_ISA_EXT_ZBB ), +__RISCV_ISA_EXT_DATA (zbc,RISCV_ISA_EXT_ZBC ),__RISCV_ISA_EXT_DATA (sscofpmf,RISCV_ISA_EXT_SSCOFPMF ),__RISCV_ISA_EXT_DATA (sstc,RISCV_ISA_EXT_SSTC ), __RISCV_ISA_EXT_DATA(svinval, RISCV_ISA_EXT_SVINVAL),
arch/riscv/kernel/cpufeature.c +1 -0
@@ -234,6 +234,7 @@ void __init riscv_fill_hwcap(void)SET_ISA_EXT_MAP ("svinval" ,RISCV_ISA_EXT_SVINVAL );SET_ISA_EXT_MAP ("svpbmt" ,RISCV_ISA_EXT_SVPBMT );SET_ISA_EXT_MAP ("zbb" ,RISCV_ISA_EXT_ZBB ); +SET_ISA_EXT_MAP ("zbc" ,RISCV_ISA_EXT_ZBC );SET_ISA_EXT_MAP ("zicbom" ,RISCV_ISA_EXT_ZICBOM );SET_ISA_EXT_MAP ("zihintpause" ,RISCV_ISA_EXT_ZIHINTPAUSE ); }
Signature · 1 line
2.39.0
RFC PATCH v2 4/16 RISC-V: add Zbkb extension detection
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
Add detection for Zbkb extension.
Zbkb is part of the set of scalar cryptography extensions and provides bitmanip instructions for cryptography, with them being a "subset of the Zbb extension particularly useful for cryptography".
Zbkb was ratified in january 2022.
Expect code using the extension to pre-encode zbkb instructions, so don't introduce special toolchain requirements for now.
--- arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/kernel/cpu.c | 1 + arch/riscv/kernel/cpufeature.c | 1 + 3 files changed, 3 insertions(+)
arch/riscv/include/asm/hwcap.h +1 -0
@@ -61,6 +61,7 @@ enum riscv_isa_ext_id {RISCV_ISA_EXT_SVPBMT ,RISCV_ISA_EXT_ZBB ,RISCV_ISA_EXT_ZBC , +RISCV_ISA_EXT_ZBKB ,RISCV_ISA_EXT_ZICBOM ,RISCV_ISA_EXT_ZIHINTPAUSE ,RISCV_ISA_EXT_ID_MAX
arch/riscv/kernel/cpu.c +1 -0
@@ -189,6 +189,7 @@ static struct riscv_isa_ext_data isa_ext_arr[] = {__RISCV_ISA_EXT_DATA (zihintpause,RISCV_ISA_EXT_ZIHINTPAUSE ),__RISCV_ISA_EXT_DATA (zbb,RISCV_ISA_EXT_ZBB ),__RISCV_ISA_EXT_DATA (zbc,RISCV_ISA_EXT_ZBC ), +__RISCV_ISA_EXT_DATA (zbkb,RISCV_ISA_EXT_ZBKB ),__RISCV_ISA_EXT_DATA (sscofpmf,RISCV_ISA_EXT_SSCOFPMF ),__RISCV_ISA_EXT_DATA (sstc,RISCV_ISA_EXT_SSTC ), __RISCV_ISA_EXT_DATA(svinval, RISCV_ISA_EXT_SVINVAL),
arch/riscv/kernel/cpufeature.c +1 -0
@@ -235,6 +235,7 @@ void __init riscv_fill_hwcap(void)SET_ISA_EXT_MAP ("svpbmt" ,RISCV_ISA_EXT_SVPBMT );SET_ISA_EXT_MAP ("zbb" ,RISCV_ISA_EXT_ZBB );SET_ISA_EXT_MAP ("zbc" ,RISCV_ISA_EXT_ZBC ); +SET_ISA_EXT_MAP ("zbkb" ,RISCV_ISA_EXT_ZBKB );SET_ISA_EXT_MAP ("zicbom" ,RISCV_ISA_EXT_ZICBOM );SET_ISA_EXT_MAP ("zihintpause" ,RISCV_ISA_EXT_ZIHINTPAUSE ); }
Signature · 1 line
2.39.0
RFC PATCH v2 5/16 RISC-V: hook new crypto subdir into build-system
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
Create a crypto subdirectory for added accelerated cryptography routines and hook it into the riscv Kbuild and the main crypto Kconfig.
--- arch/riscv/Kbuild | 1 + arch/riscv/crypto/Kconfig | 5 +++++ arch/riscv/crypto/Makefile | 4 ++++ crypto/Kconfig | 3 +++ 4 files changed, 13 insertions(+) create mode 100644 arch/riscv/crypto/Kconfig create mode 100644 arch/riscv/crypto/Makefile
arch/riscv/Kbuild +1 -0
@@ -2,6 +2,7 @@ obj-y += kernel/ mm/ net/ obj-$(CONFIG_BUILTIN_DTB) += boot/dts/ +obj-$(CONFIG_CRYPTO) += crypto/ obj-y += errata/ obj-$(CONFIG_KVM) += kvm/
arch/riscv/crypto/Kconfig new file +5 -0
@@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + +menu "Accelerated Cryptographic Algorithms for CPU (riscv)" + +endmenu
arch/riscv/crypto/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# linux/arch/riscv/crypto/Makefile +#
crypto/Kconfig +3 -0
@@ -1401,6 +1401,9 @@ endif if PPC source "arch/powerpc/crypto/Kconfig" endif +if RISCV +source "arch/riscv/crypto/Kconfig" +endif if S390 source "arch/s390/crypto/Kconfig" endif
Signature · 1 line
2.39.0
RFC PATCH v2 6/16 RISC-V: crypto: add accelerated GCM GHASH implementation
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
With different sets of available extensions a number of different implementation variants are possible. Quite a number of them are already implemented in openSSL or are in the process of being implemented, so pick the relevant openSSL coden and add suitable glue code similar to arm64 and powerpc to use it for kernel-specific cryptography.
The prioritization of the algorithms follows the ifdef chain for the assembly callbacks done in openssl but here algorithms will get registered separately so that all of them can be part of the crypto selftests.
The crypto subsystem will select the most performant of all registered algorithms on the running system but will selftest all registered ones.
In a first step this adds scalar variants using the Zbc, Zbb and possible Zbkb (bitmanip crypto extension) and the perl implementation stems from openSSL pull request on
https://github.com/openssl/openssl/pull/20078
--- arch/riscv/crypto/Kconfig | 11 + arch/riscv/crypto/Makefile | 14 + arch/riscv/crypto/ghash-riscv64-glue.c | 258 ++++++++++++++++ arch/riscv/crypto/ghash-riscv64-zbc.pl | 400 +++++++++++++++++++++++++ arch/riscv/crypto/riscv.pm | 230 ++++++++++++++ 5 files changed, 913 insertions(+) create mode 100644 arch/riscv/crypto/ghash-riscv64-glue.c create mode 100644 arch/riscv/crypto/ghash-riscv64-zbc.pl create mode 100644 arch/riscv/crypto/riscv.pm
arch/riscv/crypto/Kconfig +11 -0
@@ -2,4 +2,15 @@ menu "Accelerated Cryptographic Algorithms for CPU (riscv)" +config CRYPTO_GHASH_RISCV64 + tristate "Hash functions: GHASH" + depends on 64BIT && RISCV_ISA_ZBC + select CRYPTO_HASH + select CRYPTO_LIB_GF128MUL + help + GCM GHASH function (NIST SP800-38D) + + Architecture: riscv64 using one of: + - ZBC extension + endmenu
arch/riscv/crypto/Makefile +14 -0
@@ -2,3 +2,17 @@ # # linux/arch/riscv/crypto/Makefile # + +obj-$(CONFIG_CRYPTO_GHASH_RISCV64) += ghash-riscv64.o +ghash-riscv64-y := ghash-riscv64-glue.o +ifdef CONFIG_RISCV_ISA_ZBC +ghash-riscv64-y += ghash-riscv64-zbc.o +endif + +quiet_cmd_perlasm = PERLASM $@ + cmd_perlasm = $(PERL) $(<) void $(@) + +$(obj)/ghash-riscv64-zbc.S: $(src)/ghash-riscv64-zbc.pl + $(call cmd,perlasm) + +clean-files += ghash-riscv64-zbc.S
arch/riscv/crypto/ghash-riscv64-glue.c new file +258 -0
258 lines not on this page · raw message
arch/riscv/crypto/ghash-riscv64-zbc.pl new file +400 -0
400 lines not on this page · raw message
arch/riscv/crypto/riscv.pm new file +230 -0
230 lines not on this page · raw message
Signature · 7 lines
2.39.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH v2 7/16 RISC-V: add helper function to read the vector VLEN
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
VLEN describes the length of each vector register and some instructions need specific minimal VLENs to work correctly.
The vector code already includes a variable riscv_vsize that contains the value of "32 vector registers with vlenb length" that gets filled during boot. vlenb is the value contained in the CSR_VLENB register and the value represents "VLEN / 8".
So add riscv_vector_vlen() to return the actual VLEN value for in-kernel users when they need to check the available VLEN.
--- arch/riscv/include/asm/vector.h | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/riscv/include/asm/vector.h +11 -0
@@ -163,4 +163,15 @@ static inline bool vstate_query(struct pt_regs *regs) { return false; }#endif /* CONFIG_RISCV_ISA_V */ +/* +* Return the implementation's vlen value. +* +* riscv_vsize contains the value of "32 vector registers with vlenb length" +* so rebuild the vlen value in bits from it. +*/ +static inline int riscv_vector_vlen (void ) +{ +return riscv_vsize /32 *8 ; +} +#endif /* ! __ASM_RISCV_VECTOR_H */
Signature · 7 lines
2.39.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH v2 8/16 RISC-V: add vector crypto extension detection
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
Add detection for some extensions of the vector-crypto specification, namely
- Zvkb: Vector Bit-manipulation used in Cryptography
- Zvkg: Vector GCM/GMAC
- Zvknha and Zvknhb: NIST Algorithm Suite
- Zvkns: AES-128, AES-256 Single Round Suite
- Zvksed: ShangMi Algorithm Suite
- Zvksh: ShangMi Algorithm Suite
As their use is very specific and will likely be limited to special places we expect current code to just pre-encode those instructions, so right now we don't introduce toolchain requirements.
--- arch/riscv/include/asm/hwcap.h | 7 +++++++ arch/riscv/kernel/cpu.c | 7 +++++++ arch/riscv/kernel/cpufeature.c | 7 +++++++ 3 files changed, 21 insertions(+)
arch/riscv/include/asm/hwcap.h +7 -0
@@ -64,6 +64,13 @@ enum riscv_isa_ext_id {RISCV_ISA_EXT_ZBKB ,RISCV_ISA_EXT_ZICBOM ,RISCV_ISA_EXT_ZIHINTPAUSE , +RISCV_ISA_EXT_ZVKB , +RISCV_ISA_EXT_ZVKG , +RISCV_ISA_EXT_ZVKNED , +RISCV_ISA_EXT_ZVKNHA , +RISCV_ISA_EXT_ZVKNHB , +RISCV_ISA_EXT_ZVKSED , +RISCV_ISA_EXT_ZVKSH ,RISCV_ISA_EXT_ID_MAX };static_assert (RISCV_ISA_EXT_ID_MAX <=RISCV_ISA_EXT_MAX );
arch/riscv/kernel/cpu.c +7 -0
@@ -190,6 +190,13 @@ static struct riscv_isa_ext_data isa_ext_arr[] = {__RISCV_ISA_EXT_DATA (zbb,RISCV_ISA_EXT_ZBB ),__RISCV_ISA_EXT_DATA (zbc,RISCV_ISA_EXT_ZBC ),__RISCV_ISA_EXT_DATA (zbkb,RISCV_ISA_EXT_ZBKB ), +__RISCV_ISA_EXT_DATA (zvkb,RISCV_ISA_EXT_ZVKB ), +__RISCV_ISA_EXT_DATA (zvkg,RISCV_ISA_EXT_ZVKG ), +__RISCV_ISA_EXT_DATA (zvkned,RISCV_ISA_EXT_ZVKNED ), +__RISCV_ISA_EXT_DATA (zvknha,RISCV_ISA_EXT_ZVKNHA ), +__RISCV_ISA_EXT_DATA (zvknhb,RISCV_ISA_EXT_ZVKNHB ), +__RISCV_ISA_EXT_DATA (zvksed,RISCV_ISA_EXT_ZVKSED ), +__RISCV_ISA_EXT_DATA (zvksh,RISCV_ISA_EXT_ZVKSH ),__RISCV_ISA_EXT_DATA (sscofpmf,RISCV_ISA_EXT_SSCOFPMF ),__RISCV_ISA_EXT_DATA (sstc,RISCV_ISA_EXT_SSTC ), __RISCV_ISA_EXT_DATA(svinval, RISCV_ISA_EXT_SVINVAL),
arch/riscv/kernel/cpufeature.c +7 -0
@@ -238,6 +238,13 @@ void __init riscv_fill_hwcap(void)SET_ISA_EXT_MAP ("zbkb" ,RISCV_ISA_EXT_ZBKB );SET_ISA_EXT_MAP ("zicbom" ,RISCV_ISA_EXT_ZICBOM );SET_ISA_EXT_MAP ("zihintpause" ,RISCV_ISA_EXT_ZIHINTPAUSE ); +SET_ISA_EXT_MAP ("zvkb" ,RISCV_ISA_EXT_ZVKB ); +SET_ISA_EXT_MAP ("zvkg" ,RISCV_ISA_EXT_ZVKG ); +SET_ISA_EXT_MAP ("zvkned" ,RISCV_ISA_EXT_ZVKNED ); +SET_ISA_EXT_MAP ("zvknha" ,RISCV_ISA_EXT_ZVKNHA ); +SET_ISA_EXT_MAP ("zvknhb" ,RISCV_ISA_EXT_ZVKNHB ); +SET_ISA_EXT_MAP ("zvksed" ,RISCV_ISA_EXT_ZVKSED ); +SET_ISA_EXT_MAP ("zvksh" ,RISCV_ISA_EXT_ZVKSH ); }#undef SET_ISA_EXT_MAP }
Signature · 1 line
2.39.0
RFC PATCH v2 9/16 RISC-V: crypto: update perl include with helpers for vector (crypto) instructions
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
The openSSL scripts use a number of helpers for handling vector instructions and instructions from the vector-crypto-extensions.
Therefore port these over from openSSL.
--- arch/riscv/crypto/riscv.pm | 433 ++++++++++++++++++++++++++++++++++++- 1 file changed, 431 insertions(+), 2 deletions(-)
arch/riscv/crypto/riscv.pm +431 -2
@@ -48,11 +48,34 @@ sub read_reg { return $1; } +my @vregs = map("v$_",(0..31)); +my %vreglookup; +@vreglookup{@vregs} = @vregs; + +sub read_vreg { + my $vreg = lc shift; + if (!exists($vreglookup{$vreg})) { + my $trace = ""; + if ($have_stacktrace) { + $trace = Devel::StackTrace->new->as_string; + } + die("Unknown vector register ".$vreg."\n".$trace); + } + if (!($vreg =~ /^v([0-9]+)$/)) { + my $trace = ""; + if ($have_stacktrace) { + $trace = Devel::StackTrace->new->as_string; + } + die("Could not process vector register ".$vreg."\n".$trace); + } + return $1; +} + # Helper functions sub brev8_rv64i { - # brev8 without `brev8` instruction (only in Zkbk) - # Bit-reverses the first argument and needs three scratch registers + # brev8 without `brev8` instruction (only in Zbkb) + # Bit-reverses the first argument and needs two scratch registers my $val = shift; my $t0 = shift; my $t1 = shift;
@@ -227,4 +250,410 @@ sub rev8 { return ".word ".($template | ($rs << 15) | ($rd << 7)); } +# Vector instructions + +sub vadd_vv { + # vadd.vv vd, vs2, vs1 + my $template = 0b0000001_00000_00000_000_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15) | ($vd << 7)); +} + +sub vid_v { + # vid.v vd + my $template = 0b0101001_00000_10001_010_00000_1010111; + my $vd = read_vreg shift; + return ".word ".($template | ($vd << 7)); +} + +sub vle32_v { + # vle32.v vd, (rs1) + my $template = 0b0000001_00000_00000_110_00000_0000111; + my $vd = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($rs1 << 15) | ($vd << 7)); +} + +sub vle64_v { + # vle64.v vd, (rs1) + my $template = 0b0000001_00000_00000_111_00000_0000111; + my $vd = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($rs1 << 15) | ($vd << 7)); +} + +sub vlse32_v { + # vlse32.v vd, (rs1), rs2 + my $template = 0b0000101_00000_00000_110_00000_0000111; + my $vd = read_vreg shift; + my $rs1 = read_reg shift; + my $rs2 = read_reg shift; + return ".word ".($template | ($rs2 << 20) | ($rs1 << 15) | ($vd << 7)); +} + +sub vlse64_v { + # vlse64.v vd, (rs1), rs2 + my $template = 0b0000101_00000_00000_111_00000_0000111; + my $vd = read_vreg shift; + my $rs1 = read_reg shift; + my $rs2 = read_reg shift; + return ".word ".($template | ($rs2 << 20) | ($rs1 << 15) | ($vd << 7)); +} + +sub vmerge_vim { + # vmerge.vim vd, vs2, imm, v0 + my $template = 0b0101110_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $imm = shift; + return ".word ".($template | ($vs2 << 20) | ($imm << 15) | ($vd << 7)); +} + +sub vmerge_vvm { + # vmerge.vvm vd vs2 vs1 + my $template = 0b0101110_00000_00000_000_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15) | ($vd << 7)) +} + +sub vmseq_vi { + # vmseq vd vs1, imm + my $template = 0b0110001_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs1 = read_vreg shift; + my $imm = shift; + return ".word ".($template | ($vs1 << 20) | ($imm << 15) | ($vd << 7)) +} + +sub vmv_v_i { + # vmv.v.i vd, imm + my $template = 0b0101111_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $imm = shift; + return ".word ".($template | ($imm << 15) | ($vd << 7)); +} + +sub vmv_v_v { + # vmv.v.v vd, vs1 + my $template = 0b0101111_00000_00000_000_00000_1010111; + my $vd = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs1 << 15) | ($vd << 7)); +} + +sub vor_vv_v0t { + # vor.vv vd, vs2, vs1, v0.t + my $template = 0b0010100_00000_00000_000_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15) | ($vd << 7)); +} + +sub vse32_v { + # vse32.v vd, (rs1) + my $template = 0b0000001_00000_00000_110_00000_0100111; + my $vd = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($rs1 << 15) | ($vd << 7)); +} + +sub vse64_v { + # vse64.v vd, (rs1) + my $template = 0b0000001_00000_00000_111_00000_0100111; + my $vd = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($rs1 << 15) | ($vd << 7)); +} + +sub vsetivli__x0_2_e64_m1_ta_ma { + # vsetivli x0, 2, e64, m1, ta, ma + return ".word 0xcd817057"; +} + +sub vsetivli__x0_4_e32_m1_ta_ma { + # vsetivli x0, 4, e32, m1, ta, ma + return ".word 0xcd027057"; +} + +sub vsetivli__x0_4_e64_m1_ta_ma { + # vsetivli x0,4,e64,m1,ta,ma + return ".word 0xcd827057"; +} + +sub vsetivli__x0_8_e32_m1_ta_ma { + return ".word 0xcd047057"; +} + +sub vslidedown_vi { + # vslidedown.vi vd, vs2, uimm + my $template = 0b0011111_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($vs2 << 20) | ($uimm << 15) | ($vd << 7)); +} + +sub vslideup_vi_v0t { + # vslideup.vi vd, vs2, uimm, v0.t + my $template = 0b0011100_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($vs2 << 20) | ($uimm << 15) | ($vd << 7)); +} + +sub vslideup_vi { + # vslideup.vi vd, vs2, uimm + my $template = 0b0011101_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($vs2 << 20) | ($uimm << 15) | ($vd << 7)); +} + +sub vsll_vi { + # vsll.vi vd, vs2, uimm, vm + my $template = 0b1001011_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($vs2 << 20) | ($uimm << 15) | ($vd << 7)); +} + +sub vsrl_vx { + # vsrl.vx vd, vs2, rs1 + my $template = 0b1010001_00000_00000_100_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($vs2 << 20) | ($rs1 << 15) | ($vd << 7)); +} + +sub vsse32_v { + # vse32.v vs3, (rs1), rs2 + my $template = 0b0000101_00000_00000_110_00000_0100111; + my $vs3 = read_vreg shift; + my $rs1 = read_reg shift; + my $rs2 = read_reg shift; + return ".word ".($template | ($rs2 << 20) | ($rs1 << 15) | ($vs3 << 7)); +} + +sub vsse64_v { + # vsse64.v vs3, (rs1), rs2 + my $template = 0b0000101_00000_00000_111_00000_0100111; + my $vs3 = read_vreg shift; + my $rs1 = read_reg shift; + my $rs2 = read_reg shift; + return ".word ".($template | ($rs2 << 20) | ($rs1 << 15) | ($vs3 << 7)); +} + +sub vxor_vv_v0t { + # vxor.vv vd, vs2, vs1, v0.t + my $template = 0b0010110_00000_00000_000_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15) | ($vd << 7)); +} + +sub vxor_vv { + # vxor.vv vd, vs2, vs1 + my $template = 0b0010111_00000_00000_000_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15) | ($vd << 7)); +} + +# Vector crypto instructions + +## Zvkb instructions + +sub vclmulh_vx { + # vclmulh.vx vd, vs2, rs1 + my $template = 0b0011011_00000_00000_110_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($vs2 << 20) | ($rs1 << 15) | ($vd << 7)); +} + +sub vclmul_vx_v0t { + # vclmul.vx vd, vs2, rs1, v0.t + my $template = 0b0011000_00000_00000_110_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($vs2 << 20) | ($rs1 << 15) | ($vd << 7)); +} + +sub vclmul_vx { + # vclmul.vx vd, vs2, rs1 + my $template = 0b0011001_00000_00000_110_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($vs2 << 20) | ($rs1 << 15) | ($vd << 7)); +} + +sub vrev8_v { + # vrev8.v vd, vs2 + my $template = 0b0100101_00000_01001_010_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +## Zvkg instructions + +sub vghsh_vv { + # vghsh.vv vd, vs2, vs1 + my $template = 0b1011001_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15) | ($vd << 7)); +} + +sub vgmul_vv { + # vgmul.vv vd, vs2 + my $template = 0b1010001_00000_10001_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +## Zvkned instructions + +sub vaesdf_vs { + # vaesdf.vs vd, vs2 + my $template = 0b101001_1_00000_00001_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +sub vaesdm_vs { + # vaesdm.vs vd, vs2 + my $template = 0b101001_1_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +sub vaesef_vs { + # vaesef.vs vd, vs2 + my $template = 0b101001_1_00000_00011_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +sub vaesem_vs { + # vaesem.vs vd, vs2 + my $template = 0b101001_1_00000_00010_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +sub vaeskf1_vi { + # vaeskf1.vi vd, vs2, uimmm + my $template = 0b100010_1_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($uimm << 15) | ($vs2 << 20) | ($vd << 7)); +} + +sub vaeskf2_vi { + # vaeskf2.vi vd, vs2, uimm + my $template = 0b101010_1_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($vs2 << 20) | ($uimm << 15) | ($vd << 7)); +} + +sub vaesz_vs { + # vaesz.vs vd, vs2 + my $template = 0b101001_1_00000_00111_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +## Zvknha and Zvknhb instructions + +sub vsha2ms_vv { + # vsha2ms.vv vd, vs2, vs1 + my $template = 0b1011011_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20)| ($vs1 << 15 )| ($vd << 7)); +} + +sub vsha2ch_vv { + # vsha2ch.vv vd, vs2, vs1 + my $template = 0b101110_10000_00000_001_00000_01110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20)| ($vs1 << 15 )| ($vd << 7)); +} + +sub vsha2cl_vv { + # vsha2cl.vv vd, vs2, vs1 + my $template = 0b101111_10000_00000_001_00000_01110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20)| ($vs1 << 15 )| ($vd << 7)); +} + +## Zvksed instructions + +sub vsm4k_vi { + # vsm4k.vi vd, vs2, uimm + my $template = 0b1000011_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($vs2 << 20) | ($uimm << 15) | ($vd << 7)); +} + +sub vsm4r_vs { + # vsm4r.vs vd, vs2 + my $template = 0b1010011_00000_10000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +## zvksh instructions + +sub vsm3c_vi { + # vsm3c.vi vd, vs2, uimm + my $template = 0b1010111_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($vs2 << 20) | ($uimm << 15 ) | ($vd << 7)); +} + +sub vsm3me_vv { + # vsm3me.vv vd, vs2, vs1 + my $template = 0b1000001_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15 ) | ($vd << 7)); +} + 1;
Signature · 7 lines
2.39.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH v2 10/16 RISC-V: crypto: add Zvkb accelerated GCM GHASH implementation
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
Add a gcm hash implementation using the Zvkb crypto extension. It gets possibly registered alongside the Zbc-based variant, with a higher priority so that the crypto subsystem will be able to select the most performant variant, but the algorithm itself will still be part of the crypto selftests that run during registration.
--- arch/riscv/crypto/Kconfig | 3 +- arch/riscv/crypto/Makefile | 8 +- arch/riscv/crypto/ghash-riscv64-glue.c | 147 ++++++++++ arch/riscv/crypto/ghash-riscv64-zvkb.pl | 349 ++++++++++++++++++++++++ 4 files changed, 505 insertions(+), 2 deletions(-) create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkb.pl
arch/riscv/crypto/Kconfig +2 -1
@@ -4,7 +4,7 @@ menu "Accelerated Cryptographic Algorithms for CPU (riscv)" config CRYPTO_GHASH_RISCV64 tristate "Hash functions: GHASH" - depends on 64BIT && RISCV_ISA_ZBC + depends on 64BIT && (RISCV_ISA_ZBC || RISCV_ISA_V) select CRYPTO_HASH select CRYPTO_LIB_GF128MUL help
@@ -12,5 +12,6 @@ config CRYPTO_GHASH_RISCV64 Architecture: riscv64 using one of: - ZBC extension + - ZVKB vector crypto extension endmenu
arch/riscv/crypto/Makefile +7 -1
@@ -8,6 +8,9 @@ ghash-riscv64-y := ghash-riscv64-glue.o ifdef CONFIG_RISCV_ISA_ZBC ghash-riscv64-y += ghash-riscv64-zbc.o endif +ifdef CONFIG_RISCV_ISA_V +ghash-riscv64-y += ghash-riscv64-zvkb.o +endif quiet_cmd_perlasm = PERLASM $@ cmd_perlasm = $(PERL) $(<) void $(@)
@@ -15,4 +18,7 @@ quiet_cmd_perlasm = PERLASM $@ $(obj)/ghash-riscv64-zbc.S: $(src)/ghash-riscv64-zbc.pl $(call cmd,perlasm) -clean-files += ghash-riscv64-zbc.S +$(obj)/ghash-riscv64-zvkb.S: $(src)/ghash-riscv64-zvkb.pl + $(call cmd,perlasm) + +clean-files += ghash-riscv64-zbc.S ghash-riscv64-zvkb.S
arch/riscv/crypto/ghash-riscv64-glue.c +147 -0
171 lines not on this page · raw message
arch/riscv/crypto/ghash-riscv64-zvkb.pl new file +349 -0
349 lines not on this page · raw message
Signature · 7 lines
2.39.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH v2 11/16 RISC-V: crypto: add Zvkg accelerated GCM GHASH implementation
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
When the Zvkg vector crypto extension is available another optimized gcm ghash variant is possible, so add it as another implmentation.
--- arch/riscv/crypto/Kconfig | 1 + arch/riscv/crypto/Makefile | 7 +- arch/riscv/crypto/ghash-riscv64-glue.c | 80 ++++++++++++ arch/riscv/crypto/ghash-riscv64-zvkg.pl | 161 ++++++++++++++++++++++++ 4 files changed, 247 insertions(+), 2 deletions(-) create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkg.pl
arch/riscv/crypto/Kconfig +1 -0
@@ -13,5 +13,6 @@ config CRYPTO_GHASH_RISCV64 Architecture: riscv64 using one of: - ZBC extension - ZVKB vector crypto extension + - ZVKG vector crypto extension endmenu
arch/riscv/crypto/Makefile +5 -2
@@ -9,7 +9,7 @@ ifdef CONFIG_RISCV_ISA_ZBC ghash-riscv64-y += ghash-riscv64-zbc.o endif ifdef CONFIG_RISCV_ISA_V -ghash-riscv64-y += ghash-riscv64-zvkb.o +ghash-riscv64-y += ghash-riscv64-zvkb.o ghash-riscv64-zvkg.o endif quiet_cmd_perlasm = PERLASM $@
@@ -21,4 +21,7 @@ $(obj)/ghash-riscv64-zbc.S: $(src)/ghash-riscv64-zbc.pl $(obj)/ghash-riscv64-zvkb.S: $(src)/ghash-riscv64-zvkb.pl $(call cmd,perlasm) -clean-files += ghash-riscv64-zbc.S ghash-riscv64-zvkb.S +$(obj)/ghash-riscv64-zvkg.S: $(src)/ghash-riscv64-zvkg.pl + $(call cmd,perlasm) + +clean-files += ghash-riscv64-zbc.S ghash-riscv64-zvkb.S ghash-riscv64-zvkg.S
arch/riscv/crypto/ghash-riscv64-glue.c +80 -0
98 lines not on this page · raw message
arch/riscv/crypto/ghash-riscv64-zvkg.pl new file +161 -0
161 lines not on this page · raw message
Signature · 7 lines
2.39.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH v2 12/16 RISC-V: crypto: add a vector-crypto-accelerated SHA256 implementation
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
This adds an accelerated SHA256 algorithm using either the Zvknha or Zvknhb vector crypto extensions. The spec says that
Zvknhb supports SHA-256 and SHA-512. Zvknha supports only SHA-256.
so the relevant acclerating instructions are included in both.
--- arch/riscv/crypto/Kconfig | 11 + arch/riscv/crypto/Makefile | 7 + arch/riscv/crypto/sha256-riscv64-glue.c | 114 +++++++++ arch/riscv/crypto/sha256-riscv64-zvknha.pl | 284 +++++++++++++++++++++ 4 files changed, 416 insertions(+) create mode 100644 arch/riscv/crypto/sha256-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha256-riscv64-zvknha.pl
arch/riscv/crypto/Kconfig +11 -0
@@ -15,4 +15,15 @@ config CRYPTO_GHASH_RISCV64 - ZVKB vector crypto extension - ZVKG vector crypto extension +config CRYPTO_SHA256_RISCV64 + tristate "Hash functions: SHA-256" + depends on 64BIT && RISCV_ISA_V + select CRYPTO_HASH + select CRYPTO_LIB_SHA256 + help + SHA-256 secure hash algorithm (FIPS 180) + + Architecture: riscv64 using + - Zvknha or Zvknhb vector crypto extensions + endmenu
arch/riscv/crypto/Makefile +7 -0
@@ -12,6 +12,9 @@ ifdef CONFIG_RISCV_ISA_V ghash-riscv64-y += ghash-riscv64-zvkb.o ghash-riscv64-zvkg.o endif +obj-$(CONFIG_CRYPTO_SHA256_RISCV64) += sha256-riscv64.o +sha256-riscv64-y := sha256-riscv64-glue.o sha256-riscv64-zvknhb.o + quiet_cmd_perlasm = PERLASM $@ cmd_perlasm = $(PERL) $(<) void $(@)
@@ -24,4 +27,8 @@ $(obj)/ghash-riscv64-zvkb.S: $(src)/ghash-riscv64-zvkb.pl $(obj)/ghash-riscv64-zvkg.S: $(src)/ghash-riscv64-zvkg.pl $(call cmd,perlasm) +$(obj)/sha256-riscv64-zvknhb.S: $(src)/sha256-riscv64-zvknha.pl + $(call cmd,perlasm) + clean-files += ghash-riscv64-zbc.S ghash-riscv64-zvkb.S ghash-riscv64-zvkg.S +clean-files += sha256-riscv64-zvknha.S
arch/riscv/crypto/sha256-riscv64-glue.c new file +114 -0
114 lines not on this page · raw message
arch/riscv/crypto/sha256-riscv64-zvknha.pl new file +284 -0
284 lines not on this page · raw message
Signature · 7 lines
2.39.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH v2 13/16 RISC-V: crypto: add a vector-crypto-accelerated SHA512 implementation
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
This adds an accelerated SHA512 algorithm using either the Zvknhb vector crypto extension.
--- arch/riscv/crypto/Kconfig | 11 + arch/riscv/crypto/Makefile | 8 +- arch/riscv/crypto/sha512-riscv64-glue.c | 104 ++++++ arch/riscv/crypto/sha512-riscv64-zvknhb.pl | 347 +++++++++++++++++++++ 4 files changed, 469 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/crypto/sha512-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha512-riscv64-zvknhb.pl
arch/riscv/crypto/Kconfig +11 -0
@@ -26,4 +26,15 @@ config CRYPTO_SHA256_RISCV64 Architecture: riscv64 using - Zvknha or Zvknhb vector crypto extensions +config CRYPTO_SHA512_RISCV64 + tristate "Hash functions: SHA-512" + depends on 64BIT && RISCV_ISA_V + select CRYPTO_HASH + select CRYPTO_SHA512 + help + SHA-512 secure hash algorithm (FIPS 180) + + Architecture: riscv64 + - Zvknhb vector crypto extension + endmenu
arch/riscv/crypto/Makefile +7 -1
@@ -15,6 +15,9 @@ endif obj-$(CONFIG_CRYPTO_SHA256_RISCV64) += sha256-riscv64.o sha256-riscv64-y := sha256-riscv64-glue.o sha256-riscv64-zvknhb.o +obj-$(CONFIG_CRYPTO_SHA512_RISCV64) += sha512-riscv64.o +sha512-riscv64-y := sha512-riscv64-glue.o sha512-riscv64-zvknhb.o + quiet_cmd_perlasm = PERLASM $@ cmd_perlasm = $(PERL) $(<) void $(@)
@@ -30,5 +33,8 @@ $(obj)/ghash-riscv64-zvkg.S: $(src)/ghash-riscv64-zvkg.pl $(obj)/sha256-riscv64-zvknhb.S: $(src)/sha256-riscv64-zvknha.pl $(call cmd,perlasm) +$(obj)/sha512-riscv64-zvknhb.S: $(src)/sha512-riscv64-zvknhb.pl + $(call cmd,perlasm) + clean-files += ghash-riscv64-zbc.S ghash-riscv64-zvkb.S ghash-riscv64-zvkg.S -clean-files += sha256-riscv64-zvknha.S +clean-files += sha256-riscv64-zvknha.S sha512-riscv64-zvknhb.S
arch/riscv/crypto/sha512-riscv64-glue.c new file +104 -0
104 lines not on this page · raw message
arch/riscv/crypto/sha512-riscv64-zvknhb.pl new file +347 -0
347 lines not on this page · raw message
Signature · 7 lines
2.39.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH v2 14/16 RISC-V: crypto: add Zvkned accelerated AES encryption implementation
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
This adds an AES implementation using the Zvkned vector crypto instructions.
--- arch/riscv/crypto/Kconfig | 14 + arch/riscv/crypto/Makefile | 7 + arch/riscv/crypto/aes-riscv-glue.c | 169 ++++++++ arch/riscv/crypto/aes-riscv64-zvkned.pl | 500 ++++++++++++++++++++++++ 4 files changed, 690 insertions(+) create mode 100644 arch/riscv/crypto/aes-riscv-glue.c create mode 100644 arch/riscv/crypto/aes-riscv64-zvkned.pl
arch/riscv/crypto/Kconfig +14 -0
@@ -2,6 +2,20 @@ menu "Accelerated Cryptographic Algorithms for CPU (riscv)" +config CRYPTO_AES_RISCV + tristate "Ciphers: AES (RISCV)" + depends on 64BIT && RISCV_ISA_V + select CRYPTO_AES + help + Block ciphers: AES cipher algorithms (FIPS-197) + Length-preserving ciphers: AES with ECB, CBC, CTR, CTS, + XCTR, and XTS modes + AEAD cipher: AES with CBC, ESSIV, and SHA-256 + for fscrypt and dm-crypt + + Architecture: riscv using one of + - Zvkns + config CRYPTO_GHASH_RISCV64 tristate "Hash functions: GHASH" depends on 64BIT && (RISCV_ISA_ZBC || RISCV_ISA_V)
arch/riscv/crypto/Makefile +7 -0
@@ -3,6 +3,9 @@ # linux/arch/riscv/crypto/Makefile # +obj-$(CONFIG_CRYPTO_AES_RISCV) += aes-riscv.o +aes-riscv-y := aes-riscv-glue.o aes-riscv64-zvkned.o + obj-$(CONFIG_CRYPTO_GHASH_RISCV64) += ghash-riscv64.o ghash-riscv64-y := ghash-riscv64-glue.o ifdef CONFIG_RISCV_ISA_ZBC
@@ -21,6 +24,9 @@ sha512-riscv64-y := sha512-riscv64-glue.o sha512-riscv64-zvknhb.o quiet_cmd_perlasm = PERLASM $@ cmd_perlasm = $(PERL) $(<) void $(@) +$(obj)/aes-riscv64-zvkned.S: $(src)/aes-riscv64-zvkned.pl + $(call cmd,perlasm) + $(obj)/ghash-riscv64-zbc.S: $(src)/ghash-riscv64-zbc.pl $(call cmd,perlasm)
@@ -36,5 +42,6 @@ $(obj)/sha256-riscv64-zvknhb.S: $(src)/sha256-riscv64-zvknha.pl $(obj)/sha512-riscv64-zvknhb.S: $(src)/sha512-riscv64-zvknhb.pl $(call cmd,perlasm) +clean-files += aes-riscv64-zvkned.S clean-files += ghash-riscv64-zbc.S ghash-riscv64-zvkb.S ghash-riscv64-zvkg.S clean-files += sha256-riscv64-zvknha.S sha512-riscv64-zvknhb.S
arch/riscv/crypto/aes-riscv-glue.c new file +169 -0
169 lines not on this page · raw message
arch/riscv/crypto/aes-riscv64-zvkned.pl new file +500 -0
500 lines not on this page · raw message
Signature · 7 lines
2.39.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH v2 15/16 RISC-V: crypto: add Zvksed accelerated SM4 encryption implementation
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
Add support for the SM4 symmetric cipher implemented using the special instructions provided by the Zvksed vector crypto instructions.
--- arch/riscv/crypto/Kconfig | 17 ++ arch/riscv/crypto/Makefile | 7 + arch/riscv/crypto/sm4-riscv64-glue.c | 163 ++++++++++++++ arch/riscv/crypto/sm4-riscv64-zvksed.pl | 270 ++++++++++++++++++++++++ 4 files changed, 457 insertions(+) create mode 100644 arch/riscv/crypto/sm4-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm4-riscv64-zvksed.pl
arch/riscv/crypto/Kconfig +17 -0
@@ -51,4 +51,21 @@ config CRYPTO_SHA512_RISCV64 Architecture: riscv64 - Zvknhb vector crypto extension +config CRYPTO_SM4_RISCV64 + tristate "Ciphers: SM4 (ShangMi 4)" + depends on 64BIT && RISCV_ISA_V + select CRYPTO_ALGAPI + select CRYPTO_SM4 + select CRYPTO_SM4_GENERIC + help + SM4 cipher algorithms (OSCCA GB/T 32907-2016, + ISO/IEC 18033-3:2010/Amd 1:2021) + + SM4 (GBT.32907-2016) is a cryptographic standard issued by the + Organization of State Commercial Administration of China (OSCCA) + as an authorized cryptographic algorithms for the use within China. + + Architecture: riscv64 + - Zvksed vector crypto extension + endmenu
arch/riscv/crypto/Makefile +7 -0
@@ -21,6 +21,9 @@ sha256-riscv64-y := sha256-riscv64-glue.o sha256-riscv64-zvknhb.o obj-$(CONFIG_CRYPTO_SHA512_RISCV64) += sha512-riscv64.o sha512-riscv64-y := sha512-riscv64-glue.o sha512-riscv64-zvknhb.o +obj-$(CONFIG_CRYPTO_SM4_RISCV64) += sm4-riscv64.o +sm4-riscv64-y := sm4-riscv64-glue.o sm4-riscv64-zvksed.o + quiet_cmd_perlasm = PERLASM $@ cmd_perlasm = $(PERL) $(<) void $(@)
@@ -42,6 +45,10 @@ $(obj)/sha256-riscv64-zvknhb.S: $(src)/sha256-riscv64-zvknha.pl $(obj)/sha512-riscv64-zvknhb.S: $(src)/sha512-riscv64-zvknhb.pl $(call cmd,perlasm) +$(obj)/sm4-riscv64-zvksed.S: $(src)/sm4-riscv64-zvksed.pl + $(call cmd,perlasm) + clean-files += aes-riscv64-zvkned.S clean-files += ghash-riscv64-zbc.S ghash-riscv64-zvkb.S ghash-riscv64-zvkg.S clean-files += sha256-riscv64-zvknha.S sha512-riscv64-zvknhb.S +clean-files += sm4-riscv64-zvksed.S
arch/riscv/crypto/sm4-riscv64-glue.c new file +163 -0
163 lines not on this page · raw message
arch/riscv/crypto/sm4-riscv64-zvksed.pl new file +270 -0
270 lines not on this page · raw message
Signature · 7 lines
2.39.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH v2 16/16 RISC-V: crypto: add Zvksh accelerated SM3 hash implementation
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
Add support for the SM3 hash function implemented using the special instructions provided by the Zvksh vector crypto instructions.
--- arch/riscv/crypto/Kconfig | 11 ++ arch/riscv/crypto/Makefile | 8 +- arch/riscv/crypto/sm3-riscv64-glue.c | 112 ++++++++++++++ arch/riscv/crypto/sm3-riscv64-zvksh.pl | 195 +++++++++++++++++++++++++ 4 files changed, 325 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/crypto/sm3-riscv64-glue.c create mode 100644 arch/riscv/crypto/sm3-riscv64-zvksh.pl
arch/riscv/crypto/Kconfig +11 -0
@@ -51,6 +51,17 @@ config CRYPTO_SHA512_RISCV64 Architecture: riscv64 - Zvknhb vector crypto extension +config CRYPTO_SM3_RISCV64 + tristate "Hash functions: SM3 (ShangMi 3)" + depends on 64BIT && RISCV_ISA_V + select CRYPTO_HASH + select CRYPTO_SM3 + help + SHA-512 secure hash algorithm (FIPS 180) + + Architecture: riscv64 + - Zvknhb vector crypto extension + config CRYPTO_SM4_RISCV64 tristate "Ciphers: SM4 (ShangMi 4)" depends on 64BIT && RISCV_ISA_V
arch/riscv/crypto/Makefile +7 -1
@@ -21,6 +21,9 @@ sha256-riscv64-y := sha256-riscv64-glue.o sha256-riscv64-zvknhb.o obj-$(CONFIG_CRYPTO_SHA512_RISCV64) += sha512-riscv64.o sha512-riscv64-y := sha512-riscv64-glue.o sha512-riscv64-zvknhb.o +obj-$(CONFIG_CRYPTO_SM3_RISCV64) += sm3-riscv64.o +sm3-riscv64-y := sm3-riscv64-glue.o sm3-riscv64-zvksh.o + obj-$(CONFIG_CRYPTO_SM4_RISCV64) += sm4-riscv64.o sm4-riscv64-y := sm4-riscv64-glue.o sm4-riscv64-zvksed.o
@@ -45,10 +48,13 @@ $(obj)/sha256-riscv64-zvknhb.S: $(src)/sha256-riscv64-zvknha.pl $(obj)/sha512-riscv64-zvknhb.S: $(src)/sha512-riscv64-zvknhb.pl $(call cmd,perlasm) +$(obj)/sm3-riscv64-zvksh.S: $(src)/sm3-riscv64-zvksh.pl + $(call cmd,perlasm) + $(obj)/sm4-riscv64-zvksed.S: $(src)/sm4-riscv64-zvksed.pl $(call cmd,perlasm) clean-files += aes-riscv64-zvkned.S clean-files += ghash-riscv64-zbc.S ghash-riscv64-zvkb.S ghash-riscv64-zvkg.S clean-files += sha256-riscv64-zvknha.S sha512-riscv64-zvknhb.S -clean-files += sm4-riscv64-zvksed.S +clean-files += sm3-riscv64-zvksh.S sm4-riscv64-zvksed.S
arch/riscv/crypto/sm3-riscv64-glue.c new file +112 -0
112 lines not on this page · raw message
arch/riscv/crypto/sm3-riscv64-zvksh.pl new file +195 -0
195 lines not on this page · raw message
Signature · 7 lines
2.39.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv