RISC-V: support some cryptography accelerations
Subsystems (2)
cover RISC-V: support some cryptography accelerations
From: Heiko Stuebner <[email protected]>
So this was my playground the last days.
The base is v13 of the vector patchset but the first patches up to doing the Zbc-based GCM GHash can also run without those. Of course the vector- crypto extensions are also not ratified yet, hence the marking as RFC.
As v13 of the vector patchset dropped the patches for in-kernel usage of vector instructions, I picked the ones from v12 over into this series for now.
My basic goal was to not re-invent cryptographic code, so the heavy lifting is done by those perl-asm scripts used in openssl and the perl code used here-in stems from code that is targetted at openssl [0] and is unmodified from there to limit needed review effort.
With a matching qemu (there are patches for vector-crypto flying around) the in-kernel crypto-selftests (also the extended ones) are very happy so far.
Things to do:
- The series still includes its own Zbb extension detection, I'll need to untangle my branches to adapt to the applied Zbb patchset [and address Andrews comments]
- use correct Co-developed-attribution for the code coming from openssl
- some more cryptographic algorithms (aes, sha512, sm3, sm4)
[0] both still open
https://github.com/openssl/openssl/pull/20078
https://github.com/openssl/openssl/pull/20149
Greentime Hu (2):
riscv: Add support for kernel mode vector
riscv: Add vector extension XOR implementation
Heiko Stuebner (10):
RISC-V: add Zbb extension detection
RISC-V: add Zbc extension detection
RISC-V: add Zbkb extension detection
RISC-V: hook new crypto subdir into build-system
RISC-V: crypto: add accelerated GCM GHASH implementation
RISC-V: add vector crypto extension detection
RISC-V: crypto: update perl include with helpers for vector (crypto)
instructions
RISC-V: crypto: add Zvkb accelerated GCM GHASH implementation
RISC-V: crypto: add Zvkg accelerated GCM GHASH implementation
RISC-V: crypto: add a vector-crypto-accelerated SHA256 implementation
arch/riscv/Kbuild | 1 + arch/riscv/Kconfig | 45 ++ arch/riscv/crypto/Kconfig | 28 + arch/riscv/crypto/Makefile | 34 + arch/riscv/crypto/ghash-riscv64-glue.c | 489 ++++++++++++++ arch/riscv/crypto/ghash-riscv64-zbc.pl | 400 ++++++++++++ arch/riscv/crypto/ghash-riscv64-zvkb.pl | 346 ++++++++++ arch/riscv/crypto/ghash-riscv64-zvkg.pl | 172 +++++ arch/riscv/crypto/riscv.pm | 723 +++++++++++++++++++++ arch/riscv/crypto/sha256-riscv64-glue.c | 103 +++ arch/riscv/crypto/sha256-riscv64-zvknha.pl | 502 ++++++++++++++ arch/riscv/include/asm/hwcap.h | 7 + arch/riscv/include/asm/vector.h | 14 + arch/riscv/include/asm/xor.h | 82 +++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/cpu.c | 7 + arch/riscv/kernel/cpufeature.c | 7 + arch/riscv/kernel/kernel_mode_vector.c | 132 ++++ arch/riscv/lib/Makefile | 1 + arch/riscv/lib/xor.S | 81 +++ crypto/Kconfig | 3 + 21 files changed, 3178 insertions(+) create mode 100644 arch/riscv/crypto/Kconfig create mode 100644 arch/riscv/crypto/Makefile create mode 100644 arch/riscv/crypto/ghash-riscv64-glue.c create mode 100644 arch/riscv/crypto/ghash-riscv64-zbc.pl create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkb.pl create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkg.pl create mode 100644 arch/riscv/crypto/riscv.pm create mode 100644 arch/riscv/crypto/sha256-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha256-riscv64-zvknha.pl create mode 100644 arch/riscv/include/asm/xor.h create mode 100644 arch/riscv/kernel/kernel_mode_vector.c create mode 100644 arch/riscv/lib/xor.S
Signature · 1 line
2.39.0
RFC PATCH 1/12 riscv: Add support for kernel mode vector
From: Greentime Hu <[email protected]>
Add kernel_rvv_begin() and kernel_rvv_end() function declarations and corresponding definitions in kernel_mode_vector.c
These are needed to wrap uses of vector in kernel mode.
--- arch/riscv/include/asm/vector.h | 14 +++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/kernel_mode_vector.c | 132 +++++++++++++++++++++++++ 3 files changed, 147 insertions(+) create mode 100644 arch/riscv/kernel/kernel_mode_vector.c
arch/riscv/include/asm/vector.h +14 -0
@@ -134,6 +134,20 @@ static inline void vstate_restore(struct task_struct *task, } } +static inline void vector_flush_cpu_state (void ) +{ + asmvolatile ( +"vsetvli t0, x0, e8, m8, ta, ma\n\t" +"vmv.v.i v0, 0\n\t" +"vmv.v.i v8, 0\n\t" +"vmv.v.i v16, 0\n\t" +"vmv.v.i v24, 0\n\t" + : : :"t0" ); +} + +void kernel_rvv_begin (void ); +void kernel_rvv_end (void ); +#else /* ! CONFIG_RISCV_ISA_V */ struct pt_regs ;
arch/riscv/kernel/Makefile +1 -0
@@ -56,6 +56,7 @@ obj-$(CONFIG_MMU) += vdso.o vdso/ obj-$(CONFIG_RISCV_M_MODE) += traps_misaligned.o obj-$(CONFIG_FPU) += fpu.o obj-$(CONFIG_RISCV_ISA_V) += vector.o +obj-$(CONFIG_RISCV_ISA_V) += kernel_mode_vector.o obj-$(CONFIG_SMP) += smpboot.o obj-$(CONFIG_SMP) += smp.o obj-$(CONFIG_SMP) += cpu_ops.o
arch/riscv/kernel/kernel_mode_vector.c new file +132 -0
132 lines not on this page · raw message
Signature · 1 line
2.39.0
RFC PATCH 2/12 riscv: Add vector extension XOR implementation
In reply to Heiko StuebnerFrom: Greentime Hu <[email protected]>
This patch adds support for vector optimized XOR and it is tested in qemu.
--- arch/riscv/include/asm/xor.h | 82 ++++++++++++++++++++++++++++++++++++ arch/riscv/lib/Makefile | 1 + arch/riscv/lib/xor.S | 81 +++++++++++++++++++++++++++++++++++ 3 files changed, 164 insertions(+) create mode 100644 arch/riscv/include/asm/xor.h create mode 100644 arch/riscv/lib/xor.S
arch/riscv/include/asm/xor.h new file +82 -0
@@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* Copyright (C) 2021 SiFive +*/ + +#include <linux/hardirq.h> +#include <asm-generic/xor.h> +#ifdef CONFIG_VECTOR +#include <asm/vector.h> +#include <asm/switch_to.h> + +void xor_regs_2_ (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2); +void xor_regs_3_ (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2, +const unsigned long *__restrict p3); +void xor_regs_4_ (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2, +const unsigned long *__restrict p3, +const unsigned long *__restrict p4); +void xor_regs_5_ (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2, +const unsigned long *__restrict p3, +const unsigned long *__restrict p4, +const unsigned long *__restrict p5); + +static void xor_rvv_2 (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2) +{ +kernel_rvv_begin (); +xor_regs_2_ (bytes, p1, p2); +kernel_rvv_end (); +} + +static void xor_rvv_3 (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2, +const unsigned long *__restrict p3) +{ +kernel_rvv_begin (); +xor_regs_3_ (bytes, p1, p2, p3); +kernel_rvv_end (); +} + +static void xor_rvv_4 (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2, +const unsigned long *__restrict p3, +const unsigned long *__restrict p4) +{ +kernel_rvv_begin (); +xor_regs_4_ (bytes, p1, p2, p3, p4); +kernel_rvv_end (); +} + +static void xor_rvv_5 (unsigned long bytes,unsigned long *__restrict p1, +const unsigned long *__restrict p2, +const unsigned long *__restrict p3, +const unsigned long *__restrict p4, +const unsigned long *__restrict p5) +{ +kernel_rvv_begin (); +xor_regs_5_ (bytes, p1, p2, p3, p4, p5); +kernel_rvv_end (); +} + +static struct xor_block_template xor_block_rvv = { + .name ="rvv" , + .do_2 = xor_rvv_2, + .do_3 = xor_rvv_3, + .do_4 = xor_rvv_4, + .do_5 = xor_rvv_5 +}; + +#undef XOR_TRY_TEMPLATES +#define XOR_TRY_TEMPLATES \ + do { \ + xor_speed(&xor_block_8regs); \ + xor_speed(&xor_block_32regs); \ + if (has_vector()) { \ + xor_speed(&xor_block_rvv);\ + } \ + } while (0) +#endif
arch/riscv/lib/Makefile +1 -0
@@ -7,3 +7,4 @@ lib-$(CONFIG_MMU) += uaccess.o lib-$(CONFIG_64BIT) += tishift.o obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o +lib-$(CONFIG_VECTOR) += xor.o
arch/riscv/lib/xor.S new file +81 -0
81 lines not on this page · raw message
Signature · 1 line
2.39.0
RFC PATCH 3/12 RISC-V: add Zbb extension detection
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
Add handling for Zbb extension.
Zbb provides basic bit-manipulation instructions.
As multiple subsequent features want to check for zbb presence, add the extension handling without directly including code using it.
--- arch/riscv/Kconfig | 23 +++++++++++++++++++++++ arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/kernel/cpu.c | 1 + arch/riscv/kernel/cpufeature.c | 1 + 4 files changed, 26 insertions(+)
arch/riscv/Kconfig +23 -0
@@ -426,6 +426,29 @@ config RISCV_ISA_V If you don't know what to do here, say Y. +config TOOLCHAIN_HAS_ZBB + bool + default y + depends on !64BIT || $(cc-option,-mabi=lp64 -march=rv64ima_zbb) + depends on !32BIT || $(cc-option,-mabi=ilp32 -march=rv32ima_zbb) + depends on LLD_VERSION >= 150000 || LD_VERSION >= 23900 + depends on AS_IS_GNU + +config RISCV_ISA_ZBB + bool "Zbb extension support for bit manipulation instructions" + depends on TOOLCHAIN_HAS_ZBB + depends on !XIP_KERNEL && MMU + default y + help + Adds support to dynamically detect the presence of the ZBB + extension (basic bit manipulation) and enable its usage. + + The Zbb extension provides instructions to accelerate a number + of bit-specific operations (count bit population, sign extending, + bitrotation, etc). + + If you don't know what to do here, say Y. + config TOOLCHAIN_HAS_ZICBOM bool default y
arch/riscv/include/asm/hwcap.h +1 -0
@@ -59,6 +59,7 @@ enum riscv_isa_ext_id {RISCV_ISA_EXT_SSTC ,RISCV_ISA_EXT_SVINVAL ,RISCV_ISA_EXT_SVPBMT , +RISCV_ISA_EXT_ZBB ,RISCV_ISA_EXT_ZICBOM ,RISCV_ISA_EXT_ZIHINTPAUSE ,RISCV_ISA_EXT_ID_MAX
arch/riscv/kernel/cpu.c +1 -0
@@ -185,6 +185,7 @@ arch_initcall(riscv_cpuinfo_init);* New entries to this struct should follow the ordering rules described above. */ static struct riscv_isa_ext_data isa_ext_arr[] = { +__RISCV_ISA_EXT_DATA (zbb,RISCV_ISA_EXT_ZBB ),__RISCV_ISA_EXT_DATA (zicbom,RISCV_ISA_EXT_ZICBOM ),__RISCV_ISA_EXT_DATA (zihintpause,RISCV_ISA_EXT_ZIHINTPAUSE ),__RISCV_ISA_EXT_DATA (sscofpmf,RISCV_ISA_EXT_SSCOFPMF ),
arch/riscv/kernel/cpufeature.c +1 -0
@@ -234,6 +234,7 @@ printk("!!!! isa-string: %s\n\n\n", isa);SET_ISA_EXT_MAP ("sstc" ,RISCV_ISA_EXT_SSTC );SET_ISA_EXT_MAP ("svinval" ,RISCV_ISA_EXT_SVINVAL );SET_ISA_EXT_MAP ("svpbmt" ,RISCV_ISA_EXT_SVPBMT ); +SET_ISA_EXT_MAP ("zbb" ,RISCV_ISA_EXT_ZBB );SET_ISA_EXT_MAP ("zicbom" ,RISCV_ISA_EXT_ZICBOM );SET_ISA_EXT_MAP ("zihintpause" ,RISCV_ISA_EXT_ZIHINTPAUSE ); }
Signature · 1 line
2.39.0
RFC PATCH 4/12 RISC-V: add Zbc extension detection
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
Add handling for Zbc extension.
Zbc provides instruction for carry-less multiplication.
--- arch/riscv/Kconfig | 22 ++++++++++++++++++++++ arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/kernel/cpu.c | 1 + arch/riscv/kernel/cpufeature.c | 1 + 4 files changed, 25 insertions(+)
arch/riscv/Kconfig +22 -0
@@ -449,6 +449,28 @@ config RISCV_ISA_ZBB If you don't know what to do here, say Y. +config TOOLCHAIN_HAS_ZBC + bool + default y + depends on !64BIT || $(cc-option,-mabi=lp64 -march=rv64ima_zbc) + depends on !32BIT || $(cc-option,-mabi=ilp32 -march=rv32ima_zbc) + depends on LLD_VERSION >= 150000 || LD_VERSION >= 23900 + depends on AS_IS_GNU + +config RISCV_ISA_ZBC + bool "Zbc extension support for bit manipulation instructions" + depends on TOOLCHAIN_HAS_ZBC + depends on !XIP_KERNEL && MMU + default y + help + Adds support to dynamically detect the presence of the ZBC + extension (carry-less multiplication) and enable its usage. + + The Zbc extension provides instructions clmul, clmulh and clmulr + to accelerate carry-less multiplications. + + If you don't know what to do here, say Y. + config TOOLCHAIN_HAS_ZICBOM bool default y
arch/riscv/include/asm/hwcap.h +1 -0
@@ -60,6 +60,7 @@ enum riscv_isa_ext_id {RISCV_ISA_EXT_SVINVAL ,RISCV_ISA_EXT_SVPBMT ,RISCV_ISA_EXT_ZBB , +RISCV_ISA_EXT_ZBC ,RISCV_ISA_EXT_ZICBOM ,RISCV_ISA_EXT_ZIHINTPAUSE ,RISCV_ISA_EXT_ID_MAX
arch/riscv/kernel/cpu.c +1 -0
@@ -186,6 +186,7 @@ arch_initcall(riscv_cpuinfo_init);*/ static struct riscv_isa_ext_data isa_ext_arr[] = {__RISCV_ISA_EXT_DATA (zbb,RISCV_ISA_EXT_ZBB ), +__RISCV_ISA_EXT_DATA (zbc,RISCV_ISA_EXT_ZBC ),__RISCV_ISA_EXT_DATA (zicbom,RISCV_ISA_EXT_ZICBOM ),__RISCV_ISA_EXT_DATA (zihintpause,RISCV_ISA_EXT_ZIHINTPAUSE ),__RISCV_ISA_EXT_DATA (sscofpmf,RISCV_ISA_EXT_SSCOFPMF ),
arch/riscv/kernel/cpufeature.c +1 -0
@@ -235,6 +235,7 @@ printk("!!!! isa-string: %s\n\n\n", isa);SET_ISA_EXT_MAP ("svinval" ,RISCV_ISA_EXT_SVINVAL );SET_ISA_EXT_MAP ("svpbmt" ,RISCV_ISA_EXT_SVPBMT );SET_ISA_EXT_MAP ("zbb" ,RISCV_ISA_EXT_ZBB ); +SET_ISA_EXT_MAP ("zbc" ,RISCV_ISA_EXT_ZBC );SET_ISA_EXT_MAP ("zicbom" ,RISCV_ISA_EXT_ZICBOM );SET_ISA_EXT_MAP ("zihintpause" ,RISCV_ISA_EXT_ZIHINTPAUSE ); }
Signature · 1 line
2.39.0
RFC PATCH 5/12 RISC-V: add Zbkb extension detection
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
Add detection for Zbkb extension.
Zbkb is part of the set of scalar cryptography extensions and provides bitmanip instructions for cryptography, with them being a "subset of the Zbb extension particularly useful for cryptography".
Zbkb was ratified in january 2022.
Expect code using the extension to pre-encode zbkb instructions, so don't introduce special toolchain requirements for now.
--- arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/kernel/cpu.c | 1 + arch/riscv/kernel/cpufeature.c | 1 + 3 files changed, 3 insertions(+)
arch/riscv/include/asm/hwcap.h +1 -0
@@ -61,6 +61,7 @@ enum riscv_isa_ext_id {RISCV_ISA_EXT_SVPBMT ,RISCV_ISA_EXT_ZBB ,RISCV_ISA_EXT_ZBC , +RISCV_ISA_EXT_ZBKB ,RISCV_ISA_EXT_ZICBOM ,RISCV_ISA_EXT_ZIHINTPAUSE ,RISCV_ISA_EXT_ID_MAX
arch/riscv/kernel/cpu.c +1 -0
@@ -187,6 +187,7 @@ arch_initcall(riscv_cpuinfo_init);static struct riscv_isa_ext_data isa_ext_arr[] = {__RISCV_ISA_EXT_DATA (zbb,RISCV_ISA_EXT_ZBB ),__RISCV_ISA_EXT_DATA (zbc,RISCV_ISA_EXT_ZBC ), +__RISCV_ISA_EXT_DATA (zbkb,RISCV_ISA_EXT_ZBKB ),__RISCV_ISA_EXT_DATA (zicbom,RISCV_ISA_EXT_ZICBOM ),__RISCV_ISA_EXT_DATA (zihintpause,RISCV_ISA_EXT_ZIHINTPAUSE ),__RISCV_ISA_EXT_DATA (sscofpmf,RISCV_ISA_EXT_SSCOFPMF ),
arch/riscv/kernel/cpufeature.c +1 -0
@@ -236,6 +236,7 @@ printk("!!!! isa-string: %s\n\n\n", isa);SET_ISA_EXT_MAP ("svpbmt" ,RISCV_ISA_EXT_SVPBMT );SET_ISA_EXT_MAP ("zbb" ,RISCV_ISA_EXT_ZBB );SET_ISA_EXT_MAP ("zbc" ,RISCV_ISA_EXT_ZBC ); +SET_ISA_EXT_MAP ("zbkb" ,RISCV_ISA_EXT_ZBKB );SET_ISA_EXT_MAP ("zicbom" ,RISCV_ISA_EXT_ZICBOM );SET_ISA_EXT_MAP ("zihintpause" ,RISCV_ISA_EXT_ZIHINTPAUSE ); }
Signature · 7 lines
2.39.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH 6/12 RISC-V: hook new crypto subdir into build-system
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
Create a crypto subdirectory for added accelerated cryptography routines and hook it into the riscv Kbuild and the main crypto Kconfig.
--- arch/riscv/Kbuild | 1 + arch/riscv/crypto/Kconfig | 5 +++++ arch/riscv/crypto/Makefile | 4 ++++ crypto/Kconfig | 3 +++ 4 files changed, 13 insertions(+) create mode 100644 arch/riscv/crypto/Kconfig create mode 100644 arch/riscv/crypto/Makefile
arch/riscv/Kbuild +1 -0
@@ -2,6 +2,7 @@ obj-y += kernel/ mm/ net/ obj-$(CONFIG_BUILTIN_DTB) += boot/dts/ +obj-$(CONFIG_CRYPTO) += crypto/ obj-y += errata/ obj-$(CONFIG_KVM) += kvm/
arch/riscv/crypto/Kconfig new file +5 -0
@@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + +menu "Accelerated Cryptographic Algorithms for CPU (riscv)" + +endmenu
arch/riscv/crypto/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# linux/arch/riscv/crypto/Makefile +#
crypto/Kconfig +3 -0
@@ -1401,6 +1401,9 @@ endif if PPC source "arch/powerpc/crypto/Kconfig" endif +if RISCV +source "arch/riscv/crypto/Kconfig" +endif if S390 source "arch/s390/crypto/Kconfig" endif
Signature · 1 line
2.39.0
RFC PATCH 7/12 RISC-V: crypto: add accelerated GCM GHASH implementation
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
With different sets of available extensions a number of different implementation variants are possible. Quite a number of them are already implemented in openSSL or are in the process of being implemented, so pick the relevant openSSL coden and add suitable glue code similar to arm64 and powerpc to use it for kernel-specific cryptography.
The prioritization of the algorithms follows the ifdef chain for the assembly callbacks done in openssl but here algorithms will get registered separately so that all of them can be part of the crypto selftests.
The crypto subsystem will select the most performant of all registered algorithms on the running system but will selftest all registered ones.
In a first step this adds scalar variants using the Zbc, Zbb and possible Zbkb (bitmanip crypto extension) and the perl implementation stems from openSSL pull request on
https://github.com/openssl/openssl/pull/20078
--- arch/riscv/crypto/Kconfig | 11 + arch/riscv/crypto/Makefile | 14 + arch/riscv/crypto/ghash-riscv64-glue.c | 263 ++++++++++++++++ arch/riscv/crypto/ghash-riscv64-zbc.pl | 400 +++++++++++++++++++++++++ arch/riscv/crypto/riscv.pm | 230 ++++++++++++++ 5 files changed, 918 insertions(+) create mode 100644 arch/riscv/crypto/ghash-riscv64-glue.c create mode 100644 arch/riscv/crypto/ghash-riscv64-zbc.pl create mode 100644 arch/riscv/crypto/riscv.pm
arch/riscv/crypto/Kconfig +11 -0
@@ -2,4 +2,15 @@ menu "Accelerated Cryptographic Algorithms for CPU (riscv)" +config CRYPTO_GHASH_RISCV64 + tristate "Hash functions: GHASH" + depends on 64BIT && RISCV_ISA_ZBC + select CRYPTO_HASH + select CRYPTO_LIB_GF128MUL + help + GCM GHASH function (NIST SP800-38D) + + Architecture: riscv64 using one of: + - ZBC extension + endmenu
arch/riscv/crypto/Makefile +14 -0
@@ -2,3 +2,17 @@ # # linux/arch/riscv/crypto/Makefile # + +obj-$(CONFIG_CRYPTO_GHASH_RISCV64) += ghash-riscv64.o +ghash-riscv64-y := ghash-riscv64-glue.o +ifdef CONFIG_RISCV_ISA_ZBC +ghash-riscv64-y += ghash-riscv64-zbc.o +endif + +quiet_cmd_perlasm = PERLASM $@ + cmd_perlasm = $(PERL) $(<) void $(@) + +$(obj)/ghash-riscv64-zbc.S: $(src)/ghash-riscv64-zbc.pl + $(call cmd,perlasm) + +clean-files += ghash-riscv64-zbc.S
arch/riscv/crypto/ghash-riscv64-glue.c new file +263 -0
263 lines not on this page · raw message
arch/riscv/crypto/ghash-riscv64-zbc.pl new file +400 -0
400 lines not on this page · raw message
arch/riscv/crypto/riscv.pm new file +230 -0
230 lines not on this page · raw message
Signature · 7 lines
2.39.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
RFC PATCH 8/12 RISC-V: add vector crypto extension detection
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
Add detection for some extensions of the vector-crypto specification, namely
- Zvkb: Vector Bit-manipulation used in Cryptography
- Zvkg: Vector GCM/GMAC
- Zvknha and Zvknhb: NIST Algorithm Suite
As their use is very specific and will likely be limited to special places we expect current code to just pre-encode those instructions, so right now we don't introduce toolchain requirements.
--- arch/riscv/include/asm/hwcap.h | 4 ++++ arch/riscv/kernel/cpu.c | 4 ++++ arch/riscv/kernel/cpufeature.c | 4 ++++ 3 files changed, 12 insertions(+)
arch/riscv/include/asm/hwcap.h +4 -0
@@ -64,6 +64,10 @@ enum riscv_isa_ext_id {RISCV_ISA_EXT_ZBKB ,RISCV_ISA_EXT_ZICBOM ,RISCV_ISA_EXT_ZIHINTPAUSE , +RISCV_ISA_EXT_ZVKB , +RISCV_ISA_EXT_ZVKG , +RISCV_ISA_EXT_ZVKNHA , +RISCV_ISA_EXT_ZVKNHB , RISCV_ISA_EXT_ID_MAX };static_assert (RISCV_ISA_EXT_ID_MAX <=RISCV_ISA_EXT_MAX );
arch/riscv/kernel/cpu.c +4 -0
@@ -190,6 +190,10 @@ static struct riscv_isa_ext_data isa_ext_arr[] = {__RISCV_ISA_EXT_DATA (zbkb,RISCV_ISA_EXT_ZBKB ),__RISCV_ISA_EXT_DATA (zicbom,RISCV_ISA_EXT_ZICBOM ),__RISCV_ISA_EXT_DATA (zihintpause,RISCV_ISA_EXT_ZIHINTPAUSE ), +__RISCV_ISA_EXT_DATA (zvkb,RISCV_ISA_EXT_ZVKB ), +__RISCV_ISA_EXT_DATA (zvkg,RISCV_ISA_EXT_ZVKG ), +__RISCV_ISA_EXT_DATA (zvknha,RISCV_ISA_EXT_ZVKNHA ), +__RISCV_ISA_EXT_DATA (zvknhb,RISCV_ISA_EXT_ZVKNHB ),__RISCV_ISA_EXT_DATA (sscofpmf,RISCV_ISA_EXT_SSCOFPMF ),__RISCV_ISA_EXT_DATA (sstc,RISCV_ISA_EXT_SSTC ), __RISCV_ISA_EXT_DATA(svinval, RISCV_ISA_EXT_SVINVAL),
arch/riscv/kernel/cpufeature.c +4 -0
@@ -239,6 +239,10 @@ printk("!!!! isa-string: %s\n\n\n", isa);SET_ISA_EXT_MAP ("zbkb" ,RISCV_ISA_EXT_ZBKB );SET_ISA_EXT_MAP ("zicbom" ,RISCV_ISA_EXT_ZICBOM );SET_ISA_EXT_MAP ("zihintpause" ,RISCV_ISA_EXT_ZIHINTPAUSE ); +SET_ISA_EXT_MAP ("zvkb" ,RISCV_ISA_EXT_ZVKB ); +SET_ISA_EXT_MAP ("zvkg" ,RISCV_ISA_EXT_ZVKG ); +SET_ISA_EXT_MAP ("zvknha" ,RISCV_ISA_EXT_ZVKNHA ); +SET_ISA_EXT_MAP ("zvknhb" ,RISCV_ISA_EXT_ZVKNHB ); }#undef SET_ISA_EXT_MAP }
Signature · 1 line
2.39.0
RFC PATCH 9/12 RISC-V: crypto: update perl include with helpers for vector (crypto) instructions
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
The openSSL scripts use a number of helpers for handling vector instructions and instructions from the vector-crypto-extensions.
Therefore port these over from openSSL.
--- arch/riscv/crypto/riscv.pm | 493 +++++++++++++++++++++++++++++++++++++ 1 file changed, 493 insertions(+)
arch/riscv/crypto/riscv.pm +493 -0
@@ -48,6 +48,29 @@ sub read_reg { return $1; } +my @vregs = map("v$_",(0..31)); +my %vreglookup; +@vreglookup{@vregs} = @vregs; + +sub read_vreg { + my $vreg = lc shift; + if (!exists($vreglookup{$vreg})) { + my $trace = ""; + if ($have_stacktrace) { + $trace = Devel::StackTrace->new->as_string; + } + die("Unknown vector register ".$vreg."\n".$trace); + } + if (!($vreg =~ /^v([0-9]+)$/)) { + my $trace = ""; + if ($have_stacktrace) { + $trace = Devel::StackTrace->new->as_string; + } + die("Could not process vector register ".$vreg."\n".$trace); + } + return $1; +} + # Helper functions sub brev8_rv64i {
@@ -227,4 +250,474 @@ sub rev8 { return ".word ".($template | ($rs << 15) | ($rd << 7)); } +# Vector instructions + +sub vadd_vv { + # vadd.vv vd, vs2, vs1 + my $template = 0b0000001_00000_00000_000_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15) | ($vd << 7)); +} + +sub vand_vi { + # vand.vi vd, vs1, imm + my $template = 0b0010011_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs1 = read_vreg shift; + my $imm = shift; + return ".word ".($template | ($vs1 << 20) | ($imm << 15) | ($vd << 7)) +} + +sub vid_v { + # vid.v vd + my $template = 0b0101001_00000_10001_010_00000_1010111; + my $vd = read_vreg shift; + return ".word ".($template | ($vd << 7)); +} + +sub vl1re32_v { + # vl1re32.v vd, (rs) + my $template = 0b0000001_00000_00000_110_00000_0000111; + my $vd = read_vreg shift; + my $rs = read_reg shift; + return ".word ".($template | ($rs << 15)| ($vd << 7)); +} + +sub vl1re64_v { + # vl1re64.v vd, (rs) + my $template = 0b0000001_00000_00000_111_00000_0000111; + my $vd = read_vreg shift; + my $rs = read_reg shift; + return ".word ".($template | ($rs << 15)| ($vd << 7)); +} + +sub vle32_v { + # vle32.v vd, (rs1) + my $template = 0b0000001_00000_00000_110_00000_0000111; + my $vd = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($rs1 << 15) | ($vd << 7)); +} + +sub vle64_v { + # vle64.v vd, (rs1) + my $template = 0b0000001_00000_00000_111_00000_0000111; + my $vd = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($rs1 << 15) | ($vd << 7)); +} + +sub vlse32_v { + # vlse32.v vd, (rs1), rs2 + my $template = 0b0000101_00000_00000_110_00000_0000111; + my $vd = read_vreg shift; + my $rs1 = read_reg shift; + my $rs2 = read_reg shift; + return ".word ".($template | ($rs2 << 20) | ($rs1 << 15) | ($vd << 7)); +} + +sub vlse64_v { + # vlse64.v vd, (rs1), rs2 + my $template = 0b0000101_00000_00000_111_00000_0000111; + my $vd = read_vreg shift; + my $rs1 = read_reg shift; + my $rs2 = read_reg shift; + return ".word ".($template | ($rs2 << 20) | ($rs1 << 15) | ($vd << 7)); +} + +sub vmerge_vim { + # vmerge.vim vd, vs2, imm, v0 + my $template = 0b0101110_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $imm = shift; + return ".word ".($template | ($vs2 << 20) | ($imm << 15) | ($vd << 7)); +} + +sub vmerge_vvm { + # vmerge.vvm vd vs2 vs1 + my $template = 0b0101110_00000_00000_000_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15) | ($vd << 7)) +} + +sub vmseq_vi { + # vmseq vd vs1, imm + my $template = 0b0110001_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs1 = read_vreg shift; + my $imm = shift; + return ".word ".($template | ($vs1 << 20) | ($imm << 15) | ($vd << 7)) +} + +sub vmv_v_i { + # vmv.v.i vd, imm + my $template = 0b0101111_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $imm = shift; + return ".word ".($template | ($imm << 15) | ($vd << 7)); +} + +sub vmv_v_v { + # vmv.v.v vd, vs1 + my $template = 0b0101111_00000_00000_000_00000_1010111; + my $vd = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs1 << 15) | ($vd << 7)); +} + +sub vmv_v_x { + # vmv.v.x vd, rs1 + my $template = 0b0101111_00000_00000_100_00000_1010111; + my $vd = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($rs1 << 15) | ($vd << 7)); +} + +sub vor_vv_v0t { + # vor.vv vd, vs2, vs1, v0.t + my $template = 0b0010100_00000_00000_000_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15) | ($vd << 7)); +} + +sub vrgather_vv { + # vrgather.vv vd, vs2, vs1 + my $template = 0b0011001_00000_00000_000_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + if ($vd == $vs1 || $vd == $vs2) { + my $trace = Devel::StackTrace->new; + die("Source operands and destination operand must not overlap!\n".$trace->as_string); + } + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15) | ($vd << 7)); +} + +sub vse32_v { + # vse32.v vd, (rs1) + my $template = 0b0000001_00000_00000_110_00000_0100111; + my $vd = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($rs1 << 15) | ($vd << 7)); +} + +sub vse64_v { + # vse64.v vd, (rs1) + my $template = 0b0000001_00000_00000_111_00000_0100111; + my $vd = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($rs1 << 15) | ($vd << 7)); +} + +sub vsetivli__x0_2_e64_m1_ta_ma { + # vsetivli x0, 2, e64, m1, ta, ma + return ".word 0xcd817057"; +} + +sub vsetivli__x0_4_e32_m1_ta_ma { + # vsetivli x0, 4, e32, m1, ta, ma + return ".word 0xcd027057"; +} + +sub vsetivli__x0_4_e64_m1_ta_ma { + # vsetivli x0,4,e64,m1,ta,ma + return ".word 0xcd827057"; +} + +sub vslidedown_vi { + # vslidedown.vi vd, vs2, uimm + my $template = 0b0011111_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($vs2 << 20) | ($uimm << 15) | ($vd << 7)); +} + +sub vslideup_vi_v0t { + # vslideup.vi vd, vs2, uimm, v0.t + my $template = 0b0011100_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($vs2 << 20) | ($uimm << 15) | ($vd << 7)); +} + +sub vslideup_vi { + # vslideup.vi vd, vs2, uimm + my $template = 0b0011101_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($vs2 << 20) | ($uimm << 15) | ($vd << 7)); +} + +sub vsll_vi { + # vsll.vi vd, vs2, uimm, vm + my $template = 0b1001011_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($vs2 << 20) | ($uimm << 15) | ($vd << 7)); +} + +sub vsrl_vx { + # vsrl.vx vd, vs2, rs1 + my $template = 0b1010001_00000_00000_100_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($vs2 << 20) | ($rs1 << 15) | ($vd << 7)); +} + +sub vsse32_v { + # vse32.v vs3, (rs1), rs2 + my $template = 0b0000101_00000_00000_110_00000_0100111; + my $vs3 = read_vreg shift; + my $rs1 = read_reg shift; + my $rs2 = read_reg shift; + return ".word ".($template | ($rs2 << 20) | ($rs1 << 15) | ($vs3 << 7)); +} + +sub vsse64_v { + # vsse64.v vs3, (rs1), rs2 + my $template = 0b0000101_00000_00000_111_00000_0100111; + my $vs3 = read_vreg shift; + my $rs1 = read_reg shift; + my $rs2 = read_reg shift; + return ".word ".($template | ($rs2 << 20) | ($rs1 << 15) | ($vs3 << 7)); +} + +sub vxor_vi { + # vxor.vi vd, vs2, imm + my $template = 0b0010111_00000_00000_011_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $imm = shift; + return ".word ".($template | ($vs2 << 20) | ($imm << 15) | ($vd << 7)); +} + +sub vxor_vv_v0t { + # vxor.vv vd, vs2, vs1, v0.t + my $template = 0b0010110_00000_00000_000_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15) | ($vd << 7)); +} + +sub vxor_vv { + # vxor.vv vd, vs2, vs1 + my $template = 0b0010111_00000_00000_000_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15) | ($vd << 7)); +} + +# Vector crypto instructions + +## Zvkb instructions + +sub vclmulh_vx { + # vclmulh.vx vd, vs2, rs1 + my $template = 0b0011011_00000_00000_110_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($vs2 << 20) | ($rs1 << 15) | ($vd << 7)); +} + +sub vclmul_vx_v0t { + # vclmul.vx vd, vs2, rs1, v0.t + my $template = 0b0011000_00000_00000_110_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($vs2 << 20) | ($rs1 << 15) | ($vd << 7)); +} + +sub vclmul_vx { + # vclmul.vx vd, vs2, rs1 + my $template = 0b0011001_00000_00000_110_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $rs1 = read_reg shift; + return ".word ".($template | ($vs2 << 20) | ($rs1 << 15) | ($vd << 7)); +} + +sub vrev8_v { + # vrev8.v vd, vs2 + my $template = 0b0100101_00000_01001_010_00000_1010111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +## Zvkg instructions + +sub vghmac_vv { + # vhgmac.vv vd, vs2, vs1 + my $template = 0b1011001_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + + return ".word ".($template | ($vs2 << 20) | ($vs1 << 15) | ($vd << 7)); +} + +## Zvknha and Zvknhb instructions + +sub vsha2ms_vv { + # vsha2ms.vv vd, vs2, vs1 + my $template = 0b1011011_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20)| ($vs1 << 15 )| ($vd << 7)); +} + +sub vsha2cl_vv { + # vsha2cl.vv vd, vs2, vs1 + my $template = 0b101111_10000_00000_001_00000_01110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20)| ($vs1 << 15 )| ($vd << 7)); +} + +sub vsha2ch_vv { + # vsha2ch.vv vd, vs2, vs1 + my $template = 0b101110_10000_00000_001_00000_01110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $vs1 = read_vreg shift; + return ".word ".($template | ($vs2 << 20)| ($vs1 << 15 )| ($vd << 7)); +} + +## Zvkns instructions + +sub vaesdf_vs { + # vaesdf.vs vd, vs2 + my $template = 0b101001_1_00000_00001_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +sub vaesdf_vv { + # vaesdf.vv vd, vs2 + my $template = 0b101000_1_00000_00001_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +sub vaesdm_vs { + # vaesdm.vs vd, vs2 + my $template = 0b101001_1_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +sub vaesdm_vv { + # vaesdm.vv vd, vs2 + my $template = 0b101000_1_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +sub vaesef_vs { + # vaesef.vs vd, vs2 + my $template = 0b101001_1_00000_00011_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +sub vaesef_vv { + # vaesef.vv vd, vs2 + my $template = 0b101000_1_00000_00011_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +sub vaesem_vs { + # vaesem.vs vd, vs2 + my $template = 0b101001_1_00000_00010_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +sub vaesem_vv { + # vaesem.vv vd, vs2 + my $template = 0b101000_1_00000_00010_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +sub vaeskf1_vi { + # vaeskf1.vi vd, vs2, uimmm + my $template = 0b100010_1_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($uimm << 15) | ($vs2 << 20) | ($vd << 7)); +} + +sub vaeskf2_vi { + # vaeskf2.vi vd, vs2, uimm + my $template = 0b101010_1_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($vs2 << 20) | ($uimm << 15) | ($vd << 7)); +} + +sub vaesz_vs { + # vaesz.vs vd, vs2 + my $template = 0b101001_1_00000_00111_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +## Zvksed instructions + +sub vsm4k_vi { + # vsm4k.vi vd, vs2, uimm + my $template = 0b1000011_00000_00000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + my $uimm = shift; + return ".word ".($template | ($vs2 << 20) | ($uimm << 15) | ($vd << 7)); +} + +sub vsm4r_vv { + # vsm4r.vv vd, vs2 + my $template = 0b1010001_00000_10000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + +sub vsm4r_vs { + # vsm4r.vs vd, vs2 + my $template = 0b1010011_00000_10000_010_00000_1110111; + my $vd = read_vreg shift; + my $vs2 = read_vreg shift; + return ".word ".($template | ($vs2 << 20) | ($vd << 7)); +} + 1;
Signature · 1 line
2.39.0
RFC PATCH 10/12 RISC-V: crypto: add Zvkb accelerated GCM GHASH implementation
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
Add a gcm hash implementation using the Zvkb crypto extension. It gets possibly registered alongside the Zbc-based variant, with a higher priority so that the crypto subsystem will be able to select the most performant variant, but the algorithm itself will still be part of the crypto selftests that run during registration.
--- arch/riscv/crypto/Kconfig | 3 +- arch/riscv/crypto/Makefile | 8 +- arch/riscv/crypto/ghash-riscv64-glue.c | 146 ++++++++++ arch/riscv/crypto/ghash-riscv64-zvkb.pl | 346 ++++++++++++++++++++++++ 4 files changed, 501 insertions(+), 2 deletions(-) create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkb.pl
arch/riscv/crypto/Kconfig +2 -1
@@ -4,7 +4,7 @@ menu "Accelerated Cryptographic Algorithms for CPU (riscv)" config CRYPTO_GHASH_RISCV64 tristate "Hash functions: GHASH" - depends on 64BIT && RISCV_ISA_ZBC + depends on 64BIT && (RISCV_ISA_ZBC || RISCV_ISA_V) select CRYPTO_HASH select CRYPTO_LIB_GF128MUL help
@@ -12,5 +12,6 @@ config CRYPTO_GHASH_RISCV64 Architecture: riscv64 using one of: - ZBC extension + - ZVKB vector crypto extension endmenu
arch/riscv/crypto/Makefile +7 -1
@@ -8,6 +8,9 @@ ghash-riscv64-y := ghash-riscv64-glue.o ifdef CONFIG_RISCV_ISA_ZBC ghash-riscv64-y += ghash-riscv64-zbc.o endif +ifdef CONFIG_RISCV_ISA_V +ghash-riscv64-y += ghash-riscv64-zvkb.o +endif quiet_cmd_perlasm = PERLASM $@ cmd_perlasm = $(PERL) $(<) void $(@)
@@ -15,4 +18,7 @@ quiet_cmd_perlasm = PERLASM $@ $(obj)/ghash-riscv64-zbc.S: $(src)/ghash-riscv64-zbc.pl $(call cmd,perlasm) -clean-files += ghash-riscv64-zbc.S +$(obj)/ghash-riscv64-zvkb.S: $(src)/ghash-riscv64-zvkb.pl + $(call cmd,perlasm) + +clean-files += ghash-riscv64-zbc.S ghash-riscv64-zvkb.S
arch/riscv/crypto/ghash-riscv64-glue.c +146 -0
170 lines not on this page · raw message
arch/riscv/crypto/ghash-riscv64-zvkb.pl new file +346 -0
346 lines not on this page · raw message
Signature · 1 line
2.39.0
RFC PATCH 11/12 RISC-V: crypto: add Zvkg accelerated GCM GHASH implementation
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
When the Zvkg vector crypto extension is available another optimized gcm ghash variant is possible, so add it as another implmentation.
--- arch/riscv/crypto/Kconfig | 1 + arch/riscv/crypto/Makefile | 7 +- arch/riscv/crypto/ghash-riscv64-glue.c | 80 +++++++++++ arch/riscv/crypto/ghash-riscv64-zvkg.pl | 172 ++++++++++++++++++++++++ 4 files changed, 258 insertions(+), 2 deletions(-) create mode 100644 arch/riscv/crypto/ghash-riscv64-zvkg.pl
arch/riscv/crypto/Kconfig +1 -0
@@ -13,5 +13,6 @@ config CRYPTO_GHASH_RISCV64 Architecture: riscv64 using one of: - ZBC extension - ZVKB vector crypto extension + - ZVKG vector crypto extension endmenu
arch/riscv/crypto/Makefile +5 -2
@@ -9,7 +9,7 @@ ifdef CONFIG_RISCV_ISA_ZBC ghash-riscv64-y += ghash-riscv64-zbc.o endif ifdef CONFIG_RISCV_ISA_V -ghash-riscv64-y += ghash-riscv64-zvkb.o +ghash-riscv64-y += ghash-riscv64-zvkb.o ghash-riscv64-zvkg.o endif quiet_cmd_perlasm = PERLASM $@
@@ -21,4 +21,7 @@ $(obj)/ghash-riscv64-zbc.S: $(src)/ghash-riscv64-zbc.pl $(obj)/ghash-riscv64-zvkb.S: $(src)/ghash-riscv64-zvkb.pl $(call cmd,perlasm) -clean-files += ghash-riscv64-zbc.S ghash-riscv64-zvkb.S +$(obj)/ghash-riscv64-zvkg.S: $(src)/ghash-riscv64-zvkg.pl + $(call cmd,perlasm) + +clean-files += ghash-riscv64-zbc.S ghash-riscv64-zvkb.S ghash-riscv64-zvkg.S
arch/riscv/crypto/ghash-riscv64-glue.c +80 -0
98 lines not on this page · raw message
arch/riscv/crypto/ghash-riscv64-zvkg.pl new file +172 -0
172 lines not on this page · raw message
Signature · 1 line
2.39.0
RFC PATCH 12/12 RISC-V: crypto: add a vector-crypto-accelerated SHA256 implementation
In reply to Heiko StuebnerFrom: Heiko Stuebner <[email protected]>
This adds an accelerated SHA256 algorithm using either the Zvknha or Zvknhb vector crypto extensions. The spec says that
Zvknhb supports SHA-256 and SHA-512. Zvknha supports only SHA-256.
so the relevant acclerating instructions are included in both.
--- arch/riscv/crypto/Kconfig | 10 + arch/riscv/crypto/Makefile | 7 + arch/riscv/crypto/sha256-riscv64-glue.c | 103 +++++ arch/riscv/crypto/sha256-riscv64-zvknha.pl | 502 +++++++++++++++++++++ 4 files changed, 622 insertions(+) create mode 100644 arch/riscv/crypto/sha256-riscv64-glue.c create mode 100644 arch/riscv/crypto/sha256-riscv64-zvknha.pl
arch/riscv/crypto/Kconfig +10 -0
@@ -15,4 +15,14 @@ config CRYPTO_GHASH_RISCV64 - ZVKB vector crypto extension - ZVKG vector crypto extension +config CRYPTO_SHA256_RISCV64 + tristate "Hash functions: SHA-256" + depends on 64BIT && RISCV_ISA_V + select CRYPTO_HASH + help + SHA-256 secure hash algorithm (FIPS 180) + + Architecture: riscv64 using + - Zvknha or Zvknhb vector crypto extensions + endmenu
arch/riscv/crypto/Makefile +7 -0
@@ -12,6 +12,9 @@ ifdef CONFIG_RISCV_ISA_V ghash-riscv64-y += ghash-riscv64-zvkb.o ghash-riscv64-zvkg.o endif +obj-$(CONFIG_CRYPTO_SHA256_RISCV64) += sha256-riscv64.o +sha256-riscv64-y := sha256-riscv64-glue.o sha256-riscv64-zvknhb.o + quiet_cmd_perlasm = PERLASM $@ cmd_perlasm = $(PERL) $(<) void $(@)
@@ -24,4 +27,8 @@ $(obj)/ghash-riscv64-zvkb.S: $(src)/ghash-riscv64-zvkb.pl $(obj)/ghash-riscv64-zvkg.S: $(src)/ghash-riscv64-zvkg.pl $(call cmd,perlasm) +$(obj)/sha256-riscv64-zvknhb.S: $(src)/sha256-riscv64-zvknha.pl + $(call cmd,perlasm) + clean-files += ghash-riscv64-zbc.S ghash-riscv64-zvkb.S ghash-riscv64-zvkg.S +clean-files += sha256-riscv64-zvknha.S
arch/riscv/crypto/sha256-riscv64-glue.c new file +103 -0
103 lines not on this page · raw message
arch/riscv/crypto/sha256-riscv64-zvknha.pl new file +502 -0
502 lines not on this page · raw message
Signature · 1 line
2.39.0