Add RISC-V vector cryptographic instruction set support
Subsystems (4)
v3 cover Add RISC-V vector cryptographic instruction set support
This patchset provides an implementation for Zvbb, Zvbc, Zvkned, Zvknh, Zvksh, Zvkg, and Zvksed of the draft RISC-V vector cryptography extensions as per the v20230425 version of the specification(1) (6a7ae7f2). This is an update to the patchset submitted to qemu-devel on Monday, 17 Apr 2023 14:58:36 +0100.
v2:
squashed commits into one commit per extension with separate commits for each refactoring
unified trans_rvzvk*.c.inc files into one trans_rvvk.c.inc style fixes in insn32.decode and other files
added macros for EGS values in translation functions. updated from v20230303 to v20230407 of the spec:
Zvkb has been split into Zvbb and Zvbc
vbrev, vclz, vctz, vcpop and vwsll have been added to Zvbb.
v3:
New patch 03/19 removes redundant “cpu_vl == 0” checks from trans_rvv.c.inc Introduction of new tcg ops has been factored out of patch 11/19 and into 09/19
These ops are now added to non riscv-specific files
As v20230425 is a freeze candidate, we are not expecting any significant changes to the specification or this patch series.
Please note that the Zvkt data-independent execution latency extension (and all extensions including it) has not been implemented, and we would recommend not using these patches in an environment where timing attacks are an issue.
Work performed by Dickon, Lawrence, Nazar, Kiran, and William from Codethink sponsored by SiFive, as well as Max Chou and Frank Chang from SiFive.
For convenience we have created a git repo with our patches on top of a recent master. https://github.com/CodethinkLabs/qemu-ct
https://github.com/riscv/riscv-crypto/releases
Thanks to those who have already reviewed:
Richard Henderson [email protected]
[PATCH v2 02/17] target/riscv: Refactor vector-vector translation macro [PATCH v2 04/17] target/riscv: Move vector translation checks
[PATCH v2 05/17] target/riscv: Refactor translation of vector-widening instruction [PATCH v2 07/17] qemu/bitops.h: Limit rotate amounts
[PATCH v2 08/17] qemu/host-utils.h: Add clz and ctz functions for lower-bit integers [PATCH v2 14/17] crypto: Create sm4_subword
Alistair Francis [email protected]
[PATCH v2 02/17] target/riscv: Refactor vector-vector translation macro
Philipp Tomsich [email protected]
Various v1 reviews
Christoph Müllner [email protected]
Various v1 reviews
Dickon Hood (3):
target/riscv: Refactor translation of vector-widening instruction qemu/bitops.h: Limit rotate amounts
target/riscv: Add Zvbb ISA extension support
Kiran Ostrolenk (5):
target/riscv: Refactor some of the generic vector functionality target/riscv: Refactor vector-vector translation macro target/riscv: Refactor some of the generic vector functionality qemu/host-utils.h: Add clz and ctz functions for lower-bit integers target/riscv: Add Zvknh ISA extension support
Lawrence Hunter (2):
target/riscv: Add Zvbc ISA extension support
target/riscv: Add Zvksh ISA extension support
Max Chou (3):
crypto: Create sm4_subword
crypto: Add SM4 constant parameter CK
target/riscv: Add Zvksed ISA extension support
Nazar Kazakov (6):
target/riscv: Remove redundant "cpu_vl == 0" checks target/riscv: Move vector translation checks
tcg: Add andcs and rotrs tcg gvec ops target/riscv: Add Zvkned ISA extension support target/riscv: Add Zvkg ISA extension support target/riscv: Expose Zvk* and Zvb[b,c] cpu properties
accel/tcg/tcg-runtime-gvec.c | 11 + accel/tcg/tcg-runtime.h | 1 + crypto/sm4.c | 10 + include/crypto/sm4.h | 9 + include/qemu/bitops.h | 24 +- include/qemu/host-utils.h | 54 ++ include/tcg/tcg-op-gvec.h | 4 + target/arm/tcg/crypto_helper.c | 10 +- target/riscv/cpu.c | 39 + target/riscv/cpu.h | 8 + target/riscv/helper.h | 95 ++ target/riscv/insn32.decode | 58 ++ target/riscv/insn_trans/trans_rvv.c.inc | 174 ++-- target/riscv/insn_trans/trans_rvvk.c.inc | 593 ++++++++++++ target/riscv/meson.build | 4 +- target/riscv/op_helper.c | 6 + target/riscv/translate.c | 1 + target/riscv/vcrypto_helper.c | 1052 ++++++++++++++++++++++ target/riscv/vector_helper.c | 243 +---- target/riscv/vector_internals.c | 81 ++ target/riscv/vector_internals.h | 228 +++++ tcg/tcg-op-gvec.c | 23 + 22 files changed, 2365 insertions(+), 363 deletions(-) create mode 100644 target/riscv/insn_trans/trans_rvvk.c.inc create mode 100644 target/riscv/vcrypto_helper.c create mode 100644 target/riscv/vector_internals.c create mode 100644 target/riscv/vector_internals.h
Signature · 1 line
2.40.1
PATCH v3 1/19 target/riscv: Refactor some of the generic vector functionality
From: Kiran Ostrolenk <[email protected]>
Take some functions/macros out of vector_helper and put them in a new module called vector_internals. This ensures they can be used by both vector and vector-crypto helpers (latter implemented in proceeding commits).
--- target/riscv/meson.build | 1 + target/riscv/vector_helper.c | 201 +------------------------------- target/riscv/vector_internals.c | 81 +++++++++++++ target/riscv/vector_internals.h | 182 +++++++++++++++++++++++++++++ 4 files changed, 265 insertions(+), 200 deletions(-) create mode 100644 target/riscv/vector_internals.c create mode 100644 target/riscv/vector_internals.h
target/riscv/meson.build +1 -0
@@ -16,6 +16,7 @@ riscv_ss.add(files( 'gdbstub.c', 'op_helper.c', 'vector_helper.c', + 'vector_internals.c', 'bitmanip_helper.c', 'translate.c', 'm128_helper.c',
target/riscv/vector_helper.c +1 -200
266 lines not on this page · raw message
target/riscv/vector_internals.c new file +81 -0
81 lines not on this page · raw message
target/riscv/vector_internals.h new file +182 -0
182 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH v3 01/19] target/riscv: Refactor some of the generic vector functionality
Quoted 9 lines · Lawrence Hunter
From: Kiran Ostrolenk <[email protected]>
Take some functions/macros out of
vector_helperand put them in a new module calledvector_internals. This ensures they can be used by both vector and vector-crypto helpers (latter implemented in proceeding commits).Signed-off-by Kiran Ostrolenk <[email protected]>---
Weiwei Li
Quoted 575 lines
target/riscv/meson.build | 1 + target/riscv/vector_helper.c | 201 +------------------------------- target/riscv/vector_internals.c | 81 +++++++++++++ target/riscv/vector_internals.h | 182 +++++++++++++++++++++++++++++ 4 files changed, 265 insertions(+), 200 deletions(-)create mode 100644
target/riscv/vector_internals.ccreate mode 100644target/riscv/vector_internals.htarget/riscv/meson.build +1
-0@@ -16,6 +16,7 @@ riscv_ss.add(files( 'gdbstub.c', 'op_helper.c', 'vector_helper.c', + 'vector_internals.c', 'bitmanip_helper.c', 'translate.c', 'm128_helper.c',target/riscv/vector_helper.c +1
-200@@ -26,6 +26,7 @@#include "fpu/softfloat.h" #include "tcg/tcg-gvec-desc.h" #include "internals.h" +#include "vector_internals.h" #include <math.h> target_ulong HELPER(vsetvl)(CPURISCVState *env, target_ulong s1,@@ -75,68 +76,6 @@ target_ulong HELPER(vsetvl)(CPURISCVState *env, target_ulong s1,return vl; } -/* -* Note that vector data is stored in host-endian 64-bit chunks, -* so addressing units smaller than that needs a host-endian fixup. -*/ -#if HOST_BIG_ENDIAN -#define H1 (x) ((x) ^ 7) -#define H1_2 (x) ((x) ^ 6) -#define H1_4 (x) ((x) ^ 4) -#define H2 (x) ((x) ^ 3) -#define H4 (x) ((x) ^ 1) -#define H8 (x) ((x)) -#else -#define H1 (x) (x) -#define H1_2 (x) (x) -#define H1_4 (x) (x) -#define H2 (x) (x) -#define H4 (x) (x) -#define H8 (x) (x) -#endif - -static inline uint32_t vext_nf (uint32_t desc) -{ -return FIELD_EX32 (simd_data (desc),VDATA ,NF ); -} - -static inline uint32_t vext_vm (uint32_t desc) -{ -return FIELD_EX32 (simd_data (desc),VDATA ,VM ); -} - -/* -* Encode LMUL to lmul as following: -* LMUL vlmul lmul -* 1 000 0 -* 2 001 1 -* 4 010 2 -* 8 011 3 -* - 100 - -* 1/8 101 -3 -* 1/4 110 -2 -* 1/2 111 -1 -*/ -static inline int32_t vext_lmul (uint32_t desc) -{ -return sextract32 (FIELD_EX32 (simd_data (desc),VDATA ,LMUL ),0 ,3 ); -} - -static inline uint32_t vext_vta (uint32_t desc) -{ -return FIELD_EX32 (simd_data (desc),VDATA ,VTA ); -} - -static inline uint32_t vext_vma (uint32_t desc) -{ -return FIELD_EX32 (simd_data (desc),VDATA ,VMA ); -} - -static inline uint32_t vext_vta_all_1s (uint32_t desc) -{ -return FIELD_EX32 (simd_data (desc),VDATA ,VTA_ALL_1S ); -} -/* * Get the maximum number of elements can be operated. * @@ -155,21 +94,6 @@ static inline uint32_t vext_max_elems(uint32_t desc, uint32_t log2_esz)return scale <0 ? vlenb >> -scale : vlenb << scale; } -/* -* Get number of total elements, including prestart, body and tail elements. -* Note that when LMUL < 1, the tail includes the elements past VLMAX that -* are held in the same vector register. -*/ -static inline uint32_t vext_get_total_elems (CPURISCVState *env,uint32_t desc, -uint32_t esz) -{ -uint32_t vlenb =simd_maxsz (desc); -uint32_t sew =1 <<FIELD_EX64 (env->vtype,VTYPE ,VSEW ); -int8_t emul =ctzl (esz) -ctzl (sew) +vext_lmul (desc) <0 ?0 : -ctzl (esz) -ctzl (sew) +vext_lmul (desc); -return (vlenb << emul) / esz; -} -static inline target_ulong adjust_addr (CPURISCVState *env,target_ulong addr) {return (addr & env->cur_pmmask) | env->cur_pmbase;@@ -202,20 +126,6 @@ static void probe_pages(CPURISCVState *env, target_ulong addr, } } -/* set agnostic elements to 1s */ -static void vext_set_elems_1s (void *base,uint32_t is_agnostic,uint32_t cnt, -uint32_t tot) -{ -if (is_agnostic ==0 ) { -/* policy undisturbed */ -return ; - } -if (tot - cnt ==0 ) { -return ; - } -memset (base + cnt,-1 , tot - cnt); -} -static inline void vext_set_elem_mask (void *v0,int index,uint8_t value) {@@ -225,18 +135,6 @@ static inline void vext_set_elem_mask(void *v0, int index, ((uint64_t *)v0)[idx] =deposit64 (old, pos,1 , value); } -/* -* Earlier designs (pre-0.9) had a varying number of bits -* per mask value (MLEN). In the 0.9 design, MLEN=1. -* (Section 4.5) -*/ -static inline int vext_elem_mask (void *v0,int index) -{ -int idx = index /64 ; -int pos = index %64 ; -return (((uint64_t *)v0)[idx] >> pos) &1 ; -} -/* elements operations for load and store */ typedef void vext_ldst_elem_fn (CPURISCVState *env,target_ulong addr,uint32_t idx,void *vd,uintptr_t retaddr);@@ -739,18 +637,11 @@ GEN_VEXT_ST_WHOLE(vs8r_v, int8_t, ste_b)*** Vector Integer Arithmetic Instructions */ -/* expand macro args before macro */ -#define RVVCALL (macro, ...) macro(__VA_ARGS__) -/* (TD, T1, T2, TX1, TX2) */ #define OP_SSS_B int8_t, int8_t, int8_t, int8_t, int8_t#define OP_SSS_H int16_t, int16_t, int16_t, int16_t, int16_t#define OP_SSS_W int32_t, int32_t, int32_t, int32_t, int32_t#define OP_SSS_D int64_t, int64_t, int64_t, int64_t, int64_t -#define OP_UUU_B uint8_t, uint8_t, uint8_t, uint8_t, uint8_t -#define OP_UUU_H uint16_t, uint16_t, uint16_t, uint16_t, uint16_t -#define OP_UUU_W uint32_t, uint32_t, uint32_t, uint32_t, uint32_t -#define OP_UUU_D uint64_t, uint64_t, uint64_t, uint64_t, uint64_t#define OP_SUS_B int8_t, uint8_t, int8_t, uint8_t, int8_t#define OP_SUS_H int16_t, uint16_t, int16_t, uint16_t, int16_t#define OP_SUS_W int32_t, uint32_t, int32_t, uint32_t, int32_t@@ -774,16 +665,6 @@ GEN_VEXT_ST_WHOLE(vs8r_v, int8_t, ste_b)#define NOP_UUU_H uint16_t, uint16_t, uint32_t, uint16_t, uint32_t#define NOP_UUU_W uint32_t, uint32_t, uint64_t, uint32_t, uint64_t -/* operation of two vector elements */ -typedef void opivv2_fn (void *vd,void *vs1,void *vs2,int i); - -#define OPIVV2 (NAME ,TD ,T1 ,T2 ,TX1 ,TX2 ,HD ,HS1 ,HS2 ,OP ) \ -static void do_##NAME(void *vd, void *vs1, void *vs2, int i) \ -{ \ - TX1 s1 = *((T1 *)vs1 + HS1(i)); \ - TX2 s2 = *((T2 *)vs2 + HS2(i)); \ - *((TD *)vd + HD(i)) = OP(s2, s1); \ -}#define DO_SUB (N ,M ) (N - M)#define DO_RSUB (N ,M ) (M - N)@@ -796,40 +677,6 @@ RVVCALL(OPIVV2, vsub_vv_h, OP_SSS_H, H2, H2, H2, DO_SUB) RVVCALL(OPIVV2, vsub_vv_w, OP_SSS_W, H4, H4, H4, DO_SUB) RVVCALL(OPIVV2, vsub_vv_d, OP_SSS_D, H8, H8, H8, DO_SUB) -static void do_vext_vv (void *vd,void *v0,void *vs1,void *vs2, -CPURISCVState *env,uint32_t desc, -opivv2_fn *fn,uint32_t esz) -{ -uint32_t vm =vext_vm (desc); -uint32_t vl = env->vl; -uint32_t total_elems =vext_get_total_elems (env, desc, esz); -uint32_t vta =vext_vta (desc); -uint32_t vma =vext_vma (desc); -uint32_t i; - -for (i = env->vstart; i < vl; i++) { -if (!vm && !vext_elem_mask (v0, i)) { -/* set masked-off elements to 1s */ -vext_set_elems_1s (vd, vma, i * esz, (i +1 ) * esz); -continue ; - } -fn (vd, vs1, vs2, i); - } - env->vstart =0 ; -/* set tail elements to 1s */ -vext_set_elems_1s (vd, vta, vl * esz, total_elems * esz); -} - -/* generate the helpers for OPIVV */ -#define GEN_VEXT_VV (NAME ,ESZ ) \ -void HELPER(NAME)(void *vd, void *v0, void *vs1, \ - void *vs2, CPURISCVState *env, \ - uint32_t desc) \ -{ \ - do_vext_vv(vd, v0, vs1, vs2, env, desc, \ - do_##NAME, ESZ); \ -} - GEN_VEXT_VV(vadd_vv_b, 1) GEN_VEXT_VV(vadd_vv_h, 2)GEN_VEXT_VV (vadd_vv_w,4 )@@ -839,18 +686,6 @@ GEN_VEXT_VV(vsub_vv_h, 2) GEN_VEXT_VV(vsub_vv_w, 4) GEN_VEXT_VV(vsub_vv_d, 8) -typedef void opivx2_fn (void *vd,target_long s1,void *vs2,int i); - -/* -* (T1)s1 gives the real operator type. -* (TX1)(T1)s1 expands the operator type of widen or narrow operations. -*/ -#define OPIVX2 (NAME ,TD ,T1 ,T2 ,TX1 ,TX2 ,HD ,HS2 ,OP ) \ -static void do_##NAME(void *vd, target_long s1, void *vs2, int i) \ -{ \ - TX2 s2 = *((T2 *)vs2 + HS2(i)); \ - *((TD *)vd + HD(i)) = OP(s2, (TX1)(T1)s1); \ -} RVVCALL(OPIVX2, vadd_vx_b, OP_SSS_B, H1, H1, DO_ADD)RVVCALL (OPIVX2 , vadd_vx_h,OP_SSS_H ,H2 ,H2 ,DO_ADD )@@ -865,40 +700,6 @@ RVVCALL(OPIVX2, vrsub_vx_h, OP_SSS_H, H2, H2, DO_RSUB) RVVCALL(OPIVX2, vrsub_vx_w, OP_SSS_W, H4, H4, DO_RSUB) RVVCALL(OPIVX2, vrsub_vx_d, OP_SSS_D, H8, H8, DO_RSUB) -static void do_vext_vx (void *vd,void *v0,target_long s1,void *vs2, -CPURISCVState *env,uint32_t desc, -opivx2_fn fn,uint32_t esz) -{ -uint32_t vm =vext_vm (desc); -uint32_t vl = env->vl; -uint32_t total_elems =vext_get_total_elems (env, desc, esz); -uint32_t vta =vext_vta (desc); -uint32_t vma =vext_vma (desc); -uint32_t i; - -for (i = env->vstart; i < vl; i++) { -if (!vm && !vext_elem_mask (v0, i)) { -/* set masked-off elements to 1s */ -vext_set_elems_1s (vd, vma, i * esz, (i +1 ) * esz); -continue ; - } -fn (vd, s1, vs2, i); - } - env->vstart =0 ; -/* set tail elements to 1s */ -vext_set_elems_1s (vd, vta, vl * esz, total_elems * esz); -} - -/* generate the helpers for OPIVX */ -#define GEN_VEXT_VX (NAME ,ESZ ) \ -void HELPER(NAME)(void *vd, void *v0, target_ulong s1, \ - void *vs2, CPURISCVState *env, \ - uint32_t desc) \ -{ \ - do_vext_vx(vd, v0, s1, vs2, env, desc, \ - do_##NAME, ESZ); \ -} - GEN_VEXT_VX(vadd_vx_b, 1) GEN_VEXT_VX(vadd_vx_h, 2)GEN_VEXT_VX (vadd_vx_w,4 )target/riscv/vector_internals.c new file +81
-0@@ -0,0 +1,81 @@ +/* +* RISC-V Vector Extension Internals +* +* Copyright (c) 2020 T-Head Semiconductor Co., Ltd. All rights reserved. +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ + +#include "vector_internals.h" + +/* set agnostic elements to 1s */ +void vext_set_elems_1s (void *base,uint32_t is_agnostic,uint32_t cnt, +uint32_t tot) +{ +if (is_agnostic ==0 ) { +/* policy undisturbed */ +return ; + } +if (tot - cnt ==0 ) { +return ; + } +memset (base + cnt,-1 , tot - cnt); +} + +void do_vext_vv (void *vd,void *v0,void *vs1,void *vs2, +CPURISCVState *env,uint32_t desc, +opivv2_fn *fn,uint32_t esz) +{ +uint32_t vm =vext_vm (desc); +uint32_t vl = env->vl; +uint32_t total_elems =vext_get_total_elems (env, desc, esz); +uint32_t vta =vext_vta (desc); +uint32_t vma =vext_vma (desc); +uint32_t i; + +for (i = env->vstart; i < vl; i++) { +if (!vm && !vext_elem_mask (v0, i)) { +/* set masked-off elements to 1s */ +vext_set_elems_1s (vd, vma, i * esz, (i +1 ) * esz); +continue ; + } +fn (vd, vs1, vs2, i); + } + env->vstart =0 ; +/* set tail elements to 1s */ +vext_set_elems_1s (vd, vta, vl * esz, total_elems * esz); +} + +void do_vext_vx (void *vd,void *v0,target_long s1,void *vs2, +CPURISCVState *env,uint32_t desc, +opivx2_fn fn,uint32_t esz) +{ +uint32_t vm =vext_vm (desc); +uint32_t vl = env->vl; +uint32_t total_elems =vext_get_total_elems (env, desc, esz); +uint32_t vta =vext_vta (desc); +uint32_t vma =vext_vma (desc); +uint32_t i; + +for (i = env->vstart; i < vl; i++) { +if (!vm && !vext_elem_mask (v0, i)) { +/* set masked-off elements to 1s */ +vext_set_elems_1s (vd, vma, i * esz, (i +1 ) * esz); +continue ; + } +fn (vd, s1, vs2, i); + } + env->vstart =0 ; +/* set tail elements to 1s */ +vext_set_elems_1s (vd, vta, vl * esz, total_elems * esz); +}target/riscv/vector_internals.h new file +182
-0@@ -0,0 +1,182 @@ +/* +* RISC-V Vector Extension Internals +* +* Copyright (c) 2020 T-Head Semiconductor Co., Ltd. All rights reserved. +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ + +#ifndef TARGET_RISCV_VECTOR_INTERNALS_H +#define TARGET_RISCV_VECTOR_INTERNALS_H + +#include "qemu/osdep.h" +#include "qemu/bitops.h" +#include "cpu.h" +#include "tcg/tcg-gvec-desc.h" +#include "internals.h" + +static inline uint32_t vext_nf (uint32_t desc) +{ +return FIELD_EX32 (simd_data (desc),VDATA ,NF ); +} + +/* +* Note that vector data is stored in host-endian 64-bit chunks, +* so addressing units smaller than that needs a host-endian fixup. +*/ +#if HOST_BIG_ENDIAN +#define H1 (x) ((x) ^ 7) +#define H1_2 (x) ((x) ^ 6) +#define H1_4 (x) ((x) ^ 4) +#define H2 (x) ((x) ^ 3) +#define H4 (x) ((x) ^ 1) +#define H8 (x) ((x)) +#else +#define H1 (x) (x) +#define H1_2 (x) (x) +#define H1_4 (x) (x) +#define H2 (x) (x) +#define H4 (x) (x) +#define H8 (x) (x) +#endif + +/* +* Encode LMUL to lmul as following: +* LMUL vlmul lmul +* 1 000 0 +* 2 001 1 +* 4 010 2 +* 8 011 3 +* - 100 - +* 1/8 101 -3 +* 1/4 110 -2 +* 1/2 111 -1 +*/ +static inline int32_t vext_lmul (uint32_t desc) +{ +return sextract32 (FIELD_EX32 (simd_data (desc),VDATA ,LMUL ),0 ,3 ); +} + +static inline uint32_t vext_vm (uint32_t desc) +{ +return FIELD_EX32 (simd_data (desc),VDATA ,VM ); +} + +static inline uint32_t vext_vma (uint32_t desc) +{ +return FIELD_EX32 (simd_data (desc),VDATA ,VMA ); +} + +static inline uint32_t vext_vta (uint32_t desc) +{ +return FIELD_EX32 (simd_data (desc),VDATA ,VTA ); +} + +static inline uint32_t vext_vta_all_1s (uint32_t desc) +{ +return FIELD_EX32 (simd_data (desc),VDATA ,VTA_ALL_1S ); +} + +/* +* Earlier designs (pre-0.9) had a varying number of bits +* per mask value (MLEN). In the 0.9 design, MLEN=1. +* (Section 4.5) +*/ +static inline int vext_elem_mask (void *v0,int index) +{ +int idx = index /64 ; +int pos = index %64 ; +return (((uint64_t *)v0)[idx] >> pos) &1 ; +} + +/* +* Get number of total elements, including prestart, body and tail elements. +* Note that when LMUL < 1, the tail includes the elements past VLMAX that +* are held in the same vector register. +*/ +static inline uint32_t vext_get_total_elems (CPURISCVState *env,uint32_t desc, +uint32_t esz) +{ +uint32_t vlenb =simd_maxsz (desc); +uint32_t sew =1 <<FIELD_EX64 (env->vtype,VTYPE ,VSEW ); +int8_t emul =ctzl (esz) -ctzl (sew) +vext_lmul (desc) <0 ?0 : +ctzl (esz) -ctzl (sew) +vext_lmul (desc); +return (vlenb << emul) / esz; +} + +/* set agnostic elements to 1s */ +void vext_set_elems_1s (void *base,uint32_t is_agnostic,uint32_t cnt, +uint32_t tot); + +/* expand macro args before macro */ +#define RVVCALL (macro, ...) macro(__VA_ARGS__) + +/* (TD, T1, T2, TX1, TX2) */ +#define OP_UUU_B uint8_t, uint8_t, uint8_t, uint8_t, uint8_t +#define OP_UUU_H uint16_t, uint16_t, uint16_t, uint16_t, uint16_t +#define OP_UUU_W uint32_t, uint32_t, uint32_t, uint32_t, uint32_t +#define OP_UUU_D uint64_t, uint64_t, uint64_t, uint64_t, uint64_t + +/* operation of two vector elements */ +typedef void opivv2_fn (void *vd,void *vs1,void *vs2,int i); + +#define OPIVV2 (NAME ,TD ,T1 ,T2 ,TX1 ,TX2 ,HD ,HS1 ,HS2 ,OP ) \ +static void do_##NAME(void *vd, void *vs1, void *vs2, int i) \ +{ \ + TX1 s1 = *((T1 *)vs1 + HS1(i)); \ + TX2 s2 = *((T2 *)vs2 + HS2(i)); \ + *((TD *)vd + HD(i)) = OP(s2, s1); \ +} + +void do_vext_vv (void *vd,void *v0,void *vs1,void *vs2, +CPURISCVState *env,uint32_t desc, +opivv2_fn *fn,uint32_t esz); + +/* generate the helpers for OPIVV */ +#define GEN_VEXT_VV (NAME ,ESZ ) \ +void HELPER(NAME)(void *vd, void *v0, void *vs1, \ + void *vs2, CPURISCVState *env, \ + uint32_t desc) \ +{ \ + do_vext_vv(vd, v0, vs1, vs2, env, desc, \ + do_##NAME, ESZ); \ +} + +typedef void opivx2_fn (void *vd,target_long s1,void *vs2,int i); + +/* +* (T1)s1 gives the real operator type. +* (TX1)(T1)s1 expands the operator type of widen or narrow operations. +*/ +#define OPIVX2 (NAME ,TD ,T1 ,T2 ,TX1 ,TX2 ,HD ,HS2 ,OP ) \ +static void do_##NAME(void *vd, target_long s1, void *vs2, int i) \ +{ \ + TX2 s2 = *((T2 *)vs2 + HS2(i)); \ + *((TD *)vd + HD(i)) = OP(s2, (TX1)(T1)s1); \ +} + +void do_vext_vx (void *vd,void *v0,target_long s1,void *vs2, +CPURISCVState *env,uint32_t desc, +opivx2_fn fn,uint32_t esz); + +/* generate the helpers for OPIVX */ +#define GEN_VEXT_VX (NAME ,ESZ ) \ +void HELPER(NAME)(void *vd, void *v0, target_ulong s1, \ + void *vs2, CPURISCVState *env, \ + uint32_t desc) \ +{ \ + do_vext_vx(vd, v0, s1, vs2, env, desc, \ + do_##NAME, ESZ); \ +} + +#endif /* TARGET_RISCV_VECTOR_INTERNALS_H */
PATCH v3 2/19 target/riscv: Refactor vector-vector translation macro
In reply to Lawrence HunterFrom: Kiran Ostrolenk <[email protected]>
Refactor the non SEW-specific stuff out of GEN_OPIVV_TRANS into function opivv_trans (similar to opivi_trans). opivv_trans will be used in proceeding vector-crypto commits.
--- target/riscv/insn_trans/trans_rvv.c.inc | 62 +++++++++++++------------ 1 file changed, 32 insertions(+), 30 deletions(-)
target/riscv/insn_trans/trans_rvv.c.inc +32 -30
@@ -1643,38 +1643,40 @@ GEN_OPIWX_WIDEN_TRANS(vwadd_wx) GEN_OPIWX_WIDEN_TRANS(vwsubu_wx) GEN_OPIWX_WIDEN_TRANS(vwsub_wx) +static bool opivv_trans(uint32_t vd, uint32_t vs1, uint32_t vs2, uint32_t vm, + gen_helper_gvec_4_ptr *fn, DisasContext *s) +{ + uint32_t data = 0; + TCGLabel *over = gen_new_label(); + tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); + tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); + + data = FIELD_DP32(data, VDATA, VM, vm); + data = FIELD_DP32(data, VDATA, LMUL, s->lmul); + data = FIELD_DP32(data, VDATA, VTA, s->vta); + data = FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s); + data = FIELD_DP32(data, VDATA, VMA, s->vma); + tcg_gen_gvec_4_ptr(vreg_ofs(s, vd), vreg_ofs(s, 0), vreg_ofs(s, vs1), + vreg_ofs(s, vs2), cpu_env, s->cfg_ptr->vlen / 8, + s->cfg_ptr->vlen / 8, data, fn); + mark_vs_dirty(s); + gen_set_label(over); + return true; +} + /* Vector Integer Add-with-Carry / Subtract-with-Borrow Instructions */ /* OPIVV without GVEC IR */ -#define GEN_OPIVV_TRANS(NAME, CHECK) \ -static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ -{ \ - if (CHECK(s, a)) { \ - uint32_t data = 0; \ - static gen_helper_gvec_4_ptr * const fns[4] = { \ - gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ - gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ - }; \ - TCGLabel *over = gen_new_label(); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ - tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ - \ - data = FIELD_DP32(data, VDATA, VM, a->vm); \ - data = FIELD_DP32(data, VDATA, LMUL, s->lmul); \ - data = FIELD_DP32(data, VDATA, VTA, s->vta); \ - data = \ - FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s);\ - data = FIELD_DP32(data, VDATA, VMA, s->vma); \ - tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0), \ - vreg_ofs(s, a->rs1), \ - vreg_ofs(s, a->rs2), cpu_env, \ - s->cfg_ptr->vlen / 8, \ - s->cfg_ptr->vlen / 8, data, \ - fns[s->sew]); \ - mark_vs_dirty(s); \ - gen_set_label(over); \ - return true; \ - } \ - return false; \ +#define GEN_OPIVV_TRANS(NAME, CHECK) \ +static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ +{ \ + if (CHECK(s, a)) { \ + static gen_helper_gvec_4_ptr * const fns[4] = { \ + gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ + gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ + }; \ + return opivv_trans(a->rd, a->rs1, a->rs2, a->vm, fns[s->sew], s);\ + } \ + return false; \ } /*
Signature · 1 line
2.40.1
Re: [PATCH v3 02/19] target/riscv: Refactor vector-vector translation macro
Quoted 10 lines · Lawrence Hunter
From: Kiran Ostrolenk <[email protected]>
Refactor the non SEW-specific stuff out of
GEN_OPIVV_TRANSinto functionopivv_trans(similar toopivi_trans).opivv_transwill be used in proceeding vector-crypto commits.Signed-off-by Kiran Ostrolenk <[email protected]>Reviewed-by Richard Henderson <[email protected]>Reviewed-by Alistair Francis <[email protected]>---
Weiwei Li
Quoted 78 lines
target/riscv/insn_trans/trans_rvv.c.inc | 62 +++++++++++++------------ 1 file changed, 32 insertions(+), 30 deletions(-)target/riscv/insn_trans/trans_rvv.c.inc +32
-30@@ -1643,38 +1643,40 @@ GEN_OPIWX_WIDEN_TRANS(vwadd_wx) GEN_OPIWX_WIDEN_TRANS(vwsubu_wx) GEN_OPIWX_WIDEN_TRANS(vwsub_wx) +static bool opivv_trans(uint32_t vd, uint32_t vs1, uint32_t vs2, uint32_t vm, + gen_helper_gvec_4_ptr *fn, DisasContext *s) +{ + uint32_t data = 0; + TCGLabel *over = gen_new_label(); + tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); + tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); + + data = FIELD_DP32(data, VDATA, VM, vm); + data = FIELD_DP32(data, VDATA, LMUL, s->lmul); + data = FIELD_DP32(data, VDATA, VTA, s->vta); + data = FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s); + data = FIELD_DP32(data, VDATA, VMA, s->vma); + tcg_gen_gvec_4_ptr(vreg_ofs(s, vd), vreg_ofs(s, 0), vreg_ofs(s, vs1), + vreg_ofs(s, vs2), cpu_env, s->cfg_ptr->vlen / 8, + s->cfg_ptr->vlen / 8, data, fn); + mark_vs_dirty(s); + gen_set_label(over); + return true; +} + /* Vector Integer Add-with-Carry / Subtract-with-Borrow Instructions */ /* OPIVV without GVEC IR */ -#define GEN_OPIVV_TRANS(NAME, CHECK) \ -static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ -{ \ - if (CHECK(s, a)) { \ - uint32_t data = 0; \ - static gen_helper_gvec_4_ptr * const fns[4] = { \ - gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ - gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ - }; \ - TCGLabel *over = gen_new_label(); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ - tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ - \ - data = FIELD_DP32(data, VDATA, VM, a->vm); \ - data = FIELD_DP32(data, VDATA, LMUL, s->lmul); \ - data = FIELD_DP32(data, VDATA, VTA, s->vta); \ - data = \ - FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s);\ - data = FIELD_DP32(data, VDATA, VMA, s->vma); \ - tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0), \ - vreg_ofs(s, a->rs1), \ - vreg_ofs(s, a->rs2), cpu_env, \ - s->cfg_ptr->vlen / 8, \ - s->cfg_ptr->vlen / 8, data, \ - fns[s->sew]); \ - mark_vs_dirty(s); \ - gen_set_label(over); \ - return true; \ - } \ - return false; \ +#define GEN_OPIVV_TRANS(NAME, CHECK) \ +static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ +{ \ + if (CHECK(s, a)) { \ + static gen_helper_gvec_4_ptr * const fns[4] = { \ + gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ + gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ + }; \ + return opivv_trans(a->rd, a->rs1, a->rs2, a->vm, fns[s->sew], s);\ + } \ + return false; \ } /*
PATCH v3 3/19 target/riscv: Remove redundant "cpu_vl == 0" checks
In reply to Lawrence HunterFrom: Nazar Kazakov <[email protected]>
Remove the redundant "vl == 0" check which is already included within the vstart >= vl check, when vl == 0.
--- target/riscv/insn_trans/trans_rvv.c.inc | 31 +------------------------ 1 file changed, 1 insertion(+), 30 deletions(-)
target/riscv/insn_trans/trans_rvv.c.inc +1 -30
@@ -617,7 +617,6 @@ static bool ldst_us_trans(uint32_t vd, uint32_t rs1, uint32_t data, TCGv_i32 desc; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();
@@ -786,7 +785,6 @@ static bool ldst_stride_trans(uint32_t vd, uint32_t rs1, uint32_t rs2, TCGv_i32 desc; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();
@@ -893,7 +891,6 @@ static bool ldst_index_trans(uint32_t vd, uint32_t rs1, uint32_t vs2, TCGv_i32 desc; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();
@@ -1034,7 +1031,6 @@ static bool ldff_trans(uint32_t vd, uint32_t rs1, uint32_t data, TCGv_i32 desc; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();
@@ -1191,7 +1187,6 @@ do_opivv_gvec(DisasContext *s, arg_rmrr *a, GVecGen3Fn *gvec_fn, return false; } - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) {
@@ -1241,7 +1236,6 @@ static bool opivx_trans(uint32_t vd, uint32_t rs1, uint32_t vs2, uint32_t vm, uint32_t data = 0; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();
@@ -1405,7 +1399,6 @@ static bool opivi_trans(uint32_t vd, uint32_t imm, uint32_t vs2, uint32_t vm, uint32_t data = 0; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();
@@ -1492,7 +1485,6 @@ static bool do_opivv_widen(DisasContext *s, arg_rmrr *a, if (checkfn(s, a)) { uint32_t data = 0; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); data = FIELD_DP32(data, VDATA, VM, a->vm);
@@ -1575,7 +1567,6 @@ static bool do_opiwv_widen(DisasContext *s, arg_rmrr *a, if (opiwv_widen_check(s, a)) { uint32_t data = 0; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); data = FIELD_DP32(data, VDATA, VM, a->vm);
@@ -1648,7 +1639,6 @@ static bool opivv_trans(uint32_t vd, uint32_t vs1, uint32_t vs2, uint32_t vm, { uint32_t data = 0; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); data = FIELD_DP32(data, VDATA, VM, vm);
@@ -1842,7 +1832,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##NAME##_w, \ }; \ TCGLabel *over = gen_new_label(); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \
@@ -2054,7 +2043,6 @@ static bool trans_vmv_v_v(DisasContext *s, arg_vmv_v_v *a) gen_helper_vmv_v_v_w, gen_helper_vmv_v_v_d, }; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); tcg_gen_gvec_2_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, a->rs1),
@@ -2078,7 +2066,6 @@ static bool trans_vmv_v_x(DisasContext *s, arg_vmv_v_x *a) vext_check_ss(s, a->rd, 0, 1)) { TCGv s1; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); s1 = get_gpr(s, a->rs1, EXT_SIGN);
@@ -2140,7 +2127,6 @@ static bool trans_vmv_v_i(DisasContext *s, arg_vmv_v_i *a) gen_helper_vmv_v_x_w, gen_helper_vmv_v_x_d, }; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); s1 = tcg_constant_i64(simm);
@@ -2288,7 +2274,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm(s, RISCV_FRM_DYN); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \
@@ -2323,7 +2308,6 @@ static bool opfvf_trans(uint32_t vd, uint32_t rs1, uint32_t vs2, TCGv_i64 t1; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();
@@ -2408,7 +2392,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm(s, RISCV_FRM_DYN); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);\ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \
@@ -2483,7 +2466,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm(s, RISCV_FRM_DYN); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \
@@ -2601,7 +2583,6 @@ static bool do_opfv(DisasContext *s, arg_rmr *a, uint32_t data = 0; TCGLabel *over = gen_new_label(); gen_set_rm_chkfrm(s, rm); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); data = FIELD_DP32(data, VDATA, VM, a->vm);
@@ -2713,7 +2694,6 @@ static bool trans_vfmv_v_f(DisasContext *s, arg_vfmv_v_f *a) gen_helper_vmv_v_x_d, }; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); t1 = tcg_temp_new_i64();
@@ -2792,7 +2772,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm_chkfrm(s, FRM); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \
@@ -2844,7 +2823,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm(s, RISCV_FRM_DYN); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \
@@ -2912,7 +2890,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm_chkfrm(s, FRM); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \
@@ -2962,7 +2939,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm_chkfrm(s, FRM); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \
@@ -3053,7 +3029,6 @@ static bool trans_##NAME(DisasContext *s, arg_r *a) \ uint32_t data = 0; \ gen_helper_gvec_4_ptr *fn = gen_helper_##NAME; \ TCGLabel *over = gen_new_label(); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, LMUL, s->lmul); \
@@ -3222,7 +3197,6 @@ static bool trans_vid_v(DisasContext *s, arg_vid_v *a) require_vm(a->vm, a->rd)) { uint32_t data = 0; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); data = FIELD_DP32(data, VDATA, VM, a->vm);
@@ -3409,7 +3383,6 @@ static bool trans_vmv_s_x(DisasContext *s, arg_vmv_s_x *a) TCGv s1; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); t1 = tcg_temp_new_i64();
@@ -3466,8 +3439,7 @@ static bool trans_vfmv_s_f(DisasContext *s, arg_vfmv_s_f *a) TCGv_i64 t1; TCGLabel *over = gen_new_label(); - /* if vl == 0 or vstart >= vl, skip vector register write back */ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); + /* if vstart >= vl, skip vector register write back */ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); /* NaN-box f[rs1] */
@@ -3718,7 +3690,6 @@ static bool int_ext_op(DisasContext *s, arg_rmr *a, uint8_t seq) uint32_t data = 0; gen_helper_gvec_3_ptr *fn; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); static gen_helper_gvec_3_ptr * const fns[6][4] = {
Signature · 1 line
2.40.1
Re: [PATCH v3 03/19] target/riscv: Remove redundant "cpu_vl == 0" checks
Quoted 6 lines · Lawrence Hunter
From: Nazar Kazakov <[email protected]>
Remove the redundant "vl == 0" check which is already included within the vstart >= vl check, when vl == 0.
Signed-off-by Nazar Kazakov <[email protected]>---
Weiwei Li
Quoted 241 lines
target/riscv/insn_trans/trans_rvv.c.inc | 31 +------------------------ 1 file changed, 1 insertion(+), 30 deletions(-)target/riscv/insn_trans/trans_rvv.c.inc +1
-30@@ -617,7 +617,6 @@ static bool ldst_us_trans(uint32_t vd, uint32_t rs1, uint32_t data, TCGv_i32 desc; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();@@ -786,7 +785,6 @@ static bool ldst_stride_trans(uint32_t vd, uint32_t rs1, uint32_t rs2, TCGv_i32 desc; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();@@ -893,7 +891,6 @@ static bool ldst_index_trans(uint32_t vd, uint32_t rs1, uint32_t vs2, TCGv_i32 desc; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();@@ -1034,7 +1031,6 @@ static bool ldff_trans(uint32_t vd, uint32_t rs1, uint32_t data, TCGv_i32 desc; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();@@ -1191,7 +1187,6 @@ do_opivv_gvec(DisasContext *s, arg_rmrr *a, GVecGen3Fn *gvec_fn, return false; } - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) {@@ -1241,7 +1236,6 @@ static bool opivx_trans(uint32_t vd, uint32_t rs1, uint32_t vs2, uint32_t vm, uint32_t data = 0; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();@@ -1405,7 +1399,6 @@ static bool opivi_trans(uint32_t vd, uint32_t imm, uint32_t vs2, uint32_t vm, uint32_t data = 0; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();@@ -1492,7 +1485,6 @@ static bool do_opivv_widen(DisasContext *s, arg_rmrr *a, if (checkfn(s, a)) { uint32_t data = 0; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); data = FIELD_DP32(data, VDATA, VM, a->vm);@@ -1575,7 +1567,6 @@ static bool do_opiwv_widen(DisasContext *s, arg_rmrr *a, if (opiwv_widen_check(s, a)) { uint32_t data = 0; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); data = FIELD_DP32(data, VDATA, VM, a->vm);@@ -1648,7 +1639,6 @@ static bool opivv_trans(uint32_t vd, uint32_t vs1, uint32_t vs2, uint32_t vm, { uint32_t data = 0; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); data = FIELD_DP32(data, VDATA, VM, vm);@@ -1842,7 +1832,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##NAME##_w, \ }; \ TCGLabel *over = gen_new_label(); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \@@ -2054,7 +2043,6 @@ static bool trans_vmv_v_v(DisasContext *s, arg_vmv_v_v *a) gen_helper_vmv_v_v_w, gen_helper_vmv_v_v_d, }; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); tcg_gen_gvec_2_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, a->rs1),@@ -2078,7 +2066,6 @@ static bool trans_vmv_v_x(DisasContext *s, arg_vmv_v_x *a) vext_check_ss(s, a->rd, 0, 1)) { TCGv s1; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); s1 = get_gpr(s, a->rs1, EXT_SIGN);@@ -2140,7 +2127,6 @@ static bool trans_vmv_v_i(DisasContext *s, arg_vmv_v_i *a) gen_helper_vmv_v_x_w, gen_helper_vmv_v_x_d, }; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); s1 = tcg_constant_i64(simm);@@ -2288,7 +2274,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm(s, RISCV_FRM_DYN); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \@@ -2323,7 +2308,6 @@ static bool opfvf_trans(uint32_t vd, uint32_t rs1, uint32_t vs2, TCGv_i64 t1; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); dest = tcg_temp_new_ptr();@@ -2408,7 +2392,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm(s, RISCV_FRM_DYN); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);\ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \@@ -2483,7 +2466,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm(s, RISCV_FRM_DYN); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \@@ -2601,7 +2583,6 @@ static bool do_opfv(DisasContext *s, arg_rmr *a, uint32_t data = 0; TCGLabel *over = gen_new_label(); gen_set_rm_chkfrm(s, rm); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); data = FIELD_DP32(data, VDATA, VM, a->vm);@@ -2713,7 +2694,6 @@ static bool trans_vfmv_v_f(DisasContext *s, arg_vfmv_v_f *a) gen_helper_vmv_v_x_d, }; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); t1 = tcg_temp_new_i64();@@ -2792,7 +2772,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm_chkfrm(s, FRM); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \@@ -2844,7 +2823,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm(s, RISCV_FRM_DYN); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \@@ -2912,7 +2890,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm_chkfrm(s, FRM); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \@@ -2962,7 +2939,6 @@ static bool trans_##NAME(DisasContext *s, arg_rmr *a) \ }; \ TCGLabel *over = gen_new_label(); \ gen_set_rm_chkfrm(s, FRM); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, VM, a->vm); \@@ -3053,7 +3029,6 @@ static bool trans_##NAME(DisasContext *s, arg_r *a) \ uint32_t data = 0; \ gen_helper_gvec_4_ptr *fn = gen_helper_##NAME; \ TCGLabel *over = gen_new_label(); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ \ data = FIELD_DP32(data, VDATA, LMUL, s->lmul); \@@ -3222,7 +3197,6 @@ static bool trans_vid_v(DisasContext *s, arg_vid_v *a) require_vm(a->vm, a->rd)) { uint32_t data = 0; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); data = FIELD_DP32(data, VDATA, VM, a->vm);@@ -3409,7 +3383,6 @@ static bool trans_vmv_s_x(DisasContext *s, arg_vmv_s_x *a) TCGv s1; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); t1 = tcg_temp_new_i64();@@ -3466,8 +3439,7 @@ static bool trans_vfmv_s_f(DisasContext *s, arg_vfmv_s_f *a) TCGv_i64 t1; TCGLabel *over = gen_new_label(); - /* if vl == 0 or vstart >= vl, skip vector register write back */ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); + /* if vstart >= vl, skip vector register write back */ tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); /* NaN-box f[rs1] */@@ -3718,7 +3690,6 @@ static bool int_ext_op(DisasContext *s, arg_rmr *a, uint8_t seq) uint32_t data = 0; gen_helper_gvec_3_ptr *fn; TCGLabel *over = gen_new_label(); - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); static gen_helper_gvec_3_ptr * const fns[6][4] = {
PATCH v3 4/19 target/riscv: Add Zvbc ISA extension support
In reply to Lawrence HunterThis commit adds support for the Zvbc vector-crypto extension, which consists of the following instructions:
* vclmulh.[vx,vv]
* vclmul.[vx,vv]
Translation functions are defined in target/riscv/insn_trans/trans_rvvk.c.inc and helpers are defined in target/riscv/vcrypto_helper.c.
--- target/riscv/cpu.c | 7 ++ target/riscv/cpu.h | 1 + target/riscv/helper.h | 6 ++ target/riscv/insn32.decode | 6 ++ target/riscv/insn_trans/trans_rvvk.c.inc | 88 ++++++++++++++++++++++++ target/riscv/meson.build | 3 +- target/riscv/translate.c | 1 + target/riscv/vcrypto_helper.c | 59 ++++++++++++++++ 8 files changed, 170 insertions(+), 1 deletion(-) create mode 100644 target/riscv/insn_trans/trans_rvvk.c.inc create mode 100644 target/riscv/vcrypto_helper.c
target/riscv/cpu.c +7 -0
@@ -109,6 +109,7 @@ static const struct isa_ext_data isa_edata_arr[] = {ISA_EXT_DATA_ENTRY (zve64d, true,PRIV_VERSION_1_12_0 , ext_zve64d),ISA_EXT_DATA_ENTRY (zvfh, true,PRIV_VERSION_1_12_0 , ext_zvfh),ISA_EXT_DATA_ENTRY (zvfhmin, true,PRIV_VERSION_1_12_0 , ext_zvfhmin), +ISA_EXT_DATA_ENTRY (zvbc, true,PRIV_VERSION_1_12_0 , ext_zvbc),ISA_EXT_DATA_ENTRY (zhinx, true,PRIV_VERSION_1_12_0 , ext_zhinx),ISA_EXT_DATA_ENTRY (zhinxmin, true,PRIV_VERSION_1_12_0 , ext_zhinxmin), ISA_EXT_DATA_ENTRY(smaia, true, PRIV_VERSION_1_12_0, ext_smaia),
@@ -1211,6 +1212,12 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)return ; } +if (cpu->cfg.ext_zvbc && + !(cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { +error_setg (errp,"Zvbc extension requires V or Zve64{f,d} extensions" ); +return ; + } +#ifndef CONFIG_USER_ONLY if (cpu->cfg.pmu_num) {if (!riscv_pmu_init (cpu, cpu->cfg.pmu_num) && cpu->cfg.ext_sscofpmf) {
target/riscv/cpu.h +1 -0
@@ -470,6 +470,7 @@ struct RISCVCPUConfig {bool ext_zve32f;bool ext_zve64f;bool ext_zve64d; +bool ext_zvbc;bool ext_zmmul;bool ext_zvfh;bool ext_zvfhmin;
target/riscv/helper.h +6 -0
@@ -1142,3 +1142,9 @@ DEF_HELPER_FLAGS_1(aes64im, TCG_CALL_NO_RWG_SE, tl, tl) DEF_HELPER_FLAGS_3(sm4ed, TCG_CALL_NO_RWG_SE, tl, tl, tl, tl)DEF_HELPER_FLAGS_3 (sm4ks,TCG_CALL_NO_RWG_SE , tl, tl, tl, tl) + +/* Vector crypto functions */ +DEF_HELPER_6(vclmul_vv, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vclmul_vx, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vclmulh_vv, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6 (vclmulh_vx, void, ptr, ptr, tl, ptr, env, i32)
target/riscv/insn32.decode +6 -0
@@ -908,3 +908,9 @@ sm4ks .. 11010 ..... ..... 000 ..... 0110011 @k_aes # *** RV32 Zicond Standard Extension *** czero_eqz 0000111 ..... ..... 101 ..... 0110011 @r czero_nez 0000111 ..... ..... 111 ..... 0110011 @r + +# *** Zvbc vector crypto extension *** +vclmul_vv 001100 . ..... ..... 010 ..... 1010111 @r_vm +vclmul_vx 001100 . ..... ..... 110 ..... 1010111 @r_vm +vclmulh_vv 001101 . ..... ..... 010 ..... 1010111 @r_vm +vclmulh_vx 001101 . ..... ..... 110 ..... 1010111 @r_vm
target/riscv/insn_trans/trans_rvvk.c.inc new file +88 -0
88 lines not on this page · raw message
target/riscv/meson.build +2 -1
@@ -20,7 +20,8 @@ riscv_ss.add(files( 'bitmanip_helper.c', 'translate.c', 'm128_helper.c', - 'crypto_helper.c' + 'crypto_helper.c', + 'vcrypto_helper.c' )) riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'), if_false: files('kvm-stub.c'))
target/riscv/translate.c +1 -0
@@ -1083,6 +1083,7 @@ static uint32_t opcode_at(DisasContextBase *dcbase, target_ulong pc)#include "insn_trans/trans_rvzicbo.c.inc" #include "insn_trans/trans_rvzfh.c.inc" #include "insn_trans/trans_rvk.c.inc" +#include "insn_trans/trans_rvvk.c.inc" #include "insn_trans/trans_privileged.c.inc" #include "insn_trans/trans_svinval.c.inc" #include "decode-xthead.c.inc"
target/riscv/vcrypto_helper.c new file +59 -0
59 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH v3 04/19] target/riscv: Add Zvbc ISA extension support
Quoted 41 lines · Lawrence Hunter
This commit adds support for the Zvbc vector-crypto extension, which consists of the following instructions:
* vclmulh.[vx,vv]
* vclmul.[vx,vv]Translation functions are defined in
target/riscv/insn_trans/trans_rvvk.c.incand helpers are defined intarget/riscv/vcrypto_helper.c.Co-authored-by Nazar Kazakov <[email protected]>Co-authored-by Max Chou <[email protected]>Signed-off-by Nazar Kazakov <[email protected]>Signed-off-by Max Chou <[email protected]>Signed-off-by Lawrence Hunter <[email protected]>--- target/riscv/cpu.c | 7 ++ target/riscv/cpu.h | 1 + target/riscv/helper.h | 6 ++ target/riscv/insn32.decode | 6 ++ target/riscv/insn_trans/trans_rvvk.c.inc | 88 ++++++++++++++++++++++++ target/riscv/meson.build | 3 +- target/riscv/translate.c | 1 + target/riscv/vcrypto_helper.c | 59 ++++++++++++++++ 8 files changed, 170 insertions(+), 1 deletion(-)create mode 100644 target/riscv/insn_trans/trans_rvvk.c.inc create mode 100644
target/riscv/vcrypto_helper.ctarget/riscv/cpu.c +1
-0@@ -109,6 +109,7 @@ static const struct isa_ext_data isa_edata_arr[] = {ISA_EXT_DATA_ENTRY (zve64d, true,PRIV_VERSION_1_12_0 , ext_zve64d),ISA_EXT_DATA_ENTRY (zvfh, true,PRIV_VERSION_1_12_0 , ext_zvfh),ISA_EXT_DATA_ENTRY (zvfhmin, true,PRIV_VERSION_1_12_0 , ext_zvfhmin), +ISA_EXT_DATA_ENTRY (zvbc, true,PRIV_VERSION_1_12_0 , ext_zvbc),ISA_EXT_DATA_ENTRY (zhinx, true,PRIV_VERSION_1_12_0 , ext_zhinx),ISA_EXT_DATA_ENTRY (zhinxmin, true,PRIV_VERSION_1_12_0 , ext_zhinxmin), ISA_EXT_DATA_ENTRY(smaia, true, PRIV_VERSION_1_12_0, ext_smaia),
target/riscv/cpu.c static void riscv_cpu_realize(DeviceState *dev, Error **errp) · line 1220
Quoted 9 lines
target/riscv/cpu.c +6
-0@@ -1211,6 +1212,12 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)return ; } +if (cpu->cfg.ext_zvbc && + !(cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { +error_setg (errp,"Zvbc extension requires V or Zve64{f,d} extensions" ); +return ; + } +
You can only check ext_zve64f here, since V depends on Zve64d and Zve64d depends on Zve64f.
Quoted 84 lines
target/riscv/cpu.c +0
-0#ifndef CONFIG_USER_ONLY if (cpu->cfg.pmu_num) {if (!riscv_pmu_init (cpu, cpu->cfg.pmu_num) && cpu->cfg.ext_sscofpmf) {target/riscv/cpu.h +1
-0@@ -470,6 +470,7 @@ struct RISCVCPUConfig {bool ext_zve32f;bool ext_zve64f;bool ext_zve64d; +bool ext_zvbc;bool ext_zmmul;bool ext_zvfh;bool ext_zvfhmin;target/riscv/helper.h +6
-0@@ -1142,3 +1142,9 @@ DEF_HELPER_FLAGS_1(aes64im, TCG_CALL_NO_RWG_SE, tl, tl) DEF_HELPER_FLAGS_3(sm4ed, TCG_CALL_NO_RWG_SE, tl, tl, tl, tl)DEF_HELPER_FLAGS_3 (sm4ks,TCG_CALL_NO_RWG_SE , tl, tl, tl, tl) + +/* Vector crypto functions */ +DEF_HELPER_6(vclmul_vv, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vclmul_vx, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vclmulh_vv, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6 (vclmulh_vx, void, ptr, ptr, tl, ptr, env, i32)target/riscv/insn32.decode +6
-0@@ -908,3 +908,9 @@ sm4ks .. 11010 ..... ..... 000 ..... 0110011 @k_aes # *** RV32 Zicond Standard Extension *** czero_eqz 0000111 ..... ..... 101 ..... 0110011 @r czero_nez 0000111 ..... ..... 111 ..... 0110011 @r + +# *** Zvbc vector crypto extension *** +vclmul_vv 001100 . ..... ..... 010 ..... 1010111 @r_vm +vclmul_vx 001100 . ..... ..... 110 ..... 1010111 @r_vm +vclmulh_vv 001101 . ..... ..... 010 ..... 1010111 @r_vm +vclmulh_vx 001101 . ..... ..... 110 ..... 1010111 @r_vmtarget/riscv/insn_trans/trans_rvvk.c.inc new file +35
-0@@ -0,0 +1,88 @@ +/* + * RISC-V translation routines for the vector crypto extension. + * + * Copyright (C) 2023 SiFive, Inc. + * Written by Codethink Ltd and SiFive. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see <http://www.gnu.org/licenses/>. + */ + +/* + * Zvbc + */ + +#define GEN_VV_MASKED_TRANS(NAME, CHECK) \ + static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ + { \ + if (CHECK(s, a)) { \ + return opivv_trans(a->rd, a->rs1, a->rs2, a->vm, \ + gen_helper_##NAME, s); \ + } \ + return false; \ + } + +static bool vclmul_vv_check(DisasContext *s, arg_rmrr *a) +{
target/riscv/insn_trans/trans_rvvk.c.inc line 45
Quoted 10 lines
target/riscv/insn_trans/trans_rvvk.c.inc new file +10
-0+ return opivv_check(s, a) && + s->cfg_ptr->ext_zvbc == true && + s->sew == MO_64; +} + +GEN_VV_MASKED_TRANS(vclmul_vv, vclmul_vv_check) +GEN_VV_MASKED_TRANS(vclmulh_vv, vclmul_vv_check) + +#define GEN_VX_MASKED_TRANS(NAME, CHECK) \ + static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \
Most of code of this function is similar to opivx_trans. Maybe we can reuse it here.
Regards,
Weiwei Li
Quoted 134 lines
target/riscv/insn_trans/trans_rvvk.c.inc new file +43
-0+ { \ + if (CHECK(s, a)) { \ + TCGv_ptr rd_v, v0_v, rs2_v; \ + TCGv rs1; \ + TCGv_i32 desc; \ + uint32_t data = 0; \ + \ + TCGLabel *over = gen_new_label(); \ + tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ + \ + data = FIELD_DP32(data, VDATA, VM, a->vm); \ + data = FIELD_DP32(data, VDATA, LMUL, s->lmul); \ + data = FIELD_DP32(data, VDATA, VTA, s->vta); \ + data = FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s); \ + data = FIELD_DP32(data, VDATA, VMA, s->vma); \ + \ + rd_v = tcg_temp_new_ptr(); \ + v0_v = tcg_temp_new_ptr(); \ + rs1 = get_gpr(s, a->rs1, EXT_ZERO); \ + rs2_v = tcg_temp_new_ptr(); \ + desc = tcg_constant_i32( \ + simd_desc(s->cfg_ptr->vlen / 8, s->cfg_ptr->vlen / 8, data)); \ + tcg_gen_addi_ptr(rd_v, cpu_env, vreg_ofs(s, a->rd)); \ + tcg_gen_addi_ptr(v0_v, cpu_env, vreg_ofs(s, 0)); \ + tcg_gen_addi_ptr(rs2_v, cpu_env, vreg_ofs(s, a->rs2)); \ + gen_helper_##NAME(rd_v, v0_v, rs1, rs2_v, cpu_env, desc); \ + \ + mark_vs_dirty(s); \ + gen_set_label(over); \ + return true; \ + } \ + return false; \ + } + +static bool vclmul_vx_check(DisasContext *s, arg_rmrr *a) +{ + return opivx_check(s, a) && + s->cfg_ptr->ext_zvbc == true && + s->sew == MO_64; +} + +GEN_VX_MASKED_TRANS(vclmul_vx, vclmul_vx_check) +GEN_VX_MASKED_TRANS(vclmulh_vx, vclmul_vx_check)target/riscv/meson.build +2
-1@@ -20,7 +20,8 @@ riscv_ss.add(files( 'bitmanip_helper.c', 'translate.c', 'm128_helper.c', - 'crypto_helper.c' + 'crypto_helper.c', + 'vcrypto_helper.c' )) riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'), if_false: files('kvm-stub.c'))target/riscv/translate.c +1
-0@@ -1083,6 +1083,7 @@ static uint32_t opcode_at(DisasContextBase *dcbase, target_ulong pc)#include "insn_trans/trans_rvzicbo.c.inc" #include "insn_trans/trans_rvzfh.c.inc" #include "insn_trans/trans_rvk.c.inc" +#include "insn_trans/trans_rvvk.c.inc" #include "insn_trans/trans_privileged.c.inc" #include "insn_trans/trans_svinval.c.inc" #include "decode-xthead.c.inc" target/riscv/vcrypto_helper.c new file +59
-0@@ -0,0 +1,59 @@ +/* +* RISC-V Vector Crypto Extension Helpers for QEMU. +* +* Copyright (C) 2023 SiFive, Inc. +* Written by Codethink Ltd and SiFive. +* +* This program is free software; you can redistribute it and/or modify it +* under the terms and conditions of the GNU General Public License, +* version 2 or later, as published by the Free Software Foundation. +* +* This program is distributed in the hope it will be useful, but WITHOUT +* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +* more details. +* +* You should have received a copy of the GNU General Public License along with +* this program. If not, see <http://www.gnu.org/licenses/>. +*/ + +#include "qemu/osdep.h" +#include "qemu/host-utils.h" +#include "qemu/bitops.h" +#include "cpu.h" +#include "exec/memop.h" +#include "exec/exec-all.h" +#include "exec/helper-proto.h" +#include "internals.h" +#include "vector_internals.h" + +static uint64_t clmul64 (uint64_t y,uint64_t x) +{ +uint64_t result =0 ; +for (int j =63 ; j >=0 ; j--) { +if ((y >> j) &1 ) { + result ^= (x << j); + } + } +return result; +} + +static uint64_t clmulh64 (uint64_t y,uint64_t x) +{ +uint64_t result =0 ; +for (int j =63 ; j >=1 ; j--) { +if ((y >> j) &1 ) { + result ^= (x >> (64 - j)); + } + } +return result; +} + +RVVCALL(OPIVV2, vclmul_vv, OP_UUU_D, H8, H8, H8, clmul64) +GEN_VEXT_VV(vclmul_vv, 8) +RVVCALL(OPIVX2, vclmul_vx, OP_UUU_D, H8, H8, clmul64) +GEN_VEXT_VX(vclmul_vx, 8) +RVVCALL(OPIVV2, vclmulh_vv, OP_UUU_D, H8, H8, H8, clmulh64) +GEN_VEXT_VV(vclmulh_vv, 8) +RVVCALL(OPIVX2, vclmulh_vx, OP_UUU_D, H8, H8, clmulh64) +GEN_VEXT_VX (vclmulh_vx,8 )
PATCH v3 5/19 target/riscv: Move vector translation checks
In reply to Lawrence HunterFrom: Nazar Kazakov <[email protected]>
Move the checks out of do_opiv{v,x,i}_gvec{,_shift} functions and into the corresponding macros. This enables the functions to be reused in proceeding commits without check duplication.
--- target/riscv/insn_trans/trans_rvv.c.inc | 28 +++++++++++-------------- 1 file changed, 12 insertions(+), 16 deletions(-)
target/riscv/insn_trans/trans_rvv.c.inc +12 -16
@@ -1183,9 +1183,6 @@ do_opivv_gvec(DisasContext *s, arg_rmrr *a, GVecGen3Fn *gvec_fn, gen_helper_gvec_4_ptr *fn) { TCGLabel *over = gen_new_label(); - if (!opivv_check(s, a)) { - return false; - } tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);
@@ -1218,6 +1215,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ }; \ + if (!opivv_check(s, a)) { \ + return false; \ + } \ return do_opivv_gvec(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ }
@@ -1276,10 +1276,6 @@ static inline bool do_opivx_gvec(DisasContext *s, arg_rmrr *a, GVecGen2sFn *gvec_fn, gen_helper_opivx *fn) { - if (!opivx_check(s, a)) { - return false; - } - if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) { TCGv_i64 src1 = tcg_temp_new_i64();
@@ -1301,6 +1297,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ }; \ + if (!opivx_check(s, a)) { \ + return false; \ + } \ return do_opivx_gvec(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ }
@@ -1432,10 +1431,6 @@ static inline bool do_opivi_gvec(DisasContext *s, arg_rmrr *a, GVecGen2iFn *gvec_fn, gen_helper_opivx *fn, imm_mode_t imm_mode) { - if (!opivx_check(s, a)) { - return false; - } - if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) { gvec_fn(s->sew, vreg_ofs(s, a->rd), vreg_ofs(s, a->rs2), extract_imm(s, a->rs1, imm_mode), MAXSZ(s), MAXSZ(s));
@@ -1453,6 +1448,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##OPIVX##_b, gen_helper_##OPIVX##_h, \ gen_helper_##OPIVX##_w, gen_helper_##OPIVX##_d, \ }; \ + if (!opivx_check(s, a)) { \ + return false; \ + } \ return do_opivi_gvec(s, a, tcg_gen_gvec_##SUF, \ fns[s->sew], IMM_MODE); \ }
@@ -1775,10 +1773,6 @@ static inline bool do_opivx_gvec_shift(DisasContext *s, arg_rmrr *a, GVecGen2sFn32 *gvec_fn, gen_helper_opivx *fn) { - if (!opivx_check(s, a)) { - return false; - } - if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) { TCGv_i32 src1 = tcg_temp_new_i32();
@@ -1800,7 +1794,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ }; \ - \ + if (!opivx_check(s, a)) { \ + return false; \ + } \ return do_opivx_gvec_shift(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ }
Signature · 1 line
2.40.1
Re: [PATCH v3 05/19] target/riscv: Move vector translation checks
Quoted 9 lines · Lawrence Hunter
From: Nazar Kazakov <[email protected]>
Move the checks out of
do_opiv{v,x,i}_gvec{,_shift}functions and into the corresponding macros. This enables the functions to be reused in proceeding commits without check duplication.Signed-off-by Nazar Kazakov <[email protected]>Reviewed-by Richard Henderson <[email protected]>---
Weiwei Li
Quoted 91 lines
target/riscv/insn_trans/trans_rvv.c.inc | 28 +++++++++++-------------- 1 file changed, 12 insertions(+), 16 deletions(-)target/riscv/insn_trans/trans_rvv.c.inc +12
-16@@ -1183,9 +1183,6 @@ do_opivv_gvec(DisasContext *s, arg_rmrr *a, GVecGen3Fn *gvec_fn, gen_helper_gvec_4_ptr *fn) { TCGLabel *over = gen_new_label(); - if (!opivv_check(s, a)) { - return false; - } tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over);@@ -1218,6 +1215,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ }; \ + if (!opivv_check(s, a)) { \ + return false; \ + } \ return do_opivv_gvec(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ }@@ -1276,10 +1276,6 @@ static inline bool do_opivx_gvec(DisasContext *s, arg_rmrr *a, GVecGen2sFn *gvec_fn, gen_helper_opivx *fn) { - if (!opivx_check(s, a)) { - return false; - } - if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) { TCGv_i64 src1 = tcg_temp_new_i64();@@ -1301,6 +1297,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ }; \ + if (!opivx_check(s, a)) { \ + return false; \ + } \ return do_opivx_gvec(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ }@@ -1432,10 +1431,6 @@ static inline bool do_opivi_gvec(DisasContext *s, arg_rmrr *a, GVecGen2iFn *gvec_fn, gen_helper_opivx *fn, imm_mode_t imm_mode) { - if (!opivx_check(s, a)) { - return false; - } - if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) { gvec_fn(s->sew, vreg_ofs(s, a->rd), vreg_ofs(s, a->rs2), extract_imm(s, a->rs1, imm_mode), MAXSZ(s), MAXSZ(s));@@ -1453,6 +1448,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##OPIVX##_b, gen_helper_##OPIVX##_h, \ gen_helper_##OPIVX##_w, gen_helper_##OPIVX##_d, \ }; \ + if (!opivx_check(s, a)) { \ + return false; \ + } \ return do_opivi_gvec(s, a, tcg_gen_gvec_##SUF, \ fns[s->sew], IMM_MODE); \ }@@ -1775,10 +1773,6 @@ static inline bool do_opivx_gvec_shift(DisasContext *s, arg_rmrr *a, GVecGen2sFn32 *gvec_fn, gen_helper_opivx *fn) { - if (!opivx_check(s, a)) { - return false; - } - if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) { TCGv_i32 src1 = tcg_temp_new_i32();@@ -1800,7 +1794,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ }; \ - \ + if (!opivx_check(s, a)) { \ + return false; \ + } \ return do_opivx_gvec_shift(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ }
PATCH v3 6/19 target/riscv: Refactor translation of vector-widening instruction
In reply to Lawrence HunterFrom: Dickon Hood <[email protected]>
Zvbb (implemented in later commit) has a widening instruction, which requires an extra check on the enabled extensions. Refactor GEN_OPIVX_WIDEN_TRANS() to take a check function to avoid reimplementing it.
--- target/riscv/insn_trans/trans_rvv.c.inc | 52 +++++++++++-------------- 1 file changed, 23 insertions(+), 29 deletions(-)
target/riscv/insn_trans/trans_rvv.c.inc +23 -29
@@ -1526,30 +1526,24 @@ static bool opivx_widen_check(DisasContext *s, arg_rmrr *a) vext_check_ds(s, a->rd, a->rs2, a->vm); } -static bool do_opivx_widen(DisasContext *s, arg_rmrr *a, - gen_helper_opivx *fn) -{ - if (opivx_widen_check(s, a)) { - return opivx_trans(a->rd, a->rs1, a->rs2, a->vm, fn, s); - } - return false; +#define GEN_OPIVX_WIDEN_TRANS(NAME, CHECK) \ +static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ +{ \ + if (CHECK(s, a)) { \ + static gen_helper_opivx * const fns[3] = { \ + gen_helper_##NAME##_b, \ + gen_helper_##NAME##_h, \ + gen_helper_##NAME##_w \ + }; \ + return opivx_trans(a->rd, a->rs1, a->rs2, a->vm, fns[s->sew], s); \ + } \ + return false; \ } -#define GEN_OPIVX_WIDEN_TRANS(NAME) \ -static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ -{ \ - static gen_helper_opivx * const fns[3] = { \ - gen_helper_##NAME##_b, \ - gen_helper_##NAME##_h, \ - gen_helper_##NAME##_w \ - }; \ - return do_opivx_widen(s, a, fns[s->sew]); \ -} - -GEN_OPIVX_WIDEN_TRANS(vwaddu_vx) -GEN_OPIVX_WIDEN_TRANS(vwadd_vx) -GEN_OPIVX_WIDEN_TRANS(vwsubu_vx) -GEN_OPIVX_WIDEN_TRANS(vwsub_vx) +GEN_OPIVX_WIDEN_TRANS(vwaddu_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwadd_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwsubu_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwsub_vx, opivx_widen_check) /* WIDEN OPIVV with WIDEN */ static bool opiwv_widen_check(DisasContext *s, arg_rmrr *a)
@@ -1997,9 +1991,9 @@ GEN_OPIVX_TRANS(vrem_vx, opivx_check) GEN_OPIVV_WIDEN_TRANS(vwmul_vv, opivv_widen_check) GEN_OPIVV_WIDEN_TRANS(vwmulu_vv, opivv_widen_check) GEN_OPIVV_WIDEN_TRANS(vwmulsu_vv, opivv_widen_check) -GEN_OPIVX_WIDEN_TRANS(vwmul_vx) -GEN_OPIVX_WIDEN_TRANS(vwmulu_vx) -GEN_OPIVX_WIDEN_TRANS(vwmulsu_vx) +GEN_OPIVX_WIDEN_TRANS(vwmul_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwmulu_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwmulsu_vx, opivx_widen_check) /* Vector Single-Width Integer Multiply-Add Instructions */ GEN_OPIVV_TRANS(vmacc_vv, opivv_check)
@@ -2015,10 +2009,10 @@ GEN_OPIVX_TRANS(vnmsub_vx, opivx_check) GEN_OPIVV_WIDEN_TRANS(vwmaccu_vv, opivv_widen_check) GEN_OPIVV_WIDEN_TRANS(vwmacc_vv, opivv_widen_check) GEN_OPIVV_WIDEN_TRANS(vwmaccsu_vv, opivv_widen_check) -GEN_OPIVX_WIDEN_TRANS(vwmaccu_vx) -GEN_OPIVX_WIDEN_TRANS(vwmacc_vx) -GEN_OPIVX_WIDEN_TRANS(vwmaccsu_vx) -GEN_OPIVX_WIDEN_TRANS(vwmaccus_vx) +GEN_OPIVX_WIDEN_TRANS(vwmaccu_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwmacc_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwmaccsu_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwmaccus_vx, opivx_widen_check) /* Vector Integer Merge and Move Instructions */ static bool trans_vmv_v_v(DisasContext *s, arg_vmv_v_v *a)
Signature · 1 line
2.40.1
Re: [PATCH v3 06/19] target/riscv: Refactor translation of vector-widening instruction
Quoted 10 lines · Lawrence Hunter
From: Dickon Hood <[email protected]>
Zvbb (implemented in later commit) has a widening instruction, which requires an extra check on the enabled extensions. Refactor
GEN_OPIVX_WIDEN_TRANS()to take a check function to avoid reimplementing it.Signed-off-by Dickon Hood <[email protected]>Reviewed-by Richard Henderson <[email protected]>---
Weiwei Li
Quoted 82 lines
target/riscv/insn_trans/trans_rvv.c.inc | 52 +++++++++++-------------- 1 file changed, 23 insertions(+), 29 deletions(-)target/riscv/insn_trans/trans_rvv.c.inc +23
-29@@ -1526,30 +1526,24 @@ static bool opivx_widen_check(DisasContext *s, arg_rmrr *a) vext_check_ds(s, a->rd, a->rs2, a->vm); } -static bool do_opivx_widen(DisasContext *s, arg_rmrr *a, - gen_helper_opivx *fn) -{ - if (opivx_widen_check(s, a)) { - return opivx_trans(a->rd, a->rs1, a->rs2, a->vm, fn, s); - } - return false; +#define GEN_OPIVX_WIDEN_TRANS(NAME, CHECK) \ +static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ +{ \ + if (CHECK(s, a)) { \ + static gen_helper_opivx * const fns[3] = { \ + gen_helper_##NAME##_b, \ + gen_helper_##NAME##_h, \ + gen_helper_##NAME##_w \ + }; \ + return opivx_trans(a->rd, a->rs1, a->rs2, a->vm, fns[s->sew], s); \ + } \ + return false; \ } -#define GEN_OPIVX_WIDEN_TRANS(NAME) \ -static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ -{ \ - static gen_helper_opivx * const fns[3] = { \ - gen_helper_##NAME##_b, \ - gen_helper_##NAME##_h, \ - gen_helper_##NAME##_w \ - }; \ - return do_opivx_widen(s, a, fns[s->sew]); \ -} - -GEN_OPIVX_WIDEN_TRANS(vwaddu_vx) -GEN_OPIVX_WIDEN_TRANS(vwadd_vx) -GEN_OPIVX_WIDEN_TRANS(vwsubu_vx) -GEN_OPIVX_WIDEN_TRANS(vwsub_vx) +GEN_OPIVX_WIDEN_TRANS(vwaddu_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwadd_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwsubu_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwsub_vx, opivx_widen_check) /* WIDEN OPIVV with WIDEN */ static bool opiwv_widen_check(DisasContext *s, arg_rmrr *a)@@ -1997,9 +1991,9 @@ GEN_OPIVX_TRANS(vrem_vx, opivx_check) GEN_OPIVV_WIDEN_TRANS(vwmul_vv, opivv_widen_check) GEN_OPIVV_WIDEN_TRANS(vwmulu_vv, opivv_widen_check) GEN_OPIVV_WIDEN_TRANS(vwmulsu_vv, opivv_widen_check) -GEN_OPIVX_WIDEN_TRANS(vwmul_vx) -GEN_OPIVX_WIDEN_TRANS(vwmulu_vx) -GEN_OPIVX_WIDEN_TRANS(vwmulsu_vx) +GEN_OPIVX_WIDEN_TRANS(vwmul_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwmulu_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwmulsu_vx, opivx_widen_check) /* Vector Single-Width Integer Multiply-Add Instructions */ GEN_OPIVV_TRANS(vmacc_vv, opivv_check)@@ -2015,10 +2009,10 @@ GEN_OPIVX_TRANS(vnmsub_vx, opivx_check) GEN_OPIVV_WIDEN_TRANS(vwmaccu_vv, opivv_widen_check) GEN_OPIVV_WIDEN_TRANS(vwmacc_vv, opivv_widen_check) GEN_OPIVV_WIDEN_TRANS(vwmaccsu_vv, opivv_widen_check) -GEN_OPIVX_WIDEN_TRANS(vwmaccu_vx) -GEN_OPIVX_WIDEN_TRANS(vwmacc_vx) -GEN_OPIVX_WIDEN_TRANS(vwmaccsu_vx) -GEN_OPIVX_WIDEN_TRANS(vwmaccus_vx) +GEN_OPIVX_WIDEN_TRANS(vwmaccu_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwmacc_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwmaccsu_vx, opivx_widen_check) +GEN_OPIVX_WIDEN_TRANS(vwmaccus_vx, opivx_widen_check) /* Vector Integer Merge and Move Instructions */ static bool trans_vmv_v_v(DisasContext *s, arg_vmv_v_v *a)
PATCH v3 7/19 target/riscv: Refactor some of the generic vector functionality
In reply to Lawrence HunterFrom: Kiran Ostrolenk <[email protected]>
Move some macros out of vector_helper and into vector_internals. This ensures they can be used by both vector and vector-crypto helpers (latter implemented in proceeding commits).
--- target/riscv/vector_helper.c | 42 ------------------------------ target/riscv/vector_internals.h | 46 +++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+), 42 deletions(-)
target/riscv/vector_helper.c +0 -42
@@ -646,9 +646,6 @@ GEN_VEXT_ST_WHOLE(vs8r_v, int8_t, ste_b)#define OP_SUS_H int16_t, uint16_t, int16_t, uint16_t, int16_t#define OP_SUS_W int32_t, uint32_t, int32_t, uint32_t, int32_t#define OP_SUS_D int64_t, uint64_t, int64_t, uint64_t, int64_t -#define WOP_UUU_B uint16_t, uint8_t, uint8_t, uint16_t, uint16_t -#define WOP_UUU_H uint32_t, uint16_t, uint16_t, uint32_t, uint32_t -#define WOP_UUU_W uint64_t, uint32_t, uint32_t, uint64_t, uint64_t#define WOP_SSS_B int16_t, int8_t, int8_t, int16_t, int16_t#define WOP_SSS_H int32_t, int16_t, int16_t, int32_t, int32_t#define WOP_SSS_W int64_t, int32_t, int32_t, int64_t, int64_t
@@ -3412,11 +3409,6 @@ GEN_VEXT_VF(vfwnmsac_vf_h, 4)GEN_VEXT_VF (vfwnmsac_vf_w,8 ) /* Vector Floating-Point Square-Root Instruction */ -/* (TD, T2, TX2) */ -#define OP_UU_H uint16_t, uint16_t, uint16_t -#define OP_UU_W uint32_t, uint32_t, uint32_t -#define OP_UU_D uint64_t, uint64_t, uint64_t -#define OPFVV1 (NAME ,TD ,T2 ,TX2 ,HD ,HS2 ,OP ) \ static void do_##NAME(void *vd, void *vs2, int i, \ CPURISCVState *env) \
@@ -4109,40 +4101,6 @@ GEN_VEXT_CMP_VF(vmfge_vf_w, uint32_t, H4, vmfge32)GEN_VEXT_CMP_VF (vmfge_vf_d, uint64_t,H8 , vmfge64) /* Vector Floating-Point Classify Instruction */ -#define OPIVV1 (NAME ,TD ,T2 ,TX2 ,HD ,HS2 ,OP ) \ -static void do_##NAME(void *vd, void *vs2, int i) \ -{ \ - TX2 s2 = *((T2 *)vs2 + HS2(i)); \ - *((TD *)vd + HD(i)) = OP(s2); \ -} - -#define GEN_VEXT_V (NAME ,ESZ ) \ -void HELPER(NAME)(void *vd, void *v0, void *vs2, \ - CPURISCVState *env, uint32_t desc) \ -{ \ - uint32_t vm = vext_vm(desc); \ - uint32_t vl = env->vl; \ - uint32_t total_elems = \ - vext_get_total_elems(env, desc, ESZ); \ - uint32_t vta = vext_vta(desc); \ - uint32_t vma = vext_vma(desc); \ - uint32_t i; \ - \ - for (i = env->vstart; i < vl; i++) { \ - if (!vm && !vext_elem_mask(v0, i)) { \ -/* set masked-off elements to 1s */ \ -vext_set_elems_1s (vd, vma, i *ESZ , \ - (i +1 ) *ESZ ); \ -continue ; \ - } \ - do_##NAME(vd, vs2, i); \ - } \ - env->vstart = 0; \ -/* set tail elements to 1s */ \ -vext_set_elems_1s (vd, vta, vl *ESZ , \ - total_elems *ESZ ); \ -} -target_ulong fclass_h (uint64_t frs1) {float16 f = frs1;
target/riscv/vector_internals.h +46 -0
62 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH v3 07/19] target/riscv: Refactor some of the generic vector functionality
Quoted 8 lines · Lawrence Hunter
From: Kiran Ostrolenk <[email protected]>
Move some macros out of
vector_helperand intovector_internals. This ensures they can be used by both vector and vector-crypto helpers (latter implemented in proceeding commits).Signed-off-by Kiran Ostrolenk <[email protected]>---
Weiwei Li
Quoted 139 lines
target/riscv/vector_helper.c | 42 ------------------------------ target/riscv/vector_internals.h | 46 +++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+), 42 deletions(-)target/riscv/vector_helper.c +0
-42@@ -646,9 +646,6 @@ GEN_VEXT_ST_WHOLE(vs8r_v, int8_t, ste_b)#define OP_SUS_H int16_t, uint16_t, int16_t, uint16_t, int16_t#define OP_SUS_W int32_t, uint32_t, int32_t, uint32_t, int32_t#define OP_SUS_D int64_t, uint64_t, int64_t, uint64_t, int64_t -#define WOP_UUU_B uint16_t, uint8_t, uint8_t, uint16_t, uint16_t -#define WOP_UUU_H uint32_t, uint16_t, uint16_t, uint32_t, uint32_t -#define WOP_UUU_W uint64_t, uint32_t, uint32_t, uint64_t, uint64_t#define WOP_SSS_B int16_t, int8_t, int8_t, int16_t, int16_t#define WOP_SSS_H int32_t, int16_t, int16_t, int32_t, int32_t#define WOP_SSS_W int64_t, int32_t, int32_t, int64_t, int64_t@@ -3412,11 +3409,6 @@ GEN_VEXT_VF(vfwnmsac_vf_h, 4)GEN_VEXT_VF (vfwnmsac_vf_w,8 ) /* Vector Floating-Point Square-Root Instruction */ -/* (TD, T2, TX2) */ -#define OP_UU_H uint16_t, uint16_t, uint16_t -#define OP_UU_W uint32_t, uint32_t, uint32_t -#define OP_UU_D uint64_t, uint64_t, uint64_t -#define OPFVV1 (NAME ,TD ,T2 ,TX2 ,HD ,HS2 ,OP ) \ static void do_##NAME(void *vd, void *vs2, int i, \ CPURISCVState *env) \@@ -4109,40 +4101,6 @@ GEN_VEXT_CMP_VF(vmfge_vf_w, uint32_t, H4, vmfge32)GEN_VEXT_CMP_VF (vmfge_vf_d, uint64_t,H8 , vmfge64) /* Vector Floating-Point Classify Instruction */ -#define OPIVV1 (NAME ,TD ,T2 ,TX2 ,HD ,HS2 ,OP ) \ -static void do_##NAME(void *vd, void *vs2, int i) \ -{ \ - TX2 s2 = *((T2 *)vs2 + HS2(i)); \ - *((TD *)vd + HD(i)) = OP(s2); \ -} - -#define GEN_VEXT_V (NAME ,ESZ ) \ -void HELPER(NAME)(void *vd, void *v0, void *vs2, \ - CPURISCVState *env, uint32_t desc) \ -{ \ - uint32_t vm = vext_vm(desc); \ - uint32_t vl = env->vl; \ - uint32_t total_elems = \ - vext_get_total_elems(env, desc, ESZ); \ - uint32_t vta = vext_vta(desc); \ - uint32_t vma = vext_vma(desc); \ - uint32_t i; \ - \ - for (i = env->vstart; i < vl; i++) { \ - if (!vm && !vext_elem_mask(v0, i)) { \ -/* set masked-off elements to 1s */ \ -vext_set_elems_1s (vd, vma, i *ESZ , \ - (i +1 ) *ESZ ); \ -continue ; \ - } \ - do_##NAME(vd, vs2, i); \ - } \ - env->vstart = 0; \ -/* set tail elements to 1s */ \ -vext_set_elems_1s (vd, vta, vl *ESZ , \ - total_elems *ESZ ); \ -} -target_ulong fclass_h (uint64_t frs1) {float16 f = frs1;target/riscv/vector_internals.h +46
-0@@ -121,12 +121,52 @@ void vext_set_elems_1s(void *base, uint32_t is_agnostic, uint32_t cnt,/* expand macro args before macro */ #define RVVCALL (macro, ...) macro(__VA_ARGS__) +/* (TD, T2, TX2) */ +#define OP_UU_B uint8_t, uint8_t, uint8_t +#define OP_UU_H uint16_t, uint16_t, uint16_t +#define OP_UU_W uint32_t, uint32_t, uint32_t +#define OP_UU_D uint64_t, uint64_t, uint64_t +/* (TD, T1, T2, TX1, TX2) */ #define OP_UUU_B uint8_t, uint8_t, uint8_t, uint8_t, uint8_t#define OP_UUU_H uint16_t, uint16_t, uint16_t, uint16_t, uint16_t#define OP_UUU_W uint32_t, uint32_t, uint32_t, uint32_t, uint32_t#define OP_UUU_D uint64_t, uint64_t, uint64_t, uint64_t, uint64_t +#define OPIVV1 (NAME ,TD ,T2 ,TX2 ,HD ,HS2 ,OP ) \ +static void do_##NAME(void *vd, void *vs2, int i) \ +{ \ + TX2 s2 = *((T2 *)vs2 + HS2(i)); \ + *((TD *)vd + HD(i)) = OP(s2); \ +} + +#define GEN_VEXT_V (NAME ,ESZ ) \ +void HELPER(NAME)(void *vd, void *v0, void *vs2, \ + CPURISCVState *env, uint32_t desc) \ +{ \ + uint32_t vm = vext_vm(desc); \ + uint32_t vl = env->vl; \ + uint32_t total_elems = \ + vext_get_total_elems(env, desc, ESZ); \ + uint32_t vta = vext_vta(desc); \ + uint32_t vma = vext_vma(desc); \ + uint32_t i; \ + \ + for (i = env->vstart; i < vl; i++) { \ + if (!vm && !vext_elem_mask(v0, i)) { \ +/* set masked-off elements to 1s */ \ +vext_set_elems_1s (vd, vma, i *ESZ , \ + (i +1 ) *ESZ ); \ +continue ; \ + } \ + do_##NAME(vd, vs2, i); \ + } \ + env->vstart = 0; \ +/* set tail elements to 1s */ \ +vext_set_elems_1s (vd, vta, vl *ESZ , \ + total_elems *ESZ ); \ +} +/* operation of two vector elements */ typedef void opivv2_fn (void *vd,void *vs1,void *vs2,int i);@@ -179,4 +219,10 @@ void HELPER(NAME)(void *vd, void *v0, target_ulong s1, \ do_##NAME, ESZ); \ } +/* Three of the widening shortening macros: */ +/* (TD, T1, T2, TX1, TX2) */ +#define WOP_UUU_B uint16_t, uint8_t, uint8_t, uint16_t, uint16_t +#define WOP_UUU_H uint32_t, uint16_t, uint16_t, uint32_t, uint32_t +#define WOP_UUU_W uint64_t, uint32_t, uint32_t, uint64_t, uint64_t +#endif /* TARGET_RISCV_VECTOR_INTERNALS_H */
PATCH v3 8/19 qemu/bitops.h: Limit rotate amounts
In reply to Lawrence HunterFrom: Dickon Hood <[email protected]>
Rotates have been fixed up to only allow for reasonable rotate amounts (ie, no rotates >7 on an 8b value etc.) This fixes a problem with riscv vector rotate instructions.
--- include/qemu/bitops.h | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-)
include/qemu/bitops.h +16 -8
@@ -218,7 +218,8 @@ static inline unsigned long find_first_zero_bit(const unsigned long *addr,*/ static inline uint8_t rol8 (uint8_t word,unsigned int shift) { -return (word << shift) | (word >> ((8 - shift) &7 )); + shift &=7 ; +return (word << shift) | (word >> (8 - shift)); }/**
@@ -228,7 +229,8 @@ static inline uint8_t rol8(uint8_t word, unsigned int shift)*/ static inline uint8_t ror8 (uint8_t word,unsigned int shift) { -return (word >> shift) | (word << ((8 - shift) &7 )); + shift &=7 ; +return (word >> shift) | (word << (8 - shift)); }/**
@@ -238,7 +240,8 @@ static inline uint8_t ror8(uint8_t word, unsigned int shift)*/ static inline uint16_t rol16 (uint16_t word,unsigned int shift) { -return (word << shift) | (word >> ((16 - shift) &15 )); + shift &=15 ; +return (word << shift) | (word >> (16 - shift)); }/**
@@ -248,7 +251,8 @@ static inline uint16_t rol16(uint16_t word, unsigned int shift)*/ static inline uint16_t ror16 (uint16_t word,unsigned int shift) { -return (word >> shift) | (word << ((16 - shift) &15 )); + shift &=15 ; +return (word >> shift) | (word << (16 - shift)); }/**
@@ -258,7 +262,8 @@ static inline uint16_t ror16(uint16_t word, unsigned int shift)*/ static inline uint32_t rol32 (uint32_t word,unsigned int shift) { -return (word << shift) | (word >> ((32 - shift) &31 )); + shift &=31 ; +return (word << shift) | (word >> (32 - shift)); }/**
@@ -268,7 +273,8 @@ static inline uint32_t rol32(uint32_t word, unsigned int shift)*/ static inline uint32_t ror32 (uint32_t word,unsigned int shift) { -return (word >> shift) | (word << ((32 - shift) &31 )); + shift &=31 ; +return (word >> shift) | (word << (32 - shift)); }/**
@@ -278,7 +284,8 @@ static inline uint32_t ror32(uint32_t word, unsigned int shift)*/ static inline uint64_t rol64 (uint64_t word,unsigned int shift) { -return (word << shift) | (word >> ((64 - shift) &63 )); + shift &=63 ; +return (word << shift) | (word >> (64 - shift)); }/**
@@ -288,7 +295,8 @@ static inline uint64_t rol64(uint64_t word, unsigned int shift)*/ static inline uint64_t ror64 (uint64_t word,unsigned int shift) { -return (word >> shift) | (word << ((64 - shift) &63 )); + shift &=63 ; +return (word >> shift) | (word << (64 - shift)); }/**
Signature · 1 line
2.40.1
Richard Henderson
AppliedRe: [PATCH v3 08/19] qemu/bitops.h: Limit rotate amounts
Quoted 11 lines · Lawrence Hunter
From: Dickon Hood<[email protected]>
Rotates have been fixed up to only allow for reasonable rotate amounts (ie, no rotates >7 on an 8b value etc.) This fixes a problem with riscv vector rotate instructions.
Signed-off-by Dickon Hood<[email protected]>Reviewed-by Richard Henderson<[email protected]>--- include/qemu/bitops.h | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-)
Queued to tcg-next.
r~
Re: [PATCH v3 08/19] qemu/bitops.h: Limit rotate amounts
In reply to Lawrence Hunterinclude/qemu/bitops.h static inline uint32_t rol32(uint32_t word, unsigned int shift) · line 277
Quoted 5 lines · Lawrence Hunter
static inline
uint32_tror32(uint32_t word, unsigned int shift){include/qemu/bitops.h +2
-1-return (word >> shift) | (word << ((32 - shift) &31 )); + shift &=31 ; +return (word >> shift) | (word << (32 - shift));
This is incorrect, because if shift == 0, you are now performing (word << 32).
I agree with your original intent though, to mask and accept any rotation. I've changed these like so:
- return (word >> shift) | (word << ((32 - shift) & 31));
+ return (word >> (shift & 31)) | (word << (-shift & 31));
which also eliminates the useless subtract from word-size-before-mask.
r~
PATCH v3 9/19 tcg: Add andcs and rotrs tcg gvec ops
In reply to Lawrence HunterFrom: Nazar Kazakov <[email protected]>
This commit adds helper functions and tcg operation definitions for the andcs and rotrs instructions
--- accel/tcg/tcg-runtime-gvec.c | 11 +++++++++++ accel/tcg/tcg-runtime.h | 1 + include/tcg/tcg-op-gvec.h | 4 ++++ tcg/tcg-op-gvec.c | 23 +++++++++++++++++++++++ 4 files changed, 39 insertions(+)
accel/tcg/tcg-runtime-gvec.c +11 -0
@@ -550,6 +550,17 @@ void HELPER(gvec_ands)(void *d, void *a, uint64_t b, uint32_t desc)clear_high (d, oprsz, desc); } +void HELPER (gvec_andcs )(void *d,void *a,uint64_t b,uint32_t desc) +{ +intptr_t oprsz =simd_oprsz (desc); +intptr_t i; + +for (i =0 ; i < oprsz; i +=sizeof (uint64_t )) { + *(uint64_t *)(d + i) = *(uint64_t *)(a + i) & ~b; + } +clear_high (d, oprsz, desc); +} +void HELPER (gvec_xors )(void *d,void *a,uint64_t b,uint32_t desc) {intptr_t oprsz =simd_oprsz (desc);
accel/tcg/tcg-runtime.h +1 -0
@@ -217,6 +217,7 @@ DEF_HELPER_FLAGS_4(gvec_nor, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_eqv, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_ands, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) +DEF_HELPER_FLAGS_4(gvec_andcs, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) DEF_HELPER_FLAGS_4(gvec_xors, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32)DEF_HELPER_FLAGS_4 (gvec_ors,TCG_CALL_NO_RWG , void, ptr, ptr, i64, i32)
include/tcg/tcg-op-gvec.h +4 -0
@@ -330,6 +330,8 @@ void tcg_gen_gvec_ori(unsigned vece, uint32_t dofs, uint32_t aofs,void tcg_gen_gvec_ands (unsigned vece,uint32_t dofs,uint32_t aofs,TCGv_i64 c,uint32_t oprsz,uint32_t maxsz); +void tcg_gen_gvec_andcs (unsigned vece,uint32_t dofs,uint32_t aofs, +TCGv_i64 c,uint32_t oprsz,uint32_t maxsz);void tcg_gen_gvec_xors (unsigned vece,uint32_t dofs,uint32_t aofs,TCGv_i64 c,uint32_t oprsz,uint32_t maxsz); void tcg_gen_gvec_ors(unsigned vece, uint32_t dofs, uint32_t aofs,
@@ -369,6 +371,8 @@ void tcg_gen_gvec_sars(unsigned vece, uint32_t dofs, uint32_t aofs, TCGv_i32 shift, uint32_t oprsz, uint32_t maxsz);void tcg_gen_gvec_rotls (unsigned vece,uint32_t dofs,uint32_t aofs,TCGv_i32 shift,uint32_t oprsz,uint32_t maxsz); +void tcg_gen_gvec_rotrs (unsigned vece,uint32_t dofs,uint32_t aofs, +TCGv_i32 shift,uint32_t oprsz,uint32_t maxsz);/* * Perform vector shift by vector element, modulo the element size.
tcg/tcg-op-gvec.c +23 -0
@@ -2761,6 +2761,21 @@ void tcg_gen_gvec_andi(unsigned vece, uint32_t dofs, uint32_t aofs,tcg_gen_gvec_2s (dofs, aofs, oprsz, maxsz, tmp, &gop_ands); } +void tcg_gen_gvec_andcs (unsigned vece,uint32_t dofs,uint32_t aofs, +TCGv_i64 c,uint32_t oprsz,uint32_t maxsz) +{ +static GVecGen2s g = { + .fni8 = tcg_gen_andc_i64, + .fniv = tcg_gen_andc_vec, + .fno = gen_helper_gvec_andcs, + .prefer_i64 =TCG_TARGET_REG_BITS ==64 , + .vece =MO_64 + }; + +tcg_gen_dup_i64 (vece, c, c); +tcg_gen_gvec_2s (dofs, aofs, oprsz, maxsz, c, &g); +} +static const GVecGen2s gop_xors = { .fni8 = tcg_gen_xor_i64, .fniv = tcg_gen_xor_vec,
@@ -3336,6 +3351,14 @@ void tcg_gen_gvec_rotls(unsigned vece, uint32_t dofs, uint32_t aofs,do_gvec_shifts (vece, dofs, aofs, shift, oprsz, maxsz, &g); } +void tcg_gen_gvec_rotrs (unsigned vece,uint32_t dofs,uint32_t aofs, +TCGv_i32 shift,uint32_t oprsz,uint32_t maxsz) +{ +TCGv_i32 tmp =tcg_temp_new_i32 (); +tcg_gen_sub_i32 (tmp,tcg_constant_i32 (1 << (vece +3 )), shift); +tcg_gen_gvec_rotls (vece, dofs, aofs, tmp, oprsz, maxsz); +} +/* * Expand D = A << (B % element bits) *
Signature · 1 line
2.40.1
Richard Henderson
AppliedRe: [PATCH v3 09/19] tcg: Add andcs and rotrs tcg gvec ops
Quoted 11 lines · Lawrence Hunter
From: Nazar Kazakov <[email protected]>
This commit adds helper functions and tcg operation definitions for the andcs and rotrs instructions
Signed-off-by Nazar Kazakov <[email protected]>--- accel/tcg/tcg-runtime-gvec.c | 11 +++++++++++ accel/tcg/tcg-runtime.h | 1 + include/tcg/tcg-op-gvec.h | 4 ++++ tcg/tcg-op-gvec.c | 23 +++++++++++++++++++++++ 4 files changed, 39 insertions(+)
Queued to tcg-next as two patches, and with alterations:
Quoted 4 lines
tcg/tcg-op-gvec.c +4
-0+void tcg_gen_gvec_andcs (unsigned vece,uint32_t dofs,uint32_t aofs, +TCGv_i64 c,uint32_t oprsz,uint32_t maxsz) +{ +static GVecGen2s g = {
tcg/tcg-op-gvec.c void tcg_gen_gvec_andi(unsigned vece, uint32_t dofs, uint32_t aofs, · line 2777
Quoted 10 lines
tcg/tcg-op-gvec.c +10
-0+ .fni8 = tcg_gen_andc_i64, + .fniv = tcg_gen_andc_vec, + .fno = gen_helper_gvec_andcs, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + .vece = MO_64 + }; + +tcg_gen_dup_i64 (vece, c, c); +tcg_gen_gvec_2s (dofs, aofs, oprsz, maxsz, c, &g); +}
This needed a temporary.
tcg/tcg-op-gvec.c void tcg_gen_gvec_rotls(unsigned vece, uint32_t dofs, uint32_t aofs, · line 3360
Quoted 7 lines
tcg/tcg-op-gvec.c +7
-0+void tcg_gen_gvec_rotrs (unsigned vece,uint32_t dofs,uint32_t aofs, +TCGv_i32 shift,uint32_t oprsz,uint32_t maxsz) +{ +TCGv_i32 tmp =tcg_temp_new_i32 (); +tcg_gen_sub_i32 (tmp,tcg_constant_i32 (1 << (vece +3 )), shift); +tcg_gen_gvec_rotls (vece, dofs, aofs, tmp, oprsz, maxsz); +}
This needed the rotation count to be masked (32 - 0 == 32 is illegal). Simplified as (-shift & mask).
r~
PATCH v3 10/19 qemu/host-utils.h: Add clz and ctz functions for lower-bit integers
In reply to Lawrence HunterFrom: Kiran Ostrolenk <[email protected]>
This is for use in the RISC-V vclz and vctz instructions (implemented in proceeding commit).
--- include/qemu/host-utils.h | 54 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+)
include/qemu/host-utils.h +54 -0
@@ -107,6 +107,36 @@ static inline uint64_t muldiv64(uint64_t a, uint32_t b, uint32_t c) }#endif +/** +* clz8 - count leading zeros in a 8-bit value. +* @val: The value to search +* +* Returns 8 if the value is zero. Note that the GCC builtin is +* undefined if the value is zero. +* +* Note that the GCC builtin will upcast its argument to an `unsigned int` +* so this function subtracts off the number of prepended zeroes. +*/ +static inline int clz8 (uint8_t val) +{ +return val ?__builtin_clz (val) -24 :8 ; +} + +/** +* clz16 - count leading zeros in a 16-bit value. +* @val: The value to search +* +* Returns 16 if the value is zero. Note that the GCC builtin is +* undefined if the value is zero. +* +* Note that the GCC builtin will upcast its argument to an `unsigned int` +* so this function subtracts off the number of prepended zeroes. +*/ +static inline int clz16 (uint16_t val) +{ +return val ?__builtin_clz (val) -16 :16 ; +} +/** * clz32 - count leading zeros in a 32-bit value. * @val: The value to search
@@ -153,6 +183,30 @@ static inline int clo64(uint64_t val)return clz64 (~val); } +/** +* ctz8 - count trailing zeros in a 8-bit value. +* @val: The value to search +* +* Returns 8 if the value is zero. Note that the GCC builtin is +* undefined if the value is zero. +*/ +static inline int ctz8 (uint8_t val) +{ +return val ?__builtin_ctz (val) :8 ; +} + +/** +* ctz16 - count trailing zeros in a 16-bit value. +* @val: The value to search +* +* Returns 16 if the value is zero. Note that the GCC builtin is +* undefined if the value is zero. +*/ +static inline int ctz16 (uint16_t val) +{ +return val ?__builtin_ctz (val) :16 ; +} +/** * ctz32 - count trailing zeros in a 32-bit value. * @val: The value to search
Signature · 1 line
2.40.1
Richard Henderson
AppliedRe: [PATCH v3 10/19] qemu/host-utils.h: Add clz and ctz functions for lower-bit integers
Quoted 10 lines · Lawrence Hunter
From: Kiran Ostrolenk<[email protected]>
This is for use in the RISC-V vclz and vctz instructions (implemented in proceeding commit).
Signed-off-by Kiran Ostrolenk<[email protected]>Reviewed-by Richard Henderson<[email protected]>--- include/qemu/host-utils.h | 54 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+)
Queued to tcg-next.
r~
PATCH v3 11/19 target/riscv: Add Zvbb ISA extension support
In reply to Lawrence HunterFrom: Dickon Hood <[email protected]>
This commit adds support for the Zvbb vector-crypto extension, which consists of the following instructions:
* vrol.[vv,vx]
* vror.[vv,vx,vi]
* vbrev8.v
* vrev8.v
* vandn.[vv,vx]
* vbrev.v
* vclz.v
* vctz.v
* vcpop.v
* vwsll.[vv,vx,vi]
Translation functions are defined in target/riscv/insn_trans/trans_rvvk.c.inc and helpers are defined in target/riscv/vcrypto_helper.c.
--- target/riscv/cpu.c | 12 ++ target/riscv/cpu.h | 1 + target/riscv/helper.h | 62 +++++++++ target/riscv/insn32.decode | 20 +++ target/riscv/insn_trans/trans_rvv.c.inc | 3 + target/riscv/insn_trans/trans_rvvk.c.inc | 164 +++++++++++++++++++++++ target/riscv/vcrypto_helper.c | 138 +++++++++++++++++++ 7 files changed, 400 insertions(+)
target/riscv/cpu.c +12 -0
@@ -109,6 +109,7 @@ static const struct isa_ext_data isa_edata_arr[] = {ISA_EXT_DATA_ENTRY (zve64d, true,PRIV_VERSION_1_12_0 , ext_zve64d),ISA_EXT_DATA_ENTRY (zvfh, true,PRIV_VERSION_1_12_0 , ext_zvfh),ISA_EXT_DATA_ENTRY (zvfhmin, true,PRIV_VERSION_1_12_0 , ext_zvfhmin), +ISA_EXT_DATA_ENTRY (zvbb, true,PRIV_VERSION_1_12_0 , ext_zvbb),ISA_EXT_DATA_ENTRY (zvbc, true,PRIV_VERSION_1_12_0 , ext_zvbc),ISA_EXT_DATA_ENTRY (zhinx, true,PRIV_VERSION_1_12_0 , ext_zhinx), ISA_EXT_DATA_ENTRY(zhinxmin, true, PRIV_VERSION_1_12_0, ext_zhinxmin),
@@ -1212,6 +1213,17 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)return ; } +/* +* In principle Zve*x would also suffice here, were they supported +* in qemu +*/ +if (cpu->cfg.ext_zvbb && !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || + cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { +error_setg (errp, +"Vector crypto extensions require V or Zve* extensions" ); +return ; + } +if (cpu->cfg.ext_zvbc && !(cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) {error_setg (errp,"Zvbc extension requires V or Zve64{f,d} extensions" );
target/riscv/cpu.h +1 -0
@@ -470,6 +470,7 @@ struct RISCVCPUConfig {bool ext_zve32f;bool ext_zve64f;bool ext_zve64d; +bool ext_zvbb;bool ext_zvbc;bool ext_zmmul;bool ext_zvfh;
target/riscv/helper.h +62 -0
65 lines not on this page · raw message
target/riscv/insn32.decode +20 -0
@@ -37,6 +37,7 @@ %imm_u 12:s20 !function=ex_shift_12 %imm_bs 30:2 !function=ex_shift_3 %imm_rnum 20:4 +%imm_z6 26:1 15:5 # Argument sets: &empty
@@ -82,6 +83,7 @@ @r_vm ...... vm:1 ..... ..... ... ..... ....... &rmrr %rs2 %rs1 %rd @r_vm_1 ...... . ..... ..... ... ..... ....... &rmrr vm=1 %rs2 %rs1 %rd @r_vm_0 ...... . ..... ..... ... ..... ....... &rmrr vm=0 %rs2 %rs1 %rd +@r2_zimm6 ..... . vm:1 ..... ..... ... ..... ....... &rmrr %rs2 rs1=%imm_z6 %rd @r2_zimm11 . zimm:11 ..... ... ..... ....... %rs1 %rd @r2_zimm10 .. zimm:10 ..... ... ..... ....... %rs1 %rd @r2_s ....... ..... ..... ... ..... ....... %rs2 %rs1
@@ -914,3 +916,21 @@ vclmul_vv 001100 . ..... ..... 010 ..... 1010111 @r_vm vclmul_vx 001100 . ..... ..... 110 ..... 1010111 @r_vm vclmulh_vv 001101 . ..... ..... 010 ..... 1010111 @r_vm vclmulh_vx 001101 . ..... ..... 110 ..... 1010111 @r_vm + +# *** Zvbb vector crypto extension *** +vrol_vv 010101 . ..... ..... 000 ..... 1010111 @r_vm +vrol_vx 010101 . ..... ..... 100 ..... 1010111 @r_vm +vror_vv 010100 . ..... ..... 000 ..... 1010111 @r_vm +vror_vx 010100 . ..... ..... 100 ..... 1010111 @r_vm +vror_vi 01010. . ..... ..... 011 ..... 1010111 @r2_zimm6 +vbrev8_v 010010 . ..... 01000 010 ..... 1010111 @r2_vm +vrev8_v 010010 . ..... 01001 010 ..... 1010111 @r2_vm +vandn_vv 000001 . ..... ..... 000 ..... 1010111 @r_vm +vandn_vx 000001 . ..... ..... 100 ..... 1010111 @r_vm +vbrev_v 010010 . ..... 01010 010 ..... 1010111 @r2_vm +vclz_v 010010 . ..... 01100 010 ..... 1010111 @r2_vm +vctz_v 010010 . ..... 01101 010 ..... 1010111 @r2_vm +vcpop_v 010010 . ..... 01110 010 ..... 1010111 @r2_vm +vwsll_vv 110101 . ..... ..... 000 ..... 1010111 @r_vm +vwsll_vx 110101 . ..... ..... 100 ..... 1010111 @r_vm +vwsll_vi 110101 . ..... ..... 011 ..... 1010111 @r_vm
target/riscv/insn_trans/trans_rvv.c.inc +3 -0
@@ -1368,6 +1368,7 @@ GEN_OPIVX_GVEC_TRANS(vrsub_vx, rsubs) typedef enum { IMM_ZX, /* Zero-extended */ IMM_SX, /* Sign-extended */ + IMM_ZIMM6, /* Truncate to 6 bits */ IMM_TRUNC_SEW, /* Truncate to log(SEW) bits */ IMM_TRUNC_2SEW, /* Truncate to log(2*SEW) bits */ } imm_mode_t;
@@ -1383,6 +1384,8 @@ static int64_t extract_imm(DisasContext *s, uint32_t imm, imm_mode_t imm_mode) return extract64(imm, 0, s->sew + 3); case IMM_TRUNC_2SEW: return extract64(imm, 0, s->sew + 4); + case IMM_ZIMM6: + return extract64(imm, 0, 6); default: g_assert_not_reached(); }
target/riscv/insn_trans/trans_rvvk.c.inc +164 -0
167 lines not on this page · raw message
target/riscv/vcrypto_helper.c +138 -0
147 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH v3 11/19] target/riscv: Add Zvbb ISA extension support
Quoted 54 lines · Lawrence Hunter
From: Dickon Hood <[email protected]>
This commit adds support for the Zvbb vector-crypto extension, which consists of the following instructions:
* vrol.[vv,vx]
* vror.[vv,vx,vi]
* vbrev8.v
* vrev8.v
* vandn.[vv,vx]
* vbrev.v
* vclz.v
* vctz.v
* vcpop.v
* vwsll.[vv,vx,vi]Translation functions are defined in
target/riscv/insn_trans/trans_rvvk.c.incand helpers are defined intarget/riscv/vcrypto_helper.c.Co-authored-by Nazar Kazakov <[email protected]>Co-authored-by William Salmon <[email protected]>Co-authored-by Kiran Ostrolenk <[email protected]>Signed-off-by Nazar Kazakov <[email protected]>Signed-off-by William Salmon <[email protected]>Signed-off-by Kiran Ostrolenk <[email protected]>Signed-off-by Dickon Hood <[email protected]>--- target/riscv/cpu.c | 12 ++ target/riscv/cpu.h | 1 + target/riscv/helper.h | 62 +++++++++ target/riscv/insn32.decode | 20 +++ target/riscv/insn_trans/trans_rvv.c.inc | 3 + target/riscv/insn_trans/trans_rvvk.c.inc | 164 +++++++++++++++++++++++ target/riscv/vcrypto_helper.c | 138 +++++++++++++++++++ 7 files changed, 400 insertions(+)target/riscv/cpu.c +2
-0@@ -109,6 +109,7 @@ static const struct isa_ext_data isa_edata_arr[] = {ISA_EXT_DATA_ENTRY (zve64d, true,PRIV_VERSION_1_12_0 , ext_zve64d),ISA_EXT_DATA_ENTRY (zvfh, true,PRIV_VERSION_1_12_0 , ext_zvfh),ISA_EXT_DATA_ENTRY (zvfhmin, true,PRIV_VERSION_1_12_0 , ext_zvfhmin), +ISA_EXT_DATA_ENTRY (zvbb, true,PRIV_VERSION_1_12_0 , ext_zvbb),ISA_EXT_DATA_ENTRY (zvbc, true,PRIV_VERSION_1_12_0 , ext_zvbc),ISA_EXT_DATA_ENTRY (zhinx, true,PRIV_VERSION_1_12_0 , ext_zhinx), ISA_EXT_DATA_ENTRY(zhinxmin, true, PRIV_VERSION_1_12_0, ext_zhinxmin),@@ -1212,6 +1213,17 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)return ; } +/*
target/riscv/cpu.c static void riscv_cpu_realize(DeviceState *dev, Error **errp) · line 1226
Quoted 10 lines
target/riscv/cpu.c +10
-0+* In principle Zve*x would also suffice here, were they supported +* in qemu +*/ +if (cpu->cfg.ext_zvbb && !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || + cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { +error_setg (errp, +"Vector crypto extensions require V or Zve* extensions" ); +return ; + } +
Similar to previous patch. We can only check zve32f here.
Regards,
Weiwei Li
Quoted 473 lines
target/riscv/cpu.c +0
-0if (cpu->cfg.ext_zvbc && !(cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) {error_setg (errp,"Zvbc extension requires V or Zve64{f,d} extensions" );target/riscv/cpu.h +1
-0@@ -470,6 +470,7 @@ struct RISCVCPUConfig {bool ext_zve32f;bool ext_zve64f;bool ext_zve64d; +bool ext_zvbb;bool ext_zvbc;bool ext_zmmul;bool ext_zvfh;target/riscv/helper.h +62
-0@@ -1148,3 +1148,65 @@ DEF_HELPER_6(vclmul_vv, void, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_6(vclmul_vx, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vclmulh_vv, void, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_6(vclmulh_vx, void, ptr, ptr, tl, ptr, env, i32) + +DEF_HELPER_6(vror_vv_b, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vror_vv_h, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vror_vv_w, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vror_vv_d, void, ptr, ptr, ptr, ptr, env, i32) + +DEF_HELPER_6(vror_vx_b, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vror_vx_h, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vror_vx_w, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vror_vx_d, void, ptr, ptr, tl, ptr, env, i32) + +DEF_HELPER_6(vrol_vv_b, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vrol_vv_h, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vrol_vv_w, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vrol_vv_d, void, ptr, ptr, ptr, ptr, env, i32) + +DEF_HELPER_6(vrol_vx_b, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vrol_vx_h, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vrol_vx_w, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vrol_vx_d, void, ptr, ptr, tl, ptr, env, i32) + +DEF_HELPER_5(vrev8_v_b, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vrev8_v_h, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vrev8_v_w, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vrev8_v_d, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vbrev8_v_b, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vbrev8_v_h, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vbrev8_v_w, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vbrev8_v_d, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vbrev_v_b, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vbrev_v_h, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vbrev_v_w, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vbrev_v_d, void, ptr, ptr, ptr, env, i32) + +DEF_HELPER_5(vclz_v_b, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vclz_v_h, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vclz_v_w, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vclz_v_d, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vctz_v_b, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vctz_v_h, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vctz_v_w, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vctz_v_d, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vcpop_v_b, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vcpop_v_h, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vcpop_v_w, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vcpop_v_d, void, ptr, ptr, ptr, env, i32) + +DEF_HELPER_6(vwsll_vv_b, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vwsll_vv_h, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vwsll_vv_w, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vwsll_vx_b, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vwsll_vx_h, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vwsll_vx_w, void, ptr, ptr, tl, ptr, env, i32) + +DEF_HELPER_6(vandn_vv_b, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vandn_vv_h, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vandn_vv_w, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vandn_vv_d, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vandn_vx_b, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vandn_vx_h, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vandn_vx_w, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6 (vandn_vx_d, void, ptr, ptr, tl, ptr, env, i32)target/riscv/insn32.decode +20
-0@@ -37,6 +37,7 @@ %imm_u 12:s20 !function=ex_shift_12 %imm_bs 30:2 !function=ex_shift_3 %imm_rnum 20:4 +%imm_z6 26:1 15:5 # Argument sets: &empty@@ -82,6 +83,7 @@ @r_vm ...... vm:1 ..... ..... ... ..... ....... &rmrr %rs2 %rs1 %rd @r_vm_1 ...... . ..... ..... ... ..... ....... &rmrr vm=1 %rs2 %rs1 %rd @r_vm_0 ...... . ..... ..... ... ..... ....... &rmrr vm=0 %rs2 %rs1 %rd +@r2_zimm6 ..... . vm:1 ..... ..... ... ..... ....... &rmrr %rs2 rs1=%imm_z6 %rd @r2_zimm11 . zimm:11 ..... ... ..... ....... %rs1 %rd @r2_zimm10 .. zimm:10 ..... ... ..... ....... %rs1 %rd @r2_s ....... ..... ..... ... ..... ....... %rs2 %rs1@@ -914,3 +916,21 @@ vclmul_vv 001100 . ..... ..... 010 ..... 1010111 @r_vm vclmul_vx 001100 . ..... ..... 110 ..... 1010111 @r_vm vclmulh_vv 001101 . ..... ..... 010 ..... 1010111 @r_vm vclmulh_vx 001101 . ..... ..... 110 ..... 1010111 @r_vm + +# *** Zvbb vector crypto extension *** +vrol_vv 010101 . ..... ..... 000 ..... 1010111 @r_vm +vrol_vx 010101 . ..... ..... 100 ..... 1010111 @r_vm +vror_vv 010100 . ..... ..... 000 ..... 1010111 @r_vm +vror_vx 010100 . ..... ..... 100 ..... 1010111 @r_vm +vror_vi 01010. . ..... ..... 011 ..... 1010111 @r2_zimm6 +vbrev8_v 010010 . ..... 01000 010 ..... 1010111 @r2_vm +vrev8_v 010010 . ..... 01001 010 ..... 1010111 @r2_vm +vandn_vv 000001 . ..... ..... 000 ..... 1010111 @r_vm +vandn_vx 000001 . ..... ..... 100 ..... 1010111 @r_vm +vbrev_v 010010 . ..... 01010 010 ..... 1010111 @r2_vm +vclz_v 010010 . ..... 01100 010 ..... 1010111 @r2_vm +vctz_v 010010 . ..... 01101 010 ..... 1010111 @r2_vm +vcpop_v 010010 . ..... 01110 010 ..... 1010111 @r2_vm +vwsll_vv 110101 . ..... ..... 000 ..... 1010111 @r_vm +vwsll_vx 110101 . ..... ..... 100 ..... 1010111 @r_vm +vwsll_vi 110101 . ..... ..... 011 ..... 1010111 @r_vmtarget/riscv/insn_trans/trans_rvv.c.inc +3
-0@@ -1368,6 +1368,7 @@ GEN_OPIVX_GVEC_TRANS(vrsub_vx, rsubs) typedef enum { IMM_ZX, /* Zero-extended */ IMM_SX, /* Sign-extended */ + IMM_ZIMM6, /* Truncate to 6 bits */ IMM_TRUNC_SEW, /* Truncate to log(SEW) bits */ IMM_TRUNC_2SEW, /* Truncate to log(2*SEW) bits */ } imm_mode_t;@@ -1383,6 +1384,8 @@ static int64_t extract_imm(DisasContext *s, uint32_t imm, imm_mode_t imm_mode) return extract64(imm, 0, s->sew + 3); case IMM_TRUNC_2SEW: return extract64(imm, 0, s->sew + 4); + case IMM_ZIMM6: + return extract64(imm, 0, 6); default: g_assert_not_reached(); }target/riscv/insn_trans/trans_rvvk.c.inc +164
-0@@ -86,3 +86,167 @@ static bool vclmul_vx_check(DisasContext *s, arg_rmrr *a) GEN_VX_MASKED_TRANS(vclmul_vx, vclmul_vx_check) GEN_VX_MASKED_TRANS(vclmulh_vx, vclmul_vx_check) + +/* + * Zvbb + */ + +#define GEN_OPIVI_GVEC_TRANS_CHECK(NAME, IMM_MODE, OPIVX, SUF, CHECK) \ + static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ + { \ + if (CHECK(s, a)) { \ + static gen_helper_opivx *const fns[4] = { \ + gen_helper_##OPIVX##_b, \ + gen_helper_##OPIVX##_h, \ + gen_helper_##OPIVX##_w, \ + gen_helper_##OPIVX##_d, \ + }; \ + return do_opivi_gvec(s, a, tcg_gen_gvec_##SUF, fns[s->sew], \ + IMM_MODE); \ + } \ + return false; \ + } + +#define GEN_OPIVV_GVEC_TRANS_CHECK(NAME, SUF, CHECK) \ + static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ + { \ + if (CHECK(s, a)) { \ + static gen_helper_gvec_4_ptr *const fns[4] = { \ + gen_helper_##NAME##_b, \ + gen_helper_##NAME##_h, \ + gen_helper_##NAME##_w, \ + gen_helper_##NAME##_d, \ + }; \ + return do_opivv_gvec(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ + } \ + return false; \ + } + +#define GEN_OPIVX_GVEC_SHIFT_TRANS_CHECK(NAME, SUF, CHECK) \ + static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ + { \ + if (CHECK(s, a)) { \ + static gen_helper_opivx *const fns[4] = { \ + gen_helper_##NAME##_b, \ + gen_helper_##NAME##_h, \ + gen_helper_##NAME##_w, \ + gen_helper_##NAME##_d, \ + }; \ + return do_opivx_gvec_shift(s, a, tcg_gen_gvec_##SUF, \ + fns[s->sew]); \ + } \ + return false; \ + } + +static bool zvbb_vv_check(DisasContext *s, arg_rmrr *a) +{ + return opivv_check(s, a) && s->cfg_ptr->ext_zvbb == true; +} + +static bool zvbb_vx_check(DisasContext *s, arg_rmrr *a) +{ + return opivx_check(s, a) && s->cfg_ptr->ext_zvbb == true; +} + +/* vrol.v[vx] */ +GEN_OPIVV_GVEC_TRANS_CHECK(vrol_vv, rotlv, zvbb_vv_check) +GEN_OPIVX_GVEC_SHIFT_TRANS_CHECK(vrol_vx, rotls, zvbb_vx_check) + +/* vror.v[vxi] */ +GEN_OPIVV_GVEC_TRANS_CHECK(vror_vv, rotrv, zvbb_vv_check) +GEN_OPIVX_GVEC_SHIFT_TRANS_CHECK(vror_vx, rotrs, zvbb_vx_check) +GEN_OPIVI_GVEC_TRANS_CHECK(vror_vi, IMM_ZIMM6, vror_vx, rotri, zvbb_vx_check) + +#define GEN_OPIVX_GVEC_TRANS_CHECK(NAME, SUF, CHECK) \ + static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ + { \ + if (CHECK(s, a)) { \ + static gen_helper_opivx *const fns[4] = { \ + gen_helper_##NAME##_b, \ + gen_helper_##NAME##_h, \ + gen_helper_##NAME##_w, \ + gen_helper_##NAME##_d, \ + }; \ + return do_opivx_gvec(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ + } \ + return false; \ + } + +/* vandn.v[vx] */ +GEN_OPIVV_GVEC_TRANS_CHECK(vandn_vv, andc, zvbb_vv_check) +GEN_OPIVX_GVEC_TRANS_CHECK(vandn_vx, andcs, zvbb_vx_check) + +#define GEN_OPIV_TRANS(NAME, CHECK) \ + static bool trans_##NAME(DisasContext *s, arg_rmr *a) \ + { \ + if (CHECK(s, a)) { \ + uint32_t data = 0; \ + static gen_helper_gvec_3_ptr *const fns[4] = { \ + gen_helper_##NAME##_b, \ + gen_helper_##NAME##_h, \ + gen_helper_##NAME##_w, \ + gen_helper_##NAME##_d, \ + }; \ + TCGLabel *over = gen_new_label(); \ + tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ + \ + data = FIELD_DP32(data, VDATA, VM, a->vm); \ + data = FIELD_DP32(data, VDATA, LMUL, s->lmul); \ + data = FIELD_DP32(data, VDATA, VTA, s->vta); \ + data = FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s); \ + data = FIELD_DP32(data, VDATA, VMA, s->vma); \ + tcg_gen_gvec_3_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0), \ + vreg_ofs(s, a->rs2), cpu_env, \ + s->cfg_ptr->vlen / 8, s->cfg_ptr->vlen / 8, \ + data, fns[s->sew]); \ + mark_vs_dirty(s); \ + gen_set_label(over); \ + return true; \ + } \ + return false; \ + } + +static bool zvbb_opiv_check(DisasContext *s, arg_rmr *a) +{ + return s->cfg_ptr->ext_zvbb == true && + require_rvv(s) && + vext_check_isa_ill(s) && + vext_check_ss(s, a->rd, a->rs2, a->vm); +} + +GEN_OPIV_TRANS(vbrev8_v, zvbb_opiv_check) +GEN_OPIV_TRANS(vrev8_v, zvbb_opiv_check) +GEN_OPIV_TRANS(vbrev_v, zvbb_opiv_check) +GEN_OPIV_TRANS(vclz_v, zvbb_opiv_check) +GEN_OPIV_TRANS(vctz_v, zvbb_opiv_check) +GEN_OPIV_TRANS(vcpop_v, zvbb_opiv_check) + +static bool vwsll_vv_check(DisasContext *s, arg_rmrr *a) +{ + return s->cfg_ptr->ext_zvbb && opivv_widen_check(s, a); +} + +static bool vwsll_vx_check(DisasContext *s, arg_rmrr *a) +{ + return s->cfg_ptr->ext_zvbb && opivx_widen_check(s, a); +} + +/* OPIVI without GVEC IR */ +#define GEN_OPIVI_WIDEN_TRANS(NAME, IMM_MODE, OPIVX, CHECK) \ + static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ + { \ + if (CHECK(s, a)) { \ + static gen_helper_opivx *const fns[3] = { \ + gen_helper_##OPIVX##_b, \ + gen_helper_##OPIVX##_h, \ + gen_helper_##OPIVX##_w, \ + }; \ + return opivi_trans(a->rd, a->rs1, a->rs2, a->vm, fns[s->sew], s, \ + IMM_MODE); \ + } \ + return false; \ + } + +GEN_OPIVV_WIDEN_TRANS(vwsll_vv, vwsll_vv_check) +GEN_OPIVX_WIDEN_TRANS(vwsll_vx, vwsll_vx_check) +GEN_OPIVI_WIDEN_TRANS(vwsll_vi, IMM_ZX, vwsll_vx, vwsll_vx_check)target/riscv/vcrypto_helper.c +138
-0@@ -20,6 +20,7 @@#include "qemu/osdep.h" #include "qemu/host-utils.h" #include "qemu/bitops.h" +#include "qemu/bswap.h" #include "cpu.h" #include "exec/memop.h" #include "exec/exec-all.h" @@ -57,3 +58,140 @@ RVVCALL(OPIVV2, vclmulh_vv, OP_UUU_D, H8, H8, H8, clmulh64) GEN_VEXT_VV(vclmulh_vv, 8) RVVCALL(OPIVX2, vclmulh_vx, OP_UUU_D, H8, H8, clmulh64) GEN_VEXT_VX(vclmulh_vx, 8) + +RVVCALL(OPIVV2, vror_vv_b, OP_UUU_B, H1, H1, H1, ror8) +RVVCALL(OPIVV2, vror_vv_h, OP_UUU_H, H2, H2, H2, ror16) +RVVCALL(OPIVV2, vror_vv_w, OP_UUU_W, H4, H4, H4, ror32) +RVVCALL(OPIVV2, vror_vv_d, OP_UUU_D, H8, H8, H8, ror64) +GEN_VEXT_VV(vror_vv_b, 1) +GEN_VEXT_VV(vror_vv_h, 2) +GEN_VEXT_VV(vror_vv_w, 4) +GEN_VEXT_VV(vror_vv_d, 8) + +RVVCALL(OPIVX2, vror_vx_b, OP_UUU_B, H1, H1, ror8) +RVVCALL(OPIVX2, vror_vx_h, OP_UUU_H, H2, H2, ror16) +RVVCALL(OPIVX2, vror_vx_w, OP_UUU_W, H4, H4, ror32) +RVVCALL(OPIVX2, vror_vx_d, OP_UUU_D, H8, H8, ror64) +GEN_VEXT_VX(vror_vx_b, 1) +GEN_VEXT_VX(vror_vx_h, 2) +GEN_VEXT_VX(vror_vx_w, 4) +GEN_VEXT_VX(vror_vx_d, 8) + +RVVCALL(OPIVV2, vrol_vv_b, OP_UUU_B, H1, H1, H1, rol8) +RVVCALL(OPIVV2, vrol_vv_h, OP_UUU_H, H2, H2, H2, rol16) +RVVCALL(OPIVV2, vrol_vv_w, OP_UUU_W, H4, H4, H4, rol32) +RVVCALL(OPIVV2, vrol_vv_d, OP_UUU_D, H8, H8, H8, rol64) +GEN_VEXT_VV(vrol_vv_b, 1) +GEN_VEXT_VV(vrol_vv_h, 2) +GEN_VEXT_VV(vrol_vv_w, 4) +GEN_VEXT_VV(vrol_vv_d, 8) + +RVVCALL(OPIVX2, vrol_vx_b, OP_UUU_B, H1, H1, rol8) +RVVCALL(OPIVX2, vrol_vx_h, OP_UUU_H, H2, H2, rol16) +RVVCALL(OPIVX2, vrol_vx_w, OP_UUU_W, H4, H4, rol32) +RVVCALL(OPIVX2, vrol_vx_d, OP_UUU_D, H8, H8, rol64) +GEN_VEXT_VX(vrol_vx_b, 1) +GEN_VEXT_VX(vrol_vx_h, 2) +GEN_VEXT_VX(vrol_vx_w, 4) +GEN_VEXT_VX(vrol_vx_d, 8) + +static uint64_t brev8 (uint64_t val) +{ + val = ((val &0x5555555555555555ull ) <<1 ) | + ((val &0xAAAAAAAAAAAAAAAAull ) >>1 ); + val = ((val &0x3333333333333333ull ) <<2 ) | + ((val &0xCCCCCCCCCCCCCCCCull ) >>2 ); + val = ((val &0x0F0F0F0F0F0F0F0Full ) <<4 ) | + ((val &0xF0F0F0F0F0F0F0F0ull ) >>4 ); + +return val; +} + +RVVCALL(OPIVV1, vbrev8_v_b, OP_UU_B, H1, H1, brev8) +RVVCALL(OPIVV1, vbrev8_v_h, OP_UU_H, H2, H2, brev8) +RVVCALL(OPIVV1, vbrev8_v_w, OP_UU_W, H4, H4, brev8) +RVVCALL(OPIVV1, vbrev8_v_d, OP_UU_D, H8, H8, brev8) +GEN_VEXT_V(vbrev8_v_b, 1) +GEN_VEXT_V(vbrev8_v_h, 2) +GEN_VEXT_V(vbrev8_v_w, 4) +GEN_VEXT_V(vbrev8_v_d, 8) + +#define DO_IDENTITY (a) (a) +RVVCALL(OPIVV1, vrev8_v_b, OP_UU_B, H1, H1, DO_IDENTITY) +RVVCALL(OPIVV1, vrev8_v_h, OP_UU_H, H2, H2, bswap16) +RVVCALL(OPIVV1, vrev8_v_w, OP_UU_W, H4, H4, bswap32) +RVVCALL(OPIVV1, vrev8_v_d, OP_UU_D, H8, H8, bswap64) +GEN_VEXT_V(vrev8_v_b, 1) +GEN_VEXT_V(vrev8_v_h, 2) +GEN_VEXT_V(vrev8_v_w, 4) +GEN_VEXT_V(vrev8_v_d, 8) + +#define DO_ANDN (a, b) ((a) & ~(b)) +RVVCALL(OPIVV2, vandn_vv_b, OP_UUU_B, H1, H1, H1, DO_ANDN) +RVVCALL(OPIVV2, vandn_vv_h, OP_UUU_H, H2, H2, H2, DO_ANDN) +RVVCALL(OPIVV2, vandn_vv_w, OP_UUU_W, H4, H4, H4, DO_ANDN) +RVVCALL(OPIVV2, vandn_vv_d, OP_UUU_D, H8, H8, H8, DO_ANDN) +GEN_VEXT_VV(vandn_vv_b, 1) +GEN_VEXT_VV(vandn_vv_h, 2) +GEN_VEXT_VV(vandn_vv_w, 4) +GEN_VEXT_VV(vandn_vv_d, 8) + +RVVCALL(OPIVX2, vandn_vx_b, OP_UUU_B, H1, H1, DO_ANDN) +RVVCALL(OPIVX2, vandn_vx_h, OP_UUU_H, H2, H2, DO_ANDN) +RVVCALL(OPIVX2, vandn_vx_w, OP_UUU_W, H4, H4, DO_ANDN) +RVVCALL(OPIVX2, vandn_vx_d, OP_UUU_D, H8, H8, DO_ANDN) +GEN_VEXT_VX(vandn_vx_b, 1) +GEN_VEXT_VX(vandn_vx_h, 2) +GEN_VEXT_VX(vandn_vx_w, 4) +GEN_VEXT_VX(vandn_vx_d, 8) + +RVVCALL(OPIVV1, vbrev_v_b, OP_UU_B, H1, H1, revbit8) +RVVCALL(OPIVV1, vbrev_v_h, OP_UU_H, H2, H2, revbit16) +RVVCALL(OPIVV1, vbrev_v_w, OP_UU_W, H4, H4, revbit32) +RVVCALL(OPIVV1, vbrev_v_d, OP_UU_D, H8, H8, revbit64) +GEN_VEXT_V(vbrev_v_b, 1) +GEN_VEXT_V(vbrev_v_h, 2) +GEN_VEXT_V(vbrev_v_w, 4) +GEN_VEXT_V(vbrev_v_d, 8) + +RVVCALL(OPIVV1, vclz_v_b, OP_UU_B, H1, H1, clz8) +RVVCALL(OPIVV1, vclz_v_h, OP_UU_H, H2, H2, clz16) +RVVCALL(OPIVV1, vclz_v_w, OP_UU_W, H4, H4, clz32) +RVVCALL(OPIVV1, vclz_v_d, OP_UU_D, H8, H8, clz64) +GEN_VEXT_V(vclz_v_b, 1) +GEN_VEXT_V(vclz_v_h, 2) +GEN_VEXT_V(vclz_v_w, 4) +GEN_VEXT_V(vclz_v_d, 8) + +RVVCALL(OPIVV1, vctz_v_b, OP_UU_B, H1, H1, ctz8) +RVVCALL(OPIVV1, vctz_v_h, OP_UU_H, H2, H2, ctz16) +RVVCALL(OPIVV1, vctz_v_w, OP_UU_W, H4, H4, ctz32) +RVVCALL(OPIVV1, vctz_v_d, OP_UU_D, H8, H8, ctz64) +GEN_VEXT_V(vctz_v_b, 1) +GEN_VEXT_V(vctz_v_h, 2) +GEN_VEXT_V(vctz_v_w, 4) +GEN_VEXT_V(vctz_v_d, 8) + +RVVCALL(OPIVV1, vcpop_v_b, OP_UU_B, H1, H1, ctpop8) +RVVCALL(OPIVV1, vcpop_v_h, OP_UU_H, H2, H2, ctpop16) +RVVCALL(OPIVV1, vcpop_v_w, OP_UU_W, H4, H4, ctpop32) +RVVCALL(OPIVV1, vcpop_v_d, OP_UU_D, H8, H8, ctpop64) +GEN_VEXT_V(vcpop_v_b, 1) +GEN_VEXT_V(vcpop_v_h, 2) +GEN_VEXT_V(vcpop_v_w, 4) +GEN_VEXT_V(vcpop_v_d, 8) + +#define DO_SLL (N ,M ) (N << (M & (sizeof(N) * 8 - 1))) +RVVCALL(OPIVV2, vwsll_vv_b, WOP_UUU_B, H2, H1, H1, DO_SLL) +RVVCALL(OPIVV2, vwsll_vv_h, WOP_UUU_H, H4, H2, H2, DO_SLL) +RVVCALL(OPIVV2, vwsll_vv_w, WOP_UUU_W, H8, H4, H4, DO_SLL) +GEN_VEXT_VV(vwsll_vv_b, 2) +GEN_VEXT_VV(vwsll_vv_h, 4) +GEN_VEXT_VV(vwsll_vv_w, 8) + +RVVCALL(OPIVX2, vwsll_vx_b, WOP_UUU_B, H2, H1, DO_SLL) +RVVCALL(OPIVX2, vwsll_vx_h, WOP_UUU_H, H4, H2, DO_SLL) +RVVCALL(OPIVX2, vwsll_vx_w, WOP_UUU_W, H8, H4, DO_SLL) +GEN_VEXT_VX(vwsll_vx_b, 2) +GEN_VEXT_VX(vwsll_vx_h, 4) +GEN_VEXT_VX (vwsll_vx_w,8 )
PATCH v3 12/19 target/riscv: Add Zvkned ISA extension support
In reply to Lawrence HunterFrom: Nazar Kazakov <[email protected]>
This commit adds support for the Zvkned vector-crypto extension, which consists of the following instructions:
* vaesef.[vv,vs]
* vaesdf.[vv,vs]
* vaesdm.[vv,vs]
* vaesz.vs
* vaesem.[vv,vs]
* vaeskf1.vi
* vaeskf2.vi
Translation functions are defined in target/riscv/insn_trans/trans_rvvk.c.inc and helpers are defined in target/riscv/vcrypto_helper.c.
--- target/riscv/cpu.c | 6 +- target/riscv/cpu.h | 1 + target/riscv/helper.h | 13 + target/riscv/insn32.decode | 14 ++ target/riscv/insn_trans/trans_rvvk.c.inc | 163 ++++++++++++ target/riscv/op_helper.c | 6 + target/riscv/vcrypto_helper.c | 308 +++++++++++++++++++++++ 7 files changed, 509 insertions(+), 2 deletions(-)
target/riscv/cpu.c +4 -2
@@ -111,6 +111,7 @@ static const struct isa_ext_data isa_edata_arr[] = {ISA_EXT_DATA_ENTRY (zvfhmin, true,PRIV_VERSION_1_12_0 , ext_zvfhmin),ISA_EXT_DATA_ENTRY (zvbb, true,PRIV_VERSION_1_12_0 , ext_zvbb),ISA_EXT_DATA_ENTRY (zvbc, true,PRIV_VERSION_1_12_0 , ext_zvbc), +ISA_EXT_DATA_ENTRY (zvkned, true,PRIV_VERSION_1_12_0 , ext_zvkned),ISA_EXT_DATA_ENTRY (zhinx, true,PRIV_VERSION_1_12_0 , ext_zhinx),ISA_EXT_DATA_ENTRY (zhinxmin, true,PRIV_VERSION_1_12_0 , ext_zhinxmin), ISA_EXT_DATA_ENTRY(smaia, true, PRIV_VERSION_1_12_0, ext_smaia),
@@ -1217,8 +1218,9 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)* In principle Zve*x would also suffice here, were they supported * in qemu */ -if (cpu->cfg.ext_zvbb && !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || - cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { +if ((cpu->cfg.ext_zvbb || cpu->cfg.ext_zvkned) && + !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || + cpu->cfg.ext_v)) {error_setg (errp,"Vector crypto extensions require V or Zve* extensions" );return ;
target/riscv/cpu.h +1 -0
@@ -472,6 +472,7 @@ struct RISCVCPUConfig {bool ext_zve64d;bool ext_zvbb;bool ext_zvbc; +bool ext_zvkned;bool ext_zmmul;bool ext_zvfh;bool ext_zvfhmin;
target/riscv/helper.h +13 -0
@@ -1,5 +1,6 @@/* Exceptions */ DEF_HELPER_2(raise_exception, noreturn, env, i32) +DEF_HELPER_2 (restore_cpu_and_raise_exception, noreturn, env, i32) /* Floating Point - rounding mode */DEF_HELPER_FLAGS_2 (set_rounding_mode,TCG_CALL_NO_WG , void, env, i32)
@@ -1210,3 +1211,15 @@ DEF_HELPER_6(vandn_vx_b, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vandn_vx_h, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vandn_vx_w, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vandn_vx_d, void, ptr, ptr, tl, ptr, env, i32) + +DEF_HELPER_4(vaesef_vv, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesef_vs, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesdf_vv, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesdf_vs, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesem_vv, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesem_vs, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesdm_vv, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesdm_vs, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesz_vs, void, ptr, ptr, env, i32) +DEF_HELPER_5(vaeskf1_vi, void, ptr, ptr, i32, env, i32) +DEF_HELPER_5 (vaeskf2_vi, void, ptr, ptr, i32, env, i32)
target/riscv/insn32.decode +14 -0
@@ -75,6 +75,7 @@ @r_rm ....... ..... ..... ... ..... ....... %rs2 %rs1 %rm %rd @r2_rm ....... ..... ..... ... ..... ....... %rs1 %rm %rd @r2 ....... ..... ..... ... ..... ....... &r2 %rs1 %rd +@r2_vm_1 ...... . ..... ..... ... ..... ....... &rmr vm=1 %rs2 %rd @r2_nfvm ... ... vm:1 ..... ..... ... ..... ....... &r2nfvm %nf %rs1 %rd @r2_vm ...... vm:1 ..... ..... ... ..... ....... &rmr %rs2 %rd @r1_vm ...... vm:1 ..... ..... ... ..... ....... %rd
@@ -934,3 +935,16 @@ vcpop_v 010010 . ..... 01110 010 ..... 1010111 @r2_vm vwsll_vv 110101 . ..... ..... 000 ..... 1010111 @r_vm vwsll_vx 110101 . ..... ..... 100 ..... 1010111 @r_vm vwsll_vi 110101 . ..... ..... 011 ..... 1010111 @r_vm + +# *** Zvkned vector crypto extension *** +vaesef_vv 101000 1 ..... 00011 010 ..... 1110111 @r2_vm_1 +vaesef_vs 101001 1 ..... 00011 010 ..... 1110111 @r2_vm_1 +vaesdf_vv 101000 1 ..... 00001 010 ..... 1110111 @r2_vm_1 +vaesdf_vs 101001 1 ..... 00001 010 ..... 1110111 @r2_vm_1 +vaesem_vv 101000 1 ..... 00010 010 ..... 1110111 @r2_vm_1 +vaesem_vs 101001 1 ..... 00010 010 ..... 1110111 @r2_vm_1 +vaesdm_vv 101000 1 ..... 00000 010 ..... 1110111 @r2_vm_1 +vaesdm_vs 101001 1 ..... 00000 010 ..... 1110111 @r2_vm_1 +vaesz_vs 101001 1 ..... 00111 010 ..... 1110111 @r2_vm_1 +vaeskf1_vi 100010 1 ..... ..... 010 ..... 1110111 @r_vm_1 +vaeskf2_vi 101010 1 ..... ..... 010 ..... 1110111 @r_vm_1
target/riscv/insn_trans/trans_rvvk.c.inc +163 -0
166 lines not on this page · raw message
target/riscv/op_helper.c +6 -0
@@ -38,6 +38,12 @@ void helper_raise_exception(CPURISCVState *env, uint32_t exception)riscv_raise_exception (env, exception,0 ); } +void helper_restore_cpu_and_raise_exception (CPURISCVState *env, +uint32_t exception) +{ +riscv_raise_exception (env, exception,GETPC ()); +} +target_ulong helper_csrr (CPURISCVState *env,int csr) {/*
target/riscv/vcrypto_helper.c +308 -0
317 lines not on this page · raw message
Signature · 1 line
2.40.1
PATCH v3 13/19 target/riscv: Add Zvknh ISA extension support
In reply to Lawrence HunterFrom: Kiran Ostrolenk <[email protected]>
This commit adds support for the Zvknh vector-crypto extension, which consists of the following instructions:
* vsha2ms.vv
* vsha2c[hl].vv
Translation functions are defined in target/riscv/insn_trans/trans_rvvk.c.inc and helpers are defined in target/riscv/vcrypto_helper.c.
--- target/riscv/cpu.c | 10 +- target/riscv/cpu.h | 2 + target/riscv/helper.h | 4 + target/riscv/insn32.decode | 5 + target/riscv/insn_trans/trans_rvvk.c.inc | 70 ++++++++ target/riscv/vcrypto_helper.c | 214 +++++++++++++++++++++++ 6 files changed, 302 insertions(+), 3 deletions(-)
target/riscv/cpu.c +7 -3
@@ -112,6 +112,8 @@ static const struct isa_ext_data isa_edata_arr[] = {ISA_EXT_DATA_ENTRY (zvbb, true,PRIV_VERSION_1_12_0 , ext_zvbb),ISA_EXT_DATA_ENTRY (zvbc, true,PRIV_VERSION_1_12_0 , ext_zvbc),ISA_EXT_DATA_ENTRY (zvkned, true,PRIV_VERSION_1_12_0 , ext_zvkned), +ISA_EXT_DATA_ENTRY (zvknha, true,PRIV_VERSION_1_12_0 , ext_zvknha), +ISA_EXT_DATA_ENTRY (zvknhb, true,PRIV_VERSION_1_12_0 , ext_zvknhb),ISA_EXT_DATA_ENTRY (zhinx, true,PRIV_VERSION_1_12_0 , ext_zhinx),ISA_EXT_DATA_ENTRY (zhinxmin, true,PRIV_VERSION_1_12_0 , ext_zhinxmin), ISA_EXT_DATA_ENTRY(smaia, true, PRIV_VERSION_1_12_0, ext_smaia),
@@ -1218,7 +1220,7 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)* In principle Zve*x would also suffice here, were they supported * in qemu */ -if ((cpu->cfg.ext_zvbb || cpu->cfg.ext_zvkned) && +if ((cpu->cfg.ext_zvbb || cpu->cfg.ext_zvkned || cpu->cfg.ext_zvknha) && !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { error_setg(errp,
@@ -1226,9 +1228,11 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)return ; } -if (cpu->cfg.ext_zvbc && +if ((cpu->cfg.ext_zvbc || cpu->cfg.ext_zvknhb) && !(cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { -error_setg (errp,"Zvbc extension requires V or Zve64{f,d} extensions" ); +error_setg ( + errp, +"Zvbc and Zvknhb extensions require V or Zve64{f,d} extensions" );return ; }
target/riscv/cpu.h +2 -0
@@ -473,6 +473,8 @@ struct RISCVCPUConfig {bool ext_zvbb;bool ext_zvbc;bool ext_zvkned; +bool ext_zvknha; +bool ext_zvknhb;bool ext_zmmul;bool ext_zvfh;bool ext_zvfhmin;
target/riscv/helper.h +4 -0
@@ -1223,3 +1223,7 @@ DEF_HELPER_4(vaesdm_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesz_vs, void, ptr, ptr, env, i32) DEF_HELPER_5(vaeskf1_vi, void, ptr, ptr, i32, env, i32) DEF_HELPER_5(vaeskf2_vi, void, ptr, ptr, i32, env, i32) + +DEF_HELPER_5(vsha2ms_vv, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vsha2ch_vv, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5 (vsha2cl_vv, void, ptr, ptr, ptr, env, i32)
target/riscv/insn32.decode +5 -0
@@ -948,3 +948,8 @@ vaesdm_vs 101001 1 ..... 00000 010 ..... 1110111 @r2_vm_1 vaesz_vs 101001 1 ..... 00111 010 ..... 1110111 @r2_vm_1 vaeskf1_vi 100010 1 ..... ..... 010 ..... 1110111 @r_vm_1 vaeskf2_vi 101010 1 ..... ..... 010 ..... 1110111 @r_vm_1 + +# *** Zvknh vector crypto extension *** +vsha2ms_vv 101101 1 ..... ..... 010 ..... 1110111 @r_vm_1 +vsha2ch_vv 101110 1 ..... ..... 010 ..... 1110111 @r_vm_1 +vsha2cl_vv 101111 1 ..... ..... 010 ..... 1110111 @r_vm_1
target/riscv/insn_trans/trans_rvvk.c.inc +70 -0
73 lines not on this page · raw message
target/riscv/vcrypto_helper.c +214 -0
217 lines not on this page · raw message
Signature · 1 line
2.40.1
PATCH v3 14/19 target/riscv: Add Zvksh ISA extension support
In reply to Lawrence HunterThis commit adds support for the Zvksh vector-crypto extension, which consists of the following instructions:
* vsm3me.vv
* vsm3c.vi
Translation functions are defined in target/riscv/insn_trans/trans_rvvk.c.inc and helpers are defined in target/riscv/vcrypto_helper.c.
--- target/riscv/cpu.c | 4 +- target/riscv/cpu.h | 1 + target/riscv/helper.h | 3 + target/riscv/insn32.decode | 4 + target/riscv/insn_trans/trans_rvvk.c.inc | 32 ++++++ target/riscv/vcrypto_helper.c | 134 +++++++++++++++++++++++ 6 files changed, 177 insertions(+), 1 deletion(-)
target/riscv/cpu.c +3 -1
@@ -114,6 +114,7 @@ static const struct isa_ext_data isa_edata_arr[] = {ISA_EXT_DATA_ENTRY (zvkned, true,PRIV_VERSION_1_12_0 , ext_zvkned),ISA_EXT_DATA_ENTRY (zvknha, true,PRIV_VERSION_1_12_0 , ext_zvknha),ISA_EXT_DATA_ENTRY (zvknhb, true,PRIV_VERSION_1_12_0 , ext_zvknhb), +ISA_EXT_DATA_ENTRY (zvksh, true,PRIV_VERSION_1_12_0 , ext_zvksh),ISA_EXT_DATA_ENTRY (zhinx, true,PRIV_VERSION_1_12_0 , ext_zhinx),ISA_EXT_DATA_ENTRY (zhinxmin, true,PRIV_VERSION_1_12_0 , ext_zhinxmin), ISA_EXT_DATA_ENTRY(smaia, true, PRIV_VERSION_1_12_0, ext_smaia),
@@ -1220,7 +1221,8 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)* In principle Zve*x would also suffice here, were they supported * in qemu */ -if ((cpu->cfg.ext_zvbb || cpu->cfg.ext_zvkned || cpu->cfg.ext_zvknha) && +if ((cpu->cfg.ext_zvbb || cpu->cfg.ext_zvkned || cpu->cfg.ext_zvknha || + cpu->cfg.ext_zvksh) && !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { error_setg(errp,
target/riscv/cpu.h +1 -0
@@ -475,6 +475,7 @@ struct RISCVCPUConfig {bool ext_zvkned;bool ext_zvknha;bool ext_zvknhb; +bool ext_zvksh;bool ext_zmmul;bool ext_zvfh;bool ext_zvfhmin;
target/riscv/helper.h +3 -0
@@ -1227,3 +1227,6 @@ DEF_HELPER_5(vaeskf2_vi, void, ptr, ptr, i32, env, i32) DEF_HELPER_5(vsha2ms_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vsha2ch_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vsha2cl_vv, void, ptr, ptr, ptr, env, i32) + +DEF_HELPER_5(vsm3me_vv, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5 (vsm3c_vi, void, ptr, ptr, i32, env, i32)
target/riscv/insn32.decode +4 -0
@@ -953,3 +953,7 @@ vaeskf2_vi 101010 1 ..... ..... 010 ..... 1110111 @r_vm_1 vsha2ms_vv 101101 1 ..... ..... 010 ..... 1110111 @r_vm_1 vsha2ch_vv 101110 1 ..... ..... 010 ..... 1110111 @r_vm_1 vsha2cl_vv 101111 1 ..... ..... 010 ..... 1110111 @r_vm_1 + +# *** Zvksh vector crypto extension *** +vsm3me_vv 100000 1 ..... ..... 010 ..... 1110111 @r_vm_1 +vsm3c_vi 101011 1 ..... ..... 010 ..... 1110111 @r_vm_1
target/riscv/insn_trans/trans_rvvk.c.inc +32 -0
@@ -483,3 +483,35 @@ static bool vsha_check(DisasContext *s, arg_rmrr *a) GEN_VV_UNMASKED_TRANS(vsha2ms_vv, vsha_check, ZVKNH_EGS) GEN_VV_UNMASKED_TRANS(vsha2cl_vv, vsha_check, ZVKNH_EGS) GEN_VV_UNMASKED_TRANS(vsha2ch_vv, vsha_check, ZVKNH_EGS) + +/* + * Zvksh + */ + +#define ZVKSH_EGS 8 + +static inline bool vsm3_check(DisasContext *s, arg_rmrr *a) +{ + int egw_bytes = ZVKSH_EGS << s->sew; + int mult = 1 << MAX(s->lmul, 0); + return s->cfg_ptr->ext_zvksh == true && + require_rvv(s) && + vext_check_isa_ill(s) && + !is_overlapped(a->rd, mult, a->rs2, mult) && + MAXSZ(s) >= egw_bytes && + s->vstart % ZVKSH_EGS == 0 && + s->sew == MO_32; +} + +static inline bool vsm3me_check(DisasContext *s, arg_rmrr *a) +{ + return vsm3_check(s, a) && vext_check_sss(s, a->rd, a->rs1, a->rs2, a->vm); +} + +static inline bool vsm3c_check(DisasContext *s, arg_rmrr *a) +{ + return vsm3_check(s, a) && vext_check_ss(s, a->rd, a->rs2, a->vm); +} + +GEN_VV_UNMASKED_TRANS(vsm3me_vv, vsm3me_check, ZVKSH_EGS) +GEN_VI_UNMASKED_TRANS(vsm3c_vi, vsm3c_check, ZVKSH_EGS)
target/riscv/vcrypto_helper.c +134 -0
137 lines not on this page · raw message
Signature · 1 line
2.40.1
PATCH v3 15/19 target/riscv: Add Zvkg ISA extension support
In reply to Lawrence HunterFrom: Nazar Kazakov <[email protected]>
This commit adds support for the Zvkg vector-crypto extension, which consists of the following instructions:
* vgmul.vv
* vghsh.vv
Translation functions are defined in target/riscv/insn_trans/trans_rvvk.c.inc and helpers are defined in target/riscv/vcrypto_helper.c.
--- target/riscv/cpu.c | 5 +- target/riscv/cpu.h | 1 + target/riscv/helper.h | 3 + target/riscv/insn32.decode | 4 ++ target/riscv/insn_trans/trans_rvvk.c.inc | 32 +++++++++++ target/riscv/vcrypto_helper.c | 72 ++++++++++++++++++++++++ 6 files changed, 115 insertions(+), 2 deletions(-)
target/riscv/cpu.c +3 -2
@@ -111,6 +111,7 @@ static const struct isa_ext_data isa_edata_arr[] = {ISA_EXT_DATA_ENTRY (zvfhmin, true,PRIV_VERSION_1_12_0 , ext_zvfhmin),ISA_EXT_DATA_ENTRY (zvbb, true,PRIV_VERSION_1_12_0 , ext_zvbb),ISA_EXT_DATA_ENTRY (zvbc, true,PRIV_VERSION_1_12_0 , ext_zvbc), +ISA_EXT_DATA_ENTRY (zvkg, true,PRIV_VERSION_1_12_0 , ext_zvkg),ISA_EXT_DATA_ENTRY (zvkned, true,PRIV_VERSION_1_12_0 , ext_zvkned),ISA_EXT_DATA_ENTRY (zvknha, true,PRIV_VERSION_1_12_0 , ext_zvknha), ISA_EXT_DATA_ENTRY(zvknhb, true, PRIV_VERSION_1_12_0, ext_zvknhb),
@@ -1221,8 +1222,8 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)* In principle Zve*x would also suffice here, were they supported * in qemu */ -if ((cpu->cfg.ext_zvbb || cpu->cfg.ext_zvkned || cpu->cfg.ext_zvknha || - cpu->cfg.ext_zvksh) && +if ((cpu->cfg.ext_zvbb || cpu->cfg.ext_zvkg || cpu->cfg.ext_zvkned || + cpu->cfg.ext_zvknha || cpu->cfg.ext_zvksh) && !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { error_setg(errp,
target/riscv/cpu.h +1 -0
@@ -472,6 +472,7 @@ struct RISCVCPUConfig {bool ext_zve64d;bool ext_zvbb;bool ext_zvbc; +bool ext_zvkg;bool ext_zvkned;bool ext_zvknha;bool ext_zvknhb;
target/riscv/helper.h +3 -0
@@ -1230,3 +1230,6 @@ DEF_HELPER_5(vsha2cl_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vsm3me_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vsm3c_vi, void, ptr, ptr, i32, env, i32) + +DEF_HELPER_5(vghsh_vv, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_4 (vgmul_vv, void, ptr, ptr, env, i32)
target/riscv/insn32.decode +4 -0
@@ -957,3 +957,7 @@ vsha2cl_vv 101111 1 ..... ..... 010 ..... 1110111 @r_vm_1 # *** Zvksh vector crypto extension *** vsm3me_vv 100000 1 ..... ..... 010 ..... 1110111 @r_vm_1 vsm3c_vi 101011 1 ..... ..... 010 ..... 1110111 @r_vm_1 + +# *** Zvkg vector crypto extension *** +vghsh_vv 101100 1 ..... ..... 010 ..... 1110111 @r_vm_1 +vgmul_vv 101000 1 ..... 10001 010 ..... 1110111 @r2_vm_1
target/riscv/insn_trans/trans_rvvk.c.inc +32 -0
@@ -515,3 +515,35 @@ static inline bool vsm3c_check(DisasContext *s, arg_rmrr *a) GEN_VV_UNMASKED_TRANS(vsm3me_vv, vsm3me_check, ZVKSH_EGS) GEN_VI_UNMASKED_TRANS(vsm3c_vi, vsm3c_check, ZVKSH_EGS) + +/* + * Zvkg + */ + +#define ZVKG_EGS 4 + +static bool vgmul_check(DisasContext *s, arg_rmr *a) +{ + int egw_bytes = ZVKG_EGS << s->sew; + return s->cfg_ptr->ext_zvkg == true && + vext_check_isa_ill(s) && + require_rvv(s) && + MAXSZ(s) >= egw_bytes && + vext_check_ss(s, a->rd, a->rs2, a->vm) && + s->vstart % ZVKG_EGS == 0 && + s->sew == MO_32; +} + +GEN_V_UNMASKED_TRANS(vgmul_vv, vgmul_check) + +static bool vghsh_check(DisasContext *s, arg_rmrr *a) +{ + int egw_bytes = ZVKG_EGS << s->sew; + return s->cfg_ptr->ext_zvkg == true && + opivv_check(s, a) && + MAXSZ(s) >= egw_bytes && + s->vstart % ZVKG_EGS == 0 && + s->sew == MO_32; +} + +GEN_VV_UNMASKED_TRANS(vghsh_vv, vghsh_check, ZVKG_EGS)
target/riscv/vcrypto_helper.c +72 -0
75 lines not on this page · raw message
Signature · 1 line
2.40.1
PATCH v3 16/19 crypto: Create sm4_subword
In reply to Lawrence HunterFrom: Max Chou <[email protected]>
Allows sharing of sm4_subword between different targets.
--- include/crypto/sm4.h | 8 ++++++++ target/arm/tcg/crypto_helper.c | 10 ++-------- 2 files changed, 10 insertions(+), 8 deletions(-)
include/crypto/sm4.h +8 -0
@@ -3,4 +3,12 @@extern const uint8_t sm4_sbox[256 ]; +static inline uint32_t sm4_subword (uint32_t word) +{ +return sm4_sbox[word &0xff ] | + sm4_sbox[(word >>8 ) &0xff ] <<8 | + sm4_sbox[(word >>16 ) &0xff ] <<16 | + sm4_sbox[(word >>24 ) &0xff ] <<24 ; +} +#endif
target/arm/tcg/crypto_helper.c +2 -8
@@ -707,10 +707,7 @@ static void do_crypto_sm4e(uint64_t *rd, uint64_t *rn, uint64_t *rm)CR_ST_WORD (d, (i +3 ) %4 ) ^CR_ST_WORD (n, i); - t = sm4_sbox[t &0xff ] | - sm4_sbox[(t >>8 ) &0xff ] <<8 | - sm4_sbox[(t >>16 ) &0xff ] <<16 | - sm4_sbox[(t >>24 ) &0xff ] <<24 ; + t =sm4_subword (t);CR_ST_WORD (d, i) ^= t ^rol32 (t,2 ) ^rol32 (t,10 ) ^rol32 (t,18 ) ^rol32 (t,24 );
@@ -744,10 +741,7 @@ static void do_crypto_sm4ekey(uint64_t *rd, uint64_t *rn, uint64_t *rm)CR_ST_WORD (d, (i +3 ) %4 ) ^CR_ST_WORD (m, i); - t = sm4_sbox[t &0xff ] | - sm4_sbox[(t >>8 ) &0xff ] <<8 | - sm4_sbox[(t >>16 ) &0xff ] <<16 | - sm4_sbox[(t >>24 ) &0xff ] <<24 ; + t =sm4_subword (t);CR_ST_WORD (d, i) ^= t ^rol32 (t,13 ) ^rol32 (t,23 ); }
Signature · 1 line
2.40.1
PATCH v3 17/19 crypto: Add SM4 constant parameter CK
In reply to Lawrence HunterFrom: Max Chou <[email protected]>
Adds sm4_ck constant for use in sm4 cryptography across different targets.
--- crypto/sm4.c | 10 ++++++++++ include/crypto/sm4.h | 1 + 2 files changed, 11 insertions(+)
crypto/sm4.c +10 -0
@@ -47,3 +47,13 @@ uint8_t const sm4_sbox[] = { 0x79, 0xee, 0x5f, 0x3e, 0xd7, 0xcb, 0x39, 0x48, }; +uint32_t const sm4_ck[] = { +0x00070e15 ,0x1c232a31 ,0x383f464d ,0x545b6269 , +0x70777e85 ,0x8c939aa1 ,0xa8afb6bd ,0xc4cbd2d9 , +0xe0e7eef5 ,0xfc030a11 ,0x181f262d ,0x343b4249 , +0x50575e65 ,0x6c737a81 ,0x888f969d ,0xa4abb2b9 , +0xc0c7ced5 ,0xdce3eaf1 ,0xf8ff060d ,0x141b2229 , +0x30373e45 ,0x4c535a61 ,0x686f767d ,0x848b9299 , +0xa0a7aeb5 ,0xbcc3cad1 ,0xd8dfe6ed ,0xf4fb0209 , +0x10171e25 ,0x2c333a41 ,0x484f565d ,0x646b7279 +};
include/crypto/sm4.h +1 -0
@@ -2,6 +2,7 @@#define QEMU_SM4_H extern const uint8_t sm4_sbox[256 ]; +extern const uint32_t sm4_ck[32 ];static inline uint32_t sm4_subword (uint32_t word) {
Signature · 1 line
2.40.1
PATCH v3 18/19 target/riscv: Add Zvksed ISA extension support
In reply to Lawrence HunterFrom: Max Chou <[email protected]>
This commit adds support for the Zvksed vector-crypto extension, which consists of the following instructions:
* vsm4k.vi
* vsm4r.[vv,vs]
Translation functions are defined in target/riscv/insn_trans/trans_rvvk.c.inc and helpers are defined in target/riscv/vcrypto_helper.c.
[[email protected]: Moved SM4 functions from crypto_helper.c to vcrypto_helper.c] [[email protected]: Added alignment checks, refactored code to use macros, and minor style changes]
--- target/riscv/cpu.c | 3 +- target/riscv/cpu.h | 1 + target/riscv/helper.h | 4 + target/riscv/insn32.decode | 5 + target/riscv/insn_trans/trans_rvvk.c.inc | 44 ++++++++ target/riscv/vcrypto_helper.c | 127 +++++++++++++++++++++++ 6 files changed, 183 insertions(+), 1 deletion(-)
target/riscv/cpu.c +2 -1
@@ -115,6 +115,7 @@ static const struct isa_ext_data isa_edata_arr[] = {ISA_EXT_DATA_ENTRY (zvkned, true,PRIV_VERSION_1_12_0 , ext_zvkned),ISA_EXT_DATA_ENTRY (zvknha, true,PRIV_VERSION_1_12_0 , ext_zvknha),ISA_EXT_DATA_ENTRY (zvknhb, true,PRIV_VERSION_1_12_0 , ext_zvknhb), +ISA_EXT_DATA_ENTRY (zvksed, true,PRIV_VERSION_1_12_0 , ext_zvksed),ISA_EXT_DATA_ENTRY (zvksh, true,PRIV_VERSION_1_12_0 , ext_zvksh),ISA_EXT_DATA_ENTRY (zhinx, true,PRIV_VERSION_1_12_0 , ext_zhinx), ISA_EXT_DATA_ENTRY(zhinxmin, true, PRIV_VERSION_1_12_0, ext_zhinxmin),
@@ -1223,7 +1224,7 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp)* in qemu */ if ((cpu->cfg.ext_zvbb || cpu->cfg.ext_zvkg || cpu->cfg.ext_zvkned || - cpu->cfg.ext_zvknha || cpu->cfg.ext_zvksh) && + cpu->cfg.ext_zvknha || cpu->cfg.ext_zvksed || cpu->cfg.ext_zvksh) && !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { error_setg(errp,
target/riscv/cpu.h +1 -0
@@ -476,6 +476,7 @@ struct RISCVCPUConfig {bool ext_zvkned;bool ext_zvknha;bool ext_zvknhb; +bool ext_zvksed;bool ext_zvksh;bool ext_zmmul;bool ext_zvfh;
target/riscv/helper.h +4 -0
@@ -1233,3 +1233,7 @@ DEF_HELPER_5(vsm3c_vi, void, ptr, ptr, i32, env, i32) DEF_HELPER_5(vghsh_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_4(vgmul_vv, void, ptr, ptr, env, i32) + +DEF_HELPER_5(vsm4k_vi, void, ptr, ptr, i32, env, i32) +DEF_HELPER_4(vsm4r_vv, void, ptr, ptr, env, i32) +DEF_HELPER_4 (vsm4r_vs, void, ptr, ptr, env, i32)
target/riscv/insn32.decode +5 -0
@@ -961,3 +961,8 @@ vsm3c_vi 101011 1 ..... ..... 010 ..... 1110111 @r_vm_1 # *** Zvkg vector crypto extension *** vghsh_vv 101100 1 ..... ..... 010 ..... 1110111 @r_vm_1 vgmul_vv 101000 1 ..... 10001 010 ..... 1110111 @r2_vm_1 + +# *** Zvksed vector crypto extension *** +vsm4k_vi 100001 1 ..... ..... 010 ..... 1110111 @r_vm_1 +vsm4r_vv 101000 1 ..... 10000 010 ..... 1110111 @r2_vm_1 +vsm4r_vs 101001 1 ..... 10000 010 ..... 1110111 @r2_vm_1
target/riscv/insn_trans/trans_rvvk.c.inc +44 -0
47 lines not on this page · raw message
target/riscv/vcrypto_helper.c +127 -0
136 lines not on this page · raw message
Signature · 1 line
2.40.1
PATCH v3 19/19 target/riscv: Expose Zvk* and Zvb[b,c] cpu properties
In reply to Lawrence HunterFrom: Nazar Kazakov <[email protected]>
Exposes earlier CPU flags allowing the use of the vector cryptography extensions.
--- target/riscv/cpu.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
target/riscv/cpu.c +10 -0
@@ -1485,6 +1485,16 @@ static Property riscv_cpu_extensions[] = {DEFINE_PROP_BOOL ("x-zvfh" ,RISCVCPU , cfg.ext_zvfh, false),DEFINE_PROP_BOOL ("x-zvfhmin" ,RISCVCPU , cfg.ext_zvfhmin, false), +/* Vector cryptography extensions */ +DEFINE_PROP_BOOL ("x-zvbb" ,RISCVCPU , cfg.ext_zvbb, false), +DEFINE_PROP_BOOL ("x-zvbc" ,RISCVCPU , cfg.ext_zvbc, false), +DEFINE_PROP_BOOL ("x-zvkg" ,RISCVCPU , cfg.ext_zvkg, false), +DEFINE_PROP_BOOL ("x-zvkned" ,RISCVCPU , cfg.ext_zvkned, false), +DEFINE_PROP_BOOL ("x-zvknha" ,RISCVCPU , cfg.ext_zvknha, false), +DEFINE_PROP_BOOL ("x-zvknhb" ,RISCVCPU , cfg.ext_zvknhb, false), +DEFINE_PROP_BOOL ("x-zvksed" ,RISCVCPU , cfg.ext_zvksed, false), +DEFINE_PROP_BOOL ("x-zvksh" ,RISCVCPU , cfg.ext_zvksh, false), + DEFINE_PROP_END_OF_LIST(), };
Signature · 1 line
2.40.1
Re: [PATCH v3 19/19] target/riscv: Expose Zvk* and Zvb[b, c] cpu properties
Quoted 6 lines · Lawrence Hunter
From: Nazar Kazakov <[email protected]>
Exposes earlier CPU flags allowing the use of the vector cryptography extensions.
Signed-off-by Nazar Kazakov <[email protected]>---
Weiwei Li
Quoted 24 lines
target/riscv/cpu.c | 10 ++++++++++ 1 file changed, 10 insertions(+)target/riscv/cpu.c +10
-0@@ -1485,6 +1485,16 @@ static Property riscv_cpu_extensions[] = {DEFINE_PROP_BOOL ("x-zvfh" ,RISCVCPU , cfg.ext_zvfh, false),DEFINE_PROP_BOOL ("x-zvfhmin" ,RISCVCPU , cfg.ext_zvfhmin, false), +/* Vector cryptography extensions */ +DEFINE_PROP_BOOL ("x-zvbb" ,RISCVCPU , cfg.ext_zvbb, false), +DEFINE_PROP_BOOL ("x-zvbc" ,RISCVCPU , cfg.ext_zvbc, false), +DEFINE_PROP_BOOL ("x-zvkg" ,RISCVCPU , cfg.ext_zvkg, false), +DEFINE_PROP_BOOL ("x-zvkned" ,RISCVCPU , cfg.ext_zvkned, false), +DEFINE_PROP_BOOL ("x-zvknha" ,RISCVCPU , cfg.ext_zvknha, false), +DEFINE_PROP_BOOL ("x-zvknhb" ,RISCVCPU , cfg.ext_zvknhb, false), +DEFINE_PROP_BOOL ("x-zvksed" ,RISCVCPU , cfg.ext_zvksed, false), +DEFINE_PROP_BOOL ("x-zvksh" ,RISCVCPU , cfg.ext_zvksh, false), + DEFINE_PROP_END_OF_LIST(), };
Hi Lawrence,
Can you please re-send, rebased on top of Alistair's riscv-to-apply.next? There are some comments from Weiwei Li that are worth considering. Richard Henderson also took patches 8-9-10 via his tcg queue so you don't have to worry about those.
CC my email in the next version and I'll get some reviews going. QEMU feature freeze for 8.1 is July 11th - perhaps we can squeeze this in for 8.1.
Thanks,
Daniel
Quoted 103 lines · Lawrence Hunter
This patchset provides an implementation for Zvbb, Zvbc, Zvkned, Zvknh, Zvksh, Zvkg, and Zvksed of the draft RISC-V vector cryptography extensions as per the v20230425 version of the specification(1) (
6a7ae7f2). This is an update to the patchset submitted to qemu-devel on Monday, 17 Apr 2023 14:58:36 +0100.v2:
squashed commits into one commit per extension with separate commits for each refactoring
unified trans_rvzvk*.c.inc files into one trans_rvvk.c.inc style fixes in insn32.decode and other files
added macros for EGS values in translation functions. updated from v20230303 to v20230407 of the spec:
Zvkb has been split into Zvbb and Zvbc
vbrev, vclz, vctz, vcpop and vwsll have been added to Zvbb.v3:
New patch 03/19 removes redundant “cpu_vl == 0” checks from trans_rvv.c.inc Introduction of new tcg ops has been factored out of patch 11/19 and into 09/19
These ops are now added to non riscv-specific filesAs v20230425 is a freeze candidate, we are not expecting any significant changes to the specification or this patch series.
Please note that the Zvkt data-independent execution latency extension (and all extensions including it) has not been implemented, and we would recommend not using these patches in an environment where timing attacks are an issue.
Work performed by Dickon, Lawrence, Nazar, Kiran, and William from Codethink sponsored by SiFive, as well as Max Chou and Frank Chang from SiFive.
For convenience we have created a git repo with our patches on top of a recent master. https://github.com/CodethinkLabs/qemu-ct
https://github.com/riscv/riscv-crypto/releases
Thanks to those who have already reviewed:
Richard Henderson [email protected]
[PATCH v2 02/17] target/riscv: Refactor vector-vector translation macro [PATCH v2 04/17] target/riscv: Move vector translation checks
[PATCH v2 05/17] target/riscv: Refactor translation of vector-widening instruction [PATCH v2 07/17]qemu/bitops.h: Limit rotate amounts
[PATCH v2 08/17]qemu/host-utils.h: Add clz and ctz functions for lower-bit integers [PATCH v2 14/17] crypto: Createsm4_subword
Alistair Francis [email protected]
[PATCH v2 02/17] target/riscv: Refactor vector-vector translation macro
Philipp Tomsich [email protected]
Various v1 reviews
Christoph Müllner [email protected]
Various v1 reviewsDickon Hood (3):
target/riscv: Refactor translation of vector-widening instructionqemu/bitops.h: Limit rotate amounts
target/riscv: Add Zvbb ISA extension supportKiran Ostrolenk (5):
target/riscv: Refactor some of the generic vector functionality target/riscv: Refactor vector-vector translation macro target/riscv: Refactor some of the generic vector functionalityqemu/host-utils.h: Add clz and ctz functions for lower-bit integers target/riscv: Add Zvknh ISA extension supportLawrence Hunter (2):
target/riscv: Add Zvbc ISA extension support
target/riscv: Add Zvksh ISA extension supportMax Chou (3):
crypto: Createsm4_subword
crypto: Add SM4 constant parameter CK
target/riscv: Add Zvksed ISA extension supportNazar Kazakov (6):
target/riscv: Remove redundant "cpu_vl== 0" checks target/riscv: Move vector translation checks
tcg: Add andcs and rotrs tcg gvec ops target/riscv: Add Zvkned ISA extension support target/riscv: Add Zvkg ISA extension support target/riscv: Expose Zvk* and Zvb[b,c] cpu propertiesaccel/tcg/tcg-runtime-gvec.c | 11 + accel/tcg/tcg-runtime.h | 1 + crypto/sm4.c | 10 + include/crypto/sm4.h | 9 + include/qemu/bitops.h | 24 +- include/qemu/host-utils.h | 54 ++ include/tcg/tcg-op-gvec.h | 4 + target/arm/tcg/crypto_helper.c | 10 +- target/riscv/cpu.c | 39 + target/riscv/cpu.h | 8 + target/riscv/helper.h | 95 ++ target/riscv/insn32.decode | 58 ++ target/riscv/insn_trans/trans_rvv.c.inc | 174 ++-- target/riscv/insn_trans/trans_rvvk.c.inc | 593 ++++++++++++ target/riscv/meson.build | 4 +- target/riscv/op_helper.c | 6 + target/riscv/translate.c | 1 + target/riscv/vcrypto_helper.c | 1052 ++++++++++++++++++++++ target/riscv/vector_helper.c | 243 +---- target/riscv/vector_internals.c | 81 ++ target/riscv/vector_internals.h | 228 +++++ tcg/tcg-op-gvec.c | 23 + 22 files changed, 2365 insertions(+), 363 deletions(-)create mode 100644 target/riscv/insn_trans/trans_rvvk.c.inc create mode 100644
target/riscv/vcrypto_helper.ccreate mode 100644target/riscv/vector_internals.ccreate mode 100644target/riscv/vector_internals.h
Hi Daniel,
I'm Max Chou from SiFive, one of the authors of this patchset.
I'll take over to update this patchset to the v20230531 version of the RISC-V vector cryptography specification and take the comments from Weiwei Li into consideration. Then I'll re-send, rebased on top of Alistair's riscv-to-apply.next in the next few days.
Thanks,
Max
Quoted 119 lines · Daniel Henrique Barboza
Hi Lawrence,
Can you please re-send, rebased on top of Alistair's riscv-to-apply.next? There are some comments from Weiwei Li that are worth considering. Richard Henderson also took patches 8-9-10 via his tcg queue so you don't have to worry about those.
CC my email in the next version and I'll get some reviews going. QEMU feature freeze for 8.1 is July 11th - perhaps we can squeeze this in for 8.1.
Thanks,
Daniel
Quoted 103 lines · Lawrence Hunter
This patchset provides an implementation for Zvbb, Zvbc, Zvkned, Zvknh, Zvksh, Zvkg, and Zvksed of the draft RISC-V vector cryptography extensions as per the v20230425 version of the specification(1) (
6a7ae7f2). This is an update to the patchset submitted to qemu-devel on Monday, 17 Apr 2023 14:58:36 +0100.v2:
squashed commits into one commit per extension with separate commits for each refactoring
unified trans_rvzvk*.c.inc files into one trans_rvvk.c.inc style fixes in insn32.decode and other files
added macros for EGS values in translation functions. updated from v20230303 to v20230407 of the spec:
Zvkb has been split into Zvbb and Zvbc
vbrev, vclz, vctz, vcpop and vwsll have been added to Zvbb.v3:
New patch 03/19 removes redundant “cpu_vl == 0” checks from trans_rvv.c.inc Introduction of new tcg ops has been factored out of patch 11/19 and into 09/19 These ops are now added to non riscv-specific files
As v20230425 is a freeze candidate, we are not expecting any significant changes to the specification or this patch series.
Please note that the Zvkt data-independent execution latency extension (and all extensions including it) has not been implemented, and we would recommend not using these patches in an environment where timing attacks are an issue.
Work performed by Dickon, Lawrence, Nazar, Kiran, and William from Codethink sponsored by SiFive, as well as Max Chou and Frank Chang from SiFive.
For convenience we have created a git repo with our patches on top of a recent master. https://github.com/CodethinkLabs/qemu-ct
https://github.com/riscv/riscv-crypto/releases
Thanks to those who have already reviewed:
Richard Henderson [email protected]
[PATCH v2 02/17] target/riscv: Refactor vector-vector translation macro [PATCH v2 04/17] target/riscv: Move vector translation checks
[PATCH v2 05/17] target/riscv: Refactor translation of vector-widening instruction [PATCH v2 07/17]qemu/bitops.h: Limit rotate amounts
[PATCH v2 08/17]qemu/host-utils.h: Add clz and ctz functions for lower-bit integers [PATCH v2 14/17] crypto: Createsm4_subword
Alistair Francis [email protected]
[PATCH v2 02/17] target/riscv: Refactor vector-vector translation macro Philipp Tomsich [email protected]
Various v1 reviews
Christoph Müllner [email protected]
Various v1 reviewsDickon Hood (3):
target/riscv: Refactor translation of vector-widening instructionqemu/bitops.h: Limit rotate amounts
target/riscv: Add Zvbb ISA extension supportKiran Ostrolenk (5):
target/riscv: Refactor some of the generic vector functionality target/riscv: Refactor vector-vector translation macro target/riscv: Refactor some of the generic vector functionalityqemu/host-utils.h: Add clz and ctz functions for lower-bit integers target/riscv: Add Zvknh ISA extension supportLawrence Hunter (2):
target/riscv: Add Zvbc ISA extension support
target/riscv: Add Zvksh ISA extension supportMax Chou (3):
crypto: Createsm4_subword
crypto: Add SM4 constant parameter CK
target/riscv: Add Zvksed ISA extension supportNazar Kazakov (6):
target/riscv: Remove redundant "cpu_vl== 0" checks target/riscv: Move vector translation checks
tcg: Add andcs and rotrs tcg gvec ops target/riscv: Add Zvkned ISA extension support target/riscv: Add Zvkg ISA extension support target/riscv: Expose Zvk* and Zvb[b,c] cpu propertiesaccel/tcg/tcg-runtime-gvec.c | 11 + accel/tcg/tcg-runtime.h | 1 + crypto/sm4.c | 10 + include/crypto/sm4.h | 9 + include/qemu/bitops.h | 24 +- include/qemu/host-utils.h | 54 ++ include/tcg/tcg-op-gvec.h | 4 + target/arm/tcg/crypto_helper.c | 10 +- target/riscv/cpu.c | 39 + target/riscv/cpu.h | 8 + target/riscv/helper.h | 95 ++ target/riscv/insn32.decode | 58 ++ target/riscv/insn_trans/trans_rvv.c.inc | 174 ++-- target/riscv/insn_trans/trans_rvvk.c.inc | 593 ++++++++++++
target/riscv/meson.build | 4 +- target/riscv/op_helper.c | 6 + target/riscv/translate.c | 1 +target/riscv/vcrypto_helper.c | 1052 ++++++++++++++++++++++ target/riscv/vector_helper.c | 243 +---- target/riscv/vector_internals.c | 81 ++ target/riscv/vector_internals.h | 228 +++++ tcg/tcg-op-gvec.c | 23 +
22 files changed, 2365 insertions(+), 363 deletions(-)create mode 100644 target/riscv/insn_trans/trans_rvvk.c.inc create mode 100644
target/riscv/vcrypto_helper.ccreate mode 100644target/riscv/vector_internals.ccreate mode 100644target/riscv/vector_internals.h