tcg: Support for Int128 with helpers
Subsystems (3)
v3 cover tcg: Support for Int128 with helpers
This is working toward improving atomicity within TCG, especially with respect to Arm FEAT_LSE2, which guarantees that any operation that does not cross a 16-byte boundary is treated atomically.
(Incidentally, I've also stumbled across language in the Intel SDM that shows the feature is required there too, and the guarantee is even bigger -- anything that does not cross a cache line boundary. Given that we've only ever got 16-byte atomic operations on other hosts, there's no chance of supporting full cache line generically. But we could turn on the same "within 16" atomicity that will be used for FEAT_LSE2.)
That goal is somewhat down the road. This patch set contains two
items: paired register allocation and TCGv_i128 usage with helpers.
The next step will be putting these two together to provide atomic 128-bit load/store operations within TCG. Via, e.g. AArch64 LDP, Power7 LDQ, or S390x LPQ -- all of which require allocating a pair of registers. (Intel will require that I go through AVX, which is a bit of a complication, but I'll figure that out.) And then of course separately via the helpers used by the slow path.
Patches for target/ to use and test this to follow.
Changes for v3:
* Testing showed that trying to make things "easier" for the register allocator on 32-bit hosts by keeping TCGv_i128 as a blob was a mistake. Now split to 4 parts, similar to how we treat TCGv_i64 as 2 parts.
* Fallout from the above is that we now have to support more than 14 call arguments, which meant expanding TCGOp. Now allocated variable sized, using only the nubmer of operands required. This could in fact result in less memory usage on average, but haven't collected any numbers.
* Implement (non-atomic) load/store on TCGv_i128, which gives some of the helpers required for...
* Implement tcg_gen_atomic_cmpxchg_i128, which will eliminate the primary source of 128-bit hacks/ifdefs in target/.
Changes for v2:
* Fixes and r-b (philmd).
* Include i386 atomic16 patch, which avoids minor conflicts later.
* Split a few larger patches.
* Bug fixes for TCI.
r~
Richard Henderson (45):
meson: Move CONFIG_TCG_INTERPRETER to config_host
tcg: Tidy tcg_reg_alloc_op
tcg: Introduce paired register allocation
tcg/s390x: Use register pair allocation for div and mulu2 tcg/arm: Use register pair allocation for qemu_{ld,st}_i64
tcg: Remove TCG_TARGET_STACK_GROWSUP
accel/tcg: Set cflags_next_tb in cpu_common_initfn target/sparc: Avoid TCGV_{LOW,HIGH}
tcg: Move TCG_{LOW,HIGH} to tcg-internal.h
tcg: Add temp_subindex to TCGTemp
tcg: Simplify calls to temp_sync vs mem_coherent
tcg: Allocate TCGTemp pairs in host memory order
tcg: Move TCG_TYPE_COUNT outside enum
tcg: Introduce tcg_type_size
tcg: Introduce TCGCallReturnKind and TCGCallArgumentKind
tcg: Replace TCG_TARGET_CALL_ALIGN_ARGS with TCG_TARGET_CALL_ARG_I64
tcg: Replace TCG_TARGET_EXTEND_ARGS with TCG_TARGET_CALL_ARG_I32
tcg: Use TCG_CALL_ARG_EVEN for TCI special case accel/tcg/plugin: Don't search for the function pointer index accel/tcg/plugin: Avoid duplicate copy in copy_call accel/tcg/plugin: Use copy_op in append_{udata,mem}_cb
tci: MAX_OPC_PARAM_IARGS is no longer used
tcg: Vary the allocation size for TCGOp
tcg: Use output_pref wrapper function
tcg: Reorg function calls
tcg: Move ffi_cif pointer into TCGHelperInfo
tcg/aarch64: Merge tcg_out_callr into tcg_out_call
tcg: Add TCGHelperInfo argument to tcg_out_call
tcg: Define TCG_TYPE_I128 and related helper macros
tcg: Handle dh_typecode_i128 with TCG_CALL_{RET,ARG}_NORMAL
tcg: Allocate objects contiguously in temp_allocate_frame
tcg: Introduce tcg_out_addi_ptr
tcg: Add TCG_CALL_{RET,ARG}_BY_REF
tcg: Introduce tcg_target_call_oarg_reg
tcg: Add TCG_CALL_RET_BY_VEC
include/qemu/int128: Use Int128 structure for TCI tcg/i386: Add TCG_TARGET_CALL_{RET,ARG}_I128
tcg/tci: Fix big-endian return register ordering
tcg/tci: Add TCG_TARGET_CALL_{RET,ARG}_I128
tcg: Add TCG_TARGET_CALL_{RET,ARG}_I128
tcg: Add temp allocation for TCGv_i128
tcg: Add basic data movement for TCGv_i128
tcg: Add guest load/store primitives for TCGv_i128
tcg: Add tcg_gen_{non}atomic_cmpxchg_i128
tcg: Split out tcg_gen_nonatomic_cmpxchg_i{32,64}
accel/tcg/tcg-runtime.h | 11 + include/exec/cpu_ldst.h | 10 + include/exec/helper-head.h | 9 +- include/qemu/atomic128.h | 29 +- include/qemu/int128.h | 25 +- include/tcg/tcg-op.h | 50 +- include/tcg/tcg.h | 145 ++- tcg/aarch64/tcg-target.h | 6 +- tcg/arm/tcg-target-con-set.h | 7 +- tcg/arm/tcg-target-con-str.h | 2 + tcg/arm/tcg-target.h | 6 +- tcg/i386/tcg-target.h | 12 + tcg/loongarch64/tcg-target.h | 5 +- tcg/mips/tcg-target.h | 6 +- tcg/riscv/tcg-target.h | 10 +- tcg/s390x/tcg-target-con-set.h | 4 +- tcg/s390x/tcg-target-con-str.h | 8 +- tcg/s390x/tcg-target.h | 5 +- tcg/sparc64/tcg-target.h | 5 +- tcg/tcg-internal.h | 75 +- tcg/tci/tcg-target.h | 10 + accel/tcg/cputlb.c | 112 ++ accel/tcg/plugin-gen.c | 54 +- accel/tcg/user-exec.c | 66 ++ hw/core/cpu-common.c | 1 + target/sparc/translate.c | 21 +- tcg/optimize.c | 10 +- tcg/tcg-op-vec.c | 10 +- tcg/tcg-op.c | 442 ++++++-- tcg/tcg.c | 1679 +++++++++++++++++++++--------- tcg/tci.c | 66 +- util/int128.c | 42 + accel/tcg/atomic_common.c.inc | 45 + meson.build | 4 +- tcg/aarch64/tcg-target.c.inc | 36 +- tcg/arm/tcg-target.c.inc | 68 +- tcg/i386/tcg-target.c.inc | 57 +- tcg/loongarch64/tcg-target.c.inc | 24 +- tcg/mips/tcg-target.c.inc | 20 +- tcg/ppc/tcg-target.c.inc | 56 +- tcg/riscv/tcg-target.c.inc | 24 +- tcg/s390x/tcg-target.c.inc | 71 +- tcg/sparc64/tcg-target.c.inc | 22 +- tcg/tci/tcg-target.c.inc | 36 +- 44 files changed, 2506 insertions(+), 900 deletions(-)
Signature · 1 line
2.34.1
PATCH v3 1/45 meson: Move CONFIG_TCG_INTERPRETER to config_host
Like CONFIG_TCG, the enabled method of execution is a host property not a guest property. This exposes the define to compile-once files.
--- meson.build | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
meson.build +1 -3
@@ -469,6 +469,7 @@ if get_option('tcg').allowed() endif if get_option('tcg_interpreter') tcg_arch = 'tci' + config_host += { 'CONFIG_TCG_INTERPRETER': 'y' } elif host_arch == 'x86_64' tcg_arch = 'i386' elif host_arch == 'ppc64'
@@ -2540,9 +2541,6 @@ foreach target : target_dirs if sym == 'CONFIG_TCG' or target in accelerator_targets.get(sym, []) config_target += { sym: 'y' } config_all += { sym: 'y' } - if sym == 'CONFIG_TCG' and tcg_arch == 'tci' - config_target += { 'CONFIG_TCG_INTERPRETER': 'y' } - endif if target in modular_tcg config_target += { 'CONFIG_TCG_MODULAR': 'y' } else
Signature · 1 line
2.34.1
PATCH v3 2/45 tcg: Tidy tcg_reg_alloc_op
In reply to Richard HendersonReplace goto allocate_in_reg with a boolean. Remove o_preferred_regs which isn't used, except to copy.
--- tcg/tcg.c | 45 +++++++++++++++++++++------------------------ 1 file changed, 21 insertions(+), 24 deletions(-)
tcg/tcg.c +21 -24
@@ -3606,7 +3606,8 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op)/* satisfy input constraints */ for (k =0 ; k < nb_iargs; k++) { -TCGRegSet i_preferred_regs, o_preferred_regs; +TCGRegSet i_preferred_regs; +bool allocate_new_reg; i = def->args_ct[nb_oargs + k].sort_index; arg = op->args[i];
@@ -3621,9 +3622,12 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op)continue ; } - i_preferred_regs = o_preferred_regs =0 ; + reg = ts->reg; + i_preferred_regs =0 ; + allocate_new_reg = false; +if (arg_ct->ialias) { - o_preferred_regs = op->output_pref[arg_ct->alias_index]; + i_preferred_regs = op->output_pref[arg_ct->alias_index];/* * If the input is readonly, then it cannot also be an
@@ -3632,30 +3636,23 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op)* register and move it. */ if (temp_readonly (ts) || !IS_DEAD_ARG (i)) { - goto allocate_in_reg; + allocate_new_reg = true; + }else if (ts->val_type ==TEMP_VAL_REG ) { +/* +* Check if the current register has already been +* allocated for another input. +*/ + allocate_new_reg =tcg_regset_test_reg (i_allocated_regs, reg); } - -/* -* Check if the current register has already been allocated -* for another input aliased to an output. -*/ -if (ts->val_type ==TEMP_VAL_REG ) { - reg = ts->reg; -for (int k2 =0 ; k2 < k; k2++) { -int i2 = def->args_ct[nb_oargs + k2].sort_index; -if (def->args_ct[i2].ialias && reg == new_args[i2]) { - goto allocate_in_reg; - } - } - } - i_preferred_regs = o_preferred_regs; } -temp_load (s, ts, arg_ct->regs, i_allocated_regs, i_preferred_regs); - reg = ts->reg; +if (!allocate_new_reg) { +temp_load (s, ts, arg_ct->regs, i_allocated_regs, i_preferred_regs); + reg = ts->reg; + allocate_new_reg = !tcg_regset_test_reg (arg_ct->regs, reg); + } -if (!tcg_regset_test_reg (arg_ct->regs, reg)) { - allocate_in_reg: +if (allocate_new_reg) {/* * Allocate a new register matching the constraint * and move the temporary register into it.
@@ -3663,7 +3660,7 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op)temp_load (s, ts, tcg_target_available_regs[ts->type], i_allocated_regs,0 ); reg =tcg_reg_alloc (s, arg_ct->regs, i_allocated_regs, - o_preferred_regs, ts->indirect_base); + i_preferred_regs, ts->indirect_base);if (!tcg_out_mov (s, ts->type, reg, ts->reg)) {/* * Cross register class move not supported. Sync the
Signature · 1 line
2.34.1
PATCH v3 3/45 tcg: Introduce paired register allocation
In reply to Richard HendersonThere are several instances where we need to be able to allocate a pair of registers to related inputs/outputs. Add 'p' and 'm' register constraints for this, in order to be able to allocate the even/odd register first or second.
--- include/tcg/tcg.h | 2 + tcg/tcg.c | 469 ++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 393 insertions(+), 78 deletions(-)
include/tcg/tcg.h +2 -0
@@ -951,6 +951,8 @@ typedef struct TCGArgConstraint { unsigned ct : 16; unsigned alias_index : 4; unsigned sort_index : 4; + unsigned pair_index : 4; + unsigned pair : 2; /* 0: none, 1: first, 2: second, 3: second alias */ bool oalias : 1; bool ialias : 1; bool newreg : 1;
tcg/tcg.c +391 -78
569 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v3 4/45 tcg/s390x: Use register pair allocation for div and mulu2
In reply to Richard HendersonPreviously we hard-coded R2 and R3.
--- tcg/s390x/tcg-target-con-set.h | 4 ++-- tcg/s390x/tcg-target-con-str.h | 8 +------ tcg/s390x/tcg-target.c.inc | 43 +++++++++++++++++++++++++--------- 3 files changed, 35 insertions(+), 20 deletions(-)
tcg/s390x/tcg-target-con-set.h +2 -2
@@ -29,8 +29,8 @@ C_O1_I2(v, v, v) C_O1_I3(v, v, v, v) C_O1_I4(r, r, ri, r, 0) C_O1_I4(r, r, ri, rI, 0) -C_O2_I2(b, a, 0, r) -C_O2_I3(b, a, 0, 1, r) +C_O2_I2(o, m, 0, r) +C_O2_I3(o, m, 0, 1, r) C_O2_I4(r, r, 0, 1, rA, r) C_O2_I4(r, r, 0, 1, ri, r)C_O2_I4 (r, r,0 ,1 , r, r)
tcg/s390x/tcg-target-con-str.h +1 -7
@@ -11,13 +11,7 @@ REGS('r', ALL_GENERAL_REGS) REGS('L', ALL_GENERAL_REGS & ~SOFTMMU_RESERVE_REGS) REGS('v', ALL_VECTOR_REGS) -/* -* A (single) even/odd pair for division. -* TODO: Add something to the register allocator to allow -* this kind of regno+1 pairing to be done more generally. - */ -REGS('a', 1u << TCG_REG_R2) -REGS ('b' ,1u <<TCG_REG_R3 ) +REGS ('o' ,0xaaaa )/* odd numbered general regs */ /* * Define constraint letters for constants:
tcg/s390x/tcg-target.c.inc +32 -11
69 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v3 5/45 tcg/arm: Use register pair allocation for qemu_{ld, st}_i64
In reply to Richard HendersonAlthough we still can't use ldrd and strd for all operations, increase the chances by getting the register allocation correct.
--- tcg/arm/tcg-target-con-set.h | 7 ++++--- tcg/arm/tcg-target-con-str.h | 2 ++ tcg/arm/tcg-target.c.inc | 28 ++++++++++++++++++---------- 3 files changed, 24 insertions(+), 13 deletions(-)
tcg/arm/tcg-target-con-set.h +4 -3
@@ -15,8 +15,9 @@ C_O0_I2(r, rIN) C_O0_I2(s, s) C_O0_I2(w, r) C_O0_I3(s, s, s) +C_O0_I3(S, p, s) C_O0_I4(r, r, rI, rI) -C_O0_I4(s, s, s, s) +C_O0_I4(S, p, s, s) C_O1_I1(r, l) C_O1_I1(r, r)C_O1_I1 (w, r)
@@ -38,8 +39,8 @@ C_O1_I2(w, w, wZ) C_O1_I3(w, w, w, w) C_O1_I4(r, r, r, rI, rI) C_O1_I4(r, r, rIN, rIK, 0) -C_O2_I1(r, r, l) -C_O2_I2(r, r, l, l) +C_O2_I1(e, p, l) +C_O2_I2(e, p, l, l) C_O2_I2(r, r, r, r) C_O2_I4(r, r, r, r, rIN, rIK)C_O2_I4 (r, r, rI, rI, rIN, rIK)
tcg/arm/tcg-target-con-str.h +2 -0
@@ -8,9 +8,11 @@* Define constraint letters for register sets: * REGS(letter, register_mask) */ +REGS('e', ALL_GENERAL_REGS & 0x5555) /* even regs */ REGS('r', ALL_GENERAL_REGS) REGS('l', ALL_QLOAD_REGS) REGS('s', ALL_QSTORE_REGS) +REGS('S', ALL_QSTORE_REGS & 0x5555) /* even qstore */REGS ('w' ,ALL_VECTOR_REGS )/*
tcg/arm/tcg-target.c.inc +18 -10
@@ -1686,9 +1686,11 @@ static void tcg_out_qemu_ld_index(TCGContext *s, MemOp opc, tcg_out_ld32_r(s, COND_AL, datalo, addrlo, addend); break; case MO_UQ: + /* We used pair allocation for datalo, so already should be aligned. */ + tcg_debug_assert((datalo & 1) == 0); + tcg_debug_assert(datahi == datalo + 1); /* LDRD requires alignment; double-check that. */ - if (get_alignment_bits(opc) >= MO_64 - && (datalo & 1) == 0 && datahi == datalo + 1) { + if (get_alignment_bits(opc) >= MO_64) { /* * Rm (the second address op) must not overlap Rt or Rt + 1. * Since datalo is aligned, we can simplify the test via alignment.
@@ -1742,9 +1744,11 @@ static void tcg_out_qemu_ld_direct(TCGContext *s, MemOp opc, TCGReg datalo, tcg_out_ld32_12(s, COND_AL, datalo, addrlo, 0); break; case MO_UQ: + /* We used pair allocation for datalo, so already should be aligned. */ + tcg_debug_assert((datalo & 1) == 0); + tcg_debug_assert(datahi == datalo + 1); /* LDRD requires alignment; double-check that. */ - if (get_alignment_bits(opc) >= MO_64 - && (datalo & 1) == 0 && datahi == datalo + 1) { + if (get_alignment_bits(opc) >= MO_64) { tcg_out_ldrd_8(s, COND_AL, datalo, addrlo, 0); } else if (datalo == addrlo) { tcg_out_ld32_12(s, COND_AL, datahi, addrlo, 4);
@@ -1826,9 +1830,11 @@ static void tcg_out_qemu_st_index(TCGContext *s, ARMCond cond, MemOp opc, tcg_out_st32_r(s, cond, datalo, addrlo, addend); break; case MO_64: + /* We used pair allocation for datalo, so already should be aligned. */ + tcg_debug_assert((datalo & 1) == 0); + tcg_debug_assert(datahi == datalo + 1); /* STRD requires alignment; double-check that. */ - if (get_alignment_bits(opc) >= MO_64 - && (datalo & 1) == 0 && datahi == datalo + 1) { + if (get_alignment_bits(opc) >= MO_64) { tcg_out_strd_r(s, cond, datalo, addrlo, addend); } else if (scratch_addend) { tcg_out_st32_rwb(s, cond, datalo, addend, addrlo);
@@ -1863,9 +1869,11 @@ static void tcg_out_qemu_st_direct(TCGContext *s, MemOp opc, TCGReg datalo, tcg_out_st32_12(s, COND_AL, datalo, addrlo, 0); break; case MO_64: + /* We used pair allocation for datalo, so already should be aligned. */ + tcg_debug_assert((datalo & 1) == 0); + tcg_debug_assert(datahi == datalo + 1); /* STRD requires alignment; double-check that. */ - if (get_alignment_bits(opc) >= MO_64 - && (datalo & 1) == 0 && datahi == datalo + 1) { + if (get_alignment_bits(opc) >= MO_64) { tcg_out_strd_8(s, COND_AL, datalo, addrlo, 0); } else { tcg_out_st32_12(s, COND_AL, datalo, addrlo, 0);
@@ -2333,11 +2341,11 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) case INDEX_op_qemu_ld_i32: return TARGET_LONG_BITS == 32 ? C_O1_I1(r, l) : C_O1_I2(r, l, l); case INDEX_op_qemu_ld_i64: - return TARGET_LONG_BITS == 32 ? C_O2_I1(r, r, l) : C_O2_I2(r, r, l, l); + return TARGET_LONG_BITS == 32 ? C_O2_I1(e, p, l) : C_O2_I2(e, p, l, l); case INDEX_op_qemu_st_i32: return TARGET_LONG_BITS == 32 ? C_O0_I2(s, s) : C_O0_I3(s, s, s); case INDEX_op_qemu_st_i64: - return TARGET_LONG_BITS == 32 ? C_O0_I3(s, s, s) : C_O0_I4(s, s, s, s); + return TARGET_LONG_BITS == 32 ? C_O0_I3(S, p, s) : C_O0_I4(S, p, s, s); case INDEX_op_st_vec: return C_O0_I2(w, r);
Signature · 1 line
2.34.1
PATCH v3 6/45 tcg: Remove TCG_TARGET_STACK_GROWSUP
In reply to Richard HendersonThe hppa host code has been removed since 2013; this should have been deleted at the same time.
--- tcg/aarch64/tcg-target.h | 1 - tcg/arm/tcg-target.h | 1 - tcg/tcg.c | 32 ++------------------------------ 3 files changed, 2 insertions(+), 32 deletions(-)
tcg/aarch64/tcg-target.h +0 -1
@@ -16,7 +16,6 @@#define TCG_TARGET_INSN_UNIT_SIZE 4#define TCG_TARGET_TLB_DISPLACEMENT_BITS 24#define MAX_CODE_GEN_BUFFER_SIZE (2 * GiB) -#undef TCG_TARGET_STACK_GROWSUP typedef enum { TCG_REG_X0, TCG_REG_X1, TCG_REG_X2, TCG_REG_X3,
tcg/arm/tcg-target.h +0 -1
@@ -30,7 +30,6 @@ extern int arm_arch;#define use_armv7_instructions (__ARM_ARCH >= 7 || arm_arch >= 7) -#undef TCG_TARGET_STACK_GROWSUP#define TCG_TARGET_INSN_UNIT_SIZE 4#define TCG_TARGET_TLB_DISPLACEMENT_BITS 16#define MAX_CODE_GEN_BUFFER_SIZE UINT32_MAX
tcg/tcg.c +2 -30
@@ -1551,25 +1551,8 @@ void tcg_gen_callN(void *func, TCGTemp *ret, int nargs, TCGTemp **args) }if (TCG_TARGET_REG_BITS <64 && is_64bit) { -/* -* If stack grows up, then we will be placing successive -* arguments at lower addresses, which means we need to -* reverse the order compared to how we would normally -* treat either big or little-endian. For those arguments -* that will wind up in registers, this still works for -* HPPA (the only current STACK_GROWSUP target) since the -* argument registers are *also* allocated in decreasing -* order. If another such target is added, this logic may -* have to get more complicated to differentiate between -* stack arguments and register arguments. -*/ -#if HOST_BIG_ENDIAN != defined(TCG_TARGET_STACK_GROWSUP ) - op->args[pi++] =temp_arg (args[i] +1 ); - op->args[pi++] =temp_arg (args[i]); -#else - op->args[pi++] =temp_arg (args[i]); - op->args[pi++] =temp_arg (args[i] +1 ); -#endif + op->args[pi++] =temp_arg (args[i] +HOST_BIG_ENDIAN ); + op->args[pi++] =temp_arg (args[i] + !HOST_BIG_ENDIAN ); real_args +=2 ;continue ; }
@@ -4166,12 +4149,6 @@ static bool tcg_reg_alloc_dup2(TCGContext *s, const TCGOp *op)return true; } -#ifdef TCG_TARGET_STACK_GROWSUP -#define STACK_DIR (x) (-(x)) -#else -#define STACK_DIR (x) (x) -#endif -static void tcg_reg_alloc_call (TCGContext *s,TCGOp *op) {const int nb_oargs =TCGOP_CALLO (op);
@@ -4211,18 +4188,13 @@ static void tcg_reg_alloc_call(TCGContext *s, TCGOp *op) stack_offset =TCG_TARGET_CALL_STACK_OFFSET ;for (i = nb_regs; i < nb_iargs; i++) { arg = op->args[nb_oargs + i]; -#ifdef TCG_TARGET_STACK_GROWSUP - stack_offset -=sizeof (tcg_target_long); -#endif if (arg !=TCG_CALL_DUMMY_ARG ) { ts =arg_temp (arg);temp_load (s, ts, tcg_target_available_regs[ts->type], s->reserved_regs,0 );tcg_out_st (s, ts->type, ts->reg,TCG_REG_CALL_STACK , stack_offset); } -#ifndef TCG_TARGET_STACK_GROWSUP stack_offset +=sizeof (tcg_target_long); -#endif }/* assign input registers */
Signature · 1 line
2.34.1
PATCH v3 7/45 accel/tcg: Set cflags_next_tb in cpu_common_initfn
In reply to Richard HendersonWhile we initialize this value in cpu_common_reset, that isn't called during startup, so set it as well in init. This fixes -singlestep versus the very first TB.
--- hw/core/cpu-common.c | 1 + 1 file changed, 1 insertion(+)
hw/core/cpu-common.c +1 -0
@@ -235,6 +235,7 @@ static void cpu_common_initfn(Object *obj)/* the default value is changed by qemu_init_vcpu() for softmmu */ cpu->nr_cores =1 ; cpu->nr_threads =1 ; + cpu->cflags_next_tb =-1 ;qemu_mutex_init (&cpu->work_mutex);QSIMPLEQ_INIT (&cpu->work_list);
Signature · 1 line
2.34.1
PATCH v3 8/45 target/sparc: Avoid TCGV_{LOW,HIGH}
In reply to Richard HendersonUse the official extend/extract functions instead of routines that will shortly be internal to tcg.
--- target/sparc/translate.c | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-)
target/sparc/translate.c +4 -17
@@ -163,13 +163,6 @@ static inline void gen_update_fprs_dirty(DisasContext *dc, int rd)/* floating point registers moves */ static TCGv_i32 gen_load_fpr_F (DisasContext *dc,unsigned int src) { -#if TCG_TARGET_REG_BITS ==32 -if (src &1 ) { -return TCGV_LOW (cpu_fpr[src /2 ]); - }else { -return TCGV_HIGH (cpu_fpr[src /2 ]); - } -#else TCGv_i32 ret =get_temp_i32 (dc);if (src &1 ) {tcg_gen_extrl_i64_i32 (ret, cpu_fpr[src /2 ]);
@@ -177,22 +170,16 @@ static TCGv_i32 gen_load_fpr_F(DisasContext *dc, unsigned int src)tcg_gen_extrh_i64_i32 (ret, cpu_fpr[src /2 ]); }return ret; -#endif }static void gen_store_fpr_F (DisasContext *dc,unsigned int dst,TCGv_i32 v) { -#if TCG_TARGET_REG_BITS ==32 -if (dst &1 ) { -tcg_gen_mov_i32 (TCGV_LOW (cpu_fpr[dst /2 ]), v); - }else { -tcg_gen_mov_i32 (TCGV_HIGH (cpu_fpr[dst /2 ]), v); - } -#else -TCGv_i64 t = (TCGv_i64 )v; +TCGv_i64 t =tcg_temp_new_i64 (); + +tcg_gen_extu_i32_i64 (t, v);tcg_gen_deposit_i64 (cpu_fpr[dst /2 ], cpu_fpr[dst /2 ], t, (dst &1 ?0 :32 ),32 ); -#endif +tcg_temp_free_i64 (t);gen_update_fprs_dirty (dc, dst); }
Signature · 1 line
2.34.1
PATCH v3 9/45 tcg: Move TCG_{LOW,HIGH} to tcg-internal.h
In reply to Richard HendersonMove the error-generating fallback from tcg-op.c, and replace "_link_error" with modern QEMU_ERROR markup.
--- include/tcg/tcg-op.h | 33 +++++---------------------------- include/tcg/tcg.h | 12 ------------ tcg/tcg-internal.h | 14 ++++++++++++++ tcg/tcg-op-vec.c | 2 ++ tcg/tcg-op.c | 37 ++++++++++++++++++++++++++++--------- 5 files changed, 49 insertions(+), 49 deletions(-)
include/tcg/tcg-op.h +5 -28
@@ -667,35 +667,12 @@ static inline void tcg_gen_mul_i64(TCGv_i64 ret, TCGv_i64 arg1, TCGv_i64 arg2) tcg_gen_op3_i64(INDEX_op_mul_i64, ret, arg1, arg2); }#else /* TCG_TARGET_REG_BITS == 32 */ -static inline void tcg_gen_st8_i64 (TCGv_i64 arg1,TCGv_ptr arg2, -tcg_target_long offset) -{ -tcg_gen_st8_i32 (TCGV_LOW (arg1), arg2, offset); -} +void tcg_gen_st8_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset); +void tcg_gen_st16_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset); +void tcg_gen_st32_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset); -static inline void tcg_gen_st16_i64 (TCGv_i64 arg1,TCGv_ptr arg2, -tcg_target_long offset) -{ -tcg_gen_st16_i32 (TCGV_LOW (arg1), arg2, offset); -} - -static inline void tcg_gen_st32_i64 (TCGv_i64 arg1,TCGv_ptr arg2, -tcg_target_long offset) -{ -tcg_gen_st_i32 (TCGV_LOW (arg1), arg2, offset); -} - -static inline void tcg_gen_add_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2) -{ -tcg_gen_add2_i32 (TCGV_LOW (ret),TCGV_HIGH (ret),TCGV_LOW (arg1), -TCGV_HIGH (arg1),TCGV_LOW (arg2),TCGV_HIGH (arg2)); -} - -static inline void tcg_gen_sub_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2) -{ -tcg_gen_sub2_i32 (TCGV_LOW (ret),TCGV_HIGH (ret),TCGV_LOW (arg1), -TCGV_HIGH (arg1),TCGV_LOW (arg2),TCGV_HIGH (arg2)); -} +void tcg_gen_add_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2); +void tcg_gen_sub_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2);void tcg_gen_discard_i64 (TCGv_i64 arg);void tcg_gen_mov_i64 (TCGv_i64 ret,TCGv_i64 arg);
include/tcg/tcg.h +0 -12
@@ -737,18 +737,6 @@ static inline TCGv_vec temp_tcgv_vec(TCGTemp *t)return (TCGv_vec )temp_tcgv_i32 (t); } -#if TCG_TARGET_REG_BITS ==32 -static inline TCGv_i32 TCGV_LOW (TCGv_i64 t) -{ -return temp_tcgv_i32 (tcgv_i64_temp (t)); -} - -static inline TCGv_i32 TCGV_HIGH (TCGv_i64 t) -{ -return temp_tcgv_i32 (tcgv_i64_temp (t) +1 ); -} -#endif -static inline TCGArg tcg_get_insn_param (TCGOp *op,int arg) {return op->args[arg];
tcg/tcg-internal.h +14 -0
@@ -59,4 +59,18 @@ static inline unsigned tcg_call_flags(TCGOp *op)return tcg_call_info (op)->flags; } +#if TCG_TARGET_REG_BITS ==32 +static inline TCGv_i32 TCGV_LOW (TCGv_i64 t) +{ +return temp_tcgv_i32 (tcgv_i64_temp (t)); +} +static inline TCGv_i32 TCGV_HIGH (TCGv_i64 t) +{ +return temp_tcgv_i32 (tcgv_i64_temp (t) +1 ); +} +#else +extern TCGv_i32 TCGV_LOW(TCGv_i64) QEMU_ERROR("32-bit code path is reachable"); +extern TCGv_i32 TCGV_HIGH(TCGv_i64) QEMU_ERROR("32-bit code path is reachable"); +#endif +#endif /* TCG_INTERNAL_H */
tcg/tcg-op-vec.c +2 -0
@@ -21,6 +21,8 @@#include "tcg/tcg.h" #include "tcg/tcg-op.h" #include "tcg/tcg-mo.h" +#include "tcg-internal.h" +/* Reduce the number of ifdefs below. This assumes that all uses of TCGV_HIGH and TCGV_LOW are properly protected by a conditional that
tcg/tcg-op.c +28 -9
@@ -28,16 +28,8 @@#include "tcg/tcg-op.h" #include "tcg/tcg-mo.h" #include "exec/plugin-gen.h" +#include "tcg-internal.h" -/* Reduce the number of ifdefs below. This assumes that all uses of -TCGV_HIGH and TCGV_LOW are properly protected by a conditional that -the compiler can eliminate. */ -#if TCG_TARGET_REG_BITS ==64 -extern TCGv_i32 TCGV_LOW_link_error (TCGv_i64 ); -extern TCGv_i32 TCGV_HIGH_link_error (TCGv_i64 ); -#define TCGV_LOW TCGV_LOW_link_error -#define TCGV_HIGH TCGV_HIGH_link_error -#endif void tcg_gen_op1 (TCGOpcode opc,TCGArg a1) {
@@ -1171,6 +1163,21 @@ void tcg_gen_ld_i64(TCGv_i64 ret, TCGv_ptr arg2, tcg_target_long offset)#endif } +void tcg_gen_st8_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset) +{ +tcg_gen_st8_i32 (TCGV_LOW (arg1), arg2, offset); +} + +void tcg_gen_st16_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset) +{ +tcg_gen_st16_i32 (TCGV_LOW (arg1), arg2, offset); +} + +void tcg_gen_st32_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset) +{ +tcg_gen_st_i32 (TCGV_LOW (arg1), arg2, offset); +} +void tcg_gen_st_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset) {#if HOST_BIG_ENDIAN
@@ -1182,6 +1189,18 @@ void tcg_gen_st_i64(TCGv_i64 arg1, TCGv_ptr arg2, tcg_target_long offset)#endif } +void tcg_gen_add_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2) +{ +tcg_gen_add2_i32 (TCGV_LOW (ret),TCGV_HIGH (ret),TCGV_LOW (arg1), +TCGV_HIGH (arg1),TCGV_LOW (arg2),TCGV_HIGH (arg2)); +} + +void tcg_gen_sub_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2) +{ +tcg_gen_sub2_i32 (TCGV_LOW (ret),TCGV_HIGH (ret),TCGV_LOW (arg1), +TCGV_HIGH (arg1),TCGV_LOW (arg2),TCGV_HIGH (arg2)); +} +void tcg_gen_and_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2) {tcg_gen_and_i32 (TCGV_LOW (ret),TCGV_LOW (arg1),TCGV_LOW (arg2));
Signature · 1 line
2.34.1
PATCH v3 10/45 tcg: Add temp_subindex to TCGTemp
In reply to Richard HendersonRecord the location of a TCGTemp within a larger object.
--- include/tcg/tcg.h | 1 + tcg/tcg.c | 3 +++ 2 files changed, 4 insertions(+)
include/tcg/tcg.h +1 -0
@@ -456,6 +456,7 @@ typedef struct TCGTemp { unsigned int mem_coherent:1; unsigned int mem_allocated:1; unsigned int temp_allocated:1; + unsigned int temp_subindex:1;int64_t val;struct TCGTemp *mem_base;
tcg/tcg.c +3 -0
@@ -927,6 +927,7 @@ TCGTemp *tcg_global_mem_new_internal(TCGType type, TCGv_ptr base, ts2->mem_allocated =1 ; ts2->mem_base = base_ts; ts2->mem_offset = offset + (1 - bigendian) *4 ; + ts2->temp_subindex =1 ;pstrcpy (buf,sizeof (buf), name);pstrcat (buf,sizeof (buf),"_1" ); ts2->name =strdup (buf);
@@ -973,6 +974,7 @@ TCGTemp *tcg_temp_new_internal(TCGType type, bool temp_local) ts2->base_type =TCG_TYPE_I64 ; ts2->type =TCG_TYPE_I32 ; ts2->temp_allocated =1 ; + ts2->temp_subindex =1 ; ts2->kind = kind; }else { ts->base_type = type;
@@ -1091,6 +1093,7 @@ TCGTemp *tcg_constant_internal(TCGType type, int64_t val) ts2->type =TCG_TYPE_I32 ; ts2->kind =TEMP_CONST ; ts2->temp_allocated =1 ; + ts2->temp_subindex =1 ; ts2->val = val >>32 ; }else { ts->base_type = type;
Signature · 1 line
2.34.1
PATCH v3 11/45 tcg: Simplify calls to temp_sync vs mem_coherent
In reply to Richard HendersonThe first thing that temp_sync does is check mem_coherent, so there's no need for the caller to do so.
--- tcg/tcg.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
tcg/tcg.c +2 -6
@@ -4117,12 +4117,8 @@ static bool tcg_reg_alloc_dup2(TCGContext *s, const TCGOp *op)/* If the two inputs form one 64-bit value, try dupm_vec. */ if (itsl +1 == itsh && itsl->base_type ==TCG_TYPE_I64 ) { -if (!itsl->mem_coherent) { -temp_sync (s, itsl, s->reserved_regs,0 ,0 ); - } -if (!itsh->mem_coherent) { -temp_sync (s, itsh, s->reserved_regs,0 ,0 ); - } +temp_sync (s, itsl, s->reserved_regs,0 ,0 ); +temp_sync (s, itsh, s->reserved_regs,0 ,0 );#if HOST_BIG_ENDIAN TCGTemp *its = itsh;#else
Signature · 1 line
2.34.1
PATCH v3 12/45 tcg: Allocate TCGTemp pairs in host memory order
In reply to Richard HendersonAllocate the first of a pair at the lower address, and the second of a pair at the higher address. This will make it easier to find the beginning of the larger memory block.
--- tcg/tcg-internal.h | 4 ++-- tcg/tcg.c | 58 ++++++++++++++++++++++------------------------ 2 files changed, 30 insertions(+), 32 deletions(-)
tcg/tcg-internal.h +2 -2
@@ -62,11 +62,11 @@ static inline unsigned tcg_call_flags(TCGOp *op)#if TCG_TARGET_REG_BITS ==32 static inline TCGv_i32 TCGV_LOW (TCGv_i64 t) { -return temp_tcgv_i32 (tcgv_i64_temp (t)); +return temp_tcgv_i32 (tcgv_i64_temp (t) +HOST_BIG_ENDIAN ); }static inline TCGv_i32 TCGV_HIGH (TCGv_i64 t) { -return temp_tcgv_i32 (tcgv_i64_temp (t) +1 ); +return temp_tcgv_i32 (tcgv_i64_temp (t) + !HOST_BIG_ENDIAN ); }#else extern TCGv_i32 TCGV_LOW(TCGv_i64) QEMU_ERROR("32-bit code path is reachable");
tcg/tcg.c +28 -30
124 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH for-8.0 v3 12/45] tcg: Allocate TCGTemp pairs in host memory order
Quoted 9 lines · Richard Henderson
Allocate the first of a pair at the lower address, and the second of a pair at the higher address. This will make it easier to find the beginning of the larger memory block.
Signed-off-by Richard Henderson <[email protected]>--- tcg/tcg-internal.h | 4 ++-- tcg/tcg.c | 58 ++++++++++++++++++++++------------------------ 2 files changed, 30 insertions(+), 32 deletions(-)
PATCH v3 13/45 tcg: Move TCG_TYPE_COUNT outside enum
In reply to Richard HendersonThe count is not itself an enumerator. Move it outside to prevent the compiler from considering it with -Wswitch-enum.
--- include/tcg/tcg.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
include/tcg/tcg.h +2 -1
@@ -294,7 +294,8 @@ typedef enum TCGType { TCG_TYPE_V128, TCG_TYPE_V256, -TCG_TYPE_COUNT ,/* number of different types */ +/* Number of different types (integer not enum) */ +#define TCG_TYPE_COUNT (TCG_TYPE_V256 + 1)/* An alias for the size of the host register. */ #if TCG_TARGET_REG_BITS ==32
Signature · 1 line
2.34.1
PATCH v3 14/45 tcg: Introduce tcg_type_size
In reply to Richard HendersonAdd a helper function for computing the size of a type.
--- include/tcg/tcg.h | 16 ++++++++++++++++ tcg/tcg.c | 26 ++++++++++++-------------- 2 files changed, 28 insertions(+), 14 deletions(-)
include/tcg/tcg.h +16 -0
@@ -319,6 +319,22 @@ typedef enum TCGType {#endif } TCGType; +/** +* tcg_type_size +* @t: type +* +* Return the size of the type in bytes. +*/ +static inline int tcg_type_size (TCGType t) +{ +unsigned i = t; +if (i >=TCG_TYPE_V64 ) { +tcg_debug_assert (i <TCG_TYPE_COUNT ); + i -=TCG_TYPE_V64 -1 ; + } +return 4 << i; +} +/** * get_alignment_bits * @memop: MemOp value
tcg/tcg.c +12 -14
@@ -3110,22 +3110,22 @@ static void check_regs(TCGContext *s)static void temp_allocate_frame (TCGContext *s,TCGTemp *ts) { -intptr_t off, size, align; +int size =tcg_type_size (ts->type); +int align; +intptr_t off;switch (ts->type) {case TCG_TYPE_I32 : - size = align =4 ; + align =4 ;break ;case TCG_TYPE_I64 :case TCG_TYPE_V64 : - size = align =8 ; + align =8 ;break ;case TCG_TYPE_V128 : - size = align =16 ; -break ;case TCG_TYPE_V256 :/* Note that we do not require aligned storage for V256. */ - size =32 , align =16 ; + align =16 ;break ;default :g_assert_not_reached ();
@@ -3641,8 +3641,8 @@ static void tcg_reg_alloc_dup(TCGContext *s, const TCGOp *op)TCGRegSet dup_out_regs, dup_in_regs;TCGTemp *its, *ots;TCGType itype, vtype; -intptr_t endian_fixup;unsigned vece; +int lowpart_ofs;bool ok; ots =arg_temp (op->args[0 ]);
@@ -3711,14 +3711,12 @@ static void tcg_reg_alloc_dup(TCGContext *s, const TCGOp *op)/* fall through */ case TEMP_VAL_MEM : -#if HOST_BIG_ENDIAN - endian_fixup = itype ==TCG_TYPE_I32 ?4 :8 ; - endian_fixup -=1 << vece; -#else - endian_fixup =0 ; -#endif + lowpart_ofs =0 ; +if (HOST_BIG_ENDIAN ) { + lowpart_ofs =tcg_type_size (itype) - (1 << vece); + }if (tcg_out_dupm_vec (s, vtype, vece, ots->reg, its->mem_base->reg, - its->mem_offset + endian_fixup)) { + its->mem_offset + lowpart_ofs)) { goto done; }tcg_out_ld (s, itype, ots->reg, its->mem_base->reg, its->mem_offset);
Signature · 1 line
2.34.1
Re: [PATCH for-8.0 v3 14/45] tcg: Introduce tcg_type_size
Quoted 20 lines · Richard Henderson
Add a helper function for computing the size of a type.
Signed-off-by Richard Henderson <[email protected]>--- include/tcg/tcg.h | 16 ++++++++++++++++ tcg/tcg.c | 26 ++++++++++++-------------- 2 files changed, 28 insertions(+), 14 deletions(-)include/tcg/tcg.h +4
-0@@ -319,6 +319,22 @@ typedef enum TCGType {#endif } TCGType; +/** +* tcg_type_size +* @t: type +*
include/tcg/tcg.h typedef enum TCGType { · line 335
Quoted 10 lines
include/tcg/tcg.h +10
-0+* Return the size of the type in bytes. +*/ +static inline int tcg_type_size (TCGType t) +{ +unsigned i = t; +if (i >=TCG_TYPE_V64 ) { +tcg_debug_assert (i <TCG_TYPE_COUNT ); + i -=TCG_TYPE_V64 -1 ; + } +return 4 << i;
I'd feel safer if we assign TCG_TYPE_I32 .. TCG_TYPE_V256 in TCGType, just in case.
include/tcg/tcg.h typedef enum TCGType { · line 336
Quoted 1 line
include/tcg/tcg.h +1
-0+}
Re: [PATCH for-8.0 v3 14/45] tcg: Introduce tcg_type_size
Quoted 25 lines · Philippe Mathieu-Daudé
Quoted 24 lines · Richard Henderson
Add a helper function for computing the size of a type.
Signed-off-by Richard Henderson <[email protected]>---
include/tcg/tcg.h| 16 ++++++++++++++++ tcg/tcg.c | 26 ++++++++++++--------------2 files changed, 28 insertions(+), 14 deletions(-)include/tcg/tcg.h +9
-0@@ -319,6 +319,22 @@ typedef enum TCGType { #endif } TCGType; +/** +* tcg_type_size +* @t: type +* +* Return the size of the type in bytes. +*/ +static inline int tcg_type_size (TCGType t) +{ + unsigned i = t;
Quoted 8 lines
include/tcg/tcg.h typedef enum TCGType { · line 335
Quoted 5 lines
include/tcg/tcg.h +5
-0+ if (i >= TCG_TYPE_V64) { + tcg_debug_assert(i < TCG_TYPE_COUNT); + i -= TCG_TYPE_V64 - 1; + } + return 4 << i;I'd feel safer if we assign
TCG_TYPE_I32..TCG_TYPE_V256in TCGType, just in case.
What do you mean?
r~
Re: [PATCH for-8.0 v3 14/45] tcg: Introduce tcg_type_size
Quoted 35 lines · Richard Henderson
Quoted 25 lines · Philippe Mathieu-Daudé
Quoted 24 lines · Richard Henderson
Add a helper function for computing the size of a type.
Signed-off-by Richard Henderson <[email protected]>---
include/tcg/tcg.h| 16 ++++++++++++++++ tcg/tcg.c | 26 ++++++++++++--------------2 files changed, 28 insertions(+), 14 deletions(-)include/tcg/tcg.h +9
-0@@ -319,6 +319,22 @@ typedef enum TCGType { #endif } TCGType; +/** +* tcg_type_size +* @t: type +* +* Return the size of the type in bytes. +*/ +static inline int tcg_type_size (TCGType t) +{ + unsigned i = t;Quoted 8 lines
include/tcg/tcg.h typedef enum TCGType { · line 334
Quoted 5 lines
include/tcg/tcg.h +5
-0+ if (i >= TCG_TYPE_V64) { + tcg_debug_assert(i < TCG_TYPE_COUNT); + i -= TCG_TYPE_V64 - 1; + } + return 4 << i;I'd feel safer if we assign
TCG_TYPE_I32..TCG_TYPE_V256in TCGType, just in case.
Quoted 1 line
What do you mean?
-- >8 --
include/tcg/tcg.h +5 -5
@@ -289,8 +289,8 @@ typedef struct TCGPool { typedef enum TCGType { - TCG_TYPE_I32, - TCG_TYPE_I64, + TCG_TYPE_I32 = 0, + TCG_TYPE_I64 = 1, - TCG_TYPE_V64, - TCG_TYPE_V128, - TCG_TYPE_V256, + TCG_TYPE_V64 = 2, + TCG_TYPE_V128 = 3, + TCG_TYPE_V256 = 4,
---
Re: [PATCH for-8.0 v3 14/45] tcg: Introduce tcg_type_size
Quoted 10 lines · Philippe Mathieu-Daudé
Quoted 4 lines
Quoted 2 lines
I'd feel safer if we assign
TCG_TYPE_I32..TCG_TYPE_V256in TCGType, just in case.What do you mean?
-- >8 --
include/tcg/tcg.h +0
-1@@ -289,8 +289,8 @@ typedef struct TCGPool { typedef enum TCGType { - TCG_TYPE_I32,
include/tcg/tcg.h typedef struct TCGPool { · line 295
Quoted 10 lines
include/tcg/tcg.h +5
-4- TCG_TYPE_I64, + TCG_TYPE_I32 = 0, + TCG_TYPE_I64 = 1, - TCG_TYPE_V64, - TCG_TYPE_V128, - TCG_TYPE_V256, + TCG_TYPE_V64 = 2, + TCG_TYPE_V128 = 3, + TCG_TYPE_V256 = 4,
But that's what C does. I don't see the point.
r~
PATCH v3 15/45 tcg: Introduce TCGCallReturnKind and TCGCallArgumentKind
In reply to Richard HendersonPrepare to replace a bunch of separate ifdefs with a consistent way to describe the abi of a function call.
--- tcg/tcg-internal.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
tcg/tcg-internal.h +15 -0
@@ -27,6 +27,21 @@#define TCG_HIGHWATER 1024 +/* +* Describe the calling convention of a given argument type. +*/ +typedef enum { +TCG_CALL_RET_NORMAL ,/* by registers */ +}TCGCallReturnKind ; + +typedef enum { +TCG_CALL_ARG_NORMAL ,/* by registers (continuing onto stack) */ +TCG_CALL_ARG_EVEN ,/* like normal, but skipping odd slots */ +TCG_CALL_ARG_EXTEND ,/* for i32, as a sign/zero-extended i64 */ +TCG_CALL_ARG_EXTEND_U ,/* ... as a zero-extended i64 */ +TCG_CALL_ARG_EXTEND_S ,/* ... as a sign-extended i64 */ +}TCGCallArgumentKind ; + typedef struct TCGHelperInfo { void *func; const char *name;
Signature · 1 line
2.34.1
Re: [PATCH for-8.0 v3 15/45] tcg: Introduce TCGCallReturnKind and TCGCallArgumentKind
Quoted 2 lines · Richard Henderson
Prepare to replace a bunch of separate ifdefs with a consistent way to describe the abi of a function call.
Nitpicking, s/abi/ABI/ like in patch 40 "Fill in the parameters for the host ABI for Int128".
Quoted 5 lines
Signed-off-by Richard Henderson <[email protected]>--- tcg/tcg-internal.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
PATCH v3 16/45 tcg: Replace TCG_TARGET_CALL_ALIGN_ARGS with TCG_TARGET_CALL_ARG_I64
In reply to Richard HendersonFor 32-bit hosts when TCG_TARGET_CALL_ALIGN_ARGS was set, use TCG_CALL_ARG_EVEN. For 64-bit hosts, TCG_TARGET_CALL_ALIGN_ARGS was silently ignored, so always use TCG_CALL_ARG_NORMAL.
--- tcg/aarch64/tcg-target.h | 2 +- tcg/arm/tcg-target.h | 2 +- tcg/i386/tcg-target.h | 1 + tcg/loongarch64/tcg-target.h | 2 +- tcg/mips/tcg-target.h | 3 ++- tcg/riscv/tcg-target.h | 6 +++++- tcg/s390x/tcg-target.h | 1 + tcg/sparc64/tcg-target.h | 1 + tcg/tci/tcg-target.h | 5 +++++ tcg/tcg.c | 6 ++++-- tcg/ppc/tcg-target.c.inc | 21 ++++++++------------- 11 files changed, 30 insertions(+), 20 deletions(-)
tcg/aarch64/tcg-target.h +1 -1
@@ -51,8 +51,8 @@ typedef enum {/* used for function call generation */ #define TCG_REG_CALL_STACK TCG_REG_SP#define TCG_TARGET_STACK_ALIGN 16 -#define TCG_TARGET_CALL_ALIGN_ARGS 1#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL/* optional instructions */ #define TCG_TARGET_HAS_div_i32 1
tcg/arm/tcg-target.h +1 -1
@@ -88,8 +88,8 @@ extern bool use_neon_instructions;/* used for function call generation */ #define TCG_TARGET_STACK_ALIGN 8 -#define TCG_TARGET_CALL_ALIGN_ARGS 1#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN/* optional instructions */ #define TCG_TARGET_HAS_ext8s_i32 1
tcg/i386/tcg-target.h +1 -0
@@ -98,6 +98,7 @@ typedef enum {#else #define TCG_TARGET_CALL_STACK_OFFSET 0#endif +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMALextern bool have_bmi1;extern bool have_popcnt;
tcg/loongarch64/tcg-target.h +1 -1
@@ -92,8 +92,8 @@ typedef enum {/* used for function call generation */ #define TCG_REG_CALL_STACK TCG_REG_SP#define TCG_TARGET_STACK_ALIGN 16 -#define TCG_TARGET_CALL_ALIGN_ARGS 1#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL/* optional instructions */ #define TCG_TARGET_HAS_movcond_i32 0
tcg/mips/tcg-target.h +2 -1
@@ -83,10 +83,11 @@ typedef enum {#define TCG_TARGET_STACK_ALIGN 16#if _MIPS_SIM == _ABIO32# define TCG_TARGET_CALL_STACK_OFFSET 16 +# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN#else # define TCG_TARGET_CALL_STACK_OFFSET 0 +# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL#endif -#define TCG_TARGET_CALL_ALIGN_ARGS 1/* MOVN/MOVZ instructions detection */ #if (defined(__mips_isa_rev) && (__mips_isa_rev >= 1)) || \
tcg/riscv/tcg-target.h +5 -1
@@ -81,8 +81,12 @@ typedef enum {/* used for function call generation */ #define TCG_REG_CALL_STACK TCG_REG_SP#define TCG_TARGET_STACK_ALIGN 16 -#define TCG_TARGET_CALL_ALIGN_ARGS 1#define TCG_TARGET_CALL_STACK_OFFSET 0 +#if TCG_TARGET_REG_BITS ==32 +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN +#else +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +#endif /* optional instructions */ #define TCG_TARGET_HAS_movcond_i32 0
tcg/s390x/tcg-target.h +1 -0
@@ -166,6 +166,7 @@ extern uint64_t s390_facilities[3];/* used for function call generation */ #define TCG_TARGET_STACK_ALIGN 8#define TCG_TARGET_CALL_STACK_OFFSET 160 +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_RET_NORMAL#define TCG_TARGET_EXTEND_ARGS 1#define TCG_TARGET_HAS_MEMORY_BSWAP 1
tcg/sparc64/tcg-target.h +1 -0
@@ -72,6 +72,7 @@ typedef enum {#define TCG_TARGET_STACK_ALIGN 16#define TCG_TARGET_CALL_STACK_OFFSET (128 + 6*8 + TCG_TARGET_STACK_BIAS)#define TCG_TARGET_EXTEND_ARGS 1 +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL#if defined(__VIS__) && __VIS__ >=0x300 #define use_vis3_instructions 1
tcg/tci/tcg-target.h +5 -0
@@ -158,6 +158,11 @@ typedef enum {/* Used for function call generation. */ #define TCG_TARGET_CALL_STACK_OFFSET 0#define TCG_TARGET_STACK_ALIGN 8 +#if TCG_TARGET_REG_BITS ==32 +# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN +#else +# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +#endif #define HAVE_TCG_QEMU_TB_EXEC #define TCG_TARGET_NEED_POOL_LABELS
tcg/tcg.c +4 -2
@@ -1541,9 +1541,11 @@ void tcg_gen_callN(void *func, TCGTemp *ret, int nargs, TCGTemp **args)* for passing off to ffi_call. */ want_align = true; -#elif defined(TCG_TARGET_CALL_ALIGN_ARGS ) +#else /* Some targets want aligned 64 bit args */ - want_align = is_64bit; +if (is_64bit) { + want_align =TCG_TARGET_CALL_ARG_I64 ==TCG_CALL_ARG_EVEN ; + }#endif if (TCG_TARGET_REG_BITS <64 && want_align && (real_args &1 )) {
tcg/ppc/tcg-target.c.inc +8 -13
@@ -45,7 +45,9 @@ #endif #ifdef _CALL_SYSV -# define TCG_TARGET_CALL_ALIGN_ARGS 1 +# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN +#else +# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL #endif /* For some memory operations, we need a scratch that isn't R0. For the AIX
@@ -2202,9 +2204,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) lo = lb->addrlo_reg; hi = lb->addrhi_reg; if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { -#ifdef TCG_TARGET_CALL_ALIGN_ARGS - arg |= 1; -#endif + arg |= (TCG_TARGET_CALL_ARG_I64 == TCG_CALL_ARG_EVEN); tcg_out_mov(s, TCG_TYPE_I32, arg++, hi); tcg_out_mov(s, TCG_TYPE_I32, arg++, lo); } else {
@@ -2250,9 +2250,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) lo = lb->addrlo_reg; hi = lb->addrhi_reg; if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { -#ifdef TCG_TARGET_CALL_ALIGN_ARGS - arg |= 1; -#endif + arg |= (TCG_TARGET_CALL_ARG_I64 == TCG_CALL_ARG_EVEN); tcg_out_mov(s, TCG_TYPE_I32, arg++, hi); tcg_out_mov(s, TCG_TYPE_I32, arg++, lo); } else {
@@ -2266,9 +2264,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) if (TCG_TARGET_REG_BITS == 32) { switch (s_bits) { case MO_64: -#ifdef TCG_TARGET_CALL_ALIGN_ARGS - arg |= 1; -#endif + arg |= (TCG_TARGET_CALL_ARG_I64 == TCG_CALL_ARG_EVEN); tcg_out_mov(s, TCG_TYPE_I32, arg++, hi); /* FALLTHRU */ case MO_32:
@@ -2324,9 +2320,8 @@ static bool tcg_out_fail_alignment(TCGContext *s, TCGLabelQemuLdst *l) if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { TCGReg arg = TCG_REG_R4; -#ifdef TCG_TARGET_CALL_ALIGN_ARGS - arg |= 1; -#endif + + arg |= (TCG_TARGET_CALL_ARG_I64 == TCG_CALL_ARG_EVEN); if (l->addrlo_reg != arg) { tcg_out_mov(s, TCG_TYPE_I32, arg, l->addrhi_reg); tcg_out_mov(s, TCG_TYPE_I32, arg + 1, l->addrlo_reg);
Signature · 1 line
2.34.1
PATCH v3 17/45 tcg: Replace TCG_TARGET_EXTEND_ARGS with TCG_TARGET_CALL_ARG_I32
In reply to Richard HendersonFor 64-bit hosts that had TCG_TARGET_EXTEND_ARGS, set TCG_TARGET_CALL_ARG_I32 to TCG_CALL_ARG_EXTEND. Otherwise, use TCG_CALL_ARG_NORMAL.
--- tcg/aarch64/tcg-target.h | 1 + tcg/arm/tcg-target.h | 1 + tcg/i386/tcg-target.h | 1 + tcg/loongarch64/tcg-target.h | 1 + tcg/mips/tcg-target.h | 1 + tcg/riscv/tcg-target.h | 1 + tcg/s390x/tcg-target.h | 2 +- tcg/sparc64/tcg-target.h | 2 +- tcg/tci/tcg-target.h | 1 + tcg/tcg.c | 42 ++++++++++++++++++------------------ tcg/ppc/tcg-target.c.inc | 6 +++++- 11 files changed, 35 insertions(+), 24 deletions(-)
tcg/aarch64/tcg-target.h +1 -0
@@ -52,6 +52,7 @@ typedef enum {#define TCG_REG_CALL_STACK TCG_REG_SP#define TCG_TARGET_STACK_ALIGN 16#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL/* optional instructions */
tcg/arm/tcg-target.h +1 -0
@@ -89,6 +89,7 @@ extern bool use_neon_instructions;/* used for function call generation */ #define TCG_TARGET_STACK_ALIGN 8#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN/* optional instructions */
tcg/i386/tcg-target.h +1 -0
@@ -98,6 +98,7 @@ typedef enum {#else #define TCG_TARGET_CALL_STACK_OFFSET 0#endif +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMALextern bool have_bmi1;
tcg/loongarch64/tcg-target.h +1 -0
@@ -93,6 +93,7 @@ typedef enum {#define TCG_REG_CALL_STACK TCG_REG_SP#define TCG_TARGET_STACK_ALIGN 16#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL/* optional instructions */
tcg/mips/tcg-target.h +1 -0
@@ -88,6 +88,7 @@ typedef enum {# define TCG_TARGET_CALL_STACK_OFFSET 0# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL#endif +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL/* MOVN/MOVZ instructions detection */ #if (defined(__mips_isa_rev) && (__mips_isa_rev >= 1)) || \
tcg/riscv/tcg-target.h +1 -0
@@ -82,6 +82,7 @@ typedef enum {#define TCG_REG_CALL_STACK TCG_REG_SP#define TCG_TARGET_STACK_ALIGN 16#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#if TCG_TARGET_REG_BITS ==32 #define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN#else
tcg/s390x/tcg-target.h +1 -1
@@ -166,9 +166,9 @@ extern uint64_t s390_facilities[3];/* used for function call generation */ #define TCG_TARGET_STACK_ALIGN 8#define TCG_TARGET_CALL_STACK_OFFSET 160 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_EXTEND#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_RET_NORMAL -#define TCG_TARGET_EXTEND_ARGS 1#define TCG_TARGET_HAS_MEMORY_BSWAP 1#define TCG_TARGET_DEFAULT_MO (TCG_MO_ALL & ~TCG_MO_ST_LD)
tcg/sparc64/tcg-target.h +1 -1
@@ -71,7 +71,7 @@ typedef enum {#define TCG_TARGET_STACK_BIAS 2047#define TCG_TARGET_STACK_ALIGN 16#define TCG_TARGET_CALL_STACK_OFFSET (128 + 6*8 + TCG_TARGET_STACK_BIAS) -#define TCG_TARGET_EXTEND_ARGS 1 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_EXTEND#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL#if defined(__VIS__) && __VIS__ >=0x300
tcg/tci/tcg-target.h +1 -0
@@ -158,6 +158,7 @@ typedef enum {/* Used for function call generation. */ #define TCG_TARGET_CALL_STACK_OFFSET 0#define TCG_TARGET_STACK_ALIGN 8 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#if TCG_TARGET_REG_BITS ==32 # define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN#else
tcg/tcg.c +21 -21
61 lines not on this page · raw message
tcg/ppc/tcg-target.c.inc +5 -1
@@ -44,6 +44,11 @@ # endif #endif +#if TCG_TARGET_REG_BITS == 64 +# define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_EXTEND +#else +# define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL +#endif #ifdef _CALL_SYSV # define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN #else
@@ -2520,7 +2525,6 @@ static void tcg_out_nop_fill(tcg_insn_unit *p, int count) /* Parameters for function call generation, used in tcg.c. */ #define TCG_TARGET_STACK_ALIGN 16 -#define TCG_TARGET_EXTEND_ARGS 1 #ifdef _CALL_AIX # define LINK_AREA_SIZE (6 * SZR)
Signature · 1 line
2.34.1
PATCH v3 18/45 tcg: Use TCG_CALL_ARG_EVEN for TCI special case
In reply to Richard HendersonChange 32-bit tci TCG_TARGET_CALL_ARG_I32 to TCG_CALL_ARG_EVEN, to force 32-bit values to be aligned to 64-bit. With a small reorg to the argument processing loop, this neatly replaces an ifdef for CONFIG_TCG_INTERPRETER.
--- tcg/tci/tcg-target.h | 3 ++- tcg/tcg.c | 62 ++++++++++++++++++++++++++------------------ 2 files changed, 39 insertions(+), 26 deletions(-)
tcg/tci/tcg-target.h +2 -1
@@ -158,10 +158,11 @@ typedef enum {/* Used for function call generation. */ #define TCG_TARGET_CALL_STACK_OFFSET 0#define TCG_TARGET_STACK_ALIGN 8 -#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#if TCG_TARGET_REG_BITS ==32 +# define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_EVEN# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN#else +# define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL#endif
tcg/tcg.c +37 -25
73 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v3 19/45 accel/tcg/plugin: Don't search for the function pointer index
In reply to Richard HendersonThe function pointer is immediately after the output and input operands; no need to search.
--- accel/tcg/plugin-gen.c | 29 +++++++++++------------------ 1 file changed, 11 insertions(+), 18 deletions(-)
accel/tcg/plugin-gen.c +11 -18
@@ -381,32 +381,25 @@ static TCGOp *copy_st_ptr(TCGOp **begin_op, TCGOp *op)static TCGOp *copy_call (TCGOp **begin_op,TCGOp *op,void *empty_func,void *func,int *cb_idx) { +TCGOp *old_op; +int func_idx; +/* copy all ops until the call */ do { op =copy_op_nocheck (begin_op, op); }while (op->opc != INDEX_op_call);/* fill in the op call */ - op->param1 = (*begin_op)->param1; - op->param2 = (*begin_op)->param2; + old_op = *begin_op; +TCGOP_CALLI (op) =TCGOP_CALLI (old_op); +TCGOP_CALLO (op) =TCGOP_CALLO (old_op);tcg_debug_assert (op->life ==0 ); -if (*cb_idx ==-1 ) { -int i; -/* -* Instead of working out the position of the callback in args[], just -* look for @empty_func, since it should be a unique pointer. -*/ -for (i =0 ; i <MAX_OPC_PARAM_ARGS ; i++) { -if ((uintptr_t )(*begin_op)->args[i] == (uintptr_t )empty_func) { - *cb_idx = i; -break ; - } - } -tcg_debug_assert (i <MAX_OPC_PARAM_ARGS ); - } - op->args[*cb_idx] = (uintptr_t )func; - op->args[*cb_idx +1 ] = (*begin_op)->args[*cb_idx +1 ]; + func_idx =TCGOP_CALLO (op) +TCGOP_CALLI (op); + *cb_idx = func_idx; + + op->args[func_idx] = (uintptr_t )func; + op->args[func_idx +1 ] = old_op->args[func_idx +1 ];return op; }
Signature · 1 line
2.34.1
PATCH v3 20/45 accel/tcg/plugin: Avoid duplicate copy in copy_call
In reply to Richard HendersonWe copied all of the arguments in copy_op_nocheck. We only need to replace the one argument that we change.
--- accel/tcg/plugin-gen.c | 2 -- 1 file changed, 2 deletions(-)
accel/tcg/plugin-gen.c +0 -2
@@ -397,9 +397,7 @@ static TCGOp *copy_call(TCGOp **begin_op, TCGOp *op, void *empty_func, func_idx =TCGOP_CALLO (op) +TCGOP_CALLI (op); *cb_idx = func_idx; - op->args[func_idx] = (uintptr_t )func; - op->args[func_idx +1 ] = old_op->args[func_idx +1 ];return op; }
Signature · 1 line
2.34.1
Re: [PATCH for-8.0 v3 20/45] accel/tcg/plugin: Avoid duplicate copy in copy_call
Quoted 7 lines · Richard Henderson
We copied all of the arguments in
copy_op_nocheck. We only need to replace the one argument that we change.Signed-off-by Richard Henderson <[email protected]>--- accel/tcg/plugin-gen.c | 2 -- 1 file changed, 2 deletions(-)
PATCH v3 21/45 accel/tcg/plugin: Use copy_op in append_{udata, mem}_cb
In reply to Richard HendersonBetter to re-use the existing function for copying ops.
--- accel/tcg/plugin-gen.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
accel/tcg/plugin-gen.c +8 -8
@@ -415,11 +415,11 @@ static TCGOp *append_udata_cb(const struct qemu_plugin_dyn_cb *cb, op =copy_const_ptr (&begin_op, op, cb->userp);/* copy the ld_i32, but note that we only have to copy it once */ - begin_op =QTAILQ_NEXT (begin_op, link); -tcg_debug_assert (begin_op && begin_op->opc == INDEX_op_ld_i32);if (*cb_idx ==-1 ) { - op =tcg_op_insert_after (tcg_ctx, op, INDEX_op_ld_i32); -memcpy (op->args, begin_op->args,sizeof (op->args)); + op =copy_op (&begin_op, op, INDEX_op_ld_i32); + }else { + begin_op =QTAILQ_NEXT (begin_op, link); +tcg_debug_assert (begin_op && begin_op->opc == INDEX_op_ld_i32); }/* call */
@@ -462,11 +462,11 @@ static TCGOp *append_mem_cb(const struct qemu_plugin_dyn_cb *cb, op =copy_const_ptr (&begin_op, op, cb->userp);/* copy the ld_i32, but note that we only have to copy it once */ - begin_op =QTAILQ_NEXT (begin_op, link); -tcg_debug_assert (begin_op && begin_op->opc == INDEX_op_ld_i32);if (*cb_idx ==-1 ) { - op =tcg_op_insert_after (tcg_ctx, op, INDEX_op_ld_i32); -memcpy (op->args, begin_op->args,sizeof (op->args)); + op =copy_op (&begin_op, op, INDEX_op_ld_i32); + }else { + begin_op =QTAILQ_NEXT (begin_op, link); +tcg_debug_assert (begin_op && begin_op->opc == INDEX_op_ld_i32); }/* extu_tl_i64 */
Signature · 1 line
2.34.1
Re: [PATCH for-8.0 v3 21/45] accel/tcg/plugin: Use copy_op in append_{udata, mem}_cb
Quoted 6 lines · Richard Henderson
Better to re-use the existing function for copying ops.
Signed-off-by Richard Henderson <[email protected]>--- accel/tcg/plugin-gen.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
PATCH v3 22/45 tci: MAX_OPC_PARAM_IARGS is no longer used
In reply to Richard Henderson--- tcg/tci.c | 1 - tcg/tci/tcg-target.c.inc | 4 ---- 2 files changed, 5 deletions(-)
tcg/tci.c +0 -1
@@ -18,7 +18,6 @@*/ #include "qemu/osdep.h" -#include "tcg/tcg.h" /* MAX_OPC_PARAM_IARGS */ #include "exec/cpu_ldst.h" #include "tcg/tcg-op.h" #include "tcg/tcg-ldst.h"
tcg/tci/tcg-target.c.inc +0 -4
@@ -197,10 +197,6 @@ static const int tcg_target_reg_alloc_order[] = { TCG_REG_R0, }; -#if MAX_OPC_PARAM_IARGS != 7 -# error Fix needed, number of supported input arguments changed! -#endif - /* No call arguments via registers. All will be stored on the "stack". */ static const int tcg_target_call_iarg_regs[] = { };
Signature · 1 line
2.34.1
Re: [PATCH for-8.0 v3 22/45] tci: MAX_OPC_PARAM_IARGS is no longer used
Quoted 5 lines · Richard Henderson
Signed-off-by Richard Henderson <[email protected]>--- tcg/tci.c | 1 - tcg/tci/tcg-target.c.inc | 4 ---- 2 files changed, 5 deletions(-)
Since commit 7b7d8b2d9a ("tcg/tci: Use ffi for calls")?
PATCH v3 23/45 tcg: Vary the allocation size for TCGOp
In reply to Richard HendersonWe have been allocating a worst case number of arguments to support calls. Instead, allow the size to vary. By default leave space for 4 args, to maximize reuse, but allow calls to increase the number of args to 32.
--- include/exec/helper-head.h | 2 -- include/tcg/tcg-op.h | 2 +- include/tcg/tcg.h | 54 +++++++++++++-------------------- accel/tcg/plugin-gen.c | 11 ++++--- tcg/optimize.c | 4 +-- tcg/tcg-op-vec.c | 8 ++--- tcg/tcg-op.c | 12 ++++---- tcg/tcg.c | 61 +++++++++++++++++++++++++------------- 8 files changed, 82 insertions(+), 72 deletions(-)
include/exec/helper-head.h +0 -2
@@ -133,6 +133,4 @@#define DEF_HELPER_7 (name, ret, t1, t2, t3, t4, t5, t6, t7) \ DEF_HELPER_FLAGS_7(name, 0, ret, t1, t2, t3, t4, t5, t6, t7) -/* MAX_OPC_PARAM_IARGS must be set to n if last entry is DEF_HELPER_FLAGS_n. */ -#endif /* EXEC_HELPER_HEAD_H */
include/tcg/tcg-op.h +1 -1
@@ -818,7 +818,7 @@ static inline void tcg_gen_plugin_cb_start(unsigned from, unsigned type,static inline void tcg_gen_plugin_cb_end (void ) { -tcg_emit_op (INDEX_op_plugin_cb_end); +tcg_emit_op (INDEX_op_plugin_cb_end,0 ); }#if TARGET_LONG_BITS ==32
include/tcg/tcg.h +21 -33
85 lines not on this page · raw message
accel/tcg/plugin-gen.c +7 -4
@@ -258,10 +258,13 @@ static TCGOp *rm_ops(TCGOp *op)static TCGOp *copy_op_nocheck (TCGOp **begin_op,TCGOp *op) { - *begin_op =QTAILQ_NEXT (*begin_op, link); -tcg_debug_assert (*begin_op); - op =tcg_op_insert_after (tcg_ctx, op, (*begin_op)->opc); -memcpy (op->args, (*begin_op)->args,sizeof (op->args)); +TCGOp *old_op =QTAILQ_NEXT (*begin_op, link); +unsigned nargs = old_op->nargs; + + *begin_op = old_op; + op =tcg_op_insert_after (tcg_ctx, op, old_op->opc, nargs); +memcpy (op->args, old_op->args,sizeof (op->args[0 ]) * nargs); +return op; }
tcg/optimize.c +2 -2
@@ -962,7 +962,7 @@ static bool fold_addsub2(OptContext *ctx, TCGOp *op, bool add) rh = op->args[1 ];/* The proper opcode is supplied by tcg_opt_gen_mov. */ - op2 =tcg_op_insert_before (ctx->tcg, op,0 ); + op2 =tcg_op_insert_before (ctx->tcg, op,0 ,2 );tcg_opt_gen_movi (ctx, op, rl, al);tcg_opt_gen_movi (ctx, op2, rh, ah);
@@ -1613,7 +1613,7 @@ static bool fold_multiply2(OptContext *ctx, TCGOp *op) rh = op->args[1 ];/* The proper opcode is supplied by tcg_opt_gen_mov. */ - op2 =tcg_op_insert_before (ctx->tcg, op,0 ); + op2 =tcg_op_insert_before (ctx->tcg, op,0 ,2 );tcg_opt_gen_movi (ctx, op, rl, l);tcg_opt_gen_movi (ctx, op2, rh, h);
tcg/tcg-op-vec.c +4 -4
@@ -152,7 +152,7 @@ bool tcg_can_emit_vecop_list(const TCGOpcode *list,void vec_gen_2 (TCGOpcode opc,TCGType type,unsigned vece,TCGArg r,TCGArg a) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,2 );TCGOP_VECL (op) = type -TCG_TYPE_V64 ;TCGOP_VECE (op) = vece; op->args[0 ] = r;
@@ -162,7 +162,7 @@ void vec_gen_2(TCGOpcode opc, TCGType type, unsigned vece, TCGArg r, TCGArg a)void vec_gen_3 (TCGOpcode opc,TCGType type,unsigned vece,TCGArg r,TCGArg a,TCGArg b) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,3 );TCGOP_VECL (op) = type -TCG_TYPE_V64 ;TCGOP_VECE (op) = vece; op->args[0 ] = r;
@@ -173,7 +173,7 @@ void vec_gen_3(TCGOpcode opc, TCGType type, unsigned vece,void vec_gen_4 (TCGOpcode opc,TCGType type,unsigned vece,TCGArg r,TCGArg a,TCGArg b,TCGArg c) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,4 );TCGOP_VECL (op) = type -TCG_TYPE_V64 ;TCGOP_VECE (op) = vece; op->args[0 ] = r;
@@ -185,7 +185,7 @@ void vec_gen_4(TCGOpcode opc, TCGType type, unsigned vece,static void vec_gen_6 (TCGOpcode opc,TCGType type,unsigned vece,TCGArg r,TCGArg a,TCGArg b,TCGArg c,TCGArg d,TCGArg e) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,6 );TCGOP_VECL (op) = type -TCG_TYPE_V64 ;TCGOP_VECE (op) = vece; op->args[0 ] = r;
tcg/tcg-op.c +6 -6
@@ -33,20 +33,20 @@void tcg_gen_op1 (TCGOpcode opc,TCGArg a1) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,1 ); op->args[0 ] = a1; }void tcg_gen_op2 (TCGOpcode opc,TCGArg a1,TCGArg a2) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,2 ); op->args[0 ] = a1; op->args[1 ] = a2; }void tcg_gen_op3 (TCGOpcode opc,TCGArg a1,TCGArg a2,TCGArg a3) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,3 ); op->args[0 ] = a1; op->args[1 ] = a2; op->args[2 ] = a3;
@@ -54,7 +54,7 @@ void tcg_gen_op3(TCGOpcode opc, TCGArg a1, TCGArg a2, TCGArg a3)void tcg_gen_op4 (TCGOpcode opc,TCGArg a1,TCGArg a2,TCGArg a3,TCGArg a4) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,4 ); op->args[0 ] = a1; op->args[1 ] = a2; op->args[2 ] = a3;
@@ -64,7 +64,7 @@ void tcg_gen_op4(TCGOpcode opc, TCGArg a1, TCGArg a2, TCGArg a3, TCGArg a4)void tcg_gen_op5 (TCGOpcode opc,TCGArg a1,TCGArg a2,TCGArg a3,TCGArg a4,TCGArg a5) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,5 ); op->args[0 ] = a1; op->args[1 ] = a2; op->args[2 ] = a3;
@@ -75,7 +75,7 @@ void tcg_gen_op5(TCGOpcode opc, TCGArg a1, TCGArg a2, TCGArg a3,void tcg_gen_op6 (TCGOpcode opc,TCGArg a1,TCGArg a2,TCGArg a3,TCGArg a4,TCGArg a5,TCGArg a6) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,6 ); op->args[0 ] = a1; op->args[1 ] = a2; op->args[2 ] = a3;
tcg/tcg.c +41 -20
124 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v3 24/45 tcg: Use output_pref wrapper function
In reply to Richard HendersonWe will shortly have the possibility of more that two outputs, though only for calls (for which preferences are moot). Avoid direct references to op->output_pref[] when possible.
--- include/tcg/tcg.h | 5 +++++ tcg/tcg.c | 34 ++++++++++++++++++---------------- 2 files changed, 23 insertions(+), 16 deletions(-)
include/tcg/tcg.h +5 -0
@@ -518,6 +518,11 @@ typedef struct TCGOp {/* Make sure operands fit in the bitfields above. */ QEMU_BUILD_BUG_ON (NB_OPS > (1 <<8 )); +static inline TCGRegSet output_pref (const TCGOp *op,unsigned i) +{ +return i <ARRAY_SIZE (op->output_pref) ? op->output_pref[i] :0 ; +} + typedef struct TCGProfile { int64_t cpu_exec_time; int64_t tb_count1;
tcg/tcg.c +18 -16
@@ -1957,7 +1957,7 @@ static void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs)if (have_prefs) {for (i =0 ; i < nb_oargs; ++i) { -TCGRegSet set = op->output_pref[i]; +TCGRegSet set =output_pref (op, i);if (i ==0 ) {ne_fprintf (f," pref=" );
@@ -2627,11 +2627,11 @@ static void liveness_pass_1(TCGContext *s) } ts->state =TS_DEAD ;la_reset_pref (ts); - -/* Not used -- it will be tcg_target_call_oarg_regs[i]. */ - op->output_pref[i] =0 ; } +/* Not used -- it will be tcg_target_call_oarg_reg(). */ +memset (op->output_pref,0 ,sizeof (op->output_pref)); +if (!(call_flags & (TCG_CALL_NO_WRITE_GLOBALS |TCG_CALL_NO_READ_GLOBALS ))) {la_global_kill (s, nb_globals);
@@ -2793,7 +2793,9 @@ static void liveness_pass_1(TCGContext *s) ts =arg_temp (op->args[i]);/* Remember the preference of the uses that followed. */ - op->output_pref[i] = *la_temp_pref (ts); +if (i <ARRAY_SIZE (op->output_pref)) { + op->output_pref[i] = *la_temp_pref (ts); + }/* Output args are dead. */ if (ts->state &TS_DEAD ) {
@@ -2863,7 +2865,7 @@ static void liveness_pass_1(TCGContext *s) set &= ct->regs;if (ct->ialias) { - set &= op->output_pref[ct->alias_index]; + set &=output_pref (op, ct->alias_index); }/* If the combination is not possible, restart. */ if (set ==0 ) {
@@ -3581,7 +3583,7 @@ static void tcg_reg_alloc_mov(TCGContext *s, const TCGOp *op)TCGType otype, itype; allocated_regs = s->reserved_regs; - preferred_regs = op->output_pref[0 ]; + preferred_regs =output_pref (op,0 ); ots =arg_temp (op->args[0 ]); ts =arg_temp (op->args[1 ]);
@@ -3696,7 +3698,7 @@ static void tcg_reg_alloc_dup(TCGContext *s, const TCGOp *op)if (IS_DEAD_ARG (1 )) {temp_dead (s, its); } -tcg_reg_alloc_do_movi (s, ots, val, arg_life, op->output_pref[0 ]); +tcg_reg_alloc_do_movi (s, ots, val, arg_life,output_pref (op,0 ));return ; }
@@ -3712,7 +3714,7 @@ static void tcg_reg_alloc_dup(TCGContext *s, const TCGOp *op)tcg_regset_set_reg (allocated_regs, its->reg); } ots->reg =tcg_reg_alloc (s, dup_out_regs, allocated_regs, - op->output_pref[0 ], ots->indirect_base); +output_pref (op,0 ), ots->indirect_base); ots->val_type =TEMP_VAL_REG ; ots->mem_coherent =0 ; s->reg_to_temp[ots->reg] = ots;
@@ -3831,7 +3833,7 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op)switch (arg_ct->pair) {case 0 :/* not paired */ if (arg_ct->ialias) { - i_preferred_regs = op->output_pref[arg_ct->alias_index]; + i_preferred_regs =output_pref (op, arg_ct->alias_index);/* * If the input is readonly, then it cannot also be an
@@ -3882,7 +3884,7 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op) ts2 = i1 != i2 ?arg_temp (op->args[i2]) :NULL ;if (arg_ct->ialias) { - i_preferred_regs = op->output_pref[arg_ct->alias_index]; + i_preferred_regs =output_pref (op, arg_ct->alias_index); if (IS_DEAD_ARG(i1) && IS_DEAD_ARG(i2) && !temp_readonly(ts) &&
@@ -3928,7 +3930,7 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op)case 3 :/* ialias with second output, no first input */ tcg_debug_assert (arg_ct->ialias); - i_preferred_regs = op->output_pref[arg_ct->alias_index]; + i_preferred_regs =output_pref (op, arg_ct->alias_index); allocate_new_reg = true; if (IS_DEAD_ARG(i) &&
@@ -4018,10 +4020,10 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op) }else if (arg_ct->newreg) { reg =tcg_reg_alloc (s, arg_ct->regs, i_allocated_regs | o_allocated_regs, - op->output_pref[k], ts->indirect_base); +output_pref (op, k), ts->indirect_base); }else { reg =tcg_reg_alloc (s, arg_ct->regs, o_allocated_regs, - op->output_pref[k], ts->indirect_base); +output_pref (op, k), ts->indirect_base); }break ;
@@ -4032,7 +4034,7 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op)break ; } reg =tcg_reg_alloc_pair (s, arg_ct->regs, o_allocated_regs, - op->output_pref[k], ts->indirect_base); +output_pref (op, k), ts->indirect_base);break ;case 2 :/* second of pair */
@@ -4123,7 +4125,7 @@ static bool tcg_reg_alloc_dup2(TCGContext *s, const TCGOp *op) } ots->reg =tcg_reg_alloc (s, dup_out_regs, allocated_regs, - op->output_pref[0 ], ots->indirect_base); +output_pref (op,0 ), ots->indirect_base); ots->val_type =TEMP_VAL_REG ; ots->mem_coherent =0 ; s->reg_to_temp[ots->reg] = ots;
Signature · 1 line
2.34.1
Re: [PATCH for-8.0 v3 24/45] tcg: Use output_pref wrapper function
Quoted 9 lines · Richard Henderson
We will shortly have the possibility of more that two outputs, though only for calls (for which preferences are moot). Avoid direct references to op->output_pref[] when possible.
Signed-off-by Richard Henderson <[email protected]>--- include/tcg/tcg.h | 5 +++++ tcg/tcg.c | 34 ++++++++++++++++++---------------- 2 files changed, 23 insertions(+), 16 deletions(-)
PATCH v3 25/45 tcg: Reorg function calls
In reply to Richard HendersonPre-compute the function call layout for each helper at startup. Drop TCG_CALL_DUMMY_ARG, as we no longer need to leave gaps in the op->args[] array. This allows several places to stop checking for NULL TCGTemp, to which TCG_CALL_DUMMY_ARG mapped.
For tcg_gen_callN, loop over the arguments once. Allocate the TCGOp for the call early but delay emitting it, collecting arguments first. This allows the argument processing loop to emit code for extensions and have them sequenced before the call.
For tcg_reg_alloc_call, loop over the arguments in reverse order, which allows stack slots to be filled first naturally.
--- include/exec/helper-head.h | 2 + include/tcg/tcg.h | 5 +- tcg/tcg-internal.h | 22 +- tcg/optimize.c | 6 +- tcg/tcg.c | 601 ++++++++++++++++++++++--------------- 5 files changed, 387 insertions(+), 249 deletions(-)
include/exec/helper-head.h +2 -0
@@ -133,4 +133,6 @@#define DEF_HELPER_7 (name, ret, t1, t2, t3, t4, t5, t6, t7) \ DEF_HELPER_FLAGS_7(name, 0, ret, t1, t2, t3, t4, t5, t6, t7) +/* MAX_CALL_IARGS must be set to n if last entry is DEF_HELPER_FLAGS_n. */ +#endif /* EXEC_HELPER_HEAD_H */
include/tcg/tcg.h +2 -3
@@ -38,6 +38,8 @@/* XXX: make safe guess about sizes */ #define MAX_OP_PER_INSTR 266 +#define MAX_CALL_IARGS 7 +#define CPU_TEMP_BUF_NLONGS 128#define TCG_STATIC_FRAME_SIZE (CPU_TEMP_BUF_NLONGS * sizeof(long))
@@ -411,9 +413,6 @@ typedef TCGv_ptr TCGv_env;#define TCG_CALL_NO_RWG_SE (TCG_CALL_NO_RWG | TCG_CALL_NO_SE)#define TCG_CALL_NO_WG_SE (TCG_CALL_NO_WG | TCG_CALL_NO_SE) -/* Used to align parameters. See the comment before tcgv_i32_temp. */ -#define TCG_CALL_DUMMY_ARG ((TCGArg)0) -/* * Flags for the bswap opcodes. * If IZ, the input is zero-extended, otherwise unknown.
tcg/tcg-internal.h +20 -2
@@ -42,11 +42,29 @@ typedef enum {TCG_CALL_ARG_EXTEND_S ,/* ... as a sign-extended i64 */ } TCGCallArgumentKind; +typedef struct TCGCallArgumentLoc { +TCGCallArgumentKind kind :8 ; +unsigned arg_slot :8 ; +unsigned ref_slot :8 ; +unsigned arg_idx :4 ; +unsigned tmp_subindex :2 ; +}TCGCallArgumentLoc ; + +/* Avoid "unsigned < 0 is always false" Werror, when iarg_regs is empty. */ +#define REG_P (L ) \ + ((int)(L)->arg_slot < (int)ARRAY_SIZE(tcg_target_call_iarg_regs)) +typedef struct TCGHelperInfo {void *func;const char *name; -unsigned flags; -unsigned typemask; +unsigned typemask :32 ; +unsigned flags :8 ; +unsigned nr_in :8 ; +unsigned nr_out :8 ; +TCGCallReturnKind out_kind :8 ; + +/* Maximum physical arguments are constrained by TCG_TYPE_I128. */ +TCGCallArgumentLoc in[MAX_CALL_IARGS * (128 /TCG_TARGET_REG_BITS )]; }TCGHelperInfo ;extern TCGContext tcg_init_ctx;
tcg/optimize.c +2 -4
@@ -667,9 +667,7 @@ static void init_arguments(OptContext *ctx, TCGOp *op, int nb_args) {for (int i =0 ; i < nb_args; i++) {TCGTemp *ts =arg_temp (op->args[i]); -if (ts) { -init_ts_info (ctx, ts); - } +init_ts_info (ctx, ts); } }
@@ -680,7 +678,7 @@ static void copy_propagate(OptContext *ctx, TCGOp *op,for (int i = nb_oargs; i < nb_oargs + nb_iargs; i++) {TCGTemp *ts =arg_temp (op->args[i]); -if (ts &&ts_is_copy (ts)) { +if (ts_is_copy (ts)) { op->args[i] =temp_arg (find_better_copy (s, ts)); } }
tcg/tcg.c +361 -240
741 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v3 26/45 tcg: Move ffi_cif pointer into TCGHelperInfo
In reply to Richard HendersonInstead of requiring a separate hash table lookup, put a pointer to the CIF into TCGHelperInfo.
--- tcg/tcg-internal.h | 7 +++ tcg/tcg.c | 129 +++++++++++++++++++++++++-------------------- 2 files changed, 78 insertions(+), 58 deletions(-)
tcg/tcg-internal.h +7 -0
@@ -25,6 +25,10 @@#ifndef TCG_INTERNAL_H #define TCG_INTERNAL_H +#ifdef CONFIG_TCG_INTERPRETER +#include <ffi.h> +#endif +#define TCG_HIGHWATER 1024/*
@@ -57,6 +61,9 @@ typedef struct TCGCallArgumentLoc { typedef struct TCGHelperInfo { void *func; const char *name; +#ifdef CONFIG_TCG_INTERPRETER + ffi_cif *cif; +#endif unsigned typemask : 32; unsigned flags : 8; unsigned nr_in : 8;
tcg/tcg.c +71 -58
154 lines not on this page · raw message
Signature · 1 line
2.34.1
cover tcg: Move ffi_cif pointer into TCGHelperInfo (splitted)
Hi Richard,
Your "Move ffi_cif pointer into TCGHelperInfo" patch was not obvious (to me) at first, so I split it in 3 trivial patches.
Philippe Mathieu-Daudé (2):
tcg: Convert typecode_to_ffi from array to function
tcg: Factor init_ffi_layouts() out of tcg_context_init()
Richard Henderson (1):
tcg: Move ffi_cif pointer into TCGHelperInfo
tcg/tcg-internal.h | 7 +++ tcg/tcg.c | 125 +++++++++++++++++++++++++-------------------- 2 files changed, 78 insertions(+), 54 deletions(-)
Signature · 1 line
2.38.1
PATCH 1/3 tcg: Convert typecode_to_ffi from array to function
In the unlikely case of invalid typecode mask, the function will abort instead of returning a NULL pointer.
Message-Id: <[email protected]>
--- tcg/tcg.c | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-)
tcg/tcg.c +20 -10
@@ -554,14 +554,24 @@ static GHashTable *helper_table;#ifdef CONFIG_TCG_INTERPRETER static GHashTable *ffi_table; -static ffi_type *const typecode_to_ffi[8 ] = { - [dh_typecode_void] = &ffi_type_void, - [dh_typecode_i32] = &ffi_type_uint32, - [dh_typecode_s32] = &ffi_type_sint32, - [dh_typecode_i64] = &ffi_type_uint64, - [dh_typecode_s64] = &ffi_type_sint64, - [dh_typecode_ptr] = &ffi_type_pointer, -}; +static ffi_type *typecode_to_ffi (int argmask) +{ +switch (argmask) { +case dh_typecode_void: +return &ffi_type_void; +case dh_typecode_i32: +return &ffi_type_uint32; +case dh_typecode_s32: +return &ffi_type_sint32; +case dh_typecode_i64: +return &ffi_type_uint64; +case dh_typecode_s64: +return &ffi_type_sint64; +case dh_typecode_ptr: +return &ffi_type_pointer; + } +g_assert_not_reached (); +}#endif typedef struct TCGCumulativeArgs {
@@ -764,14 +774,14 @@ static void tcg_context_init(unsigned max_cpus) nargs =DIV_ROUND_UP (nargs,3 ); ca =g_malloc0 (sizeof (*ca) + nargs *sizeof (ffi_type *)); - ca->cif.rtype = typecode_to_ffi[typemask &7 ]; + ca->cif.rtype =typecode_to_ffi (typemask &7 ); ca->cif.nargs = nargs;if (nargs !=0 ) { ca->cif.arg_types = ca->args;for (int j =0 ; j < nargs; ++j) {int typecode =extract32 (typemask, (j +1 ) *3 ,3 ); - ca->args[j] = typecode_to_ffi[typecode]; + ca->args[j] =typecode_to_ffi (typecode); } }
Signature · 1 line
2.38.1
PATCH 2/3 tcg: Factor init_ffi_layouts() out of tcg_context_init()
In reply to Philippe Mathieu-DaudéMessage-Id: <[email protected]>
--- tcg/tcg.c | 83 +++++++++++++++++++++++++++++-------------------------- 1 file changed, 44 insertions(+), 39 deletions(-)
tcg/tcg.c +44 -39
@@ -572,7 +572,49 @@ static ffi_type *typecode_to_ffi(int argmask) }g_assert_not_reached (); } -#endif + +static void init_ffi_layouts (void ) +{ +/* g_direct_hash/equal for direct comparisons on uint32_t. */ + ffi_table =g_hash_table_new (NULL ,NULL ); +for (int i =0 ; i <ARRAY_SIZE (all_helpers); ++i) { +uint32_t typemask = all_helpers[i].typemask; +gpointer hash = (gpointer )(uintptr_t )typemask; +struct { +ffi_cif cif; +ffi_type *args[]; + } *ca; +ffi_status status; +int nargs; + +if (g_hash_table_lookup (ffi_table, hash)) { +continue ; + } + +/* Ignoring the return type, find the last non-zero field. */ + nargs =32 -clz32 (typemask >>3 ); + nargs =DIV_ROUND_UP (nargs,3 ); + + ca =g_malloc0 (sizeof (*ca) + nargs *sizeof (ffi_type *)); + ca->cif.rtype =typecode_to_ffi (typemask &7 ); + ca->cif.nargs = nargs; + +if (nargs !=0 ) { + ca->cif.arg_types = ca->args; +for (int j =0 ; j < nargs; ++j) { +int typecode =extract32 (typemask, (j +1 ) *3 ,3 ); + ca->args[j] =typecode_to_ffi (typecode); + } + } + + status =ffi_prep_cif (&ca->cif,FFI_DEFAULT_ABI , nargs, + ca->cif.rtype, ca->cif.arg_types); +assert (status ==FFI_OK ); + +g_hash_table_insert (ffi_table, hash, (gpointer)&ca->cif); + } +} +#endif /* CONFIG_TCG_INTERPRETER */ typedef struct TCGCumulativeArgs { int arg_idx; /* tcg_gen_callN args[] */
@@ -753,44 +795,7 @@ static void tcg_context_init(unsigned max_cpus) }#ifdef CONFIG_TCG_INTERPRETER -/* g_direct_hash/equal for direct comparisons on uint32_t. */ - ffi_table =g_hash_table_new (NULL ,NULL ); -for (i =0 ; i <ARRAY_SIZE (all_helpers); ++i) { -struct { -ffi_cif cif; -ffi_type *args[]; - } *ca; -uint32_t typemask = all_helpers[i].typemask; -gpointer hash = (gpointer )(uintptr_t )typemask; -ffi_status status; -int nargs; - -if (g_hash_table_lookup (ffi_table, hash)) { -continue ; - } - -/* Ignoring the return type, find the last non-zero field. */ - nargs =32 -clz32 (typemask >>3 ); - nargs =DIV_ROUND_UP (nargs,3 ); - - ca =g_malloc0 (sizeof (*ca) + nargs *sizeof (ffi_type *)); - ca->cif.rtype =typecode_to_ffi (typemask &7 ); - ca->cif.nargs = nargs; - -if (nargs !=0 ) { - ca->cif.arg_types = ca->args; -for (int j =0 ; j < nargs; ++j) { -int typecode =extract32 (typemask, (j +1 ) *3 ,3 ); - ca->args[j] =typecode_to_ffi (typecode); - } - } - - status =ffi_prep_cif (&ca->cif,FFI_DEFAULT_ABI , nargs, - ca->cif.rtype, ca->cif.arg_types); -assert (status ==FFI_OK ); - -g_hash_table_insert (ffi_table, hash, (gpointer)&ca->cif); - } +init_ffi_layouts ();#endif tcg_target_init (s);
Signature · 1 line
2.38.1
PATCH 3/3 tcg: Move ffi_cif pointer into TCGHelperInfo
In reply to Philippe Mathieu-DaudéFrom: Richard Henderson <[email protected]>
Instead of requiring a separate hash table lookup, put a pointer to the CIF into TCGHelperInfo.
Message-Id: <[email protected]>
--- tcg/tcg-internal.h | 7 +++++++ tcg/tcg.c | 26 ++++++++++++++------------ 2 files changed, 21 insertions(+), 12 deletions(-)
tcg/tcg-internal.h +7 -0
@@ -25,6 +25,10 @@#ifndef TCG_INTERNAL_H #define TCG_INTERNAL_H +#ifdef CONFIG_TCG_INTERPRETER +#include <ffi.h> +#endif +#define TCG_HIGHWATER 1024/*
@@ -57,6 +61,9 @@ typedef struct TCGCallArgumentLoc { typedef struct TCGHelperInfo { void *func; const char *name; +#ifdef CONFIG_TCG_INTERPRETER + ffi_cif *cif; +#endif unsigned typemask : 32; unsigned flags : 8; unsigned nr_in : 8;
tcg/tcg.c +14 -12
@@ -552,8 +552,6 @@ static TCGHelperInfo all_helpers[] = {static GHashTable *helper_table;#ifdef CONFIG_TCG_INTERPRETER -static GHashTable *ffi_table; -static ffi_type *typecode_to_ffi (int argmask) {switch (argmask) {
@@ -576,9 +574,11 @@ static ffi_type *typecode_to_ffi(int argmask)static void init_ffi_layouts (void ) {/* g_direct_hash/equal for direct comparisons on uint32_t. */ - ffi_table =g_hash_table_new (NULL ,NULL ); +GHashTable *ffi_table =g_hash_table_new (NULL ,NULL ); +for (int i =0 ; i <ARRAY_SIZE (all_helpers); ++i) { -uint32_t typemask = all_helpers[i].typemask; +TCGHelperInfo *info = &all_helpers[i]; +unsigned typemask = info->typemask;gpointer hash = (gpointer )(uintptr_t )typemask; struct { ffi_cif cif;
@@ -586,8 +586,11 @@ static void init_ffi_layouts(void) } *ca;ffi_status status;int nargs; +ffi_cif *cif; -if (g_hash_table_lookup (ffi_table, hash)) { + cif =g_hash_table_lookup (ffi_table, hash); +if (cif) { + info->cif = cif;continue ; }
@@ -611,8 +614,12 @@ static void init_ffi_layouts(void) ca->cif.rtype, ca->cif.arg_types);assert (status ==FFI_OK ); - g_hash_table_insert(ffi_table, hash, (gpointer)&ca->cif); + cif = &ca->cif; + info->cif = cif; + g_hash_table_insert(ffi_table, hash, (gpointer)cif); } + +g_hash_table_destroy (ffi_table); }#endif /* CONFIG_TCG_INTERPRETER */
@@ -4413,12 +4420,7 @@ static void tcg_reg_alloc_call(TCGContext *s, TCGOp *op) }#ifdef CONFIG_TCG_INTERPRETER - { -gpointer hash = (gpointer )(uintptr_t )info->typemask; -ffi_cif *cif =g_hash_table_lookup (ffi_table, hash); -assert (cif !=NULL ); -tcg_out_call (s,tcg_call_func (op), cif); - } +tcg_out_call (s,tcg_call_func (op), info->cif);#else tcg_out_call (s,tcg_call_func (op));#endif
Signature · 1 line
2.38.1
Re: [PATCH 3/3] tcg: Move ffi_cif pointer into TCGHelperInfo
tcg/tcg.c
Quoted 43 lines · Philippe Mathieu-Daudé
From: Richard Henderson <[email protected]>
Instead of requiring a separate hash table lookup, put a pointer to the CIF into TCGHelperInfo.
Signed-off-by Richard Henderson <[email protected]>Message-Id: <[email protected]>
[PMD: Split from bigger patch]Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- tcg/tcg-internal.h | 7 +++++++ tcg/tcg.c | 26 ++++++++++++++------------ 2 files changed, 21 insertions(+), 12 deletions(-)tcg/tcg-internal.h +7
-0@@ -25,6 +25,10 @@#ifndef TCG_INTERNAL_H #define TCG_INTERNAL_H +#ifdef CONFIG_TCG_INTERPRETER +#include <ffi.h> +#endif +#define TCG_HIGHWATER 1024/* @@ -57,6 +61,9 @@ typedef struct TCGCallArgumentLoc { typedef struct TCGHelperInfo { void *func; const char *name; +#ifdef CONFIG_TCG_INTERPRETER + ffi_cif *cif; +#endif unsigned typemask : 32; unsigned flags : 8; unsigned nr_in : 8;tcg/tcg.c +0
-0
I forgot this hunk:
-- >8 --
@@ -62,10 +62,6 @@
#include "tcg/tcg-ldst.h"
#include "tcg-internal.h"
-#ifdef CONFIG_TCG_INTERPRETER
-#include <ffi.h>
-#endif
-
---
Quoted 64 lines
tcg/tcg.c +14
-12@@ -552,8 +552,6 @@ static TCGHelperInfo all_helpers[] = {static GHashTable *helper_table;#ifdef CONFIG_TCG_INTERPRETER -static GHashTable *ffi_table; -static ffi_type *typecode_to_ffi (int argmask) {switch (argmask) {@@ -576,9 +574,11 @@ static ffi_type *typecode_to_ffi(int argmask)static void init_ffi_layouts (void ) {/* g_direct_hash/equal for direct comparisons on uint32_t. */ - ffi_table =g_hash_table_new (NULL ,NULL ); +GHashTable *ffi_table =g_hash_table_new (NULL ,NULL ); +for (int i =0 ; i <ARRAY_SIZE (all_helpers); ++i) { -uint32_t typemask = all_helpers[i].typemask; +TCGHelperInfo *info = &all_helpers[i]; +unsigned typemask = info->typemask;gpointer hash = (gpointer )(uintptr_t )typemask; struct { ffi_cif cif;@@ -586,8 +586,11 @@ static void init_ffi_layouts(void) } *ca;ffi_status status;int nargs; +ffi_cif *cif; -if (g_hash_table_lookup (ffi_table, hash)) { + cif =g_hash_table_lookup (ffi_table, hash); +if (cif) { + info->cif = cif;continue ; }@@ -611,8 +614,12 @@ static void init_ffi_layouts(void) ca->cif.rtype, ca->cif.arg_types);assert (status ==FFI_OK ); - g_hash_table_insert(ffi_table, hash, (gpointer)&ca->cif); + cif = &ca->cif; + info->cif = cif; + g_hash_table_insert(ffi_table, hash, (gpointer)cif); } + +g_hash_table_destroy (ffi_table); }#endif /* CONFIG_TCG_INTERPRETER */ @@ -4413,12 +4420,7 @@ static void tcg_reg_alloc_call(TCGContext *s, TCGOp *op) }#ifdef CONFIG_TCG_INTERPRETER - { -gpointer hash = (gpointer )(uintptr_t )info->typemask; -ffi_cif *cif =g_hash_table_lookup (ffi_table, hash); -assert (cif !=NULL ); -tcg_out_call (s,tcg_call_func (op), cif); - } +tcg_out_call (s,tcg_call_func (op), info->cif);#else tcg_out_call (s,tcg_call_func (op));#endif
PATCH v3 27/45 tcg/aarch64: Merge tcg_out_callr into tcg_out_call
In reply to Richard HendersonThere is only one use, and BLR is perhaps even more self-documentary than CALLR.
--- tcg/aarch64/tcg-target.c.inc | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-)
tcg/aarch64/tcg-target.c.inc +1 -6
@@ -1336,11 +1336,6 @@ static void tcg_out_goto_long(TCGContext *s, const tcg_insn_unit *target) } } -static inline void tcg_out_callr(TCGContext *s, TCGReg reg) -{ - tcg_out_insn(s, 3207, BLR, reg); -} - static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target) { ptrdiff_t offset = tcg_pcrel_diff(s, target) >> 2;
@@ -1348,7 +1343,7 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target) tcg_out_insn(s, 3206, BL, offset); } else { tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_TMP, (intptr_t)target); - tcg_out_callr(s, TCG_REG_TMP); + tcg_out_insn(s, 3207, BLR, TCG_REG_TMP); } }
Signature · 1 line
2.34.1
PATCH v3 28/45 tcg: Add TCGHelperInfo argument to tcg_out_call
In reply to Richard HendersonThis eliminates an ifdef for TCI, and will be required for expanding the call for TCGv_i128.
--- tcg/tcg.c | 12 ++---------- tcg/aarch64/tcg-target.c.inc | 12 +++++++++--- tcg/arm/tcg-target.c.inc | 10 ++++++++-- tcg/i386/tcg-target.c.inc | 5 +++-- tcg/loongarch64/tcg-target.c.inc | 7 ++++--- tcg/mips/tcg-target.c.inc | 3 ++- tcg/ppc/tcg-target.c.inc | 7 ++++--- tcg/riscv/tcg-target.c.inc | 7 ++++--- tcg/s390x/tcg-target.c.inc | 12 +++++++++--- tcg/sparc64/tcg-target.c.inc | 3 ++- tcg/tci/tcg-target.c.inc | 3 ++- 11 files changed, 49 insertions(+), 32 deletions(-)
tcg/tcg.c +2 -10
@@ -145,12 +145,8 @@ static void tcg_out_st(TCGContext *s, TCGType type, TCGReg arg, TCGReg arg1, intptr_t arg2);static bool tcg_out_sti (TCGContext *s,TCGType type,TCGArg val,TCGReg base,intptr_t ofs); -#ifdef CONFIG_TCG_INTERPRETER static void tcg_out_call (TCGContext *s,const tcg_insn_unit *target, -ffi_cif *cif); -#else -static void tcg_out_call (TCGContext *s,const tcg_insn_unit *target); -#endif +const TCGHelperInfo *info);static bool tcg_target_const_match (int64_t val,TCGType type,int ct);#ifdef TCG_TARGET_NEED_LDST_LABELS static int tcg_out_ldst_finalize (TCGContext *s);
@@ -4415,11 +4411,7 @@ static void tcg_reg_alloc_call(TCGContext *s, TCGOp *op)save_globals (s, allocated_regs); } -#ifdef CONFIG_TCG_INTERPRETER -tcg_out_call (s,tcg_call_func (op), info->cif); -#else -tcg_out_call (s,tcg_call_func (op)); -#endif +tcg_out_call (s,tcg_call_func (op), info);/* Assign output registers and emit moves if needed. */ switch (info->out_kind) {
tcg/aarch64/tcg-target.c.inc +9 -3
@@ -1336,7 +1336,7 @@ static void tcg_out_goto_long(TCGContext *s, const tcg_insn_unit *target) } } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target) +static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *target) { ptrdiff_t offset = tcg_pcrel_diff(s, target) >> 2; if (offset == sextract64(offset, 0, 26)) {
@@ -1347,6 +1347,12 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target) } } +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target, + const TCGHelperInfo *info) +{ + tcg_out_call_int(s, target); +} + void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx, uintptr_t jmp_rw, uintptr_t addr) {
@@ -1594,7 +1600,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) tcg_out_mov(s, TARGET_LONG_BITS == 64, TCG_REG_X1, lb->addrlo_reg); tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_X2, oi); tcg_out_adr(s, TCG_REG_X3, lb->raddr); - tcg_out_call(s, qemu_ld_helpers[opc & MO_SIZE]); + tcg_out_call_int(s, qemu_ld_helpers[opc & MO_SIZE]); if (opc & MO_SIGN) { tcg_out_sxt(s, lb->type, size, lb->datalo_reg, TCG_REG_X0); } else {
@@ -1620,7 +1626,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) tcg_out_mov(s, size == MO_64, TCG_REG_X2, lb->datalo_reg); tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_X3, oi); tcg_out_adr(s, TCG_REG_X4, lb->raddr); - tcg_out_call(s, qemu_st_helpers[opc & MO_SIZE]); + tcg_out_call_int(s, qemu_st_helpers[opc & MO_SIZE]); tcg_out_goto(s, lb->raddr); return true; }
tcg/arm/tcg-target.c.inc +8 -2
@@ -1131,7 +1131,7 @@ static void tcg_out_goto(TCGContext *s, ARMCond cond, const tcg_insn_unit *addr) * The call case is mostly used for helpers - so it's not unreasonable * for them to be beyond branch range. */ -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *addr) +static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *addr) { intptr_t addri = (intptr_t)addr; ptrdiff_t disp = tcg_pcrel_diff(s, addr);
@@ -1150,6 +1150,12 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *addr) tcg_out_blx_reg(s, COND_AL, TCG_REG_TMP); } +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *addr, + const TCGHelperInfo *info) +{ + tcg_out_call_int(s, addr); +} + static void tcg_out_goto_label(TCGContext *s, ARMCond cond, TCGLabel *l) { if (l->has_value) {
@@ -1515,7 +1521,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) argreg = tcg_out_arg_reg32(s, argreg, TCG_REG_R14); /* Use the canonical unsigned helpers and minimize icache usage. */ - tcg_out_call(s, qemu_ld_helpers[opc & MO_SIZE]); + tcg_out_call_int(s, qemu_ld_helpers[opc & MO_SIZE]); datalo = lb->datalo_reg; datahi = lb->datahi_reg;
tcg/i386/tcg-target.c.inc +3 -2
@@ -1661,7 +1661,8 @@ static void tcg_out_branch(TCGContext *s, int call, const tcg_insn_unit *dest) } } -static inline void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest) +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest, + const TCGHelperInfo *info) { tcg_out_branch(s, 1, dest); }
@@ -1885,7 +1886,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) (uintptr_t)l->raddr); } - tcg_out_call(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_branch(s, 1, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); data_reg = l->datalo_reg; switch (opc & MO_SSIZE) {
tcg/loongarch64/tcg-target.c.inc +4 -3
@@ -567,7 +567,8 @@ static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *arg, bool tail) } } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg) +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg, + const TCGHelperInfo *info) { tcg_out_call_int(s, arg, false); }
@@ -760,7 +761,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_A2, oi); tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_A3, (tcg_target_long)l->raddr); - tcg_out_call(s, qemu_ld_helpers[size]); + tcg_out_call_int(s, qemu_ld_helpers[size], false); switch (opc & MO_SSIZE) { case MO_SB:
@@ -821,7 +822,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_A3, oi); tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_A4, (tcg_target_long)l->raddr); - tcg_out_call(s, qemu_st_helpers[size]); + tcg_out_call_int(s, qemu_st_helpers[size], false); return tcg_out_goto(s, l->raddr); }
tcg/mips/tcg-target.c.inc +2 -1
@@ -1020,7 +1020,8 @@ static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *arg, bool tail) } } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg) +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg, + const TCGHelperInfo *info) { tcg_out_call_int(s, arg, false); tcg_out_nop(s);
tcg/ppc/tcg-target.c.inc +4 -3
@@ -2002,7 +2002,8 @@ static void tcg_out_call_int(TCGContext *s, int lk, #endif } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target) +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target, + const TCGHelperInfo *info) { tcg_out_call_int(s, LK, target); }
@@ -2221,7 +2222,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) tcg_out_movi(s, TCG_TYPE_I32, arg++, oi); tcg_out32(s, MFSPR | RT(arg) | LR); - tcg_out_call(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_call_int(s, LK, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); lo = lb->datalo_reg; hi = lb->datahi_reg;
@@ -2290,7 +2291,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) tcg_out_movi(s, TCG_TYPE_I32, arg++, oi); tcg_out32(s, MFSPR | RT(arg) | LR); - tcg_out_call(s, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_call_int(s, LK, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); tcg_out_b(s, 0, lb->raddr); return true;
tcg/riscv/tcg-target.c.inc +4 -3
@@ -819,7 +819,8 @@ static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *arg, bool tail) } } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg) +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg, + const TCGHelperInfo *info) { tcg_out_call_int(s, arg, false); }
@@ -1002,7 +1003,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) tcg_out_movi(s, TCG_TYPE_PTR, a2, oi); tcg_out_movi(s, TCG_TYPE_PTR, a3, (tcg_target_long)l->raddr); - tcg_out_call(s, qemu_ld_helpers[opc & MO_SSIZE]); + tcg_out_call_int(s, qemu_ld_helpers[opc & MO_SSIZE], false); tcg_out_mov(s, (opc & MO_SIZE) == MO_64, l->datalo_reg, a0); tcg_out_goto(s, l->raddr);
@@ -1047,7 +1048,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) tcg_out_movi(s, TCG_TYPE_PTR, a3, oi); tcg_out_movi(s, TCG_TYPE_PTR, a4, (tcg_target_long)l->raddr); - tcg_out_call(s, qemu_st_helpers[opc & MO_SIZE]); + tcg_out_call_int(s, qemu_st_helpers[opc & MO_SIZE], false); tcg_out_goto(s, l->raddr); return true;
tcg/s390x/tcg-target.c.inc +9 -3
@@ -1691,7 +1691,7 @@ static void tgen_brcond(TCGContext *s, TCGType type, TCGCond c, tgen_branch(s, cc, l); } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest) +static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *dest) { ptrdiff_t off = tcg_pcrel_diff(s, dest) >> 1; if (off == (int32_t)off) {
@@ -1702,6 +1702,12 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest) } } +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest, + const TCGHelperInfo *info) +{ + tcg_out_call_int(s, dest); +} + static void tcg_out_qemu_ld_direct(TCGContext *s, MemOp opc, TCGReg data, TCGReg base, TCGReg index, int disp) {
@@ -1897,7 +1903,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) } tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_R4, oi); tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R5, (uintptr_t)lb->raddr); - tcg_out_call(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SSIZE)]); + tcg_out_call_int(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SSIZE)]); tcg_out_mov(s, TCG_TYPE_I64, data_reg, TCG_REG_R2); tgen_gotoi(s, S390_CC_ALWAYS, lb->raddr);
@@ -1938,7 +1944,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) } tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_R5, oi); tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R6, (uintptr_t)lb->raddr); - tcg_out_call(s, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_call_int(s, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); tgen_gotoi(s, S390_CC_ALWAYS, lb->raddr); return true;
tcg/sparc64/tcg-target.c.inc +2 -1
@@ -859,7 +859,8 @@ static void tcg_out_call_nodelay(TCGContext *s, const tcg_insn_unit *dest, } } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest) +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest, + const TCGHelperInfo *info) { tcg_out_call_nodelay(s, dest, false); tcg_out_nop(s);
tcg/tci/tcg-target.c.inc +2 -1
@@ -558,8 +558,9 @@ static void tcg_out_movi(TCGContext *s, TCGType type, } static void tcg_out_call(TCGContext *s, const tcg_insn_unit *func, - ffi_cif *cif) + const TCGHelperInfo *info) { + ffi_cif *cif = info->cif; tcg_insn_unit insn = 0; uint8_t which;
Signature · 1 line
2.34.1
PATCH v3 29/45 tcg: Define TCG_TYPE_I128 and related helper macros
In reply to Richard HendersonBegin staging in support for TCGv_i128 with Int128. Define the type enumerator, the typedef, and the helper-head.h macros.
This cannot yet be used, because you can't allocate temporaries of this new type.
--- include/exec/helper-head.h | 7 +++++++ include/tcg/tcg.h | 17 ++++++++++------- 2 files changed, 17 insertions(+), 7 deletions(-)
include/exec/helper-head.h +7 -0
@@ -26,6 +26,7 @@#define dh_alias_int i32#define dh_alias_i64 i64#define dh_alias_s64 i64 +#define dh_alias_i128 i128#define dh_alias_f16 i32#define dh_alias_f32 i32#define dh_alias_f64 i64
@@ -40,6 +41,7 @@#define dh_ctype_int int#define dh_ctype_i64 uint64_t#define dh_ctype_s64 int64_t +#define dh_ctype_i128 Int128#define dh_ctype_f16 uint32_t#define dh_ctype_f32 float32#define dh_ctype_f64 float64
@@ -71,6 +73,7 @@#define dh_retvar_decl0_noreturn void#define dh_retvar_decl0_i32 TCGv_i32 retval#define dh_retvar_decl0_i64 TCGv_i64 retval +#define dh_retval_decl0_i128 TCGv_i128 retval#define dh_retvar_decl0_ptr TCGv_ptr retval#define dh_retvar_decl0 (t) glue(dh_retvar_decl0_, dh_alias(t))
@@ -78,6 +81,7 @@#define dh_retvar_decl_noreturn#define dh_retvar_decl_i32 TCGv_i32 retval,#define dh_retvar_decl_i64 TCGv_i64 retval, +#define dh_retvar_decl_i128 TCGv_i128 retval,#define dh_retvar_decl_ptr TCGv_ptr retval,#define dh_retvar_decl (t) glue(dh_retvar_decl_, dh_alias(t))
@@ -85,6 +89,7 @@#define dh_retvar_noreturn NULL#define dh_retvar_i32 tcgv_i32_temp(retval)#define dh_retvar_i64 tcgv_i64_temp(retval) +#define dh_retvar_i128 tcgv_i128_temp(retval)#define dh_retvar_ptr tcgv_ptr_temp(retval)#define dh_retvar (t) glue(dh_retvar_, dh_alias(t))
@@ -95,6 +100,7 @@#define dh_typecode_i64 4#define dh_typecode_s64 5#define dh_typecode_ptr 6 +#define dh_typecode_i128 7#define dh_typecode_int dh_typecode_s32#define dh_typecode_f16 dh_typecode_i32#define dh_typecode_f32 dh_typecode_i32
@@ -104,6 +110,7 @@#define dh_callflag_i32 0#define dh_callflag_i64 0 +#define dh_callflag_i128 0#define dh_callflag_ptr 0#define dh_callflag_void 0#define dh_callflag_noreturn TCG_CALL_NO_RETURN
include/tcg/tcg.h +10 -7
@@ -277,6 +277,7 @@ typedef struct TCGPool { typedef enum TCGType { TCG_TYPE_I32, TCG_TYPE_I64, + TCG_TYPE_I128, TCG_TYPE_V64, TCG_TYPE_V128,
@@ -358,13 +359,14 @@ typedef tcg_target_ulong TCGArg; in tcg/README. Target CPU front-end code uses these types to deal with TCG variables as it emits TCG code via the tcg_gen_* functions. They come in several flavours: - * TCGv_i32 : 32 bit integer type - * TCGv_i64 : 64 bit integer type - * TCGv_ptr : a host pointer type - * TCGv_vec : a host vector type; the exact size is not exposed - to the CPU front-end code. - * TCGv : an integer type the same size as target_ulong - (an alias for either TCGv_i32 or TCGv_i64) + * TCGv_i32 : 32 bit integer type + * TCGv_i64 : 64 bit integer type + * TCGv_i128 : 128 bit integer type + * TCGv_ptr : a host pointer type + * TCGv_vec : a host vector type; the exact size is not exposed + to the CPU front-end code. + * TCGv : an integer type the same size as target_ulong + (an alias for either TCGv_i32 or TCGv_i64) The compiler's type checking will complain if you mix them up and pass the wrong sized TCGv to a function.
@@ -384,6 +386,7 @@ typedef tcg_target_ulong TCGArg;typedef struct TCGv_i32_d *TCGv_i32 ;typedef struct TCGv_i64_d *TCGv_i64 ; +typedef struct TCGv_i128_d *TCGv_i128 ;typedef struct TCGv_ptr_d *TCGv_ptr ;typedef struct TCGv_vec_d *TCGv_vec ;typedef TCGv_ptr TCGv_env ;
Signature · 1 line
2.34.1
PATCH v3 30/45 tcg: Handle dh_typecode_i128 with TCG_CALL_{RET, ARG}_NORMAL
In reply to Richard HendersonMany hosts pass and return 128-bit quantities like sequential 64-bit quantities. Treat this just like we currently break down 64-bit quantities for a 32-bit host.
--- tcg/tcg.c | 33 +++++++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-)
tcg/tcg.c +29 -4
@@ -685,11 +685,22 @@ static void init_call_layout(TCGHelperInfo *info)case dh_typecode_s64: info->nr_out =64 /TCG_TARGET_REG_BITS ; info->out_kind =TCG_CALL_RET_NORMAL ; +assert (info->nr_out <=ARRAY_SIZE (tcg_target_call_oarg_regs)); +break ; +case dh_typecode_i128: + info->nr_out =128 /TCG_TARGET_REG_BITS ; + info->out_kind =TCG_CALL_RET_NORMAL ;/* TODO */ +switch (/* TODO */ TCG_CALL_RET_NORMAL ) { +case TCG_CALL_RET_NORMAL : +assert (info->nr_out <=ARRAY_SIZE (tcg_target_call_oarg_regs)); +break ; +default : +qemu_build_not_reached (); + }break ;default :g_assert_not_reached (); } -assert (info->nr_out <=ARRAY_SIZE (tcg_target_call_oarg_regs));/* * Parse and place function arguments.
@@ -736,6 +747,18 @@ static void init_call_layout(TCGHelperInfo *info)case dh_typecode_ptr:layout_arg_1 (&cum, info,TCG_CALL_ARG_NORMAL );break ; +case dh_typecode_i128: +switch (/* TODO */ TCG_CALL_ARG_NORMAL ) { +case TCG_CALL_ARG_EVEN : +layout_arg_even (&cum); +/* fall through */ +case TCG_CALL_ARG_NORMAL : +layout_arg_normal_n (&cum, info,128 /TCG_TARGET_REG_BITS ); +break ; +default : +qemu_build_not_reached (); + } +break ;default :g_assert_not_reached (); }
@@ -1653,11 +1676,13 @@ void tcg_gen_callN(void *func, TCGTemp *ret, int nargs, TCGTemp **args) op->args[pi++] =temp_arg (ret);break ;case 2 : +case 4 :tcg_debug_assert (ret !=NULL ); -tcg_debug_assert (ret->base_type == ret->type +1 ); +tcg_debug_assert (ret->base_type == ret->type +ctz32 (n));tcg_debug_assert (ret->temp_subindex ==0 ); - op->args[pi++] =temp_arg (ret); - op->args[pi++] =temp_arg (ret +1 ); +for (i =0 ; i < n; ++i) { + op->args[pi++] =temp_arg (ret + i); + }break ;default :g_assert_not_reached ();
Signature · 1 line
2.34.1
PATCH v3 31/45 tcg: Allocate objects contiguously in temp_allocate_frame
In reply to Richard HendersonWhen allocating a temp to the stack frame, consider the base type and allocate all parts at once.
--- tcg/tcg.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-)
tcg/tcg.c +22 -8
@@ -3296,11 +3296,12 @@ static void check_regs(TCGContext *s)static void temp_allocate_frame (TCGContext *s,TCGTemp *ts) { -int size =tcg_type_size (ts->type); -int align;intptr_t off; +int size, align; -switch (ts->type) { +/* When allocating an object, look at the full type. */ + size =tcg_type_size (ts->base_type); +switch (ts->base_type) {case TCG_TYPE_I32 : align =4 ;break ;
@@ -3331,13 +3332,26 @@ static void temp_allocate_frame(TCGContext *s, TCGTemp *ts)tcg_raise_tb_overflow (s); } s->current_frame_offset = off + size; - - ts->mem_offset = off;#if defined(__sparc__) - ts->mem_offset +=TCG_TARGET_STACK_BIAS ; + off +=TCG_TARGET_STACK_BIAS ;#endif - ts->mem_base = s->frame_temp; - ts->mem_allocated =1 ; + +/* If the object was subdivided, assign memory to all the parts. */ +if (ts->base_type != ts->type) { +int part_size =tcg_type_size (ts->type); +int part_count = size / part_size; + + ts -= ts->temp_subindex; +for (int i =0 ; i < part_count; ++i) { + ts[i].mem_offset = off + i * part_size; + ts[i].mem_base = s->frame_temp; + ts[i].mem_allocated =1 ; + } + }else { + ts->mem_offset = off; + ts->mem_base = s->frame_temp; + ts->mem_allocated =1 ; + } }static void temp_load (TCGContext *,TCGTemp *,TCGRegSet ,TCGRegSet ,TCGRegSet );
Signature · 1 line
2.34.1
PATCH v3 32/45 tcg: Introduce tcg_out_addi_ptr
In reply to Richard HendersonImplement the function for arm, i386, and s390x, which will use it. Add stubs for all other backends.
--- tcg/tcg.c | 2 ++ tcg/aarch64/tcg-target.c.inc | 7 +++++++ tcg/arm/tcg-target.c.inc | 20 ++++++++++++++++++++ tcg/i386/tcg-target.c.inc | 8 ++++++++ tcg/loongarch64/tcg-target.c.inc | 7 +++++++ tcg/mips/tcg-target.c.inc | 7 +++++++ tcg/ppc/tcg-target.c.inc | 7 +++++++ tcg/riscv/tcg-target.c.inc | 7 +++++++ tcg/s390x/tcg-target.c.inc | 7 +++++++ tcg/sparc64/tcg-target.c.inc | 7 +++++++ tcg/tci/tcg-target.c.inc | 7 +++++++ 11 files changed, 86 insertions(+)
tcg/tcg.c +2 -0
@@ -103,6 +103,8 @@ static void tcg_out_ld(TCGContext *s, TCGType type, TCGReg ret, TCGReg arg1,static bool tcg_out_mov (TCGContext *s,TCGType type,TCGReg ret,TCGReg arg);static void tcg_out_movi (TCGContext *s,TCGType type,TCGReg ret,tcg_target_long arg); +static void tcg_out_addi_ptr (TCGContext *s,TCGReg ,TCGReg ,tcg_target_long ) + __attribute__((unused));static void tcg_out_op (TCGContext *s,TCGOpcode opc,const TCGArg args[TCG_MAX_OP_ARGS ],const int const_args[TCG_MAX_OP_ARGS ]);
tcg/aarch64/tcg-target.c.inc +7 -0
@@ -1102,6 +1102,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, TCGReg rd, tcg_out_insn(s, 3305, LDR, 0, rd); } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + /* Define something more legible for general use. */ #define tcg_out_ldst_r tcg_out_insn_3310
tcg/arm/tcg-target.c.inc +20 -0
@@ -2548,6 +2548,26 @@ static void tcg_out_movi(TCGContext *s, TCGType type, tcg_out_movi32(s, COND_AL, ret, arg); } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + int enc, opc = ARITH_ADD; + + /* All of the easiest immediates to encode are positive. */ + if (imm < 0) { + imm = -imm; + opc = ARITH_SUB; + } + enc = encode_imm(imm); + if (enc >= 0) { + tcg_out_dat_imm(s, COND_AL, opc, rd, rs, enc); + } else { + tcg_out_movi32(s, COND_AL, TCG_REG_TMP, imm); + tcg_out_dat_reg(s, COND_AL, opc, rd, rs, + TCG_REG_TMP, SHIFT_IMM_LSL(0)); + } +} + /* Type is always V128, with I64 elements. */ static void tcg_out_dup2_vec(TCGContext *s, TCGReg rd, TCGReg rl, TCGReg rh) {
tcg/i386/tcg-target.c.inc +8 -0
@@ -1069,6 +1069,14 @@ static void tcg_out_movi(TCGContext *s, TCGType type, } } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + tcg_debug_assert(TCG_TARGET_REG_BITS == 32); + tcg_out_modrm_offset(s, OPC_LEA, rd, rs, imm); +} + static inline void tcg_out_pushi(TCGContext *s, tcg_target_long val) { if (val == (int8_t)val) {
tcg/loongarch64/tcg-target.c.inc +7 -0
@@ -389,6 +389,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, TCGReg rd, } } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + static void tcg_out_ext8u(TCGContext *s, TCGReg ret, TCGReg arg) { tcg_out_opc_andi(s, ret, arg, 0xff);
tcg/mips/tcg-target.c.inc +7 -0
@@ -550,6 +550,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, } } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + static void tcg_out_bswap16(TCGContext *s, TCGReg ret, TCGReg arg, int flags) { /* ret and arg can't be register tmp0 */
tcg/ppc/tcg-target.c.inc +7 -0
@@ -1125,6 +1125,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, TCGReg ret, } } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + static bool mask_operand(uint32_t c, int *mb, int *me) { uint32_t lsb, test;
tcg/riscv/tcg-target.c.inc +7 -0
@@ -545,6 +545,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, TCGReg rd, tcg_out_opc_imm(s, OPC_LD, rd, rd, 0); } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + static void tcg_out_ext8u(TCGContext *s, TCGReg ret, TCGReg arg) { tcg_out_opc_imm(s, OPC_ANDI, ret, arg, 0xff);
tcg/s390x/tcg-target.c.inc +7 -0
@@ -1020,6 +1020,13 @@ static inline bool tcg_out_sti(TCGContext *s, TCGType type, TCGArg val, return false; } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + tcg_out_mem(s, RX_LA, RXY_LAY, rd, rs, TCG_REG_NONE, imm); +} + /* load data from an absolute host address */ static void tcg_out_ld_abs(TCGContext *s, TCGType type, TCGReg dest, const void *abs)
tcg/sparc64/tcg-target.c.inc +7 -0
@@ -497,6 +497,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, tcg_out_movi_int(s, type, ret, arg, false, TCG_REG_T2); } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + static void tcg_out_ldst_rr(TCGContext *s, TCGReg data, TCGReg a1, TCGReg a2, int op) {
tcg/tci/tcg-target.c.inc +7 -0
@@ -557,6 +557,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, } } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + static void tcg_out_call(TCGContext *s, const tcg_insn_unit *func, const TCGHelperInfo *info) {
Signature · 1 line
2.34.1
Re: [PATCH for-8.0 v3 32/45] tcg: Introduce tcg_out_addi_ptr
Quoted 5 lines · Richard Henderson
Implement the function for arm, i386, and s390x, which will use it. Add stubs for all other backends.
Signed-off-by Richard Henderson <[email protected]>---
For ppc and common code bits:
Quoted 220 lines
tcg/tcg.c | 2 ++ tcg/aarch64/tcg-target.c.inc | 7 +++++++ tcg/arm/tcg-target.c.inc | 20 ++++++++++++++++++++ tcg/i386/tcg-target.c.inc | 8 ++++++++ tcg/loongarch64/tcg-target.c.inc | 7 +++++++ tcg/mips/tcg-target.c.inc | 7 +++++++ tcg/ppc/tcg-target.c.inc | 7 +++++++ tcg/riscv/tcg-target.c.inc | 7 +++++++ tcg/s390x/tcg-target.c.inc | 7 +++++++ tcg/sparc64/tcg-target.c.inc | 7 +++++++ tcg/tci/tcg-target.c.inc | 7 +++++++ 11 files changed, 86 insertions(+)tcg/tcg.c +2
-0@@ -103,6 +103,8 @@ static void tcg_out_ld(TCGContext *s, TCGType type, TCGReg ret, TCGReg arg1,static bool tcg_out_mov (TCGContext *s,TCGType type,TCGReg ret,TCGReg arg);static void tcg_out_movi (TCGContext *s,TCGType type,TCGReg ret,tcg_target_long arg); +static void tcg_out_addi_ptr (TCGContext *s,TCGReg ,TCGReg ,tcg_target_long ) + __attribute__((unused));static void tcg_out_op (TCGContext *s,TCGOpcode opc,const TCGArg args[TCG_MAX_OP_ARGS ],const int const_args[TCG_MAX_OP_ARGS ]);tcg/aarch64/tcg-target.c.inc +7
-0@@ -1102,6 +1102,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, TCGReg rd, tcg_out_insn(s, 3305, LDR, 0, rd); } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + /* Define something more legible for general use. */ #define tcg_out_ldst_r tcg_out_insn_3310tcg/arm/tcg-target.c.inc +20
-0@@ -2548,6 +2548,26 @@ static void tcg_out_movi(TCGContext *s, TCGType type, tcg_out_movi32(s, COND_AL, ret, arg); } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + int enc, opc = ARITH_ADD; + + /* All of the easiest immediates to encode are positive. */ + if (imm < 0) { + imm = -imm; + opc = ARITH_SUB; + } + enc = encode_imm(imm); + if (enc >= 0) { + tcg_out_dat_imm(s, COND_AL, opc, rd, rs, enc); + } else { + tcg_out_movi32(s, COND_AL, TCG_REG_TMP, imm); + tcg_out_dat_reg(s, COND_AL, opc, rd, rs, + TCG_REG_TMP, SHIFT_IMM_LSL(0)); + } +} + /* Type is always V128, with I64 elements. */ static void tcg_out_dup2_vec(TCGContext *s, TCGReg rd, TCGReg rl, TCGReg rh) {tcg/i386/tcg-target.c.inc +8
-0@@ -1069,6 +1069,14 @@ static void tcg_out_movi(TCGContext *s, TCGType type, } } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + tcg_debug_assert(TCG_TARGET_REG_BITS == 32); + tcg_out_modrm_offset(s, OPC_LEA, rd, rs, imm); +} + static inline void tcg_out_pushi(TCGContext *s, tcg_target_long val) { if (val == (int8_t)val) {tcg/loongarch64/tcg-target.c.inc +7
-0@@ -389,6 +389,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, TCGReg rd, } } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + static void tcg_out_ext8u(TCGContext *s, TCGReg ret, TCGReg arg) { tcg_out_opc_andi(s, ret, arg, 0xff);tcg/mips/tcg-target.c.inc +7
-0@@ -550,6 +550,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, } } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + static void tcg_out_bswap16(TCGContext *s, TCGReg ret, TCGReg arg, int flags) { /* ret and arg can't be register tmp0 */tcg/ppc/tcg-target.c.inc +7
-0@@ -1125,6 +1125,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, TCGReg ret, } } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + static bool mask_operand(uint32_t c, int *mb, int *me) { uint32_t lsb, test;tcg/riscv/tcg-target.c.inc +7
-0@@ -545,6 +545,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, TCGReg rd, tcg_out_opc_imm(s, OPC_LD, rd, rd, 0); } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + static void tcg_out_ext8u(TCGContext *s, TCGReg ret, TCGReg arg) { tcg_out_opc_imm(s, OPC_ANDI, ret, arg, 0xff);tcg/s390x/tcg-target.c.inc +7
-0@@ -1020,6 +1020,13 @@ static inline bool tcg_out_sti(TCGContext *s, TCGType type, TCGArg val, return false; } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + tcg_out_mem(s, RX_LA, RXY_LAY, rd, rs, TCG_REG_NONE, imm); +} + /* load data from an absolute host address */ static void tcg_out_ld_abs(TCGContext *s, TCGType type, TCGReg dest, const void *abs)tcg/sparc64/tcg-target.c.inc +7
-0@@ -497,6 +497,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, tcg_out_movi_int(s, type, ret, arg, false, TCG_REG_T2); } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + static void tcg_out_ldst_rr(TCGContext *s, TCGReg data, TCGReg a1, TCGReg a2, int op) {tcg/tci/tcg-target.c.inc +7
-0@@ -557,6 +557,13 @@ static void tcg_out_movi(TCGContext *s, TCGType type, } } +static void tcg_out_addi_ptr(TCGContext *s, TCGReg rd, TCGReg rs, + tcg_target_long imm) +{ + /* This function is only used for passing structs by reference. */ + g_assert_not_reached(); +} + static void tcg_out_call(TCGContext *s, const tcg_insn_unit *func, const TCGHelperInfo *info) {
PATCH v3 33/45 tcg: Add TCG_CALL_{RET,ARG}_BY_REF
In reply to Richard HendersonThese will be used by some hosts, both 32 and 64-bit, to pass and return i128. Not yet used, because allocation is not yet enabled.
--- tcg/tcg-internal.h | 3 + tcg/tcg.c | 135 ++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 135 insertions(+), 3 deletions(-)
tcg/tcg-internal.h +3 -0
@@ -36,6 +36,7 @@*/ typedef enum {TCG_CALL_RET_NORMAL ,/* by registers */ +TCG_CALL_RET_BY_REF ,/* for i128, by reference */ }TCGCallReturnKind ; typedef enum {
@@ -44,6 +45,8 @@ typedef enum {TCG_CALL_ARG_EXTEND ,/* for i32, as a sign/zero-extended i64 */ TCG_CALL_ARG_EXTEND_U ,/* ... as a zero-extended i64 */ TCG_CALL_ARG_EXTEND_S ,/* ... as a sign-extended i64 */ +TCG_CALL_ARG_BY_REF ,/* for i128, by reference, first */ +TCG_CALL_ARG_BY_REF_N ,/* ... by reference, subsequent */ } TCGCallArgumentKind; typedef struct TCGCallArgumentLoc {
tcg/tcg.c +132 -3
195 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v3 34/45 tcg: Introduce tcg_target_call_oarg_reg
In reply to Richard HendersonReplace the flat array tcg_target_call_oarg_regs[] with a function call including the TCGCallReturnKind.
--- tcg/tcg.c | 9 ++++++--- tcg/aarch64/tcg-target.c.inc | 10 +++++++--- tcg/arm/tcg-target.c.inc | 10 +++++++--- tcg/i386/tcg-target.c.inc | 16 ++++++++++------ tcg/loongarch64/tcg-target.c.inc | 10 ++++++---- tcg/mips/tcg-target.c.inc | 10 ++++++---- tcg/ppc/tcg-target.c.inc | 10 ++++++---- tcg/riscv/tcg-target.c.inc | 10 ++++++---- tcg/s390x/tcg-target.c.inc | 9 ++++++--- tcg/sparc64/tcg-target.c.inc | 12 ++++++------ tcg/tci/tcg-target.c.inc | 12 ++++++------ 11 files changed, 72 insertions(+), 46 deletions(-)
tcg/tcg.c +6 -3
@@ -148,6 +148,7 @@ static bool tcg_out_sti(TCGContext *s, TCGType type, TCGArg val, TCGReg base, intptr_t ofs);static void tcg_out_call (TCGContext *s,const tcg_insn_unit *target,const TCGHelperInfo *info); +static TCGReg tcg_target_call_oarg_reg (TCGCallReturnKind kind,int slot);static bool tcg_target_const_match (int64_t val,TCGType type,int ct);#ifdef TCG_TARGET_NEED_LDST_LABELS static int tcg_out_ldst_finalize (TCGContext *s);
@@ -718,14 +719,16 @@ static void init_call_layout(TCGHelperInfo *info)case dh_typecode_s64: info->nr_out =64 /TCG_TARGET_REG_BITS ; info->out_kind =TCG_CALL_RET_NORMAL ; -assert (info->nr_out <=ARRAY_SIZE (tcg_target_call_oarg_regs)); +/* Query the last register now to trigger any assert early. */ +tcg_target_call_oarg_reg (info->out_kind, info->nr_out -1 );break ;case dh_typecode_i128: info->nr_out =128 /TCG_TARGET_REG_BITS ; info->out_kind =TCG_CALL_RET_NORMAL ;/* TODO */ switch (/* TODO */ TCG_CALL_RET_NORMAL ) {case TCG_CALL_RET_NORMAL : -assert (info->nr_out <=ARRAY_SIZE (tcg_target_call_oarg_regs)); +/* Query the last register now to trigger any assert early. */ +tcg_target_call_oarg_reg (info->out_kind, info->nr_out -1 );break ;case TCG_CALL_RET_BY_REF :/*
@@ -4579,7 +4582,7 @@ static void tcg_reg_alloc_call(TCGContext *s, TCGOp *op)case TCG_CALL_RET_NORMAL :for (i =0 ; i < nb_oargs; i++) {TCGTemp *ts =arg_temp (op->args[i]); -TCGReg reg = tcg_target_call_oarg_regs[i]; +TCGReg reg =tcg_target_call_oarg_reg (TCG_CALL_RET_NORMAL , i);/* ENV should not be modified. */ tcg_debug_assert (!temp_readonly (ts));
tcg/aarch64/tcg-target.c.inc +7 -3
@@ -63,9 +63,13 @@ static const int tcg_target_call_iarg_regs[8] = { TCG_REG_X0, TCG_REG_X1, TCG_REG_X2, TCG_REG_X3, TCG_REG_X4, TCG_REG_X5, TCG_REG_X6, TCG_REG_X7 }; -static const int tcg_target_call_oarg_regs[1] = { - TCG_REG_X0 -}; + +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 1); + return TCG_REG_X0 + slot; +} #define TCG_REG_TMP TCG_REG_X30 #define TCG_VEC_TMP TCG_REG_V31
tcg/arm/tcg-target.c.inc +7 -3
@@ -79,9 +79,13 @@ static const int tcg_target_reg_alloc_order[] = { static const int tcg_target_call_iarg_regs[4] = { TCG_REG_R0, TCG_REG_R1, TCG_REG_R2, TCG_REG_R3 }; -static const int tcg_target_call_oarg_regs[2] = { - TCG_REG_R0, TCG_REG_R1 -}; + +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 3); + return TCG_REG_R0 + slot; +} #define TCG_REG_TMP TCG_REG_R12 #define TCG_VEC_TMP TCG_REG_Q15
tcg/i386/tcg-target.c.inc +10 -6
@@ -109,12 +109,16 @@ static const int tcg_target_call_iarg_regs[] = { #endif }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_EAX, -#if TCG_TARGET_REG_BITS == 32 - TCG_REG_EDX -#endif -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + switch (kind) { + case TCG_CALL_RET_NORMAL: + tcg_debug_assert(slot >= 0 && slot <= 1); + return slot ? TCG_REG_EDX : TCG_REG_EAX; + default: + g_assert_not_reached(); + } +} /* Constants we accept. */ #define TCG_CT_CONST_S32 0x100
tcg/loongarch64/tcg-target.c.inc +6 -4
@@ -114,10 +114,12 @@ static const int tcg_target_call_iarg_regs[] = { TCG_REG_A7, }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_A0, - TCG_REG_A1, -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 1); + return TCG_REG_A0 + slot; +} #ifndef CONFIG_SOFTMMU #define USE_GUEST_BASE (guest_base != 0)
tcg/mips/tcg-target.c.inc +6 -4
@@ -136,10 +136,12 @@ static const TCGReg tcg_target_call_iarg_regs[] = { #endif }; -static const TCGReg tcg_target_call_oarg_regs[2] = { - TCG_REG_V0, - TCG_REG_V1 -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 1); + return TCG_REG_V0 + slot; +} static const tcg_insn_unit *tb_ret_addr; static const tcg_insn_unit *bswap32_addr;
tcg/ppc/tcg-target.c.inc +6 -4
@@ -186,10 +186,12 @@ static const int tcg_target_call_iarg_regs[] = { TCG_REG_R10 }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_R3, - TCG_REG_R4 -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 1); + return TCG_REG_R3 + slot; +} static const int tcg_target_callee_save_regs[] = { #ifdef _CALL_DARWIN
tcg/riscv/tcg-target.c.inc +6 -4
@@ -113,10 +113,12 @@ static const int tcg_target_call_iarg_regs[] = { TCG_REG_A7, }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_A0, - TCG_REG_A1, -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 1); + return TCG_REG_A0 + slot; +} #define TCG_CT_CONST_ZERO 0x100 #define TCG_CT_CONST_S12 0x200
tcg/s390x/tcg-target.c.inc +6 -3
@@ -390,9 +390,12 @@ static const int tcg_target_call_iarg_regs[] = { TCG_REG_R6, }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_R2, -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot == 0); + return TCG_REG_R2; +} #define S390_CC_EQ 8 #define S390_CC_LT 4
tcg/sparc64/tcg-target.c.inc +6 -6
@@ -133,12 +133,12 @@ static const int tcg_target_call_iarg_regs[6] = { TCG_REG_O5, }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_O0, - TCG_REG_O1, - TCG_REG_O2, - TCG_REG_O3, -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 3); + return TCG_REG_O0 + slot; +} #define INSN_OP(x) ((x) << 30) #define INSN_OP2(x) ((x) << 22)
tcg/tci/tcg-target.c.inc +6 -6
@@ -200,12 +200,12 @@ static const int tcg_target_reg_alloc_order[] = { /* No call arguments via registers. All will be stored on the "stack". */ static const int tcg_target_call_iarg_regs[] = { }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_R0, -#if TCG_TARGET_REG_BITS == 32 - TCG_REG_R1 -#endif -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot < 64 / TCG_TARGET_REG_BITS); + return TCG_REG_R0 + slot; +} #ifdef CONFIG_DEBUG_TCG static const char *const tcg_target_reg_names[TCG_TARGET_NB_REGS] = {
Signature · 1 line
2.34.1
Re: [PATCH for-8.0 v3 34/45] tcg: Introduce tcg_target_call_oarg_reg
Quoted 5 lines · Richard Henderson
Replace the flat array tcg_target_call_oarg_regs[] with a function call including the TCGCallReturnKind.
Signed-off-by Richard Henderson <[email protected]>---
For ppc and common code bits:
Quoted 272 lines
tcg/tcg.c | 9 ++++++--- tcg/aarch64/tcg-target.c.inc | 10 +++++++--- tcg/arm/tcg-target.c.inc | 10 +++++++--- tcg/i386/tcg-target.c.inc | 16 ++++++++++------ tcg/loongarch64/tcg-target.c.inc | 10 ++++++---- tcg/mips/tcg-target.c.inc | 10 ++++++---- tcg/ppc/tcg-target.c.inc | 10 ++++++---- tcg/riscv/tcg-target.c.inc | 10 ++++++---- tcg/s390x/tcg-target.c.inc | 9 ++++++--- tcg/sparc64/tcg-target.c.inc | 12 ++++++------ tcg/tci/tcg-target.c.inc | 12 ++++++------ 11 files changed, 72 insertions(+), 46 deletions(-)tcg/tcg.c +6
-3@@ -148,6 +148,7 @@ static bool tcg_out_sti(TCGContext *s, TCGType type, TCGArg val, TCGReg base, intptr_t ofs);static void tcg_out_call (TCGContext *s,const tcg_insn_unit *target,const TCGHelperInfo *info); +static TCGReg tcg_target_call_oarg_reg (TCGCallReturnKind kind,int slot);static bool tcg_target_const_match (int64_t val,TCGType type,int ct);#ifdef TCG_TARGET_NEED_LDST_LABELS static int tcg_out_ldst_finalize (TCGContext *s);@@ -718,14 +719,16 @@ static void init_call_layout(TCGHelperInfo *info)case dh_typecode_s64: info->nr_out =64 /TCG_TARGET_REG_BITS ; info->out_kind =TCG_CALL_RET_NORMAL ; -assert (info->nr_out <=ARRAY_SIZE (tcg_target_call_oarg_regs)); +/* Query the last register now to trigger any assert early. */ +tcg_target_call_oarg_reg (info->out_kind, info->nr_out -1 );break ;case dh_typecode_i128: info->nr_out =128 /TCG_TARGET_REG_BITS ; info->out_kind =TCG_CALL_RET_NORMAL ;/* TODO */ switch (/* TODO */ TCG_CALL_RET_NORMAL ) {case TCG_CALL_RET_NORMAL : -assert (info->nr_out <=ARRAY_SIZE (tcg_target_call_oarg_regs)); +/* Query the last register now to trigger any assert early. */ +tcg_target_call_oarg_reg (info->out_kind, info->nr_out -1 );break ;case TCG_CALL_RET_BY_REF :/* @@ -4579,7 +4582,7 @@ static void tcg_reg_alloc_call(TCGContext *s, TCGOp *op)case TCG_CALL_RET_NORMAL :for (i =0 ; i < nb_oargs; i++) {TCGTemp *ts =arg_temp (op->args[i]); -TCGReg reg = tcg_target_call_oarg_regs[i]; +TCGReg reg =tcg_target_call_oarg_reg (TCG_CALL_RET_NORMAL , i);/* ENV should not be modified. */ tcg_debug_assert (!temp_readonly (ts));tcg/aarch64/tcg-target.c.inc +7
-3@@ -63,9 +63,13 @@ static const int tcg_target_call_iarg_regs[8] = { TCG_REG_X0, TCG_REG_X1, TCG_REG_X2, TCG_REG_X3, TCG_REG_X4, TCG_REG_X5, TCG_REG_X6, TCG_REG_X7 }; -static const int tcg_target_call_oarg_regs[1] = { - TCG_REG_X0 -}; + +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 1); + return TCG_REG_X0 + slot; +} #define TCG_REG_TMP TCG_REG_X30 #define TCG_VEC_TMP TCG_REG_V31tcg/arm/tcg-target.c.inc +7
-3@@ -79,9 +79,13 @@ static const int tcg_target_reg_alloc_order[] = { static const int tcg_target_call_iarg_regs[4] = { TCG_REG_R0, TCG_REG_R1, TCG_REG_R2, TCG_REG_R3 }; -static const int tcg_target_call_oarg_regs[2] = { - TCG_REG_R0, TCG_REG_R1 -}; + +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 3); + return TCG_REG_R0 + slot; +} #define TCG_REG_TMP TCG_REG_R12 #define TCG_VEC_TMP TCG_REG_Q15tcg/i386/tcg-target.c.inc +10
-6@@ -109,12 +109,16 @@ static const int tcg_target_call_iarg_regs[] = { #endif }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_EAX, -#if TCG_TARGET_REG_BITS == 32 - TCG_REG_EDX -#endif -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + switch (kind) { + case TCG_CALL_RET_NORMAL: + tcg_debug_assert(slot >= 0 && slot <= 1); + return slot ? TCG_REG_EDX : TCG_REG_EAX; + default: + g_assert_not_reached(); + } +} /* Constants we accept. */ #define TCG_CT_CONST_S32 0x100tcg/loongarch64/tcg-target.c.inc +6
-4@@ -114,10 +114,12 @@ static const int tcg_target_call_iarg_regs[] = { TCG_REG_A7, }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_A0, - TCG_REG_A1, -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 1); + return TCG_REG_A0 + slot; +} #ifndef CONFIG_SOFTMMU #define USE_GUEST_BASE (guest_base != 0)tcg/mips/tcg-target.c.inc +6
-4@@ -136,10 +136,12 @@ static const TCGReg tcg_target_call_iarg_regs[] = { #endif }; -static const TCGReg tcg_target_call_oarg_regs[2] = { - TCG_REG_V0, - TCG_REG_V1 -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 1); + return TCG_REG_V0 + slot; +} static const tcg_insn_unit *tb_ret_addr; static const tcg_insn_unit *bswap32_addr;tcg/ppc/tcg-target.c.inc +6
-4@@ -186,10 +186,12 @@ static const int tcg_target_call_iarg_regs[] = { TCG_REG_R10 }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_R3, - TCG_REG_R4 -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 1); + return TCG_REG_R3 + slot; +} static const int tcg_target_callee_save_regs[] = { #ifdef _CALL_DARWINtcg/riscv/tcg-target.c.inc +6
-4@@ -113,10 +113,12 @@ static const int tcg_target_call_iarg_regs[] = { TCG_REG_A7, }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_A0, - TCG_REG_A1, -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 1); + return TCG_REG_A0 + slot; +} #define TCG_CT_CONST_ZERO 0x100 #define TCG_CT_CONST_S12 0x200tcg/s390x/tcg-target.c.inc +6
-3@@ -390,9 +390,12 @@ static const int tcg_target_call_iarg_regs[] = { TCG_REG_R6, }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_R2, -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot == 0); + return TCG_REG_R2; +} #define S390_CC_EQ 8 #define S390_CC_LT 4tcg/sparc64/tcg-target.c.inc +6
-6@@ -133,12 +133,12 @@ static const int tcg_target_call_iarg_regs[6] = { TCG_REG_O5, }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_O0, - TCG_REG_O1, - TCG_REG_O2, - TCG_REG_O3, -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot <= 3); + return TCG_REG_O0 + slot; +} #define INSN_OP(x) ((x) << 30) #define INSN_OP2(x) ((x) << 22)tcg/tci/tcg-target.c.inc +6
-6@@ -200,12 +200,12 @@ static const int tcg_target_reg_alloc_order[] = { /* No call arguments via registers. All will be stored on the "stack". */ static const int tcg_target_call_iarg_regs[] = { }; -static const int tcg_target_call_oarg_regs[] = { - TCG_REG_R0, -#if TCG_TARGET_REG_BITS == 32 - TCG_REG_R1 -#endif -}; +static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) +{ + tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); + tcg_debug_assert(slot >= 0 && slot < 64 / TCG_TARGET_REG_BITS); + return TCG_REG_R0 + slot; +} #ifdef CONFIG_DEBUG_TCG static const char *const tcg_target_reg_names[TCG_TARGET_NB_REGS] = {
PATCH v3 35/45 tcg: Add TCG_CALL_RET_BY_VEC
In reply to Richard HendersonThis will be used by _WIN64 to return i128. Not yet used, because allocation is not yet enabled.
--- tcg/tcg-internal.h | 1 + tcg/tcg.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)
tcg/tcg-internal.h +1 -0
@@ -37,6 +37,7 @@typedef enum {TCG_CALL_RET_NORMAL ,/* by registers */ TCG_CALL_RET_BY_REF ,/* for i128, by reference */ +TCG_CALL_RET_BY_VEC ,/* for i128, by vector register */ }TCGCallReturnKind ; typedef enum {
tcg/tcg.c +19 -0
@@ -730,6 +730,10 @@ static void init_call_layout(TCGHelperInfo *info)/* Query the last register now to trigger any assert early. */ tcg_target_call_oarg_reg (info->out_kind, info->nr_out -1 );break ; +case TCG_CALL_RET_BY_VEC : +/* Query the single register now to trigger any assert early. */ +tcg_target_call_oarg_reg (TCG_CALL_RET_BY_VEC ,0 ); +break ;case TCG_CALL_RET_BY_REF :/* * Allocate the first argument to the output.
@@ -4598,6 +4602,21 @@ static void tcg_reg_alloc_call(TCGContext *s, TCGOp *op) }break ; +case TCG_CALL_RET_BY_VEC : + { +TCGTemp *ts =arg_temp (op->args[0 ]); + +tcg_debug_assert (ts->base_type ==TCG_TYPE_I128 ); +tcg_debug_assert (ts->temp_subindex ==0 ); +if (!ts->mem_allocated) { +temp_allocate_frame (s, ts); + } +tcg_out_st (s,TCG_TYPE_V128 , +tcg_target_call_oarg_reg (TCG_CALL_RET_BY_VEC ,0 ), + ts->mem_base->reg, ts->mem_offset); + } +/* fall through to mark all parts in memory */ +case TCG_CALL_RET_BY_REF :/* The callee has performed a write through the reference. */ for (i =0 ; i < nb_oargs; i++) {
Signature · 1 line
2.34.1
PATCH v3 36/45 include/qemu/int128: Use Int128 structure for TCI
In reply to Richard HendersonWe are about to allow passing Int128 to/from tcg helper functions, but libffi doesn't support __int128_t, so use the structure.
In order for atomic128.h to continue working, we must provide a mechanism to frob between real __int128_t and the structure. Provide a new union, Int128Alias, for this. We cannot modify Int128 itself, as any changed alignment would also break libffi.
--- include/qemu/atomic128.h | 29 +++++++++++++++++++++------ include/qemu/int128.h | 25 +++++++++++++++++++++--- util/int128.c | 42 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 87 insertions(+), 9 deletions(-)
include/qemu/atomic128.h +23 -6
@@ -44,13 +44,23 @@#if defined(CONFIG_ATOMIC128 )static inline Int128 atomic16_cmpxchg (Int128 *ptr,Int128 cmp,Int128 new) { -return qatomic_cmpxchg__nocheck (ptr, cmp, new); +Int128Alias r, c, n; + + c.s = cmp; + n.s = new; + r.i =qatomic_cmpxchg__nocheck ((__int128_t *)ptr, c.i, n.i); +return r.s; }# define HAVE_CMPXCHG128 1#elif defined(CONFIG_CMPXCHG128 )static inline Int128 atomic16_cmpxchg (Int128 *ptr,Int128 cmp,Int128 new) { -return __sync_val_compare_and_swap_16 (ptr, cmp, new); +Int128Alias r, c, n; + + c.s = cmp; + n.s = new; + r.i =__sync_val_compare_and_swap_16 ((__int128_t *)ptr, c.i, n.i); +return r.s; }# define HAVE_CMPXCHG128 1#elif defined(__aarch64__)
@@ -89,12 +99,18 @@ Int128 QEMU_ERROR("unsupported atomic")#if defined(CONFIG_ATOMIC128 )static inline Int128 atomic16_read (Int128 *ptr) { -return qatomic_read__nocheck (ptr); +Int128Alias r; + + r.i =qatomic_read__nocheck ((__int128_t *)ptr); +return r.s; }static inline void atomic16_set (Int128 *ptr,Int128 val) { -qatomic_set__nocheck (ptr, val); +Int128Alias v; + + v.s = val; +qatomic_set__nocheck ((__int128_t *)ptr, v.i); }# define HAVE_ATOMIC128 1
@@ -132,7 +148,8 @@ static inline void atomic16_set(Int128 *ptr, Int128 val)static inline Int128 atomic16_read (Int128 *ptr) {/* Maybe replace 0 with 0, returning the old value. */ -return atomic16_cmpxchg (ptr,0 ,0 ); +Int128 z =int128_make64 (0 ); +return atomic16_cmpxchg (ptr, z, z); }static inline void atomic16_set (Int128 *ptr,Int128 val)
@@ -141,7 +158,7 @@ static inline void atomic16_set(Int128 *ptr, Int128 val)do { cmp = old; old =atomic16_cmpxchg (ptr, cmp, val); - }while (old != cmp); + }while (int128_ne (old, cmp)); }# define HAVE_ATOMIC128 1
include/qemu/int128.h +22 -3
@@ -3,7 +3,12 @@#include "qemu/bswap.h" -#ifdef CONFIG_INT128 +/* +* With TCI, we need to use libffi for interfacing with TCG helpers. +* But libffi does not support __int128_t, and therefore cannot pass +* or return values of this type, force use of the Int128 struct. +*/ +#if defined(CONFIG_INT128 ) && !defined(CONFIG_TCG_INTERPRETER )typedef __int128_t Int128 ;static inline Int128 int128_make64 (uint64_t a)
@@ -460,8 +465,7 @@ Int128 int128_divu(Int128, Int128);Int128 int128_remu (Int128 ,Int128 );Int128 int128_divs (Int128 ,Int128 );Int128 int128_rems (Int128 ,Int128 ); - -#endif /* CONFIG_INT128 */ +#endif /* CONFIG_INT128 && !CONFIG_TCG_INTERPRETER */ static inline void bswap128s (Int128 *s) {
@@ -472,4 +476,19 @@ static inline void bswap128s(Int128 *s)#define INT128_MAX int128_make128(UINT64_MAX, INT64_MAX)#define INT128_MIN int128_make128(0, INT64_MIN) +/* +* When compiler supports a 128-bit type, define a combination of +* a possible structure and the native types. Ease parameter passing +* via use of the transparent union extension. +*/ +#ifdef CONFIG_INT128 +typedef union { +Int128 s; +__int128_t i; +__uint128_t u; +}Int128Alias __attribute__((transparent_union)); +#else +typedef Int128 Int128Alias ; +#endif /* CONFIG_INT128 */ +#endif /* INT128_H */
util/int128.c +42 -0
46 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v3 37/45 tcg/i386: Add TCG_TARGET_CALL_{RET, ARG}_I128
In reply to Richard HendersonFill in the parameters for the host ABI for Int128. Adjust tcg_target_call_oarg_reg for _WIN64, and tcg_out_call for i386 sysv. Allow TCG_TYPE_V128 stores without AVX enabled.
--- tcg/i386/tcg-target.h | 10 ++++++++++ tcg/i386/tcg-target.c.inc | 30 +++++++++++++++++++++++++++++- 2 files changed, 39 insertions(+), 1 deletion(-)
tcg/i386/tcg-target.h +10 -0
@@ -100,6 +100,16 @@ typedef enum {#endif #define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +#if defined(_WIN64) +# define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_BY_REF +# define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_BY_VEC +#elif TCG_TARGET_REG_BITS ==64 +# define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_NORMAL +# define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL +#else +# define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_NORMAL +# define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_BY_REF +#endif extern bool have_bmi1;extern bool have_popcnt;
tcg/i386/tcg-target.c.inc +29 -1
@@ -115,6 +115,11 @@ static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) case TCG_CALL_RET_NORMAL: tcg_debug_assert(slot >= 0 && slot <= 1); return slot ? TCG_REG_EDX : TCG_REG_EAX; +#ifdef _WIN64 + case TCG_CALL_RET_BY_VEC: + tcg_debug_assert(slot == 0); + return TCG_REG_XMM0; +#endif default: g_assert_not_reached(); }
@@ -1188,9 +1193,16 @@ static void tcg_out_st(TCGContext *s, TCGType type, TCGReg arg, * The gvec infrastructure is asserts that v128 vector loads * and stores use a 16-byte aligned offset. Validate that the * final pointer is aligned by using an insn that will SIGSEGV. + * + * This specific instance is also used by TCG_CALL_RET_BY_VEC, + * for _WIN64, which must have SSE2 but may not have AVX. */ tcg_debug_assert(arg >= 16); - tcg_out_vex_modrm_offset(s, OPC_MOVDQA_WxVx, arg, 0, arg1, arg2); + if (have_avx1) { + tcg_out_vex_modrm_offset(s, OPC_MOVDQA_WxVx, arg, 0, arg1, arg2); + } else { + tcg_out_modrm_offset(s, OPC_MOVDQA_WxVx, arg, arg1, arg2); + } break; case TCG_TYPE_V256: /*
@@ -1677,6 +1689,22 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest, const TCGHelperInfo *info) { tcg_out_branch(s, 1, dest); + +#ifndef _WIN32 + if (TCG_TARGET_REG_BITS == 32 && info->out_kind == TCG_CALL_RET_BY_REF) { + /* + * The sysv i386 abi for struct return places a reference as the + * first argument of the stack, and pops that argument with the + * return statement. Since we want to retain the aligned stack + * pointer for the callee, we do not want to actually push that + * argument before the call but rely on the normal store to the + * stack slot. But we do need to compensate for the pop in order + * to reset our correct stack pointer value. + * Pushing a garbage value back onto the stack is quickest. + */ + tcg_out_push(s, TCG_REG_EAX); + } +#endif } static void tcg_out_jmp(TCGContext *s, const tcg_insn_unit *dest)
Signature · 1 line
2.34.1
PATCH v3 38/45 tcg/tci: Fix big-endian return register ordering
In reply to Richard HendersonWe expect the backend to require register pairs in host-endian ordering, thus for big-endian the first register of a pair contains the high part.
We were forcing R0 to contain the low part for calls.
--- tcg/tci.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-)
tcg/tci.c +11 -10
@@ -520,27 +520,28 @@ uintptr_t QEMU_DISABLE_CFI tcg_qemu_tb_exec(CPUArchState *env,ffi_call (pptr[1 ], pptr[0 ], stack, call_slots); } -/* Any result winds up "left-aligned" in the stack[0] slot. */ switch (len) {case 0 :/* void */ break ;case 1 :/* uint32_t */ /* +* The result winds up "left-aligned" in the stack[0] slot. * Note that libffi has an odd special case in that it will * always widen an integral result to ffi_arg. */ -if (sizeof (ffi_arg) ==4 ) { - regs[TCG_REG_R0 ] = *(uint32_t *)stack; -break ; - } -/* fall through */ -case 2 :/* uint64_t */ -if (TCG_TARGET_REG_BITS ==32 ) { -tci_write_reg64 (regs,TCG_REG_R1 ,TCG_REG_R0 , stack[0 ]); +if (sizeof (ffi_arg) ==8 ) { + regs[TCG_REG_R0 ] = (uint32_t )stack[0 ]; }else { - regs[TCG_REG_R0 ] = stack[0 ]; + regs[TCG_REG_R0 ] = *(uint32_t *)stack; }break ; +case 2 :/* uint64_t */ +/* +* For TCG_TARGET_REG_BITS == 32, the register pair +* must stay in host memory order. +*/ +memcpy (®s[TCG_REG_R0 ], stack,8 ); +break ;default :g_assert_not_reached (); }
Signature · 1 line
2.34.1
PATCH v3 39/45 tcg/tci: Add TCG_TARGET_CALL_{RET,ARG}_I128
In reply to Richard HendersonFill in the parameters for libffi for Int128. Adjust the interpreter to allow for 16-byte return values. Adjust tcg_out_call to record the return value length.
Call parameters are no longer all the same size, so we cannot reuse the same call_slots array for every function. Compute it each time now, but only fill in slots required for the call we're about to make.
--- tcg/tci/tcg-target.h | 3 +++ tcg/tcg.c | 19 +++++++++++++++++ tcg/tci.c | 44 ++++++++++++++++++++-------------------- tcg/tci/tcg-target.c.inc | 10 ++++----- 4 files changed, 49 insertions(+), 27 deletions(-)
tcg/tci/tcg-target.h +3 -0
@@ -161,10 +161,13 @@ typedef enum {#if TCG_TARGET_REG_BITS ==32 # define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_EVEN# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN +# define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_EVEN#else # define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +# define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_NORMAL#endif +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL#define HAVE_TCG_QEMU_TB_EXEC #define TCG_TARGET_NEED_POOL_LABELS
tcg/tcg.c +19 -0
@@ -548,6 +548,22 @@ static GHashTable *helper_table;#ifdef CONFIG_TCG_INTERPRETER static ffi_type *typecode_to_ffi (int argmask) { +/* +* libffi does not support __int128_t, so we have forced Int128 +* to use the structure definition instead of the builtin type. +*/ +static ffi_type *ffi_type_i128_elements[3 ] = { + &ffi_type_uint64, + &ffi_type_uint64, +NULL + }; +static ffi_type ffi_type_i128 = { + .size =16 , + .alignment = __alignof__(Int128 ), + .type =FFI_TYPE_STRUCT , + .elements = ffi_type_i128_elements, + }; +switch (argmask) {case dh_typecode_void:return &ffi_type_void;
@@ -561,6 +577,8 @@ static ffi_type *typecode_to_ffi(int argmask)return &ffi_type_sint64;case dh_typecode_ptr:return &ffi_type_pointer; +case dh_typecode_i128: +return &ffi_type_i128; }g_assert_not_reached (); }
@@ -591,6 +609,7 @@ static void init_ffi_layouts(void)/* Ignoring the return type, find the last non-zero field. */ nargs =32 -clz32 (typemask >>3 ); nargs =DIV_ROUND_UP (nargs,3 ); +assert (nargs <=MAX_CALL_IARGS ); ca =g_malloc0 (sizeof (*ca) + nargs *sizeof (ffi_type *)); ca->cif.rtype =typecode_to_ffi (typemask &7 );
tcg/tci.c +22 -22
66 lines not on this page · raw message
tcg/tci/tcg-target.c.inc +5 -5
@@ -203,7 +203,7 @@ static const int tcg_target_call_iarg_regs[] = { }; static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) { tcg_debug_assert(kind == TCG_CALL_RET_NORMAL); - tcg_debug_assert(slot >= 0 && slot < 64 / TCG_TARGET_REG_BITS); + tcg_debug_assert(slot >= 0 && slot < 128 / TCG_TARGET_REG_BITS); return TCG_REG_R0 + slot; }
@@ -573,11 +573,11 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *func, if (cif->rtype == &ffi_type_void) { which = 0; - } else if (cif->rtype->size == 4) { - which = 1; } else { - tcg_debug_assert(cif->rtype->size == 8); - which = 2; + tcg_debug_assert(cif->rtype->size == 4 || + cif->rtype->size == 8 || + cif->rtype->size == 16); + which = ctz32(cif->rtype->size) - 1; } new_pool_l2(s, 20, s->code_ptr, 0, (uintptr_t)func, (uintptr_t)cif); insn = deposit32(insn, 0, 8, INDEX_op_call);
Signature · 1 line
2.34.1
PATCH v3 40/45 tcg: Add TCG_TARGET_CALL_{RET,ARG}_I128
In reply to Richard HendersonFill in the parameters for the host ABI for Int128 for those backends which require no extra modification.
--- tcg/aarch64/tcg-target.h | 2 ++ tcg/arm/tcg-target.h | 2 ++ tcg/loongarch64/tcg-target.h | 2 ++ tcg/mips/tcg-target.h | 2 ++ tcg/riscv/tcg-target.h | 3 +++ tcg/s390x/tcg-target.h | 2 ++ tcg/sparc64/tcg-target.h | 2 ++ tcg/tcg.c | 6 +++--- tcg/ppc/tcg-target.c.inc | 3 +++ 9 files changed, 21 insertions(+), 3 deletions(-)
tcg/aarch64/tcg-target.h +2 -0
@@ -54,6 +54,8 @@ typedef enum {#define TCG_TARGET_CALL_STACK_OFFSET 0#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL/* optional instructions */ #define TCG_TARGET_HAS_div_i32 1
tcg/arm/tcg-target.h +2 -0
@@ -91,6 +91,8 @@ extern bool use_neon_instructions;#define TCG_TARGET_CALL_STACK_OFFSET 0#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_EVEN +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_BY_REF/* optional instructions */ #define TCG_TARGET_HAS_ext8s_i32 1
tcg/loongarch64/tcg-target.h +2 -0
@@ -95,6 +95,8 @@ typedef enum {#define TCG_TARGET_CALL_STACK_OFFSET 0#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL/* optional instructions */ #define TCG_TARGET_HAS_movcond_i32 0
tcg/mips/tcg-target.h +2 -0
@@ -89,6 +89,8 @@ typedef enum {# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL#endif #define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_EVEN +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL/* MOVN/MOVZ instructions detection */ #if (defined(__mips_isa_rev) && (__mips_isa_rev >= 1)) || \
tcg/riscv/tcg-target.h +3 -0
@@ -85,9 +85,12 @@ typedef enum {#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#if TCG_TARGET_REG_BITS ==32 #define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_EVEN#else #define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_NORMAL#endif +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL/* optional instructions */ #define TCG_TARGET_HAS_movcond_i32 0
tcg/s390x/tcg-target.h +2 -0
@@ -168,6 +168,8 @@ extern uint64_t s390_facilities[3];#define TCG_TARGET_CALL_STACK_OFFSET 160#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_EXTEND#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_RET_NORMAL +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_BY_REF +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_BY_REF#define TCG_TARGET_HAS_MEMORY_BSWAP 1
tcg/sparc64/tcg-target.h +2 -0
@@ -73,6 +73,8 @@ typedef enum {#define TCG_TARGET_CALL_STACK_OFFSET (128 + 6*8 + TCG_TARGET_STACK_BIAS)#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_EXTEND#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL#if defined(__VIS__) && __VIS__ >=0x300 #define use_vis3_instructions 1
tcg/tcg.c +3 -3
@@ -743,8 +743,8 @@ static void init_call_layout(TCGHelperInfo *info)break ;case dh_typecode_i128: info->nr_out =128 /TCG_TARGET_REG_BITS ; - info->out_kind =TCG_CALL_RET_NORMAL ;/* TODO */ -switch (/* TODO */ TCG_CALL_RET_NORMAL ) { + info->out_kind =TCG_TARGET_CALL_RET_I128 ; +switch (TCG_TARGET_CALL_RET_I128 ) {case TCG_CALL_RET_NORMAL :/* Query the last register now to trigger any assert early. */ tcg_target_call_oarg_reg (info->out_kind, info->nr_out -1 );
@@ -815,7 +815,7 @@ static void init_call_layout(TCGHelperInfo *info)layout_arg_1 (&cum, info,TCG_CALL_ARG_NORMAL );break ;case dh_typecode_i128: -switch (/* TODO */ TCG_CALL_ARG_NORMAL ) { +switch (TCG_TARGET_CALL_ARG_I128 ) {case TCG_CALL_ARG_EVEN :layout_arg_even (&cum);/* fall through */
tcg/ppc/tcg-target.c.inc +3 -0
@@ -54,6 +54,9 @@ #else # define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL #endif +/* Note sysv arg alignment applies only to 2-word types, not more. */ +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL /* For some memory operations, we need a scratch that isn't R0. For the AIX calling convention, we can re-use the TOC register since we'll be reloading
Signature · 1 line
2.34.1
Re: [PATCH for-8.0 v3 40/45] tcg: Add TCG_TARGET_CALL_{RET,ARG}_I128
Quoted 5 lines · Richard Henderson
Fill in the parameters for the host ABI for Int128 for those backends which require no extra modification.
Signed-off-by Richard Henderson <[email protected]>---
Quoted 144 lines
tcg/aarch64/tcg-target.h | 2 ++ tcg/arm/tcg-target.h | 2 ++ tcg/loongarch64/tcg-target.h | 2 ++ tcg/mips/tcg-target.h | 2 ++ tcg/riscv/tcg-target.h | 3 +++ tcg/s390x/tcg-target.h | 2 ++ tcg/sparc64/tcg-target.h | 2 ++ tcg/tcg.c | 6 +++--- tcg/ppc/tcg-target.c.inc | 3 +++ 9 files changed, 21 insertions(+), 3 deletions(-)tcg/aarch64/tcg-target.h +2
-0@@ -54,6 +54,8 @@ typedef enum {#define TCG_TARGET_CALL_STACK_OFFSET 0#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL/* optional instructions */ #define TCG_TARGET_HAS_div_i32 1tcg/arm/tcg-target.h +2
-0@@ -91,6 +91,8 @@ extern bool use_neon_instructions;#define TCG_TARGET_CALL_STACK_OFFSET 0#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_EVEN +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_BY_REF/* optional instructions */ #define TCG_TARGET_HAS_ext8s_i32 1tcg/loongarch64/tcg-target.h +2
-0@@ -95,6 +95,8 @@ typedef enum {#define TCG_TARGET_CALL_STACK_OFFSET 0#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL/* optional instructions */ #define TCG_TARGET_HAS_movcond_i32 0tcg/mips/tcg-target.h +2
-0@@ -89,6 +89,8 @@ typedef enum {# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL#endif #define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_EVEN +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL/* MOVN/MOVZ instructions detection */ #if (defined(__mips_isa_rev) && (__mips_isa_rev >= 1)) || \tcg/riscv/tcg-target.h +3
-0@@ -85,9 +85,12 @@ typedef enum {#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#if TCG_TARGET_REG_BITS ==32 #define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_EVEN#else #define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_NORMAL#endif +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL/* optional instructions */ #define TCG_TARGET_HAS_movcond_i32 0tcg/s390x/tcg-target.h +2
-0@@ -168,6 +168,8 @@ extern uint64_t s390_facilities[3];#define TCG_TARGET_CALL_STACK_OFFSET 160#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_EXTEND#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_RET_NORMAL +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_BY_REF +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_BY_REF#define TCG_TARGET_HAS_MEMORY_BSWAP 1tcg/sparc64/tcg-target.h +2
-0@@ -73,6 +73,8 @@ typedef enum {#define TCG_TARGET_CALL_STACK_OFFSET (128 + 6*8 + TCG_TARGET_STACK_BIAS)#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_EXTEND#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL#if defined(__VIS__) && __VIS__ >=0x300 #define use_vis3_instructions 1tcg/tcg.c +3
-3@@ -743,8 +743,8 @@ static void init_call_layout(TCGHelperInfo *info)break ;case dh_typecode_i128: info->nr_out =128 /TCG_TARGET_REG_BITS ; - info->out_kind =TCG_CALL_RET_NORMAL ;/* TODO */ -switch (/* TODO */ TCG_CALL_RET_NORMAL ) { + info->out_kind =TCG_TARGET_CALL_RET_I128 ; +switch (TCG_TARGET_CALL_RET_I128 ) {case TCG_CALL_RET_NORMAL :/* Query the last register now to trigger any assert early. */ tcg_target_call_oarg_reg (info->out_kind, info->nr_out -1 );@@ -815,7 +815,7 @@ static void init_call_layout(TCGHelperInfo *info)layout_arg_1 (&cum, info,TCG_CALL_ARG_NORMAL );break ;case dh_typecode_i128: -switch (/* TODO */ TCG_CALL_ARG_NORMAL ) { +switch (TCG_TARGET_CALL_ARG_I128 ) {case TCG_CALL_ARG_EVEN :layout_arg_even (&cum);/* fall through */ tcg/ppc/tcg-target.c.inc +3
-0@@ -54,6 +54,9 @@ #else # define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL #endif +/* Note sysv arg alignment applies only to 2-word types, not more. */ +#define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_NORMAL +#define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL /* For some memory operations, we need a scratch that isn't R0. For the AIX calling convention, we can re-use the TOC register since we'll be reloading
PATCH v3 41/45 tcg: Add temp allocation for TCGv_i128
In reply to Richard HendersonThis enables allocation of i128. The type is not yet usable, as we have not yet added data movement ops.
--- include/tcg/tcg.h | 32 +++++++++++++++++++++++++ tcg/tcg.c | 60 +++++++++++++++++++++++++++++++++-------------- 2 files changed, 74 insertions(+), 18 deletions(-)
include/tcg/tcg.h +32 -0
@@ -697,6 +697,11 @@ static inline TCGTemp *tcgv_i64_temp(TCGv_i64 v)return tcgv_i32_temp ((TCGv_i32 )v); } +static inline TCGTemp *tcgv_i128_temp (TCGv_i128 v) +{ +return tcgv_i32_temp ((TCGv_i32 )v); +} +static inline TCGTemp *tcgv_ptr_temp (TCGv_ptr v) {return tcgv_i32_temp ((TCGv_i32 )v);
@@ -717,6 +722,11 @@ static inline TCGArg tcgv_i64_arg(TCGv_i64 v)return temp_arg (tcgv_i64_temp (v)); } +static inline TCGArg tcgv_i128_arg (TCGv_i128 v) +{ +return temp_arg (tcgv_i128_temp (v)); +} +static inline TCGArg tcgv_ptr_arg (TCGv_ptr v) {return temp_arg (tcgv_ptr_temp (v));
@@ -738,6 +748,11 @@ static inline TCGv_i64 temp_tcgv_i64(TCGTemp *t)return (TCGv_i64 )temp_tcgv_i32 (t); } +static inline TCGv_i128 temp_tcgv_i128 (TCGTemp *t) +{ +return (TCGv_i128 )temp_tcgv_i32 (t); +} +static inline TCGv_ptr temp_tcgv_ptr (TCGTemp *t) {return (TCGv_ptr )temp_tcgv_i32 (t);
@@ -860,6 +875,11 @@ static inline void tcg_temp_free_i64(TCGv_i64 arg)tcg_temp_free_internal (tcgv_i64_temp (arg)); } +static inline void tcg_temp_free_i128 (TCGv_i128 arg) +{ +tcg_temp_free_internal (tcgv_i128_temp (arg)); +} +static inline void tcg_temp_free_ptr (TCGv_ptr arg) {tcg_temp_free_internal (tcgv_ptr_temp (arg));
@@ -908,6 +928,18 @@ static inline TCGv_i64 tcg_temp_local_new_i64(void)return temp_tcgv_i64 (t); } +static inline TCGv_i128 tcg_temp_new_i128 (void ) +{ +TCGTemp *t =tcg_temp_new_internal (TCG_TYPE_I128 , false); +return temp_tcgv_i128 (t); +} + +static inline TCGv_i128 tcg_temp_local_new_i128 (void ) +{ +TCGTemp *t =tcg_temp_new_internal (TCG_TYPE_I128 , true); +return temp_tcgv_i128 (t); +} +static inline TCGv_ptr tcg_global_mem_new_ptr (TCGv_ptr reg,intptr_t offset,const char *name) {
tcg/tcg.c +42 -18
77 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v3 42/45 tcg: Add basic data movement for TCGv_i128
In reply to Richard HendersonAdd code generation functions for data movement between TCGv_i128 (mov) and to/from TCGv_i64 (concat, extract).
--- include/tcg/tcg-op.h | 4 ++++ tcg/tcg-internal.h | 13 +++++++++++++ tcg/tcg-op.c | 20 ++++++++++++++++++++ 3 files changed, 37 insertions(+)
include/tcg/tcg-op.h +4 -0
@@ -712,6 +712,10 @@ void tcg_gen_extrh_i64_i32(TCGv_i32 ret, TCGv_i64 arg);void tcg_gen_extr_i64_i32 (TCGv_i32 lo,TCGv_i32 hi,TCGv_i64 arg);void tcg_gen_extr32_i64 (TCGv_i64 lo,TCGv_i64 hi,TCGv_i64 arg); +void tcg_gen_mov_i128 (TCGv_i128 dst,TCGv_i128 src); +void tcg_gen_extr_i128_i64 (TCGv_i64 lo,TCGv_i64 hi,TCGv_i128 arg); +void tcg_gen_concat_i64_i128 (TCGv_i128 ret,TCGv_i64 lo,TCGv_i64 hi); +static inline void tcg_gen_concat32_i64 (TCGv_i64 ret,TCGv_i64 lo,TCGv_i64 hi) {tcg_gen_deposit_i64 (ret, lo, hi,32 ,32 );
tcg/tcg-internal.h +13 -0
@@ -117,4 +117,17 @@ extern TCGv_i32 TCGV_LOW(TCGv_i64) QEMU_ERROR("32-bit code path is reachable"); extern TCGv_i32 TCGV_HIGH(TCGv_i64) QEMU_ERROR("32-bit code path is reachable");#endif +static inline TCGv_i64 TCGV128_LOW (TCGv_i128 t) +{ +/* For 32-bit, offset by 2, which may then have TCGV_{LOW,HIGH} applied. */ +int o =HOST_BIG_ENDIAN ?64 /TCG_TARGET_REG_BITS :0 ; +return temp_tcgv_i64 (tcgv_i128_temp (t) + o); +} + +static inline TCGv_i64 TCGV128_HIGH (TCGv_i128 t) +{ +int o =HOST_BIG_ENDIAN ?0 :64 /TCG_TARGET_REG_BITS ; +return temp_tcgv_i64 (tcgv_i128_temp (t) + o); +} +#endif /* TCG_INTERNAL_H */
tcg/tcg-op.c +20 -0
@@ -2745,6 +2745,26 @@ void tcg_gen_extr32_i64(TCGv_i64 lo, TCGv_i64 hi, TCGv_i64 arg)tcg_gen_shri_i64 (hi, arg,32 ); } +void tcg_gen_extr_i128_i64 (TCGv_i64 lo,TCGv_i64 hi,TCGv_i128 arg) +{ +tcg_gen_mov_i64 (lo,TCGV128_LOW (arg)); +tcg_gen_mov_i64 (hi,TCGV128_HIGH (arg)); +} + +void tcg_gen_concat_i64_i128 (TCGv_i128 ret,TCGv_i64 lo,TCGv_i64 hi) +{ +tcg_gen_mov_i64 (TCGV128_LOW (ret), lo); +tcg_gen_mov_i64 (TCGV128_HIGH (ret), hi); +} + +void tcg_gen_mov_i128 (TCGv_i128 dst,TCGv_i128 src) +{ +if (dst != src) { +tcg_gen_mov_i64 (TCGV128_LOW (dst),TCGV128_LOW (src)); +tcg_gen_mov_i64 (TCGV128_HIGH (dst),TCGV128_HIGH (src)); + } +} +/* QEMU specific operations. */ void tcg_gen_exit_tb (const TranslationBlock *tb,unsigned idx)
Signature · 1 line
2.34.1
PATCH v3 43/45 tcg: Add guest load/store primitives for TCGv_i128
In reply to Richard HendersonThese are not yet considering atomicity of the 16-byte value; this is a direct replacement for the current target code which uses a pair of 8-byte operations.
--- include/exec/cpu_ldst.h | 10 +++ include/tcg/tcg-op.h | 2 + accel/tcg/cputlb.c | 112 +++++++++++++++++++++++++++++++++ accel/tcg/user-exec.c | 66 ++++++++++++++++++++ tcg/tcg-op.c | 134 ++++++++++++++++++++++++++++++++++++++++ 5 files changed, 324 insertions(+)
include/exec/cpu_ldst.h +10 -0
@@ -220,6 +220,11 @@ uint32_t cpu_ldl_le_mmu(CPUArchState *env, abi_ptr ptr,uint64_t cpu_ldq_le_mmu (CPUArchState *env,abi_ptr ptr,MemOpIdx oi,uintptr_t ra); +Int128 cpu_ld16_be_mmu (CPUArchState *env,abi_ptr addr, +MemOpIdx oi,uintptr_t ra); +Int128 cpu_ld16_le_mmu (CPUArchState *env,abi_ptr addr, +MemOpIdx oi,uintptr_t ra); +void cpu_stb_mmu (CPUArchState *env,abi_ptr ptr,uint8_t val,MemOpIdx oi,uintptr_t ra); void cpu_stw_be_mmu(CPUArchState *env, abi_ptr ptr, uint16_t val,
@@ -235,6 +240,11 @@ void cpu_stl_le_mmu(CPUArchState *env, abi_ptr ptr, uint32_t val,void cpu_stq_le_mmu (CPUArchState *env,abi_ptr ptr,uint64_t val,MemOpIdx oi,uintptr_t ra); +void cpu_st16_be_mmu (CPUArchState *env,abi_ptr addr,Int128 val, +MemOpIdx oi,uintptr_t ra); +void cpu_st16_le_mmu (CPUArchState *env,abi_ptr addr,Int128 val, +MemOpIdx oi,uintptr_t ra); +uint32_t cpu_atomic_cmpxchgb_mmu (CPUArchState *env,target_ulong addr,uint32_t cmpv,uint32_t newv,MemOpIdx oi,uintptr_t retaddr);
include/tcg/tcg-op.h +2 -0
@@ -845,6 +845,8 @@ void tcg_gen_qemu_ld_i32(TCGv_i32, TCGv, TCGArg, MemOp);void tcg_gen_qemu_st_i32 (TCGv_i32 ,TCGv ,TCGArg ,MemOp );void tcg_gen_qemu_ld_i64 (TCGv_i64 ,TCGv ,TCGArg ,MemOp );void tcg_gen_qemu_st_i64 (TCGv_i64 ,TCGv ,TCGArg ,MemOp ); +void tcg_gen_qemu_ld_i128 (TCGv_i128 ,TCGv ,TCGArg ,MemOp ); +void tcg_gen_qemu_st_i128 (TCGv_i128 ,TCGv ,TCGArg ,MemOp );static inline void tcg_gen_qemu_ld8u (TCGv ret,TCGv addr,int mem_index) {
accel/tcg/cputlb.c +112 -0
124 lines not on this page · raw message
accel/tcg/user-exec.c +66 -0
78 lines not on this page · raw message
tcg/tcg-op.c +134 -0
140 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v3 44/45 tcg: Add tcg_gen_{non}atomic_cmpxchg_i128
In reply to Richard HendersonThis will allow targets to avoid rolling their own.
--- accel/tcg/tcg-runtime.h | 11 +++++ include/tcg/tcg-op.h | 5 +++ tcg/tcg-op.c | 85 +++++++++++++++++++++++++++++++++++ accel/tcg/atomic_common.c.inc | 45 +++++++++++++++++++ 4 files changed, 146 insertions(+)
accel/tcg/tcg-runtime.h +11 -0
@@ -55,6 +55,17 @@ DEF_HELPER_FLAGS_5(atomic_cmpxchgq_be, TCG_CALL_NO_WG,DEF_HELPER_FLAGS_5 (atomic_cmpxchgq_le,TCG_CALL_NO_WG , i64, env, tl, i64, i64, i32)#endif +#ifdef CONFIG_CMPXCHG128 +DEF_HELPER_FLAGS_5 (atomic_cmpxchgo_be,TCG_CALL_NO_WG , + i128, env, tl, i128, i128, i32) +DEF_HELPER_FLAGS_5 (atomic_cmpxchgo_le,TCG_CALL_NO_WG , + i128, env, tl, i128, i128, i32) +#endif + +DEF_HELPER_FLAGS_5 (nonatomic_cmpxchgo_be,TCG_CALL_NO_WG , + i128, env, tl, i128, i128, i32) +DEF_HELPER_FLAGS_5 (nonatomic_cmpxchgo_le,TCG_CALL_NO_WG , + i128, env, tl, i128, i128, i32)#ifdef CONFIG_ATOMIC64 #define GEN_ATOMIC_HELPERS (NAME ) \
include/tcg/tcg-op.h +5 -0
@@ -907,6 +907,11 @@ void tcg_gen_atomic_cmpxchg_i32(TCGv_i32, TCGv, TCGv_i32, TCGv_i32, TCGArg, MemOp);void tcg_gen_atomic_cmpxchg_i64 (TCGv_i64 ,TCGv ,TCGv_i64 ,TCGv_i64 ,TCGArg ,MemOp ); +void tcg_gen_atomic_cmpxchg_i128 (TCGv_i128 ,TCGv ,TCGv_i128 ,TCGv_i128 , +TCGArg ,MemOp ); + +void tcg_gen_nonatomic_cmpxchg_i128 (TCGv_i128 ,TCGv ,TCGv_i128 ,TCGv_i128 , +TCGArg ,MemOp );void tcg_gen_atomic_xchg_i32 (TCGv_i32 ,TCGv ,TCGv_i32 ,TCGArg ,MemOp );void tcg_gen_atomic_xchg_i64 (TCGv_i64 ,TCGv ,TCGv_i64 ,TCGArg ,MemOp );
tcg/tcg-op.c +85 -0
109 lines not on this page · raw message
accel/tcg/atomic_common.c.inc +45 -0
53 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v3 45/45 tcg: Split out tcg_gen_nonatomic_cmpxchg_i{32, 64}
In reply to Richard HendersonNormally this is automatically handled by the CF_PARALLEL checks with in tcg_gen_atomic_cmpxchg_i{32,64}, but x86 has a special case of !PREFIX_LOCK where it always wants the non-atomic version.
Split these out so that x86 does not have to roll its own.
--- include/tcg/tcg-op.h | 4 ++ tcg/tcg-op.c | 154 +++++++++++++++++++++++++++---------------- 2 files changed, 101 insertions(+), 57 deletions(-)
include/tcg/tcg-op.h +4 -0
@@ -910,6 +910,10 @@ void tcg_gen_atomic_cmpxchg_i64(TCGv_i64, TCGv, TCGv_i64, TCGv_i64,void tcg_gen_atomic_cmpxchg_i128 (TCGv_i128 ,TCGv ,TCGv_i128 ,TCGv_i128 ,TCGArg ,MemOp ); +void tcg_gen_nonatomic_cmpxchg_i32 (TCGv_i32 ,TCGv ,TCGv_i32 ,TCGv_i32 , +TCGArg ,MemOp ); +void tcg_gen_nonatomic_cmpxchg_i64 (TCGv_i64 ,TCGv ,TCGv_i64 ,TCGv_i64 , +TCGArg ,MemOp );void tcg_gen_nonatomic_cmpxchg_i128 (TCGv_i128 ,TCGv ,TCGv_i128 ,TCGv_i128 ,TCGArg ,MemOp );
tcg/tcg-op.c +97 -57
179 lines not on this page · raw message
Signature · 1 line
2.34.1