tcg misc patches
v4 cover tcg misc patches
For v4, I've sorted some of the reviewed patches to the beginning of the queue and omitted them; this was not possible for all. This set begins with the first patch that is missing review. I've not tested this set separately, so in case it doesn't apply, the entire tree is
https://gitlab.com/rth7680/qemu/-/tree/tcg-next
Patches lacking review:
01-tcg-Fix-tcg_reg_alloc_dup.patch 02-tcg-Centralize-updates-to-reg_to_temp.patch 03-tcg-Remove-check_regs.patch 04-tcg-Introduce-paired-register-allocation.patch 05-accel-tcg-Set-cflags_next_tb-in-cpu_common_initfn.patch 06-target-sparc-Avoid-TCGV_-LOW-HIGH.patch 09-tcg-Simplify-calls-to-temp_sync-vs-mem_coherent.patch 17-accel-tcg-plugin-Don-t-search-for-the-function-po.patch 20-tcg-Vary-the-allocation-size-for-TCGOp.patch 22-tcg-Reorg-function-calls.patch 23-tcg-Convert-typecode_to_ffi-from-array-to-functio.patch 24-tcg-Factor-init_ffi_layouts-out-of-tcg_context_in.patch 25-tcg-Move-ffi_cif-pointer-into-TCGHelperInfo.patch 26-tcg-aarch64-Merge-tcg_out_callr-into-tcg_out_call.patch
r~
Philippe Mathieu-Daudé (2):
tcg: Convert typecode_to_ffi from array to function
tcg: Factor init_ffi_layouts() out of tcg_context_init()
Richard Henderson (25):
tcg: Fix tcg_reg_alloc_dup*
tcg: Centralize updates to reg_to_temp
tcg: Remove check_regs
tcg: Introduce paired register allocation
accel/tcg: Set cflags_next_tb in cpu_common_initfn target/sparc: Avoid TCGV_{LOW,HIGH}
tcg: Move TCG_{LOW,HIGH} to tcg-internal.h
tcg: Add temp_subindex to TCGTemp
tcg: Simplify calls to temp_sync vs mem_coherent
tcg: Allocate TCGTemp pairs in host memory order
tcg: Move TCG_TYPE_COUNT outside enum
tcg: Introduce tcg_type_size
tcg: Introduce TCGCallReturnKind and TCGCallArgumentKind
tcg: Replace TCG_TARGET_CALL_ALIGN_ARGS with TCG_TARGET_CALL_ARG_I64
tcg: Replace TCG_TARGET_EXTEND_ARGS with TCG_TARGET_CALL_ARG_I32
tcg: Use TCG_CALL_ARG_EVEN for TCI special case accel/tcg/plugin: Don't search for the function pointer index accel/tcg/plugin: Avoid duplicate copy in copy_call accel/tcg/plugin: Use copy_op in append_{udata,mem}_cb
tcg: Vary the allocation size for TCGOp
tcg: Use output_pref wrapper function
tcg: Reorg function calls
tcg: Move ffi_cif pointer into TCGHelperInfo
tcg/aarch64: Merge tcg_out_callr into tcg_out_call
tcg: Add TCGHelperInfo argument to tcg_out_call
include/exec/helper-head.h | 2 +- include/tcg/tcg-op.h | 35 +- include/tcg/tcg.h | 96 +- tcg/aarch64/tcg-target.h | 3 +- tcg/arm/tcg-target.h | 3 +- tcg/i386/tcg-target.h | 2 + tcg/loongarch64/tcg-target.h | 3 +- tcg/mips/tcg-target.h | 4 +- tcg/riscv/tcg-target.h | 7 +- tcg/s390x/tcg-target.h | 3 +- tcg/sparc64/tcg-target.h | 3 +- tcg/tcg-internal.h | 58 +- tcg/tci/tcg-target.h | 7 + accel/tcg/plugin-gen.c | 54 +- hw/core/cpu-common.c | 1 + target/sparc/translate.c | 21 +- tcg/optimize.c | 10 +- tcg/tcg-op-vec.c | 10 +- tcg/tcg-op.c | 49 +- tcg/tcg.c | 1591 +++++++++++++++++++----------- tcg/aarch64/tcg-target.c.inc | 19 +- tcg/arm/tcg-target.c.inc | 10 +- tcg/i386/tcg-target.c.inc | 5 +- tcg/loongarch64/tcg-target.c.inc | 7 +- tcg/mips/tcg-target.c.inc | 3 +- tcg/ppc/tcg-target.c.inc | 34 +- tcg/riscv/tcg-target.c.inc | 7 +- tcg/s390x/tcg-target.c.inc | 12 +- tcg/sparc64/tcg-target.c.inc | 3 +- tcg/tci/tcg-target.c.inc | 3 +- 30 files changed, 1285 insertions(+), 780 deletions(-)
Signature · 1 line
2.34.1
PATCH v4 1/27 tcg: Fix tcg_reg_alloc_dup*
The assignment to mem_coherent should be done with any modification, not simply with a newly allocated register.
--- tcg/tcg.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tcg/tcg.c +2 -2
@@ -3498,7 +3498,6 @@ static void tcg_reg_alloc_dup(TCGContext *s, const TCGOp *op) ots->reg =tcg_reg_alloc (s, dup_out_regs, allocated_regs, op->output_pref[0 ], ots->indirect_base); ots->val_type =TEMP_VAL_REG ; - ots->mem_coherent =0 ; s->reg_to_temp[ots->reg] = ots; }
@@ -3552,6 +3551,7 @@ static void tcg_reg_alloc_dup(TCGContext *s, const TCGOp *op)tcg_debug_assert (ok); done: + ots->mem_coherent =0 ;if (IS_DEAD_ARG (1 )) {temp_dead (s, its); }
@@ -3779,7 +3779,6 @@ static bool tcg_reg_alloc_dup2(TCGContext *s, const TCGOp *op) ots->reg =tcg_reg_alloc (s, dup_out_regs, allocated_regs, op->output_pref[0 ], ots->indirect_base); ots->val_type =TEMP_VAL_REG ; - ots->mem_coherent =0 ; s->reg_to_temp[ots->reg] = ots; }
@@ -3823,6 +3822,7 @@ static bool tcg_reg_alloc_dup2(TCGContext *s, const TCGOp *op)return false; done: + ots->mem_coherent =0 ;if (IS_DEAD_ARG (1 )) {temp_dead (s, itsl); }
Signature · 1 line
2.34.1
Re: [PATCH v4 01/27] tcg: Fix tcg_reg_alloc_dup*
Quoted 2 lines · Richard Henderson
The assignment to
mem_coherentshould be done with any modification, not simply with a newly allocated register.
What exactly does mem_coherent mean in this case? Is it that our register store is potentially out of sync with live values in temp regs or that we have memory loads and stores in flight?
I think it would be useful to add a doc patch for TCGTemp do specify what the various fields mean. It would certainly help reviewers that don't have it committed to memory ;-)
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
Re: [PATCH v4 01/27] tcg: Fix tcg_reg_alloc_dup*
Quoted 6 lines · Alex Bennée
Quoted 2 lines · Richard Henderson
The assignment to
mem_coherentshould be done with any modification, not simply with a newly allocated register.
Quoted 8 lines
What exactly does
mem_coherentmean in this case? Is it that our register store is potentially out of sync with live values in temp regs or that we have memory loads and stores in flight?I think it would be useful to add a doc patch for TCGTemp do specify what the various fields mean. It would certainly help reviewers that don't have it committed to memory ;-)
mem_coherent means that the register value and the memory value are the same, so that if we must release the register we do not need to save the value back to memory.
r~
PATCH v4 2/27 tcg: Centralize updates to reg_to_temp
In reply to Richard HendersonCreate two new functions, set_temp_val_{reg,nonreg}. Assert that the reg_to_temp mapping is correct before any changes are made.
--- tcg/tcg.c | 159 +++++++++++++++++++++++++++++------------------------- 1 file changed, 85 insertions(+), 74 deletions(-)
tcg/tcg.c +85 -74
@@ -3019,6 +3019,35 @@ static void temp_allocate_frame(TCGContext *s, TCGTemp *ts) ts->mem_allocated =1 ; } +/* Assign @reg to @ts, and update reg_to_temp[]. */ +static void set_temp_val_reg (TCGContext *s,TCGTemp *ts,TCGReg reg) +{ +if (ts->val_type ==TEMP_VAL_REG ) { +TCGReg old = ts->reg; +tcg_debug_assert (s->reg_to_temp[old] == ts); +if (old == reg) { +return ; + } + s->reg_to_temp[old] =NULL ; + } +tcg_debug_assert (s->reg_to_temp[reg] ==NULL ); + s->reg_to_temp[reg] = ts; + ts->val_type =TEMP_VAL_REG ; + ts->reg = reg; +} + +/* Assign a non-register value type to @ts, and update reg_to_temp[]. */ +static void set_temp_val_nonreg (TCGContext *s,TCGTemp *ts,TCGTempVal type) +{ +tcg_debug_assert (type !=TEMP_VAL_REG ); +if (ts->val_type ==TEMP_VAL_REG ) { +TCGReg reg = ts->reg; +tcg_debug_assert (s->reg_to_temp[reg] == ts); + s->reg_to_temp[reg] =NULL ; + } + ts->val_type = type; +} +static void temp_load (TCGContext *,TCGTemp *,TCGRegSet ,TCGRegSet ,TCGRegSet );/* Mark a temporary as free or dead. If 'free_or_dead' is negative,
@@ -3044,10 +3073,7 @@ static void temp_free_or_dead(TCGContext *s, TCGTemp *ts, int free_or_dead)default :g_assert_not_reached (); } -if (ts->val_type ==TEMP_VAL_REG ) { - s->reg_to_temp[ts->reg] =NULL ; - } - ts->val_type = new_type; +set_temp_val_nonreg (s, ts, new_type); }/* Mark a temporary as dead. */
@@ -3227,9 +3253,7 @@ static void temp_load(TCGContext *s, TCGTemp *ts, TCGRegSet desired_regs,default :tcg_abort (); } - ts->reg = reg; - ts->val_type =TEMP_VAL_REG ; - s->reg_to_temp[reg] = ts; +set_temp_val_reg (s, ts, reg); }/* Save a temporary to memory. 'allocated_regs' is used in case a
@@ -3341,10 +3365,7 @@ static void tcg_reg_alloc_do_movi(TCGContext *s, TCGTemp *ots,tcg_debug_assert (!temp_readonly (ots));/* The movi is not explicitly generated here. */ -if (ots->val_type ==TEMP_VAL_REG ) { - s->reg_to_temp[ots->reg] =NULL ; - } - ots->val_type =TEMP_VAL_CONST ; +set_temp_val_nonreg (s, ots,TEMP_VAL_CONST ); ots->val = val; ots->mem_coherent =0 ;if (NEED_SYNC_ARG (0 )) {
@@ -3363,6 +3384,7 @@ static void tcg_reg_alloc_mov(TCGContext *s, const TCGOp *op)TCGRegSet allocated_regs, preferred_regs;TCGTemp *ts, *ots;TCGType otype, itype; +TCGReg oreg, ireg; allocated_regs = s->reserved_regs; preferred_regs = op->output_pref[0 ];
@@ -3394,8 +3416,9 @@ static void tcg_reg_alloc_mov(TCGContext *s, const TCGOp *op)temp_load (s, ts, tcg_target_available_regs[itype], allocated_regs, preferred_regs); } -tcg_debug_assert (ts->val_type ==TEMP_VAL_REG ); + ireg = ts->reg; +if (IS_DEAD_ARG (0 )) {/* mov to a non-saved dead register makes no sense (even with liveness analysis disabled). */
@@ -3403,52 +3426,53 @@ static void tcg_reg_alloc_mov(TCGContext *s, const TCGOp *op)if (!ots->mem_allocated) {temp_allocate_frame (s, ots); } -tcg_out_st (s, otype, ts->reg, ots->mem_base->reg, ots->mem_offset); +tcg_out_st (s, otype, ireg, ots->mem_base->reg, ots->mem_offset);if (IS_DEAD_ARG (1 )) {temp_dead (s, ts); }temp_dead (s, ots); +return ; + } + +if (IS_DEAD_ARG (1 ) && ts->kind !=TEMP_FIXED ) { +/* +* The mov can be suppressed. Kill input first, so that it +* is unlinked from reg_to_temp, then set the output to the +* reg that we saved from the input. +*/ +temp_dead (s, ts); + oreg = ireg; }else { -if (IS_DEAD_ARG (1 ) && ts->kind !=TEMP_FIXED ) { -/* the mov can be suppressed */ -if (ots->val_type ==TEMP_VAL_REG ) { - s->reg_to_temp[ots->reg] =NULL ; - } - ots->reg = ts->reg; -temp_dead (s, ts); +if (ots->val_type ==TEMP_VAL_REG ) { + oreg = ots->reg; }else { -if (ots->val_type !=TEMP_VAL_REG ) { -/* When allocating a new register, make sure to not spill the -input one. */ -tcg_regset_set_reg (allocated_regs, ts->reg); - ots->reg =tcg_reg_alloc (s, tcg_target_available_regs[otype], - allocated_regs, preferred_regs, - ots->indirect_base); - } -if (!tcg_out_mov (s, otype, ots->reg, ts->reg)) { -/* -* Cross register class move not supported. -* Store the source register into the destination slot -* and leave the destination temp as TEMP_VAL_MEM. -*/ -assert (!temp_readonly (ots)); -if (!ts->mem_allocated) { -temp_allocate_frame (s, ots); - } -tcg_out_st (s, ts->type, ts->reg, - ots->mem_base->reg, ots->mem_offset); - ots->mem_coherent =1 ; -temp_free_or_dead (s, ots,-1 ); -return ; - } +/* Make sure to not spill the input register during allocation. */ + oreg =tcg_reg_alloc (s, tcg_target_available_regs[otype], + allocated_regs | ((TCGRegSet )1 << ireg), + preferred_regs, ots->indirect_base); } - ots->val_type =TEMP_VAL_REG ; - ots->mem_coherent =0 ; - s->reg_to_temp[ots->reg] = ots; -if (NEED_SYNC_ARG (0 )) { -temp_sync (s, ots, allocated_regs,0 ,0 ); +if (!tcg_out_mov (s, otype, oreg, ireg)) { +/* +* Cross register class move not supported. +* Store the source register into the destination slot +* and leave the destination temp as TEMP_VAL_MEM. +*/ +assert (!temp_readonly (ots)); +if (!ts->mem_allocated) { +temp_allocate_frame (s, ots); + } +tcg_out_st (s, ts->type, ireg, ots->mem_base->reg, ots->mem_offset); +set_temp_val_nonreg (s, ts,TEMP_VAL_MEM ); + ots->mem_coherent =1 ; +return ; } } +set_temp_val_reg (s, ots, oreg); + ots->mem_coherent =0 ; + +if (NEED_SYNC_ARG (0 )) { +temp_sync (s, ots, allocated_regs,0 ,0 ); + } }/*
@@ -3490,15 +3514,15 @@ static void tcg_reg_alloc_dup(TCGContext *s, const TCGOp *op)/* Allocate the output register now. */ if (ots->val_type !=TEMP_VAL_REG ) {TCGRegSet allocated_regs = s->reserved_regs; +TCGReg oreg;if (!IS_DEAD_ARG (1 ) && its->val_type ==TEMP_VAL_REG ) {/* Make sure to not spill the input register. */ tcg_regset_set_reg (allocated_regs, its->reg); } - ots->reg =tcg_reg_alloc (s, dup_out_regs, allocated_regs, - op->output_pref[0 ], ots->indirect_base); - ots->val_type =TEMP_VAL_REG ; - s->reg_to_temp[ots->reg] = ots; + oreg =tcg_reg_alloc (s, dup_out_regs, allocated_regs, + op->output_pref[0 ], ots->indirect_base); +set_temp_val_reg (s, ots, oreg); }switch (its->val_type) {
@@ -3535,10 +3559,12 @@ static void tcg_reg_alloc_dup(TCGContext *s, const TCGOp *op)#else endian_fixup =0 ;#endif +/* Attempt to dup directly from the input memory slot. */ if (tcg_out_dupm_vec (s, vtype, vece, ots->reg, its->mem_base->reg, its->mem_offset + endian_fixup)) { goto done; } +/* Load the input into the destination vector register. */ tcg_out_ld (s, itype, ots->reg, its->mem_base->reg, its->mem_offset);break ;
@@ -3707,17 +3733,8 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op) op->output_pref[k], ts->indirect_base); }tcg_regset_set_reg (o_allocated_regs, reg); -if (ts->val_type ==TEMP_VAL_REG ) { - s->reg_to_temp[ts->reg] =NULL ; - } - ts->val_type =TEMP_VAL_REG ; - ts->reg = reg; -/* -* Temp value is modified, so the value kept in memory is -* potentially not the same. -*/ +set_temp_val_reg (s, ts, reg); ts->mem_coherent =0 ; - s->reg_to_temp[reg] = ts; new_args[i] = reg; } }
@@ -3767,6 +3784,7 @@ static bool tcg_reg_alloc_dup2(TCGContext *s, const TCGOp *op)TCGRegSet allocated_regs = s->reserved_regs;TCGRegSet dup_out_regs = tcg_op_defs[INDEX_op_dup_vec].args_ct[0 ].regs; +TCGReg oreg;/* Make sure to not spill the input registers. */ if (!IS_DEAD_ARG (1 ) && itsl->val_type ==TEMP_VAL_REG ) {
@@ -3776,10 +3794,9 @@ static bool tcg_reg_alloc_dup2(TCGContext *s, const TCGOp *op)tcg_regset_set_reg (allocated_regs, itsh->reg); } - ots->reg =tcg_reg_alloc (s, dup_out_regs, allocated_regs, - op->output_pref[0 ], ots->indirect_base); - ots->val_type =TEMP_VAL_REG ; - s->reg_to_temp[ots->reg] = ots; + oreg =tcg_reg_alloc (s, dup_out_regs, allocated_regs, + op->output_pref[0 ], ots->indirect_base); +set_temp_val_reg (s, ots, oreg); }/* Promote dup2 of immediates to dupi_vec. */
@@ -3962,14 +3979,8 @@ static void tcg_reg_alloc_call(TCGContext *s, TCGOp *op)tcg_debug_assert (!temp_readonly (ts)); reg = tcg_target_call_oarg_regs[i]; -tcg_debug_assert (s->reg_to_temp[reg] ==NULL ); -if (ts->val_type ==TEMP_VAL_REG ) { - s->reg_to_temp[ts->reg] =NULL ; - } - ts->val_type =TEMP_VAL_REG ; - ts->reg = reg; +set_temp_val_reg (s, ts, reg); ts->mem_coherent =0 ; - s->reg_to_temp[reg] = ts;if (NEED_SYNC_ARG (i)) {temp_sync (s, ts, allocated_regs,0 ,IS_DEAD_ARG (i)); }else if (IS_DEAD_ARG (i)) {
Signature · 1 line
2.34.1
Re: [PATCH v4 02/27] tcg: Centralize updates to reg_to_temp
Quoted 5 lines · Richard Henderson
Create two new functions, set_temp_val_{reg,nonreg}. Assert that the
reg_to_tempmapping is correct before any changes are made.Signed-off-by Richard Henderson <[email protected]>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
PATCH v4 3/27 tcg: Remove check_regs
In reply to Richard HendersonWe now check the consistency of reg_to_temp[] with each update, so the utility of checking consistency at the end of each opcode is minimal. In addition, the form of this check is quite expensive, consuming 10% of a checking-enabled build.
--- tcg/tcg.c | 76 ------------------------------------------------------- 1 file changed, 76 deletions(-)
tcg/tcg.c +0 -76
@@ -2900,79 +2900,6 @@ static bool liveness_pass_2(TCGContext *s)return changes; } -#ifdef CONFIG_DEBUG_TCG -static void dump_regs (TCGContext *s) -{ -TCGTemp *ts; -int i; -char buf[64 ]; - -for (i =0 ; i < s->nb_temps; i++) { - ts = &s->temps[i]; -printf (" %10s: " ,tcg_get_arg_str_ptr (s, buf,sizeof (buf), ts)); -switch (ts->val_type) { -case TEMP_VAL_REG : -printf ("%s" , tcg_target_reg_names[ts->reg]); -break ; -case TEMP_VAL_MEM : -printf ("%d(%s)" , (int )ts->mem_offset, - tcg_target_reg_names[ts->mem_base->reg]); -break ; -case TEMP_VAL_CONST : -printf ("$0x%" PRIx64, ts->val); -break ; -case TEMP_VAL_DEAD : -printf ("D" ); -break ; -default : -printf ("???" ); -break ; - } -printf ("\n" ); - } - -for (i =0 ; i <TCG_TARGET_NB_REGS ; i++) { -if (s->reg_to_temp[i] !=NULL ) { -printf ("%s: %s\n" , - tcg_target_reg_names[i], -tcg_get_arg_str_ptr (s, buf,sizeof (buf), s->reg_to_temp[i])); - } - } -} - -static void check_regs (TCGContext *s) -{ -int reg; -int k; -TCGTemp *ts; -char buf[64 ]; - -for (reg =0 ; reg <TCG_TARGET_NB_REGS ; reg++) { - ts = s->reg_to_temp[reg]; -if (ts !=NULL ) { -if (ts->val_type !=TEMP_VAL_REG || ts->reg != reg) { -printf ("Inconsistency for register %s:\n" , - tcg_target_reg_names[reg]); - goto fail; - } - } - } -for (k =0 ; k < s->nb_temps; k++) { - ts = &s->temps[k]; -if (ts->val_type ==TEMP_VAL_REG - && ts->kind !=TEMP_FIXED - && s->reg_to_temp[ts->reg] != ts) { -printf ("Inconsistency for temp %s:\n" , -tcg_get_arg_str_ptr (s, buf,sizeof (buf), ts)); - fail: -printf ("reg state:\n" ); -dump_regs (s); -tcg_abort (); - } - } -} -#endif -static void temp_allocate_frame (TCGContext *s,TCGTemp *ts) {intptr_t off, size, align;
@@ -4297,9 +4224,6 @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start)tcg_reg_alloc_op (s, op);break ; } -#ifdef CONFIG_DEBUG_TCG -check_regs (s); -#endif /* Test for (pending) buffer overflow. The assumption is that any one operation beginning below the high water mark cannot overrun the buffer completely. Thus we can test for overflow after
Signature · 1 line
2.34.1
Re: [PATCH v4 03/27] tcg: Remove check_regs
Quoted 6 lines · Richard Henderson
We now check the consistency of reg_to_temp[] with each update, so the utility of checking consistency at the end of each opcode is minimal. In addition, the form of this check is quite expensive, consuming 10% of a checking-enabled build.
Signed-off-by Richard Henderson <[email protected]>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
PATCH v4 4/27 tcg: Introduce paired register allocation
In reply to Richard HendersonThere are several instances where we need to be able to allocate a pair of registers to related inputs/outputs. Add 'p' and 'm' register constraints for this, in order to be able to allocate the even/odd register first or second.
--- include/tcg/tcg.h | 2 + tcg/tcg.c | 475 ++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 403 insertions(+), 74 deletions(-)
include/tcg/tcg.h +2 -0
@@ -951,6 +951,8 @@ typedef struct TCGArgConstraint { unsigned ct : 16; unsigned alias_index : 4; unsigned sort_index : 4; + unsigned pair_index : 4; + unsigned pair : 2; /* 0: none, 1: first, 2: second, 3: second alias */ bool oalias : 1; bool ialias : 1; bool newreg : 1;
tcg/tcg.c +401 -74
553 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v4 04/27] tcg: Introduce paired register allocation
Splitting this patch in 3 to ease review.
Philippe Mathieu-Daudé (2):
tcg: Massage process_op_defs()
tcg: Massage tcg_reg_alloc_op()
Richard Henderson (1):
tcg: Introduce paired register allocation
include/tcg/tcg.h | 2 + tcg/tcg.c | 475 ++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 403 insertions(+), 74 deletions(-)
Signature · 1 line
2.38.1
PATCH v4 1/1 tcg: Massage process_op_defs()
In preparation of introducing paired registers,
massage a bit process_op_defs()'s switch case.
--- tcg/tcg.c | 61 +++++++++++++++++++++++++++++++------------------------ 1 file changed, 34 insertions(+), 27 deletions(-)
tcg/tcg.c +34 -27
@@ -2012,7 +2012,7 @@ static void process_op_defs(TCGContext *s)for (op =0 ; op <NB_OPS ; op++) {TCGOpDef *def = &tcg_op_defs[op];const TCGTargetOpDef *tdefs; -int i, nb_args; +int i, o, nb_args;if (def->flags &TCG_OPF_NOT_PRESENT ) {continue ;
@@ -2034,53 +2034,60 @@ static void process_op_defs(TCGContext *s)for (i =0 ; i < nb_args; i++) {const char *ct_str = tdefs->args_ct_str[i]; +bool input_p = i >= def->nb_oargs; +/* Incomplete TCGTargetOpDef entry. */ tcg_debug_assert (ct_str !=NULL ); -while (*ct_str !='\0' ) { -switch (*ct_str) { - case '0' ... '9': - { -int oarg = *ct_str -'0' ; -tcg_debug_assert (ct_str == tdefs->args_ct_str[i]); -tcg_debug_assert (oarg < def->nb_oargs); -tcg_debug_assert (def->args_ct[oarg].regs !=0 ); - def->args_ct[i] = def->args_ct[oarg]; -/* The output sets oalias. */ - def->args_ct[oarg].oalias = true; - def->args_ct[oarg].alias_index = i; -/* The input sets ialias. */ - def->args_ct[i].ialias = true; - def->args_ct[i].alias_index = oarg; - } - ct_str++; -break ; -case '&' : - def->args_ct[i].newreg = true; - ct_str++; -break ; +switch (*ct_str) { + case '0' ... '9': + o = *ct_str -'0' ; +tcg_debug_assert (input_p); +tcg_debug_assert (o < def->nb_oargs); +tcg_debug_assert (def->args_ct[o].regs !=0 ); +tcg_debug_assert (!def->args_ct[o].oalias); + def->args_ct[i] = def->args_ct[o]; +/* The output sets oalias. */ + def->args_ct[o].oalias =1 ; + def->args_ct[o].alias_index = i; +/* The input sets ialias. */ + def->args_ct[i].ialias =1 ; + def->args_ct[i].alias_index = o; +tcg_debug_assert (ct_str[1 ] =='\0' ); +continue ; + +case '&' : +tcg_debug_assert (!input_p); + def->args_ct[i].newreg = true; + ct_str++; +break ; + } + +do { +switch (*ct_str) {case 'i' : def->args_ct[i].ct |=TCG_CT_CONST ; - ct_str++;break ;/* Include all of the target-specific constraints. */ #undef CONST#define CONST (CASE ,MASK ) \ - case CASE: def->args_ct[i].ct |= MASK; ct_str++; break; + case CASE: def->args_ct[i].ct |= MASK; break;#define REGS (CASE ,MASK ) \ - case CASE: def->args_ct[i].regs |= MASK; ct_str++; break; + case CASE: def->args_ct[i].regs |= MASK; break;#include "tcg-target-con-str.h" #undef REGS#undef CONSTdefault : + case '0' ... '9': +case '&' :/* Typo in TCGTargetOpDef constraint. */ g_assert_not_reached (); } - } + }while (*++ct_str !='\0' ); }/* TCGTargetOpDef entry with too much information? */
Signature · 1 line
2.38.1
PATCH v4 1/1 tcg: Massage tcg_reg_alloc_op()
In reply to Philippe Mathieu-DaudéIn preparation of introducing paired registers in the next commit, massage tcg_reg_alloc_op() a bit by converting it to a switch with a single case: the current non-paired register.
--- include/tcg/tcg.h | 1 + tcg/tcg.c | 98 ++++++++++++++++++++++++++++------------------- 2 files changed, 59 insertions(+), 40 deletions(-)
include/tcg/tcg.h +1 -0
@@ -951,6 +951,7 @@ typedef struct TCGArgConstraint { unsigned ct : 16; unsigned alias_index : 4; unsigned sort_index : 4; + unsigned pair : 1; /* 0: not paired, 1: illegal */ bool oalias : 1; bool ialias : 1; bool newreg : 1;
tcg/tcg.c +58 -40
124 lines not on this page · raw message
Signature · 1 line
2.38.1
PATCH v4 1/1 tcg: Introduce paired register allocation
In reply to Philippe Mathieu-DaudéFrom: Richard Henderson <[email protected]>
There are several instances where we need to be able to allocate a pair of registers to related inputs/outputs. Add 'p' and 'm' register constraints for this, in order to be able to allocate the even/odd register first or second.
--- include/tcg/tcg.h | 3 +- tcg/tcg.c | 318 ++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 312 insertions(+), 9 deletions(-)
include/tcg/tcg.h +2 -1
@@ -951,7 +951,8 @@ typedef struct TCGArgConstraint { unsigned ct : 16; unsigned alias_index : 4; unsigned sort_index : 4; - unsigned pair : 1; /* 0: not paired, 1: illegal */ + unsigned pair_index : 4; + unsigned pair : 2; /* 0: none, 1: first, 2: second, 3: second alias */ bool oalias : 1; bool ialias : 1; bool newreg : 1;
tcg/tcg.c +310 -8
380 lines not on this page · raw message
Signature · 1 line
2.38.1
PATCH v4 5/27 accel/tcg: Set cflags_next_tb in cpu_common_initfn
In reply to Richard HendersonWhile we initialize this value in cpu_common_reset, that isn't called during startup, so set it as well in init. This fixes -singlestep versus the very first TB.
--- hw/core/cpu-common.c | 1 + 1 file changed, 1 insertion(+)
hw/core/cpu-common.c +1 -0
@@ -235,6 +235,7 @@ static void cpu_common_initfn(Object *obj)/* the default value is changed by qemu_init_vcpu() for softmmu */ cpu->nr_cores =1 ; cpu->nr_threads =1 ; + cpu->cflags_next_tb =-1 ;qemu_mutex_init (&cpu->work_mutex);QSIMPLEQ_INIT (&cpu->work_list);
Signature · 1 line
2.34.1
Re: [PATCH v4 05/27] accel/tcg: Set cflags_next_tb in cpu_common_initfn
Quoted 6 lines · Richard Henderson
While we initialize this value in
cpu_common_reset, that isn't called during startup, so set it as well in init. This fixes -singlestep versus the very first TB.Fixes 04f5b647ed07 ("accel/tcg: Handle -singlestep in curr_cflags")Signed-off-by Richard Henderson <[email protected]>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
PATCH v4 6/27 target/sparc: Avoid TCGV_{LOW,HIGH}
In reply to Richard HendersonUse the official extend/extract functions instead of routines that will shortly be internal to tcg.
--- target/sparc/translate.c | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-)
target/sparc/translate.c +4 -17
@@ -163,13 +163,6 @@ static inline void gen_update_fprs_dirty(DisasContext *dc, int rd)/* floating point registers moves */ static TCGv_i32 gen_load_fpr_F (DisasContext *dc,unsigned int src) { -#if TCG_TARGET_REG_BITS ==32 -if (src &1 ) { -return TCGV_LOW (cpu_fpr[src /2 ]); - }else { -return TCGV_HIGH (cpu_fpr[src /2 ]); - } -#else TCGv_i32 ret =get_temp_i32 (dc);if (src &1 ) {tcg_gen_extrl_i64_i32 (ret, cpu_fpr[src /2 ]);
@@ -177,22 +170,16 @@ static TCGv_i32 gen_load_fpr_F(DisasContext *dc, unsigned int src)tcg_gen_extrh_i64_i32 (ret, cpu_fpr[src /2 ]); }return ret; -#endif }static void gen_store_fpr_F (DisasContext *dc,unsigned int dst,TCGv_i32 v) { -#if TCG_TARGET_REG_BITS ==32 -if (dst &1 ) { -tcg_gen_mov_i32 (TCGV_LOW (cpu_fpr[dst /2 ]), v); - }else { -tcg_gen_mov_i32 (TCGV_HIGH (cpu_fpr[dst /2 ]), v); - } -#else -TCGv_i64 t = (TCGv_i64 )v; +TCGv_i64 t =tcg_temp_new_i64 (); + +tcg_gen_extu_i32_i64 (t, v);tcg_gen_deposit_i64 (cpu_fpr[dst /2 ], cpu_fpr[dst /2 ], t, (dst &1 ?0 :32 ),32 ); -#endif +tcg_temp_free_i64 (t);gen_update_fprs_dirty (dc, dst); }
Signature · 1 line
2.34.1
Re: [PATCH v4 06/27] target/sparc: Avoid TCGV_{LOW,HIGH}
Quoted 8 lines · Richard Henderson
Use the official extend/extract functions instead of routines that will shortly be internal to tcg.
Cc Mark Cave-Ayland <[email protected]>Signed-off-by Richard Henderson <[email protected]>--- target/sparc/translate.c | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-)
PATCH v4 7/27 tcg: Move TCG_{LOW,HIGH} to tcg-internal.h
In reply to Richard HendersonMove the error-generating fallback from tcg-op.c, and replace "_link_error" with modern QEMU_ERROR markup.
--- include/tcg/tcg-op.h | 33 +++++---------------------------- include/tcg/tcg.h | 12 ------------ tcg/tcg-internal.h | 14 ++++++++++++++ tcg/tcg-op-vec.c | 2 ++ tcg/tcg-op.c | 37 ++++++++++++++++++++++++++++--------- 5 files changed, 49 insertions(+), 49 deletions(-)
include/tcg/tcg-op.h +5 -28
@@ -667,35 +667,12 @@ static inline void tcg_gen_mul_i64(TCGv_i64 ret, TCGv_i64 arg1, TCGv_i64 arg2) tcg_gen_op3_i64(INDEX_op_mul_i64, ret, arg1, arg2); }#else /* TCG_TARGET_REG_BITS == 32 */ -static inline void tcg_gen_st8_i64 (TCGv_i64 arg1,TCGv_ptr arg2, -tcg_target_long offset) -{ -tcg_gen_st8_i32 (TCGV_LOW (arg1), arg2, offset); -} +void tcg_gen_st8_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset); +void tcg_gen_st16_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset); +void tcg_gen_st32_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset); -static inline void tcg_gen_st16_i64 (TCGv_i64 arg1,TCGv_ptr arg2, -tcg_target_long offset) -{ -tcg_gen_st16_i32 (TCGV_LOW (arg1), arg2, offset); -} - -static inline void tcg_gen_st32_i64 (TCGv_i64 arg1,TCGv_ptr arg2, -tcg_target_long offset) -{ -tcg_gen_st_i32 (TCGV_LOW (arg1), arg2, offset); -} - -static inline void tcg_gen_add_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2) -{ -tcg_gen_add2_i32 (TCGV_LOW (ret),TCGV_HIGH (ret),TCGV_LOW (arg1), -TCGV_HIGH (arg1),TCGV_LOW (arg2),TCGV_HIGH (arg2)); -} - -static inline void tcg_gen_sub_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2) -{ -tcg_gen_sub2_i32 (TCGV_LOW (ret),TCGV_HIGH (ret),TCGV_LOW (arg1), -TCGV_HIGH (arg1),TCGV_LOW (arg2),TCGV_HIGH (arg2)); -} +void tcg_gen_add_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2); +void tcg_gen_sub_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2);void tcg_gen_discard_i64 (TCGv_i64 arg);void tcg_gen_mov_i64 (TCGv_i64 ret,TCGv_i64 arg);
include/tcg/tcg.h +0 -12
@@ -737,18 +737,6 @@ static inline TCGv_vec temp_tcgv_vec(TCGTemp *t)return (TCGv_vec )temp_tcgv_i32 (t); } -#if TCG_TARGET_REG_BITS ==32 -static inline TCGv_i32 TCGV_LOW (TCGv_i64 t) -{ -return temp_tcgv_i32 (tcgv_i64_temp (t)); -} - -static inline TCGv_i32 TCGV_HIGH (TCGv_i64 t) -{ -return temp_tcgv_i32 (tcgv_i64_temp (t) +1 ); -} -#endif -static inline TCGArg tcg_get_insn_param (TCGOp *op,int arg) {return op->args[arg];
tcg/tcg-internal.h +14 -0
@@ -59,4 +59,18 @@ static inline unsigned tcg_call_flags(TCGOp *op)return tcg_call_info (op)->flags; } +#if TCG_TARGET_REG_BITS ==32 +static inline TCGv_i32 TCGV_LOW (TCGv_i64 t) +{ +return temp_tcgv_i32 (tcgv_i64_temp (t)); +} +static inline TCGv_i32 TCGV_HIGH (TCGv_i64 t) +{ +return temp_tcgv_i32 (tcgv_i64_temp (t) +1 ); +} +#else +extern TCGv_i32 TCGV_LOW(TCGv_i64) QEMU_ERROR("32-bit code path is reachable"); +extern TCGv_i32 TCGV_HIGH(TCGv_i64) QEMU_ERROR("32-bit code path is reachable"); +#endif +#endif /* TCG_INTERNAL_H */
tcg/tcg-op-vec.c +2 -0
@@ -21,6 +21,8 @@#include "tcg/tcg.h" #include "tcg/tcg-op.h" #include "tcg/tcg-mo.h" +#include "tcg-internal.h" +/* Reduce the number of ifdefs below. This assumes that all uses of TCGV_HIGH and TCGV_LOW are properly protected by a conditional that
tcg/tcg-op.c +28 -9
@@ -28,16 +28,8 @@#include "tcg/tcg-op.h" #include "tcg/tcg-mo.h" #include "exec/plugin-gen.h" +#include "tcg-internal.h" -/* Reduce the number of ifdefs below. This assumes that all uses of -TCGV_HIGH and TCGV_LOW are properly protected by a conditional that -the compiler can eliminate. */ -#if TCG_TARGET_REG_BITS ==64 -extern TCGv_i32 TCGV_LOW_link_error (TCGv_i64 ); -extern TCGv_i32 TCGV_HIGH_link_error (TCGv_i64 ); -#define TCGV_LOW TCGV_LOW_link_error -#define TCGV_HIGH TCGV_HIGH_link_error -#endif void tcg_gen_op1 (TCGOpcode opc,TCGArg a1) {
@@ -1171,6 +1163,21 @@ void tcg_gen_ld_i64(TCGv_i64 ret, TCGv_ptr arg2, tcg_target_long offset)#endif } +void tcg_gen_st8_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset) +{ +tcg_gen_st8_i32 (TCGV_LOW (arg1), arg2, offset); +} + +void tcg_gen_st16_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset) +{ +tcg_gen_st16_i32 (TCGV_LOW (arg1), arg2, offset); +} + +void tcg_gen_st32_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset) +{ +tcg_gen_st_i32 (TCGV_LOW (arg1), arg2, offset); +} +void tcg_gen_st_i64 (TCGv_i64 arg1,TCGv_ptr arg2,tcg_target_long offset) {#if HOST_BIG_ENDIAN
@@ -1182,6 +1189,18 @@ void tcg_gen_st_i64(TCGv_i64 arg1, TCGv_ptr arg2, tcg_target_long offset)#endif } +void tcg_gen_add_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2) +{ +tcg_gen_add2_i32 (TCGV_LOW (ret),TCGV_HIGH (ret),TCGV_LOW (arg1), +TCGV_HIGH (arg1),TCGV_LOW (arg2),TCGV_HIGH (arg2)); +} + +void tcg_gen_sub_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2) +{ +tcg_gen_sub2_i32 (TCGV_LOW (ret),TCGV_HIGH (ret),TCGV_LOW (arg1), +TCGV_HIGH (arg1),TCGV_LOW (arg2),TCGV_HIGH (arg2)); +} +void tcg_gen_and_i64 (TCGv_i64 ret,TCGv_i64 arg1,TCGv_i64 arg2) {tcg_gen_and_i32 (TCGV_LOW (ret),TCGV_LOW (arg1),TCGV_LOW (arg2));
Signature · 1 line
2.34.1
PATCH v4 8/27 tcg: Add temp_subindex to TCGTemp
In reply to Richard HendersonRecord the location of a TCGTemp within a larger object.
--- include/tcg/tcg.h | 1 + tcg/tcg.c | 3 +++ 2 files changed, 4 insertions(+)
include/tcg/tcg.h +1 -0
@@ -456,6 +456,7 @@ typedef struct TCGTemp { unsigned int mem_coherent:1; unsigned int mem_allocated:1; unsigned int temp_allocated:1; + unsigned int temp_subindex:1;int64_t val;struct TCGTemp *mem_base;
tcg/tcg.c +3 -0
@@ -928,6 +928,7 @@ TCGTemp *tcg_global_mem_new_internal(TCGType type, TCGv_ptr base, ts2->mem_allocated =1 ; ts2->mem_base = base_ts; ts2->mem_offset = offset + (1 - bigendian) *4 ; + ts2->temp_subindex =1 ;pstrcpy (buf,sizeof (buf), name);pstrcat (buf,sizeof (buf),"_1" ); ts2->name =strdup (buf);
@@ -974,6 +975,7 @@ TCGTemp *tcg_temp_new_internal(TCGType type, bool temp_local) ts2->base_type =TCG_TYPE_I64 ; ts2->type =TCG_TYPE_I32 ; ts2->temp_allocated =1 ; + ts2->temp_subindex =1 ; ts2->kind = kind; }else { ts->base_type = type;
@@ -1092,6 +1094,7 @@ TCGTemp *tcg_constant_internal(TCGType type, int64_t val) ts2->type =TCG_TYPE_I32 ; ts2->kind =TEMP_CONST ; ts2->temp_allocated =1 ; + ts2->temp_subindex =1 ; ts2->val = val >>32 ; }else { ts->base_type = type;
Signature · 1 line
2.34.1
PATCH v4 9/27 tcg: Simplify calls to temp_sync vs mem_coherent
In reply to Richard HendersonThe first thing that temp_sync does is check mem_coherent, so there's no need for the caller to do so.
--- tcg/tcg.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
tcg/tcg.c +2 -6
@@ -4075,12 +4075,8 @@ static bool tcg_reg_alloc_dup2(TCGContext *s, const TCGOp *op)/* If the two inputs form one 64-bit value, try dupm_vec. */ if (itsl +1 == itsh && itsl->base_type ==TCG_TYPE_I64 ) { -if (!itsl->mem_coherent) { -temp_sync (s, itsl, s->reserved_regs,0 ,0 ); - } -if (!itsh->mem_coherent) { -temp_sync (s, itsh, s->reserved_regs,0 ,0 ); - } +temp_sync (s, itsl, s->reserved_regs,0 ,0 ); +temp_sync (s, itsh, s->reserved_regs,0 ,0 );#if HOST_BIG_ENDIAN TCGTemp *its = itsh;#else
Signature · 1 line
2.34.1
Re: [PATCH v4 09/27] tcg: Simplify calls to temp_sync vs mem_coherent
Quoted 7 lines · Richard Henderson
The first thing that
temp_syncdoes is checkmem_coherent, so there's no need for the caller to do so.Signed-off-by Richard Henderson <[email protected]>--- tcg/tcg.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
PATCH v4 10/27 tcg: Allocate TCGTemp pairs in host memory order
In reply to Richard HendersonAllocate the first of a pair at the lower address, and the second of a pair at the higher address. This will make it easier to find the beginning of the larger memory block.
--- tcg/tcg-internal.h | 4 ++-- tcg/tcg.c | 58 ++++++++++++++++++++++------------------------ 2 files changed, 30 insertions(+), 32 deletions(-)
tcg/tcg-internal.h +2 -2
@@ -62,11 +62,11 @@ static inline unsigned tcg_call_flags(TCGOp *op)#if TCG_TARGET_REG_BITS ==32 static inline TCGv_i32 TCGV_LOW (TCGv_i64 t) { -return temp_tcgv_i32 (tcgv_i64_temp (t)); +return temp_tcgv_i32 (tcgv_i64_temp (t) +HOST_BIG_ENDIAN ); }static inline TCGv_i32 TCGV_HIGH (TCGv_i64 t) { -return temp_tcgv_i32 (tcgv_i64_temp (t) +1 ); +return temp_tcgv_i32 (tcgv_i64_temp (t) + !HOST_BIG_ENDIAN ); }#else extern TCGv_i32 TCGV_LOW(TCGv_i64) QEMU_ERROR("32-bit code path is reachable");
tcg/tcg.c +28 -30
124 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v4 11/27 tcg: Move TCG_TYPE_COUNT outside enum
In reply to Richard HendersonThe count is not itself an enumerator. Move it outside to prevent the compiler from considering it with -Wswitch-enum.
--- include/tcg/tcg.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
include/tcg/tcg.h +2 -1
@@ -294,7 +294,8 @@ typedef enum TCGType { TCG_TYPE_V128, TCG_TYPE_V256, -TCG_TYPE_COUNT ,/* number of different types */ +/* Number of different types (integer not enum) */ +#define TCG_TYPE_COUNT (TCG_TYPE_V256 + 1)/* An alias for the size of the host register. */ #if TCG_TARGET_REG_BITS ==32
Signature · 1 line
2.34.1
PATCH v4 12/27 tcg: Introduce tcg_type_size
In reply to Richard HendersonAdd a helper function for computing the size of a type.
--- include/tcg/tcg.h | 16 ++++++++++++++++ tcg/tcg.c | 27 ++++++++++++--------------- 2 files changed, 28 insertions(+), 15 deletions(-)
include/tcg/tcg.h +16 -0
@@ -319,6 +319,22 @@ typedef enum TCGType {#endif } TCGType; +/** +* tcg_type_size +* @t: type +* +* Return the size of the type in bytes. +*/ +static inline int tcg_type_size (TCGType t) +{ +unsigned i = t; +if (i >=TCG_TYPE_V64 ) { +tcg_debug_assert (i <TCG_TYPE_COUNT ); + i -=TCG_TYPE_V64 -1 ; + } +return 4 << i; +} +/** * get_alignment_bits * @memop: MemOp value
tcg/tcg.c +12 -15
@@ -3038,22 +3038,22 @@ static bool liveness_pass_2(TCGContext *s)static void temp_allocate_frame (TCGContext *s,TCGTemp *ts) { -intptr_t off, size, align; +int size =tcg_type_size (ts->type); +int align; +intptr_t off;switch (ts->type) {case TCG_TYPE_I32 : - size = align =4 ; + align =4 ;break ;case TCG_TYPE_I64 :case TCG_TYPE_V64 : - size = align =8 ; + align =8 ;break ;case TCG_TYPE_V128 : - size = align =16 ; -break ;case TCG_TYPE_V256 :/* Note that we do not require aligned storage for V256. */ - size =32 , align =16 ; + align =16 ;break ;default :g_assert_not_reached ();
@@ -3593,8 +3593,8 @@ static void tcg_reg_alloc_dup(TCGContext *s, const TCGOp *op)TCGRegSet dup_out_regs, dup_in_regs;TCGTemp *its, *ots;TCGType itype, vtype; -intptr_t endian_fixup;unsigned vece; +int lowpart_ofs;bool ok; ots =arg_temp (op->args[0 ]);
@@ -3662,15 +3662,12 @@ static void tcg_reg_alloc_dup(TCGContext *s, const TCGOp *op)/* fall through */ case TEMP_VAL_MEM : -#if HOST_BIG_ENDIAN - endian_fixup = itype ==TCG_TYPE_I32 ?4 :8 ; - endian_fixup -=1 << vece; -#else - endian_fixup =0 ; -#endif -/* Attempt to dup directly from the input memory slot. */ + lowpart_ofs =0 ; +if (HOST_BIG_ENDIAN ) { + lowpart_ofs =tcg_type_size (itype) - (1 << vece); + }if (tcg_out_dupm_vec (s, vtype, vece, ots->reg, its->mem_base->reg, - its->mem_offset + endian_fixup)) { + its->mem_offset + lowpart_ofs)) { goto done; }/* Load the input into the destination vector register. */
Signature · 1 line
2.34.1
PATCH v4 13/27 tcg: Introduce TCGCallReturnKind and TCGCallArgumentKind
In reply to Richard HendersonPrepare to replace a bunch of separate ifdefs with a consistent way to describe the ABI of a function call.
--- tcg/tcg-internal.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
tcg/tcg-internal.h +15 -0
@@ -27,6 +27,21 @@#define TCG_HIGHWATER 1024 +/* +* Describe the calling convention of a given argument type. +*/ +typedef enum { +TCG_CALL_RET_NORMAL ,/* by registers */ +}TCGCallReturnKind ; + +typedef enum { +TCG_CALL_ARG_NORMAL ,/* by registers (continuing onto stack) */ +TCG_CALL_ARG_EVEN ,/* like normal, but skipping odd slots */ +TCG_CALL_ARG_EXTEND ,/* for i32, as a sign/zero-extended i64 */ +TCG_CALL_ARG_EXTEND_U ,/* ... as a zero-extended i64 */ +TCG_CALL_ARG_EXTEND_S ,/* ... as a sign-extended i64 */ +}TCGCallArgumentKind ; + typedef struct TCGHelperInfo { void *func; const char *name;
Signature · 1 line
2.34.1
Re: [PATCH v4 13/27] tcg: Introduce TCGCallReturnKind and TCGCallArgumentKind
Quoted 5 lines · Richard Henderson
Prepare to replace a bunch of separate ifdefs with a consistent way to describe the ABI of a function call.
Reviewed-by Philippe Mathieu-Daudé <[email protected]>Signed-off-by Richard Henderson <[email protected]>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
PATCH v4 14/27 tcg: Replace TCG_TARGET_CALL_ALIGN_ARGS with TCG_TARGET_CALL_ARG_I64
In reply to Richard HendersonFor 32-bit hosts when TCG_TARGET_CALL_ALIGN_ARGS was set, use TCG_CALL_ARG_EVEN. For 64-bit hosts, TCG_TARGET_CALL_ALIGN_ARGS was silently ignored, so always use TCG_CALL_ARG_NORMAL.
--- tcg/aarch64/tcg-target.h | 2 +- tcg/arm/tcg-target.h | 2 +- tcg/i386/tcg-target.h | 1 + tcg/loongarch64/tcg-target.h | 2 +- tcg/mips/tcg-target.h | 3 ++- tcg/riscv/tcg-target.h | 6 +++++- tcg/s390x/tcg-target.h | 1 + tcg/sparc64/tcg-target.h | 1 + tcg/tci/tcg-target.h | 5 +++++ tcg/tcg.c | 6 ++++-- tcg/ppc/tcg-target.c.inc | 21 ++++++++------------- 11 files changed, 30 insertions(+), 20 deletions(-)
tcg/aarch64/tcg-target.h +1 -1
@@ -51,8 +51,8 @@ typedef enum {/* used for function call generation */ #define TCG_REG_CALL_STACK TCG_REG_SP#define TCG_TARGET_STACK_ALIGN 16 -#define TCG_TARGET_CALL_ALIGN_ARGS 1#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL/* optional instructions */ #define TCG_TARGET_HAS_div_i32 1
tcg/arm/tcg-target.h +1 -1
@@ -88,8 +88,8 @@ extern bool use_neon_instructions;/* used for function call generation */ #define TCG_TARGET_STACK_ALIGN 8 -#define TCG_TARGET_CALL_ALIGN_ARGS 1#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN/* optional instructions */ #define TCG_TARGET_HAS_ext8s_i32 1
tcg/i386/tcg-target.h +1 -0
@@ -98,6 +98,7 @@ typedef enum {#else #define TCG_TARGET_CALL_STACK_OFFSET 0#endif +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMALextern bool have_bmi1;extern bool have_popcnt;
tcg/loongarch64/tcg-target.h +1 -1
@@ -92,8 +92,8 @@ typedef enum {/* used for function call generation */ #define TCG_REG_CALL_STACK TCG_REG_SP#define TCG_TARGET_STACK_ALIGN 16 -#define TCG_TARGET_CALL_ALIGN_ARGS 1#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL/* optional instructions */ #define TCG_TARGET_HAS_movcond_i32 0
tcg/mips/tcg-target.h +2 -1
@@ -83,10 +83,11 @@ typedef enum {#define TCG_TARGET_STACK_ALIGN 16#if _MIPS_SIM == _ABIO32# define TCG_TARGET_CALL_STACK_OFFSET 16 +# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN#else # define TCG_TARGET_CALL_STACK_OFFSET 0 +# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL#endif -#define TCG_TARGET_CALL_ALIGN_ARGS 1/* MOVN/MOVZ instructions detection */ #if (defined(__mips_isa_rev) && (__mips_isa_rev >= 1)) || \
tcg/riscv/tcg-target.h +5 -1
@@ -81,8 +81,12 @@ typedef enum {/* used for function call generation */ #define TCG_REG_CALL_STACK TCG_REG_SP#define TCG_TARGET_STACK_ALIGN 16 -#define TCG_TARGET_CALL_ALIGN_ARGS 1#define TCG_TARGET_CALL_STACK_OFFSET 0 +#if TCG_TARGET_REG_BITS ==32 +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN +#else +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +#endif /* optional instructions */ #define TCG_TARGET_HAS_movcond_i32 0
tcg/s390x/tcg-target.h +1 -0
@@ -166,6 +166,7 @@ extern uint64_t s390_facilities[3];/* used for function call generation */ #define TCG_TARGET_STACK_ALIGN 8#define TCG_TARGET_CALL_STACK_OFFSET 160 +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL#define TCG_TARGET_EXTEND_ARGS 1#define TCG_TARGET_HAS_MEMORY_BSWAP 1
tcg/sparc64/tcg-target.h +1 -0
@@ -72,6 +72,7 @@ typedef enum {#define TCG_TARGET_STACK_ALIGN 16#define TCG_TARGET_CALL_STACK_OFFSET (128 + 6*8 + TCG_TARGET_STACK_BIAS)#define TCG_TARGET_EXTEND_ARGS 1 +#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL#if defined(__VIS__) && __VIS__ >=0x300 #define use_vis3_instructions 1
tcg/tci/tcg-target.h +5 -0
@@ -158,6 +158,11 @@ typedef enum {/* Used for function call generation. */ #define TCG_TARGET_CALL_STACK_OFFSET 0#define TCG_TARGET_STACK_ALIGN 8 +#if TCG_TARGET_REG_BITS ==32 +# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN +#else +# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL +#endif #define HAVE_TCG_QEMU_TB_EXEC #define TCG_TARGET_NEED_POOL_LABELS
tcg/tcg.c +4 -2
@@ -1542,9 +1542,11 @@ void tcg_gen_callN(void *func, TCGTemp *ret, int nargs, TCGTemp **args)* for passing off to ffi_call. */ want_align = true; -#elif defined(TCG_TARGET_CALL_ALIGN_ARGS ) +#else /* Some targets want aligned 64 bit args */ - want_align = is_64bit; +if (is_64bit) { + want_align =TCG_TARGET_CALL_ARG_I64 ==TCG_CALL_ARG_EVEN ; + }#endif if (TCG_TARGET_REG_BITS <64 && want_align && (real_args &1 )) {
tcg/ppc/tcg-target.c.inc +8 -13
@@ -45,7 +45,9 @@ #endif #ifdef _CALL_SYSV -# define TCG_TARGET_CALL_ALIGN_ARGS 1 +# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN +#else +# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL #endif /* For some memory operations, we need a scratch that isn't R0. For the AIX
@@ -2202,9 +2204,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) lo = lb->addrlo_reg; hi = lb->addrhi_reg; if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { -#ifdef TCG_TARGET_CALL_ALIGN_ARGS - arg |= 1; -#endif + arg |= (TCG_TARGET_CALL_ARG_I64 == TCG_CALL_ARG_EVEN); tcg_out_mov(s, TCG_TYPE_I32, arg++, hi); tcg_out_mov(s, TCG_TYPE_I32, arg++, lo); } else {
@@ -2250,9 +2250,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) lo = lb->addrlo_reg; hi = lb->addrhi_reg; if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { -#ifdef TCG_TARGET_CALL_ALIGN_ARGS - arg |= 1; -#endif + arg |= (TCG_TARGET_CALL_ARG_I64 == TCG_CALL_ARG_EVEN); tcg_out_mov(s, TCG_TYPE_I32, arg++, hi); tcg_out_mov(s, TCG_TYPE_I32, arg++, lo); } else {
@@ -2266,9 +2264,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) if (TCG_TARGET_REG_BITS == 32) { switch (s_bits) { case MO_64: -#ifdef TCG_TARGET_CALL_ALIGN_ARGS - arg |= 1; -#endif + arg |= (TCG_TARGET_CALL_ARG_I64 == TCG_CALL_ARG_EVEN); tcg_out_mov(s, TCG_TYPE_I32, arg++, hi); /* FALLTHRU */ case MO_32:
@@ -2324,9 +2320,8 @@ static bool tcg_out_fail_alignment(TCGContext *s, TCGLabelQemuLdst *l) if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { TCGReg arg = TCG_REG_R4; -#ifdef TCG_TARGET_CALL_ALIGN_ARGS - arg |= 1; -#endif + + arg |= (TCG_TARGET_CALL_ARG_I64 == TCG_CALL_ARG_EVEN); if (l->addrlo_reg != arg) { tcg_out_mov(s, TCG_TYPE_I32, arg, l->addrhi_reg); tcg_out_mov(s, TCG_TYPE_I32, arg + 1, l->addrlo_reg);
Signature · 1 line
2.34.1
PATCH v4 15/27 tcg: Replace TCG_TARGET_EXTEND_ARGS with TCG_TARGET_CALL_ARG_I32
In reply to Richard HendersonFor 64-bit hosts that had TCG_TARGET_EXTEND_ARGS, set TCG_TARGET_CALL_ARG_I32 to TCG_CALL_ARG_EXTEND. Otherwise, use TCG_CALL_ARG_NORMAL.
--- tcg/aarch64/tcg-target.h | 1 + tcg/arm/tcg-target.h | 1 + tcg/i386/tcg-target.h | 1 + tcg/loongarch64/tcg-target.h | 1 + tcg/mips/tcg-target.h | 1 + tcg/riscv/tcg-target.h | 1 + tcg/s390x/tcg-target.h | 2 +- tcg/sparc64/tcg-target.h | 2 +- tcg/tci/tcg-target.h | 1 + tcg/tcg.c | 42 ++++++++++++++++++------------------ tcg/ppc/tcg-target.c.inc | 6 +++++- 11 files changed, 35 insertions(+), 24 deletions(-)
tcg/aarch64/tcg-target.h +1 -0
@@ -52,6 +52,7 @@ typedef enum {#define TCG_REG_CALL_STACK TCG_REG_SP#define TCG_TARGET_STACK_ALIGN 16#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL/* optional instructions */
tcg/arm/tcg-target.h +1 -0
@@ -89,6 +89,7 @@ extern bool use_neon_instructions;/* used for function call generation */ #define TCG_TARGET_STACK_ALIGN 8#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN/* optional instructions */
tcg/i386/tcg-target.h +1 -0
@@ -98,6 +98,7 @@ typedef enum {#else #define TCG_TARGET_CALL_STACK_OFFSET 0#endif +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMALextern bool have_bmi1;
tcg/loongarch64/tcg-target.h +1 -0
@@ -93,6 +93,7 @@ typedef enum {#define TCG_REG_CALL_STACK TCG_REG_SP#define TCG_TARGET_STACK_ALIGN 16#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL/* optional instructions */
tcg/mips/tcg-target.h +1 -0
@@ -88,6 +88,7 @@ typedef enum {# define TCG_TARGET_CALL_STACK_OFFSET 0# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL#endif +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL/* MOVN/MOVZ instructions detection */ #if (defined(__mips_isa_rev) && (__mips_isa_rev >= 1)) || \
tcg/riscv/tcg-target.h +1 -0
@@ -82,6 +82,7 @@ typedef enum {#define TCG_REG_CALL_STACK TCG_REG_SP#define TCG_TARGET_STACK_ALIGN 16#define TCG_TARGET_CALL_STACK_OFFSET 0 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#if TCG_TARGET_REG_BITS ==32 #define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN#else
tcg/s390x/tcg-target.h +1 -1
@@ -166,9 +166,9 @@ extern uint64_t s390_facilities[3];/* used for function call generation */ #define TCG_TARGET_STACK_ALIGN 8#define TCG_TARGET_CALL_STACK_OFFSET 160 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_EXTEND#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL -#define TCG_TARGET_EXTEND_ARGS 1#define TCG_TARGET_HAS_MEMORY_BSWAP 1#define TCG_TARGET_DEFAULT_MO (TCG_MO_ALL & ~TCG_MO_ST_LD)
tcg/sparc64/tcg-target.h +1 -1
@@ -71,7 +71,7 @@ typedef enum {#define TCG_TARGET_STACK_BIAS 2047#define TCG_TARGET_STACK_ALIGN 16#define TCG_TARGET_CALL_STACK_OFFSET (128 + 6*8 + TCG_TARGET_STACK_BIAS) -#define TCG_TARGET_EXTEND_ARGS 1 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_EXTEND#define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL#if defined(__VIS__) && __VIS__ >=0x300
tcg/tci/tcg-target.h +1 -0
@@ -158,6 +158,7 @@ typedef enum {/* Used for function call generation. */ #define TCG_TARGET_CALL_STACK_OFFSET 0#define TCG_TARGET_STACK_ALIGN 8 +#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#if TCG_TARGET_REG_BITS ==32 # define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN#else
tcg/tcg.c +21 -21
61 lines not on this page · raw message
tcg/ppc/tcg-target.c.inc +5 -1
@@ -44,6 +44,11 @@ # endif #endif +#if TCG_TARGET_REG_BITS == 64 +# define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_EXTEND +#else +# define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL +#endif #ifdef _CALL_SYSV # define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN #else
@@ -2520,7 +2525,6 @@ static void tcg_out_nop_fill(tcg_insn_unit *p, int count) /* Parameters for function call generation, used in tcg.c. */ #define TCG_TARGET_STACK_ALIGN 16 -#define TCG_TARGET_EXTEND_ARGS 1 #ifdef _CALL_AIX # define LINK_AREA_SIZE (6 * SZR)
Signature · 1 line
2.34.1
PATCH v4 16/27 tcg: Use TCG_CALL_ARG_EVEN for TCI special case
In reply to Richard HendersonChange 32-bit tci TCG_TARGET_CALL_ARG_I32 to TCG_CALL_ARG_EVEN, to force 32-bit values to be aligned to 64-bit. With a small reorg to the argument processing loop, this neatly replaces an ifdef for CONFIG_TCG_INTERPRETER.
--- tcg/tci/tcg-target.h | 3 +- tcg/tcg.c | 70 ++++++++++++++++++++++++++++---------------- 2 files changed, 47 insertions(+), 26 deletions(-)
tcg/tci/tcg-target.h +2 -1
@@ -158,10 +158,11 @@ typedef enum {/* Used for function call generation. */ #define TCG_TARGET_CALL_STACK_OFFSET 0#define TCG_TARGET_STACK_ALIGN 8 -#define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL#if TCG_TARGET_REG_BITS ==32 +# define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_EVEN# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_EVEN#else +# define TCG_TARGET_CALL_ARG_I32 TCG_CALL_ARG_NORMAL# define TCG_TARGET_CALL_ARG_I64 TCG_CALL_ARG_NORMAL#endif
tcg/tcg.c +45 -25
81 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v4 17/27 accel/tcg/plugin: Don't search for the function pointer index
In reply to Richard HendersonThe function pointer is immediately after the output and input operands; no need to search.
---
Cc: Alex Bennée <[email protected]>
--- accel/tcg/plugin-gen.c | 29 +++++++++++------------------ 1 file changed, 11 insertions(+), 18 deletions(-)
accel/tcg/plugin-gen.c +11 -18
@@ -381,32 +381,25 @@ static TCGOp *copy_st_ptr(TCGOp **begin_op, TCGOp *op)static TCGOp *copy_call (TCGOp **begin_op,TCGOp *op,void *empty_func,void *func,int *cb_idx) { +TCGOp *old_op; +int func_idx; +/* copy all ops until the call */ do { op =copy_op_nocheck (begin_op, op); }while (op->opc != INDEX_op_call);/* fill in the op call */ - op->param1 = (*begin_op)->param1; - op->param2 = (*begin_op)->param2; + old_op = *begin_op; +TCGOP_CALLI (op) =TCGOP_CALLI (old_op); +TCGOP_CALLO (op) =TCGOP_CALLO (old_op);tcg_debug_assert (op->life ==0 ); -if (*cb_idx ==-1 ) { -int i; -/* -* Instead of working out the position of the callback in args[], just -* look for @empty_func, since it should be a unique pointer. -*/ -for (i =0 ; i <MAX_OPC_PARAM_ARGS ; i++) { -if ((uintptr_t )(*begin_op)->args[i] == (uintptr_t )empty_func) { - *cb_idx = i; -break ; - } - } -tcg_debug_assert (i <MAX_OPC_PARAM_ARGS ); - } - op->args[*cb_idx] = (uintptr_t )func; - op->args[*cb_idx +1 ] = (*begin_op)->args[*cb_idx +1 ]; + func_idx =TCGOP_CALLO (op) +TCGOP_CALLI (op); + *cb_idx = func_idx; + + op->args[func_idx] = (uintptr_t )func; + op->args[func_idx +1 ] = old_op->args[func_idx +1 ];return op; }
Signature · 1 line
2.34.1
Re: [PATCH v4 17/27] accel/tcg/plugin: Don't search for the function pointer index
Quoted 9 lines · Richard Henderson
The function pointer is immediately after the output and input operands; no need to search.
Signed-off-by Richard Henderson <[email protected]>---
Cc: Alex Bennée <[email protected]>--- accel/tcg/plugin-gen.c | 29 +++++++++++------------------ 1 file changed, 11 insertions(+), 18 deletions(-)
PATCH v4 18/27 accel/tcg/plugin: Avoid duplicate copy in copy_call
In reply to Richard HendersonWe copied all of the arguments in copy_op_nocheck. We only need to replace the one argument that we change.
---
Cc: Alex Bennée <[email protected]>
--- accel/tcg/plugin-gen.c | 2 -- 1 file changed, 2 deletions(-)
accel/tcg/plugin-gen.c +0 -2
@@ -397,9 +397,7 @@ static TCGOp *copy_call(TCGOp **begin_op, TCGOp *op, void *empty_func, func_idx =TCGOP_CALLO (op) +TCGOP_CALLI (op); *cb_idx = func_idx; - op->args[func_idx] = (uintptr_t )func; - op->args[func_idx +1 ] = old_op->args[func_idx +1 ];return op; }
Signature · 1 line
2.34.1
Re: [PATCH v4 18/27] accel/tcg/plugin: Avoid duplicate copy in copy_call
Quoted 5 lines · Richard Henderson
We copied all of the arguments in
copy_op_nocheck. We only need to replace the one argument that we change.Reviewed-by Philippe Mathieu-Daudé <[email protected]>Signed-off-by Richard Henderson <[email protected]>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
PATCH v4 19/27 accel/tcg/plugin: Use copy_op in append_{udata, mem}_cb
In reply to Richard HendersonBetter to re-use the existing function for copying ops.
---
Cc: Alex Bennée <[email protected]>
--- accel/tcg/plugin-gen.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
accel/tcg/plugin-gen.c +8 -8
@@ -415,11 +415,11 @@ static TCGOp *append_udata_cb(const struct qemu_plugin_dyn_cb *cb, op =copy_const_ptr (&begin_op, op, cb->userp);/* copy the ld_i32, but note that we only have to copy it once */ - begin_op =QTAILQ_NEXT (begin_op, link); -tcg_debug_assert (begin_op && begin_op->opc == INDEX_op_ld_i32);if (*cb_idx ==-1 ) { - op =tcg_op_insert_after (tcg_ctx, op, INDEX_op_ld_i32); -memcpy (op->args, begin_op->args,sizeof (op->args)); + op =copy_op (&begin_op, op, INDEX_op_ld_i32); + }else { + begin_op =QTAILQ_NEXT (begin_op, link); +tcg_debug_assert (begin_op && begin_op->opc == INDEX_op_ld_i32); }/* call */
@@ -462,11 +462,11 @@ static TCGOp *append_mem_cb(const struct qemu_plugin_dyn_cb *cb, op =copy_const_ptr (&begin_op, op, cb->userp);/* copy the ld_i32, but note that we only have to copy it once */ - begin_op =QTAILQ_NEXT (begin_op, link); -tcg_debug_assert (begin_op && begin_op->opc == INDEX_op_ld_i32);if (*cb_idx ==-1 ) { - op =tcg_op_insert_after (tcg_ctx, op, INDEX_op_ld_i32); -memcpy (op->args, begin_op->args,sizeof (op->args)); + op =copy_op (&begin_op, op, INDEX_op_ld_i32); + }else { + begin_op =QTAILQ_NEXT (begin_op, link); +tcg_debug_assert (begin_op && begin_op->opc == INDEX_op_ld_i32); }/* extu_tl_i64 */
Signature · 1 line
2.34.1
Re: [PATCH v4 19/27] accel/tcg/plugin: Use copy_op in append_{udata,mem}_cb
Quoted 4 lines · Richard Henderson
Better to re-use the existing function for copying ops.
Reviewed-by Philippe Mathieu-Daudé <[email protected]>Signed-off-by Richard Henderson <[email protected]>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
PATCH v4 20/27 tcg: Vary the allocation size for TCGOp
In reply to Richard HendersonWe have been allocating a worst case number of arguments to support calls. Instead, allow the size to vary. By default leave space for 4 args, to maximize reuse, but allow calls to increase the number of args to 32.
--- include/exec/helper-head.h | 2 -- include/tcg/tcg-op.h | 2 +- include/tcg/tcg.h | 54 +++++++++++++-------------------- accel/tcg/plugin-gen.c | 11 ++++--- tcg/optimize.c | 4 +-- tcg/tcg-op-vec.c | 8 ++--- tcg/tcg-op.c | 12 ++++---- tcg/tcg.c | 61 +++++++++++++++++++++++++------------- 8 files changed, 82 insertions(+), 72 deletions(-)
include/exec/helper-head.h +0 -2
@@ -133,6 +133,4 @@#define DEF_HELPER_7 (name, ret, t1, t2, t3, t4, t5, t6, t7) \ DEF_HELPER_FLAGS_7(name, 0, ret, t1, t2, t3, t4, t5, t6, t7) -/* MAX_OPC_PARAM_IARGS must be set to n if last entry is DEF_HELPER_FLAGS_n. */ -#endif /* EXEC_HELPER_HEAD_H */
include/tcg/tcg-op.h +1 -1
@@ -818,7 +818,7 @@ static inline void tcg_gen_plugin_cb_start(unsigned from, unsigned type,static inline void tcg_gen_plugin_cb_end (void ) { -tcg_emit_op (INDEX_op_plugin_cb_end); +tcg_emit_op (INDEX_op_plugin_cb_end,0 ); }#if TARGET_LONG_BITS ==32
include/tcg/tcg.h +21 -33
85 lines not on this page · raw message
accel/tcg/plugin-gen.c +7 -4
@@ -258,10 +258,13 @@ static TCGOp *rm_ops(TCGOp *op)static TCGOp *copy_op_nocheck (TCGOp **begin_op,TCGOp *op) { - *begin_op =QTAILQ_NEXT (*begin_op, link); -tcg_debug_assert (*begin_op); - op =tcg_op_insert_after (tcg_ctx, op, (*begin_op)->opc); -memcpy (op->args, (*begin_op)->args,sizeof (op->args)); +TCGOp *old_op =QTAILQ_NEXT (*begin_op, link); +unsigned nargs = old_op->nargs; + + *begin_op = old_op; + op =tcg_op_insert_after (tcg_ctx, op, old_op->opc, nargs); +memcpy (op->args, old_op->args,sizeof (op->args[0 ]) * nargs); +return op; }
tcg/optimize.c +2 -2
@@ -962,7 +962,7 @@ static bool fold_addsub2(OptContext *ctx, TCGOp *op, bool add) rh = op->args[1 ];/* The proper opcode is supplied by tcg_opt_gen_mov. */ - op2 =tcg_op_insert_before (ctx->tcg, op,0 ); + op2 =tcg_op_insert_before (ctx->tcg, op,0 ,2 );tcg_opt_gen_movi (ctx, op, rl, al);tcg_opt_gen_movi (ctx, op2, rh, ah);
@@ -1613,7 +1613,7 @@ static bool fold_multiply2(OptContext *ctx, TCGOp *op) rh = op->args[1 ];/* The proper opcode is supplied by tcg_opt_gen_mov. */ - op2 =tcg_op_insert_before (ctx->tcg, op,0 ); + op2 =tcg_op_insert_before (ctx->tcg, op,0 ,2 );tcg_opt_gen_movi (ctx, op, rl, l);tcg_opt_gen_movi (ctx, op2, rh, h);
tcg/tcg-op-vec.c +4 -4
@@ -152,7 +152,7 @@ bool tcg_can_emit_vecop_list(const TCGOpcode *list,void vec_gen_2 (TCGOpcode opc,TCGType type,unsigned vece,TCGArg r,TCGArg a) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,2 );TCGOP_VECL (op) = type -TCG_TYPE_V64 ;TCGOP_VECE (op) = vece; op->args[0 ] = r;
@@ -162,7 +162,7 @@ void vec_gen_2(TCGOpcode opc, TCGType type, unsigned vece, TCGArg r, TCGArg a)void vec_gen_3 (TCGOpcode opc,TCGType type,unsigned vece,TCGArg r,TCGArg a,TCGArg b) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,3 );TCGOP_VECL (op) = type -TCG_TYPE_V64 ;TCGOP_VECE (op) = vece; op->args[0 ] = r;
@@ -173,7 +173,7 @@ void vec_gen_3(TCGOpcode opc, TCGType type, unsigned vece,void vec_gen_4 (TCGOpcode opc,TCGType type,unsigned vece,TCGArg r,TCGArg a,TCGArg b,TCGArg c) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,4 );TCGOP_VECL (op) = type -TCG_TYPE_V64 ;TCGOP_VECE (op) = vece; op->args[0 ] = r;
@@ -185,7 +185,7 @@ void vec_gen_4(TCGOpcode opc, TCGType type, unsigned vece,static void vec_gen_6 (TCGOpcode opc,TCGType type,unsigned vece,TCGArg r,TCGArg a,TCGArg b,TCGArg c,TCGArg d,TCGArg e) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,6 );TCGOP_VECL (op) = type -TCG_TYPE_V64 ;TCGOP_VECE (op) = vece; op->args[0 ] = r;
tcg/tcg-op.c +6 -6
@@ -33,20 +33,20 @@void tcg_gen_op1 (TCGOpcode opc,TCGArg a1) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,1 ); op->args[0 ] = a1; }void tcg_gen_op2 (TCGOpcode opc,TCGArg a1,TCGArg a2) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,2 ); op->args[0 ] = a1; op->args[1 ] = a2; }void tcg_gen_op3 (TCGOpcode opc,TCGArg a1,TCGArg a2,TCGArg a3) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,3 ); op->args[0 ] = a1; op->args[1 ] = a2; op->args[2 ] = a3;
@@ -54,7 +54,7 @@ void tcg_gen_op3(TCGOpcode opc, TCGArg a1, TCGArg a2, TCGArg a3)void tcg_gen_op4 (TCGOpcode opc,TCGArg a1,TCGArg a2,TCGArg a3,TCGArg a4) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,4 ); op->args[0 ] = a1; op->args[1 ] = a2; op->args[2 ] = a3;
@@ -64,7 +64,7 @@ void tcg_gen_op4(TCGOpcode opc, TCGArg a1, TCGArg a2, TCGArg a3, TCGArg a4)void tcg_gen_op5 (TCGOpcode opc,TCGArg a1,TCGArg a2,TCGArg a3,TCGArg a4,TCGArg a5) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,5 ); op->args[0 ] = a1; op->args[1 ] = a2; op->args[2 ] = a3;
@@ -75,7 +75,7 @@ void tcg_gen_op5(TCGOpcode opc, TCGArg a1, TCGArg a2, TCGArg a3,void tcg_gen_op6 (TCGOpcode opc,TCGArg a1,TCGArg a2,TCGArg a3,TCGArg a4,TCGArg a5,TCGArg a6) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,6 ); op->args[0 ] = a1; op->args[1 ] = a2; op->args[2 ] = a3;
tcg/tcg.c +41 -20
124 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v4 20/27] tcg: Vary the allocation size for TCGOp
Splitting this patch in 2 to ease review.
Philippe Mathieu-Daudé (1):
tcg: Pass number of arguments to tcg_emit_op() / tcg_op_insert_*()
Richard Henderson (1):
tcg: Vary the allocation size for TCGOp
accel/tcg/plugin-gen.c | 11 ++++--- include/exec/helper-head.h | 2 -- include/tcg/tcg-op.h | 2 +- include/tcg/tcg.h | 54 +++++++++++++-------------------- tcg/optimize.c | 4 +-- tcg/tcg-op-vec.c | 8 ++--- tcg/tcg-op.c | 12 ++++---- tcg/tcg.c | 61 +++++++++++++++++++++++++------------- 8 files changed, 82 insertions(+), 72 deletions(-)
Signature · 1 line
2.38.1
PATCH v4 1/1 tcg: Pass number of arguments to tcg_emit_op() / tcg_op_insert_*()
In order to have variable size allocated TCGOp, pass the number of arguments we use (and would allocate) up to tcg_op_alloc().
This alters tcg_emit_op(), tcg_op_insert_before() and tcg_op_insert_after() prototypes.
In tcg_op_alloc() ensure the number of arguments is in range.
--- accel/tcg/plugin-gen.c | 5 ++++- include/tcg/tcg-op.h | 2 +- include/tcg/tcg.h | 8 +++++--- tcg/optimize.c | 4 ++-- tcg/tcg-op-vec.c | 8 ++++---- tcg/tcg-op.c | 12 ++++++------ tcg/tcg.c | 30 +++++++++++++++++------------- 7 files changed, 39 insertions(+), 30 deletions(-)
accel/tcg/plugin-gen.c +4 -1
@@ -258,10 +258,13 @@ static TCGOp *rm_ops(TCGOp *op)static TCGOp *copy_op_nocheck (TCGOp **begin_op,TCGOp *op) { +unsigned nargs =ARRAY_SIZE (op->args); + *begin_op =QTAILQ_NEXT (*begin_op, link);tcg_debug_assert (*begin_op); - op =tcg_op_insert_after (tcg_ctx, op, (*begin_op)->opc); + op =tcg_op_insert_after (tcg_ctx, op, (*begin_op)->opc, nargs);memcpy (op->args, (*begin_op)->args,sizeof (op->args)); +return op; }
include/tcg/tcg-op.h +1 -1
@@ -818,7 +818,7 @@ static inline void tcg_gen_plugin_cb_start(unsigned from, unsigned type,static inline void tcg_gen_plugin_cb_end (void ) { -tcg_emit_op (INDEX_op_plugin_cb_end); +tcg_emit_op (INDEX_op_plugin_cb_end,0 ); }#if TARGET_LONG_BITS ==32
include/tcg/tcg.h +5 -3
@@ -1014,10 +1014,12 @@ bool tcg_op_supported(TCGOpcode op);void tcg_gen_callN (void *func,TCGTemp *ret,int nargs,TCGTemp **args); -TCGOp *tcg_emit_op (TCGOpcode opc); +TCGOp *tcg_emit_op (TCGOpcode opc,unsigned nargs);void tcg_op_remove (TCGContext *s,TCGOp *op); -TCGOp *tcg_op_insert_before (TCGContext *s,TCGOp *op,TCGOpcode opc); -TCGOp *tcg_op_insert_after (TCGContext *s,TCGOp *op,TCGOpcode opc); +TCGOp *tcg_op_insert_before (TCGContext *s,TCGOp *op, +TCGOpcode opc,unsigned nargs); +TCGOp *tcg_op_insert_after (TCGContext *s,TCGOp *op, +TCGOpcode opc,unsigned nargs);/** * tcg_remove_ops_after:
tcg/optimize.c +2 -2
@@ -962,7 +962,7 @@ static bool fold_addsub2(OptContext *ctx, TCGOp *op, bool add) rh = op->args[1 ];/* The proper opcode is supplied by tcg_opt_gen_mov. */ - op2 =tcg_op_insert_before (ctx->tcg, op,0 ); + op2 =tcg_op_insert_before (ctx->tcg, op,0 ,2 );tcg_opt_gen_movi (ctx, op, rl, al);tcg_opt_gen_movi (ctx, op2, rh, ah);
@@ -1613,7 +1613,7 @@ static bool fold_multiply2(OptContext *ctx, TCGOp *op) rh = op->args[1 ];/* The proper opcode is supplied by tcg_opt_gen_mov. */ - op2 =tcg_op_insert_before (ctx->tcg, op,0 ); + op2 =tcg_op_insert_before (ctx->tcg, op,0 ,2 );tcg_opt_gen_movi (ctx, op, rl, l);tcg_opt_gen_movi (ctx, op2, rh, h);
tcg/tcg-op-vec.c +4 -4
@@ -152,7 +152,7 @@ bool tcg_can_emit_vecop_list(const TCGOpcode *list,void vec_gen_2 (TCGOpcode opc,TCGType type,unsigned vece,TCGArg r,TCGArg a) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,2 );TCGOP_VECL (op) = type -TCG_TYPE_V64 ;TCGOP_VECE (op) = vece; op->args[0 ] = r;
@@ -162,7 +162,7 @@ void vec_gen_2(TCGOpcode opc, TCGType type, unsigned vece, TCGArg r, TCGArg a)void vec_gen_3 (TCGOpcode opc,TCGType type,unsigned vece,TCGArg r,TCGArg a,TCGArg b) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,3 );TCGOP_VECL (op) = type -TCG_TYPE_V64 ;TCGOP_VECE (op) = vece; op->args[0 ] = r;
@@ -173,7 +173,7 @@ void vec_gen_3(TCGOpcode opc, TCGType type, unsigned vece,void vec_gen_4 (TCGOpcode opc,TCGType type,unsigned vece,TCGArg r,TCGArg a,TCGArg b,TCGArg c) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,4 );TCGOP_VECL (op) = type -TCG_TYPE_V64 ;TCGOP_VECE (op) = vece; op->args[0 ] = r;
@@ -185,7 +185,7 @@ void vec_gen_4(TCGOpcode opc, TCGType type, unsigned vece,static void vec_gen_6 (TCGOpcode opc,TCGType type,unsigned vece,TCGArg r,TCGArg a,TCGArg b,TCGArg c,TCGArg d,TCGArg e) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,6 );TCGOP_VECL (op) = type -TCG_TYPE_V64 ;TCGOP_VECE (op) = vece; op->args[0 ] = r;
tcg/tcg-op.c +6 -6
@@ -33,20 +33,20 @@void tcg_gen_op1 (TCGOpcode opc,TCGArg a1) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,1 ); op->args[0 ] = a1; }void tcg_gen_op2 (TCGOpcode opc,TCGArg a1,TCGArg a2) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,2 ); op->args[0 ] = a1; op->args[1 ] = a2; }void tcg_gen_op3 (TCGOpcode opc,TCGArg a1,TCGArg a2,TCGArg a3) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,3 ); op->args[0 ] = a1; op->args[1 ] = a2; op->args[2 ] = a3;
@@ -54,7 +54,7 @@ void tcg_gen_op3(TCGOpcode opc, TCGArg a1, TCGArg a2, TCGArg a3)void tcg_gen_op4 (TCGOpcode opc,TCGArg a1,TCGArg a2,TCGArg a3,TCGArg a4) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,4 ); op->args[0 ] = a1; op->args[1 ] = a2; op->args[2 ] = a3;
@@ -64,7 +64,7 @@ void tcg_gen_op4(TCGOpcode opc, TCGArg a1, TCGArg a2, TCGArg a3, TCGArg a4)void tcg_gen_op5 (TCGOpcode opc,TCGArg a1,TCGArg a2,TCGArg a3,TCGArg a4,TCGArg a5) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,5 ); op->args[0 ] = a1; op->args[1 ] = a2; op->args[2 ] = a3;
@@ -75,7 +75,7 @@ void tcg_gen_op5(TCGOpcode opc, TCGArg a1, TCGArg a2, TCGArg a3,void tcg_gen_op6 (TCGOpcode opc,TCGArg a1,TCGArg a2,TCGArg a3,TCGArg a4,TCGArg a5,TCGArg a6) { -TCGOp *op =tcg_emit_op (opc); +TCGOp *op =tcg_emit_op (opc,6 ); op->args[0 ] = a1; op->args[1 ] = a2; op->args[2 ] = a3;
tcg/tcg.c +17 -13
@@ -1479,7 +1479,7 @@ bool tcg_op_supported(TCGOpcode op)and endian swap in tcg_reg_alloc_call(). */ void tcg_gen_callN (void *func,TCGTemp *ret,int nargs,TCGTemp **args) { -int i, real_args, nb_rets, pi; +int i, real_args, nb_rets, pi, max_args;unsigned typemask;const TCGHelperInfo *info;TCGOp *op;
@@ -1513,7 +1513,8 @@ void tcg_gen_callN(void *func, TCGTemp *ret, int nargs, TCGTemp **args) } } - op =tcg_emit_op (INDEX_op_call); + max_args =ARRAY_SIZE (op->args); + op =tcg_emit_op (INDEX_op_call, max_args); pi =0 ;if (ret !=NULL ) {
@@ -1590,7 +1591,7 @@ void tcg_gen_callN(void *func, TCGTemp *ret, int nargs, TCGTemp **args)/* Make sure the fields didn't overflow. */ tcg_debug_assert (TCGOP_CALLI (op) == real_args); -tcg_debug_assert (pi <=ARRAY_SIZE (op->args)); +tcg_debug_assert (pi <= max_args);if (TCG_TARGET_CALL_ARG_I32 ==TCG_CALL_ARG_EXTEND ) {for (i =0 ; i < nargs; ++i) {
@@ -2294,11 +2295,12 @@ void tcg_remove_ops_after(TCGOp *op) } } -static TCGOp *tcg_op_alloc (TCGOpcode opc) +static TCGOp *tcg_op_alloc (TCGOpcode opc,unsigned nargs) {TCGContext *s = tcg_ctx;TCGOp *op; +assert (nargs <ARRAY_SIZE (op->args));if (likely (QTAILQ_EMPTY (&s->free_ops))) { op =tcg_malloc (sizeof (TCGOp)); }else {
@@ -2312,23 +2314,25 @@ static TCGOp *tcg_op_alloc(TCGOpcode opc)return op; } -TCGOp *tcg_emit_op (TCGOpcode opc) +TCGOp *tcg_emit_op (TCGOpcode opc,unsigned nargs) { -TCGOp *op =tcg_op_alloc (opc); +TCGOp *op =tcg_op_alloc (opc, nargs);QTAILQ_INSERT_TAIL (&tcg_ctx->ops, op, link);return op; } -TCGOp *tcg_op_insert_before (TCGContext *s,TCGOp *old_op,TCGOpcode opc) +TCGOp *tcg_op_insert_before (TCGContext *s,TCGOp *old_op, +TCGOpcode opc,unsigned nargs) { -TCGOp *new_op =tcg_op_alloc (opc); +TCGOp *new_op =tcg_op_alloc (opc, nargs);QTAILQ_INSERT_BEFORE (old_op, new_op, link);return new_op; } -TCGOp *tcg_op_insert_after (TCGContext *s,TCGOp *old_op,TCGOpcode opc) +TCGOp *tcg_op_insert_after (TCGContext *s,TCGOp *old_op, +TCGOpcode opc,unsigned nargs) { -TCGOp *new_op =tcg_op_alloc (opc); +TCGOp *new_op =tcg_op_alloc (opc, nargs);QTAILQ_INSERT_AFTER (&s->ops, old_op, new_op, link);return new_op; }
@@ -2937,7 +2941,7 @@ static bool liveness_pass_2(TCGContext *s)TCGOpcode lopc = (arg_ts->type ==TCG_TYPE_I32 ? INDEX_op_ld_i32 : INDEX_op_ld_i64); -TCGOp *lop =tcg_op_insert_before (s, op, lopc); +TCGOp *lop =tcg_op_insert_before (s, op, lopc,3 ); lop->args[0 ] =temp_arg (dir_ts); lop->args[1 ] =temp_arg (arg_ts->mem_base);
@@ -3003,7 +3007,7 @@ static bool liveness_pass_2(TCGContext *s)TCGOpcode sopc = (arg_ts->type ==TCG_TYPE_I32 ? INDEX_op_st_i32 : INDEX_op_st_i64); -TCGOp *sop =tcg_op_insert_after (s, op, sopc); +TCGOp *sop =tcg_op_insert_after (s, op, sopc,3 );TCGTemp *out_ts = dir_ts;if (IS_DEAD_ARG (0 )) {
@@ -3039,7 +3043,7 @@ static bool liveness_pass_2(TCGContext *s)TCGOpcode sopc = (arg_ts->type ==TCG_TYPE_I32 ? INDEX_op_st_i32 : INDEX_op_st_i64); -TCGOp *sop =tcg_op_insert_after (s, op, sopc); +TCGOp *sop =tcg_op_insert_after (s, op, sopc,3 ); sop->args[0 ] =temp_arg (dir_ts); sop->args[1 ] =temp_arg (arg_ts->mem_base);
Signature · 1 line
2.38.1
Re: [PATCH v4 20a/27] tcg: Pass number of arguments to tcg_emit_op() / tcg_op_insert_*()
Quoted 9 lines · Philippe Mathieu-Daudé
In order to have variable size allocated TCGOp, pass the number of arguments we use (and would allocate) up to
tcg_op_alloc().This alters
tcg_emit_op(),tcg_op_insert_before()andtcg_op_insert_after()prototypes.In
tcg_op_alloc()ensure the number of arguments is in range.Signed-off-by Richard Henderson <[email protected]>
Quoted 11 lines
[PMD: Extracted from bigger patch]Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- accel/tcg/plugin-gen.c | 5 ++++- include/tcg/tcg-op.h | 2 +- include/tcg/tcg.h | 8 +++++--- tcg/optimize.c | 4 ++-- tcg/tcg-op-vec.c | 8 ++++---- tcg/tcg-op.c | 12 ++++++------ tcg/tcg.c | 30 +++++++++++++++++------------- 7 files changed, 39 insertions(+), 30 deletions(-)
PATCH v4 1/1 tcg: Vary the allocation size for TCGOp
In reply to Philippe Mathieu-DaudéFrom: Richard Henderson <[email protected]>
We have been allocating a worst case number of arguments to support calls. Instead, allow the size to vary. By default leave space for 4 args, to maximize reuse, but allow calls to increase the number of args to 32.
--- accel/tcg/plugin-gen.c | 10 ++++----- include/exec/helper-head.h | 2 -- include/tcg/tcg.h | 46 +++++++++++++------------------------- tcg/tcg.c | 35 +++++++++++++++++++++-------- 4 files changed, 47 insertions(+), 46 deletions(-)
accel/tcg/plugin-gen.c +5 -5
@@ -258,12 +258,12 @@ static TCGOp *rm_ops(TCGOp *op)static TCGOp *copy_op_nocheck (TCGOp **begin_op,TCGOp *op) { -unsigned nargs =ARRAY_SIZE (op->args); +TCGOp *old_op =QTAILQ_NEXT (*begin_op, link); +unsigned nargs = old_op->nargs; - *begin_op =QTAILQ_NEXT (*begin_op, link); -tcg_debug_assert (*begin_op); - op =tcg_op_insert_after (tcg_ctx, op, (*begin_op)->opc, nargs); -memcpy (op->args, (*begin_op)->args,sizeof (op->args)); + *begin_op = old_op; + op =tcg_op_insert_after (tcg_ctx, op, old_op->opc, nargs); +memcpy (op->args, old_op->args,sizeof (op->args[0 ]) * nargs);return op; }
include/exec/helper-head.h +0 -2
@@ -133,6 +133,4 @@#define DEF_HELPER_7 (name, ret, t1, t2, t3, t4, t5, t6, t7) \ DEF_HELPER_FLAGS_7(name, 0, ret, t1, t2, t3, t4, t5, t6, t7) -/* MAX_OPC_PARAM_IARGS must be set to n if last entry is DEF_HELPER_FLAGS_n. */ -#endif /* EXEC_HELPER_HEAD_H */
include/tcg/tcg.h +16 -30
70 lines not on this page · raw message
tcg/tcg.c +26 -9
@@ -1513,7 +1513,12 @@ void tcg_gen_callN(void *func, TCGTemp *ret, int nargs, TCGTemp **args) } } - max_args =ARRAY_SIZE (op->args); +/* +* A Call op needs up to 4 + 2N parameters on 32-bit archs, +* and up to 4 + N parameters on 64-bit archs +* (N = number of input arguments + output arguments). +*/ + max_args = (64 /TCG_TARGET_REG_BITS ) * nargs +4 ; op =tcg_emit_op (INDEX_op_call, max_args); pi =0 ;
@@ -2298,19 +2303,31 @@ void tcg_remove_ops_after(TCGOp *op)static TCGOp *tcg_op_alloc (TCGOpcode opc,unsigned nargs) {TCGContext *s = tcg_ctx; -TCGOp *op; +TCGOp *op =NULL ; -assert (nargs <ARRAY_SIZE (op->args)); -if (likely (QTAILQ_EMPTY (&s->free_ops))) { - op =tcg_malloc (sizeof (TCGOp)); - }else { - op =QTAILQ_FIRST (&s->free_ops); -QTAILQ_REMOVE (&s->free_ops, op, link); +if (unlikely (!QTAILQ_EMPTY (&s->free_ops))) { + QTAILQ_FOREACH(op, &s->free_ops, link) { +if (nargs <= op->nargs) { +QTAILQ_REMOVE (&s->free_ops, op, link); + nargs = op->nargs; + goto found; + } + } } + +/* Most opcodes have 3 or 4 operands: reduce fragmentation. */ + nargs =MAX (4 , nargs); + op =tcg_malloc (sizeof (TCGOp) +sizeof (TCGArg) * nargs); + + found:memset (op,0 , offsetof(TCGOp , link)); op->opc = opc; - s->nb_ops++; + op->nargs = nargs; +/* Check for bitfield overflow. */ +tcg_debug_assert (op->nargs == nargs); + + s->nb_ops++;return op; }
Signature · 1 line
2.38.1
Re: [PATCH v4 20b/27] tcg: Vary the allocation size for TCGOp
Quoted 58 lines · Philippe Mathieu-Daudé
From: Richard Henderson <[email protected]>
We have been allocating a worst case number of arguments to support calls. Instead, allow the size to vary. By default leave space for 4 args, to maximize reuse, but allow calls to increase the number of args to 32.
Signed-off-by Richard Henderson <[email protected]>[PMD: Split patch in two]Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- accel/tcg/plugin-gen.c | 10 ++++----- include/exec/helper-head.h | 2 -- include/tcg/tcg.h | 46 +++++++++++++------------------------- tcg/tcg.c | 35 +++++++++++++++++++++-------- 4 files changed, 47 insertions(+), 46 deletions(-)tcg/tcg.c +15
-8@@ -1513,7 +1513,12 @@ void tcg_gen_callN(void *func, TCGTemp *ret, int nargs, TCGTemp **args) } } - max_args =ARRAY_SIZE (op->args); +/* +* A Call op needs up to 4 + 2N parameters on 32-bit archs, +* and up to 4 + N parameters on 64-bit archs +* (N = number of input arguments + output arguments). +*/ + max_args = (64 /TCG_TARGET_REG_BITS ) * nargs +4 ; op =tcg_emit_op (INDEX_op_call, max_args); pi =0 ;@@ -2298,19 +2303,31 @@ void tcg_remove_ops_after(TCGOp *op)static TCGOp *tcg_op_alloc (TCGOpcode opc,unsigned nargs) {TCGContext *s = tcg_ctx; -TCGOp *op; +TCGOp *op =NULL ; -assert (nargs <ARRAY_SIZE (op->args)); -if (likely (QTAILQ_EMPTY (&s->free_ops))) { - op =tcg_malloc (sizeof (TCGOp)); - }else { - op =QTAILQ_FIRST (&s->free_ops); -QTAILQ_REMOVE (&s->free_ops, op, link); +if (unlikely (!QTAILQ_EMPTY (&s->free_ops))) { + QTAILQ_FOREACH(op, &s->free_ops, link) { +if (nargs <= op->nargs) { +QTAILQ_REMOVE (&s->free_ops, op, link); + nargs = op->nargs; + goto found; + } + } }
tcg/tcg.c void tcg_remove_ops_after(TCGOp *op) · line 2325
Quoted 10 lines
tcg/tcg.c +7
-1+ +/* Most opcodes have 3 or 4 operands: reduce fragmentation. */ + nargs =MAX (4 , nargs); + op =tcg_malloc (sizeof (TCGOp) +sizeof (TCGArg) * nargs); + + found:memset (op,0 , offsetof(TCGOp , link)); op->opc = opc; - s->nb_ops++; + op->nargs = nargs;
We can move this assignation before the 'found' label.
tcg/tcg.c void tcg_remove_ops_after(TCGOp *op) · line 2333
Quoted 8 lines
tcg/tcg.c +4
-0+/* Check for bitfield overflow. */ +tcg_debug_assert (op->nargs == nargs); + + s->nb_ops++;return op; }
Re: [PATCH v4 20b/27] tcg: Vary the allocation size for TCGOp
Quoted 7 lines · Philippe Mathieu-Daudé
Quoted 5 lines
+ found:
memset(op, 0, offsetof(TCGOp, link));
op->opc= opc;
- s->nb_ops++;
+op->nargs= nargs;We can move this assignation before the 'found' label.
No, affected by the memset.
r~
Re: [PATCH v4 20b/27] tcg: Vary the allocation size for TCGOp
Quoted 10 lines · Richard Henderson
Quoted 7 lines · Philippe Mathieu-Daudé
Quoted 5 lines
+ found:
memset(op, 0, offsetof(TCGOp, link));
op->opc= opc;
- s->nb_ops++;
+op->nargs= nargs;We can move this assignation before the 'found' label.
No, affected by the memset.
Oh, I missed that, good point.
Re: [PATCH v4 20/27] tcg: Vary the allocation size for TCGOp
In reply to Richard HendersonQuoted 6 lines · Richard Henderson
We have been allocating a worst case number of arguments to support calls. Instead, allow the size to vary. By default leave space for 4 args, to maximize reuse, but allow calls to increase the number of args to 32.
Signed-off-by Richard Henderson <[email protected]>
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
PATCH v4 21/27 tcg: Use output_pref wrapper function
In reply to Richard HendersonWe will shortly have the possibility of more that two outputs, though only for calls (for which preferences are moot). Avoid direct references to op->output_pref[] when possible.
--- include/tcg/tcg.h | 5 +++++ tcg/tcg.c | 34 ++++++++++++++++++---------------- 2 files changed, 23 insertions(+), 16 deletions(-)
include/tcg/tcg.h +5 -0
@@ -518,6 +518,11 @@ typedef struct TCGOp {/* Make sure operands fit in the bitfields above. */ QEMU_BUILD_BUG_ON (NB_OPS > (1 <<8 )); +static inline TCGRegSet output_pref (const TCGOp *op,unsigned i) +{ +return i <ARRAY_SIZE (op->output_pref) ? op->output_pref[i] :0 ; +} + typedef struct TCGProfile { int64_t cpu_exec_time; int64_t tb_count1;
tcg/tcg.c +18 -16
@@ -1966,7 +1966,7 @@ static void tcg_dump_ops(TCGContext *s, FILE *f, bool have_prefs)if (have_prefs) {for (i =0 ; i < nb_oargs; ++i) { -TCGRegSet set = op->output_pref[i]; +TCGRegSet set =output_pref (op, i);if (i ==0 ) {ne_fprintf (f," pref=" );
@@ -2636,11 +2636,11 @@ static void liveness_pass_1(TCGContext *s) } ts->state =TS_DEAD ;la_reset_pref (ts); - -/* Not used -- it will be tcg_target_call_oarg_regs[i]. */ - op->output_pref[i] =0 ; } +/* Not used -- it will be tcg_target_call_oarg_reg(). */ +memset (op->output_pref,0 ,sizeof (op->output_pref)); +if (!(call_flags & (TCG_CALL_NO_WRITE_GLOBALS |TCG_CALL_NO_READ_GLOBALS ))) {la_global_kill (s, nb_globals);
@@ -2802,7 +2802,9 @@ static void liveness_pass_1(TCGContext *s) ts =arg_temp (op->args[i]);/* Remember the preference of the uses that followed. */ - op->output_pref[i] = *la_temp_pref (ts); +if (i <ARRAY_SIZE (op->output_pref)) { + op->output_pref[i] = *la_temp_pref (ts); + }/* Output args are dead. */ if (ts->state &TS_DEAD ) {
@@ -2872,7 +2874,7 @@ static void liveness_pass_1(TCGContext *s) set &= ct->regs;if (ct->ialias) { - set &= op->output_pref[ct->alias_index]; + set &=output_pref (op, ct->alias_index); }/* If the combination is not possible, restart. */ if (set ==0 ) {
@@ -3539,7 +3541,7 @@ static void tcg_reg_alloc_mov(TCGContext *s, const TCGOp *op)TCGReg oreg, ireg; allocated_regs = s->reserved_regs; - preferred_regs = op->output_pref[0 ]; + preferred_regs =output_pref (op,0 ); ots =arg_temp (op->args[0 ]); ts =arg_temp (op->args[1 ]);
@@ -3656,7 +3658,7 @@ static void tcg_reg_alloc_dup(TCGContext *s, const TCGOp *op)if (IS_DEAD_ARG (1 )) {temp_dead (s, its); } -tcg_reg_alloc_do_movi (s, ots, val, arg_life, op->output_pref[0 ]); +tcg_reg_alloc_do_movi (s, ots, val, arg_life,output_pref (op,0 ));return ; }
@@ -3673,7 +3675,7 @@ static void tcg_reg_alloc_dup(TCGContext *s, const TCGOp *op)tcg_regset_set_reg (allocated_regs, its->reg); } oreg =tcg_reg_alloc (s, dup_out_regs, allocated_regs, - op->output_pref[0 ], ots->indirect_base); +output_pref (op,0 ), ots->indirect_base);set_temp_val_reg (s, ots, oreg); }
@@ -3792,7 +3794,7 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op)switch (arg_ct->pair) {case 0 :/* not paired */ if (arg_ct->ialias) { - i_preferred_regs = op->output_pref[arg_ct->alias_index]; + i_preferred_regs =output_pref (op, arg_ct->alias_index);/* * If the input is not dead after the instruction,
@@ -3839,7 +3841,7 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op)* and to identify a few cases where it's not required. */ if (arg_ct->ialias) { - i_preferred_regs = op->output_pref[arg_ct->alias_index]; + i_preferred_regs =output_pref (op, arg_ct->alias_index); if (IS_DEAD_ARG(i1) && IS_DEAD_ARG(i2) && ts->val_type == TEMP_VAL_REG &&
@@ -3873,7 +3875,7 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op)case 3 :/* ialias with second output, no first input */ tcg_debug_assert (arg_ct->ialias); - i_preferred_regs = op->output_pref[arg_ct->alias_index]; + i_preferred_regs =output_pref (op, arg_ct->alias_index); if (IS_DEAD_ARG(i) && ts->val_type == TEMP_VAL_REG &&
@@ -3993,10 +3995,10 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op) }else if (arg_ct->newreg) { reg =tcg_reg_alloc (s, arg_ct->regs, i_allocated_regs | o_allocated_regs, - op->output_pref[k], ts->indirect_base); +output_pref (op, k), ts->indirect_base); }else { reg =tcg_reg_alloc (s, arg_ct->regs, o_allocated_regs, - op->output_pref[k], ts->indirect_base); +output_pref (op, k), ts->indirect_base); }break ;
@@ -4007,7 +4009,7 @@ static void tcg_reg_alloc_op(TCGContext *s, const TCGOp *op)break ; } reg =tcg_reg_alloc_pair (s, arg_ct->regs, o_allocated_regs, - op->output_pref[k], ts->indirect_base); +output_pref (op, k), ts->indirect_base);break ;case 2 :/* second of pair */
@@ -4090,7 +4092,7 @@ static bool tcg_reg_alloc_dup2(TCGContext *s, const TCGOp *op) } oreg =tcg_reg_alloc (s, dup_out_regs, allocated_regs, - op->output_pref[0 ], ots->indirect_base); +output_pref (op,0 ), ots->indirect_base);set_temp_val_reg (s, ots, oreg); }
Signature · 1 line
2.34.1
PATCH v4 22/27 tcg: Reorg function calls
In reply to Richard HendersonPre-compute the function call layout for each helper at startup. Drop TCG_CALL_DUMMY_ARG, as we no longer need to leave gaps in the op->args[] array. This allows several places to stop checking for NULL TCGTemp, to which TCG_CALL_DUMMY_ARG mapped.
For tcg_gen_callN, loop over the arguments once. Allocate the TCGOp for the call early but delay emitting it, collecting arguments first. This allows the argument processing loop to emit code for extensions and have them sequenced before the call.
For tcg_reg_alloc_call, loop over the arguments in reverse order, which allows stack slots to be filled first naturally.
--- include/exec/helper-head.h | 2 + include/tcg/tcg.h | 5 +- tcg/tcg-internal.h | 22 +- tcg/optimize.c | 6 +- tcg/tcg.c | 609 ++++++++++++++++++++++--------------- 5 files changed, 394 insertions(+), 250 deletions(-)
include/exec/helper-head.h +2 -0
@@ -133,4 +133,6 @@#define DEF_HELPER_7 (name, ret, t1, t2, t3, t4, t5, t6, t7) \ DEF_HELPER_FLAGS_7(name, 0, ret, t1, t2, t3, t4, t5, t6, t7) +/* MAX_CALL_IARGS must be set to n if last entry is DEF_HELPER_FLAGS_n. */ +#endif /* EXEC_HELPER_HEAD_H */
include/tcg/tcg.h +2 -3
@@ -38,6 +38,8 @@/* XXX: make safe guess about sizes */ #define MAX_OP_PER_INSTR 266 +#define MAX_CALL_IARGS 7 +#define CPU_TEMP_BUF_NLONGS 128#define TCG_STATIC_FRAME_SIZE (CPU_TEMP_BUF_NLONGS * sizeof(long))
@@ -411,9 +413,6 @@ typedef TCGv_ptr TCGv_env;#define TCG_CALL_NO_RWG_SE (TCG_CALL_NO_RWG | TCG_CALL_NO_SE)#define TCG_CALL_NO_WG_SE (TCG_CALL_NO_WG | TCG_CALL_NO_SE) -/* Used to align parameters. See the comment before tcgv_i32_temp. */ -#define TCG_CALL_DUMMY_ARG ((TCGArg)0) -/* * Flags for the bswap opcodes. * If IZ, the input is zero-extended, otherwise unknown.
tcg/tcg-internal.h +20 -2
@@ -42,11 +42,29 @@ typedef enum {TCG_CALL_ARG_EXTEND_S ,/* ... as a sign-extended i64 */ } TCGCallArgumentKind; +typedef struct TCGCallArgumentLoc { +TCGCallArgumentKind kind :8 ; +unsigned arg_slot :8 ; +unsigned ref_slot :8 ; +unsigned arg_idx :4 ; +unsigned tmp_subindex :2 ; +}TCGCallArgumentLoc ; + +/* Avoid "unsigned < 0 is always false" Werror, when iarg_regs is empty. */ +#define REG_P (L ) \ + ((int)(L)->arg_slot < (int)ARRAY_SIZE(tcg_target_call_iarg_regs)) +typedef struct TCGHelperInfo {void *func;const char *name; -unsigned flags; -unsigned typemask; +unsigned typemask :32 ; +unsigned flags :8 ; +unsigned nr_in :8 ; +unsigned nr_out :8 ; +TCGCallReturnKind out_kind :8 ; + +/* Maximum physical arguments are constrained by TCG_TYPE_I128. */ +TCGCallArgumentLoc in[MAX_CALL_IARGS * (128 /TCG_TARGET_REG_BITS )]; }TCGHelperInfo ;extern TCGContext tcg_init_ctx;
tcg/optimize.c +2 -4
@@ -667,9 +667,7 @@ static void init_arguments(OptContext *ctx, TCGOp *op, int nb_args) {for (int i =0 ; i < nb_args; i++) {TCGTemp *ts =arg_temp (op->args[i]); -if (ts) { -init_ts_info (ctx, ts); - } +init_ts_info (ctx, ts); } }
@@ -680,7 +678,7 @@ static void copy_propagate(OptContext *ctx, TCGOp *op,for (int i = nb_oargs; i < nb_oargs + nb_iargs; i++) {TCGTemp *ts =arg_temp (op->args[i]); -if (ts &&ts_is_copy (ts)) { +if (ts_is_copy (ts)) { op->args[i] =temp_arg (find_better_copy (s, ts)); } }
tcg/tcg.c +368 -241
750 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v4 23/27 tcg: Convert typecode_to_ffi from array to function
In reply to Richard HendersonFrom: Philippe Mathieu-Daudé <[email protected]>
In the unlikely case of invalid typecode mask, the function will abort instead of returning a NULL pointer.
Message-Id: <[email protected]>
Message-Id: <[email protected]>
--- tcg/tcg.c | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-)
tcg/tcg.c +20 -10
@@ -555,14 +555,24 @@ static GHashTable *helper_table;#ifdef CONFIG_TCG_INTERPRETER static GHashTable *ffi_table; -static ffi_type *const typecode_to_ffi[8 ] = { - [dh_typecode_void] = &ffi_type_void, - [dh_typecode_i32] = &ffi_type_uint32, - [dh_typecode_s32] = &ffi_type_sint32, - [dh_typecode_i64] = &ffi_type_uint64, - [dh_typecode_s64] = &ffi_type_sint64, - [dh_typecode_ptr] = &ffi_type_pointer, -}; +static ffi_type *typecode_to_ffi (int argmask) +{ +switch (argmask) { +case dh_typecode_void: +return &ffi_type_void; +case dh_typecode_i32: +return &ffi_type_uint32; +case dh_typecode_s32: +return &ffi_type_sint32; +case dh_typecode_i64: +return &ffi_type_uint64; +case dh_typecode_s64: +return &ffi_type_sint64; +case dh_typecode_ptr: +return &ffi_type_pointer; + } +g_assert_not_reached (); +}#endif typedef struct TCGCumulativeArgs {
@@ -779,14 +789,14 @@ static void tcg_context_init(unsigned max_cpus) nargs =DIV_ROUND_UP (nargs,3 ); ca =g_malloc0 (sizeof (*ca) + nargs *sizeof (ffi_type *)); - ca->cif.rtype = typecode_to_ffi[typemask &7 ]; + ca->cif.rtype =typecode_to_ffi (typemask &7 ); ca->cif.nargs = nargs;if (nargs !=0 ) { ca->cif.arg_types = ca->args;for (int j =0 ; j < nargs; ++j) {int typecode =extract32 (typemask, (j +1 ) *3 ,3 ); - ca->args[j] = typecode_to_ffi[typecode]; + ca->args[j] =typecode_to_ffi (typecode); } }
Signature · 1 line
2.34.1
PATCH v4 24/27 tcg: Factor init_ffi_layouts() out of tcg_context_init()
In reply to Richard HendersonFrom: Philippe Mathieu-Daudé <[email protected]>
Message-Id: <[email protected]>
Message-Id: <[email protected]>
--- tcg/tcg.c | 83 +++++++++++++++++++++++++++++-------------------------- 1 file changed, 44 insertions(+), 39 deletions(-)
tcg/tcg.c +44 -39
@@ -573,7 +573,49 @@ static ffi_type *typecode_to_ffi(int argmask) }g_assert_not_reached (); } -#endif + +static void init_ffi_layouts (void ) +{ +/* g_direct_hash/equal for direct comparisons on uint32_t. */ + ffi_table =g_hash_table_new (NULL ,NULL ); +for (int i =0 ; i <ARRAY_SIZE (all_helpers); ++i) { +uint32_t typemask = all_helpers[i].typemask; +gpointer hash = (gpointer )(uintptr_t )typemask; +struct { +ffi_cif cif; +ffi_type *args[]; + } *ca; +ffi_status status; +int nargs; + +if (g_hash_table_lookup (ffi_table, hash)) { +continue ; + } + +/* Ignoring the return type, find the last non-zero field. */ + nargs =32 -clz32 (typemask >>3 ); + nargs =DIV_ROUND_UP (nargs,3 ); + + ca =g_malloc0 (sizeof (*ca) + nargs *sizeof (ffi_type *)); + ca->cif.rtype =typecode_to_ffi (typemask &7 ); + ca->cif.nargs = nargs; + +if (nargs !=0 ) { + ca->cif.arg_types = ca->args; +for (int j =0 ; j < nargs; ++j) { +int typecode =extract32 (typemask, (j +1 ) *3 ,3 ); + ca->args[j] =typecode_to_ffi (typecode); + } + } + + status =ffi_prep_cif (&ca->cif,FFI_DEFAULT_ABI , nargs, + ca->cif.rtype, ca->cif.arg_types); +assert (status ==FFI_OK ); + +g_hash_table_insert (ffi_table, hash, (gpointer)&ca->cif); + } +} +#endif /* CONFIG_TCG_INTERPRETER */ typedef struct TCGCumulativeArgs { int arg_idx; /* tcg_gen_callN args[] */
@@ -768,44 +810,7 @@ static void tcg_context_init(unsigned max_cpus) }#ifdef CONFIG_TCG_INTERPRETER -/* g_direct_hash/equal for direct comparisons on uint32_t. */ - ffi_table =g_hash_table_new (NULL ,NULL ); -for (i =0 ; i <ARRAY_SIZE (all_helpers); ++i) { -struct { -ffi_cif cif; -ffi_type *args[]; - } *ca; -uint32_t typemask = all_helpers[i].typemask; -gpointer hash = (gpointer )(uintptr_t )typemask; -ffi_status status; -int nargs; - -if (g_hash_table_lookup (ffi_table, hash)) { -continue ; - } - -/* Ignoring the return type, find the last non-zero field. */ - nargs =32 -clz32 (typemask >>3 ); - nargs =DIV_ROUND_UP (nargs,3 ); - - ca =g_malloc0 (sizeof (*ca) + nargs *sizeof (ffi_type *)); - ca->cif.rtype =typecode_to_ffi (typemask &7 ); - ca->cif.nargs = nargs; - -if (nargs !=0 ) { - ca->cif.arg_types = ca->args; -for (int j =0 ; j < nargs; ++j) { -int typecode =extract32 (typemask, (j +1 ) *3 ,3 ); - ca->args[j] =typecode_to_ffi (typecode); - } - } - - status =ffi_prep_cif (&ca->cif,FFI_DEFAULT_ABI , nargs, - ca->cif.rtype, ca->cif.arg_types); -assert (status ==FFI_OK ); - -g_hash_table_insert (ffi_table, hash, (gpointer)&ca->cif); - } +init_ffi_layouts ();#endif tcg_target_init (s);
Signature · 1 line
2.34.1
PATCH v4 25/27 tcg: Move ffi_cif pointer into TCGHelperInfo
In reply to Richard HendersonInstead of requiring a separate hash table lookup, put a pointer to the CIF into TCGHelperInfo.
Message-Id: <[email protected]>
Message-Id: <[email protected]>
--- tcg/tcg-internal.h | 7 +++++++ tcg/tcg.c | 30 ++++++++++++++---------------- 2 files changed, 21 insertions(+), 16 deletions(-)
tcg/tcg-internal.h +7 -0
@@ -25,6 +25,10 @@#ifndef TCG_INTERNAL_H #define TCG_INTERNAL_H +#ifdef CONFIG_TCG_INTERPRETER +#include <ffi.h> +#endif +#define TCG_HIGHWATER 1024/*
@@ -57,6 +61,9 @@ typedef struct TCGCallArgumentLoc { typedef struct TCGHelperInfo { void *func; const char *name; +#ifdef CONFIG_TCG_INTERPRETER + ffi_cif *cif; +#endif unsigned typemask : 32; unsigned flags : 8; unsigned nr_in : 8;
tcg/tcg.c +14 -16
@@ -62,10 +62,6 @@#include "tcg/tcg-ldst.h" #include "tcg-internal.h" -#ifdef CONFIG_TCG_INTERPRETER -#include <ffi.h> -#endif -/* Forward declarations for functions declared in tcg-target.c.inc and used here. */ static void tcg_target_init (TCGContext *s);
@@ -553,8 +549,6 @@ static TCGHelperInfo all_helpers[] = {static GHashTable *helper_table;#ifdef CONFIG_TCG_INTERPRETER -static GHashTable *ffi_table; -static ffi_type *typecode_to_ffi (int argmask) {switch (argmask) {
@@ -577,9 +571,11 @@ static ffi_type *typecode_to_ffi(int argmask)static void init_ffi_layouts (void ) {/* g_direct_hash/equal for direct comparisons on uint32_t. */ - ffi_table =g_hash_table_new (NULL ,NULL ); +GHashTable *ffi_table =g_hash_table_new (NULL ,NULL ); +for (int i =0 ; i <ARRAY_SIZE (all_helpers); ++i) { -uint32_t typemask = all_helpers[i].typemask; +TCGHelperInfo *info = &all_helpers[i]; +unsigned typemask = info->typemask;gpointer hash = (gpointer )(uintptr_t )typemask; struct { ffi_cif cif;
@@ -587,8 +583,11 @@ static void init_ffi_layouts(void) } *ca;ffi_status status;int nargs; +ffi_cif *cif; -if (g_hash_table_lookup (ffi_table, hash)) { + cif =g_hash_table_lookup (ffi_table, hash); +if (cif) { + info->cif = cif;continue ; }
@@ -612,8 +611,12 @@ static void init_ffi_layouts(void) ca->cif.rtype, ca->cif.arg_types);assert (status ==FFI_OK ); - g_hash_table_insert(ffi_table, hash, (gpointer)&ca->cif); + cif = &ca->cif; + info->cif = cif; + g_hash_table_insert(ffi_table, hash, (gpointer)cif); } + +g_hash_table_destroy (ffi_table); }#endif /* CONFIG_TCG_INTERPRETER */
@@ -4385,12 +4388,7 @@ static void tcg_reg_alloc_call(TCGContext *s, TCGOp *op) }#ifdef CONFIG_TCG_INTERPRETER - { -gpointer hash = (gpointer )(uintptr_t )info->typemask; -ffi_cif *cif =g_hash_table_lookup (ffi_table, hash); -assert (cif !=NULL ); -tcg_out_call (s,tcg_call_func (op), cif); - } +tcg_out_call (s,tcg_call_func (op), info->cif);#else tcg_out_call (s,tcg_call_func (op));#endif
Signature · 1 line
2.34.1
PATCH v4 26/27 tcg/aarch64: Merge tcg_out_callr into tcg_out_call
In reply to Richard HendersonThere is only one use, and BLR is perhaps even more self-documentary than CALLR.
--- tcg/aarch64/tcg-target.c.inc | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-)
tcg/aarch64/tcg-target.c.inc +1 -6
@@ -1336,11 +1336,6 @@ static void tcg_out_goto_long(TCGContext *s, const tcg_insn_unit *target) } } -static inline void tcg_out_callr(TCGContext *s, TCGReg reg) -{ - tcg_out_insn(s, 3207, BLR, reg); -} - static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target) { ptrdiff_t offset = tcg_pcrel_diff(s, target) >> 2;
@@ -1348,7 +1343,7 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target) tcg_out_insn(s, 3206, BL, offset); } else { tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_TMP, (intptr_t)target); - tcg_out_callr(s, TCG_REG_TMP); + tcg_out_insn(s, 3207, BLR, TCG_REG_TMP); } }
Signature · 1 line
2.34.1
Re: [PATCH v4 26/27] tcg/aarch64: Merge tcg_out_callr into tcg_out_call
Quoted 7 lines · Richard Henderson
There is only one use, and BLR is perhaps even more self-documentary than CALLR.
Signed-off-by Richard Henderson <[email protected]>--- tcg/aarch64/tcg-target.c.inc | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-)
PATCH v4 27/27 tcg: Add TCGHelperInfo argument to tcg_out_call
In reply to Richard HendersonThis eliminates an ifdef for TCI, and will be required for expanding the call for TCGv_i128.
--- tcg/tcg.c | 12 ++---------- tcg/aarch64/tcg-target.c.inc | 12 +++++++++--- tcg/arm/tcg-target.c.inc | 10 ++++++++-- tcg/i386/tcg-target.c.inc | 5 +++-- tcg/loongarch64/tcg-target.c.inc | 7 ++++--- tcg/mips/tcg-target.c.inc | 3 ++- tcg/ppc/tcg-target.c.inc | 7 ++++--- tcg/riscv/tcg-target.c.inc | 7 ++++--- tcg/s390x/tcg-target.c.inc | 12 +++++++++--- tcg/sparc64/tcg-target.c.inc | 3 ++- tcg/tci/tcg-target.c.inc | 3 ++- 11 files changed, 49 insertions(+), 32 deletions(-)
tcg/tcg.c +2 -10
@@ -145,12 +145,8 @@ static void tcg_out_st(TCGContext *s, TCGType type, TCGReg arg, TCGReg arg1, intptr_t arg2);static bool tcg_out_sti (TCGContext *s,TCGType type,TCGArg val,TCGReg base,intptr_t ofs); -#ifdef CONFIG_TCG_INTERPRETER static void tcg_out_call (TCGContext *s,const tcg_insn_unit *target, -ffi_cif *cif); -#else -static void tcg_out_call (TCGContext *s,const tcg_insn_unit *target); -#endif +const TCGHelperInfo *info);static bool tcg_target_const_match (int64_t val,TCGType type,int ct);#ifdef TCG_TARGET_NEED_LDST_LABELS static int tcg_out_ldst_finalize (TCGContext *s);
@@ -4387,11 +4383,7 @@ static void tcg_reg_alloc_call(TCGContext *s, TCGOp *op)save_globals (s, allocated_regs); } -#ifdef CONFIG_TCG_INTERPRETER -tcg_out_call (s,tcg_call_func (op), info->cif); -#else -tcg_out_call (s,tcg_call_func (op)); -#endif +tcg_out_call (s,tcg_call_func (op), info);/* Assign output registers and emit moves if needed. */ switch (info->out_kind) {
tcg/aarch64/tcg-target.c.inc +9 -3
@@ -1336,7 +1336,7 @@ static void tcg_out_goto_long(TCGContext *s, const tcg_insn_unit *target) } } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target) +static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *target) { ptrdiff_t offset = tcg_pcrel_diff(s, target) >> 2; if (offset == sextract64(offset, 0, 26)) {
@@ -1347,6 +1347,12 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target) } } +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target, + const TCGHelperInfo *info) +{ + tcg_out_call_int(s, target); +} + void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx, uintptr_t jmp_rw, uintptr_t addr) {
@@ -1594,7 +1600,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) tcg_out_mov(s, TARGET_LONG_BITS == 64, TCG_REG_X1, lb->addrlo_reg); tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_X2, oi); tcg_out_adr(s, TCG_REG_X3, lb->raddr); - tcg_out_call(s, qemu_ld_helpers[opc & MO_SIZE]); + tcg_out_call_int(s, qemu_ld_helpers[opc & MO_SIZE]); if (opc & MO_SIGN) { tcg_out_sxt(s, lb->type, size, lb->datalo_reg, TCG_REG_X0); } else {
@@ -1620,7 +1626,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) tcg_out_mov(s, size == MO_64, TCG_REG_X2, lb->datalo_reg); tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_X3, oi); tcg_out_adr(s, TCG_REG_X4, lb->raddr); - tcg_out_call(s, qemu_st_helpers[opc & MO_SIZE]); + tcg_out_call_int(s, qemu_st_helpers[opc & MO_SIZE]); tcg_out_goto(s, lb->raddr); return true; }
tcg/arm/tcg-target.c.inc +8 -2
@@ -1131,7 +1131,7 @@ static void tcg_out_goto(TCGContext *s, ARMCond cond, const tcg_insn_unit *addr) * The call case is mostly used for helpers - so it's not unreasonable * for them to be beyond branch range. */ -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *addr) +static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *addr) { intptr_t addri = (intptr_t)addr; ptrdiff_t disp = tcg_pcrel_diff(s, addr);
@@ -1150,6 +1150,12 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *addr) tcg_out_blx_reg(s, COND_AL, TCG_REG_TMP); } +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *addr, + const TCGHelperInfo *info) +{ + tcg_out_call_int(s, addr); +} + static void tcg_out_goto_label(TCGContext *s, ARMCond cond, TCGLabel *l) { if (l->has_value) {
@@ -1515,7 +1521,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) argreg = tcg_out_arg_reg32(s, argreg, TCG_REG_R14); /* Use the canonical unsigned helpers and minimize icache usage. */ - tcg_out_call(s, qemu_ld_helpers[opc & MO_SIZE]); + tcg_out_call_int(s, qemu_ld_helpers[opc & MO_SIZE]); datalo = lb->datalo_reg; datahi = lb->datahi_reg;
tcg/i386/tcg-target.c.inc +3 -2
@@ -1661,7 +1661,8 @@ static void tcg_out_branch(TCGContext *s, int call, const tcg_insn_unit *dest) } } -static inline void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest) +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest, + const TCGHelperInfo *info) { tcg_out_branch(s, 1, dest); }
@@ -1885,7 +1886,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) (uintptr_t)l->raddr); } - tcg_out_call(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_branch(s, 1, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); data_reg = l->datalo_reg; switch (opc & MO_SSIZE) {
tcg/loongarch64/tcg-target.c.inc +4 -3
@@ -567,7 +567,8 @@ static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *arg, bool tail) } } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg) +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg, + const TCGHelperInfo *info) { tcg_out_call_int(s, arg, false); }
@@ -760,7 +761,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_A2, oi); tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_A3, (tcg_target_long)l->raddr); - tcg_out_call(s, qemu_ld_helpers[size]); + tcg_out_call_int(s, qemu_ld_helpers[size], false); switch (opc & MO_SSIZE) { case MO_SB:
@@ -821,7 +822,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_A3, oi); tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_A4, (tcg_target_long)l->raddr); - tcg_out_call(s, qemu_st_helpers[size]); + tcg_out_call_int(s, qemu_st_helpers[size], false); return tcg_out_goto(s, l->raddr); }
tcg/mips/tcg-target.c.inc +2 -1
@@ -1020,7 +1020,8 @@ static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *arg, bool tail) } } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg) +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg, + const TCGHelperInfo *info) { tcg_out_call_int(s, arg, false); tcg_out_nop(s);
tcg/ppc/tcg-target.c.inc +4 -3
@@ -2002,7 +2002,8 @@ static void tcg_out_call_int(TCGContext *s, int lk, #endif } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target) +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target, + const TCGHelperInfo *info) { tcg_out_call_int(s, LK, target); }
@@ -2221,7 +2222,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) tcg_out_movi(s, TCG_TYPE_I32, arg++, oi); tcg_out32(s, MFSPR | RT(arg) | LR); - tcg_out_call(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_call_int(s, LK, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); lo = lb->datalo_reg; hi = lb->datahi_reg;
@@ -2290,7 +2291,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) tcg_out_movi(s, TCG_TYPE_I32, arg++, oi); tcg_out32(s, MFSPR | RT(arg) | LR); - tcg_out_call(s, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_call_int(s, LK, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); tcg_out_b(s, 0, lb->raddr); return true;
tcg/riscv/tcg-target.c.inc +4 -3
@@ -819,7 +819,8 @@ static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *arg, bool tail) } } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg) +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *arg, + const TCGHelperInfo *info) { tcg_out_call_int(s, arg, false); }
@@ -1002,7 +1003,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) tcg_out_movi(s, TCG_TYPE_PTR, a2, oi); tcg_out_movi(s, TCG_TYPE_PTR, a3, (tcg_target_long)l->raddr); - tcg_out_call(s, qemu_ld_helpers[opc & MO_SSIZE]); + tcg_out_call_int(s, qemu_ld_helpers[opc & MO_SSIZE], false); tcg_out_mov(s, (opc & MO_SIZE) == MO_64, l->datalo_reg, a0); tcg_out_goto(s, l->raddr);
@@ -1047,7 +1048,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) tcg_out_movi(s, TCG_TYPE_PTR, a3, oi); tcg_out_movi(s, TCG_TYPE_PTR, a4, (tcg_target_long)l->raddr); - tcg_out_call(s, qemu_st_helpers[opc & MO_SIZE]); + tcg_out_call_int(s, qemu_st_helpers[opc & MO_SIZE], false); tcg_out_goto(s, l->raddr); return true;
tcg/s390x/tcg-target.c.inc +9 -3
@@ -1691,7 +1691,7 @@ static void tgen_brcond(TCGContext *s, TCGType type, TCGCond c, tgen_branch(s, cc, l); } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest) +static void tcg_out_call_int(TCGContext *s, const tcg_insn_unit *dest) { ptrdiff_t off = tcg_pcrel_diff(s, dest) >> 1; if (off == (int32_t)off) {
@@ -1702,6 +1702,12 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest) } } +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest, + const TCGHelperInfo *info) +{ + tcg_out_call_int(s, dest); +} + static void tcg_out_qemu_ld_direct(TCGContext *s, MemOp opc, TCGReg data, TCGReg base, TCGReg index, int disp) {
@@ -1897,7 +1903,7 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) } tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_R4, oi); tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R5, (uintptr_t)lb->raddr); - tcg_out_call(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SSIZE)]); + tcg_out_call_int(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SSIZE)]); tcg_out_mov(s, TCG_TYPE_I64, data_reg, TCG_REG_R2); tgen_gotoi(s, S390_CC_ALWAYS, lb->raddr);
@@ -1938,7 +1944,7 @@ static bool tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) } tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_R5, oi); tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R6, (uintptr_t)lb->raddr); - tcg_out_call(s, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_call_int(s, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); tgen_gotoi(s, S390_CC_ALWAYS, lb->raddr); return true;
tcg/sparc64/tcg-target.c.inc +2 -1
@@ -859,7 +859,8 @@ static void tcg_out_call_nodelay(TCGContext *s, const tcg_insn_unit *dest, } } -static void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest) +static void tcg_out_call(TCGContext *s, const tcg_insn_unit *dest, + const TCGHelperInfo *info) { tcg_out_call_nodelay(s, dest, false); tcg_out_nop(s);
tcg/tci/tcg-target.c.inc +2 -1
@@ -558,8 +558,9 @@ static void tcg_out_movi(TCGContext *s, TCGType type, } static void tcg_out_call(TCGContext *s, const tcg_insn_unit *func, - ffi_cif *cif) + const TCGHelperInfo *info) { + ffi_cif *cif = info->cif; tcg_insn_unit insn = 0; uint8_t which;
Signature · 1 line
2.34.1
Ping.
r~
Quoted 91 lines · Richard Henderson
For v4, I've sorted some of the reviewed patches to the beginning of the queue and omitted them; this was not possible for all. This set begins with the first patch that is missing review. I've not tested this set separately, so in case it doesn't apply, the entire tree is
https://gitlab.com/rth7680/qemu/-/tree/tcg-nextPatches lacking review:
01-tcg-Fix-tcg_reg_alloc_dup.patch 02-tcg-Centralize-updates-to-reg_to_temp.patch 03-tcg-Remove-check_regs.patch 04-tcg-Introduce-paired-register-allocation.patch 05-accel-tcg-Set-cflags_next_tb-in-cpu_common_initfn.patch 06-target-sparc-Avoid-TCGV_-LOW-HIGH.patch 09-tcg-Simplify-calls-to-temp_sync-vs-mem_coherent.patch 17-accel-tcg-plugin-Don-t-search-for-the-function-po.patch 20-tcg-Vary-the-allocation-size-for-TCGOp.patch 22-tcg-Reorg-function-calls.patch 23-tcg-Convert-typecode_to_ffi-from-array-to-functio.patch 24-tcg-Factor-init_ffi_layouts-out-of-tcg_context_in.patch 25-tcg-Move-ffi_cif-pointer-into-TCGHelperInfo.patch 26-tcg-aarch64-Merge-tcg_out_callr-into-tcg_out_call.patchr~
Philippe Mathieu-Daudé (2):
tcg: Converttypecode_to_ffifrom array to function
tcg: Factorinit_ffi_layouts()out oftcg_context_init()Richard Henderson (25):
tcg: Fix tcg_reg_alloc_dup*
tcg: Centralize updates toreg_to_temp
tcg: Removecheck_regs
tcg: Introduce paired register allocation
accel/tcg: Setcflags_next_tbincpu_common_initfntarget/sparc: Avoid TCGV_{LOW,HIGH}
tcg: Move TCG_{LOW,HIGH} totcg-internal.h
tcg: Addtemp_subindexto TCGTemp
tcg: Simplify calls totemp_syncvsmem_coherent
tcg: Allocate TCGTemp pairs in host memory order
tcg: MoveTCG_TYPE_COUNToutside enum
tcg: Introducetcg_type_size
tcg: Introduce TCGCallReturnKind and TCGCallArgumentKind
tcg: ReplaceTCG_TARGET_CALL_ALIGN_ARGSwithTCG_TARGET_CALL_ARG_I64
tcg: ReplaceTCG_TARGET_EXTEND_ARGSwithTCG_TARGET_CALL_ARG_I32
tcg: UseTCG_CALL_ARG_EVENfor TCI special case accel/tcg/plugin: Don't search for the function pointer index accel/tcg/plugin: Avoid duplicate copy incopy_callaccel/tcg/plugin: Usecopy_opin append_{udata,mem}_cb
tcg: Vary the allocation size for TCGOp
tcg: Useoutput_prefwrapper function
tcg: Reorg function calls
tcg: Moveffi_cifpointer into TCGHelperInfo
tcg/aarch64: Mergetcg_out_callrintotcg_out_call
tcg: Add TCGHelperInfo argument totcg_out_callinclude/exec/helper-head.h | 2 +- include/tcg/tcg-op.h | 35 +- include/tcg/tcg.h | 96 +- tcg/aarch64/tcg-target.h | 3 +- tcg/arm/tcg-target.h | 3 +- tcg/i386/tcg-target.h | 2 + tcg/loongarch64/tcg-target.h | 3 +- tcg/mips/tcg-target.h | 4 +- tcg/riscv/tcg-target.h | 7 +- tcg/s390x/tcg-target.h | 3 +- tcg/sparc64/tcg-target.h | 3 +- tcg/tcg-internal.h | 58 +- tcg/tci/tcg-target.h | 7 + accel/tcg/plugin-gen.c | 54 +- hw/core/cpu-common.c | 1 + target/sparc/translate.c | 21 +- tcg/optimize.c | 10 +- tcg/tcg-op-vec.c | 10 +- tcg/tcg-op.c | 49 +- tcg/tcg.c | 1591 +++++++++++++++++++----------- tcg/aarch64/tcg-target.c.inc | 19 +- tcg/arm/tcg-target.c.inc | 10 +- tcg/i386/tcg-target.c.inc | 5 +- tcg/loongarch64/tcg-target.c.inc | 7 +- tcg/mips/tcg-target.c.inc | 3 +- tcg/ppc/tcg-target.c.inc | 34 +- tcg/riscv/tcg-target.c.inc | 7 +- tcg/s390x/tcg-target.c.inc | 12 +- tcg/sparc64/tcg-target.c.inc | 3 +- tcg/tci/tcg-target.c.inc | 3 +- 30 files changed, 1285 insertions(+), 780 deletions(-)