Introduce helper-to-tcg
v2 cover Introduce helper-to-tcg
Hi all, this patchset introduces helper-to-tcg, a LLVM based build-time C to TCG translator, as a QEMU subproject. The purpose of this tool is to simplify implementation of instructions in TCG by automatically translating helper functions for a given target to TCG. It may also be used as a standalone tool for getting a base TCG implementation for complicated instructions.
See KVM forum 2023 and 2025 presentations which give an overview of its application to Hexagon and RISCV:
- https://www.youtube.com/watch?v=Gwz0kp7IZPE
- https://www.youtube.com/watch?v=7qIwraM3-4I
helper-to-tcg is also applied to the Hexagon frontend, managing to translate 1280 instructions, 162 of which are HVX instructions. In the current patchset we translate 1240 instructions due to limiting emission of the heavier vectorized instructions which requires larger TBs and more vector temporaries. For the time being, idef-parser remains translating 281 instructions consisting mostly of complicated load instructions. This count will be reduced over time until idef-parser can be deprecated.
As an example, consider the following helper function implementation of a Hexagon instruction for performing a 2-element scalar product, using signed saturated arithmetic
void HELPER(V6_vdmpyhvsat)(CPUHexagonState *env,
void * restrict VdV_void,
void * restrict VuV_void,
void * restrict VvV_void)
{
fVFOREACH(32, i) {
size8s_t accum = fMPY16SS(fGETHALF(0,VuV.w[i]),fGETHALF(0, VvV.w[i]));
accum += fMPY16SS(fGETHALF(1,VuV.w[i]),fGETHALF(1, VvV.w[i]));
VdV.w[i] = fVSATW(accum);
}
}which at the end of the helper-to-tcg pipeline will have been converted to the following LLVM IR
define void @helper_V6_vdmpyhvsat(%struct.CPUArchState* %0,
i8* %1, i8* %2, i8* %3) {
%4 = bitcast i8* %2 to <32 x i32>*
%wide.load = load <32 x i32>, <32 x i32>* %4
%5 = call <32 x i32> @VecShlScalar(<32 x i32> %wide.load, i32 16)
%6 = call <32 x i32> @VecAShrScalar(<32 x i32> %5, i32 16)
%7 = bitcast i8* %3 to <32 x i32>*
%wide.load23 = load <32 x i32>, <32 x i32>* %7
%8 = call <32 x i32> @VecShlScalar(<32 x i32> %wide.load23, i32 16)
%9 = call <32 x i32> @VecAShrScalar(<32 x i32> %8, i32 16)
%10 = mul nsw <32 x i32> %9, %6
%11 = call <32 x i32> @VecAShrScalar(<32 x i32> %wide.load, i32 16)
%12 = call <32 x i32> @VecAShrScalar(<32 x i32> %wide.load23, i32 16)
%13 = mul nsw <32 x i32> %12, %11
%14 = bitcast i8* %1 to <32 x i32>*
ret void
}which, in TCG, gets emitted as
void emit_V6_vdmpyhvsat(TCGv_env env, intptr_t vec3,
intptr_t vec7, intptr_t vec6) {
VectorMem mem = {0};
intptr_t vec0 = temp_new_gvec(&mem, 128);
tcg_gen_gvec_shli(MO_32, vec0, vec7, 16, 128, 128);
intptr_t vec5 = temp_new_gvec(&mem, 128);
tcg_gen_gvec_sari(MO_32, vec5, vec0, 16, 128, 128);
intptr_t vec1 = temp_new_gvec(&mem, 128);
tcg_gen_gvec_shli(MO_32, vec1, vec6, 16, 128, 128);
tcg_gen_gvec_sari(MO_32, vec1, vec1, 16, 128, 128);
tcg_gen_gvec_mul(MO_32, vec1, vec1, vec5, 128, 128);
intptr_t vec2 = temp_new_gvec(&mem, 128);
tcg_gen_gvec_sari(MO_32, vec2, vec7, 16, 128, 128);
tcg_gen_gvec_sari(MO_32, vec0, vec6, 16, 128, 128);
tcg_gen_gvec_mul(MO_32, vec2, vec0, vec2, 128, 128);
tcg_gen_gvec_ssadd(MO_32, vec3, vec1, vec2, 128, 128);
}consisting of a few vectorized shifts, multiplications, and a signed saturated add.
For a more in-depth usage guide see subprojects/helper-to-tcg/README.md.
Limitations:
- Does not handle functions with multiple return values. On Hexagon, a large set of instructions still translated by idef-parser fall into this category.
- Can emit complicated vector operations that are not the most host-friendly, identifying and emitting custom GVec patterns would be interesting.
Patchset overview:
1. helper-to-tcg (patches 6-40) - Introduces the actual translator as a QEMU subproject.
2. Fills gaps in TCG instructions (patches 1,2,5) - Since the tool is LLVM based it allows for translation of vector instructions to gvec instructions in Tinycode. This requires the introduction of a few new tcg_gen_gvec_*() functions for dealing with sign- and zero-extension, along with a function for initializing a vector to a constant, and functions for bitreversal and funnel shift.
3. Mapping of cpu state (patch 3) - helper-to-tcg needs information about the offsets of fields in the cpu state that correspond to TCG globals, so these can be emitted in the output code. For this purpose, a target may define an array of struct cpu_tcg_mapping to map fields in the cpu state to TCG globals in a declarative way. This global array can be parsed by helper-to-tcg, and replaces manually calling tcg_global_mem_new*() in frontend.
4. Increases max size of generated TB code (patch 4) - Due to the power of the LLVM auto-vectorizer, helper-to-tcg can emit quite complicated vectorized gvec code. Particularly for Hexagon where a single instruction packet can consist of multiple vector instructions. A single instruction packet can in rare cases exceed the TB buffer size of 128 longs.
5. Applies helper-to-tcg to Hexagon (patches 41-50) - helper-to-tcg is used on the Hexagon frontend to translate a majority of helper functions in place of idef-parser. For the time being idef-parser will remain in use to translate instructions with multiple return values that are not representable as helper functions and therefore translatable with helper-to-tcg.
Changes in v2:
- Use QEMU_ANNOTATE over introducing LLVM_ANNOTATE (Philippe);
- Move .h -> .hpp to avoid checkpatch (Philppe);
- Meson changes (Paolo);
- Gvec size changing operations (zext/sext/trunc) along with constant vector creation has been moved from general gvec operations to being emitted by helper-to-tcg on demand. A simple interface for specifying the target vector layout is available through commandline options (Richard);
- Constant vectors initialized from arrays now vectorized;
- Introduced a model of target vector layouts;
- Shifted LLVM version support from 10-14 to 15-21;
- Improved end-to-end testing, now verifies output across LLVM versions, currently these tests along with their expected output serve as usage examples;
- Added docker tests for supported LLVM versions;
- Dropped the tcg_gen_callN dispatcher in favour of emitting gen_helper_*()
definitions directly;
- Optionally allow calls to declarations, custom jump instructions, and propagation of
DisasContext to better slot into existing frontends;
- Limit vector usage for hexagon to 8 temporaries and 16 gvec
instructions, also use existing `tmp_VRegs[]` memory.- Many cleanups and bugfixes.
Anton Johansson (50):
accel/tcg: Add bitreverse and funnel-shift runtime helper functions accel/tcg: Add getpc helper
tcg: Introduce tcg-global-mappings
tcg: Increase maximum TB size
tcg: Expose tcg_gen_ussub_sat()
Add helper-to-tcg subproject
helper-to-tcg: Introduce get-llvm-ir.py
helper-to-tcg: Handle LLVM version compatibility
helper-to-tcg: Introduce custom LLVM pipeline
helper-to-tcg: Add pipeline --debug and --debug-only
helper-to-tcg: Add simple error creation helper
helper-to-tcg: Introduce PrepareForOptPass
helper-to-tcg: PrepareForOptPass, demangle function names
helper-to-tcg: PrepareForOptPass, map annotations
helper-to-tcg: PrepareForOptPass, cull unused functions
helper-to-tcg: PrepareForOptPass, undef llvm.returnaddress
helper-to-tcg: PrepareForOptPass, fixup inline attributes
helper-to-tcg: PrepareForOptPass, collect debuginfo
helper-to-tcg: Pipeline, run optimization pass
helper-to-tcg: Introduce pseudo instructions
helper-to-tcg: Add guest vector layout description
helper-to-tcg: Introduce PrepareForTcgPass
helper-to-tcg: PrepareForTcgPass, remove functions with cycles
helper-to-tcg: PrepareForTcgPass, demote PHI nodes
helper-to-tcg: PrepareForTcgPass, map TCG globals
helper-to-tcg: PrepareForTcgPass, transform GEPs
helper-to-tcg: PrepareForTcgPass, canonicalize IR
helper-to-tcg: PrepareForTcgPass, identity map trivial expressions
helper-to-tcg: Introduce TcgV structure
helper-to-tcg: Introduce TcgGenPass
helper-to-tcg: TcgGenPass, linearize basic blocks
helper-to-tcg: TcgGenPass, introduce Value <-> TcgV map
helper-to-tcg: TcgGenPass, add structs for string emission
helper-to-tcg: TcgGenPass, map arguments to TCG
helper-to-tcg: TcgGenPass, propagate constant expresssions
helper-to-tcg: TcgGenPass, allocate TCG registers
helper-to-tcg: TcgGenPass, emit TCG strings
helper-to-tcg: Add README
helper-to-tcg: Add end-to-end tests
test: helper-to-tcg docker tests
target/hexagon: Add get_tb_mmu_index()
target/hexagon: Increase VECTOR_TEMPS_MAX target/hexagon: Provide env to tcg global mapping target/hexagon: Keep gen_slotval/check_noshuf for helper-to-tcg target/hexagon: Emit annotations for helpers target/hexagon: Split probe_and_commit helper target/hexagon: Use helper-to-tcg helper calls target/hexagon: Manually call generated HVX instructions target/hexagon: Use idef-parser as a fallback target/hexagon: Use helper-to-tcg
accel/tcg/tcg-runtime.c | 33 + accel/tcg/tcg-runtime.h | 8 + configs/targets/hexagon-linux-user.mak | 1 + configs/targets/hexagon-softmmu.mak | 1 + include/tcg/tcg-global-mappings.h | 118 ++ include/tcg/tcg-op-common.h | 2 + include/tcg/tcg.h | 2 +- meson.build | 10 + meson_options.txt | 2 + scripts/meson-buildoptions.sh | 5 + subprojects/helper-to-tcg/README.md | 297 ++++ subprojects/helper-to-tcg/get-llvm-ir.py | 145 ++ .../helper-to-tcg/include/CmdLineOptions.hpp | 46 + .../helper-to-tcg/include/DebugInfo.hpp | 46 + subprojects/helper-to-tcg/include/Error.hpp | 38 + .../include/FunctionAnnotation.hpp | 104 ++ .../helper-to-tcg/include/LlvmCompat.hpp | 124 ++ .../include/PrepareForOptPass.hpp | 42 + .../include/PrepareForTcgPass.hpp | 37 + .../helper-to-tcg/include/PseudoInst.hpp | 75 + .../helper-to-tcg/include/PseudoInst.inc | 78 ++ .../helper-to-tcg/include/TcgGenPass.hpp | 63 + .../helper-to-tcg/include/TcgGlobalMap.hpp | 40 + subprojects/helper-to-tcg/include/TcgType.hpp | 276 ++++ .../helper-to-tcg/include/VectorLayout.hpp | 116 ++ subprojects/helper-to-tcg/meson.build | 86 ++ subprojects/helper-to-tcg/meson_options.txt | 2 + subprojects/helper-to-tcg/src/LlvmCompat.cpp | 89 ++ subprojects/helper-to-tcg/src/Pipeline.cpp | 399 ++++++ .../PrepareForOptPass/PrepareForOptPass.cpp | 363 +++++ .../src/PrepareForTcgPass/CanonicalizeIR.cpp | 1193 ++++++++++++++++ .../src/PrepareForTcgPass/CanonicalizeIR.hpp | 24 + .../src/PrepareForTcgPass/IdentityMap.cpp | 91 ++ .../src/PrepareForTcgPass/IdentityMap.hpp | 39 + .../PrepareForTcgPass/PrepareForTcgPass.cpp | 150 ++ .../src/PrepareForTcgPass/TransformGEPs.cpp | 355 +++++ .../src/PrepareForTcgPass/TransformGEPs.hpp | 45 + subprojects/helper-to-tcg/src/PseudoInst.cpp | 183 +++ .../src/TcgGenPass/LinearizeBlocks.cpp | 140 ++ .../src/TcgGenPass/LinearizeBlocks.hpp | 32 + .../src/TcgGenPass/MapArguments.cpp | 122 ++ .../src/TcgGenPass/MapConstantExpressions.cpp | 328 +++++ .../src/TcgGenPass/MapTcgOperations.cpp | 988 ++++++++++++++ .../src/TcgGenPass/MapTemporaries.cpp | 619 +++++++++ .../helper-to-tcg/src/TcgGenPass/TcgEmit.cpp | 1203 +++++++++++++++++ .../helper-to-tcg/src/TcgGenPass/TcgEmit.hpp | 315 +++++ .../src/TcgGenPass/TcgGenPass.cpp | 385 ++++++ .../src/TcgGenPass/ValueMapping.hpp | 110 ++ .../helper-to-tcg/tests/call-to-declaration.c | 33 + .../helper-to-tcg/tests/call-translation.c | 35 + subprojects/helper-to-tcg/tests/cpustate.c | 44 + .../helper-to-tcg/tests/forward-context.c | 21 + subprojects/helper-to-tcg/tests/ldst.c | 17 + subprojects/helper-to-tcg/tests/meson.build | 121 ++ .../tests/ref/15/call-to-declaration.c | 26 + .../tests/ref/15/call-translation.c | 28 + .../helper-to-tcg/tests/ref/15/cpustate.c | 36 + .../tests/ref/15/forward-context.c | 30 + subprojects/helper-to-tcg/tests/ref/15/ldst.c | 21 + .../helper-to-tcg/tests/ref/15/scalar.c | 21 + .../tests/ref/15/user-pcrel-jump.c | 33 + .../tests/ref/15/vector-layout-f16L.c | 527 ++++++++ .../tests/ref/15/vector-layout-f16M.c | 527 ++++++++ .../tests/ref/15/vector-layout-f32L.c | 527 ++++++++ .../tests/ref/15/vector-layout-f32M.c | 527 ++++++++ .../tests/ref/15/vector-layout-f64L.c | 527 ++++++++ .../tests/ref/15/vector-layout-f64M.c | 527 ++++++++ .../tests/ref/15/vector-layout-f8L.c | 527 ++++++++ .../tests/ref/15/vector-layout-f8M.c | 527 ++++++++ .../tests/ref/15/vector-layout-t16L.c | 527 ++++++++ .../tests/ref/15/vector-layout-t16M.c | 527 ++++++++ .../tests/ref/15/vector-layout-t32L.c | 527 ++++++++ .../tests/ref/15/vector-layout-t32M.c | 527 ++++++++ .../tests/ref/15/vector-layout-t64L.c | 527 ++++++++ .../tests/ref/15/vector-layout-t64M.c | 527 ++++++++ .../tests/ref/15/vector-layout-t8L.c | 527 ++++++++ .../tests/ref/15/vector-layout-t8M.c | 527 ++++++++ .../helper-to-tcg/tests/ref/15/vector.c | 31 + .../tests/ref/21/call-to-declaration.c | 26 + .../tests/ref/21/call-translation.c | 28 + .../helper-to-tcg/tests/ref/21/cpustate.c | 36 + .../tests/ref/21/forward-context.c | 30 + subprojects/helper-to-tcg/tests/ref/21/ldst.c | 21 + .../helper-to-tcg/tests/ref/21/scalar.c | 21 + .../tests/ref/21/user-pcrel-jump.c | 33 + .../tests/ref/21/vector-layout-f16L.c | 527 ++++++++ .../tests/ref/21/vector-layout-f16M.c | 527 ++++++++ .../tests/ref/21/vector-layout-f32L.c | 527 ++++++++ .../tests/ref/21/vector-layout-f32M.c | 527 ++++++++ .../tests/ref/21/vector-layout-f64L.c | 527 ++++++++ .../tests/ref/21/vector-layout-f64M.c | 527 ++++++++ .../tests/ref/21/vector-layout-f8L.c | 527 ++++++++ .../tests/ref/21/vector-layout-f8M.c | 527 ++++++++ .../tests/ref/21/vector-layout-t16L.c | 527 ++++++++ .../tests/ref/21/vector-layout-t16M.c | 527 ++++++++ .../tests/ref/21/vector-layout-t32L.c | 527 ++++++++ .../tests/ref/21/vector-layout-t32M.c | 527 ++++++++ .../tests/ref/21/vector-layout-t64L.c | 527 ++++++++ .../tests/ref/21/vector-layout-t64M.c | 527 ++++++++ .../tests/ref/21/vector-layout-t8L.c | 527 ++++++++ .../tests/ref/21/vector-layout-t8M.c | 527 ++++++++ .../helper-to-tcg/tests/ref/21/vector.c | 31 + subprojects/helper-to-tcg/tests/scalar.c | 15 + .../helper-to-tcg/tests/tcg-global-mappings.h | 118 ++ .../helper-to-tcg/tests/user-pcrel-jump.c | 28 + .../helper-to-tcg/tests/vector-layout.c | 51 + subprojects/helper-to-tcg/tests/vector.c | 34 + target/hexagon/cpu.h | 6 +- target/hexagon/gen_helper_funcs.py | 17 +- target/hexagon/gen_helper_protos.py | 2 +- target/hexagon/gen_idef_parser_funcs.py | 9 + target/hexagon/gen_tcg_funcs.py | 16 +- target/hexagon/genptr.c | 6 +- target/hexagon/helper.h | 9 + target/hexagon/hex_common.py | 103 +- target/hexagon/meson.build | 148 +- target/hexagon/op_helper.c | 17 +- target/hexagon/translate.c | 139 +- target/hexagon/translate.h | 4 + tcg/meson.build | 1 + tcg/tcg-global-mappings.c | 62 + tcg/tcg-op-gvec.c | 4 +- tests/docker/dockerfiles/debian-llvm.docker | 38 + tests/docker/test-helper-to-tcg | 41 + 124 files changed, 27303 insertions(+), 122 deletions(-) create mode 100644 include/tcg/tcg-global-mappings.h create mode 100644 subprojects/helper-to-tcg/README.md create mode 100755 subprojects/helper-to-tcg/get-llvm-ir.py create mode 100644 subprojects/helper-to-tcg/include/CmdLineOptions.hpp create mode 100644 subprojects/helper-to-tcg/include/DebugInfo.hpp create mode 100644 subprojects/helper-to-tcg/include/Error.hpp create mode 100644 subprojects/helper-to-tcg/include/FunctionAnnotation.hpp create mode 100644 subprojects/helper-to-tcg/include/LlvmCompat.hpp create mode 100644 subprojects/helper-to-tcg/include/PrepareForOptPass.hpp create mode 100644 subprojects/helper-to-tcg/include/PrepareForTcgPass.hpp create mode 100644 subprojects/helper-to-tcg/include/PseudoInst.hpp create mode 100644 subprojects/helper-to-tcg/include/PseudoInst.inc create mode 100644 subprojects/helper-to-tcg/include/TcgGenPass.hpp create mode 100644 subprojects/helper-to-tcg/include/TcgGlobalMap.hpp create mode 100644 subprojects/helper-to-tcg/include/TcgType.hpp create mode 100644 subprojects/helper-to-tcg/include/VectorLayout.hpp create mode 100644 subprojects/helper-to-tcg/meson.build create mode 100644 subprojects/helper-to-tcg/meson_options.txt create mode 100644 subprojects/helper-to-tcg/src/LlvmCompat.cpp create mode 100644 subprojects/helper-to-tcg/src/Pipeline.cpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/CanonicalizeIR.cpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/CanonicalizeIR.hpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/IdentityMap.cpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/IdentityMap.hpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/PrepareForTcgPass.cpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/TransformGEPs.cpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/TransformGEPs.hpp create mode 100644 subprojects/helper-to-tcg/src/PseudoInst.cpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.hpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/MapArguments.cpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/MapConstantExpressions.cpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/MapTcgOperations.cpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/MapTemporaries.cpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/TcgEmit.cpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/TcgEmit.hpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/TcgGenPass.cpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/ValueMapping.hpp create mode 100644 subprojects/helper-to-tcg/tests/call-to-declaration.c create mode 100644 subprojects/helper-to-tcg/tests/call-translation.c create mode 100644 subprojects/helper-to-tcg/tests/cpustate.c create mode 100644 subprojects/helper-to-tcg/tests/forward-context.c create mode 100644 subprojects/helper-to-tcg/tests/ldst.c create mode 100644 subprojects/helper-to-tcg/tests/meson.build create mode 100644 subprojects/helper-to-tcg/tests/ref/15/call-to-declaration.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/call-translation.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/cpustate.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/forward-context.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/ldst.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/scalar.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/user-pcrel-jump.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f16L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f16M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f32L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f32M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f64L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f64M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f8L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f8M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t16L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t16M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t32L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t32M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t64L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t64M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t8L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t8M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/call-to-declaration.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/call-translation.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/cpustate.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/forward-context.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/ldst.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/scalar.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/user-pcrel-jump.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f16L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f16M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f32L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f32M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f64L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f64M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f8L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f8M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t16L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t16M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t32L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t32M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t64L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t64M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t8L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t8M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector.c create mode 100644 subprojects/helper-to-tcg/tests/scalar.c create mode 100644 subprojects/helper-to-tcg/tests/tcg-global-mappings.h create mode 100644 subprojects/helper-to-tcg/tests/user-pcrel-jump.c create mode 100644 subprojects/helper-to-tcg/tests/vector-layout.c create mode 100644 subprojects/helper-to-tcg/tests/vector.c create mode 100644 tcg/tcg-global-mappings.c create mode 100644 tests/docker/dockerfiles/debian-llvm.docker create mode 100755 tests/docker/test-helper-to-tcg
Signature · 1 line
2.52.0
PATCH v2 1/50 accel/tcg: Add bitreverse and funnel-shift runtime helper functions
Adds necessary helper functions for mapping LLVM IR onto TCG. Specifically, helpers corresponding to the bitreverse and funnel-shift intrinsics in LLVM.
Note: these may be converted to more efficient implementations in the future, but for the time being it allows helper-to-tcg to support a wider subset of LLVM IR.
--
TODO: tcg_gen_*() variants will be added in the next version, it slipped through (Richard).
--- accel/tcg/tcg-runtime.c | 28 ++++++++++++++++++++++++++++ accel/tcg/tcg-runtime.h | 6 ++++++ 2 files changed, 34 insertions(+)
accel/tcg/tcg-runtime.c +28 -0
@@ -24,6 +24,7 @@#include "qemu/osdep.h" #include "qemu/host-utils.h" #include "exec/cpu-common.h" +#include "qemu/int128.h" #include "exec/helper-proto-common.h" #include "accel/tcg/cpu-loop.h" #include "accel/tcg/getpc.h"
@@ -54,8 +55,30 @@ uint32_t HELPER(remu_i32)(uint32_t arg1, uint32_t arg2)return arg1 % arg2; } +uint32_t HELPER (bitreverse8_i32 )(uint32_t x) +{ +return revbit8 ((uint8_t ) x); +} + +uint32_t HELPER (bitreverse16_i32 )(uint32_t x) +{ +return revbit16 ((uint16_t ) x); +} + +uint32_t HELPER (bitreverse32_i32 )(uint32_t x) +{ +return revbit32 (x); +} +/* 64-bit helpers */ +uint64_t HELPER (fshl_i64 )(uint64_t a,uint64_t b,uint64_t c) +{ +Int128 d =int128_make128 (b, a); +Int128 shift =int128_lshift (d, c); +return int128_gethi (shift); +} +int64_t HELPER (div_i64 )(int64_t arg1,int64_t arg2) {return arg1 / arg2;
@@ -76,6 +99,11 @@ uint64_t HELPER(remu_i64)(uint64_t arg1, uint64_t arg2)return arg1 % arg2; } +uint64_t HELPER (bitreverse64_i64 )(uint64_t x) +{ +return revbit64 (x); +} +uint64_t HELPER (muluh_i64 )(uint64_t arg1,uint64_t arg2) {uint64_t l, h;
accel/tcg/tcg-runtime.h +6 -0
@@ -2,11 +2,17 @@ DEF_HELPER_FLAGS_2(div_i32, TCG_CALL_NO_RWG_SE, s32, s32, s32) DEF_HELPER_FLAGS_2(rem_i32, TCG_CALL_NO_RWG_SE, s32, s32, s32) DEF_HELPER_FLAGS_2(divu_i32, TCG_CALL_NO_RWG_SE, i32, i32, i32) DEF_HELPER_FLAGS_2(remu_i32, TCG_CALL_NO_RWG_SE, i32, i32, i32) +DEF_HELPER_FLAGS_1(bitreverse8_i32, TCG_CALL_NO_RWG_SE, i32, i32) +DEF_HELPER_FLAGS_1(bitreverse16_i32, TCG_CALL_NO_RWG_SE, i32, i32) +DEF_HELPER_FLAGS_1(bitreverse32_i32, TCG_CALL_NO_RWG_SE, i32, i32) DEF_HELPER_FLAGS_2(div_i64, TCG_CALL_NO_RWG_SE, s64, s64, s64) DEF_HELPER_FLAGS_2(rem_i64, TCG_CALL_NO_RWG_SE, s64, s64, s64) DEF_HELPER_FLAGS_2(divu_i64, TCG_CALL_NO_RWG_SE, i64, i64, i64) DEF_HELPER_FLAGS_2(remu_i64, TCG_CALL_NO_RWG_SE, i64, i64, i64) +DEF_HELPER_FLAGS_1(bitreverse64_i64, TCG_CALL_NO_RWG_SE, i64, i64) + +DEF_HELPER_FLAGS_3(fshl_i64, TCG_CALL_NO_RWG_SE, i64, i64, i64, i64) DEF_HELPER_FLAGS_2(mulsh_i64, TCG_CALL_NO_RWG_SE, s64, s64, s64)DEF_HELPER_FLAGS_2 (muluh_i64,TCG_CALL_NO_RWG_SE , i64, i64, i64)
Signature · 1 line
2.52.0
Re: [PATCH v2 01/50] accel/tcg: Add bitreverse and funnel-shift runtime helper functions
Quoted 22 lines · Anton Johansson via qemu development
Adds necessary helper functions for mapping LLVM IR onto TCG. Specifically, helpers corresponding to the bitreverse and funnel-shift intrinsics in LLVM.
Note: these may be converted to more efficient implementations in the future, but for the time being it allows helper-to-tcg to support a wider subset of LLVM IR.
Signed-off-by Anton Johansson <[email protected]>--
TODO:tcg_gen_*()variants will be added in the next version, it slipped through (Richard).--- accel/tcg/tcg-runtime.c | 28 ++++++++++++++++++++++++++++ accel/tcg/tcg-runtime.h | 6 ++++++ 2 files changed, 34 insertions(+)accel/tcg/tcg-runtime.c +4
-0+uint32_t HELPER (bitreverse8_i32 )(uint32_t x) +{ +return revbit8 ((uint8_t ) x); +}
accel/tcg/tcg-runtime.c uint32_t HELPER(remu_i32)(uint32_t arg1, uint32_t arg2) · line 71
Quoted 10 lines
accel/tcg/tcg-runtime.c +10
-0+ +uint32_t HELPER (bitreverse16_i32 )(uint32_t x) +{ +return revbit16 ((uint16_t ) x); +} + +uint32_t HELPER (bitreverse32_i32 )(uint32_t x) +{ +return revbit32 (x); +}
Please split in 2 patches, adding bitreverse family in one, and fshl_i64 in another.
For bitreverse:
Quoted 3 lines
/* 64-bit helpers */
+uint64_t HELPER(fshl_i64)(uint64_t a,
uint64_tb,uint64_tc)
Please use clearer hi/lo/sh/val/ret instead of a/b/c/d/shift:
HELPER(fshl_i64)(uint64_t hi, uint64_t lo, uint64_t sh)
Quoted 5 lines
accel/tcg/tcg-runtime.c +5
-0+{ +Int128 d =int128_make128 (b, a); +Int128 shift =int128_lshift (d, c); +return int128_gethi (shift); +}
Re: [PATCH v2 01/50] accel/tcg: Add bitreverse and funnel-shift runtime helper functions
In reply to Anton Johansson via qemu developmentaccel/tcg/tcg-runtime.c uint32_t HELPER(remu_i32)(uint32_t arg1, uint32_t arg2) · line 80
Quoted 6 lines · Anton Johansson via qemu development
accel/tcg/tcg-runtime.c +6
-0+uint64_t HELPER (fshl_i64 )(uint64_t a,uint64_t b,uint64_t c) +{ +Int128 d =int128_make128 (b, a); +Int128 shift =int128_lshift (d, c); +return int128_gethi (shift); +}
The existing similar opcode is extract2, performing a constant right double-word shift. I'm open to adding a variable version, and adding left-shift, but we shouldn't have two names for the same operation.
Plus Phil's comment about not doing two things in the same patch.
r~
PATCH v2 2/50 accel/tcg: Add getpc helper
In reply to Anton Johansson via qemu developmentIntroduces a helper function to return the current pc (returnaddress of the helper), this allows helper-to-tcg to correctly translate nested helper functions that checks for faulting memory operations.
This is useful as an optimization where a helper functions has a commonly taken fast path that doesn't fault e.g.
void HELPER(outer)(...)
{
... // non faulting operations
if (some_uncommon_condition) {
helper_inner(..., GETPC())
}
}the outer helper along with the condition can then be emitted as TCG, and helper_inner() will be emitted as a gen_helper_inner(, ra), where ra is the result of gen_helper_getpc().
NOTE: This is not ideal since we're introducing extra helper calls,
and the solution doesn't deal with the "inner" function not being a helper. A better solution and what we'll probably do in the next version of the patchset is to instead emit a helper definition for the "inner" function that uses GETPC().
--- accel/tcg/tcg-runtime.c | 5 +++++ accel/tcg/tcg-runtime.h | 2 ++ 2 files changed, 7 insertions(+)
accel/tcg/tcg-runtime.c +5 -0
@@ -162,3 +162,8 @@ void HELPER(exit_atomic)(CPUArchState *env) {cpu_loop_exit_atomic (env_cpu (env),GETPC ()); } + +uint64_t HELPER (getpc )(void ) +{ +return GETPC (); +}
accel/tcg/tcg-runtime.h +2 -0
@@ -30,6 +30,8 @@ DEF_HELPER_FLAGS_1(lookup_tb_ptr, TCG_CALL_NO_WG_SE, cptr, env) DEF_HELPER_FLAGS_1(exit_atomic, TCG_CALL_NO_WG, noreturn, env) +DEF_HELPER_FLAGS_0(getpc, TCG_CALL_NO_RWG_SE, i64) +#ifndef IN_HELPER_PROTO /* * Pass calls to memset directly to libc, without a thunk in qemu.
Signature · 1 line
2.52.0
Re: [PATCH v2 02/50] accel/tcg: Add getpc helper
Quoted 19 lines · Anton Johansson via qemu development
Introduces a helper function to return the current pc (returnaddress of the helper), this allows helper-to-tcg to correctly translate nested helper functions that checks for faulting memory operations.
This is useful as an optimization where a helper functions has a commonly taken fast path that doesn't fault e.g.
void HELPER(outer)(...) { ... // non faulting operations if (some_uncommon_condition) { helper_inner(..., GETPC()) } }the outer helper along with the condition can then be emitted as TCG, and
helper_inner()will be emitted as agen_helper_inner(, ra), where ra is the result ofgen_helper_getpc().
Quoted 5 lines
NOTE: This is not ideal since we're introducing extra helper calls,
and the solution doesn't deal with the "inner" function not being a helper. A better solution and what we'll probably do in the next version of the patchset is to instead emit a helper definition for the "inner" function that usesGETPC().
This is both confusing and wrong.
If you're transforming outer to hot tcg + cold helper, the cold helper would *still* use GETPC(), which would still produce an address within the jit code block for the guest instruction.
r~
PATCH v2 3/50 tcg: Introduce tcg-global-mappings
In reply to Anton Johansson via qemu developmentAdds a cpu_mapping struct to describe, in a declarative fashion, the mapping between fields in a struct, and a corresponding TCG global. As such, tcg_global_mem_new() can be automatically called given an array of cpu_mappings.
This change is not limited to helper-to-tcg, but will be required in future commits to map between offsets into CPUArchState and TCGv globals in a target-agnostic way.
--- include/tcg/tcg-global-mappings.h | 118 ++++++++++++++++++++++++++++++ tcg/meson.build | 1 + tcg/tcg-global-mappings.c | 62 ++++++++++++++++ 3 files changed, 181 insertions(+) create mode 100644 include/tcg/tcg-global-mappings.h create mode 100644 tcg/tcg-global-mappings.c
include/tcg/tcg-global-mappings.h new file +118 -0
@@ -0,0 +1,118 @@ +/* +* Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, see <http://www.gnu.org/licenses/>. +*/ + +#ifndef TCG_GLOBAL_MAP_H +#define TCG_GLOBAL_MAP_H + +/** +* cpu_tcg_mapping: Declarative mapping of offsets into a struct to global +* TCGvs. Parseable by LLVM-based tools. +* @tcg_var_name: String name of the TCGv to use as destination of the mapping. +* @tcg_var_base_address: Address of the above TCGv. +* @cpu_type_name: String name of the base type being mapped, e.g. "CPUArchState". +* @cpu_var_names: Array of printable names of TCGvs, used when calling +* tcg_global_mem_new from init_cpu_tcg_mappings. Contains +* @number_of_elements strings. +* @cpu_var_base_offset: Base offset of field in the source struct. +* @cpu_var_size: Size of field in the source struct, if the field is an array, +* this holds the size of the element type. +* @cpu_var_stride: Stride between array elements in the source struct. This +* can be greater than the element size when mapping a field +* in an array of structs. +* @number_of_elements: Number of elements of array in the source struct. +*/ +typedef struct cpu_tcg_mapping { +const char *tcg_var_name; +void *tcg_var_base_address; + +const char *cpu_type_name; +const char *const *cpu_var_names; +size_t cpu_var_base_offset; +size_t cpu_var_size; +size_t cpu_var_stride; + +size_t number_of_elements; +}cpu_tcg_mapping ; + +#define STRUCT_SIZEOF_FIELD (S , member) sizeof(((S *)0)->member) + +#define STRUCT_ARRAY_SIZE (S , array) \ + (STRUCT_SIZEOF_FIELD(S, array) / STRUCT_SIZEOF_FIELD(S, array[0])) + +/* +* Following are a few macros that aid in constructing +* `cpu_tcg_mapping`s for a few common cases. +*/ + +/* Map between single CPU register and to TCG global */ +#define CPU_TCG_MAP (struct_type, tcg_var, cpu_var) \ + (cpu_tcg_mapping) \ + { \ + .tcg_var_name = stringify(tcg_var), \ + .tcg_var_base_address = &tcg_var, \ + .cpu_type_name = stringify(struct_type), \ + .cpu_var_names = (const char *[]){stringify(cpu_var)}, \ + .cpu_var_base_offset = offsetof(struct_type, cpu_var), \ + .cpu_var_size = STRUCT_SIZEOF_FIELD(struct_type, cpu_var), \ + .cpu_var_stride = 0, .number_of_elements = 1, \ + } + +/* Map between array of CPU registers and array of TCG globals. */ +#define CPU_TCG_MAP_ARRAY (struct_type, tcg_var, cpu_var, names) \ + (cpu_tcg_mapping) \ + { \ + .tcg_var_name = stringify(tcg_var), \ + .tcg_var_base_address = tcg_var, \ + .cpu_type_name = stringify(struct_type), \ + .cpu_var_names = names, \ + .cpu_var_base_offset = offsetof(struct_type, cpu_var), \ + .cpu_var_size = STRUCT_SIZEOF_FIELD(struct_type, cpu_var[0]), \ + .cpu_var_stride = STRUCT_SIZEOF_FIELD(struct_type, cpu_var[0]), \ + .number_of_elements = STRUCT_ARRAY_SIZE(struct_type, cpu_var), \ + } + +/* +* Map between single member in an array of structs to an array +* of TCG globals, e.g. maps +* +* cpu_state.array_of_structs[i].member +* +* to +* +* tcg_global_member[i] +*/ +#define CPU_TCG_MAP_ARRAY_OF_STRUCTS (struct_type, tcg_var, cpu_struct, \ + cpu_var, names) \ + (cpu_tcg_mapping) \ + { \ + .tcg_var_name = stringify(tcg_var), \ + .tcg_var_base_address = tcg_var, \ + .cpu_type_name = stringify(struct_type), \ + .cpu_var_names = names, \ + .cpu_var_base_offset = offsetof(struct_type, cpu_struct[0].cpu_var), \ + .cpu_var_size = \ + STRUCT_SIZEOF_FIELD(struct_type, cpu_struct[0].cpu_var), \ + .cpu_var_stride = STRUCT_SIZEOF_FIELD(struct_type, cpu_struct[0]), \ + .number_of_elements = STRUCT_ARRAY_SIZE(struct_type, cpu_struct), \ + } + +extern cpu_tcg_mapping tcg_global_mappings[]; +extern size_t tcg_global_mapping_count; + +void init_cpu_tcg_mappings (cpu_tcg_mapping *mappings,size_t size); + +#endif /* TCG_GLOBAL_MAP_H */
tcg/meson.build +1 -0
@@ -13,6 +13,7 @@ tcg_ss.add(files( 'tcg-op-ldst.c', 'tcg-op-gvec.c', 'tcg-op-vec.c', + 'tcg-global-mappings.c', )) if get_option('tcg_interpreter')
tcg/tcg-global-mappings.c new file +62 -0
62 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v2 03/50] tcg: Introduce tcg-global-mappings
include/tcg/tcg-global-mappings.h line 114
Quoted 2 lines · Anton Johansson via qemu development
include/tcg/tcg-global-mappings.h new file +2
-0+extern cpu_tcg_mapping tcg_global_mappings[]; +extern size_t tcg_global_mapping_count;
No globals. All such data structures would be local to the translators, calling
Quoted 1 line
+void
init_cpu_tcg_mappings(cpu_tcg_mapping *mappings, size_t size);
as necessary.
Quoted 16 lines
tcg/tcg-global-mappings.c new file +16
-0+void init_cpu_tcg_mappings (cpu_tcg_mapping *mappings,size_t size) +{ +uintptr_t tcg_addr; +size_t cpu_offset; +const char *name; +cpu_tcg_mapping m; + +/* +* Paranoid assertion, this should always hold since +* they're typedef'd to pointers. But you never know! +*/ +g_assert (sizeof (TCGv_i32) ==sizeof (TCGv_i64)); + +/* +* Loop over entries in tcg_global_mappings and +* create the `mapped to` TCGv's.
tcg/tcg-global-mappings.c line 48
Quoted 10 lines
tcg/tcg-global-mappings.c new file +10
-0+*/ +for (int i =0 ; i < size; ++i) { + m = mappings[i]; + +for (int j =0 ; j < m.number_of_elements; ++j) { +/* +* Here we are using the fact that +* sizeof(TCGv_i32) == sizeof(TCGv_i64) == sizeof(TCGv) +*/ +assert (sizeof (TCGv_i32) ==sizeof (TCGv_i64));
Two identical asserts in 10 lines.
Such things are better with a compile-time assert using QEMU_BUILD_BUG_ON, however...
tcg/tcg-global-mappings.c line 53
Quoted 5 lines
tcg/tcg-global-mappings.c new file +5
-0+ tcg_addr = (uintptr_t )m.tcg_var_base_address + j *sizeof (TCGv_i32); + cpu_offset = m.cpu_var_base_offset + j * m.cpu_var_stride; + name = m.cpu_var_names[j]; + +if (m.cpu_var_size <8 ) {
I can understand wanting to use sizeof in the macros, but we can do better than that with
.type = _Generic(((struct_type *)0)->cpu_var,
uint64_t: TCG_TYPE_I64,
int64_t: TCG_TYPE_I64,
uint32_t: TCG_TYPE_I32,
int32_t: TCG_TYPE_I32),which will catch mistakes earlier at compile-time.
tcg/tcg-global-mappings.c line 59
Quoted 6 lines
tcg/tcg-global-mappings.c new file +6
-0+ *(TCGv_i32 *)tcg_addr = +tcg_global_mem_new_i32 (tcg_env, cpu_offset, name); + }else { + *(TCGv_i64 *)tcg_addr = +tcg_global_mem_new_i64 (tcg_env, cpu_offset, name); + }
At which point you can use tcg_global_mem_new_internal.
r~
PATCH v2 4/50 tcg: Increase maximum TB size
In reply to Anton Johansson via qemu developmentDoubles amount of space allocated for translation blocks. This is needed, particularly for Hexagon, where a single instruction packet may consist of up to four vector instructions. If each vector instruction then gets expanded into gvec operations that utilize a small host vector size the TB blows up quite quickly.
--- include/tcg/tcg.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/tcg/tcg.h +1 -1
@@ -40,7 +40,7 @@/* XXX: make safe guess about sizes */ #define MAX_OP_PER_INSTR 266 -#define CPU_TEMP_BUF_NLONGS 128 +#define CPU_TEMP_BUF_NLONGS 256#define TCG_STATIC_FRAME_SIZE (CPU_TEMP_BUF_NLONGS * sizeof(long))typedef int64_t tcg_target_long ;
Signature · 1 line
2.52.0
Re: [PATCH v2 04/50] tcg: Increase maximum TB size
Quoted 16 lines · Anton Johansson via qemu development
Doubles amount of space allocated for translation blocks. This is needed, particularly for Hexagon, where a single instruction packet may consist of up to four vector instructions. If each vector instruction then gets expanded into gvec operations that utilize a small host vector size the TB blows up quite quickly.
Signed-off-by Anton Johansson <[email protected]>--- include/tcg/tcg.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)include/tcg/tcg.h +0
-0
include/tcg/tcg.h line 46
Quoted 8 lines
include/tcg/tcg.h +1
-1@@ -40,7 +40,7 @@/* XXX: make safe guess about sizes */ #define MAX_OP_PER_INSTR 266 -#define CPU_TEMP_BUF_NLONGS 128 +#define CPU_TEMP_BUF_NLONGS 256#define TCG_STATIC_FRAME_SIZE (CPU_TEMP_BUF_NLONGS * sizeof(long))typedef int64_t tcg_target_long ;
This isn't the size of the translation block, but the size of the stack frame. Ok, with a re-wording of the commit message.
r~
PATCH v2 5/50 tcg: Expose tcg_gen_ussub_sat()
In reply to Anton Johansson via qemu developmentMaps nicely to LLVMs usub.sat instrinsic.
--- include/tcg/tcg-op-common.h | 2 ++ tcg/tcg-op-gvec.c | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-)
include/tcg/tcg-op-common.h +2 -0
@@ -192,6 +192,7 @@ void tcg_gen_sar_i32(TCGv_i32 ret, TCGv_i32 arg1, TCGv_i32 arg2);void tcg_gen_mul_i32 (TCGv_i32 ret,TCGv_i32 arg1,TCGv_i32 arg2);void tcg_gen_neg_i32 (TCGv_i32 ret,TCGv_i32 arg);void tcg_gen_not_i32 (TCGv_i32 ret,TCGv_i32 arg); +void tcg_gen_ussub_i32 (TCGv_i32 d,TCGv_i32 a,TCGv_i32 b);/* 64 bit ops */
@@ -258,6 +259,7 @@ void tcg_gen_mulu2_i64(TCGv_i64 rl, TCGv_i64 rh, TCGv_i64 arg1, TCGv_i64 arg2);void tcg_gen_muls2_i64 (TCGv_i64 rl,TCGv_i64 rh,TCGv_i64 arg1,TCGv_i64 arg2);void tcg_gen_mulsu2_i64 (TCGv_i64 rl,TCGv_i64 rh,TCGv_i64 arg1,TCGv_i64 arg2);void tcg_gen_not_i64 (TCGv_i64 ret,TCGv_i64 arg); +void tcg_gen_ussub_i64 (TCGv_i64 d,TCGv_i64 a,TCGv_i64 b);void tcg_gen_ext8s_i64 (TCGv_i64 ret,TCGv_i64 arg);void tcg_gen_ext16s_i64 (TCGv_i64 ret,TCGv_i64 arg);void tcg_gen_ext32s_i64 (TCGv_i64 ret,TCGv_i64 arg);
tcg/tcg-op-gvec.c +2 -2
@@ -2399,14 +2399,14 @@ void tcg_gen_gvec_usadd(unsigned vece, uint32_t dofs, uint32_t aofs,tcg_gen_gvec_3 (dofs, aofs, bofs, oprsz, maxsz, &g[vece]); } -static void tcg_gen_ussub_i32 (TCGv_i32 d,TCGv_i32 a,TCGv_i32 b) +void tcg_gen_ussub_i32 (TCGv_i32 d,TCGv_i32 a,TCGv_i32 b) {TCGv_i32 min =tcg_constant_i32 (0 );tcg_gen_sub_i32 (d, a, b);tcg_gen_movcond_i32 (TCG_COND_LTU , d, a, b, min, d); } -static void tcg_gen_ussub_i64 (TCGv_i64 d,TCGv_i64 a,TCGv_i64 b) +void tcg_gen_ussub_i64 (TCGv_i64 d,TCGv_i64 a,TCGv_i64 b) {TCGv_i64 min =tcg_constant_i64 (0 );tcg_gen_sub_i64 (d, a, b);
Signature · 1 line
2.52.0
Re: [PATCH v2 05/50] tcg: Expose tcg_gen_ussub_sat()
Quoted 7 lines · Anton Johansson via qemu development
Maps nicely to LLVMs usub.sat instrinsic.
Signed-off-by Anton Johansson <[email protected]>--- include/tcg/tcg-op-common.h | 2 ++ tcg/tcg-op-gvec.c | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-)
Re: [PATCH v2 05/50] tcg: Expose tcg_gen_ussub_sat()
In reply to Anton Johansson via qemu developmenttcg/tcg-op-gvec.c void tcg_gen_gvec_usadd(unsigned vece, uint32_t dofs, uint32_t aofs, · line 2407
Quoted 11 lines · Anton Johansson via qemu development
tcg/tcg-op-gvec.c +1
-1@@ -2399,14 +2399,14 @@ void tcg_gen_gvec_usadd(unsigned vece, uint32_t dofs, uint32_t aofs,tcg_gen_gvec_3 (dofs, aofs, bofs, oprsz, maxsz, &g[vece]); } -static void tcg_gen_ussub_i32 (TCGv_i32 d,TCGv_i32 a,TCGv_i32 b) +void tcg_gen_ussub_i32 (TCGv_i32 d,TCGv_i32 a,TCGv_i32 b) {TCGv_i32 min =tcg_constant_i32 (0 );tcg_gen_sub_i32 (d, a, b);tcg_gen_movcond_i32 (TCG_COND_LTU , d, a, b, min, d); }
These aren't quite general purpose. The existing formulation "knows" that d does not overlap a or b -- which is true under the limited invocation here. If you want to expose these for wider use, you need to fix that by allocating a new temporary around this operation.
r~
PATCH v2 6/50 Add helper-to-tcg subproject
In reply to Anton Johansson via qemu developmentAdds a new bare bones subproject along with options to enable or disable it. The LLVM dependency is managed manually through llvm-config for two reasons: meson struggles with multiple system LLVM versions anyway and usually returns the newest one even if it doesn't satisfy the version requirement, and; It allows the user to override the default if they choose. As such, an option is added to the subproject to manually override llvm-config.
A new target config option TARGET_HELPER_TO_TCG is also introduced, which will used in future commits to conditionally add or remove remove code and memory required by helper-to-tcg.
Current meson option is limited to Hexagon, this might change in the future if more uses are found on different frontends.
--- meson.build | 10 ++++ meson_options.txt | 2 + scripts/meson-buildoptions.sh | 5 ++ subprojects/helper-to-tcg/meson.build | 61 +++++++++++++++++++++ subprojects/helper-to-tcg/meson_options.txt | 2 + 5 files changed, 80 insertions(+) create mode 100644 subprojects/helper-to-tcg/meson.build create mode 100644 subprojects/helper-to-tcg/meson_options.txt
meson.build +10 -0
@@ -3273,7 +3273,16 @@ host_kconfig = \ (hv_balloon ? ['CONFIG_HV_BALLOON_POSSIBLE=y'] : []) + \ (have_rust ? ['CONFIG_HAVE_RUST=y'] : []) +helper_to_tcg = subproject('helper-to-tcg', + required: get_option('hexagon_helper_to_tcg') \ + .disable_auto_if('hexagon-linux-user' not in target_dirs and \ + 'hexagon-softmmu' not in target_dirs)) + ignored = [ 'TARGET_XML_FILES', 'TARGET_ABI_DIR' ] +if not helper_to_tcg.found() + # do not define it if it is not usable + ignored += ['TARGET_HELPER_TO_TCG'] +endif default_targets = 'CONFIG_DEFAULT_TARGETS' in config_host actual_target_dirs = []
@@ -4266,6 +4275,7 @@ foreach target : target_dirs if host_os == 'linux' target_inc += include_directories('linux-headers', is_system: true) endif + if target.endswith('-softmmu') target_type='system' if target_base_arch in target_system_arch
meson_options.txt +2 -0
@@ -377,6 +377,8 @@ option('qemu_ga_version', type: 'string', value: '', option('hexagon_idef_parser', type : 'boolean', value : true, description: 'use idef-parser to automatically generate TCG code for the Hexagon frontend') +option('hexagon_helper_to_tcg', type : 'feature', value : 'enabled', + description: 'use the helper-to-tcg translator to automatically generate TCG code from helpers') option('x86_version', type : 'combo', choices : ['0', '1', '2', '3', '4'], value: '1', description: 'tweak required x86_64 architecture version beyond compiler default')
scripts/meson-buildoptions.sh +5 -0
@@ -125,6 +125,9 @@ meson_options_help() { printf "%s\n" ' gtk GTK+ user interface' printf "%s\n" ' guest-agent Build QEMU Guest Agent' printf "%s\n" ' guest-agent-msi Build MSI package for the QEMU Guest Agent' + printf "%s\n" ' hexagon-helper-to-tcg' + printf "%s\n" ' use the helper-to-tcg translator to automatically generate' + printf "%s\n" ' TCG code from helpers' printf "%s\n" ' hv-balloon hv-balloon driver (requires Glib 2.68+ GTree API)' printf "%s\n" ' hvf HVF acceleration support' printf "%s\n" ' iconv Font glyph conversion support'
@@ -337,6 +340,8 @@ _meson_option_parse() { --disable-guest-agent) printf "%s" -Dguest_agent=disabled ;; --enable-guest-agent-msi) printf "%s" -Dguest_agent_msi=enabled ;; --disable-guest-agent-msi) printf "%s" -Dguest_agent_msi=disabled ;; + --enable-hexagon-helper-to-tcg) printf "%s" -Dhexagon_helper_to_tcg=enabled ;; + --disable-hexagon-helper-to-tcg) printf "%s" -Dhexagon_helper_to_tcg=disabled ;; --enable-hexagon-idef-parser) printf "%s" -Dhexagon_idef_parser=true ;; --disable-hexagon-idef-parser) printf "%s" -Dhexagon_idef_parser=false ;; --enable-hv-balloon) printf "%s" -Dhv_balloon=enabled ;;
subprojects/helper-to-tcg/meson.build new file +61 -0
61 lines not on this page · raw message
subprojects/helper-to-tcg/meson_options.txt new file +2 -0
@@ -0,0 +1,2 @@ +option('llvm_config_path', type : 'string', value : 'llvm-config', + description: 'override default llvm-config used for finding LLVM')
Signature · 1 line
2.52.0
Re: [PATCH v2 06/50] Add helper-to-tcg subproject
subprojects/helper-to-tcg/meson.build line 28
Quoted 11 lines · On Thu, 30 Jul 2026 05:09:40 +0200 Anton Johansson via qemu development…
subprojects/helper-to-tcg/meson.build new file +8
-0+project('helper-to-tcg', ['cpp'], + version: '0.8', + default_options: ['cpp_std=c++17']) +# Find LLVM using llvm-config manually. Needed as meson struggles when multiple +# versions of LLVM are installed on the same system (always returns the most +# recent). +llvm_config = get_option('llvm_config_path') +cpp_args = run_command(llvm_config, '--cxxflags').stdout().strip().split()
llvm-config --cxxflags cotains -std=, which takes precedence over cpp_std=c++17.
As a result, on LLVM 15, we get -std=c++14 and the build fails.
I'd just add on the next line:
cpp_args += '-std=c++17'
And drop default_options.
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 7/50 helper-to-tcg: Introduce get-llvm-ir.py
In reply to Anton Johansson via qemu developmentIntroduces a new python helper script to convert a set of QEMU .c files to LLVM IR .ll using clang. Compile flags are found by looking at compile_commands.json, and llvm-link is used to link together all LLVM modules into a single module.
--- subprojects/helper-to-tcg/get-llvm-ir.py | 145 +++++++++++++++++++++++ subprojects/helper-to-tcg/meson.build | 8 ++ 2 files changed, 153 insertions(+) create mode 100755 subprojects/helper-to-tcg/get-llvm-ir.py
subprojects/helper-to-tcg/get-llvm-ir.py new file +145 -0
@@ -0,0 +1,145 @@ +#!/usr/bin/env python3 + +## +## Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, see <http://www.gnu.org/licenses/>. +## + +import argparse +import json +import os +import shlex +import sys +import subprocess + + +def log(msg): + print(msg, file=sys.stderr) + + +def run_command(command): + proc = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + out = proc.communicate() + if proc.wait() != 0: + log(f"Command: {' '.join(command)} exited with {proc.returncode}\n") + log(f"output:\n{out}\n") + + +def find_compile_commands(compile_commands_path, clang_path, input_path, target): + with open(compile_commands_path, "r") as f: + compile_commands = json.load(f) + for compile_command in compile_commands: + path = compile_command["file"] + if os.path.basename(path) != os.path.basename(input_path): + continue + + os.chdir(compile_command["directory"]) + command = compile_command["command"] + + # If building multiple targets there's a chance + # input files share the same path and name. + # This could cause us to find the wrong compile + # command, we use the target path to distinguish + # between these. + if not target in command: + continue + + argv = shlex.split(command) + argv[0] = clang_path + + return argv + + raise ValueError(f"Unable to find compile command for {input_path}") + + +def generate_llvm_ir( + compile_commands_path, clang_path, output_path, input_path, target +): + command = find_compile_commands( + compile_commands_path, clang_path, input_path, target + ) + + flags_to_remove = { + "-ftrivial-auto-var-init=zero", + "-fzero-call-used-regs=used-gpr", + "-Wimplicit-fallthrough=2", + "-Wold-style-declaration", + "-Wno-psabi", + "-Wshadow=local", + "-c", + } + + # Remove + # - output of makefile rules (-MQ,-MF target); + # - output of object files (-o target); + # - excessive zero-initialization of block-scope variables + # (-ftrivial-auto-var-init=zero); + # - and any optimization flags (-O). + for i, arg in reversed(list(enumerate(command))): + if arg in {"-MQ", "-o", "-MF"}: + del command[i : i + 2] + elif arg.startswith("-O") or arg in flags_to_remove: + del command[i] + + # Define a HELPER_TO_TCG macro for translation units wanting to + # conditionally include or exclude code during translation to TCG. + # Disable optimization (-O0) and make sure clang doesn't emit optnone + # attributes (-disable-O0-optnone) which inhibit further optimization. + # Optimization will be performed at a later stage in the helper-to-tcg + # pipeline. + command += [ + "-S", + "-emit-llvm", + "-DHELPER_TO_TCG_IR_GEN", + "-O0", + "-g", + "-Xclang", + "-disable-O0-optnone", + ] + if output_path: + command += ["-o", output_path] + + run_command(command) + + +def main(): + parser = argparse.ArgumentParser( + description="Produce the LLVM IR of a given .c file." + ) + parser.add_argument( + "--compile-commands", required=True, help="Path to compile_commands.json" + ) + parser.add_argument("--clang", default="clang", help="Path to clang.") + parser.add_argument("--llvm-link", default="llvm-link", help="Path to llvm-link.") + parser.add_argument("-o", "--output", required=True, help="Output .ll file path") + parser.add_argument( + "--target-path", help="Path to QEMU target dir. (e.q. target/i386)" + ) + parser.add_argument("inputs", nargs="+", help=".c file inputs") + args = parser.parse_args() + + outputs = [] + for input in args.inputs: + output = os.path.basename(input) + ".ll" + generate_llvm_ir( + args.compile_commands, args.clang, output, input, args.target_path + ) + outputs.append(output) + + run_command([args.llvm_link] + outputs + ["-S", "-o", args.output]) + + +if __name__ == "__main__": + sys.exit(main())
subprojects/helper-to-tcg/meson.build +8 -0
@@ -42,6 +42,14 @@ endif sources = [ ] +clang = bindir / 'clang' +llvm_link = bindir / 'llvm-link' + +get_llvm_ir_cmd = [python, meson.current_source_dir() / 'get-llvm-ir.py', + '--compile-commands', 'compile_commands.json', + '--clang', clang, + '--llvm-link', llvm_link] + # NOTE: Add -Wno-template-id-cdtor for GCC versions >= 14. This warning is # related to a change in the C++ standard in C++20, that also applies to C++14 # for some reason. See defect report DR2237 and commit
Signature · 1 line
2.52.0
Re: [PATCH v2 07/50] helper-to-tcg: Introduce get-llvm-ir.py
Quoted 39 lines · On Thu, 30 Jul 2026 05:09:41 +0200 Anton Johansson via qemu development…
--- subprojects/helper-to-tcg/get-llvm-ir.py | 145 +++++++++++++++++++++++ subprojects/helper-to-tcg/meson.build | 8 ++ 2 files changed, 153 insertions(+) create mode 100755 subprojects/helper-to-tcg/get-llvm-ir.pysubprojects/helper-to-tcg/get-llvm-ir.py new file +27
-0@@ -0,0 +1,145 @@ +#!/usr/bin/env python3 + +## +## Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, see <http://www.gnu.org/licenses/>. +## + +import argparse +import json +import os +import shlex +import sys +import subprocess + +
subprojects/helper-to-tcg/get-llvm-ir.py line 37
Quoted 10 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +10
-0+def log(msg): + print(msg, file=sys.stderr) + + +def run_command(command): + proc = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + out = proc.communicate() + if proc.wait() != 0: + log(f"Command: {' '.join(command)} exited with {proc.returncode}\n") + log(f"output:\n{out}\n")
The whole program should fail if a subcommand fails, printing about it is not enough.
Quoted 34 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +34
-0+ +def find_compile_commands(compile_commands_path, clang_path, input_path, target): + with open(compile_commands_path, "r") as f: + compile_commands = json.load(f) + for compile_command in compile_commands: + path = compile_command["file"] + if os.path.basename(path) != os.path.basename(input_path): + continue + + os.chdir(compile_command["directory"]) + command = compile_command["command"] + + # If building multiple targets there's a chance + # input files share the same path and name. + # This could cause us to find the wrong compile + # command, we use the target path to distinguish + # between these. + if not target in command: + continue + + argv = shlex.split(command) + argv[0] = clang_path + + return argv + + raise ValueError(f"Unable to find compile command for {input_path}") + + +def generate_llvm_ir( + compile_commands_path, clang_path, output_path, input_path, target +): + command = find_compile_commands( + compile_commands_path, clang_path, input_path, target + )
subprojects/helper-to-tcg/get-llvm-ir.py line 82
Quoted 10 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +10
-0+ + flags_to_remove = { + "-ftrivial-auto-var-init=zero", + "-fzero-call-used-regs=used-gpr", + "-Wimplicit-fallthrough=2", + "-Wold-style-declaration", + "-Wno-psabi", + "-Wshadow=local", + "-c", + }
How did you come up with this list?
We should put some indication to make its maintenance easier. For instance, explicitly mention what part of QEMU introduced it, so one can easily check if it's still relevant or if it disappeared.
Quoted 31 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +31
-0+ + # Remove + # - output of makefile rules (-MQ,-MF target); + # - output of object files (-o target); + # - excessive zero-initialization of block-scope variables + # (-ftrivial-auto-var-init=zero); + # - and any optimization flags (-O). + for i, arg in reversed(list(enumerate(command))): + if arg in {"-MQ", "-o", "-MF"}: + del command[i : i + 2] + elif arg.startswith("-O") or arg in flags_to_remove: + del command[i] + + # Define a HELPER_TO_TCG macro for translation units wanting to + # conditionally include or exclude code during translation to TCG. + # Disable optimization (-O0) and make sure clang doesn't emit optnone + # attributes (-disable-O0-optnone) which inhibit further optimization. + # Optimization will be performed at a later stage in the helper-to-tcg + # pipeline. + command += [ + "-S", + "-emit-llvm", + "-DHELPER_TO_TCG_IR_GEN", + "-O0", + "-g", + "-Xclang", + "-disable-O0-optnone", + ] + if output_path: + command += ["-o", output_path] +
subprojects/helper-to-tcg/get-llvm-ir.py line 123
Quoted 10 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +10
-0+ run_command(command) + + +def main(): + parser = argparse.ArgumentParser( + description="Produce the LLVM IR of a given .c file." + ) + parser.add_argument( + "--compile-commands", required=True, help="Path to compile_commands.json" + )
I'm not super happy that we use compile_commands.json, however:
1. compile_commands.json is emitted unconditionally by meson at build time.
2. There's no good alternative. In rev.ng we configure a dedicated (throwaway) clang QEMU build with -fembed-bitcode and then extract the IR downstream.
Maybe here we could manage to build a set of sources with -fembed-bitcode, but that won't work unless the compiler is clang.
We could configure a temporary clang QEMU build in a subdirectory, but that's not very nice.
So, in the end, this makes sense to me.
The proper solution would to have meson provide the full invocation used to produce a certain object file that we can then manipulate, but AFAIU there's not such a feature.
One day we could maybe explore adjusting meson, but I'd say not today.
Quoted 8 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +8
-0+ parser.add_argument("--clang", default="clang", help="Path to clang.") + parser.add_argument("--llvm-link", default="llvm-link", help="Path to llvm-link.") + parser.add_argument("-o", "--output", required=True, help="Output .ll file path") + parser.add_argument( + "--target-path", help="Path to QEMU target dir. (e.q. target/i386)" + ) + parser.add_argument("inputs", nargs="+", help=".c file inputs") + args = parser.parse_args()
subprojects/helper-to-tcg/get-llvm-ir.py line 141
Quoted 10 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +10
-0+ + outputs = [] + for input in args.inputs: + output = os.path.basename(input) + ".ll" + generate_llvm_ir( + args.compile_commands, args.clang, output, input, args.target_path + ) + outputs.append(output) + + run_command([args.llvm_link] + outputs + ["-S", "-o", args.output])
Maybe we should emit bitcode (binary form) instead of textual LLVM IR. If you go this route, don't forget to rename the output to .bc.
I know this is easy for debugging, but bitcode is what one should use "in production" and you're one opt -S away from getting the textual IR again.
Quoted 25 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +4
-0+ + +if __name__ == "__main__": + sys.exit(main())subprojects/helper-to-tcg/meson.build +8
-0@@ -42,6 +42,14 @@ endif sources = [ ] +clang = bindir / 'clang' +llvm_link = bindir / 'llvm-link' + +get_llvm_ir_cmd = [python, meson.current_source_dir() / 'get-llvm-ir.py', + '--compile-commands', 'compile_commands.json', + '--clang', clang, + '--llvm-link', llvm_link] + # NOTE: Add -Wno-template-id-cdtor for GCC versions >= 14. This warning is # related to a change in the C++ standard in C++20, that also applies to C++14 # for some reason. See defect report DR2237 and commitSignature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
Re: [PATCH v2 07/50] helper-to-tcg: Introduce get-llvm-ir.py
Quoted 47 lines · Alessandro Di Federico via qemu development
Quoted 44 lines · On Thu, 30 Jul 2026 05:09:41 +0200 Anton Johansson via qemu development…
--- subprojects/helper-to-tcg/get-llvm-ir.py | 145 +++++++++++++++++++++++ subprojects/helper-to-tcg/meson.build | 8 ++ 2 files changed, 153 insertions(+) create mode 100755 subprojects/helper-to-tcg/get-llvm-ir.pysubprojects/helper-to-tcg/get-llvm-ir.py new file +32
-0@@ -0,0 +1,145 @@ +#!/usr/bin/env python3 + +## +## Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +## +## This program is free software; you can redistribute it and/or modify +## it under the terms of the GNU General Public License as published by +## the Free Software Foundation; either version 2 of the License, or +## (at your option) any later version. +## +## This program is distributed in the hope that it will be useful, +## but WITHOUT ANY WARRANTY; without even the implied warranty of +## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +## GNU General Public License for more details. +## +## You should have received a copy of the GNU General Public License +## along with this program; if not, see <http://www.gnu.org/licenses/>. +## + +import argparse +import json +import os +import shlex +import sys +import subprocess + + +def log(msg): + print(msg, file=sys.stderr) + + +def run_command(command):
Quoted 8 lines
subprojects/helper-to-tcg/get-llvm-ir.py line 37
Quoted 5 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +5
-0+ proc = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + out = proc.communicate() + if proc.wait() != 0: + log(f"Command: {' '.join(command)} exited with {proc.returncode}\n") + log(f"output:\n{out}\n")The whole program should fail if a subcommand fails, printing about it is not enough.
makes sense
[...]
Quoted 13 lines
Quoted 13 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +13
-0+def generate_llvm_ir( + compile_commands_path, clang_path, output_path, input_path, target +): + command = find_compile_commands( + compile_commands_path, clang_path, input_path, target + ) + + flags_to_remove = { + "-ftrivial-auto-var-init=zero", + "-fzero-call-used-regs=used-gpr", + "-Wimplicit-fallthrough=2", + "-Wold-style-declaration", + "-Wno-psabi",
Quoted 8 lines
subprojects/helper-to-tcg/get-llvm-ir.py line 82
Quoted 3 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +3
-0+ "-Wshadow=local", + "-c", + }How did you come up with this list?
We should put some indication to make its maintenance easier. For instance, explicitly mention what part of QEMU introduced it, so one can easily check if it's still relevant or if it disappeared.
Agree, these are a bit arcane. I'll double check if they're still necessary and comment.
[...]
Quoted 23 lines
subprojects/helper-to-tcg/get-llvm-ir.py line 123
Quoted 7 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +7
-0+def main(): + parser = argparse.ArgumentParser( + description="Produce the LLVM IR of a given .c file." + ) + parser.add_argument( + "--compile-commands", required=True, help="Path to compile_commands.json" + )I'm not super happy that we use
compile_commands.json, however:1.
compile_commands.jsonis emitted unconditionally by meson at build time.2. There's no good alternative. In rev.ng we configure a dedicated (throwaway) clang QEMU build with
-fembed-bitcodeand then extract the IR downstream.Maybe here we could manage to build a set of sources with
-fembed-bitcode, but that won't work unless the compiler is clang.We could configure a temporary clang QEMU build in a subdirectory, but that's not very nice.
Quoted 7 lines
So, in the end, this makes sense to me.
The proper solution would to have
mesonprovide the full invocation used to produce a certain object file that we can then manipulate, but AFAIU there's not such a feature.One day we could maybe explore adjusting meson, but I'd say not today.
Yeah compile_commands.json isn't that nice but atleast there's precedent for it in QEMU (scripts/modinfo-collect.py, scripts/check_sparse.py).
Quoted 23 lines
Quoted 9 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +8
-0+ parser.add_argument("--clang", default="clang", help="Path to clang.") + parser.add_argument("--llvm-link", default="llvm-link", help="Path to llvm-link.") + parser.add_argument("-o", "--output", required=True, help="Output .ll file path") + parser.add_argument( + "--target-path", help="Path to QEMU target dir. (e.q. target/i386)" + ) + parser.add_argument("inputs", nargs="+", help=".c file inputs") + args = parser.parse_args()subprojects/helper-to-tcg/get-llvm-ir.py line 141
Quoted 10 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +10
-0+ + outputs = [] + for input in args.inputs: + output = os.path.basename(input) + ".ll" + generate_llvm_ir( + args.compile_commands, args.clang, output, input, args.target_path + ) + outputs.append(output) + + run_command([args.llvm_link] + outputs + ["-S", "-o", args.output])Maybe we should emit bitcode (binary form) instead of textual LLVM IR. If you go this route, don't forget to rename the output to
.bc.
Quoted 3 lines
I know this is easy for debugging, but bitcode is what one should use "in production" and you're one
opt -Saway from getting the textual IR again.
AGH, yes I meant to use bitcode but forgot to change it before sending:)
Quoted 13 lines
Quoted 13 lines
subprojects/helper-to-tcg/get-llvm-ir.py new file +4
-0+ + +if __name__ == "__main__": + sys.exit(main())subprojects/helper-to-tcg/meson.build +0
-0@@ -42,6 +42,14 @@ endif sources = [ ]
Quoted 15 lines
Quoted 13 lines
subprojects/helper-to-tcg/meson.build +8
-0+clang = bindir / 'clang' +llvm_link = bindir / 'llvm-link' + +get_llvm_ir_cmd = [python, meson.current_source_dir() / 'get-llvm-ir.py', + '--compile-commands', 'compile_commands.json', + '--clang', clang, + '--llvm-link', llvm_link] + # NOTE: Add -Wno-template-id-cdtor for GCC versions >= 14. This warning is # related to a change in the C++ standard in C++20, that also applies to C++14 # for some reason. See defect report DR2237 and commitSignature · 1 line
2.52.0Reviewed-by Alessandro Di Federico <[email protected]>
Thanks
Signature · 2 lines
Anton Johansson rev.ng Labs Srl.
PATCH v2 8/50 helper-to-tcg: Handle LLVM version compatibility
In reply to Anton Johansson via qemu developmentAdds a translation unit with the sole purpose of handling inter-LLVM code changes. Instead of littering the code with #ifdefs, most of them will be limited to LlvmCompat.[cpp|hpp] and a saner compat::*() function is exposed in its place.
--- .../helper-to-tcg/include/LlvmCompat.hpp | 124 ++++++++++++++++++ subprojects/helper-to-tcg/meson.build | 1 + subprojects/helper-to-tcg/src/LlvmCompat.cpp | 89 +++++++++++++ 3 files changed, 214 insertions(+) create mode 100644 subprojects/helper-to-tcg/include/LlvmCompat.hpp create mode 100644 subprojects/helper-to-tcg/src/LlvmCompat.cpp
subprojects/helper-to-tcg/include/LlvmCompat.hpp new file +124 -0
@@ -0,0 +1,124 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +// +// The purpose of this file is to both collect and hide most api-specific +// changes of LLVM [10,14]. Hopefully making it easier to keep track of the +// changes necessary to support our targeted versions. +// +// Note some #ifdefs still remain throughout the codebase for larger codeblocks +// that are specific enough such that pulling them here would be more cumbersome +// than it's worth. +// + +#include <llvm/Analysis/TargetTransformInfo.h> +#include <llvm/IR/DerivedTypes.h> +#include <llvm/IR/Intrinsics.h> +#include <llvm/IR/Module.h> +#include <llvm/IR/PassManager.h> +#include <llvm/IR/PatternMatch.h> +#include <llvm/MC/TargetRegistry.h> +#include <llvm/Passes/OptimizationLevel.h> +#include <llvm/Passes/PassBuilder.h> +#include <llvm/Support/FileSystem.h> +#include <llvm/Transforms/Utils/UnifyFunctionExitNodes.h> + +#include <stdint.h> + +namespace compat { + +constexpr auto OpenFlags = llvm::sys::fs::OF_TextWithCRLF; + +using OptimizationLevel = llvm::OptimizationLevel; + +constexpr auto LTOPhase = llvm::ThinOrFullLTOPhase::None; + +inline llvm::PassBuilder createPassBuilder(llvm::TargetMachine *TM, + llvm::PipelineTuningOptions &PTO) { +#if LLVM_VERSION_MAJOR >= 16 + return llvm::PassBuilder(TM, PTO, std::nullopt); +#else + return llvm::PassBuilder(TM, PTO, llvm::None); +#endif +} + +// LLVM 16 changes the getFixedSize() -> getFixedValue() +size_t getTypeAllocSize(const llvm::DataLayout &DL, llvm::Type *Ty); + +// These rely on string comparison with QEMU names using StringRef::startswith, +// which changed to StringRef::starts_with in LLVM 18. +bool isFunctionQemuHelper(llvm::StringRef Name); +bool isFunctionQemuLoadStore(llvm::StringRef Name); + +// LLVM 21 moved to initializing a TargetTransformInfo via unique_ptr, +// instead of by value. +template <typename T> +auto makeTTI(llvm::TargetMachine *TM, const llvm::Function &F) { + // Use explicit TargetTransformInfo() contructor +#if LLVM_VERSION_MAJOR >= 21 + return llvm::TargetTransformInfo(std::make_unique<T>(TM, F)); +#else + return llvm::TargetTransformInfo(T(TM, F)); +#endif +} + +// Note, since constexpr auto is used to alias a function, default +// arguments won't work. +// +// LLVM 20 deprecated getDeclaration() in favour of getOrInsertDeclaration() +namespace Intrinsic { +#if LLVM_VERSION_MAJOR >= 20 +constexpr auto getOrInsertDeclaration = llvm::Intrinsic::getOrInsertDeclaration; +#else +constexpr auto getOrInsertDeclaration = llvm::Intrinsic::getDeclaration; +#endif +} // namespace Intrinsic + +// Wrapper to convert Function- to Module analysis manager +template <typename T> +inline const typename T::Result * +getModuleAnalysisManagerProxyResult(llvm::FunctionAnalysisManager &FAM, + llvm::Function &F) { + auto &MAMProxy = FAM.getResult<llvm::ModuleAnalysisManagerFunctionProxy>(F); + return MAMProxy.getCachedResult<T>(*F.getParent()); +} + +llvm::TargetMachine *getTargetMachine(llvm::Triple &TheTriple); + +// +// LLVM 11 and below does not define the UnifyFunctionExitNodes pass +// for the new pass manager. Copy over the definition and use it for +// 11 and below. +// +using llvm::UnifyFunctionExitNodesPass; + +inline uint32_t getVectorElementCount(const llvm::VectorType *VecTy) { + auto ElementCount = VecTy->getElementCount(); + return ElementCount.getFixedValue(); +} + +// +// PatternMatch +// + +#define compat_m_InsertElt llvm::PatternMatch::m_InsertElt +#define compat_m_Shuffle llvm::PatternMatch::m_Shuffle +#define compat_m_ZeroMask llvm::PatternMatch::m_ZeroMask + +} // namespace compat
subprojects/helper-to-tcg/meson.build +1 -0
@@ -40,6 +40,7 @@ if version_major < 10 or version_major > 21 endif sources = [ + 'src/LlvmCompat.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/LlvmCompat.cpp new file +89 -0
89 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v2 08/50] helper-to-tcg: Handle LLVM version compatibility
Quoted 4 lines · On Thu, 30 Jul 2026 05:09:42 +0200 Anton Johansson via qemu development…
subprojects/helper-to-tcg/src/LlvmCompat.cpp new file +4
-0+bool isFunctionQemuHelper(StringRef Name) { +#if LLVM_VERSION_MAJOR >= 18 + return Name.starts_with("helper_"); +#else
subprojects/helper-to-tcg/src/LlvmCompat.cpp line 57
Quoted 10 lines
subprojects/helper-to-tcg/src/LlvmCompat.cpp new file +10
-0+ return Name.startswith("helper_"); +#endif +} + +bool isFunctionQemuLoadStore(StringRef Name) { +#if LLVM_VERSION_MAJOR >= 18 + return Name.starts_with("cpu_ld") or Name.starts_with("cpu_st"); +#else + return Name.startswith("cpu_ld") or Name.startswith("cpu_st"); +#endif
I'd just keep in compat a startswith function, not isFunctionQemuHelper/isFunctionQemuLoadStore.
Let's strive for keeping this minimal.
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 9/50 helper-to-tcg: Introduce custom LLVM pipeline
In reply to Anton Johansson via qemu developmentAdds a custom pipeline, similar to LLVM opt, with the goal of taking an input LLVM IR module to an equivalent output .c file implementing functions in TCG.
Initial LLVM boilerplate is added up until the creation of a ModulePassManager. A custom target derived from x64 is added, to ensure consistent behaviour across different hosts.
--- .../helper-to-tcg/include/CmdLineOptions.hpp | 23 +++ subprojects/helper-to-tcg/meson.build | 1 + subprojects/helper-to-tcg/src/Pipeline.cpp | 146 ++++++++++++++++++ 3 files changed, 170 insertions(+) create mode 100644 subprojects/helper-to-tcg/include/CmdLineOptions.hpp create mode 100644 subprojects/helper-to-tcg/src/Pipeline.cpp
subprojects/helper-to-tcg/include/CmdLineOptions.hpp new file +23 -0
@@ -0,0 +1,23 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/Support/CommandLine.h> + +// Options for pipeline +extern llvm::cl::list<std::string> InputFiles;
subprojects/helper-to-tcg/meson.build +1 -0
@@ -41,6 +41,7 @@ endif sources = [ 'src/LlvmCompat.cpp', + 'src/Pipeline.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/Pipeline.cpp new file +146 -0
146 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v2 10/50 helper-to-tcg: Add pipeline --debug and --debug-only
In reply to Anton Johansson via qemu developmentThese options are normally added by LLVM, however since we want a minimal set of options in --help we have to readd them ourselves.
--- subprojects/helper-to-tcg/src/Pipeline.cpp | 25 +++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-)
subprojects/helper-to-tcg/src/Pipeline.cpp +24 -1
@@ -37,11 +37,14 @@ #include <llvm/PassRegistry.h> #include <llvm/Passes/PassBuilder.h> #include <llvm/Support/CommandLine.h> +#include <llvm/Support/Debug.h> #include <llvm/Support/InitLLVM.h> #include <llvm/Support/SourceMgr.h> #include <llvm/Support/TargetSelect.h> #include <llvm/Target/TargetMachine.h> +#define DEBUG_TYPE "pipeline" + using namespace llvm; cl::OptionCategory Cat("helper-to-tcg Options");
@@ -50,6 +53,16 @@ cl::OptionCategory Cat("helper-to-tcg Options"); cl::opt<std::string> InputFile(cl::Positional, cl::desc("[input LLVM module]"), cl::cat(Cat)); +// Debug options +#ifndef NDEBUG +static cl::opt<bool> Debug("debug", cl::desc("Enable debug logging (slow)"), + cl::init(false), cl::cat(Cat)); +static cl::opt<std::string> + DebugOnly("debug-only", + cl::desc("Enable debug logging for a specific pass"), + cl::init(""), cl::cat(Cat)); +#endif + // Define a TargetTransformInfo (TTI) subclass, this allows for overriding // common per-llvm-target information expected by other LLVM passes, such // as the width of the largest scalar/vector registers. Needed for consistent
@@ -105,8 +118,18 @@ int main(int argc, char **argv) { cl::ParseCommandLineOptions(argc, argv); - LLVMContext Context; + // Enable debug logging, -debug and -debug-only are normally provided by + // LLVMs default debug options, but since we want to hide most default + // options and keep them visible during a debug build it's easiest to add + // them back manually +#ifndef NDEBUG + DebugFlag = Debug; + if (!DebugOnly.empty()) { + setCurrentDebugType(DebugOnly.c_str()); + } +#endif + LLVMContext Context; SMDiagnostic Err; std::unique_ptr<Module> M = parseIRFile(InputFile, Err, Context);
Signature · 1 line
2.52.0
PATCH v2 11/50 helper-to-tcg: Add simple error creation helper
In reply to Anton Johansson via qemu developmentSimple function for creating Expected<> with nice error messages.
--- subprojects/helper-to-tcg/include/Error.hpp | 38 +++++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 subprojects/helper-to-tcg/include/Error.hpp
subprojects/helper-to-tcg/include/Error.hpp new file +38 -0
@@ -0,0 +1,38 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/IR/Value.h> +#include <llvm/Support/Error.h> + +inline llvm::Error mkError(const llvm::StringRef Msg) { + return llvm::createStringError(llvm::inconvertibleErrorCode(), Msg); +} + +// TODO: Usage of `mkError()` and `dbgs()` for serializing `Value`s is +// __really__ slow, and should only occur for error reporting. +// `ModuleSlotTracker` could be used which would speed up repeated serialization +// of a `Value`. +inline llvm::Error mkError(const llvm::StringRef Msg, const llvm::Value *V) { + std::string Str; + llvm::raw_string_ostream Stream(Str); + Stream << Msg; + Stream << *V; + Stream.flush(); + return llvm::createStringError(llvm::inconvertibleErrorCode(), Str); +}
Signature · 1 line
2.52.0
PATCH v2 12/50 helper-to-tcg: Introduce PrepareForOptPass
In reply to Anton Johansson via qemu developmentAdds a new LLVM pass that runs early in the pipeline with the goal of preparing the input module for optimization by doing some early culling of functions and information gathering.
This commits sets up a new LLVM pass over the IR module and runs it from the pipeline.
--- .../include/PrepareForOptPass.hpp | 34 +++++++++++++++++++ subprojects/helper-to-tcg/meson.build | 1 + subprojects/helper-to-tcg/src/Pipeline.cpp | 27 +++++++++++++++ .../PrepareForOptPass/PrepareForOptPass.cpp | 29 ++++++++++++++++ 4 files changed, 91 insertions(+) create mode 100644 subprojects/helper-to-tcg/include/PrepareForOptPass.hpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp
subprojects/helper-to-tcg/include/PrepareForOptPass.hpp new file +34 -0
@@ -0,0 +1,34 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/IR/PassManager.h> + +// +// PrepareForOptPass +// +// Pass that performs either early information collection or basic culling of +// the input module. simplify the module, or to allow for further optimization. +// + +class PrepareForOptPass : public llvm::PassInfoMixin<PrepareForOptPass> { +public: + PrepareForOptPass() {} + llvm::PreservedAnalyses run(llvm::Module &M, + llvm::ModuleAnalysisManager &MAM); +};
subprojects/helper-to-tcg/meson.build +1 -0
@@ -42,6 +42,7 @@ endif sources = [ 'src/LlvmCompat.cpp', 'src/Pipeline.cpp', + 'src/PrepareForOptPass/PrepareForOptPass.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/Pipeline.cpp +27 -0
@@ -17,6 +17,7 @@ #include "CmdLineOptions.hpp" #include "LlvmCompat.hpp" +#include "PrepareForOptPass.hpp" #if LLVM_VERSION_MAJOR == 15 #include <llvm/ADT/Triple.h>
@@ -42,6 +43,7 @@ #include <llvm/Support/SourceMgr.h> #include <llvm/Support/TargetSelect.h> #include <llvm/Target/TargetMachine.h> +#include <llvm/Transforms/Scalar/SROA.h> #define DEBUG_TYPE "pipeline"
@@ -165,5 +167,30 @@ int main(int argc, char **argv) { ModulePassManager MPM; + // + // Start by Filtering out functions we don't want to translate, + // following by a pass that removes `noinline`s that are inserted + // by clang on -O0. We finally run a UnifyExitNodesPass to make sure + // the helpers we parse only has a single exit. + // + + { + FunctionPassManager FPM; +#if LLVM_VERSION_MAJOR >= 16 + FPM.addPass(SROAPass(SROAOptions::ModifyCFG)); +#else + FPM.addPass(SROAPass()); +#endif + MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM))); + } + + MPM.addPass(PrepareForOptPass()); + + { + FunctionPassManager FPM; + FPM.addPass(compat::UnifyFunctionExitNodesPass()); + MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM))); + } + return 0; }
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp new file +29 -0
@@ -0,0 +1,29 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#include "PrepareForOptPass.h" + +#include <llvm/Support/Debug.h> + +#define DEBUG_TYPE "prepare-for-opt" + +using namespace llvm; + +PreservedAnalyses PrepareForOptPass::run(Module &M, + ModuleAnalysisManager &MAM) { + return PreservedAnalyses::none(); +}
Signature · 1 line
2.52.0
PATCH v2 13/50 helper-to-tcg: PrepareForOptPass, demangle function names
In reply to Anton Johansson via qemu developmentSince input LLVM IR might come from languages which mangle function names, e.g. C++, take extra care to demangle all function names and save them on the side for future lookup.
--- .../PrepareForOptPass/PrepareForOptPass.cpp | 43 ++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-)
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +42 -1
@@ -15,15 +15,56 @@ // along with this program; if not, see <http://www.gnu.org/licenses/>. // -#include "PrepareForOptPass.h" +#include "PrepareForOptPass.hpp" +#include <llvm/ADT/StringRef.h> +#include <llvm/ADT/StringSet.h> +#include <llvm/Demangle/Demangle.h> #include <llvm/Support/Debug.h> #define DEBUG_TYPE "prepare-for-opt" using namespace llvm; +static void demangleFunctionNames(Module &M) { + StringSet DemangledNames; + for (Function &F : M) { + const std::string DemangledName = llvm::demangle(F.getName().str()); + if (DemangledName == F.getName()) { + // Name not mangled + continue; + } + // The resulting demangled name might look something like + // + // namespace::subnamespace::function(...) + // + // Extract the function name and use this to replace mangled name. If + // we previously encountered the same name, give up and leave the + // mangled name. + std::string FunctionName; + size_t Index = 0; + // Remove namespaces + Index = DemangledName.find_last_of(':'); + if (Index != std::string::npos) { + FunctionName = DemangledName.substr(Index + 1); + } + // Remove arguments + Index = FunctionName.find_first_of('('); + if (Index != std::string::npos) { + FunctionName = FunctionName.substr(0, Index); + } + + if (DemangledNames.contains(FunctionName)) { + continue; + } + DemangledNames.insert(FunctionName); + + F.setName(FunctionName); + } +} + PreservedAnalyses PrepareForOptPass::run(Module &M, ModuleAnalysisManager &MAM) { + demangleFunctionNames(M); return PreservedAnalyses::none(); }
Signature · 1 line
2.52.0
Re: [PATCH v2 13/50] helper-to-tcg: PrepareForOptPass, demangle function names
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp line 55
Quoted 9 lines · On Thu, 30 Jul 2026 05:09:47 +0200 Anton Johansson <[email protected]> wrote:
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +9
-0+ Index = DemangledName.find_last_of(':'); + if (Index != std::string::npos) { + FunctionName = DemangledName.substr(Index + 1); + } + // Remove arguments + Index = FunctionName.find_first_of('('); + if (Index != std::string::npos) { + FunctionName = FunctionName.substr(0, Index); + }
LLVM provides a dedicated function to parse mangled
names: getFunctionBaseName. It's in LLVM, so new dependencies (I thought was in clang).
The current code wouldn't handle ns::function(ns::type).
Also, more in general: why are we setting the function name to the name of the original C++ function name?
It's a feature worth documenting in the README along with its ratio.
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
Re: [PATCH v2 13/50] helper-to-tcg: PrepareForOptPass, demangle function names
Quoted 13 lines · Alessandro Di Federico
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp line 55
Quoted 9 lines · On Thu, 30 Jul 2026 05:09:47 +0200 Anton Johansson <[email protected]> wrote:
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +9
-0+ Index = DemangledName.find_last_of(':'); + if (Index != std::string::npos) { + FunctionName = DemangledName.substr(Index + 1); + } + // Remove arguments + Index = FunctionName.find_first_of('('); + if (Index != std::string::npos) { + FunctionName = FunctionName.substr(0, Index); + }
Quoted 3 lines
LLVM provides a dedicated function to parse mangled
names: getFunctionBaseName. It's in LLVM, so new dependencies (I thought was in clang).
Right, as a part of ItaniumPartialDemangler, I'll use that then.
Quoted 2 lines
The current code wouldn't handle
ns::function(ns::type).
Fair point
Quoted 4 lines
Also, more in general: why are we setting the function name to the name of the original C++ function name?
It's a feature worth documenting in the README along with its ratio.
It's a bit hacky, but I really didn't want to pass around yet another Function * -> StringRef map.
Quoted 2 lines
Reviewed-by Alessandro Di Federico <[email protected]>
Thanks
Signature · 2 lines
Anton Johansson rev.ng Labs Srl.
PATCH v2 14/50 helper-to-tcg: PrepareForOptPass, map annotations
In reply to Anton Johansson via qemu developmentIn the LLVM IR module, function annotations are stored in one big global array of strings. Traverse this array and parse the data into a format more useful for future passes. A map between Functions * and an Annotations structure is exposed.
--- .../include/FunctionAnnotation.hpp | 104 ++++++++++++++++++ .../include/PrepareForOptPass.hpp | 7 +- subprojects/helper-to-tcg/src/Pipeline.cpp | 6 +- .../PrepareForOptPass/PrepareForOptPass.cpp | 94 ++++++++++++++++ 4 files changed, 209 insertions(+), 2 deletions(-) create mode 100644 subprojects/helper-to-tcg/include/FunctionAnnotation.hpp
subprojects/helper-to-tcg/include/FunctionAnnotation.hpp new file +104 -0
@@ -0,0 +1,104 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/ADT/DenseMap.h> +#include <llvm/ADT/SmallVector.h> +#include <llvm/Support/Format.h> +#include <llvm/Support/raw_ostream.h> +#include <stdint.h> + +namespace llvm { +class Function; +} + +// Different kind of function annotations which control the behaviour +// of helper-to-tcg. +enum class ArgumentAnnotation : uint8_t { + // Declares a list of arguments as immediates + Immediate = 1, + // Declares a list of arguments as vectors, represented by offsets into + // the CPU state + PtrToOffset = 2, +}; + +// Different kind of function annotations which control the behaviour +// of helper-to-tcg. +enum class FunctionAnnotation : uint8_t { + // Function should be translated + HelperToTcg = 1, + // Return value of function is an immediate + ReturnsImmediate = 2, +}; + +// Annotation data which may be attached to a function +class Annotations { + // 8-bit flag for each argument in a function, fields defined by + // `ArgumentAnnotions`. + llvm::SmallVector<uint8_t, 4> ArgumentAnnotations; + // Flag of function annotations, fields defined by `FunctionsAnnotations`. + uint8_t FunctionAnnotations = 0; + + public: + inline uint8_t getArgFlag(size_t Index) const { + if (Index >= ArgumentAnnotations.size()) { + return 0; + } + return ArgumentAnnotations[Index]; + } + + // Getters and setters for annotations flags. + + inline void set(FunctionAnnotation FA) { + FunctionAnnotations |= (uint8_t)FA; + } + + inline void set(size_t Index, ArgumentAnnotation AA) { + if (Index >= ArgumentAnnotations.size()) { + // Resizing will default initialize any new elements. + ArgumentAnnotations.resize(Index + 1); + } + ArgumentAnnotations[Index] |= (uint8_t)AA; + } + + inline bool isSet(FunctionAnnotation FA) const { + return (FunctionAnnotations & (uint8_t)FA) != 0; + } + + inline bool isSet(size_t Index, ArgumentAnnotation AA) const { + uint8_t Flag = getArgFlag(Index); + return (Flag & (uint8_t)AA) != 0; + } + + // Pretty printing debug information + inline void dump(llvm::raw_ostream &Out) const { + Out << "Annotations:\n"; + Out << " Function: " << llvm::format_hex(FunctionAnnotations, 4) + << "\n"; + for (size_t I = 0; I < ArgumentAnnotations.size(); ++I) { + const uint8_t Flag = ArgumentAnnotations[I]; + Out << " Argument[" << I << "]: " << llvm::format_hex(Flag, 4) + << "\n"; + } + } +}; + +// Mapping from functions to annotations, this is the main structure to be used +// by other parts of the codebase when referencing annotations, filled out by +// `PrepareForOptPass`. +using AnnotationMapTy = llvm::DenseMap<llvm::Function *, Annotations>;
subprojects/helper-to-tcg/include/PrepareForOptPass.hpp +6 -1
@@ -17,6 +17,7 @@ #pragma once +#include "FunctionAnnotation.hpp" #include <llvm/IR/PassManager.h> //
@@ -27,8 +28,12 @@ // class PrepareForOptPass : public llvm::PassInfoMixin<PrepareForOptPass> { + AnnotationMapTy &ResultAnnotations; public: - PrepareForOptPass() {} + PrepareForOptPass(AnnotationMapTy &ResultAnnotations) + : ResultAnnotations(ResultAnnotations) + { + } llvm::PreservedAnalyses run(llvm::Module &M, llvm::ModuleAnalysisManager &MAM); };
subprojects/helper-to-tcg/src/Pipeline.cpp +5 -1
@@ -184,7 +184,11 @@ int main(int argc, char **argv) { MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM))); } - MPM.addPass(PrepareForOptPass()); + // TODO: Get pass results via dependencies instead? Adds more boiler-plate + // but is correlct in LLVM-terms. + + AnnotationMapTy Annotations; + MPM.addPass(PrepareForOptPass(Annotations)); { FunctionPassManager FPM;
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +94 -0
112 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v2 14/50] helper-to-tcg: PrepareForOptPass, map annotations
subprojects/helper-to-tcg/src/Pipeline.cpp int main(int argc, char **argv) { · line 189
Quoted 12 lines · On Thu, 30 Jul 2026 05:09:48 +0200 Anton Johansson <[email protected]> wrote:
subprojects/helper-to-tcg/src/Pipeline.cpp +2
-1@@ -184,7 +184,11 @@ int main(int argc, char **argv) { MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM))); } - MPM.addPass(PrepareForOptPass()); + // TODO: Get pass results via dependencies instead? Adds more boiler-plate + // but is correlct in LLVM-terms.
The more appropriate way to do this is to have an *analysis* holding the data you want to share among your custom passes (AFAIU Annotations, DebugInfo, TcgGlobals, VL). An analysis, unlike a pass (i.e., a transformation) cannot modify the IR but it can expose (non-const) data to passes. For instance the DominatorTreeAnalysis exposes a dominator tree that can be updated.
Another good example of this is the whole LLVM backend which holds the machine IR in an analysis and the backend passes modify it.
Then, in your transformation passes you can depend on the analysis and update its content.
It's always tempting to create local variables in the function creating the pass manager and passing references to custom passes in the constructor, but this makes the passes unusable from opt, which is important for unit testing.
Quoted 5 lines
subprojects/helper-to-tcg/src/Pipeline.cpp +2
-0+ AnnotationMapTy Annotations; + MPM.addPass(PrepareForOptPass(Annotations)); { FunctionPassManager FPM;
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
Re: [PATCH v2 14/50] helper-to-tcg: PrepareForOptPass, map annotations
Quoted 25 lines · Alessandro Di Federico via qemu development
subprojects/helper-to-tcg/src/Pipeline.cpp int main(int argc, char **argv) { · line 189
Quoted 12 lines · On Thu, 30 Jul 2026 05:09:48 +0200 Anton Johansson <[email protected]> wrote:
subprojects/helper-to-tcg/src/Pipeline.cpp +2
-1@@ -184,7 +184,11 @@ int main(int argc, char **argv) { MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM))); } - MPM.addPass(PrepareForOptPass()); + // TODO: Get pass results via dependencies instead? Adds more boiler-plate + // but is correlct in LLVM-terms.The more appropriate way to do this is to have an *analysis* holding the data you want to share among your custom passes (AFAIU Annotations, DebugInfo, TcgGlobals, VL). An analysis, unlike a pass (i.e., a transformation) cannot modify the IR but it can expose (non-const) data to passes. For instance the
DominatorTreeAnalysisexposes a dominator tree that can be updated.
Another good example of this is the whole LLVM backend which holds the machine IR in an analysis and the backend passes modify it.
Quoted 7 lines
Then, in your transformation passes you can depend on the analysis and update its content.
It's always tempting to create local variables in the function creating the pass manager and passing references to custom passes in the constructor, but this makes the passes unusable from
opt, which is important for unit testing.
I understand the difference between analysis and transformation passes. Previously we had one analysis pass per piece of information we gathered, e.g. one analysis pass for gathering annotations, one for mapping globals, and so on. Transformation passes would then depend on these, which is the __correct__ way to structure this in LLVM terms.
In the end we had a bunch of very small analyses and much more boiler plate for passes that make little sense to run independently in opt anyway. All small analyses/transformations were merged to PrepareForOpt and friends, which IMO is simpler and easier to follow at the cost of passing result in this ugly way.
Maybe grouping data gathering to a few analyses makes sense as a compromise? In total we would have 1-2 extra passes simply to get rid of the ugly result passing.
Signature · 2 lines
Anton Johansson rev.ng Labs Srl.
Re: [PATCH v2 14/50] helper-to-tcg: PrepareForOptPass, map annotations
In reply to Anton Johansson via qemu developmentQuoted 56 lines · On Thu, 30 Jul 2026 05:09:48 +0200 Anton Johansson <[email protected]> wrote:
In the LLVM IR module, function annotations are stored in one big global array of strings. Traverse this array and parse the data into a format more useful for future passes. A map between Functions * and an
Annotationsstructure is exposed.Signed-off-by Anton Johansson <[email protected]>--- .../include/FunctionAnnotation.hpp | 104 ++++++++++++++++++ .../include/PrepareForOptPass.hpp | 7 +- subprojects/helper-to-tcg/src/Pipeline.cpp | 6 +- .../PrepareForOptPass/PrepareForOptPass.cpp | 94 ++++++++++++++++ 4 files changed, 209 insertions(+), 2 deletions(-) create mode 100644 subprojects/helper-to-tcg/include/FunctionAnnotation.hppsubprojects/helper-to-tcg/include/FunctionAnnotation.hpp new file +36
-0@@ -0,0 +1,104 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/ADT/DenseMap.h> +#include <llvm/ADT/SmallVector.h> +#include <llvm/Support/Format.h> +#include <llvm/Support/raw_ostream.h> +#include <stdint.h> + +namespace llvm { +class Function; +} + +// Different kind of function annotations which control the behaviour +// of helper-to-tcg. +enum class ArgumentAnnotation : uint8_t { + // Declares a list of arguments as immediates + Immediate = 1, + // Declares a list of arguments as vectors, represented by offsets into + // the CPU state
subprojects/helper-to-tcg/include/FunctionAnnotation.hpp line 46
Quoted 10 lines
subprojects/helper-to-tcg/include/FunctionAnnotation.hpp new file +10
-0+ PtrToOffset = 2, +}; + +// Different kind of function annotations which control the behaviour +// of helper-to-tcg. +enum class FunctionAnnotation : uint8_t { + // Function should be translated + HelperToTcg = 1, + // Return value of function is an immediate + ReturnsImmediate = 2,
I prefer 1 << 1 instead of 2 to better convey this is a bit mask and avoid future copy-paste errors.
subprojects/helper-to-tcg/include/FunctionAnnotation.hpp line 58
Quoted 12 lines
subprojects/helper-to-tcg/include/FunctionAnnotation.hpp new file +12
-0+}; + +// Annotation data which may be attached to a function +class Annotations { + // 8-bit flag for each argument in a function, fields defined by + // `ArgumentAnnotions`. + llvm::SmallVector<uint8_t, 4> ArgumentAnnotations; + // Flag of function annotations, fields defined by `FunctionsAnnotations`. + uint8_t FunctionAnnotations = 0; + + public: + inline uint8_t getArgFlag(size_t Index) const {
Methods defined inline are implicitly inline. inline is redundant.
Quoted 36 lines
subprojects/helper-to-tcg/include/FunctionAnnotation.hpp new file +36
-0+ if (Index >= ArgumentAnnotations.size()) { + return 0; + } + return ArgumentAnnotations[Index]; + } + + // Getters and setters for annotations flags. + + inline void set(FunctionAnnotation FA) { + FunctionAnnotations |= (uint8_t)FA; + } + + inline void set(size_t Index, ArgumentAnnotation AA) { + if (Index >= ArgumentAnnotations.size()) { + // Resizing will default initialize any new elements. + ArgumentAnnotations.resize(Index + 1); + } + ArgumentAnnotations[Index] |= (uint8_t)AA; + } + + inline bool isSet(FunctionAnnotation FA) const { + return (FunctionAnnotations & (uint8_t)FA) != 0; + } + + inline bool isSet(size_t Index, ArgumentAnnotation AA) const { + uint8_t Flag = getArgFlag(Index); + return (Flag & (uint8_t)AA) != 0; + } + + // Pretty printing debug information + inline void dump(llvm::raw_ostream &Out) const { + Out << "Annotations:\n"; + Out << " Function: " << llvm::format_hex(FunctionAnnotations, 4) + << "\n"; + for (size_t I = 0; I < ArgumentAnnotations.size(); ++I) { + const uint8_t Flag = ArgumentAnnotations[I];
subprojects/helper-to-tcg/include/FunctionAnnotation.hpp line 104
Quoted 10 lines
subprojects/helper-to-tcg/include/FunctionAnnotation.hpp new file +10
-0+ Out << " Argument[" << I << "]: " << llvm::format_hex(Flag, 4) + << "\n"; + } + } +}; + +// Mapping from functions to annotations, this is the main structure to be used +// by other parts of the codebase when referencing annotations, filled out by +// `PrepareForOptPass`. +using AnnotationMapTy = llvm::DenseMap<llvm::Function *, Annotations>;
According to the LLVM developer's manual
Quoted 3 lines
DenseMap is a simple linearly probed hash table. It excels at supporting small keys and values
- https://llvm.org/docs/ProgrammersManual.html
I wouldn't use it for values bigger than 128 bits.
Not too bad as of now, but if we end up adding fields to Annotations, this might silently stop be the right tool.
Quoted 13 lines
subprojects/helper-to-tcg/include/PrepareForOptPass.hpp +1
-0@@ -17,6 +17,7 @@ #pragma once +#include "FunctionAnnotation.hpp" #include <llvm/IR/PassManager.h> //
subprojects/helper-to-tcg/include/PrepareForOptPass.hpp line 31
Quoted 4 lines
subprojects/helper-to-tcg/include/PrepareForOptPass.hpp +1
-0@@ -27,8 +28,12 @@ // class PrepareForOptPass : public llvm::PassInfoMixin<PrepareForOptPass> { + AnnotationMapTy &ResultAnnotations;
Passes should not hold module-specific data.
The assumption is that they are instantiated once and run multiple times on different modules.
You should also assume they might run in parallel, even if this doesn't happen in practice.
In short: do not store any state in the pass, if not some const configuration.
Given you also have a couple of static functions sharing the same arguments, I suggest to push all this state in a PrepareForOpt class holding the relevant state and having the static functions as methods.
Quoted 47 lines
subprojects/helper-to-tcg/include/PrepareForOptPass.hpp +4
-1public: - PrepareForOptPass() {} + PrepareForOptPass(AnnotationMapTy &ResultAnnotations) + : ResultAnnotations(ResultAnnotations) + { + } llvm::PreservedAnalyses run(llvm::Module &M, llvm::ModuleAnalysisManager &MAM); };subprojects/helper-to-tcg/src/Pipeline.cpp +5
-1@@ -184,7 +184,11 @@ int main(int argc, char **argv) { MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM))); } - MPM.addPass(PrepareForOptPass()); + // TODO: Get pass results via dependencies instead? Adds more boiler-plate + // but is correlct in LLVM-terms. + + AnnotationMapTy Annotations; + MPM.addPass(PrepareForOptPass(Annotations)); { FunctionPassManager FPM;subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +5
-0@@ -16,10 +16,15 @@ // #include "PrepareForOptPass.hpp" +#include "Error.hpp" #include <llvm/ADT/StringRef.h> #include <llvm/ADT/StringSet.h> #include <llvm/Demangle/Demangle.h> +#include <llvm/IR/Constants.h> +#include <llvm/IR/Function.h> +#include <llvm/IR/Instruction.h> +#include <llvm/IR/Module.h> #include <llvm/Support/Debug.h> #define DEBUG_TYPE "prepare-for-opt"
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp static void demangleFunctionNames(Module &M) { · line 76
Quoted 9 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +6
-0@@ -63,8 +68,97 @@ static void demangleFunctionNames(Module &M) { } } +static Error parseAnnotationStr(Annotations &Ann, StringRef Str, + size_t NumArgs) { + Str = Str.trim(); + + // Function annotations + if (Str.consume_front("helper-to-tcg")) {
Why .consume_front?
Is there some suffix we are happy to ignore?
If yes, add a comment.
If not, let's have an equality test.
Quoted 20 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +20
-0+ Ann.set(FunctionAnnotation::HelperToTcg); + return Error::success(); + } else if (Str.consume_front("returns-immediate")) { + Ann.set(FunctionAnnotation::ReturnsImmediate); + return Error::success(); + } + + // Argument annotations + ArgumentAnnotation AA; + if (Str.consume_front("immediate")) { + AA = ArgumentAnnotation::Immediate; + } else if (Str.consume_front("ptr-to-offset")) { + AA = ArgumentAnnotation::PtrToOffset; + } else { + return mkError("Unknown annotation"); + } + + // An argument annotation looks like + // + // "immediate: 0, 1, 2",
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp static void demangleFunctionNames(Module &M) { · line 106
Quoted 10 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +10
-0+ // + // parse the comma separated list of argument indices. + if (!Str.consume_front(":")) { + return mkError("Expected \":\""); + } + Str = Str.ltrim(' '); + do { + Str = Str.ltrim(' '); + size_t I = 0; + Str.consumeInteger(10, I);
Do not ignore the return value.
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp static void demangleFunctionNames(Module &M) { · line 111
Quoted 5 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +5
-0+ if (I >= NumArgs) { + return mkError("Annotation has out of bounds argument index"); + } + Ann.set(I, AA); + } while (Str.consume_front(","));
I'd check the string is now empty and fail otherwise. Let's try to be strict.
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp static void demangleFunctionNames(Module &M) { · line 116
Quoted 5 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +5
-0+ + return Error::success(); +} + +static void collectAnnotations(Module &M, AnnotationMapTy &ResultAnnotations) {
I'd make collectAnnotations return ResultAnnotations.
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp static void demangleFunctionNames(Module &M) { · line 118
Quoted 2 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +2
-0+ // cast over dyn_cast is being used here to + // assert that the structure of
Please review the comment wrapping of the patchset. Sometimes they split in multiple lines in unexpected ways.
Quoted 6 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +6
-0+ // + // llvm.global.annotation + // + // is what we expect. + + GlobalVariable *GA = M.getGlobalVariable("llvm.global.annotations");
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp static void demangleFunctionNames(Module &M) { · line 134
Quoted 10 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +10
-0+ if (!GA) { + return; + } + + // Get the metadata which is stored in the first op + auto *CA = cast<ConstantArray>(GA->getOperand(0)); + // Loop over metadata + for (Value *CAOp : CA->operands()) { + auto *Struct = cast<ConstantStruct>(CAOp); + assert(Struct->getNumOperands() >= 2);
Are we asserting an expectation specific to helper-to-tcg or a more generic one about the layout of llvm.global.annotations?
In the former case, maybe I'd avoid asserting and I'd just report some message on llvm::errs().
Quoted 32 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +25
-0+ + Function *F = cast<Function>(Struct->getOperand(0)); + ConstantDataArray *AnnData = + cast<ConstantDataArray>(Struct->getOperand(1)->getOperand(0)); + + StringRef AnnStr = AnnData->getAsString(); + AnnStr = AnnStr.substr(0, AnnStr.size() - 1); + Annotations Ann = ResultAnnotations[F]; + if (auto Err = parseAnnotationStr(Ann, AnnStr, F->arg_size()); Err) { + errs() << "Failed to parse annotation: \"" << Err + << "\" for function " << F->getName() << "\n"; + continue; + } + ResultAnnotations[F] = Ann; + } + + LLVM_DEBUG({ + for (auto &P : ResultAnnotations) { + dbgs() << "Annotations for " << P.first->getName() << "\n"; + P.second.dump(dbgs()); + } + }); +} + PreservedAnalyses PrepareForOptPass::run(Module &M, ModuleAnalysisManager &MAM) { demangleFunctionNames(M); + collectAnnotations(M, ResultAnnotations); return PreservedAnalyses::none(); }Signature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 15/50 helper-to-tcg: PrepareForOptPass, cull unused functions
In reply to Anton Johansson via qemu developmentMake an early pass over all functions in the input module and filter out functions with:
1. Invalid return type, or;
2. No helper-to-tcg annotation and not called by a function with such a annotation.
A commandline option is also added to force translation of all functions starting with "helper_".
--- .../helper-to-tcg/include/CmdLineOptions.hpp | 2 + .../include/PrepareForOptPass.hpp | 7 +- subprojects/helper-to-tcg/src/Pipeline.cpp | 5 ++ .../PrepareForOptPass/PrepareForOptPass.cpp | 86 +++++++++++++++++++ 4 files changed, 96 insertions(+), 4 deletions(-)
subprojects/helper-to-tcg/include/CmdLineOptions.hpp +2 -0
@@ -21,3 +21,5 @@ // Options for pipeline extern llvm::cl::list<std::string> InputFiles; +// Options for PrepareForOptPass +extern llvm::cl::opt<bool> TranslateAllHelpers;
subprojects/helper-to-tcg/include/PrepareForOptPass.hpp +3 -4
@@ -29,11 +29,10 @@ class PrepareForOptPass : public llvm::PassInfoMixin<PrepareForOptPass> { AnnotationMapTy &ResultAnnotations; -public: + + public: PrepareForOptPass(AnnotationMapTy &ResultAnnotations) - : ResultAnnotations(ResultAnnotations) - { - } + : ResultAnnotations(ResultAnnotations) {} llvm::PreservedAnalyses run(llvm::Module &M, llvm::ModuleAnalysisManager &MAM); };
subprojects/helper-to-tcg/src/Pipeline.cpp +5 -0
@@ -65,6 +65,11 @@ static cl::opt<std::string> cl::init(""), cl::cat(Cat)); #endif +// Options for PrepareForOptPass +cl::opt<bool> TranslateAllHelpers( + "translate-all-helpers", cl::init(false), + cl::desc("Translate all functions starting with helper_*"), cl::cat(Cat)); + // Define a TargetTransformInfo (TTI) subclass, this allows for overriding // common per-llvm-target information expected by other LLVM passes, such // as the width of the largest scalar/vector registers. Needed for consistent
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +86 -0
112 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v2 15/50] helper-to-tcg: PrepareForOptPass, cull unused functions
Quoted 106 lines · On Thu, 30 Jul 2026 05:09:49 +0200 Anton Johansson <[email protected]> wrote:
Make an early pass over all functions in the input module and filter out functions with:
1. Invalid return type, or;
2. No helper-to-tcg annotation and not called by a function with such a annotation.A commandline option is also added to force translation of all functions starting with "
helper_".Signed-off-by Anton Johansson <[email protected]>--- .../helper-to-tcg/include/CmdLineOptions.hpp | 2 + .../include/PrepareForOptPass.hpp | 7 +- subprojects/helper-to-tcg/src/Pipeline.cpp | 5 ++ .../PrepareForOptPass/PrepareForOptPass.cpp | 86 +++++++++++++++++++ 4 files changed, 96 insertions(+), 4 deletions(-)subprojects/helper-to-tcg/include/CmdLineOptions.hpp +2
-0@@ -21,3 +21,5 @@ // Options for pipeline extern llvm::cl::list<std::string> InputFiles; +// Options for PrepareForOptPass +extern llvm::cl::opt<bool> TranslateAllHelpers;subprojects/helper-to-tcg/include/PrepareForOptPass.hpp +3
-4@@ -29,11 +29,10 @@ class PrepareForOptPass : public llvm::PassInfoMixin<PrepareForOptPass> { AnnotationMapTy &ResultAnnotations; -public: + + public: PrepareForOptPass(AnnotationMapTy &ResultAnnotations) - : ResultAnnotations(ResultAnnotations) - { - } + : ResultAnnotations(ResultAnnotations) {} llvm::PreservedAnalyses run(llvm::Module &M, llvm::ModuleAnalysisManager &MAM); };subprojects/helper-to-tcg/src/Pipeline.cpp +5
-0@@ -65,6 +65,11 @@ static cl::opt<std::string> cl::init(""), cl::cat(Cat)); #endif +// Options for PrepareForOptPass +cl::opt<bool> TranslateAllHelpers( + "translate-all-helpers", cl::init(false), + cl::desc("Translate all functions starting with helper_*"), cl::cat(Cat)); + // Define a TargetTransformInfo (TTI) subclass, this allows for overriding // common per-llvm-target information expected by other LLVM passes, such // as the width of the largest scalar/vector registers. Needed for consistentsubprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +17
-0@@ -16,17 +16,25 @@ // #include "PrepareForOptPass.hpp" +#include "CmdLineOptions.hpp" #include "Error.hpp" +#include "FunctionAnnotation.hpp" +#include "LlvmCompat.hpp" +#include <llvm/ADT/SmallPtrSet.h> #include <llvm/ADT/StringRef.h> #include <llvm/ADT/StringSet.h> #include <llvm/Demangle/Demangle.h> #include <llvm/IR/Constants.h> #include <llvm/IR/Function.h> #include <llvm/IR/Instruction.h> +#include <llvm/IR/Instructions.h> #include <llvm/IR/Module.h> #include <llvm/Support/Debug.h> +#include <queue> +#include <set> + #define DEBUG_TYPE "prepare-for-opt" using namespace llvm;@@ -156,9 +164,87 @@ static void collectAnnotations(Module &M, AnnotationMapTy &ResultAnnotations) { }); } +inline bool hasValidReturnTy(const Module &M, const Function *F) { + Type *RetTy = F->getReturnType(); + return RetTy->isStructTy() || RetTy == Type::getVoidTy(F->getContext()) || + RetTy == Type::getInt8Ty(M.getContext()) || + RetTy == Type::getInt16Ty(M.getContext()) || + RetTy == Type::getInt32Ty(M.getContext()) || + RetTy == Type::getInt64Ty(M.getContext()); +} +
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp static void collectAnnotations(Module &M, AnnotationMapTy &ResultAnnotations) { · line 185
Quoted 10 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +10
-0+// Functions that should be removed: +// - No helper-to-tcg annotation (if TranslateAllHelpers == false); +// - Invalid (non-integer/void) return type +static bool shouldRemoveFunction(const Module &M, const Function &F, + const AnnotationMapTy &AnnotationMap) { + if (F.isDeclaration()) { + return false; + } + + if (!hasValidReturnTy(M, &F)) {
I'd do this check after the others and write something to llvm::errs(). If a helper has been explicitly marked as to be translated, but it's unsuitable, we need to notify the user.
Quoted 22 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +22
-0+ return true; + } + + std::queue<const Function *> Worklist; + std::set<const Function *> Visited; + Worklist.push(&F); + while (!Worklist.empty()) { + const Function *F = Worklist.front(); + Worklist.pop(); + if (F->isDeclaration() or Visited.find(F) != Visited.end()) { + continue; + } + Visited.insert(F); + + if (TranslateAllHelpers and + compat::isFunctionQemuHelper(F->getName())) { + // If --translate-all-helpers is provided and `F` starts with + // "helper_*", then don't skip it. + return false; + } else if (auto It = AnnotationMap.find(F); It != AnnotationMap.end()) { + // Otherwise check "helper-to-tcg" annotation. + const Annotations &Ann = It->second;
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp static void collectAnnotations(Module &M, AnnotationMapTy &ResultAnnotations) { · line 217
Quoted 10 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +10
-0+ if (Ann.isSet(FunctionAnnotation::HelperToTcg)) { + return false; + } + } + + // Push functions that call `F` to the worklist, this way we retain + // functions that are being called by functions with the "helper-to-tcg" + // annotation. + for (const User *U : F->users()) { + auto Call = dyn_cast<CallInst>(U);
Use CallBase so you also catch InvokeInst.
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp static void collectAnnotations(Module &M, AnnotationMapTy &ResultAnnotations) { · line 229
Quoted 12 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +12
-0+ if (!Call) { + continue; + } + const Function *ParentF = Call->getParent()->getParent(); + Worklist.push(ParentF); + } + } + + return true; +} + +static void cullUnusedFunctions(Module &M, AnnotationMapTy &Annotations) {
I'd call it purgeUnusedFunctions, but this is fine as well.
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp static void collectAnnotations(Module &M, AnnotationMapTy &ResultAnnotations) { · line 235
Quoted 6 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +6
-0+ SmallPtrSet<Function *, 16> FunctionsToRemove; + for (auto &F : M) { + if (shouldRemoveFunction(M, F, Annotations)) { + FunctionsToRemove.insert(&F); + } + }
This is not super efficient, for every function you walk the call graph.
I'd walk the list of functions and mark those we need to keep due to name/annotation.
Then I'd walk the CallGraph (check out CallGraph.h) starting from them (and stopping if you get to a function you already visited.
It'd also be nice to fail in case we have indirect calls, at some point.
Quoted 17 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +9
-0+ + for (Function *F : FunctionsToRemove) { + Annotations.erase(F); + F->setComdat(nullptr); + F->deleteBody(); + } +} + PreservedAnalyses PrepareForOptPass::run(Module &M, ModuleAnalysisManager &MAM) { demangleFunctionNames(M); collectAnnotations(M, ResultAnnotations); + cullUnusedFunctions(M, ResultAnnotations); return PreservedAnalyses::none(); }Signature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 16/50 helper-to-tcg: PrepareForOptPass, undef llvm.returnaddress
In reply to Anton Johansson via qemu developmentConvert llvm.returnaddress arguments to cpu_[ld|st]*() to undef, causing the LLVM optmizer to discard the intrinsics. Needed as llvm.returnadress is not representable in TCG, and usually results from usage of GETPC() in helper functions.
--- .../PrepareForOptPass/PrepareForOptPass.cpp | 47 +++++++++++++++++++ 1 file changed, 47 insertions(+)
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +47 -0
@@ -29,6 +29,7 @@ #include <llvm/IR/Function.h> #include <llvm/IR/Instruction.h> #include <llvm/IR/Instructions.h> +#include <llvm/IR/Intrinsics.h> #include <llvm/IR/Module.h> #include <llvm/Support/Debug.h>
@@ -241,10 +242,56 @@ static void cullUnusedFunctions(Module &M, AnnotationMapTy &Annotations) { } } +struct RetAddrReplaceInfo { + User *Parent; + unsigned OpIndex; + Type *Ty; +}; + +static void replaceRetaddrWithUndef(Module &M) { + // Replace uses of llvm.returnaddress arguments to cpu_ld* w. undef, + // and let optimizations remove it. Needed as llvm.returnaddress is + // not reprensentable in TCG. + SmallVector<RetAddrReplaceInfo, 24> UsesToReplace; + Function *Retaddr = compat::Intrinsic::getOrInsertDeclaration( + &M, Intrinsic::returnaddress, {}); + // Loop over all calls to llvm.returnaddress + for (auto *CallUser : Retaddr->users()) { + auto *Call = dyn_cast<CallInst>(CallUser); + if (!Call) { + continue; + } + for (auto *PtrToIntUser : Call->users()) { + auto *Cast = dyn_cast<PtrToIntInst>(PtrToIntUser); + if (!Cast) { + continue; + } + for (Use &U : Cast->uses()) { + auto *Call = dyn_cast<CallInst>(U.getUser()); + Function *F = Call->getCalledFunction(); + if (compat::isFunctionQemuLoadStore(F->getName())) { + UsesToReplace.push_back({ + .Parent = U.getUser(), + .OpIndex = U.getOperandNo(), + .Ty = U->getType(), + }); + } + } + } + } + + // Defer replacement to not invalidate iterators + for (RetAddrReplaceInfo &RI : UsesToReplace) { + auto *Undef = UndefValue::get(RI.Ty); + RI.Parent->setOperand(RI.OpIndex, Undef); + } +} + PreservedAnalyses PrepareForOptPass::run(Module &M, ModuleAnalysisManager &MAM) { demangleFunctionNames(M); collectAnnotations(M, ResultAnnotations); cullUnusedFunctions(M, ResultAnnotations); + replaceRetaddrWithUndef(M); return PreservedAnalyses::none(); }
Signature · 1 line
2.52.0
Re: [PATCH v2 16/50] helper-to-tcg: PrepareForOptPass, undef llvm.returnaddress
Quoted 26 lines · On Thu, 30 Jul 2026 05:09:50 +0200 Anton Johansson <[email protected]> wrote:
Convert llvm.returnaddress arguments to cpu_[ld|st]*() to undef, causing the LLVM optmizer to discard the intrinsics. Needed as llvm.returnadress is not representable in TCG, and usually results from usage of
GETPC()in helper functions.Signed-off-by Anton Johansson <[email protected]>--- .../PrepareForOptPass/PrepareForOptPass.cpp | 47 +++++++++++++++++++ 1 file changed, 47 insertions(+)subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +1
-0@@ -29,6 +29,7 @@ #include <llvm/IR/Function.h> #include <llvm/IR/Instruction.h> #include <llvm/IR/Instructions.h> +#include <llvm/IR/Intrinsics.h> #include <llvm/IR/Module.h> #include <llvm/Support/Debug.h>@@ -241,10 +242,56 @@ static void cullUnusedFunctions(Module &M, AnnotationMapTy &Annotations) { } }
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp static void cullUnusedFunctions(Module &M, AnnotationMapTy &Annotations) { · line 254
Quoted 10 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +10
-0+struct RetAddrReplaceInfo { + User *Parent; + unsigned OpIndex; + Type *Ty; +}; + +static void replaceRetaddrWithUndef(Module &M) { + // Replace uses of llvm.returnaddress arguments to cpu_ld* w. undef, + // and let optimizations remove it. Needed as llvm.returnaddress is + // not reprensentable in TCG.
I'd add a brief comment about what's the use in helpers of llvm.returnaddress, possibly with a reference to the part of code emitting it.
This way, if someone changes things around that part of the code will (hopefully) git grep this.
Quoted 45 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +36
-0+ SmallVector<RetAddrReplaceInfo, 24> UsesToReplace; + Function *Retaddr = compat::Intrinsic::getOrInsertDeclaration( + &M, Intrinsic::returnaddress, {}); + // Loop over all calls to llvm.returnaddress + for (auto *CallUser : Retaddr->users()) { + auto *Call = dyn_cast<CallInst>(CallUser); + if (!Call) { + continue; + } + for (auto *PtrToIntUser : Call->users()) { + auto *Cast = dyn_cast<PtrToIntInst>(PtrToIntUser); + if (!Cast) { + continue; + } + for (Use &U : Cast->uses()) { + auto *Call = dyn_cast<CallInst>(U.getUser()); + Function *F = Call->getCalledFunction(); + if (compat::isFunctionQemuLoadStore(F->getName())) { + UsesToReplace.push_back({ + .Parent = U.getUser(), + .OpIndex = U.getOperandNo(), + .Ty = U->getType(), + }); + } + } + } + } + + // Defer replacement to not invalidate iterators + for (RetAddrReplaceInfo &RI : UsesToReplace) { + auto *Undef = UndefValue::get(RI.Ty); + RI.Parent->setOperand(RI.OpIndex, Undef); + } +} + PreservedAnalyses PrepareForOptPass::run(Module &M, ModuleAnalysisManager &MAM) { demangleFunctionNames(M); collectAnnotations(M, ResultAnnotations); cullUnusedFunctions(M, ResultAnnotations); + replaceRetaddrWithUndef(M); return PreservedAnalyses::none(); }Signature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 17/50 helper-to-tcg: PrepareForOptPass, fixup inline attributes
In reply to Anton Johansson via qemu developmentWhen producing LLVM IR using clang -O0, a noinline attribute is added. Remove this attribute to not inhibit future optimization. Also try and force functions returning struct type to be inlined so they might be translated.
--- .../src/PrepareForOptPass/PrepareForOptPass.cpp | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +14 -0
@@ -32,6 +32,7 @@ #include <llvm/IR/Intrinsics.h> #include <llvm/IR/Module.h> #include <llvm/Support/Debug.h> +#include <llvm/Transforms/Utils/Local.h> #include <queue> #include <set>
@@ -293,5 +294,18 @@ PreservedAnalyses PrepareForOptPass::run(Module &M, collectAnnotations(M, ResultAnnotations); cullUnusedFunctions(M, ResultAnnotations); replaceRetaddrWithUndef(M); + // Remove noinline function attributes automatically added by -O0, add + // alwaysinline attribute to functions with a struct return value, these + // can not be translated to TCG currently and we rely on inlining to + // hopefully get rid of them. + for (Function &F : M) { + if (F.hasFnAttribute(Attribute::AttrKind::NoInline)) { + F.removeFnAttr(Attribute::AttrKind::NoInline); + } + if (F.getReturnType()->isStructTy()) { + F.addFnAttr(Attribute::AttrKind::AlwaysInline); + } + } + return PreservedAnalyses::none(); }
Signature · 1 line
2.52.0
Re: [PATCH v2 17/50] helper-to-tcg: PrepareForOptPass, fixup inline attributes
Quoted 27 lines · On Thu, 30 Jul 2026 05:09:51 +0200 Anton Johansson <[email protected]> wrote:
When producing LLVM IR using clang -O0, a noinline attribute is added. Remove this attribute to not inhibit future optimization. Also try and force functions returning struct type to be inlined so they might be translated.
Signed-off-by Anton Johansson <[email protected]>--- .../src/PrepareForOptPass/PrepareForOptPass.cpp | 14 ++++++++++++++ 1 file changed, 14 insertions(+)subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +2
-0@@ -32,6 +32,7 @@ #include <llvm/IR/Intrinsics.h> #include <llvm/IR/Module.h> #include <llvm/Support/Debug.h> +#include <llvm/Transforms/Utils/Local.h> #include <queue> #include <set>@@ -293,5 +294,18 @@ PreservedAnalyses PrepareForOptPass::run(Module &M, collectAnnotations(M, ResultAnnotations); cullUnusedFunctions(M, ResultAnnotations); replaceRetaddrWithUndef(M); + // Remove noinline function attributes automatically added by -O0, add
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp PreservedAnalyses PrepareForOptPass::run(Module &M, · line 307
Quoted 10 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +10
-0+ // alwaysinline attribute to functions with a struct return value, these + // can not be translated to TCG currently and we rely on inlining to + // hopefully get rid of them. + for (Function &F : M) { + if (F.hasFnAttribute(Attribute::AttrKind::NoInline)) { + F.removeFnAttr(Attribute::AttrKind::NoInline); + } + if (F.getReturnType()->isStructTy()) { + F.addFnAttr(Attribute::AttrKind::AlwaysInline); + }
I'd add a brief comment about why structs are a problem.
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp PreservedAnalyses PrepareForOptPass::run(Module &M, · line 309
Quoted 2 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +2
-0+ } +
Did you consider having dinstinct passes for:
1. demangleFunctionNames
2. collectAnnotations + cullUnusedFunctions
3. replaceRetaddrWithUndef
4. This
Quoted 5 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +0
-0return PreservedAnalyses::none(); }Signature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 18/50 helper-to-tcg: PrepareForOptPass, collect debuginfo
In reply to Anton Johansson via qemu developmentDebug information is collected before any non-preserving optimization pass is ran. Currently, only variable definitions are parsed, mapping a Value * to a name and typename.
--- .../helper-to-tcg/include/DebugInfo.hpp | 46 ++++++++++++++++ .../include/PrepareForOptPass.hpp | 10 ++-- subprojects/helper-to-tcg/src/Pipeline.cpp | 3 +- .../PrepareForOptPass/PrepareForOptPass.cpp | 52 +++++++++++++++++++ 4 files changed, 107 insertions(+), 4 deletions(-) create mode 100644 subprojects/helper-to-tcg/include/DebugInfo.hpp
subprojects/helper-to-tcg/include/DebugInfo.hpp new file +46 -0
@@ -0,0 +1,46 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/ADT/StringRef.h> +#include <llvm/IR/ValueMap.h> + +namespace llvm { +class Value; +} + +// StringRefs will refer do debug metadata fields which have the same +// lifetime as the LLVMContext and survive accross optimizations and +// possible deletions of functions/variables. +struct DebugInfo { + llvm::StringRef VarName; + llvm::StringRef BaseTypeName; +}; + +using DebugInfoMapTy = llvm::ValueMap<const llvm::Value *, DebugInfo>; + +// Helper to get the variable name from debug info associated with a particular +// value, or default construct an empty name. +inline llvm::StringRef getDebugVarName(const DebugInfoMapTy &DM, + const llvm::Value *V) { + auto It = DM.find(V); + if (It != DM.end()) { + return It->second.VarName; + } + return {}; +}
subprojects/helper-to-tcg/include/PrepareForOptPass.hpp +7 -3
@@ -17,6 +17,7 @@ #pragma once +#include "DebugInfo.hpp" #include "FunctionAnnotation.hpp" #include <llvm/IR/PassManager.h>
@@ -29,10 +30,13 @@ class PrepareForOptPass : public llvm::PassInfoMixin<PrepareForOptPass> { AnnotationMapTy &ResultAnnotations; + DebugInfoMapTy &ResultDebugInfo; - public: - PrepareForOptPass(AnnotationMapTy &ResultAnnotations) - : ResultAnnotations(ResultAnnotations) {} +public: + PrepareForOptPass(AnnotationMapTy &ResultAnnotations, + DebugInfoMapTy &ResultDebugInfo) + : ResultAnnotations(ResultAnnotations), + ResultDebugInfo(ResultDebugInfo) {} llvm::PreservedAnalyses run(llvm::Module &M, llvm::ModuleAnalysisManager &MAM); };
subprojects/helper-to-tcg/src/Pipeline.cpp +2 -1
@@ -193,7 +193,8 @@ int main(int argc, char **argv) { // but is correlct in LLVM-terms. AnnotationMapTy Annotations; - MPM.addPass(PrepareForOptPass(Annotations)); + DebugInfoMapTy DebugInfo; + MPM.addPass(PrepareForOptPass(Annotations, DebugInfo)); { FunctionPassManager FPM;
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +52 -0
70 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v2 18/50] helper-to-tcg: PrepareForOptPass, collect debuginfo
Quoted 3 lines · On Thu, 30 Jul 2026 05:09:52 +0200 Anton Johansson <[email protected]> wrote:
Debug information is collected before any non-preserving optimization pass is ran. Currently, only variable definitions are parsed, mapping a Value * to a name and typename.
What's the goal of this debug info processing?
Clearly specify in the commit message and somewhere in the code what's the end purpose.
In particular, at this stage, it's unclear whether these are just for debugging purposes (i.e., they are optional) or not.
I know it's not optional, but clearly stating it here is helpful.
Quoted 51 lines
Signed-off-by Anton Johansson <[email protected]>--- .../helper-to-tcg/include/DebugInfo.hpp | 46 ++++++++++++++++ .../include/PrepareForOptPass.hpp | 10 ++-- subprojects/helper-to-tcg/src/Pipeline.cpp | 3 +- .../PrepareForOptPass/PrepareForOptPass.cpp | 52 +++++++++++++++++++ 4 files changed, 107 insertions(+), 4 deletions(-) create mode 100644 subprojects/helper-to-tcg/include/DebugInfo.hppsubprojects/helper-to-tcg/include/DebugInfo.hpp new file +36
-0@@ -0,0 +1,46 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/ADT/StringRef.h> +#include <llvm/IR/ValueMap.h> + +namespace llvm { +class Value; +} + +// StringRefs will refer do debug metadata fields which have the same +// lifetime as the LLVMContext and survive accross optimizations and +// possible deletions of functions/variables. +struct DebugInfo { + llvm::StringRef VarName; + llvm::StringRef BaseTypeName; +}; + +using DebugInfoMapTy = llvm::ValueMap<const llvm::Value *, DebugInfo>; +
subprojects/helper-to-tcg/include/DebugInfo.hpp line 46
Quoted 10 lines
subprojects/helper-to-tcg/include/DebugInfo.hpp new file +10
-0+// Helper to get the variable name from debug info associated with a particular +// value, or default construct an empty name. +inline llvm::StringRef getDebugVarName(const DebugInfoMapTy &DM, + const llvm::Value *V) { + auto It = DM.find(V); + if (It != DM.end()) { + return It->second.VarName; + } + return {}; +}
This looks like a class under disguise.
Something like...
```
class DebugInfoMap {
private:
DebugInfoMapTy Map;
public:
StringRef
getDebugVarName(const DebugInfoMapTy &DM,
const llvm::Value *V);
};
```Quoted 63 lines
subprojects/helper-to-tcg/include/PrepareForOptPass.hpp +7
-3@@ -17,6 +17,7 @@ #pragma once +#include "DebugInfo.hpp" #include "FunctionAnnotation.hpp" #include <llvm/IR/PassManager.h>@@ -29,10 +30,13 @@ class PrepareForOptPass : public llvm::PassInfoMixin<PrepareForOptPass> { AnnotationMapTy &ResultAnnotations; + DebugInfoMapTy &ResultDebugInfo; - public: - PrepareForOptPass(AnnotationMapTy &ResultAnnotations) - : ResultAnnotations(ResultAnnotations) {} +public: + PrepareForOptPass(AnnotationMapTy &ResultAnnotations, + DebugInfoMapTy &ResultDebugInfo) + : ResultAnnotations(ResultAnnotations), + ResultDebugInfo(ResultDebugInfo) {} llvm::PreservedAnalyses run(llvm::Module &M, llvm::ModuleAnalysisManager &MAM); };subprojects/helper-to-tcg/src/Pipeline.cpp +2
-1@@ -193,7 +193,8 @@ int main(int argc, char **argv) { // but is correlct in LLVM-terms. AnnotationMapTy Annotations; - MPM.addPass(PrepareForOptPass(Annotations)); + DebugInfoMapTy DebugInfo; + MPM.addPass(PrepareForOptPass(Annotations, DebugInfo)); { FunctionPassManager FPM;subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +5
-0@@ -26,6 +26,9 @@ #include <llvm/ADT/StringSet.h> #include <llvm/Demangle/Demangle.h> #include <llvm/IR/Constants.h> +#if LLVM_VERSION_MAJOR >= 19 +#include <llvm/IR/DebugProgramInstruction.h> +#endif #include <llvm/IR/Function.h> #include <llvm/IR/Instruction.h> #include <llvm/IR/Instructions.h>@@ -288,6 +291,53 @@ static void replaceRetaddrWithUndef(Module &M) { } } +static void collectDebugInfo(Function &F, DebugInfoMapTy &ResultDebugInfo) { + StringSet EncounteredNames;
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp static void replaceRetaddrWithUndef(Module &M) { · line 305
Quoted 10 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +10
-0+ for (auto &BB : F) { + for (Instruction &I : BB) { +#if LLVM_VERSION_MAJOR >= 19 + for (DbgVariableRecord &DVR : + filterDbgVars(I.getDbgRecordRange())) { + if (!DVR.isDbgValue()) { + continue; + } + StringRef BaseType{}; + StringRef VarName{};
{} is not really necessary here. I'd avoid it altogether. You can use it to 0-initialize scalars, but even then I'd just add = 0.
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp static void replaceRetaddrWithUndef(Module &M) { · line 315
Quoted 10 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +10
-0+ if (auto *Derived = + dyn_cast<DIDerivedType>(DVR.getVariable()->getType())) { + BaseType = Derived->getBaseType()->getName(); + } + VarName = DVR.getVariable()->getName(); + if (ResultDebugInfo.find(DVR.getValue(0)) == + ResultDebugInfo.end() and + !EncounteredNames.contains(VarName.data())) { + ResultDebugInfo[DVR.getValue(0)] = {VarName, BaseType}; + EncounteredNames.insert(VarName.data());
What's the logic for having a variable name considered only once? It should be documented here.
Quoted 39 lines
subprojects/helper-to-tcg/src/PrepareForOptPass/PrepareForOptPass.cpp +27
-0+ } + } +#else + if (I.isDebugOrPseudoInst()) { + if (const auto *Dbg = dyn_cast<DbgValueInst>(&I)) { + StringRef BaseType{}; + StringRef VarName{}; + if (auto *Derived = dyn_cast<DIDerivedType>( + Dbg->getVariable()->getType())) { + BaseType = Derived->getBaseType()->getName(); + } + VarName = Dbg->getVariable()->getName(); + if (ResultDebugInfo.find(Dbg->getValue(0)) == + ResultDebugInfo.end() and + !EncounteredNames.contains(VarName.data())) { + ResultDebugInfo[Dbg->getValue(0)] = {VarName, BaseType}; + EncounteredNames.insert(VarName.data()); + } + } + } +#endif + } + } +} + PreservedAnalyses PrepareForOptPass::run(Module &M, ModuleAnalysisManager &MAM) { demangleFunctionNames(M);@@ -305,6 +355,8 @@ PreservedAnalyses PrepareForOptPass::run(Module &M, if (F.getReturnType()->isStructTy()) { F.addFnAttr(Attribute::AttrKind::AlwaysInline); } + // Populate variable and type names for `Value`s from debug info. + collectDebugInfo(F, ResultDebugInfo); } return PreservedAnalyses::none();Signature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 19/50 helper-to-tcg: Pipeline, run optimization pass
In reply to Anton Johansson via qemu developmentRun a standard LLVM -Oz optimization pass, which makes up the bulk of optimizations in helper-to-tcg.
--- subprojects/helper-to-tcg/src/Pipeline.cpp | 12 ++++++++++++ 1 file changed, 12 insertions(+)
subprojects/helper-to-tcg/src/Pipeline.cpp +12 -0
@@ -202,5 +202,17 @@ int main(int argc, char **argv) { MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM))); } + // + // Run a -Oz optimization pass. In general -Oz will prefer loop + // vectorization over unrolling, as compared to -O3. In TCG, this + // translates to more utilization of gvec and possibly smaller TBs. + // + + // Optimization passes + MPM.addPass(PB.buildModuleSimplificationPipeline( + compat::OptimizationLevel::Oz, compat::LTOPhase)); + MPM.addPass( + PB.buildModuleOptimizationPipeline(compat::OptimizationLevel::Oz, {})); + return 0; }
Signature · 1 line
2.52.0
Re: [PATCH v2 19/50] helper-to-tcg: Pipeline, run optimization pass
Quoted 32 lines · On Thu, 30 Jul 2026 05:09:53 +0200 Anton Johansson <[email protected]> wrote:
Run a standard LLVM -Oz optimization pass, which makes up the bulk of optimizations in helper-to-tcg.
Signed-off-by Anton Johansson <[email protected]>--- subprojects/helper-to-tcg/src/Pipeline.cpp | 12 ++++++++++++ 1 file changed, 12 insertions(+)subprojects/helper-to-tcg/src/Pipeline.cpp +12
-0@@ -202,5 +202,17 @@ int main(int argc, char **argv) { MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM))); } + // + // Run a -Oz optimization pass. In general -Oz will prefer loop + // vectorization over unrolling, as compared to -O3. In TCG, this + // translates to more utilization of gvec and possibly smaller TBs. + // + + // Optimization passes + MPM.addPass(PB.buildModuleSimplificationPipeline( + compat::OptimizationLevel::Oz, compat::LTOPhase)); + MPM.addPass( + PB.buildModuleOptimizationPipeline(compat::OptimizationLevel::Oz, {})); + return 0; }Signature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 20/50 helper-to-tcg: Introduce pseudo instructions
In reply to Anton Johansson via qemu development"pseudo" instructions makes it easy to add custom instructions to LLVM IR in the form of calls to undefined functions. These will be used in future commits to express functionality present in TCG that is missing from LLVM IR (certain vector ops.), or to simplify the backend by collecting similar instruction mappings into a single opcode (idendity mapping).
Mapping from a call instructions in LLVM IR to an enum representing the pseudo instruction is also handled, this avoids string comparisons in the backend, and is easy to switch over.
--- .../helper-to-tcg/include/PseudoInst.hpp | 75 +++++++ .../helper-to-tcg/include/PseudoInst.inc | 78 ++++++++ subprojects/helper-to-tcg/meson.build | 1 + subprojects/helper-to-tcg/src/PseudoInst.cpp | 183 ++++++++++++++++++ 4 files changed, 337 insertions(+) create mode 100644 subprojects/helper-to-tcg/include/PseudoInst.hpp create mode 100644 subprojects/helper-to-tcg/include/PseudoInst.inc create mode 100644 subprojects/helper-to-tcg/src/PseudoInst.cpp
subprojects/helper-to-tcg/include/PseudoInst.hpp new file +75 -0
@@ -0,0 +1,75 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <stdint.h> + +#include <llvm/ADT/ArrayRef.h> +#include <llvm/IR/IRBuilder.h> +#include <llvm/IR/Module.h> +#include <llvm/IR/Value.h> + +// Pseudo instructions refers to extra LLVM instructions implemented as +// calls to undefined functions. They are useful for amending LLVM IR to +// simplify mapping to TCG in the backend, e.g. +// +// %2 = call i32 @IdentityMap.i32.i16(i16 %1) +// +// is a pseudo opcode used to communicate that %1 and %2 should be mapped +// to the same value in TCG. + +enum PseudoInstArg { + ArgAny, + ArgInt, + ArgVec, + ArgPtr, + ArgLabel, + ArgVoid, +}; + +// Define an enum `PseudoInst` consisting of the names of pseudo instructions +// defined in "PseudoInst.inc". +#define PSEUDO_INST_DEF(name, ret, args) name +enum PseudoInst : uint8_t { +#include "PseudoInst.inc" +}; +#undef PSEUDO_INST_DEF + +// Retrieve string representation and argument counts for a given +// pseudo instruction. +const char *pseudoInstName(PseudoInst Inst); +uint8_t pseudoInstArgCount(PseudoInst Inst); +llvm::ArrayRef<PseudoInstArg> pseudoInstArgTypes(PseudoInst Inst); + +// Maps `PseudoInst`, return type, and argument types, to a `FunctionCallee` +// that can be called. Provided argument types are matched against +// "PseudoInst.inc" for verification and an assert triggers on failure. +llvm::FunctionCallee pseudoInstFunction(llvm::Module &M, PseudoInst Inst, + llvm::Type *RetType, + llvm::ArrayRef<llvm::Type *> ArgTypes); + +// Convenience function that gets a pseudo instruction via +// `pseudoInstFunction()` and creates a call to it via the provided `IRBuilder`. +llvm::CallInst *createPseudoInstCall(llvm::Module &M, + llvm::IRBuilder<> &Builder, + PseudoInst Inst, llvm::Type *RetType, + llvm::ArrayRef<llvm::Value *> Values); + +// Reverse mapping of above, takes a call instruction and attempts to map the +// callee to a `PseudoInst`. +PseudoInst getPseudoInstFromCall(const llvm::CallInst *Call);
subprojects/helper-to-tcg/include/PseudoInst.inc new file +78 -0
78 lines not on this page · raw message
subprojects/helper-to-tcg/meson.build +1 -0
@@ -43,6 +43,7 @@ sources = [ 'src/LlvmCompat.cpp', 'src/Pipeline.cpp', 'src/PrepareForOptPass/PrepareForOptPass.cpp', + 'src/PseudoInst.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/PseudoInst.cpp new file +183 -0
183 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v2 20/50] helper-to-tcg: Introduce pseudo instructions
Quoted 89 lines · On Thu, 30 Jul 2026 05:09:54 +0200 Anton Johansson <[email protected]> wrote:
"pseudo" instructions makes it easy to add custom instructions to LLVM IR in the form of calls to undefined functions. These will be used in future commits to express functionality present in TCG that is missing from LLVM IR (certain vector ops.), or to simplify the backend by collecting similar instruction mappings into a single opcode (idendity mapping).
Mapping from a call instructions in LLVM IR to an enum representing the pseudo instruction is also handled, this avoids string comparisons in the backend, and is easy to switch over.
Signed-off-by Anton Johansson <[email protected]>--- .../helper-to-tcg/include/PseudoInst.hpp | 75 +++++++ .../helper-to-tcg/include/PseudoInst.inc | 78 ++++++++ subprojects/helper-to-tcg/meson.build | 1 + subprojects/helper-to-tcg/src/PseudoInst.cpp | 183 ++++++++++++++++++ 4 files changed, 337 insertions(+) create mode 100644 subprojects/helper-to-tcg/include/PseudoInst.hpp create mode 100644 subprojects/helper-to-tcg/include/PseudoInst.inc create mode 100644 subprojects/helper-to-tcg/src/PseudoInst.cppsubprojects/helper-to-tcg/include/PseudoInst.hpp new file +61
-0@@ -0,0 +1,75 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <stdint.h> + +#include <llvm/ADT/ArrayRef.h> +#include <llvm/IR/IRBuilder.h> +#include <llvm/IR/Module.h> +#include <llvm/IR/Value.h> + +// Pseudo instructions refers to extra LLVM instructions implemented as +// calls to undefined functions. They are useful for amending LLVM IR to +// simplify mapping to TCG in the backend, e.g. +// +// %2 = call i32 @IdentityMap.i32.i16(i16 %1) +// +// is a pseudo opcode used to communicate that %1 and %2 should be mapped +// to the same value in TCG. + +enum PseudoInstArg { + ArgAny, + ArgInt, + ArgVec, + ArgPtr, + ArgLabel, + ArgVoid, +}; + +// Define an enum `PseudoInst` consisting of the names of pseudo instructions +// defined in "PseudoInst.inc". +#define PSEUDO_INST_DEF(name, ret, args) name +enum PseudoInst : uint8_t { +#include "PseudoInst.inc" +}; +#undef PSEUDO_INST_DEF + +// Retrieve string representation and argument counts for a given +// pseudo instruction. +const char *pseudoInstName(PseudoInst Inst); +uint8_t pseudoInstArgCount(PseudoInst Inst); +llvm::ArrayRef<PseudoInstArg> pseudoInstArgTypes(PseudoInst Inst); + +// Maps `PseudoInst`, return type, and argument types, to a `FunctionCallee` +// that can be called. Provided argument types are matched against +// "PseudoInst.inc" for verification and an assert triggers on failure.
subprojects/helper-to-tcg/include/PseudoInst.hpp line 71
Quoted 10 lines
subprojects/helper-to-tcg/include/PseudoInst.hpp new file +10
-0+llvm::FunctionCallee pseudoInstFunction(llvm::Module &M, PseudoInst Inst, + llvm::Type *RetType, + llvm::ArrayRef<llvm::Type *> ArgTypes); + +// Convenience function that gets a pseudo instruction via +// `pseudoInstFunction()` and creates a call to it via the provided `IRBuilder`. +llvm::CallInst *createPseudoInstCall(llvm::Module &M, + llvm::IRBuilder<> &Builder, + PseudoInst Inst, llvm::Type *RetType, + llvm::ArrayRef<llvm::Value *> Values);
This looks like a class in disguise:
```
class PseudoInst {
public:
enum Operation {
// ...
};
private:
Operation TheOperation;
public:
const char *name() const;
// ...
public:
FunctionCallee makeFunction(Module &M,
Type *RetType,
ArrayRef<llvm::Type *> ArgTypes);
FunctionCallee makeCall(...);
public:
static PseudoInst fromCall(const llvm::CallInst *Call);
};
```Quoted 12 lines
subprojects/helper-to-tcg/include/PseudoInst.hpp new file +4
-0+ +// Reverse mapping of above, takes a call instruction and attempts to map the +// callee to a `PseudoInst`. +PseudoInst getPseudoInstFromCall(const llvm::CallInst *Call);subprojects/helper-to-tcg/include/PseudoInst.inc new file +2
-0@@ -0,0 +1,78 @@ +PSEUDO_INST_DEF(InvalidPseudoInst, ArgVoid, PSEUDO_INST_ARGVEC()), +// Identity mapping
subprojects/helper-to-tcg/include/PseudoInst.inc line 12
Quoted 10 lines
subprojects/helper-to-tcg/include/PseudoInst.inc new file +10
-0+PSEUDO_INST_DEF(IdentityMap, ArgAny, PSEUDO_INST_ARGVEC(ArgAny)), +// Pointer arithmetic +PSEUDO_INST_DEF(PtrAdd, ArgPtr, PSEUDO_INST_ARGVEC(ArgPtr, ArgInt)), +// Global accesses +PSEUDO_INST_DEF(AccessGlobalArray, ArgInt, PSEUDO_INST_ARGVEC(ArgInt, ArgInt, ArgInt)), +PSEUDO_INST_DEF(AccessGlobalValue, ArgInt, PSEUDO_INST_ARGVEC(ArgInt, ArgInt)), +// Conditional branch +PSEUDO_INST_DEF(Brcond, ArgVoid, PSEUDO_INST_ARGVEC(ArgInt, ArgInt, ArgInt, ArgInt, ArgLabel, ArgLabel)), +// Conditional move +PSEUDO_INST_DEF(Movcond, ArgInt, PSEUDO_INST_ARGVEC(ArgInt, ArgInt, ArgInt, ArgInt, ArgInt)),
Add more documentation.
* AFAIU Brcond represents a icmp + br in LLVM.
* AFAIU Movcond represents icmp + select in LLVM.
* IdentityMap is not obvious
* PtrAdd is not obvious
Same for others, be more verbose about what they represent, why base LLVM is not able to capture them and what they map to in TCG. No need to go in depth for all vector operations, but for these docs would be beneficial.
Quoted 51 lines
subprojects/helper-to-tcg/include/PseudoInst.inc new file +51
-0+// Vector creation ops +PSEUDO_INST_DEF(VecSplat, ArgVec, PSEUDO_INST_ARGVEC(ArgInt)), +PSEUDO_INST_DEF(VecConstant, ArgVec, PSEUDO_INST_ARGVEC(ArgVec)), +// Vector unary ops +PSEUDO_INST_DEF(VecNot, ArgVec, PSEUDO_INST_ARGVEC(ArgVec)), +// Vector scalar binary ops +PSEUDO_INST_DEF(VecAddScalar, ArgVec, PSEUDO_INST_ARGVEC(ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecSubScalar, ArgVec, PSEUDO_INST_ARGVEC(ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecMulScalar, ArgVec, PSEUDO_INST_ARGVEC(ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecXorScalar, ArgVec, PSEUDO_INST_ARGVEC(ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecOrScalar, ArgVec, PSEUDO_INST_ARGVEC(ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecAndScalar, ArgVec, PSEUDO_INST_ARGVEC(ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecShlScalar, ArgVec, PSEUDO_INST_ARGVEC(ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecLShrScalar, ArgVec, PSEUDO_INST_ARGVEC(ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecAShrScalar, ArgVec, PSEUDO_INST_ARGVEC(ArgVec, ArgInt)), +// Vector unary ops that stores to pointer +PSEUDO_INST_DEF(VecNotStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec)), +// Vector binary ops that stores to pointer +PSEUDO_INST_DEF(VecAddStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecSubStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecMulStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecXorStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecOrStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecAndStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecShlStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecLShrStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecAShrStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecAddScalarStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecSubScalarStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecMulScalarStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecXorScalarStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecOrScalarStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecAndScalarStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecShlScalarStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecLShrScalarStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgInt)), +PSEUDO_INST_DEF(VecAShrScalarStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgInt)), +// Host memory operations +// vaddr, value sign size endian +PSEUDO_INST_DEF(GuestLoad, ArgInt, PSEUDO_INST_ARGVEC(ArgInt, ArgInt, ArgInt, ArgInt)), +PSEUDO_INST_DEF(GuestStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgInt, ArgInt, ArgInt, ArgInt)), +// ... +PSEUDO_INST_DEF(VecTruncStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgInt, ArgPtr, ArgVec)), +PSEUDO_INST_DEF(VecZExtStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgInt, ArgPtr, ArgVec)), +PSEUDO_INST_DEF(VecSExtStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgInt, ArgPtr, ArgVec)), +PSEUDO_INST_DEF(VecSignedSatAddStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecSignedSatSubStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecSelectStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgInt, ArgPtr, ArgVec, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecFunnelShrStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecAbsStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec)), +PSEUDO_INST_DEF(VecSignedMaxStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecUnsignedMaxStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)),
subprojects/helper-to-tcg/include/PseudoInst.inc line 73
Quoted 10 lines
subprojects/helper-to-tcg/include/PseudoInst.inc new file +10
-0+PSEUDO_INST_DEF(VecSignedMinStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecUnsignedMinStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecCtlzStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec)), +PSEUDO_INST_DEF(VecCttzStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec)), +PSEUDO_INST_DEF(VecCtpopStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec)), +PSEUDO_INST_DEF(VecWideCondBitsel, ArgVec, PSEUDO_INST_ARGVEC(ArgVec, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecWideCondBitselStore, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgVec, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecCompare, ArgVec, PSEUDO_INST_ARGVEC(ArgInt, ArgVec, ArgVec)), +PSEUDO_INST_DEF(VecSelect, ArgVec, PSEUDO_INST_ARGVEC(ArgInt, ArgVec, ArgVec)), +
These would also benefit from more docs:
Quoted 64 lines
subprojects/helper-to-tcg/include/PseudoInst.inc new file +5
-0+PSEUDO_INST_DEF(SignExtract, ArgInt, PSEUDO_INST_ARGVEC(ArgInt, ArgInt, ArgInt)), +PSEUDO_INST_DEF(Extract, ArgInt, PSEUDO_INST_ARGVEC(ArgInt, ArgInt, ArgInt)), + +PSEUDO_INST_DEF(Exception, ArgVoid, PSEUDO_INST_ARGVEC(ArgPtr, ArgInt)), +PSEUDO_INST_DEF(GetPC, ArgInt, PSEUDO_INST_ARGVEC()),subprojects/helper-to-tcg/meson.build +1
-0@@ -43,6 +43,7 @@ sources = [ 'src/LlvmCompat.cpp', 'src/Pipeline.cpp', 'src/PrepareForOptPass/PrepareForOptPass.cpp', + 'src/PseudoInst.cpp', ] clang = bindir / 'clang'subprojects/helper-to-tcg/src/PseudoInst.cpp new file +41
-0@@ -0,0 +1,183 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#include "PseudoInst.hpp" +#include "LlvmCompat.hpp" + +#include <llvm/ADT/DenseMap.h> +#include <llvm/ADT/Twine.h> +#include <llvm/IR/Function.h> +#include <llvm/IR/Instructions.h> +#include <llvm/Support/Casting.h> + +using namespace llvm; + +// Define an array `PseudoInstName[]`, indexed by `PseudoInst` and mapping +// to a string representation of the enum. +#define PSEUDO_INST_DEF(name, ret, args) #name +static const char *PseudoInstName[] = { +#include "PseudoInst.inc" +}; +#undef PSEUDO_INST_DEF + +// Define an array `PseudoInstArgTypes[]` indexed by `PseudoInst` and +// mapping to an array of `PseudoInstArg` representing allowed argument types. +#define PSEUDO_INST_ARGVEC(...) {__VA_ARGS__} + +#define PSEUDO_INST_DEF(name, ret, args) args
subprojects/helper-to-tcg/src/PseudoInst.cpp line 51
Quoted 10 lines
subprojects/helper-to-tcg/src/PseudoInst.cpp new file +10
-0+static const SmallVector<PseudoInstArg, 6> PseudoInstArgTypes[] = { +#include "PseudoInst.inc" +}; +#undef PSEUDO_INST_DEF +#undef PSEUDO_INST_ARGVEC + +// In order to map from a `Function *` to a `PseudoInst`, we keep a map +// of all functions created, this simplifies mapping of callees to +// a `PseudoInst` that can be switched over. +static DenseMap<Function *, PseudoInst> MapFuncToInst;
Having a llvm::Function reference in a global variable is not nice. All of this should be part of a class associated to a specific module.
subprojects/helper-to-tcg/src/PseudoInst.cpp line 62
Quoted 11 lines
subprojects/helper-to-tcg/src/PseudoInst.cpp new file +11
-0+ +// Converts llvm `Type`s to a string representation +// that can be embedded in function names for basic overloading. +// +// Ex. +// +// [8 x i8] -> "a8xi8" +// <128 x i8> -> "v128xi8" +// +// LLVM has an implementation of a similar function used by intrinsics, +// called `getMangledTypeStr`, but it's not exposed.
Is this function for debugging purposes?
Please clearly identify functions that are not critical (i.e., they are for debugging purposes).
subprojects/helper-to-tcg/src/PseudoInst.cpp line 63
Quoted 1 line
subprojects/helper-to-tcg/src/PseudoInst.cpp new file +1
-0+inline std::string getMangledTypeStr(Type *Ty) {
Use static instead of inline in .cpp files.
Quoted 91 lines
subprojects/helper-to-tcg/src/PseudoInst.cpp new file +91
-0+ std::string TypeStr = ""; + llvm::raw_string_ostream TypeStream(TypeStr); + switch (Ty->getTypeID()) { + case Type::ArrayTyID: { + auto *ArrayTy = cast<ArrayType>(Ty); + std::string ElementStr = getMangledTypeStr(ArrayTy->getElementType()); + TypeStream << "a" << ArrayTy->getNumElements() << "x" << ElementStr; + } break; + case Type::FixedVectorTyID: { + auto *VecTy = cast<VectorType>(Ty); + uint32_t ElementCount = compat::getVectorElementCount(VecTy); + std::string ElementStr = getMangledTypeStr(VecTy->getElementType()); + TypeStream << "v" << ElementCount << "x" << ElementStr; + } break; + case Type::StructTyID: { + auto *StructTy = cast<StructType>(Ty); + TypeStream << StructTy->getName(); + } break; + case Type::IntegerTyID: { + auto *IntTy = cast<IntegerType>(Ty); + TypeStream << "i" << IntTy->getBitWidth(); + } break; + case Type::PointerTyID: { + TypeStream << "p"; + } break; + default: + abort(); + } + return TypeStream.str(); +} + +// Access functions into the static defined above. + +const char *pseudoInstName(PseudoInst Inst) { return PseudoInstName[Inst]; } + +uint8_t pseudoInstArgCount(PseudoInst Inst) { + return PseudoInstArgTypes[Inst].size(); +} + +llvm::ArrayRef<PseudoInstArg> pseudoInstArgTypes(PseudoInst Inst) { + return PseudoInstArgTypes[Inst]; +} + +// Match LLVM type against `PseudoIntsArg`. +static void assertMatchingType(PseudoInstArg Expected, Type *Ty) { + const Type::TypeID Id = Ty->getTypeID(); + switch (Expected) { + case ArgAny: + return; + case ArgInt: + assert(Id == Type::IntegerTyID); + return; + case ArgVec: + assert(Id == Type::FixedVectorTyID); + return; + case ArgPtr: + assert(Id == Type::PointerTyID); + return; + case ArgLabel: + assert(Id == Type::LabelTyID); + return; + case ArgVoid: + assert(Id == Type::VoidTyID); + return; + default: + abort(); + }; +} + +llvm::FunctionCallee pseudoInstFunction(llvm::Module &M, PseudoInst Inst, + llvm::Type *RetType, + llvm::ArrayRef<llvm::Type *> ArgTypes) { + ArrayRef<PseudoInstArg> PArgTypes = pseudoInstArgTypes(Inst); + assert(PArgTypes.size() == ArgTypes.size()); + for (size_t I = 0; I < ArgTypes.size(); ++I) { + assertMatchingType(PArgTypes[I], ArgTypes[I]); + } + + auto *FT = llvm::FunctionType::get(RetType, ArgTypes, false); + + std::string FnName{PseudoInstName[Inst]}; + if (!RetType->isVoidTy()) { + FnName += "."; + FnName += getMangledTypeStr(RetType); + } + for (llvm::Type *Ty : ArgTypes) { + if (Ty->isLabelTy()) { + continue; + } + FnName += "."; + FnName += getMangledTypeStr(Ty);
subprojects/helper-to-tcg/src/PseudoInst.cpp line 164
Quoted 10 lines
subprojects/helper-to-tcg/src/PseudoInst.cpp new file +10
-0+ } + + llvm::FunctionCallee Fn = M.getOrInsertFunction(FnName, FT); + auto *F = cast<Function>(Fn.getCallee()); + MapFuncToInst.insert({F, Inst}); + + return Fn; +} + +llvm::CallInst *createPseudoInstCall(llvm::Module &M, IRBuilder<> &Builder,
You could get M from Builder.GetInsertPoint.
Quoted 21 lines
subprojects/helper-to-tcg/src/PseudoInst.cpp new file +19
-0+ PseudoInst Inst, llvm::Type *RetType, + llvm::ArrayRef<llvm::Value *> Values) { + SmallVector<Type *, 8> ArgTypes; + for (Value *V : Values) { + ArgTypes.push_back(V->getType()); + } + + FunctionCallee Fn = pseudoInstFunction(M, Inst, RetType, ArgTypes); + return Builder.CreateCall(Fn, Values); +} + +PseudoInst getPseudoInstFromCall(const CallInst *Call) { + Function *F = Call->getCalledFunction(); + auto It = MapFuncToInst.find(F); + if (It == MapFuncToInst.end()) { + return InvalidPseudoInst; + } + return It->second; +}Signature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 21/50 helper-to-tcg: Add guest vector layout description
In reply to Anton Johansson via qemu developmentVectors are represented as arrays of host-endian blocks at least 64 bits in size, the size of which is configurable. Lane 0 in each block may be placed either at the most- or least-significant bytes.
This covers a wide range of QEMU targets' vector representations.
Vector layout information is used later when mapping constant expressions such as constant vectors, and when emitting helper functions for performing size-changing operations.
--- .../helper-to-tcg/include/VectorLayout.hpp | 116 ++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100644 subprojects/helper-to-tcg/include/VectorLayout.hpp
subprojects/helper-to-tcg/include/VectorLayout.hpp new file +116 -0
@@ -0,0 +1,116 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/ADT/ArrayRef.h> +#include <stddef.h> +#include <stdint.h> + +// Vector Layout +// +// Vector layouts among QEMU guests is quite diverse, this header aims to be +// simple but still capture a large subset of targets. +// +// Guest vectors are assumed to be divided into host-endian blocks of a +// configurable size (>= 64-bit and power-of-two), with block 0 always being at +// the lowest address. Blocks in turn are divided into 64-bit columns which are +// used when interfacing with QEMUs `gvec` API. +// +// Below is a 32-byte <32 x i8> guest vector consisting of 16-byte blocks, +// 8-byte columns, and 1-byte lanes for each element in the vector. Elements +// count from 1-32 in hex to indicate their position in memory. Note this +// example is for a little-endian host with `LeastSignificant` lane preference +// meaning lane 0 is at the least significant bytes of each block. +// +// Guest vector <32 x i8> {1, 2, 3, ..., 32} +// +---------------------------------+---------------------------------+ +// | BLOCK 0 | BLOCK 1 | +// +----------------+----------------+----------------+----------------+ +// | COLUMN 0 | COLUMN 1 | COLUMN 2 | COLUMN 3 | +// |0102030405060708|090A0B0C0D0E0F10|1112131415161718|191A1B1C1D1E1F20| +// +// LOW ADDRESS HIGH ADDRESS +// +// On a big-endian host, the vector would instead be expressed in memory as +// +// Guest vector <32 x i8> {1, 2, 3, ..., 32} +// +---------------------------------+---------------------------------+ +// | BLOCK 0 | BLOCK 1 | +// +----------------+----------------+----------------+----------------+ +// | COLUMN 1 | COLUMN 0 | COLUMN 3 | COLUMN 2 | +// |100F0E0D0C0B0A09|0807060504030201|201F1E1D1C1B1A19|1817161514131211| +// +// LOW ADDRESS HIGH ADDRESS +// +// with the column order having shifted to make sure the host-endian blocks +// retain the same value. Blocks are also allowed to be equal or greater in +// size that the guest vector, at which point the entire memory view of the +// vector would reverse when going from a little to big-endian host. +// +// The lane preference may also be changed to `MostSignificant` which for the +// little-endian example places lane 0 and the highest address of block 0. +// Note, that `MostSignificant` on little-endian and `LeastSignificant` on +// big-endian hosts are equivalent in their memory representation, but will +// differ in how they are emitted in C: +// +// // MostSignificant, little endian +// uint64_t vec[] = {0x90A0B0C0D0E0F10, 0x102030405060708, ...}; +// +// // LeastSignificant, big endian +// uint64_t vec[] = {0x100F0E0D0C0B0A09, 0x807060504030201, ...}; +// + +enum LanePreference { + LeastSignificant, + MostSignificant, +}; + +struct VectorLayout { + bool HostBigEndian; + LanePreference Lane0; + size_t BlockBytes; + + inline size_t index(bool Reverse, size_t Count, size_t I, size_t J) const { + if (Reverse) { + return Count * I + (J ^ (Count - 1)); + } else { + return Count * I + J; + } + } + + // Return the index into a linear array of lane L in column C + inline size_t indexLane(size_t LanesPerColumn, size_t C, size_t L) const { + const bool Reverse = (Lane0 == MostSignificant); + return index(Reverse, LanesPerColumn, C, L); + } + + // Return the index into a linear array of + inline size_t indexBlock(size_t BlocksPerVec, size_t I) const { + const bool Reverse = (HostBigEndian ^ (Lane0 == MostSignificant)); + return index(Reverse, BlocksPerVec, 0, I); + } + + //inline uint64_t column(llvm::ArrayRef<uint64_t> Lanes) { + // uint64_t Column = 0; + // const size_t LanesPerColumn = 64 / ElementSize; + // for (size_t L = 0; L < LanesPerColumn; ++L) { + // const size_t Index = VL.indexLane(LanesPerColumn, C, L); + // Column |= Ints[Index] << (ElementSize * L); + // } + //} +};
Signature · 1 line
2.52.0
Re: [PATCH v2 21/50] helper-to-tcg: Add guest vector layout description
Quoted 100 lines · On Thu, 30 Jul 2026 05:09:55 +0200 Anton Johansson <[email protected]> wrote:
Vectors are represented as arrays of host-endian blocks at least 64 bits in size, the size of which is configurable. Lane 0 in each block may be placed either at the most- or least-significant bytes.
This covers a wide range of QEMU targets' vector representations.
Vector layout information is used later when mapping constant expressions such as constant vectors, and when emitting helper functions for performing size-changing operations.
Signed-off-by Anton Johansson <[email protected]>--- .../helper-to-tcg/include/VectorLayout.hpp | 116 ++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100644 subprojects/helper-to-tcg/include/VectorLayout.hppsubprojects/helper-to-tcg/include/VectorLayout.hpp new file +78
-0@@ -0,0 +1,116 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/ADT/ArrayRef.h> +#include <stddef.h> +#include <stdint.h> + +// Vector Layout +// +// Vector layouts among QEMU guests is quite diverse, this header aims to be +// simple but still capture a large subset of targets. +// +// Guest vectors are assumed to be divided into host-endian blocks of a +// configurable size (>= 64-bit and power-of-two), with block 0 always being at +// the lowest address. Blocks in turn are divided into 64-bit columns which are +// used when interfacing with QEMUs `gvec` API. +// +// Below is a 32-byte <32 x i8> guest vector consisting of 16-byte blocks, +// 8-byte columns, and 1-byte lanes for each element in the vector. Elements +// count from 1-32 in hex to indicate their position in memory. Note this +// example is for a little-endian host with `LeastSignificant` lane preference +// meaning lane 0 is at the least significant bytes of each block. +// +// Guest vector <32 x i8> {1, 2, 3, ..., 32} +// +---------------------------------+---------------------------------+ +// | BLOCK 0 | BLOCK 1 | +// +----------------+----------------+----------------+----------------+ +// | COLUMN 0 | COLUMN 1 | COLUMN 2 | COLUMN 3 | +// |0102030405060708|090A0B0C0D0E0F10|1112131415161718|191A1B1C1D1E1F20| +// +// LOW ADDRESS HIGH ADDRESS +// +// On a big-endian host, the vector would instead be expressed in memory as +// +// Guest vector <32 x i8> {1, 2, 3, ..., 32} +// +---------------------------------+---------------------------------+ +// | BLOCK 0 | BLOCK 1 | +// +----------------+----------------+----------------+----------------+ +// | COLUMN 1 | COLUMN 0 | COLUMN 3 | COLUMN 2 | +// |100F0E0D0C0B0A09|0807060504030201|201F1E1D1C1B1A19|1817161514131211| +// +// LOW ADDRESS HIGH ADDRESS +// +// with the column order having shifted to make sure the host-endian blocks +// retain the same value. Blocks are also allowed to be equal or greater in +// size that the guest vector, at which point the entire memory view of the +// vector would reverse when going from a little to big-endian host. +// +// The lane preference may also be changed to `MostSignificant` which for the +// little-endian example places lane 0 and the highest address of block 0. +// Note, that `MostSignificant` on little-endian and `LeastSignificant` on +// big-endian hosts are equivalent in their memory representation, but will +// differ in how they are emitted in C: +// +// // MostSignificant, little endian +// uint64_t vec[] = {0x90A0B0C0D0E0F10, 0x102030405060708, ...}; +// +// // LeastSignificant, big endian +// uint64_t vec[] = {0x100F0E0D0C0B0A09, 0x807060504030201, ...}; +// + +enum LanePreference {
subprojects/helper-to-tcg/include/VectorLayout.hpp line 88
Quoted 10 lines
subprojects/helper-to-tcg/include/VectorLayout.hpp new file +10
-0+ LeastSignificant, + MostSignificant, +}; + +struct VectorLayout { + bool HostBigEndian; + LanePreference Lane0; + size_t BlockBytes; + + inline size_t index(bool Reverse, size_t Count, size_t I, size_t J) const {
Methods defined inline are implicitly inline. Drop.
Quoted 17 lines
subprojects/helper-to-tcg/include/VectorLayout.hpp new file +17
-0+ if (Reverse) { + return Count * I + (J ^ (Count - 1)); + } else { + return Count * I + J; + } + } + + // Return the index into a linear array of lane L in column C + inline size_t indexLane(size_t LanesPerColumn, size_t C, size_t L) const { + const bool Reverse = (Lane0 == MostSignificant); + return index(Reverse, LanesPerColumn, C, L); + } + + // Return the index into a linear array of + inline size_t indexBlock(size_t BlocksPerVec, size_t I) const { + const bool Reverse = (HostBigEndian ^ (Lane0 == MostSignificant)); + return index(Reverse, BlocksPerVec, 0, I);
subprojects/helper-to-tcg/include/VectorLayout.hpp line 115
Quoted 10 lines
subprojects/helper-to-tcg/include/VectorLayout.hpp new file +10
-0+ } + + //inline uint64_t column(llvm::ArrayRef<uint64_t> Lanes) { + // uint64_t Column = 0; + // const size_t LanesPerColumn = 64 / ElementSize; + // for (size_t L = 0; L < LanesPerColumn; ++L) { + // const size_t Index = VL.indexLane(LanesPerColumn, C, L); + // Column |= Ints[Index] << (ElementSize * L); + // } + //}
Do not commit commented code.
Quoted 3 lines
subprojects/helper-to-tcg/include/VectorLayout.hpp new file +1
-0+};Signature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 22/50 helper-to-tcg: Introduce PrepareForTcgPass
In reply to Anton Johansson via qemu developmentAdds a new pass over the LLVM module which runs post-optimization with the end-goal of:
* Culling functions which aren't worth translating;
* Canonicalizing the IR to something closer to TCG, and;
* Extracting information which may be useful in the backend pass.
The bulk of IR transformations occur in the canonicalization phase, with a handful occuring later in the packend. This commits sets up a new LLVM pass over the IR module and runs it from the pipeline.
--- .../helper-to-tcg/include/CmdLineOptions.hpp | 2 ++ .../include/PrepareForTcgPass.hpp | 27 +++++++++++++++++++ subprojects/helper-to-tcg/meson.build | 1 + subprojects/helper-to-tcg/src/Pipeline.cpp | 24 +++++++++++++++++ .../PrepareForTcgPass/PrepareForTcgPass.cpp | 25 +++++++++++++++++ 5 files changed, 79 insertions(+) create mode 100644 subprojects/helper-to-tcg/include/PrepareForTcgPass.hpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/PrepareForTcgPass.cpp
subprojects/helper-to-tcg/include/CmdLineOptions.hpp +2 -0
@@ -23,3 +23,5 @@ extern llvm::cl::list<std::string> InputFiles; // Options for PrepareForOptPass extern llvm::cl::opt<bool> TranslateAllHelpers; +// Options for PrepareForTcgPass +extern llvm::cl::opt<std::string> TcgGlobalMappingsName;
subprojects/helper-to-tcg/include/PrepareForTcgPass.hpp new file +27 -0
@@ -0,0 +1,27 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/IR/PassManager.h> + +class PrepareForTcgPass : public llvm::PassInfoMixin<PrepareForTcgPass> { +public: + PrepareForTcgPass() {} + llvm::PreservedAnalyses run(llvm::Module &M, + llvm::ModuleAnalysisManager &MAM); +};
subprojects/helper-to-tcg/meson.build +1 -0
@@ -44,6 +44,7 @@ sources = [ 'src/Pipeline.cpp', 'src/PrepareForOptPass/PrepareForOptPass.cpp', 'src/PseudoInst.cpp', + 'src/PrepareForTcgPass/PrepareForTcgPass.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/Pipeline.cpp +24 -0
@@ -18,6 +18,7 @@ #include "CmdLineOptions.hpp" #include "LlvmCompat.hpp" #include "PrepareForOptPass.hpp" +#include "PrepareForTcgPass.hpp" #if LLVM_VERSION_MAJOR == 15 #include <llvm/ADT/Triple.h>
@@ -32,6 +33,7 @@ #include <llvm/IR/LLVMContext.h> #include <llvm/IR/Module.h> #include <llvm/IR/PassManager.h> +#include <llvm/IR/Verifier.h> #include <llvm/IRReader/IRReader.h> #include <llvm/InitializePasses.h> #include <llvm/Linker/Linker.h>
@@ -43,6 +45,7 @@ #include <llvm/Support/SourceMgr.h> #include <llvm/Support/TargetSelect.h> #include <llvm/Target/TargetMachine.h> +#include <llvm/Transforms/Scalar/DCE.h> #include <llvm/Transforms/Scalar/SROA.h> #define DEBUG_TYPE "pipeline"
@@ -70,6 +73,13 @@ cl::opt<bool> TranslateAllHelpers( "translate-all-helpers", cl::init(false), cl::desc("Translate all functions starting with helper_*"), cl::cat(Cat)); +// Options for PrepareForTcgPass +cl::opt<std::string> TcgGlobalMappingsName( + "tcg-global-mappings", + cl::desc("<Name of global cpu_mappings[] used for mapping accesses" + "into a struct to TCG globals>"), + cl::Required, cl::cat(Cat)); + // Define a TargetTransformInfo (TTI) subclass, this allows for overriding // common per-llvm-target information expected by other LLVM passes, such // as the width of the largest scalar/vector registers. Needed for consistent
@@ -214,5 +224,19 @@ int main(int argc, char **argv) { MPM.addPass( PB.buildModuleOptimizationPipeline(compat::OptimizationLevel::Oz, {})); + // + // Next, we run our final transformations, including removing phis and our + // own instruction combining that prioritizes instructions that map more + // easily to TCG. + // + + MPM.addPass(PrepareForTcgPass()); + MPM.addPass(VerifierPass()); + { + FunctionPassManager FPM; + FPM.addPass(DCEPass()); + MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM))); + } + return 0; }
subprojects/helper-to-tcg/src/PrepareForTcgPass/PrepareForTcgPass.cpp new file +25 -0
@@ -0,0 +1,25 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#include "PrepareForTcgPass.hpp" + +using namespace llvm; + +PreservedAnalyses PrepareForTcgPass::run(Module &M, ModuleAnalysisManager &MAM) +{ + return PreservedAnalyses::none(); +}
Signature · 1 line
2.52.0
Re: [PATCH v2 22/50] helper-to-tcg: Introduce PrepareForTcgPass
Quoted 5 lines · On Thu, 30 Jul 2026 05:09:56 +0200 Anton Johansson <[email protected]> wrote:
Adds a new pass over the LLVM module which runs post-optimization with the end-goal of:
* Culling functions which aren't worth translating;
* Canonicalizing the IR to something closer to TCG, and;
When you "canonicalize" for a backend, you're doing more "instruction selection" than canonicalization. The goal of canonicalization is to bring the IR in the most standard, frontend- and backend-agnostic representation you can.
I'd talk about a form of instruction selection.
Quoted 4 lines
* Extracting information which may be useful in the backend pass.
The bulk of IR transformations occur in the canonicalization phase, with a handful occuring later in the packend. This commits sets up a new LLVM
packend
Quoted 97 lines
pass over the IR module and runs it from the pipeline.
Signed-off-by Anton Johansson <[email protected]>--- .../helper-to-tcg/include/CmdLineOptions.hpp | 2 ++ .../include/PrepareForTcgPass.hpp | 27 +++++++++++++++++++ subprojects/helper-to-tcg/meson.build | 1 + subprojects/helper-to-tcg/src/Pipeline.cpp | 24 +++++++++++++++++ .../PrepareForTcgPass/PrepareForTcgPass.cpp | 25 +++++++++++++++++ 5 files changed, 79 insertions(+) create mode 100644 subprojects/helper-to-tcg/include/PrepareForTcgPass.hpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/PrepareForTcgPass.cppsubprojects/helper-to-tcg/include/CmdLineOptions.hpp +2
-0@@ -23,3 +23,5 @@ extern llvm::cl::list<std::string> InputFiles; // Options for PrepareForOptPass extern llvm::cl::opt<bool> TranslateAllHelpers; +// Options for PrepareForTcgPass +extern llvm::cl::opt<std::string> TcgGlobalMappingsName;subprojects/helper-to-tcg/include/PrepareForTcgPass.hpp new file +27
-0@@ -0,0 +1,27 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/IR/PassManager.h> + +class PrepareForTcgPass : public llvm::PassInfoMixin<PrepareForTcgPass> { +public: + PrepareForTcgPass() {} + llvm::PreservedAnalyses run(llvm::Module &M, + llvm::ModuleAnalysisManager &MAM); +};subprojects/helper-to-tcg/meson.build +1
-0@@ -44,6 +44,7 @@ sources = [ 'src/Pipeline.cpp', 'src/PrepareForOptPass/PrepareForOptPass.cpp', 'src/PseudoInst.cpp', + 'src/PrepareForTcgPass/PrepareForTcgPass.cpp', ] clang = bindir / 'clang'subprojects/helper-to-tcg/src/Pipeline.cpp +3
-0@@ -18,6 +18,7 @@ #include "CmdLineOptions.hpp" #include "LlvmCompat.hpp" #include "PrepareForOptPass.hpp" +#include "PrepareForTcgPass.hpp" #if LLVM_VERSION_MAJOR == 15 #include <llvm/ADT/Triple.h>@@ -32,6 +33,7 @@ #include <llvm/IR/LLVMContext.h> #include <llvm/IR/Module.h> #include <llvm/IR/PassManager.h> +#include <llvm/IR/Verifier.h> #include <llvm/IRReader/IRReader.h> #include <llvm/InitializePasses.h> #include <llvm/Linker/Linker.h>@@ -43,6 +45,7 @@ #include <llvm/Support/SourceMgr.h> #include <llvm/Support/TargetSelect.h> #include <llvm/Target/TargetMachine.h> +#include <llvm/Transforms/Scalar/DCE.h> #include <llvm/Transforms/Scalar/SROA.h> #define DEBUG_TYPE "pipeline"
subprojects/helper-to-tcg/src/Pipeline.cpp cl::opt<bool> TranslateAllHelpers( · line 82
Quoted 10 lines
subprojects/helper-to-tcg/src/Pipeline.cpp +7
-0@@ -70,6 +73,13 @@ cl::opt<bool> TranslateAllHelpers( "translate-all-helpers", cl::init(false), cl::desc("Translate all functions starting with helper_*"), cl::cat(Cat)); +// Options for PrepareForTcgPass +cl::opt<std::string> TcgGlobalMappingsName( + "tcg-global-mappings", + cl::desc("<Name of global cpu_mappings[] used for mapping accesses" + "into a struct to TCG globals>"), + cl::Required, cl::cat(Cat)); +
I'd drop this and go by convention (i.e., just assume the name) + documentation.
If you think this is useful for debugging purposes, at least give it a default so we don't need to specify the parameter in the regular invocation.
If you keep this, introduce it in the first commit using it TcgGlobalMappingsName.
Quoted 56 lines
subprojects/helper-to-tcg/src/Pipeline.cpp +14
-0// Define a TargetTransformInfo (TTI) subclass, this allows for overriding // common per-llvm-target information expected by other LLVM passes, such // as the width of the largest scalar/vector registers. Needed for consistent@@ -214,5 +224,19 @@ int main(int argc, char **argv) { MPM.addPass( PB.buildModuleOptimizationPipeline(compat::OptimizationLevel::Oz, {})); + // + // Next, we run our final transformations, including removing phis and our + // own instruction combining that prioritizes instructions that map more + // easily to TCG. + // + + MPM.addPass(PrepareForTcgPass()); + MPM.addPass(VerifierPass()); + { + FunctionPassManager FPM; + FPM.addPass(DCEPass()); + MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM))); + } + return 0; }subprojects/helper-to-tcg/src/PrepareForTcgPass/PrepareForTcgPass.cpp new file +25
-0@@ -0,0 +1,25 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#include "PrepareForTcgPass.hpp" + +using namespace llvm; + +PreservedAnalyses PrepareForTcgPass::run(Module &M, ModuleAnalysisManager &MAM) +{ + return PreservedAnalyses::none(); +}Signature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 23/50 helper-to-tcg: PrepareForTcgPass, remove functions with cycles
In reply to Anton Johansson via qemu developmentFunctions with cycles are removed for two primary reasons:
* As a simplifying assumption for register allocation which occurs down
the line, and;
* If a function contains cycles post-optimization neither unrolling or loop vectorization were deemed beneficial, and the function _might_ be better suited as a helper anyway.
Cycles are detected by iterating over Strongly Connected Components (SCCs) which imply the existence of cycles if a SCC contains more than one node, or it has a self-edge.
--- .../PrepareForTcgPass/PrepareForTcgPass.cpp | 34 +++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-)
subprojects/helper-to-tcg/src/PrepareForTcgPass/PrepareForTcgPass.cpp +32 -2
@@ -17,9 +17,39 @@ #include "PrepareForTcgPass.hpp" +#include <llvm/ADT/SCCIterator.h> +#include <llvm/ADT/SmallPtrSet.h> +#include <llvm/IR/Function.h> +#include <llvm/IR/Module.h> + using namespace llvm; -PreservedAnalyses PrepareForTcgPass::run(Module &M, ModuleAnalysisManager &MAM) -{ +static void removeFunctionsWithLoops(Module &M, ModuleAnalysisManager &MAM) { + // Iterate over all Strongly Connected Components (SCCs), a SCC implies + // the existence of loops if: + // - it has more than one node, or; + // - it has a self-edge. + SmallPtrSet<Function *, 16> FunctionsToRemove; + for (Function &F : M) { + if (F.isDeclaration()) { + continue; + } + for (auto It = scc_begin(&F); !It.isAtEnd(); ++It) { + if (It.hasCycle()) { + FunctionsToRemove.insert(&F); + break; + } + } + } + + for (Function *F : FunctionsToRemove) { + F->setComdat(nullptr); + F->deleteBody(); + } +} + +PreservedAnalyses PrepareForTcgPass::run(Module &M, + ModuleAnalysisManager &MAM) { + removeFunctionsWithLoops(M, MAM); return PreservedAnalyses::none(); }
Signature · 1 line
2.52.0
Re: [PATCH v2 23/50] helper-to-tcg: PrepareForTcgPass, remove functions with cycles
Quoted 5 lines · On Thu, 30 Jul 2026 05:09:57 +0200 Anton Johansson <[email protected]> wrote:
Functions with cycles are removed for two primary reasons:
* As a simplifying assumption for register allocation which occurs down
the line, and;
Quoted 7 lines
* If a function contains cycles post-optimization neither unrolling or loop vectorization were deemed beneficial, and the function _might_ be better suited as a helper anyway.
Cycles are detected by iterating over Strongly Connected Components (SCCs) which imply the existence of cycles if a SCC contains more than one node, or it has a self-edge.
I'd move most of the commit message to the documentation of the function.
Quoted 54 lines
Signed-off-by Anton Johansson <[email protected]>--- .../PrepareForTcgPass/PrepareForTcgPass.cpp | 34 +++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-)subprojects/helper-to-tcg/src/PrepareForTcgPass/PrepareForTcgPass.cpp +32
-2@@ -17,9 +17,39 @@ #include "PrepareForTcgPass.hpp" +#include <llvm/ADT/SCCIterator.h> +#include <llvm/ADT/SmallPtrSet.h> +#include <llvm/IR/Function.h> +#include <llvm/IR/Module.h> + using namespace llvm; -PreservedAnalyses PrepareForTcgPass::run(Module &M, ModuleAnalysisManager &MAM) -{ +static void removeFunctionsWithLoops(Module &M, ModuleAnalysisManager &MAM) { + // Iterate over all Strongly Connected Components (SCCs), a SCC implies + // the existence of loops if: + // - it has more than one node, or; + // - it has a self-edge. + SmallPtrSet<Function *, 16> FunctionsToRemove; + for (Function &F : M) { + if (F.isDeclaration()) { + continue; + } + for (auto It = scc_begin(&F); !It.isAtEnd(); ++It) { + if (It.hasCycle()) { + FunctionsToRemove.insert(&F); + break; + } + } + } + + for (Function *F : FunctionsToRemove) { + F->setComdat(nullptr); + F->deleteBody(); + } +} + +PreservedAnalyses PrepareForTcgPass::run(Module &M, + ModuleAnalysisManager &MAM) { + removeFunctionsWithLoops(M, MAM); return PreservedAnalyses::none(); }Signature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 24/50 helper-to-tcg: PrepareForTcgPass, demote PHI nodes
In reply to Anton Johansson via qemu developmentPHI nodes have no clear analogue in TCG, this commits converts them to stack accesses using a built-in LLVM transformation.
--- .../PrepareForTcgPass/PrepareForTcgPass.cpp | 23 +++++++++++++++++++ 1 file changed, 23 insertions(+)
subprojects/helper-to-tcg/src/PrepareForTcgPass/PrepareForTcgPass.cpp +23 -0
@@ -20,7 +20,10 @@ #include <llvm/ADT/SCCIterator.h> #include <llvm/ADT/SmallPtrSet.h> #include <llvm/IR/Function.h> +#include <llvm/IR/InstIterator.h> +#include <llvm/IR/Instructions.h> #include <llvm/IR/Module.h> +#include <llvm/Transforms/Utils/Local.h> using namespace llvm;
@@ -48,8 +51,28 @@ static void removeFunctionsWithLoops(Module &M, ModuleAnalysisManager &MAM) { } } +inline void demotePhis(Function &F) { + if (F.isDeclaration()) { + return; + } + + SmallVector<PHINode *, 10> Phis; + for (auto &I : instructions(F)) { + if (auto *Phi = dyn_cast<PHINode>(&I)) { + Phis.push_back(Phi); + } + } + + for (auto *Phi : Phis) { + DemotePHIToStack(Phi); + } +} + PreservedAnalyses PrepareForTcgPass::run(Module &M, ModuleAnalysisManager &MAM) { removeFunctionsWithLoops(M, MAM); + for (Function &F : M) { + demotePhis(F); + } return PreservedAnalyses::none(); }
Signature · 1 line
2.52.0
Re: [PATCH v2 24/50] helper-to-tcg: PrepareForTcgPass, demote PHI nodes
Quoted 54 lines · On Thu, 30 Jul 2026 05:09:58 +0200 Anton Johansson via qemu development…
PHI nodes have no clear analogue in TCG, this commits converts them to stack accesses using a built-in LLVM transformation.
Signed-off-by Anton Johansson <[email protected]>--- .../PrepareForTcgPass/PrepareForTcgPass.cpp | 23 +++++++++++++++++++ 1 file changed, 23 insertions(+)subprojects/helper-to-tcg/src/PrepareForTcgPass/PrepareForTcgPass.cpp +23
-0@@ -20,7 +20,10 @@ #include <llvm/ADT/SCCIterator.h> #include <llvm/ADT/SmallPtrSet.h> #include <llvm/IR/Function.h> +#include <llvm/IR/InstIterator.h> +#include <llvm/IR/Instructions.h> #include <llvm/IR/Module.h> +#include <llvm/Transforms/Utils/Local.h> using namespace llvm;@@ -48,8 +51,28 @@ static void removeFunctionsWithLoops(Module &M, ModuleAnalysisManager &MAM) { } } +inline void demotePhis(Function &F) { + if (F.isDeclaration()) { + return; + } + + SmallVector<PHINode *, 10> Phis; + for (auto &I : instructions(F)) { + if (auto *Phi = dyn_cast<PHINode>(&I)) { + Phis.push_back(Phi); + } + } + + for (auto *Phi : Phis) { + DemotePHIToStack(Phi); + } +} + PreservedAnalyses PrepareForTcgPass::run(Module &M, ModuleAnalysisManager &MAM) { removeFunctionsWithLoops(M, MAM); + for (Function &F : M) { + demotePhis(F); + } return PreservedAnalyses::none(); }Signature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 25/50 helper-to-tcg: PrepareForTcgPass, map TCG globals
In reply to Anton Johansson via qemu developmentThe input LLVM module may define an array of cpu_mapping structs, describing the mapping between fields in a specified struct (usually CPUArchState) and TCG globals.
Create a map between offsets into the specified struct and TCG globals (name, size, number of elements, stride) by iterating over the global cpu_mapping array. The name of this array is configurable via the --tcg-global-mappings flag.
--- .../include/PrepareForTcgPass.hpp | 6 +- .../helper-to-tcg/include/TcgGlobalMap.hpp | 40 ++++++++++++ subprojects/helper-to-tcg/src/Pipeline.cpp | 9 +-- .../PrepareForTcgPass/PrepareForTcgPass.cpp | 62 +++++++++++++++++++ 4 files changed, 112 insertions(+), 5 deletions(-) create mode 100644 subprojects/helper-to-tcg/include/TcgGlobalMap.hpp
subprojects/helper-to-tcg/include/PrepareForTcgPass.hpp +5 -1
@@ -17,11 +17,15 @@ #pragma once +#include "TcgGlobalMap.hpp" #include <llvm/IR/PassManager.h> class PrepareForTcgPass : public llvm::PassInfoMixin<PrepareForTcgPass> { + TcgGlobalMap &ResultTcgGlobalMap; + public: - PrepareForTcgPass() {} + PrepareForTcgPass(TcgGlobalMap &ResultTcgGlobalMap) + : ResultTcgGlobalMap(ResultTcgGlobalMap) {} llvm::PreservedAnalyses run(llvm::Module &M, llvm::ModuleAnalysisManager &MAM); };
subprojects/helper-to-tcg/include/TcgGlobalMap.hpp new file +40 -0
@@ -0,0 +1,40 @@ +#pragma once + +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#include <llvm/ADT/DenseMap.h> +#include <llvm/ADT/SmallVector.h> +#include <llvm/ADT/StringRef.h> +#include <stdint.h> + +// `TcgGlobal` describes a field in a struct which has a mapping to a TCG +// global. `Size`, `NumElements`, and `Stride` describe the mapped type, all +// types are assumed to be integer or arrays of integers. `Code` is the +// expression to be emitted when accessing the mapped global, usually the +// variable name. +struct TcgGlobal { + llvm::StringRef Code; + uint64_t Size; + uint64_t NumElements; + uint64_t Stride; +}; + +// Array of maps between offsets into a mapped struct to the resulting global +// type, outer array is indexed by the base struct type to handle multiple +// struct-to-global mappings. +using TcgGlobalMap = llvm::SmallVector<llvm::DenseMap<uint32_t, TcgGlobal>, 1>;
subprojects/helper-to-tcg/src/Pipeline.cpp +5 -4
@@ -76,9 +76,9 @@ cl::opt<bool> TranslateAllHelpers( // Options for PrepareForTcgPass cl::opt<std::string> TcgGlobalMappingsName( "tcg-global-mappings", - cl::desc("<Name of global cpu_mappings[] used for mapping accesses" - "into a struct to TCG globals>"), - cl::Required, cl::cat(Cat)); + cl::desc("Name of global cpu_mappings[] used for mapping accesses" + "into a struct to TCG globals"), + cl::init("mappings"), cl::cat(Cat)); // Define a TargetTransformInfo (TTI) subclass, this allows for overriding // common per-llvm-target information expected by other LLVM passes, such
@@ -230,7 +230,8 @@ int main(int argc, char **argv) { // easily to TCG. // - MPM.addPass(PrepareForTcgPass()); + TcgGlobalMap TcgGlobals; + MPM.addPass(PrepareForTcgPass(TcgGlobals)); MPM.addPass(VerifierPass()); { FunctionPassManager FPM;
subprojects/helper-to-tcg/src/PrepareForTcgPass/PrepareForTcgPass.cpp +62 -0
82 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v2 26/50 helper-to-tcg: PrepareForTcgPass, transform GEPs
In reply to Anton Johansson via qemu developmentgetelementpointer (GEP) instructions in LLVM IR represent general pointer arithmetic (struct field access, array indexing, ...). From the perspective of TCG, three distinct cases are important and are transformed into pseudo instructions respectively:
* Struct accesses whose offset into the struct map to a TCG global are
transformed into `call @AccessGlobalValue(structindex, offset)`;
* Struct accesses whose offset into the struct map to an array of TCG
globals are transformed into `call @AccessGlobalArray(structindex, offset, index)`;
* Otherwise, access is converted to general pointer arithmetic in LLVM IR using
`call @PtrAdd(...)`.
These three cases are treated differently in the backend and all other GEPs are considered erroneous.
--- .../include/PrepareForTcgPass.hpp | 7 +- subprojects/helper-to-tcg/meson.build | 1 + subprojects/helper-to-tcg/src/Pipeline.cpp | 2 +- .../PrepareForTcgPass/PrepareForTcgPass.cpp | 4 + .../src/PrepareForTcgPass/TransformGEPs.cpp | 355 ++++++++++++++++++ .../src/PrepareForTcgPass/TransformGEPs.hpp | 45 +++ 6 files changed, 411 insertions(+), 3 deletions(-) create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/TransformGEPs.cpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/TransformGEPs.hpp
subprojects/helper-to-tcg/include/PrepareForTcgPass.hpp +5 -2
@@ -17,15 +17,18 @@ #pragma once +#include "DebugInfo.hpp" #include "TcgGlobalMap.hpp" #include <llvm/IR/PassManager.h> class PrepareForTcgPass : public llvm::PassInfoMixin<PrepareForTcgPass> { TcgGlobalMap &ResultTcgGlobalMap; + const DebugInfoMapTy &DebugInfo; public: - PrepareForTcgPass(TcgGlobalMap &ResultTcgGlobalMap) - : ResultTcgGlobalMap(ResultTcgGlobalMap) {} + PrepareForTcgPass(TcgGlobalMap &ResultTcgGlobalMap, + const DebugInfoMapTy &DebugInfo) + : ResultTcgGlobalMap(ResultTcgGlobalMap), DebugInfo(DebugInfo) {} llvm::PreservedAnalyses run(llvm::Module &M, llvm::ModuleAnalysisManager &MAM); };
subprojects/helper-to-tcg/meson.build +1 -0
@@ -45,6 +45,7 @@ sources = [ 'src/PrepareForOptPass/PrepareForOptPass.cpp', 'src/PseudoInst.cpp', 'src/PrepareForTcgPass/PrepareForTcgPass.cpp', + 'src/PrepareForTcgPass/TransformGEPs.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/Pipeline.cpp +1 -1
@@ -231,7 +231,7 @@ int main(int argc, char **argv) { // TcgGlobalMap TcgGlobals; - MPM.addPass(PrepareForTcgPass(TcgGlobals)); + MPM.addPass(PrepareForTcgPass(TcgGlobals, DebugInfo)); MPM.addPass(VerifierPass()); { FunctionPassManager FPM;
subprojects/helper-to-tcg/src/PrepareForTcgPass/PrepareForTcgPass.cpp +4 -0
@@ -17,6 +17,7 @@ #include "PrepareForTcgPass.hpp" #include "CmdLineOptions.hpp" +#include "TransformGEPs.hpp" #include <llvm/ADT/SCCIterator.h> #include <llvm/ADT/SmallPtrSet.h>
@@ -136,5 +137,8 @@ PreservedAnalyses PrepareForTcgPass::run(Module &M, demotePhis(F); } const StringMap<size_t> TypeIndexMap = collectTcgGlobals(M, ResultTcgGlobalMap); + for (Function &F : M) { + transformGEPs(M, F, ResultTcgGlobalMap, TypeIndexMap, DebugInfo); + } return PreservedAnalyses::none(); }
subprojects/helper-to-tcg/src/PrepareForTcgPass/TransformGEPs.cpp new file +355 -0
355 lines not on this page · raw message
subprojects/helper-to-tcg/src/PrepareForTcgPass/TransformGEPs.hpp new file +45 -0
45 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v2 27/50 helper-to-tcg: PrepareForTcgPass, canonicalize IR
In reply to Anton Johansson via qemu developmentIterates over the IR with the goal of converting it to a form closer to TCG, taking care of IR discrepancies between LLVM and TCG. This also simplifies the backend by containing the bulk of custom IR transformations.
--- .../helper-to-tcg/include/CmdLineOptions.hpp | 3 + .../include/PrepareForTcgPass.hpp | 7 +- subprojects/helper-to-tcg/meson.build | 1 + subprojects/helper-to-tcg/src/Pipeline.cpp | 63 +- .../src/PrepareForTcgPass/CanonicalizeIR.cpp | 1193 +++++++++++++++++ .../src/PrepareForTcgPass/CanonicalizeIR.hpp | 24 + .../PrepareForTcgPass/PrepareForTcgPass.cpp | 2 + 7 files changed, 1290 insertions(+), 3 deletions(-) create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/CanonicalizeIR.cpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/CanonicalizeIR.hpp
subprojects/helper-to-tcg/include/CmdLineOptions.hpp +3 -0
@@ -25,3 +25,6 @@ extern llvm::cl::list<std::string> InputFiles; extern llvm::cl::opt<bool> TranslateAllHelpers; // Options for PrepareForTcgPass extern llvm::cl::opt<std::string> TcgGlobalMappingsName; +extern llvm::cl::opt<std::string> UserPCRelBranchFunc; +extern llvm::cl::opt<std::string> UserPCRelBranchFallthroughFunc; +extern llvm::cl::opt<std::string> UserPCRelBranchConditionalFunc;
subprojects/helper-to-tcg/include/PrepareForTcgPass.hpp +5 -2
@@ -19,16 +19,19 @@ #include "DebugInfo.hpp" #include "TcgGlobalMap.hpp" +#include "VectorLayout.hpp" #include <llvm/IR/PassManager.h> class PrepareForTcgPass : public llvm::PassInfoMixin<PrepareForTcgPass> { TcgGlobalMap &ResultTcgGlobalMap; const DebugInfoMapTy &DebugInfo; + const VectorLayout &VL; public: PrepareForTcgPass(TcgGlobalMap &ResultTcgGlobalMap, - const DebugInfoMapTy &DebugInfo) - : ResultTcgGlobalMap(ResultTcgGlobalMap), DebugInfo(DebugInfo) {} + const DebugInfoMapTy &DebugInfo, const VectorLayout &VL) + : ResultTcgGlobalMap(ResultTcgGlobalMap), DebugInfo(DebugInfo), VL(VL) { + } llvm::PreservedAnalyses run(llvm::Module &M, llvm::ModuleAnalysisManager &MAM); };
subprojects/helper-to-tcg/meson.build +1 -0
@@ -46,6 +46,7 @@ sources = [ 'src/PseudoInst.cpp', 'src/PrepareForTcgPass/PrepareForTcgPass.cpp', 'src/PrepareForTcgPass/TransformGEPs.cpp', + 'src/PrepareForTcgPass/CanonicalizeIR.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/Pipeline.cpp +62 -1
97 lines not on this page · raw message
subprojects/helper-to-tcg/src/PrepareForTcgPass/CanonicalizeIR.cpp new file +1193 -0
1,193 lines not on this page · raw message
subprojects/helper-to-tcg/src/PrepareForTcgPass/CanonicalizeIR.hpp new file +24 -0
@@ -0,0 +1,24 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/IR/PassManager.h> + +struct VectorLayout; + +void canonicalizeIR(llvm::Module &M, llvm::ModuleAnalysisManager &MAM, const VectorLayout &VL);
subprojects/helper-to-tcg/src/PrepareForTcgPass/PrepareForTcgPass.cpp +2 -0
@@ -16,6 +16,7 @@ // #include "PrepareForTcgPass.hpp" +#include "CanonicalizeIR.hpp" #include "CmdLineOptions.hpp" #include "TransformGEPs.hpp"
@@ -140,5 +141,6 @@ PreservedAnalyses PrepareForTcgPass::run(Module &M, for (Function &F : M) { transformGEPs(M, F, ResultTcgGlobalMap, TypeIndexMap, DebugInfo); } + canonicalizeIR(M, MAM, VL); return PreservedAnalyses::none(); }
Signature · 1 line
2.52.0
PATCH v2 28/50 helper-to-tcg: PrepareForTcgPass, identity map trivial expressions
In reply to Anton Johansson via qemu developmentTransformation of the IR, identity mapping expressions which would amount to nothing more than a move when emitted as TCG, but is required in LLVM IR to not break the IR.
Trivial expressions are mapped to a @IdentityMap pseudo instruction allowing them to be dealt with in a uniform manner down the line.
--- subprojects/helper-to-tcg/meson.build | 1 + .../src/PrepareForTcgPass/IdentityMap.cpp | 91 +++++++++++++++++++ .../src/PrepareForTcgPass/IdentityMap.hpp | 39 ++++++++ .../PrepareForTcgPass/PrepareForTcgPass.cpp | 4 + 4 files changed, 135 insertions(+) create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/IdentityMap.cpp create mode 100644 subprojects/helper-to-tcg/src/PrepareForTcgPass/IdentityMap.hpp
subprojects/helper-to-tcg/meson.build +1 -0
@@ -47,6 +47,7 @@ sources = [ 'src/PrepareForTcgPass/PrepareForTcgPass.cpp', 'src/PrepareForTcgPass/TransformGEPs.cpp', 'src/PrepareForTcgPass/CanonicalizeIR.cpp', + 'src/PrepareForTcgPass/IdentityMap.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/PrepareForTcgPass/IdentityMap.cpp new file +91 -0
91 lines not on this page · raw message
subprojects/helper-to-tcg/src/PrepareForTcgPass/IdentityMap.hpp new file +39 -0
@@ -0,0 +1,39 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/IR/Function.h> +#include <llvm/IR/Module.h> + +// +// Transformation of the IR, taking what would become trivial unary operations +// and maps them to a single @IdentityMap pseudo instruction. +// +// To motivate further, in order to produce nice IR on the other end, generally +// the operands of these trivial expressions needs to be forwarded and treated +// as the destination value (identity mapped). However, directly removing these +// instructions will result in broken LLVM IR (consider zext i8, i32 where both +// the source and destination would map to TCGv_i32). +// +// Moreover, handling these identity mapped values in an adhoc way quickly +// becomes cumbersome and spreads throughout the codebase. Therefore, +// introducing @IdentityMap allows code further down the pipeline to ignore the +// source of the identity map. +// + +void identityMap(llvm::Module &M, llvm::Function &F);
subprojects/helper-to-tcg/src/PrepareForTcgPass/PrepareForTcgPass.cpp +4 -0
@@ -18,6 +18,7 @@ #include "PrepareForTcgPass.hpp" #include "CanonicalizeIR.hpp" #include "CmdLineOptions.hpp" +#include "IdentityMap.hpp" #include "TransformGEPs.hpp" #include <llvm/ADT/SCCIterator.h>
@@ -142,5 +143,8 @@ PreservedAnalyses PrepareForTcgPass::run(Module &M, transformGEPs(M, F, ResultTcgGlobalMap, TypeIndexMap, DebugInfo); } canonicalizeIR(M, MAM, VL); + for (Function &F : M) { + identityMap(M, F); + } return PreservedAnalyses::none(); }
Signature · 1 line
2.52.0
PATCH v2 29/50 helper-to-tcg: Introduce TcgV structure
In reply to Anton Johansson via qemu developmentAdds a struct representing everything a LLVM value might map to in TCG, this includes:
* TCGv_i* (IrValue);
* TCGv_ptr (IrPtr);
* TCGLabel (IrLabel);
* 123123ull (IrImmediate);
* intptr_t (IrPtrToOffset).
Every TcgV value is assigned an increasing identifier, a kind (list above), a size, and optinally a name along with some extra flags. Two values are considered equal if their identifiers are equal. For instance, this can mean one identifier might describe different LLVM sizes at different points in a function, which is very useful when mapping semantics from LLVM IR to TCG.
--- subprojects/helper-to-tcg/include/TcgType.hpp | 276 ++++++++++++++++++ 1 file changed, 276 insertions(+) create mode 100644 subprojects/helper-to-tcg/include/TcgType.hpp
subprojects/helper-to-tcg/include/TcgType.hpp new file +276 -0
@@ -0,0 +1,276 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include "Error.hpp" +#include "LlvmCompat.hpp" + +#include <llvm/ADT/ArrayRef.h> +#include <llvm/ADT/StringRef.h> +#include <llvm/IR/DerivedTypes.h> +#include <llvm/Support/raw_ostream.h> + +#include <assert.h> +#include <stdint.h> +#include <string> +#include <utility> // std::pair + +// Data representing allowed vector and value sizes. + +// Sizes corresponding to a TCGv_i* register. +enum AllowedTcgSize : uint8_t { + T32 = 32, + T64 = 64, + T128 = 128, +}; + +// Sizes corresponding of allowed LLVM values, operations smaller than +// `AllowedTcgSize` act on these sizes. +enum AllowedLlvmSize : uint8_t { + I1 = 1, + I8 = 8, + I16 = 16, + I32 = 32, + I64 = 64, +}; + +// Despite limiting sizes to an enum, when assigned from runtime values they may +// still differ. +inline bool verifyLlvmSize(AllowedLlvmSize S) { + switch (S) { + case I1: + case I8: + case I16: + case I32: + case I64: + return true; + default: + return false; + }; +} + +// Size of an integer value, combining a TCG and LLVM size. +struct ValueSize { + AllowedTcgSize TcgBitWidth; + AllowedLlvmSize LlvmBitWidth; + + static llvm::Expected<ValueSize> fromBitWidth(size_t BitWidth) { + ValueSize Ret; + if (BitWidth <= 32) { + Ret.TcgBitWidth = T32; + } else if (BitWidth <= 64) { + Ret.TcgBitWidth = T64; + } else { + Ret.TcgBitWidth = T128; + } + Ret.LlvmBitWidth = (AllowedLlvmSize)BitWidth; + // There's no runtime sanity checks that an enum is not just any integer + // value, so verify manually. + if (verifyLlvmSize(Ret.LlvmBitWidth)) { + return Ret; + } else { + return mkError("Invalid bit width"); + } + } + + static llvm::Expected<ValueSize> fromLlvmType(llvm::IntegerType *Ty) { + return fromBitWidth(Ty->getBitWidth()); + } +}; + +struct VectorSize { + uint8_t ElementCount; + AllowedLlvmSize ElementBitWidth; + + inline size_t bytes() const { return (ElementCount * ElementBitWidth) / 8; } + + static llvm::Expected<VectorSize> fromLlvmType(llvm::VectorType *Ty) { + auto *IntTy = llvm::dyn_cast<llvm::IntegerType>(Ty->getElementType()); + if (!IntTy) { + return mkError( + "Vectors of non-integer element type not supported!\n"); + } + VectorSize Ret{ + (uint8_t)compat::getVectorElementCount(Ty), + (AllowedLlvmSize)IntTy->getBitWidth(), + }; + // There's no runtime sanity checks that an enum is not just any integer + // value, so verify manually. + if (verifyLlvmSize(Ret.ElementBitWidth)) { + return Ret; + } else { + return mkError("Invalid bit width"); + } + } +}; + +union TcgSize { + VectorSize Vec; + ValueSize Val; +}; + +// clang-format off +enum TcgKind : uint8_t { + IrInvalid = 0, + IrValue = 1, // TCG register value (TCGv_i*) + IrImmediate = 2, // Immediate argument to TCG operation ([u]int*_t) + IrPtr = 4, // TCG host pointer (TCGv_ptr) + IrPtrToOffset = 8, // Target "gvec" vector (intptr_t) + IrLabel = 16, // TCG label (TCGv_label) +}; +// clang-format on + +// Describes a single value to be output in TCG, discriminated by `TcgKind`. +// +// This data is designed to be copied around and each value is identified via a +// unique `Id`, as a result the "same" `TcgV` can take on e.g. different sizes +// at different points in function, which allows one TCG variable to represent +// multiple smaller "logical" LLVM sizes. +// +// TODO: +// +// * There is overlap between `Kind == IrImmediate` and `(ConstantExpression +// and Kind == IrValue)` that is not obvious, the condition `!Name.empty()` +// is also rather similar to `ConstantExpression` in a lot of cases. +// +// * `Name` should not be string, this is resulting in redundant copies and is +// also completely unnecessary given our usage pattern. `Name` is never +// modified, and all strings are created by the same few passes. A better +// solution would be to use a `StringSaver/BumpAllocator` in `CEmitter` and +// use `StringRef`s everywhere else. +// +struct TcgV { + uint16_t Id = 0; + TcgKind Kind = IrInvalid; + TcgSize Size; + bool ConstantExpression = false; + std::string Name = ""; + + // Static id used to identify `TcgV`s + inline static uint16_t IdCounter = 0; + static void resetId() { IdCounter = 0; } + static uint16_t getId() { return IdCounter++; } + + // Helper functions for accessing size fields. + inline ValueSize intSize() const { + assert(Kind == IrImmediate or Kind == IrValue); + return Size.Val; + } + + inline VectorSize vecSize() const { + assert(Kind == IrPtrToOffset); + return Size.Vec; + } + + // Return as `int` to reduce casts when printing and to ease comparisons. + inline int tcgBitWidth() const { return intSize().TcgBitWidth; } + inline int llvmBitWidth() const { return intSize().LlvmBitWidth; } + + static TcgV makeVector(VectorSize Size) { + return TcgV({}, {.Vec = Size}, IrPtrToOffset); + } + + static TcgV makeImmediate(llvm::StringRef Name, ValueSize Size) { + return TcgV(Name.str(), {.Val = Size}, IrImmediate); + } + + static TcgV makeTemp(ValueSize Size, TcgKind Kind) { + return TcgV({}, {.Val = Size}, Kind); + } + + static TcgV makeConstantExpression(llvm::StringRef Expression, + ValueSize Size, TcgKind Kind) { + TcgV Tcg(Expression.str(), {.Val = Size}, Kind); + Tcg.ConstantExpression = true; + return Tcg; + } + + static TcgV makeLabel() { return TcgV("", {.Val = {T32, I32}}, IrLabel); } + + TcgV() = default; + + TcgV(std::string Name, TcgSize Size, TcgKind Kind) + : Id(getId()), Kind(Kind), Size(Size), Name(Name) {} + + // Equality between two values it determined only by the assigned id, + // consider a 16- to 8-bit truncation: + // + // %1 = trunc i8, i16 %0, + // + // which after identity mapping becomes + // + // %1 = call i8 @IdentityMap.i16.i8(i16 %0). + // + // Identity mapping will copy the `TcgV` assigned to `%0` to `%1`, but `%1` + // will retain its `LlvmSize` needed to emit correctly sized operations down + // the line. Despite differing in `LlvmSize` both `%0` and `%1` will be + // emitted as the same TCG value with type `TCGv_i32`. + bool operator==(const TcgV &Other) const { return Other.Id == Id; } + bool operator!=(const TcgV &Other) const { return !operator==(Other); } + + // Print out struct fields, useful for debugging. + inline void dump(llvm::raw_ostream &Out) const { + Out << "TcgV " << Id; + if (!Name.empty()) { + Out << " (" << Name << ")"; + } + Out << ":\n"; + Out << " Kind: "; + switch (Kind) { + case IrInvalid: + Out << "IrInvalid\n"; + break; + case IrValue: + Out << "IrValue\n"; + break; + case IrImmediate: + Out << "IrImmediate\n"; + break; + case IrPtr: + Out << "IrPtr\n"; + break; + case IrPtrToOffset: + Out << "IrPtrToOffset\n"; + break; + case IrLabel: + Out << "IrLabel\n"; + break; + } + if (Kind == IrPtrToOffset) { + Out << " VectorElementCount: " << (int)Size.Vec.ElementCount + << "\n"; + Out << " VectorElementBitWidth: " << (int)Size.Vec.ElementBitWidth + << "\n"; + } else { + Out << " TcgSize: " << (int)Size.Val.TcgBitWidth << "\n"; + Out << " LlvmSize: " << (int)Size.Val.LlvmBitWidth << "\n"; + } + Out << " ConstantExpression: " << ConstantExpression << "\n"; + } +}; + +// Helper function for verifying that a set of `TcgV` are of acceptable types, +// use with flags like +// +// assertKinds({{Dst, IrPtr}, {Src, IrValue | IrImmediate})) +// +inline void assertKinds(llvm::ArrayRef<std::pair<const TcgV, uint8_t>> Pairs) { + for (auto &[Tcg, Flag] : Pairs) { + assert(Tcg.Kind & Flag); + } +}
Signature · 1 line
2.52.0
PATCH v2 30/50 helper-to-tcg: Introduce TcgGenPass
In reply to Anton Johansson via qemu developmentAdds a final backend pass over the LLVM module, taking previously optimized and canonicalized LLVM IR and for each function:
1. Maps arguments, and propagates constant expressions;
2. Allocates TCG values to non-constants;
3. Emits the final TCG.
Points above will be implemented in following commits, this commits adds boilerplate and some scaffolding for translating and emitting code.
TODO: This commit could do with some more splitting, TcgEmit.hpp and ValueMapping.hpp and the functions therein are referenced but not added in this commit, they come later.
--- .../helper-to-tcg/include/CmdLineOptions.hpp | 16 + .../helper-to-tcg/include/TcgGenPass.hpp | 63 +++ subprojects/helper-to-tcg/meson.build | 1 + subprojects/helper-to-tcg/src/Pipeline.cpp | 97 ++++- .../src/TcgGenPass/TcgGenPass.cpp | 385 ++++++++++++++++++ 5 files changed, 561 insertions(+), 1 deletion(-) create mode 100644 subprojects/helper-to-tcg/include/TcgGenPass.hpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/TcgGenPass.cpp
subprojects/helper-to-tcg/include/CmdLineOptions.hpp +16 -0
@@ -28,3 +28,19 @@ extern llvm::cl::opt<std::string> TcgGlobalMappingsName; extern llvm::cl::opt<std::string> UserPCRelBranchFunc; extern llvm::cl::opt<std::string> UserPCRelBranchFallthroughFunc; extern llvm::cl::opt<std::string> UserPCRelBranchConditionalFunc; +// Options for TcgEmit +extern llvm::cl::opt<std::string> MmuIndexFunction; +extern llvm::cl::opt<std::string> TempVectorBlock; +extern llvm::cl::opt<uint32_t> MaxVectorInstructions; +extern llvm::cl::opt<uint32_t> MaxVectorTempBytes; +// Options for MapTemporaries +extern llvm::cl::opt<uint32_t> GuestPtrSize; +// Options for TcgGenPass +extern llvm::cl::opt<std::string> OutputSourceFile; +extern llvm::cl::opt<std::string> OutputHeaderFile; +extern llvm::cl::opt<std::string> OutputEnabledFile; +extern llvm::cl::opt<bool> ErrorOnTranslationFailure; +extern llvm::cl::opt<bool> StaticOutput; +extern llvm::cl::opt<bool> AllowDeclCall; +extern llvm::cl::opt<bool> ForwardContext; +extern llvm::cl::opt<bool> EmitVectorPreamble;
subprojects/helper-to-tcg/include/TcgGenPass.hpp new file +63 -0
63 lines not on this page · raw message
subprojects/helper-to-tcg/meson.build +1 -0
@@ -48,6 +48,7 @@ sources = [ 'src/PrepareForTcgPass/TransformGEPs.cpp', 'src/PrepareForTcgPass/CanonicalizeIR.cpp', 'src/PrepareForTcgPass/IdentityMap.cpp', + 'src/TcgGenPass/TcgGenPass.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/Pipeline.cpp +96 -1
126 lines not on this page · raw message
subprojects/helper-to-tcg/src/TcgGenPass/TcgGenPass.cpp new file +385 -0
385 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v2 30/50] helper-to-tcg: Introduce TcgGenPass
subprojects/helper-to-tcg/src/Pipeline.cpp int main(int argc, char **argv) { · line 389
Quoted 10 lines · On Thu, 30 Jul 2026 05:10:04 +0200 Anton Johansson via qemu development…
subprojects/helper-to-tcg/src/Pipeline.cpp +10
-0+ std::error_code EC; + ToolOutputFile OutSource(OutputSourceFile, EC, compat::OpenFlags); + ToolOutputFile OutHeader(OutputHeaderFile, EC, compat::OpenFlags); + ToolOutputFile OutHelpers(OutputHelpersFile, EC, compat::OpenFlags); + ToolOutputFile OutEnabled(OutputEnabledFile, EC, compat::OpenFlags); + assert(!EC); + + MPM.addPass(TcgGenPass(OutSource.os(), OutHeader.os(), OutHelpers.os(), + OutEnabled.os(), OutputHeaderFile, Annotations, + DebugInfo, TcgGlobals, VL));
I'd move all the cl::opt from this file to TcgGenPass. Ideally, passes should not have constructor arguments and should be runnable from opt.
Make an effort to have all the passes be default constructible.
Quoted 9 lines
subprojects/helper-to-tcg/src/Pipeline.cpp +7
-0+ MPM.run(*M.get(), MAM); + + OutSource.keep(); + OutHeader.keep(); + OutHelpers.keep(); + OutEnabled.keep(); + return 0; }
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
Re: [PATCH v2 30/50] helper-to-tcg: Introduce TcgGenPass
Quoted 14 lines · Alessandro Di Federico via qemu development
subprojects/helper-to-tcg/src/Pipeline.cpp int main(int argc, char **argv) { · line 389
Quoted 10 lines · On Thu, 30 Jul 2026 05:10:04 +0200 Anton Johansson via qemu development…
subprojects/helper-to-tcg/src/Pipeline.cpp +10
-0+ std::error_code EC; + ToolOutputFile OutSource(OutputSourceFile, EC, compat::OpenFlags); + ToolOutputFile OutHeader(OutputHeaderFile, EC, compat::OpenFlags); + ToolOutputFile OutHelpers(OutputHelpersFile, EC, compat::OpenFlags); + ToolOutputFile OutEnabled(OutputEnabledFile, EC, compat::OpenFlags); + assert(!EC); + + MPM.addPass(TcgGenPass(OutSource.os(), OutHeader.os(), OutHelpers.os(), + OutEnabled.os(), OutputHeaderFile, Annotations, + DebugInfo, TcgGlobals, VL));
Quoted 5 lines
I'd move all the
cl::optfrom this file toTcgGenPass. Ideally, passes should not have constructor arguments and should be runnable fromopt.Make an effort to have all the passes be default constructible.
This is a good idea, maintaining all the cl::opt changes in the patchset was also really cumbersome, this is easier.
I can make it so passes are runnable from opt, but I'm not sure how useful it would be since our pipeline sets TargetTransformInfo, meaning if you run all the passes independently from opt, you wouldn't get the same result. So you couldn't get to a point where you can run TcgGenPass independently since you wouldn't have access to the correct input IR.
Signature · 2 lines
Anton Johansson rev.ng Labs Srl.
PATCH v2 31/50 helper-to-tcg: TcgGenPass, linearize basic blocks
In reply to Anton Johansson via qemu developmentPasses over the IR that will follow all depend on a specific iteration order over the IR that must match between passes, establish that iteration order. Also replace conditional branches with a pseudo instruction closer in semantics to TCG, this breaks the IR verification, but as this is the last transformation of the IR in this tool that's acceptable.
--- subprojects/helper-to-tcg/meson.build | 1 + .../src/TcgGenPass/LinearizeBlocks.cpp | 140 ++++++++++++++++++ .../src/TcgGenPass/LinearizeBlocks.hpp | 32 ++++ 3 files changed, 173 insertions(+) create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.hpp
subprojects/helper-to-tcg/meson.build +1 -0
@@ -49,6 +49,7 @@ sources = [ 'src/PrepareForTcgPass/CanonicalizeIR.cpp', 'src/PrepareForTcgPass/IdentityMap.cpp', 'src/TcgGenPass/TcgGenPass.cpp', + 'src/TcgGenPass/LinearizeBlocks.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp new file +140 -0
140 lines not on this page · raw message
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.hpp new file +32 -0
@@ -0,0 +1,32 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/ADT/SmallVector.h> +#include <llvm/IR/PassManager.h> + +namespace llvm { +class BasicBlock; +class Function; +class Module; +} // namespace llvm + +using LinearBlocks = llvm::SmallVector<llvm::BasicBlock *, 16>; +LinearBlocks linearizeBlocks(llvm::Module &M, + llvm::FunctionAnalysisManager &FAM, + llvm::Function &F);
Signature · 1 line
2.52.0
Re: [PATCH v2 31/50] helper-to-tcg: TcgGenPass, linearize basic blocks
Quoted 4 lines · On Thu, 30 Jul 2026 05:10:05 +0200 Anton Johansson via qemu development…
Passes over the IR that will follow all depend on a specific iteration order over the IR that must match between passes, establish that iteration order. Also replace conditional branches with a pseudo instruction closer in semantics to TCG, this breaks the IR verification,
Are you saying the IR is invalid after this pass?
That's not great. If the problem is the lack of terminators, just put an unreachable instruction. If you don't want to put unreachable because DCE than drops code, add a call to a opaque noreturn function. Or maybe a branch to a sink block.
I know it's not nice, but if you give up on verification to avoid checking X, you also lose the check for Y, Z and so on.
I'd be strict and make sure the result of each pass is valid, no exceptions.
Quoted 2 lines
but as this is the last transformation of the IR in this tool that's acceptable.
We run DCE after this AFAIU, so it's not the last transformation.
Quoted 64 lines
Signed-off-by Anton Johansson <[email protected]>--- subprojects/helper-to-tcg/meson.build | 1 + .../src/TcgGenPass/LinearizeBlocks.cpp | 140 ++++++++++++++++++ .../src/TcgGenPass/LinearizeBlocks.hpp | 32 ++++ 3 files changed, 173 insertions(+) create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.hppsubprojects/helper-to-tcg/meson.build +1
-0@@ -49,6 +49,7 @@ sources = [ 'src/PrepareForTcgPass/CanonicalizeIR.cpp', 'src/PrepareForTcgPass/IdentityMap.cpp', 'src/TcgGenPass/TcgGenPass.cpp', + 'src/TcgGenPass/LinearizeBlocks.cpp', ] clang = bindir / 'clang'subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp new file +37
-0@@ -0,0 +1,140 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#include "LinearizeBlocks.hpp" +#include "PseudoInst.hpp" + +#include <llvm/ADT/PostOrderIterator.h> +#include <llvm/ADT/SmallPtrSet.h> +#include <llvm/ADT/SmallSet.h> +#include <llvm/ADT/SmallVector.h> +#include <llvm/Analysis/RegionInfo.h> +#include <llvm/Analysis/VectorUtils.h> +#include <llvm/IR/Constants.h> +#include <llvm/IR/DerivedTypes.h> +#include <llvm/IR/Dominators.h> +#include <llvm/IR/Function.h> +#include <llvm/IR/IRBuilder.h> +#include <llvm/IR/InstIterator.h> +#include <llvm/IR/InstrTypes.h> +#include <llvm/IR/Instruction.h> +#include <llvm/IR/Instructions.h> +#include <llvm/IR/Intrinsics.h> +#include <llvm/IR/Module.h>
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp line 47
Quoted 10 lines
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp new file +10
-0+#include <llvm/IR/PassManager.h> +#include <llvm/IR/PatternMatch.h> +#include <llvm/Support/Casting.h> + +using namespace llvm; +using namespace PatternMatch; + +// Needed to track and remove instructions not handled by a subsequent dead code +// elimination, this applies to calls to pseudo instructions in particular. +using EraseInstVec = SmallVector<Instruction *, 16>;
Just InstructionVector? The additional meaning can be embedded in the name of the instance.
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp line 51
Quoted 4 lines
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp new file +4
-0+ +static void convertICmpBrToPseudInst(LLVMContext &Context, RegionInfo &RI, + EraseInstVec &InstToErase, Module &M, + Instruction *I, BasicBlock *NextBB) {
Docs: why do we do this?
Also, document the fact that this is an optimization, the backend can emit regular conditional branches.
Quoted 24 lines
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp new file +24
-0+ auto *ICmp = dyn_cast<ICmpInst>(I); + if (!ICmp) { + return; + } + + // Since we want to remove the icmp instruction we ensure that + // all uses are branch instructions that can be converted into + // @brcond.* calls. + for (User *U : ICmp->users()) { + if (!isa<BranchInst>(U)) { + return; + } + } + + Value *Op0 = ICmp->getOperand(0); + Value *Op1 = ICmp->getOperand(1); + auto *CmpIntTy = dyn_cast<IntegerType>(Op0->getType()); + if (!CmpIntTy) { + return; + } + for (User *U : ICmp->users()) { + auto *Br = cast<BranchInst>(U); + + BasicBlock *True = Br->getSuccessor(0);
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp line 85
Quoted 10 lines
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp new file +10
-0+ BasicBlock *False = Br->getSuccessor(1); + + IRBuilder<> Builder(Br); + + // TODO: This is strange, indeally the unreachable branch should have + // been optimized out, here we invert the conditional branch if we can + // fallthrough to the reachable branch. + bool TrueUnreachable = + True->getTerminator()->getOpcode() == Instruction::Unreachable and + False->getTerminator()->getOpcode() != Instruction::Unreachable;
I'd consider running simplify-cfg, which should do this for you. simplify-cfg is a bit invasive, maybe you can find something simpler bisecting the -O2 pipeline.
If you end up doing this by hand, I'd like at least to see a comment like "we're not using simplify-cfg because...".
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp line 89
Quoted 4 lines
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp new file +4
-0+ + // If the next basic block is either of our true/false + // branches, we can fallthrough instead of branching. + bool Fallthrough = (NextBB == True or NextBB == False);
I now understand why you call this linearizeBlocks, explain in docs above the function declaration.
Are sure you want a pseudo instruction for this?
We could have the transformation "canonicalize" the conditional branch into having the fallthrough block to be the first successor and then have the backend detect this and optimize emission on the fly.
I'd prefer to keep the number of pseudoinstructions to the minimum, but there's also an argument for keeping the backend as trivial as possible (at the cost of more pseudo-instructions).
Up to you, no biggie.
Quoted 23 lines
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp new file +23
-0+ + // If the succeeding basic block is the true branch we + // invert the condition so we can fallthrough instead. + ICmpInst::Predicate Predicate; + if (NextBB == True or (TrueUnreachable and NextBB == False)) { + std::swap(True, False); + Predicate = ICmp->getInversePredicate(); + } else { + Predicate = ICmp->getPredicate(); + } + + createPseudoInstCall(M, Builder, Brcond, Builder.getVoidTy(), + {Builder.getInt8(Fallthrough), + ConstantInt::get(CmpIntTy, Predicate), Op0, Op1, + True, False}); + + InstToErase.push_back(Br); + } + InstToErase.push_back(ICmp); +} + +LinearBlocks linearizeBlocks(Module &M, FunctionAnalysisManager &FAM, + Function &F) {
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp line 122
Quoted 10 lines
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp new file +10
-0+ assert(!F.isDeclaration()); + + LinearBlocks Blocks{}; + + LLVMContext &Context = F.getContext(); + EraseInstVec InstToErase; + auto &RI = FAM.getResult<RegionInfoAnalysis>(F); + + ReversePostOrderTraversal<Function *> RPOT(&F); + for (auto BBIt = RPOT.begin(); BBIt != RPOT.end(); ++BBIt) {
Use range-based for loop.
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp line 126
Quoted 4 lines
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp new file +4
-0+ Blocks.push_back(*BBIt); + } + + for (int i = 0; i < Blocks.size(); ++i) {
Use range-based for loop + llvm::zip(Blocks, skip_first(Blocks), something like this.
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp line 130
Quoted 4 lines
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp new file +4
-0+ BasicBlock *BB = Blocks[i]; + BasicBlock *NextBB = (i + 1 < Blocks.size()) ? Blocks[i + 1] : nullptr; + for (Instruction &I : *BB) { + convertICmpBrToPseudInst(Context, RI, InstToErase, M, &I, NextBB);
It's quite unexpected for a function called linearizeBlocks to do convertICmpBrToPseudInst. Either split or add good documentation on the function to convey the role of this function.
My initial intuition was: just reorder blocks in reverse post order.
Quoted 50 lines
subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.cpp new file +10
-0+ } + } + + // Finally clean up instructions we need to remove manually + for (Instruction *I : InstToErase) { + I->eraseFromParent(); + } + + return Blocks; +}subprojects/helper-to-tcg/src/TcgGenPass/LinearizeBlocks.hpp new file +32
-0@@ -0,0 +1,32 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include <llvm/ADT/SmallVector.h> +#include <llvm/IR/PassManager.h> + +namespace llvm { +class BasicBlock; +class Function; +class Module; +} // namespace llvm + +using LinearBlocks = llvm::SmallVector<llvm::BasicBlock *, 16>; +LinearBlocks linearizeBlocks(llvm::Module &M, + llvm::FunctionAnalysisManager &FAM, + llvm::Function &F);Signature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 32/50 helper-to-tcg: TcgGenPass, introduce Value <-> TcgV map
In reply to Anton Johansson via qemu developmentAdds a TempAllocationData structure which will be the primary struct holding information about mappings between Values and TcgVs.
Any identified return value is also stored along with flags for easier use by other passes. All functions which populate TempAllocationData will be declared in this header.
--- .../src/TcgGenPass/ValueMapping.hpp | 110 ++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/ValueMapping.hpp
subprojects/helper-to-tcg/src/TcgGenPass/ValueMapping.hpp new file +110 -0
@@ -0,0 +1,110 @@ +// +// Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved. +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with this program; if not, see <http://www.gnu.org/licenses/>. +// + +#pragma once + +#include "DebugInfo.hpp" +#include "FunctionAnnotation.hpp" +#include "LinearizeBlocks.hpp" +#include "TcgEmit.hpp" +#include "TcgGlobalMap.hpp" +#include "TcgType.hpp" + +#include <llvm/ADT/SmallSet.h> +#include <llvm/ADT/StringRef.h> +#include <llvm/IR/BasicBlock.h> +#include <llvm/IR/Instructions.h> +#include <llvm/Support/Error.h> + +// +// Value Mapping +// +// Data structures and functions needed for mapping various LLVM `Value`s to +// `TcgV`s. +// + +namespace llvm { +class Function; +} + +// Flags reprensting common special cases for function return values, used when +// emitted TCG to produce better output. +enum TempAllocationFlags { + SkipReturnMov = 1, + ReturnsImmediate = 2, + ReturnsValue = 4, + + HasReturn = (ReturnsImmediate | ReturnsValue), +}; + +// Main data structure responsible for holding mappings between LLVM `Value`s +// and `TcgV`s, populated by functions declared below. +struct TempAllocationData { + // Mapping of LLVM Values to the corresponding TcgV + llvm::DenseMap<const llvm::Value *, TcgV> Map; + + // Whether or not the final mov in an instruction can safely + // be ignored or not. + uint8_t flags = 0; + TcgV ReturnValue; + + inline bool hasReturnValue() const { return flags & HasReturn; } + + inline TcgV map(const llvm::Value *V, const TcgV &T) { + return Map.try_emplace(V, T).first->second; + } +}; + +inline const llvm::iterator_range<llvm::User::const_op_iterator> +getOperands(const llvm::Instruction *const I) { + switch (I->getOpcode()) { + case Instruction::GetElementPtr: + return llvm::cast<llvm::GetElementPtrInst>(I)->operands(); + case Instruction::Call: + return llvm::cast<llvm::CallInst>(I)->args(); + default: + return I->operands(); + } +} + +// Defined in MapArguments.cpp +llvm::Error mapArguments(const llvm::Function &F, + const AnnotationMapTy &AnnotationMap, + const DebugInfoMapTy &DebugInfo, + TempAllocationData &TAD); + +// Defined in MapConstantExpressions.cpp +llvm::Error propagateConstantExpressions(CEmitter &C, const llvm::Function &F, + const LinearBlocks &Blocks, + const AnnotationMapTy &AnnotationMap, + const DebugInfoMapTy &DebugInfo, + const TcgGlobalMap &TcgGlobals, + TempAllocationData &TAD); +// Defined in MapTemporaries.cpp +llvm::Error allocateTemporaries(const llvm::Function &F, + const LinearBlocks &Blocks, + const AnnotationMapTy &AnnotationMap, + const DebugInfoMapTy &DebugInfo, CEmitter &C, + const TcgGlobalMap &TcgGlobal, + TempAllocationData &TAD); + +// Defined MapTcgOperations.cpp +llvm::Error mapTcgOperations( + const LinearBlocks &Blocks, const TcgGlobalMap &TcgGlobals, + const AnnotationMapTy &AnnotationMap, + const llvm::SmallPtrSet<llvm::Function *, 16> &HasTranslatedFunction, + const TempAllocationData &TAD, TcgEmitter &Tcg, CEmitter &C);
Signature · 1 line
2.52.0
PATCH v2 33/50 helper-to-tcg: TcgGenPass, add structs for string emission
In reply to Anton Johansson via qemu developmentAdds two new structures TcgEmitter and CEmitter with the purpose of producing valid TCG and C expressions in string form, these will be used in following commits.
--- subprojects/helper-to-tcg/meson.build | 1 + .../helper-to-tcg/src/TcgGenPass/TcgEmit.cpp | 1203 +++++++++++++++++ .../helper-to-tcg/src/TcgGenPass/TcgEmit.hpp | 315 +++++ 3 files changed, 1519 insertions(+) create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/TcgEmit.cpp create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/TcgEmit.hpp
subprojects/helper-to-tcg/meson.build +1 -0
@@ -50,6 +50,7 @@ sources = [ 'src/PrepareForTcgPass/IdentityMap.cpp', 'src/TcgGenPass/TcgGenPass.cpp', 'src/TcgGenPass/LinearizeBlocks.cpp', + 'src/TcgGenPass/TcgEmit.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/TcgGenPass/TcgEmit.cpp new file +1203 -0
1,203 lines not on this page · raw message
subprojects/helper-to-tcg/src/TcgGenPass/TcgEmit.hpp new file +315 -0
315 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v2 34/50 helper-to-tcg: TcgGenPass, map arguments to TCG
In reply to Anton Johansson via qemu developmentMapping for arguments is slighty different than other Values and depend on function annotations, perform it as a separe pass.
--- subprojects/helper-to-tcg/meson.build | 1 + .../src/TcgGenPass/MapArguments.cpp | 122 ++++++++++++++++++ 2 files changed, 123 insertions(+) create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/MapArguments.cpp
subprojects/helper-to-tcg/meson.build +1 -0
@@ -51,6 +51,7 @@ sources = [ 'src/TcgGenPass/TcgGenPass.cpp', 'src/TcgGenPass/LinearizeBlocks.cpp', 'src/TcgGenPass/TcgEmit.cpp', + 'src/TcgGenPass/MapArguments.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/TcgGenPass/MapArguments.cpp new file +122 -0
122 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v2 35/50 helper-to-tcg: TcgGenPass, propagate constant expresssions
In reply to Anton Johansson via qemu developmentIn a forward pass over the instructions, fold and propagate constant expressions to TcgVs with a string representing the expression.
This needs to be performed before register allocation where TcgV registers are assigned to Values to get the best register reuse. Register allocation is also performed in a backwards pass over the instructions, so these two passes cannot be nicely combined.
--- subprojects/helper-to-tcg/meson.build | 1 + .../src/TcgGenPass/MapConstantExpressions.cpp | 328 ++++++++++++++++++ 2 files changed, 329 insertions(+) create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/MapConstantExpressions.cpp
subprojects/helper-to-tcg/meson.build +1 -0
@@ -52,6 +52,7 @@ sources = [ 'src/TcgGenPass/LinearizeBlocks.cpp', 'src/TcgGenPass/TcgEmit.cpp', 'src/TcgGenPass/MapArguments.cpp', + 'src/TcgGenPass/MapConstantExpressions.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/TcgGenPass/MapConstantExpressions.cpp new file +328 -0
328 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v2 36/50 helper-to-tcg: TcgGenPass, allocate TCG registers
In reply to Anton Johansson via qemu developmentBased on the assumption of a cycle free IR, this commit adds a simple register allocator for emitted values in TCG. The goal of this pass is to reduce the number of temporaries required in the output code, which is especially important when dealing with gvec vectors as to not require huge amounts of temporary storage in CPUArchState.
For each LLVM value in the IR, that has not already been mapped (arguments, constant expressions), the allocator will assign a struct TcgV reprensenting a variable in the output TCG.
--- subprojects/helper-to-tcg/meson.build | 1 + .../src/TcgGenPass/MapTemporaries.cpp | 619 ++++++++++++++++++ 2 files changed, 620 insertions(+) create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/MapTemporaries.cpp
subprojects/helper-to-tcg/meson.build +1 -0
@@ -53,6 +53,7 @@ sources = [ 'src/TcgGenPass/TcgEmit.cpp', 'src/TcgGenPass/MapArguments.cpp', 'src/TcgGenPass/MapConstantExpressions.cpp', + 'src/TcgGenPass/MapTemporaries.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/TcgGenPass/MapTemporaries.cpp new file +619 -0
619 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v2 37/50 helper-to-tcg: TcgGenPass, emit TCG strings
In reply to Anton Johansson via qemu developmentForward pass over the IR which uses previosly mapped LLVM values to emit the corresponding operations in TCG.
--- subprojects/helper-to-tcg/meson.build | 1 + .../src/TcgGenPass/MapTcgOperations.cpp | 988 ++++++++++++++++++ 2 files changed, 989 insertions(+) create mode 100644 subprojects/helper-to-tcg/src/TcgGenPass/MapTcgOperations.cpp
subprojects/helper-to-tcg/meson.build +1 -0
@@ -54,6 +54,7 @@ sources = [ 'src/TcgGenPass/MapArguments.cpp', 'src/TcgGenPass/MapConstantExpressions.cpp', 'src/TcgGenPass/MapTemporaries.cpp', + 'src/TcgGenPass/MapTcgOperations.cpp', ] clang = bindir / 'clang'
subprojects/helper-to-tcg/src/TcgGenPass/MapTcgOperations.cpp new file +988 -0
988 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v2 38/50 helper-to-tcg: Add README
In reply to Anton Johansson via qemu development--- subprojects/helper-to-tcg/README.md | 297 ++++++++++++++++++++++++++++ 1 file changed, 297 insertions(+) create mode 100644 subprojects/helper-to-tcg/README.md
subprojects/helper-to-tcg/README.md new file +297 -0
@@ -0,0 +1,297 @@ +# helper-to-tcg + +`helper-to-tcg` is a standalone LLVM IR to TCG translator, with the goal of simplifying the implementation of complicated instructions in TCG. Instruction semantics can be specified either directly in LLVM IR or any language that can be compiled to it (C, C++, ...). However, the tool is tailored towards QEMU helper functions written in C. + +Internally, `helper-to-tcg` consists of a mix of custom and built-in transformation and analysis passes that are applied to the input LLVM IR sequentially. The pipeline of passes is laid out as follows +``` + +---------------+ +-----+ +---------------+ +------------+ +LLVM IR -> | PrepareForOpt | -> | -Oz | -> | PrepareForTcg | -> | TcgGenPass | -> TCG + +---------------+ +-----+ +---------------+ +------------+ +``` +where the custom passes performs: +* `PrepareForOpt` - Early information gathering and culling pass. Removes unneeded functions, maps function annotations, gathers debuginfo data, demangles function names, etc.; +* `PrepareForTcg` - Post-optimization pass that tries to get the IR as close to TCG as possible, goal is to take complexity away from the backend; +* `TcgGenPass` - Backend pass that propagates constants, allocates TCG variables to LLVM values, and emits final TCG C code. + +As for LLVM optimization, `-Oz` strikes a good balance between unrolling and vectorization, from testing. More aggressive optimization levels would often unroll loops over compacting it with loop vectorization. + +## Project Structure + +* `get-llvm-ir.py` - Helper script to convert a QEMU `.c` file to LLVM IR by getting compile flags from `compile_commands.json`; +* `pipeline` - Implementation of a LLVM pipeline orchestrating passes and handling input; +* `src` - Implementation of custom LLVM passes: `PrepareForOptPass`,`PrepareForTcgPass`,`TcgGenPass`, and common features; +* `include` - Shared headers between `src`; +* `tests` - Simple end-to-end tests of C functions we expect to be able to translate, tests fail if any function fails to translate, output is verified against expected output for support major LLVM versions. + +## Example Translations + +`helper-to-tcg` is able to deal with a wide variety of helper functions, the following code snippet contains two examples from the Hexagon architecture implementing the semantics of a predicated and instruction (`A2_pandt`) and a vectorized signed saturated 2-element scalar product (`V6_vdmpyhvsat`). + +```c +int32_t HELPER(A2_pandt)(CPUHexagonState *env, int32_t RdV, + int32_t PuV, int32_t RsV, int32_t RtV) +{ + if(fLSBOLD(PuV)) { + RdV=RsV&RtV; + } else { + CANCEL; + } + return RdV; +} + +void HELPER(V6_vdmpyhvsat)(CPUHexagonState *env, + void * restrict VdV_void, + void * restrict VuV_void, + void * restrict VvV_void) +{ + fVFOREACH(32, i) { + size8s_t accum = fMPY16SS(fGETHALF(0,VuV.w[i]),fGETHALF(0, VvV.w[i])); + accum += fMPY16SS(fGETHALF(1,VuV.w[i]),fGETHALF(1, VvV.w[i])); + VdV.w[i] = fVSATW(accum); + } +} +``` +For the above snippet, `helper-to-tcg` produces the following TCG +```c +void emit_A2_pandt(TCGv_i32 temp0, TCGv_env env, TCGv_i32 temp4, + TCGv_i32 temp8, TCGv_i32 temp7, TCGv_i32 temp6) { + TCGv_i32 temp2 = tcg_temp_new_i32(); + tcg_gen_andi_i32(temp2, temp8, 1); + TCGv_i32 temp5 = tcg_temp_new_i32(); + tcg_gen_and_i32(temp5, temp6, temp7); + tcg_gen_movcond_i32(TCG_COND_EQ, temp0, temp2, tcg_constant_i32(0), temp4, temp5); +} + +void emit_V6_vdmpyhvsat(TCGv_env env, intptr_t vec3, + intptr_t vec7, intptr_t vec6) { + VectorMem mem = {0}; + intptr_t vec0 = temp_new_gvec(&mem, 128); + tcg_gen_gvec_shli(MO_32, vec0, vec7, 16, 128, 128); + intptr_t vec5 = temp_new_gvec(&mem, 128); + tcg_gen_gvec_sari(MO_32, vec5, vec0, 16, 128, 128); + intptr_t vec1 = temp_new_gvec(&mem, 128); + tcg_gen_gvec_shli(MO_32, vec1, vec6, 16, 128, 128); + tcg_gen_gvec_sari(MO_32, vec1, vec1, 16, 128, 128); + tcg_gen_gvec_mul(MO_32, vec1, vec1, vec5, 128, 128); + intptr_t vec2 = temp_new_gvec(&mem, 128); + tcg_gen_gvec_sari(MO_32, vec2, vec7, 16, 128, 128); + tcg_gen_gvec_sari(MO_32, vec0, vec6, 16, 128, 128); + tcg_gen_gvec_mul(MO_32, vec2, vec0, vec2, 128, 128); + tcg_gen_gvec_ssadd(MO_32, vec3, vec1, vec2, 128, 128); +} +``` + +In the first case, the predicated and instruction was made branchless by using a conditional move, and in the latter case the inner loop of the vectorized scalar product could be converted to a few vectorized shifts and multiplications, folllowed by a vectorized signed saturated addition. + +## Usage + +Building `helper-to-tcg` produces a binary implementing the pipeline outlined above, going from LLVM IR to TCG. + +### Specifying Functions to Translate + +Unless `--translate-all-helpers` is specified, the default behaviour of `helper-to-tcg` is to only translate functions annotated via a special `"helper-to-tcg"` annotation. Functions called by annotated functions will also be translated, see the following example: + +```c +// Function will be translated, annotation provided +__attribute__((annotate ("helper-to-tcg"))) +int f(int a, int b) { + return 2 * g(a, b); +} + +// Function will be translated, called by annotated `f()` function +int g(int a, int b) { + ... +} + +// Function will not be translated +int h(int a, int b) { + ... +} +``` + +### Immediate and Vector Arguments + +Function annotations are in some cases used to provide extra information to `helper-to-tcg` not otherwise present in the IR. For example, whether an integer argument should actually be treated as an immediate rather than a register, or if a pointer argument should be treated as a `gvec` vector (offset into `CPUArchState`). For instance: +```c +__attribute__((annotate ("helper-to-tcg"))) +__attribute__((annotate ("immediate: 1"))) +int f(int a, int i) { + ... +} + +__attribute__((annotate ("helper-to-tcg"))) +__attribute__((annotate ("ptr-to-offset: 0, 1"))) +void g(void * restrict a, void * restrict b) { + ... +} +``` +where `"immediate: 1"` tells `helper-to-tcg` that the argument with index `1` should be treated as an immediate (multiple arguments are specified through a comma separated list). Similarly `"ptr-to-offset: 0, 1"` indicates that arguments width index 0 and 1 should be treated as offsets from `CPUArchState` (given as `intptr_t`), rather than actual pointer arguments. For the above code, `helper-to-tcg` emits +```c +void emit_f(TCGv_i32 res, TCGv_i32 a, int i) { + ... +} + +void emit_g(intptr_t a, intptr_t b) { + ... +} +``` + +### Loads and Stores + +Translating loads and stores is slightly trickier, as some QEMU specific assumptions are made. Loads and stores in the input are assumed to go through the `cpu_[st|ld]*()` functions defined in `exec/cpu_ldst.h` that a helper function would use. + +If using standalone input functions (not QEMU helper functions), loads and stores are still represented by `cpu_[st|ld]*()` which needs to be declared, consider: +```c +/* Opaque CPU state type, will be mapped to tcg_env */ +struct CPUArchState; +typedef struct CPUArchState CPUArchState; + +/* Prototype of QEMU helper guest load/store functions, see exec/cpu_ldst.h */ +uint32_t cpu_ldub_data(CPUArchState *, uint32_t ptr); +void cpu_stb_data(CPUArchState *, uint32_t ptr, uint32_t data); + +uint32_t helper_ld8(CPUArchState *env, uint32_t addr) { + return cpu_ldub_data(env, addr); +} + +void helper_st8(CPUArchState *env, uint32_t addr, uint32_t data) { + return cpu_stb_data(env, addr, data); +} +``` +implementing an 8-bit load and store instruction, these will be translated to the following TCG. +```c +void emit_ld8(TCGv_i32 temp0, TCGv_env env, TCGv_i32 temp1) { + tcg_gen_qemu_ld_i32(temp0, temp1, tb_mmu_index(tcg_ctx->gen_tb->flags), MO_UB); +} + +void emit_st8(TCGv_env env, TCGv_i32 temp0, TCGv_i32 temp1) { + tcg_gen_qemu_st_i32(temp1, temp0, tb_mmu_index(tcg_ctx->gen_tb->flags), MO_UB); +} +``` +Note, the emitted code assumes the definition of a `tb_mmu_index()` function to retrieve the current CPU MMU index, the name of this function can be configured via the `--mmu-index-function` flag. + +### Mapping CPU State + +In QEMU, commonly accessed fields in the `CPUArchState` are often mapped to global `TCGv*` variables representing that piece of CPU state in TCG. When translating helper functions (or other C functions), a method of specifying which fields in the CPU state should be mapped to which globals is needed. To this end, a declarative approach is taken, where mappings between CPU state and globals can be consumed by both `helper-to-tcg` and runtime QEMU for instantiating the `TCGv` globals themselves. + +Users must define this mapping via a global `cpu_tcg_mapping []` array, as can be seen in the following example where `mapped_field` of `CPUArchState` is mapped to the global `tcg_field`. For more complicated examples see the tests in `tests/cpustate.c`. +```c +#include <stdint.h> +#include "tcg/tcg-global-mappings.h" + +/* Define a CPU state with some different fields */ + +typedef struct CPUArchState { + uint32_t mapped_field; + uint32_t unmapped_field; +} CPUArchState; + +/* Dummy struct, in QEMU this would correspond to TCGv_i32 in tcg.h */ +typedef struct TCGv_i32 {} TCGv_i32; + +/* Global TCGv representing CPU state */ +TCGv_i32 tcg_field; + +/* + * Finally provide a mapping of CPUArchState to TCG globals we care about, here + * we map mapped_field to tcg_field + */ +cpu_tcg_mapping mappings[] = { + CPU_TCG_MAP(CPUArchState, tcg_field, mapped_field), +}; + +uint32_t helper_mapped(CPUArchState *env) { + return env->mapped_field; +} + +uint32_t helper_unmapped(CPUArchState *env) { + return env->unmapped_field; +} +``` +Note, the name of the `cpu_tcg_mapping[]` is provided via the `--tcg-global-mappings` flag. For the above example, `helper-to-tcg` emits +```c +extern TCGv_i32 tcg_field; + +void emit_mapped(TCGv_i32 temp0, TCGv_env env) { + tcg_gen_mov_i32(temp0, tcg_field); +} + +void emit_unmapped(TCGv_i32 temp0, TCGv_env env) { + TCGv_ptr ptr1 = tcg_temp_new_ptr(); + tcg_gen_addi_ptr(ptr1, env, 128ull); + tcg_gen_ld_i32(temp0, ptr1, 0); +} +``` +where accesses in the input C code are correctly mapped to the corresponding TCG globals. The unmapped `CPUArchState` access turns into pointer math and a load, whereas the mapped access turns into a `mov` from a global. + +### Automatic Calling of Generated Code + +Finally, calling the generated code is as simple as including the output of `helper-to-tcg` into the project and manually calling `emit_*(...)`. However, when dealing with an existing frontend that has a lot of helper functions already in use, we simplify this process somewhat for non-vector instructions. For translated helper functions `helper-to-tcg` emits its own `gen_helper_*()` function definitions +```c +/* helper-to-tcg-emitted.c */ +void emit_add(TCGv_ptr env, TCGv_i32 d, TCGv_i32 a, uint32_t imm) +{ + tcg_gen_addi_i32(d, a, imm); +} + +/* helper-to-tcg-emitted.h */ +static inline void gen_helper_add_imm(TCGv_ptr env, TCGv_i32 d, TCGv_i32 a, TCGv_i32 imm) +{ + emit_add(env, d, a, tcgv_i32_temp(imm)->val); +} +``` +then from code that needs to call `gen_helper_add_imm` include +```c +/* translate.c */ + +/* + * Include generated gen_helper_*() definitions if using helper-to-tcg, but + * only when not generating input IR since the header won't exist at that point. + */ +#if defined(TARGET_HELPER_TO_TCG) && !defined(HELPER_TO_TCG_IR_GEN) +#include "helper-to-tcg-emitted.h" +#else +#include "exec/helper-gen.h" +#endif +``` + +### Simple Command Usage + +Assume a `helpers.c` file with functions to translate, then to obtain LLVM IR +```bash +$ clang helpers.c -O0 -Xclang -disable-O0-optnone -S -emit-llvm +``` +which produces `helpers.ll` to be fed into `helper-to-tcg` +```bash +$ ./helper-to-tcg helpers.ll --translate-all-helpers +``` +where `--translate-all-helpers` means "translate all functions starting with helper_*". Finally, the above command produces `helper-to-tcg-emitted.[c|h]` with emitted TCG code. + +By default meson uses `llvm-config` to find LLVM, usually this corresponds to the latest version installed on the system. Since `helper-to-tcg` only supports LLVM 15-21, the path to `llvm-config` can be manually overridden using `meson configure -Dllvm_config_path=...` when needed. This is also useful for testing multiple LLVM versions. If `helper-to-tcg` is used as a part of `QEMU`, this is specified using `../configure -Dhelper_to_tcg:llvm_config_path=...`. + +### Testing + +When building `helper-to-tcg` as a part of a QEMU project the `end-to-end` tests may be ran via either of the commands +```bash +$ make check +$ make check-helper-to-tcg +$ meson test --suite 'helper-to-tcg:end-to-end' +``` +where the last one is useful when building as a standalone project. + +On the QEMU end, a docker container has been added for testing with the supported LLVM versions, this may be triggered via +```bash +$ make docker-test-helper-to-tcg@debian-llvm +``` +and will build `helper-to-tcg` along with the current target, run all end-to-end and `check-tcg` tests for each version of LLVM that is targeted. + +Debug logging can be turned on via +```bash +$ ./helper-to-tcg --debug +``` +or to restrict logging to given pass + +```bash +$ ./helper-to-tcg --debug-only=${pass} +``` +where `${pass}` is one of `map-tcg-ops, map-temporaries, map-arguments, tcg-gen-pass, pipeline, transform-geps, prepare-for-opt`.
Signature · 1 line
2.52.0
PATCH v2 39/50 helper-to-tcg: Add end-to-end tests
In reply to Anton Johansson via qemu developmentIntroduces simple end-to-end tests, providing input functions covering a variety of features, and expected outputs for each of test and supported LLVM version. (Currently reference code for LLVM 15 and 21 is added).
--- subprojects/helper-to-tcg/meson.build | 2 + .../helper-to-tcg/tests/call-to-declaration.c | 33 ++ .../helper-to-tcg/tests/call-translation.c | 35 ++ subprojects/helper-to-tcg/tests/cpustate.c | 44 ++ .../helper-to-tcg/tests/forward-context.c | 21 + subprojects/helper-to-tcg/tests/ldst.c | 17 + subprojects/helper-to-tcg/tests/meson.build | 121 ++++ .../tests/ref/15/call-to-declaration.c | 26 + .../tests/ref/15/call-translation.c | 28 + .../helper-to-tcg/tests/ref/15/cpustate.c | 36 ++ .../tests/ref/15/forward-context.c | 30 + subprojects/helper-to-tcg/tests/ref/15/ldst.c | 21 + .../helper-to-tcg/tests/ref/15/scalar.c | 21 + .../tests/ref/15/user-pcrel-jump.c | 33 ++ .../tests/ref/15/vector-layout-f16L.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-f16M.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-f32L.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-f32M.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-f64L.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-f64M.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-f8L.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-f8M.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-t16L.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-t16M.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-t32L.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-t32M.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-t64L.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-t64M.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-t8L.c | 527 ++++++++++++++++++ .../tests/ref/15/vector-layout-t8M.c | 527 ++++++++++++++++++ .../helper-to-tcg/tests/ref/15/vector.c | 31 ++ .../tests/ref/21/call-to-declaration.c | 26 + .../tests/ref/21/call-translation.c | 28 + .../helper-to-tcg/tests/ref/21/cpustate.c | 36 ++ .../tests/ref/21/forward-context.c | 30 + subprojects/helper-to-tcg/tests/ref/21/ldst.c | 21 + .../helper-to-tcg/tests/ref/21/scalar.c | 21 + .../tests/ref/21/user-pcrel-jump.c | 33 ++ .../tests/ref/21/vector-layout-f16L.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-f16M.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-f32L.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-f32M.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-f64L.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-f64M.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-f8L.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-f8M.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-t16L.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-t16M.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-t32L.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-t32M.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-t64L.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-t64M.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-t8L.c | 527 ++++++++++++++++++ .../tests/ref/21/vector-layout-t8M.c | 527 ++++++++++++++++++ .../helper-to-tcg/tests/ref/21/vector.c | 31 ++ subprojects/helper-to-tcg/tests/scalar.c | 15 + .../helper-to-tcg/tests/tcg-global-mappings.h | 118 ++++ .../helper-to-tcg/tests/user-pcrel-jump.c | 28 + .../helper-to-tcg/tests/vector-layout.c | 51 ++ subprojects/helper-to-tcg/tests/vector.c | 34 ++ 60 files changed, 17835 insertions(+) create mode 100644 subprojects/helper-to-tcg/tests/call-to-declaration.c create mode 100644 subprojects/helper-to-tcg/tests/call-translation.c create mode 100644 subprojects/helper-to-tcg/tests/cpustate.c create mode 100644 subprojects/helper-to-tcg/tests/forward-context.c create mode 100644 subprojects/helper-to-tcg/tests/ldst.c create mode 100644 subprojects/helper-to-tcg/tests/meson.build create mode 100644 subprojects/helper-to-tcg/tests/ref/15/call-to-declaration.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/call-translation.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/cpustate.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/forward-context.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/ldst.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/scalar.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/user-pcrel-jump.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f16L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f16M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f32L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f32M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f64L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f64M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f8L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-f8M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t16L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t16M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t32L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t32M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t64L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t64M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t8L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector-layout-t8M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/15/vector.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/call-to-declaration.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/call-translation.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/cpustate.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/forward-context.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/ldst.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/scalar.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/user-pcrel-jump.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f16L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f16M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f32L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f32M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f64L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f64M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f8L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-f8M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t16L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t16M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t32L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t32M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t64L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t64M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t8L.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector-layout-t8M.c create mode 100644 subprojects/helper-to-tcg/tests/ref/21/vector.c create mode 100644 subprojects/helper-to-tcg/tests/scalar.c create mode 100644 subprojects/helper-to-tcg/tests/tcg-global-mappings.h create mode 100644 subprojects/helper-to-tcg/tests/user-pcrel-jump.c create mode 100644 subprojects/helper-to-tcg/tests/vector-layout.c create mode 100644 subprojects/helper-to-tcg/tests/vector.c
subprojects/helper-to-tcg/meson.build +2 -0
@@ -82,3 +82,5 @@ pipeline = executable('helper-to-tcg', sources, include_directories: ['include'] + [incdir], link_args: [ldflags] + [libs] + [syslibs], cpp_args: cpp_args) + +subdir('tests')
subprojects/helper-to-tcg/tests/call-to-declaration.c new file +33 -0
@@ -0,0 +1,33 @@ +/* +* Test translation of calls to declration +* +* (experimental) +*/ + +#define ANNOTATE (str) __attribute__((annotate(str))) + +int fdecl0 (int a); + +ANNOTATE("helper-to-tcg") +int f0 (int cond,int b) { +return fdecl0 (b); +} + +ANNOTATE("immediate: 0") +int fdecl1 (int a); + +ANNOTATE("helper-to-tcg") +ANNOTATE("immediate: 1") +int f1 (int cond,int b) { +return fdecl1 (b); +} + +ANNOTATE("returns-immediate") +ANNOTATE("immediate: 0") +int fdecl2 (int a); + +ANNOTATE("helper-to-tcg") +ANNOTATE("immediate: 0") +int f2 (int b) { +return fdecl2 (b); +}
subprojects/helper-to-tcg/tests/call-translation.c new file +35 -0
@@ -0,0 +1,35 @@ +/* +* Test translation of dependent functions. +* +* This test will be ran without --translate-all-helpers, meaning we have to +* manually mark them for translation. +*/ + +#define ANNOTATE (str) __attribute__((annotate(str))) + +/* +* Only inner0 is marked for translation, outer0 will be skipped. +*/ +ANNOTATE("helper-to-tcg") +int inner0 (int a) {return a +5 ; } + +int outer0 (int cond,int b) { +if (cond) { +return inner0 (b); + } +return 0 ; +} + +/* +* Both inner1 and outer1 will be translated since outer1 is marked for +* translation and depends on inner1. +*/ +int inner1 (int a) {return a +5 ; } + +ANNOTATE("helper-to-tcg") +int outer1 (int cond,int b) { +if (cond) { +return inner1 (b); + } +return 0 ; +}
subprojects/helper-to-tcg/tests/cpustate.c new file +44 -0
44 lines not on this page · raw message
subprojects/helper-to-tcg/tests/forward-context.c new file +21 -0
@@ -0,0 +1,21 @@ +/* +* Test translation of calls to declration +* +* This test will be ran without --translate-all-helpers, meaning we have to +* manually mark them for translation. +*/ + +#define ANNOTATE (str) __attribute__((annotate(str))) + +ANNOTATE("immediate: 0") +int fdecl0 (int a); + +ANNOTATE("helper-to-tcg") +int f0 (int a,int off) { +return a +fdecl0 (off); +} + +ANNOTATE("helper-to-tcg") +int f1 (int a) { +return f0 (a,a) +f0 (-a,-a); +}
subprojects/helper-to-tcg/tests/ldst.c new file +17 -0
@@ -0,0 +1,17 @@ +#include <stdint.h> + +/* Opaque CPU state type, will be mapped to tcg_env */ +struct CPUArchState ; +typedef struct CPUArchState CPUArchState ; + +/* Prototype of QEMU helper guest load/store functions, see exec/cpu_ldst.h */ +uint32_t cpu_ldub_data (CPUArchState *,uint32_t ptr); +void cpu_stb_data (CPUArchState *,uint32_t ptr,uint32_t data); + +uint32_t helper_ld8 (CPUArchState *env,uint32_t addr) { +return cpu_ldub_data (env, addr); +} + +void helper_st8 (CPUArchState *env,uint32_t addr,uint32_t data) { +return cpu_stb_data (env, addr, data); +}
subprojects/helper-to-tcg/tests/meson.build new file +121 -0
121 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/call-to-declaration.c new file +26 -0
@@ -0,0 +1,26 @@ +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "cpu.h" +#include "translate.h" +#include "tcg/tcg-op.h" +#include "tcg/tcg-op-gvec.h" +#include "tcg/tcg.h" +#include "tcg/tcg-global-mappings.h" +#include "exec/translation-block.h" +#include "call-to-declaration.h" + +// i32 f0 +void emit_f0 (TCGv_i32 tmp2,TCGv_i32 tmp0,TCGv_i32 b) { +tmp2 =fdecl0 (b); +} + +// i32 f1 +void emit_f1 (TCGv_i32 tmp2,TCGv_i32 tmp0,uint32_t b) { +tmp2 =fdecl1 (tcg_constant_i32 (b)); +} + +// i32 f2 +void emit_f2 (TCGv_i32 tmp1,uint32_t b) { +tmp1 =fdecl2 (tcg_constant_i32 (b)); +} +
subprojects/helper-to-tcg/tests/ref/15/call-translation.c new file +28 -0
@@ -0,0 +1,28 @@ +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "cpu.h" +#include "translate.h" +#include "tcg/tcg-op.h" +#include "tcg/tcg-op-gvec.h" +#include "tcg/tcg.h" +#include "tcg/tcg-global-mappings.h" +#include "exec/translation-block.h" +#include "call-translation.h" + +// i32 inner0 +void emit_inner0 (TCGv_i32 tmp2,TCGv_i32 a) { +tcg_gen_add_i32 (tmp2, a,tcg_constant_i32 (5 )); +} + +// i32 inner1 +void emit_inner1 (TCGv_i32 tmp2,TCGv_i32 a) { +tcg_gen_add_i32 (tmp2, a,tcg_constant_i32 (5 )); +} + +// i32 outer1 +void emit_outer1 (TCGv_i32 tmp5,TCGv_i32 cond,TCGv_i32 b) { +TCGv_i32 tmp6 =tcg_temp_new_i32 (); +tcg_gen_add_i32 (tmp6, b,tcg_constant_i32 (5 )); +tcg_gen_movcond_i32 (TCG_COND_EQ , tmp5, cond,tcg_constant_i32 (0 ),tcg_constant_i32 (0 ), tmp6); +} +
subprojects/helper-to-tcg/tests/ref/15/cpustate.c new file +36 -0
@@ -0,0 +1,36 @@ +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "cpu.h" +#include "translate.h" +#include "tcg/tcg-op.h" +#include "tcg/tcg-op-gvec.h" +#include "tcg/tcg.h" +#include "tcg/tcg-global-mappings.h" +#include "exec/translation-block.h" +#include "cpustate.h" + +extern TCGv_i32 tcg_regs[32 ]; +extern TCGv_i32 tcg_a[8 ]; +extern TCGv_i32 tcg_field; +// i32 helper_reg +void emit_reg (TCGv_i32 tmp5,TCGv_ptr env,uint32_t i) { +tcg_gen_mov_i32 (tmp5, tcg_regs[((uint64_t ) (uint32_t ) i)]); +} + +// i32 helper_data_a +void emit_data_a (TCGv_i32 tmp6,TCGv_ptr env,uint32_t i) { +tcg_gen_mov_i32 (tmp6, tcg_a[((uint64_t ) (uint32_t ) i)]); +} + +// i32 helper_single_mapped +void emit_single_mapped (TCGv_i32 tmp4,TCGv_ptr env) { +tcg_gen_mov_i32 (tmp4, tcg_field); +} + +// i32 helper_unmapped +void emit_unmapped (TCGv_i32 tmp2,TCGv_ptr env) { +TCGv_ptr ptr3 =tcg_temp_new_ptr (); +tcg_gen_addi_ptr (ptr3, env,128ull ); +tcg_gen_ld_i32 (tmp2, ptr3,0 ); +} +
subprojects/helper-to-tcg/tests/ref/15/forward-context.c new file +30 -0
@@ -0,0 +1,30 @@ +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "cpu.h" +#include "translate.h" +#include "tcg/tcg-op.h" +#include "tcg/tcg-op-gvec.h" +#include "tcg/tcg.h" +#include "translate.h" +#include "tcg/tcg-global-mappings.h" +#include "exec/translation-block.h" +#include "forward-context.h" + +#include "translate.h" +// i32 f0 +void emit_f0 (DisasContext *ctx,TCGv_i32 tmp2,TCGv_i32 a,TCGv_i32 off) { +TCGv_i32 tmp3 =tcg_temp_new_i32 (); +tmp3 =fdecl0 (ctx, off); +tcg_gen_add_i32 (tmp2, tmp3, a); +} + +// i32 f1 +void emit_f1 (DisasContext *ctx,TCGv_i32 tmp2,TCGv_i32 a) { +TCGv_i32 tmp4 =tcg_temp_new_i32 (); +tmp4 =fdecl0 (ctx, a); +tcg_gen_sub_i32 (tmp2,tcg_constant_i32 (0 ), a); +TCGv_i32 tmp3 =tcg_temp_new_i32 (); +tmp3 =fdecl0 (ctx, tmp2); +tcg_gen_add_i32 (tmp2, tmp3, tmp4); +} +
subprojects/helper-to-tcg/tests/ref/15/ldst.c new file +21 -0
@@ -0,0 +1,21 @@ +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "cpu.h" +#include "translate.h" +#include "tcg/tcg-op.h" +#include "tcg/tcg-op-gvec.h" +#include "tcg/tcg.h" +#include "tcg/tcg-global-mappings.h" +#include "exec/translation-block.h" +#include "ldst.h" + +// i32 helper_ld8 +void emit_ld8 (TCGv_i32 tmp4,TCGv_ptr env,TCGv_i32 addr) { +tcg_gen_qemu_ld_i32 (tmp4, addr,tb_mmu_index (tcg_ctx->gen_tb->flags),MO_8 ); +} + +// void helper_st8 +void emit_st8 (TCGv_ptr env,TCGv_i32 addr,TCGv_i32 data) { +tcg_gen_qemu_st_i32 (data, addr,tb_mmu_index (tcg_ctx->gen_tb->flags),MO_8 ); +} +
subprojects/helper-to-tcg/tests/ref/15/scalar.c new file +21 -0
@@ -0,0 +1,21 @@ +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "cpu.h" +#include "translate.h" +#include "tcg/tcg-op.h" +#include "tcg/tcg-op-gvec.h" +#include "tcg/tcg.h" +#include "tcg/tcg-global-mappings.h" +#include "exec/translation-block.h" +#include "scalar.h" + +// i32 helper_add +void emit_add (TCGv_i32 tmp2,TCGv_i32 a,TCGv_i32 b) { +tcg_gen_add_i32 (tmp2, b, a); +} + +// i32 helper_cmov +void emit_cmov (TCGv_i32 tmp5,TCGv_i32 c0,TCGv_i32 c1,TCGv_i32 a,TCGv_i32 b) { +tcg_gen_movcond_i32 (TCG_COND_LTU , tmp5, c0, c1, a, b); +} +
subprojects/helper-to-tcg/tests/ref/15/user-pcrel-jump.c new file +33 -0
@@ -0,0 +1,33 @@ +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "cpu.h" +#include "translate.h" +#include "tcg/tcg-op.h" +#include "tcg/tcg-op-gvec.h" +#include "tcg/tcg.h" +#include "tcg/tcg-global-mappings.h" +#include "exec/translation-block.h" +#include "user-pcrel-jump.h" + +// void prcel_jump +void emit_prcel_jump (TCGv_i32 off) { +TCGv_i32 ea =tcg_temp_new_i32 (); +tcg_gen_shl_i32 (ea, off,tcg_constant_i32 (2 )); +tcg_gen_or_i32 (ea, ea,tcg_constant_i32 (3 )); +target_pcrel_jump (ea); +} + +// void prcel_condjump +void emit_prcel_condjump (TCGv_i32 cond,TCGv_i32 off) { +TCGLabel * label8 =gen_new_label (); +TCGLabel * label9 =gen_new_label (); +tcg_gen_brcondi_i32 (TCG_COND_EQ , cond,0 , label8); +gen_set_label (label9); +TCGv_i32 ea =tcg_temp_new_i32 (); +tcg_gen_shl_i32 (ea, off,tcg_constant_i32 (2 )); +tcg_gen_or_i32 (ea, ea,tcg_constant_i32 (3 )); +target_pcrel_jump (ea); +tcg_gen_br (label8); +gen_set_label (label8); +} +
subprojects/helper-to-tcg/tests/ref/15/vector-layout-f16L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-f16M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-f32L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-f32M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-f64L.c new file +527 -0
421 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-f64M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-f8L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-f8M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-t16L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-t16M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-t32L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-t32M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-t64L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-t64M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-t8L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector-layout-t8M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/15/vector.c new file +31 -0
Lines of this file are not shown. View the raw message.
subprojects/helper-to-tcg/tests/ref/21/call-to-declaration.c new file +26 -0
Lines of this file are not shown. View the raw message.
subprojects/helper-to-tcg/tests/ref/21/call-translation.c new file +28 -0
Lines of this file are not shown. View the raw message.
subprojects/helper-to-tcg/tests/ref/21/cpustate.c new file +36 -0
Lines of this file are not shown. View the raw message.
subprojects/helper-to-tcg/tests/ref/21/forward-context.c new file +30 -0
Lines of this file are not shown. View the raw message.
subprojects/helper-to-tcg/tests/ref/21/ldst.c new file +21 -0
Lines of this file are not shown. View the raw message.
subprojects/helper-to-tcg/tests/ref/21/scalar.c new file +21 -0
Lines of this file are not shown. View the raw message.
subprojects/helper-to-tcg/tests/ref/21/user-pcrel-jump.c new file +33 -0
Lines of this file are not shown. View the raw message.
subprojects/helper-to-tcg/tests/ref/21/vector-layout-f16L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-f16M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-f32L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-f32M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-f64L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-f64M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-f8L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-f8M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-t16L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-t16M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-t32L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-t32M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-t64L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-t64M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-t8L.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector-layout-t8M.c new file +527 -0
527 lines not on this page · raw message
subprojects/helper-to-tcg/tests/ref/21/vector.c new file +31 -0
Lines of this file are not shown. View the raw message.
subprojects/helper-to-tcg/tests/scalar.c new file +15 -0
Lines of this file are not shown. View the raw message.
subprojects/helper-to-tcg/tests/tcg-global-mappings.h new file +118 -0
118 lines not on this page · raw message
subprojects/helper-to-tcg/tests/user-pcrel-jump.c new file +28 -0
Lines of this file are not shown. View the raw message.
subprojects/helper-to-tcg/tests/vector-layout.c new file +51 -0
51 lines not on this page · raw message
subprojects/helper-to-tcg/tests/vector.c new file +34 -0
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.52.0
Part of the diff is not shown here. View the raw message.
PATCH v2 40/50 test: helper-to-tcg docker tests
In reply to Anton Johansson via qemu developmentAdds a docker container with LLVM versions 15-21 and a docker test for building helper-to-tcg and running end-to-end tests along with running check-tcg, for testing targets such as Hexagon.
--
NOTE: Last I checked I had some troubles with C++17 support within the container, was a while ago though.
--- tests/docker/dockerfiles/debian-llvm.docker | 38 +++++++++++++++++++ tests/docker/test-helper-to-tcg | 41 +++++++++++++++++++++ 2 files changed, 79 insertions(+) create mode 100644 tests/docker/dockerfiles/debian-llvm.docker create mode 100755 tests/docker/test-helper-to-tcg
tests/docker/dockerfiles/debian-llvm.docker new file +38 -0
@@ -0,0 +1,38 @@ +FROM docker.io/library/debian:11-slim + +RUN apt update && \ + DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ + DEBIAN_FRONTEND=noninteractive eatmydata \ + apt install -y --no-install-recommends \ + bison \ + ca-certificates \ + flex \ + gawk \ + libmpc-dev \ + libmpfr-dev \ + libglib2.0-dev \ + libpixman-1-dev \ + make \ + ninja-build \ + rsync \ + pkgconf \ + wget \ + lsb-release \ + software-properties-common \ + gnupg \ + meson \ + python3-pip \ + python3-setuptools \ + python3-venv \ + python3-wheel + +RUN /usr/bin/pip3 install tomli + +RUN wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh +RUN ./llvm.sh 15 all +RUN ./llvm.sh 16 all +RUN ./llvm.sh 17 all +RUN ./llvm.sh 18 all +RUN ./llvm.sh 19 all +RUN ./llvm.sh 20 all +RUN ./llvm.sh 21 all
tests/docker/test-helper-to-tcg new file +41 -0
41 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v2 40/50] test: helper-to-tcg docker tests
Quoted 3 lines · On Thu, 30 Jul 2026 05:10:14 +0200 Anton Johansson via qemu development…
Adds a docker container with LLVM versions 15-21 and a docker test for building helper-to-tcg and running end-to-end tests along with running check-tcg, for testing targets such as Hexagon.
A report on my tests. Everything passes except:
1. test_max_temps from hvx_misc fails
2. I get follwoing 7 failures (out of 1689) on the (private) test-harness suite:
v65_Q6_P_vmpybsu_RR
v65_Q6_Vw_vinsert_VwR-compound
v65_Q6_Vw_vinsert_VwR
v65_Q6_Vw_vsub_VwVw_sat-compound
v65_Q6_Vw_vsub_VwVw_sat
v65_Q6_Ww_vsub_WwWw_sat-compound Also note that rundriver.sh exits 0 even when tests fail there.
PSA: Anton is going to be off until Aug 10th.
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
Re: [PATCH v2 40/50] test: helper-to-tcg docker tests
In reply to Anton Johansson via qemu developmentQuoted 8 lines · On Thu, 30 Jul 2026 05:10:14 +0200 Anton Johansson via qemu development…
Adds a docker container with LLVM versions 15-21 and a docker test for building helper-to-tcg and running end-to-end tests along with running check-tcg, for testing targets such as Hexagon.
Signed-off-by Anton Johansson <[email protected]>--
NOTE: Last I checked I had some troubles with C++17 support within the container, was a while ago though.
Maybe that was the clang 15 + -std=c++14 I reported on the other thread?
Quoted 13 lines
--- tests/docker/dockerfiles/debian-llvm.docker | 38 +++++++++++++++++++ tests/docker/test-helper-to-tcg | 41 +++++++++++++++++++++ 2 files changed, 79 insertions(+) create mode 100644 tests/docker/dockerfiles/debian-llvm.docker create mode 100755 tests/docker/test-helper-to-tcgtests/docker/dockerfiles/debian-llvm.docker new file +0
-0
tests/docker/dockerfiles/debian-llvm.docker line 1
Quoted 1 line
tests/docker/dockerfiles/debian-llvm.docker new file +1
-0@@ -0,0 +1,38 @@ +FROM docker.io/library/debian:11-slim
Debian 11 uses an old glibc (2.31) that prevents me from building QEMU.
Specifically, I get an error about missing MADV_POPULATE_READ and MADV_POPULATE_WRITE.
I suggest to switch to Debian 12, for which llvm.sh provides all the versions we're interested in *and* is compatible with the hexagon toolchain.
All other .dockers use Debian 13, but apt.llvm.org only provide 21 and 22 for it.
There's an entry for older version but Packages is empty:
https://apt.llvm.org/trixie/dists/llvm-toolchain-trixie-15/main/binary-amd64/Packages
Quoted 16 lines
tests/docker/dockerfiles/debian-llvm.docker new file +16
-0+ +RUN apt update && \ + DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ + DEBIAN_FRONTEND=noninteractive eatmydata \ + apt install -y --no-install-recommends \ + bison \ + ca-certificates \ + flex \ + gawk \ + libmpc-dev \ + libmpfr-dev \ + libglib2.0-dev \ + libpixman-1-dev \ + make \ + ninja-build \ + rsync \
tests/docker/dockerfiles/debian-llvm.docker line 27
Quoted 10 lines
tests/docker/dockerfiles/debian-llvm.docker new file +10
-0+ pkgconf \ + wget \ + lsb-release \ + software-properties-common \ + gnupg \ + meson \ + python3-pip \ + python3-setuptools \ + python3-venv \ + python3-wheel
I had to add bzip2 and libfdt-dev.
tests/docker/dockerfiles/debian-llvm.docker line 29
Quoted 2 lines
tests/docker/dockerfiles/debian-llvm.docker new file +2
-0+ +RUN /usr/bin/pip3 install tomli
Python 3.11 (in Debian 12) has tomllib in the standard library and this line should be no longer necessary.
tests/docker/dockerfiles/debian-llvm.docker line 38
Quoted 9 lines
tests/docker/dockerfiles/debian-llvm.docker new file +9
-0+ +RUN wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh +RUN ./llvm.sh 15 all +RUN ./llvm.sh 16 all +RUN ./llvm.sh 17 all +RUN ./llvm.sh 18 all +RUN ./llvm.sh 19 all +RUN ./llvm.sh 20 all +RUN ./llvm.sh 21 all
I suggest to limit building QEMU for the Hexagon target:
DEF_TARGET_LIST=hexagon-linux-user
Otherwise, I get a build error on the ARM target with clang 15:
../../src/hw/arm/fsl-imx8mp.c:684:56: error: initializer element is not a compile-time constant
fsl_imx8mp_memmap[FSL_IMX8MP_FLEXCAN1].addr,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~I guess this is temporary, given that clang 15 is supported AFAIU.
I also suggest to install the Hexagon toolchain, so we can actually test Hexagon.
ENV TOOLCHAIN_RELEASE=22.1.8
ENV TOOLCHAIN_URL=https://artifacts.codelinaro.org/artifactory/codelinaro-toolchain-for-hexagon/${TOOLCHAIN_RELEASE}________/hexagon-debs-${TOOLCHAIN_RELEASE}________.tar.gz
RUN ./llvm.sh 22 && \
mkdir -p /tmp/hexagon-debs && \
wget -qO- "$TOOLCHAIN_URL" | tar -xzC /tmp/hexagon-debs && \
DEBIAN_FRONTEND=noninteractive apt-get install -y /tmp/hexagon-debs/*.deb && \
rm -rf /tmp/hexagon-debsI tested all the containers from 15 to 21 and the toolchain is correctly picked up and tests pass (except for test_max_temps, reported in another thread).
Eventually we'll also want to use helper-to-tcg for xqciu RISC-V extensions, but even then, I think Hexagon is going to be the target exercising the most features of helper-to-tcg, so it's worth to throw it against the whole range of versions of LLVM we want to support.
Brian: I don't remember what's the status of the Hexagon toolchain, but being able to use the upstream LLVM for check-tcg would be nice. Can you remind me if anything major is missing there?
tests/docker/test-helper-to-tcg line 5
Quoted 11 lines
tests/docker/test-helper-to-tcg new file +5
-0@@ -0,0 +1,41 @@ +#!/bin/bash -e + +TEST_COMMAND="" +TARGET_LIST="" +
I think you need to . common.rc and...
Quoted 23 lines
tests/docker/test-helper-to-tcg new file +23
-0+cd "$BUILD_DIR" + +# Checks helper-to-tcg builds and passes unit tests +# for targetted LLVM versions. Run meson from QEMU source so helper-to-tcg tests can find +# `tcg-global-mappings.h`. +for version in {15..21}; do + llvm_config=llvm-config-${version} + build_dir="${BUILD_DIR}/build-helper-to-tcg-${version}" + cxx=$(${llvm_config} --bindir)/clang++ + [ ! -d ${build_dir} ] && mkdir ${build_dir} + CXX=${cxx} meson setup ${build_dir} ${QEMU_SRC}/subprojects/helper-to-tcg -Dllvm_config_path=${llvm_config} + meson compile -C ${build_dir} + meson test -C ${build_dir} --suite 'helper-to-tcg:helper-to-tcg' +done + +# Runs check-tcg for all LLVM versions +llvm_config_main=llvm-config-15 +bin_main=$(${llvm_config_main} --bindir) +cc_main=${bin_main}/clang +cxx_main=${bin_main}/clang++ +for version in {15..21}; do + llvm_config=llvm-config-${version} + build_dir=build-qemu-${version}
tests/docker/test-helper-to-tcg line 38
Quoted 10 lines
tests/docker/test-helper-to-tcg new file +10
-0+ bin=$(${llvm_config} --bindir) + cxx=${bin}/clang++ + cc=${bin}/clang + [ ! -d ${build_dir} ] && mkdir ${build_dir} + + pushd ${build_dir} + + TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \ + build_qemu "--cc=${cc_main} --cxx=${cxx_main} -Dhelper-to-tcg:llvm_config_path=${llvm_config} --enable-debug-tcg" + check_tcg
...
check_qemu check-tcg
Quoted 5 lines
tests/docker/test-helper-to-tcg new file +3
-0+ + popd +doneSignature · 1 line
2.52.0
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
Re: [PATCH v2 40/50] test: helper-to-tcg docker tests
In reply to Anton Johansson via qemu developmentQuoted 21 lines · Anton Johansson
Adds a docker container with LLVM versions 15-21 and a docker test for building helper-to-tcg and running end-to-end tests along with running check-tcg, for testing targets such as Hexagon.
Signed-off-by Anton Johansson <[email protected]>--
NOTE: Last I checked I had some troubles with C++17 support within the container, was a while ago though.--- tests/docker/dockerfiles/debian-llvm.docker | 38 +++++++++++++++++++ tests/docker/test-helper-to-tcg | 41 +++++++++++++++++++++ 2 files changed, 79 insertions(+) create mode 100644 tests/docker/dockerfiles/debian-llvm.docker create mode 100755 tests/docker/test-helper-to-tcgtests/docker/dockerfiles/debian-llvm.docker new file +0
-0
tests/docker/dockerfiles/debian-llvm.docker line 2
Quoted 2 lines
tests/docker/dockerfiles/debian-llvm.docker new file +2
-0@@ -0,0 +1,38 @@ +FROM docker.io/library/debian:11-slim +
As Alessandro mentioned in his answer, all others containers are using debian:13-slim, you can migrate to this one also.
Quoted 83 lines
tests/docker/dockerfiles/debian-llvm.docker new file +36
-0+RUN apt update && \ + DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \ + DEBIAN_FRONTEND=noninteractive eatmydata \ + apt install -y --no-install-recommends \ + bison \ + ca-certificates \ + flex \ + gawk \ + libmpc-dev \ + libmpfr-dev \ + libglib2.0-dev \ + libpixman-1-dev \ + make \ + ninja-build \ + rsync \ + pkgconf \ + wget \ + lsb-release \ + software-properties-common \ + gnupg \ + meson \ + python3-pip \ + python3-setuptools \ + python3-venv \ + python3-wheel + +RUN /usr/bin/pip3 install tomli + +RUN wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh +RUN ./llvm.sh 15 all +RUN ./llvm.sh 16 all +RUN ./llvm.sh 17 all +RUN ./llvm.sh 18 all +RUN ./llvm.sh 19 all +RUN ./llvm.sh 20 all +RUN ./llvm.sh 21 alltests/docker/test-helper-to-tcg new file +41
-0@@ -0,0 +1,41 @@ +#!/bin/bash -e + +TEST_COMMAND="" +TARGET_LIST="" + +cd "$BUILD_DIR" + +# Checks helper-to-tcg builds and passes unit tests +# for targetted LLVM versions. Run meson from QEMU source so helper-to-tcg tests can find +# `tcg-global-mappings.h`. +for version in {15..21}; do + llvm_config=llvm-config-${version} + build_dir="${BUILD_DIR}/build-helper-to-tcg-${version}" + cxx=$(${llvm_config} --bindir)/clang++ + [ ! -d ${build_dir} ] && mkdir ${build_dir} + CXX=${cxx} meson setup ${build_dir} ${QEMU_SRC}/subprojects/helper-to-tcg -Dllvm_config_path=${llvm_config} + meson compile -C ${build_dir} + meson test -C ${build_dir} --suite 'helper-to-tcg:helper-to-tcg' +done + +# Runs check-tcg for all LLVM versions +llvm_config_main=llvm-config-15 +bin_main=$(${llvm_config_main} --bindir) +cc_main=${bin_main}/clang +cxx_main=${bin_main}/clang++ +for version in {15..21}; do + llvm_config=llvm-config-${version} + build_dir=build-qemu-${version} + bin=$(${llvm_config} --bindir) + cxx=${bin}/clang++ + cc=${bin}/clang + [ ! -d ${build_dir} ] && mkdir ${build_dir} + + pushd ${build_dir} + + TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \ + build_qemu "--cc=${cc_main} --cxx=${cxx_main} -Dhelper-to-tcg:llvm_config_path=${llvm_config} --enable-debug-tcg" + check_tcg + + popd +done
PATCH v2 41/50 target/hexagon: Add get_tb_mmu_index()
In reply to Anton Johansson via qemu developmentAdds a functions to return the current mmu index given tb_flags of the current translation block. Required by helper-to-tcg in order to retrieve the mmu index for memory operations without changing the signature of helper functions.
--- target/hexagon/translate.c | 12 ++++++++++++ target/hexagon/translate.h | 4 ++++ 2 files changed, 16 insertions(+)
target/hexagon/translate.c +12 -0
@@ -128,6 +128,18 @@ intptr_t ctx_tmp_vreg_off(DisasContext *ctx, int regnum,return offset; } +#if defined(TARGET_HELPER_TO_TCG ) +/* +* Returns the current mmu index given tb_flags of the current translation +* block. Required by helper-to-tcg in order to retrieve the mmu index for +* memory operations without changing the signature of helper functions. +*/ +int get_tb_mmu_index (uint32_t flags) +{ +return FIELD_EX32 (flags,TB_FLAGS ,MMU_INDEX ); +} +#endif +static void gen_exception (int excp,uint32_t PC ) { gen_helper_raise_exception(tcg_env, tcg_constant_i32(excp),
target/hexagon/translate.h +4 -0
@@ -345,4 +345,8 @@ FIELD(PROBE_PKT_SCALAR_HVX_STORES, MMU_IDX, 5, 2)void gen_framecheck (DisasContext *ctx,TCGv_i32 addr,TCGv_i32 ea); +#if defined(TARGET_HELPER_TO_TCG ) +int get_tb_mmu_index (uint32_t flags); +#endif +#endif
Signature · 1 line
2.52.0
Re: [PATCH v2 41/50] target/hexagon: Add get_tb_mmu_index()
Hi Anton,
Quoted 16 lines · Anton Johansson via qemu development
Adds a functions to return the current mmu index given
tb_flagsof the current translation block. Required by helper-to-tcg in order to retrieve the mmu index for memory operations without changing the signature of helper functions.Signed-off-by Anton Johansson <[email protected]>--- target/hexagon/translate.c | 12 ++++++++++++ target/hexagon/translate.h | 4 ++++ 2 files changed, 16 insertions(+)target/hexagon/translate.c +0
-0
target/hexagon/translate.c intptr_t ctx_tmp_vreg_off(DisasContext *ctx, int regnum, · line 136
Quoted 9 lines
target/hexagon/translate.c +6
-0@@ -128,6 +128,18 @@ intptr_t ctx_tmp_vreg_off(DisasContext *ctx, int regnum,return offset; } +#if defined(TARGET_HELPER_TO_TCG ) +/* +* Returns the current mmu index given tb_flags of the current translation +* block. Required by helper-to-tcg in order to retrieve the mmu index for +* memory operations without changing the signature of helper functions. +*/
No need to guard for TARGET_HELPER_TO_TCG IMO, simply add and use in hexagon_tr_init_disas_context().
Dropping the "Required by helper-to-tcg ..." comment, the TARGET_HELPER_TO_TCG guard and using in init_disas_context:
Quoted 20 lines
target/hexagon/translate.c +6
-0+int get_tb_mmu_index (uint32_t flags) +{ +return FIELD_EX32 (flags,TB_FLAGS ,MMU_INDEX ); +} +#endif +static void gen_exception (int excp,uint32_t PC ) { gen_helper_raise_exception(tcg_env, tcg_constant_i32(excp),target/hexagon/translate.h +3
-0@@ -345,4 +345,8 @@ FIELD(PROBE_PKT_SCALAR_HVX_STORES, MMU_IDX, 5, 2)void gen_framecheck (DisasContext *ctx,TCGv_i32 addr,TCGv_i32 ea); +#if defined(TARGET_HELPER_TO_TCG ) +int get_tb_mmu_index (uint32_t flags); +#endif
PATCH v2 42/50 target/hexagon: Increase VECTOR_TEMPS_MAX
In reply to Anton Johansson via qemu developmentDoubles the number of vector temporaries from 4 to 8, useful when emitting vector instructions with helper-to-tcg to handle more complicated helpers. tmp_VRegs must be later specified later to helper-to-tcg through --temp-vector-block=tmp_VRegs.
--- target/hexagon/cpu.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
target/hexagon/cpu.h +5 -1
@@ -117,7 +117,7 @@ typedef struct {#define SET_EXCEPTION (env->status |= EXEC_STATUS_EXCEPTION)/* Maximum number of vector temps in a packet */ -#define VECTOR_TEMPS_MAX 4 +#define VECTOR_TEMPS_MAX 8 typedef struct CPUArchState { target_ulong gpr[TOTAL_PER_THREAD_REGS];
@@ -160,6 +160,10 @@ typedef struct CPUArchState { MMVector future_VRegs[VECTOR_TEMPS_MAX] QEMU_ALIGNED(16); MMVector tmp_VRegs[VECTOR_TEMPS_MAX] QEMU_ALIGNED(16); +#ifdef TARGET_HELPER_TO_TCG + uint8_t tmp_vmem[4096] QEMU_ALIGNED(16); +#endif + MMQReg QRegs[NUM_QREGS] QEMU_ALIGNED(16); MMQReg future_QRegs[NUM_QREGS] QEMU_ALIGNED(16);
Signature · 1 line
2.52.0
PATCH v2 43/50 target/hexagon: Provide env to tcg global mapping
In reply to Anton Johansson via qemu developmentReplaces previous calls to tcg_global_mem_new*() with a declarative global array of cpu_mapping structs. This array can be used to initialize all TCG globals with one function call from the target, and may additionally be used from LLVM based tools to map between offsets into a struct and a mapped TCGv global.
--- target/hexagon/translate.c | 119 ++++++++++++++++++------------------- 1 file changed, 57 insertions(+), 62 deletions(-)
target/hexagon/translate.c +57 -62
@@ -33,6 +33,7 @@#include "genptr.h" #include "printinsn.h" #include "exec/target_page.h" +#include "tcg/tcg-global-mappings.h" #define HELPER_H "helper.h"#include "exec/helper-info.c.inc"
@@ -1328,77 +1329,68 @@ static char store_val64_names[STORES_MAX][NAME_LEN];static char vstore_addr_names[VSTORES_MAX ][NAME_LEN ];static char vstore_size_names[VSTORES_MAX ][NAME_LEN ];static char vstore_pending_names[VSTORES_MAX ][NAME_LEN ]; - -void hexagon_translate_init (void ) -{ -int i; - -opcode_init (); +static const char *store_addr_names_ptr[STORES_MAX ]; +static const char *store_width_names_ptr[STORES_MAX ]; +static const char *store_val32_names_ptr[STORES_MAX ]; +static const char *store_val64_names_ptr[STORES_MAX ]; + +cpu_tcg_mapping tcg_global_mappings[] = { +/* General purpose and predicate registers */ +CPU_TCG_MAP_ARRAY (CPUHexagonState, hex_gpr, gpr, hexagon_regnames), +CPU_TCG_MAP_ARRAY (CPUHexagonState, hex_pred, pred, hexagon_prednames), + +/* Misc */ +CPU_TCG_MAP (CPUHexagonState, hex_new_value_usr, new_value_usr), +CPU_TCG_MAP (CPUHexagonState, hex_next_PC, next_PC), +CPU_TCG_MAP (CPUHexagonState, hex_slot_cancelled, slot_cancelled), +CPU_TCG_MAP (CPUHexagonState, hex_llsc_addr, llsc_addr), +CPU_TCG_MAP (CPUHexagonState, hex_llsc_val, llsc_val), +CPU_TCG_MAP (CPUHexagonState, hex_llsc_val_i64, llsc_val_i64), #ifndef CONFIG_USER_ONLY -for (i =0 ; i <NUM_GREGS ; i++) { - hex_greg[i] =tcg_global_mem_new_i32 (tcg_env, - offsetof(CPUHexagonState, greg[i]), - hexagon_gregnames[i]); - } -for (i =0 ; i <NUM_SREGS ; i++) { -if (i <HEX_SREG_GLB_START ) { - hex_t_sreg[i] =tcg_global_mem_new_i32 (tcg_env, - offsetof(CPUHexagonState, t_sreg[i]), - hexagon_sregnames[i]); - } - } +CPU_TCG_MAP (CPUHexagonState, hex_cause_code, cause_code), +CPU_TCG_MAP (CPUHexagonState, hex_cycle_count, t_cycle_count), +CPU_TCG_MAP_ARRAY (CPUHexagonState, hex_greg, greg, hexagon_gregnames), +CPU_TCG_MAP_ARRAY (CPUHexagonState, hex_t_sreg, t_sreg, hexagon_sregnames), #endif -for (i =0 ; i <TOTAL_PER_THREAD_REGS ; i++) { - hex_gpr[i] =tcg_global_mem_new (tcg_env, - offsetof(CPUHexagonState, gpr[i]), - hexagon_regnames[i]); - } - hex_new_value_usr =tcg_global_mem_new (tcg_env, - offsetof(CPUHexagonState , new_value_usr),"new_value_usr" ); - hex_next_PC =tcg_global_mem_new (tcg_env, - offsetof(CPUHexagonState , next_PC),"next_PC" ); -for (i =0 ; i <NUM_PREGS ; i++) { - hex_pred[i] =tcg_global_mem_new (tcg_env, - offsetof(CPUHexagonState, pred[i]), - hexagon_prednames[i]); - } - hex_slot_cancelled =tcg_global_mem_new (tcg_env, - offsetof(CPUHexagonState , slot_cancelled),"slot_cancelled" ); - hex_llsc_addr =tcg_global_mem_new (tcg_env, - offsetof(CPUHexagonState , llsc_addr),"llsc_addr" ); - hex_llsc_val =tcg_global_mem_new (tcg_env, - offsetof(CPUHexagonState , llsc_val),"llsc_val" ); - hex_llsc_val_i64 =tcg_global_mem_new_i64 (tcg_env, - offsetof(CPUHexagonState , llsc_val_i64),"llsc_val_i64" ); -#ifndef CONFIG_USER_ONLY - hex_cause_code =tcg_global_mem_new_i32 (tcg_env, - offsetof(CPUHexagonState , cause_code),"cause_code" ); - hex_cycle_count =tcg_global_mem_new_i64 (tcg_env, - offsetof(CPUHexagonState , t_cycle_count),"t_cycle_count" ); -#endif -for (i =0 ; i <STORES_MAX ; i++) { -snprintf (store_addr_names[i],NAME_LEN ,"store_addr_%d" , i); - hex_store_addr[i] =tcg_global_mem_new (tcg_env, - offsetof(CPUHexagonState, mem_log_stores[i].va), - store_addr_names[i]); +/* Logging stores */ +CPU_TCG_MAP_ARRAY_OF_STRUCTS (CPUHexagonState, hex_store_addr, + mem_log_stores, va, store_addr_names_ptr), +CPU_TCG_MAP_ARRAY_OF_STRUCTS (CPUHexagonState, hex_store_width, + mem_log_stores, width, store_width_names_ptr), +CPU_TCG_MAP_ARRAY_OF_STRUCTS (CPUHexagonState, hex_store_val32, + mem_log_stores, data32, store_val32_names_ptr), +CPU_TCG_MAP_ARRAY_OF_STRUCTS (CPUHexagonState, hex_store_val64, + mem_log_stores, data64, store_val64_names_ptr), +}; -snprintf (store_width_names[i],NAME_LEN ,"store_width_%d" , i); - hex_store_width[i] =tcg_global_mem_new_i32 (tcg_env, - offsetof(CPUHexagonState, mem_log_stores[i].width), - store_width_names[i]); +size_t tcg_global_mapping_count =ARRAY_SIZE (tcg_global_mappings); +static void init_cpu_reg_names (void ) { +/* +* Create register names and store them in `*_names`, +* then copy to and array of pointers in `*_names_ptr` +* which is easier to pass around. +*/ +for (int i =0 ; i <STORES_MAX ; ++i) { +snprintf (store_addr_names[i],NAME_LEN ,"store_addr_%d" , i); +snprintf (store_width_names[i],NAME_LEN ,"store_width_%d" , i);snprintf (store_val32_names[i],NAME_LEN ,"store_val32_%d" , i); - hex_store_val32[i] =tcg_global_mem_new (tcg_env, - offsetof(CPUHexagonState, mem_log_stores[i].data32), - store_val32_names[i]); -snprintf (store_val64_names[i],NAME_LEN ,"store_val64_%d" , i); - hex_store_val64[i] =tcg_global_mem_new_i64 (tcg_env, - offsetof(CPUHexagonState, mem_log_stores[i].data64), - store_val64_names[i]); + store_addr_names_ptr[i] = store_addr_names[i]; + store_width_names_ptr[i] = store_width_names[i]; + store_val32_names_ptr[i] = store_val32_names[i]; + store_val64_names_ptr[i] = store_val64_names[i]; } +} + +void hexagon_translate_init (void ) +{ +int i; + +opcode_init (); +for (i =0 ; i <VSTORES_MAX ; i++) {snprintf (vstore_addr_names[i],NAME_LEN ,"vstore_addr_%d" , i); hex_vstore_addr[i] = tcg_global_mem_new(tcg_env,
@@ -1415,4 +1407,7 @@ void hexagon_translate_init(void) offsetof(CPUHexagonState, vstore_pending[i]), vstore_pending_names[i]); } + +init_cpu_reg_names (); +init_cpu_tcg_mappings (tcg_global_mappings, tcg_global_mapping_count); }
Signature · 1 line
2.52.0
Re: [PATCH v2 43/50] target/hexagon: Provide env to tcg global mapping
Quoted 1 line · Anton Johansson via qemu development
+cpu_tcg_mapping tcg_global_mappings[] = {
static const
Quoted 1 line
+size_t
tcg_global_mapping_count=ARRAY_SIZE(tcg_global_mappings);
Drop this and propagate the ARRAY_SIZE to
Quoted 1 line
+
init_cpu_tcg_mappings(tcg_global_mappings, tcg_global_mapping_count);
here.
r~
PATCH v2 44/50 target/hexagon: Keep gen_slotval/check_noshuf for helper-to-tcg
In reply to Anton Johansson via qemu developmentMakes sure gen_slotval() and check_noshuf() remains defined when helper-to-tcg and idef-parser are both used. gen_slotval() is needed for creating a TCGv of the slot value fed to helpers (generated helper-to-tcg code), and check_noshuf() is needed for helper definitions used as input to helper-to-tcg.
--- target/hexagon/genptr.c | 2 +- target/hexagon/op_helper.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
target/hexagon/genptr.c +1 -1
@@ -533,7 +533,7 @@ static inline void gen_store_conditional8(DisasContext *ctx,tcg_gen_movi_tl (hex_llsc_addr, ~0 ); } -#ifndef CONFIG_HEXAGON_IDEF_PARSER +#if !defined(CONFIG_HEXAGON_IDEF_PARSER ) || defined(TARGET_HELPER_TO_TCG )static TCGv gen_slotval (DisasContext *ctx) { int slotval =
target/hexagon/op_helper.c +1 -1
@@ -488,7 +488,7 @@ void HELPER(probe_pkt_scalar_hvx_stores)(CPUHexagonState *env, int mask) } } -#ifndef CONFIG_HEXAGON_IDEF_PARSER +#if !defined(CONFIG_HEXAGON_IDEF_PARSER ) || defined(CONFIG_HELPER_TO_TCG )/* * mem_noshuf * Section 5.5 of the Hexagon V67 Programmer's Reference Manual
Signature · 1 line
2.52.0
Re: [PATCH v2 44/50] target/hexagon: Keep gen_slotval/check_noshuf for helper-to-tcg
target/hexagon/op_helper.c void HELPER(probe_pkt_scalar_hvx_stores)(CPUHexagonState *env, int mask) · line 494
Quoted 13 lines · On Thu, 30 Jul 2026 05:10:18 +0200 Anton Johansson via qemu development…
target/hexagon/op_helper.c +1
-1@@ -488,7 +488,7 @@ void HELPER(probe_pkt_scalar_hvx_stores)(CPUHexagonState *env, int mask) } } -#ifndef CONFIG_HEXAGON_IDEF_PARSER +#if !defined(CONFIG_HEXAGON_IDEF_PARSER ) || defined(CONFIG_HELPER_TO_TCG )/* * mem_noshuf * Section 5.5 of the Hexagon V67 Programmer's Reference Manual
This change prevents me from building with-Dhexagon_helper_to_tcg=disabled.
Maybe I'm doing something wrong.
Signature · 2 lines
Alessandro Di Federico rev.ng Labs
PATCH v2 45/50 target/hexagon: Emit annotations for helpers
In reply to Anton Johansson via qemu developmentAdds the following LLVM_ANNOTATE attributes to helper functions generated by Hexagon:
1. "helper-to-tcg", to specify that a given helper functions should be
translated, and;
2. "immediate: ..." to make sure immediate arguments to helper functions remain immediates in the emitted TCG code (e.g. slot).
3. "ptr-to-offset: ..." to make sure pointer arguments are treated as immediates representing an offset into the CPU state, needed to work with gvec.
Two functions are also added to hex_common.py, to firstly parse the generated file containing all successfully translated helper functions, and secondly to expose the indices of immediate and pointer (vector) arguments to helper functions. The latter is needed to generate the input of helper-to-tcg.
--- target/hexagon/gen_helper_funcs.py | 17 ++++++++++- target/hexagon/gen_helper_protos.py | 2 +- target/hexagon/gen_tcg_funcs.py | 2 +- target/hexagon/hex_common.py | 45 +++++++++++++++++++++++------ 4 files changed, 54 insertions(+), 12 deletions(-)
target/hexagon/gen_helper_funcs.py +16 -1
@@ -41,9 +41,23 @@ def gen_helper_function(f, tag, tagregs, tagimms): ret_type = hex_common.helper_ret_type(tag, regs).func_arg declared = [] - for arg in hex_common.helper_args(tag, regs, imms): + helper_args, imm_inds, hvx_inds = hex_common.helper_args(tag, regs, imms) + for arg in helper_args: declared.append(arg.func_arg) + ## Specify that helpers should be translated by helper-to-tcg + f.write(f'QEMU_ANNOTATE("helper-to-tcg")\n') + ## Specify which arguments to the helper function should be treated as + ## immediate arguments + if len(imm_inds) > 0: + imm_inds_str = ','.join(str(i) for i in imm_inds) + f.write(f'QEMU_ANNOTATE("immediate: {imm_inds_str}")\n') + ## Specify which arguments to the helper function should be treated as + ## gvec vectors + if len(hvx_inds) > 0: + hvx_inds_str = ','.join(str(i) for i in hvx_inds) + f.write(f'QEMU_ANNOTATE("ptr-to-offset: {hvx_inds_str}")\n') + arguments = ", ".join(declared) f.write(f"{ret_type} HELPER({tag})({arguments})\n") f.write("{\n")
@@ -51,6 +65,7 @@ def gen_helper_function(f, tag, tagregs, tagimms): f.write(hex_common.code_fmt(f"""\ uint32_t EA; """)) + ## Declare the return variable if not hex_common.is_predicated(tag): for regtype, regid in regs:
target/hexagon/gen_helper_protos.py +1 -1
@@ -36,7 +36,7 @@ def gen_helper_prototype(f, tag, tagregs, tagimms): ret_type = hex_common.helper_ret_type(tag, regs).proto_arg declared.append(ret_type) - for arg in hex_common.helper_args(tag, regs, imms): + for arg in hex_common.helper_args(tag, regs, imms)[0]: declared.append(arg.proto_arg) arguments = ", ".join(declared)
target/hexagon/gen_tcg_funcs.py +1 -1
@@ -100,7 +100,7 @@ def gen_tcg_func(f, tag, regs, imms): if ret_type != "void": declared.append(ret_type) - for arg in hex_common.helper_args(tag, regs, imms): + for arg in hex_common.helper_args(tag, regs, imms)[0]: declared.append(arg.call_arg) arguments = ", ".join(declared)
target/hexagon/hex_common.py +36 -9
176 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v2 46/50 target/hexagon: Split probe_and_commit helper
In reply to Anton Johansson via qemu developmentSplit function into a slowpath such that the faster function can be translated by helper-to-tcg.
--- target/hexagon/helper.h | 1 + target/hexagon/op_helper.c | 17 ++++++++++++++--- 2 files changed, 15 insertions(+), 3 deletions(-)
target/hexagon/helper.h +1 -0
@@ -107,6 +107,7 @@ DEF_HELPER_4(probe_noshuf_load, void, env, i32, int, int) DEF_HELPER_2(probe_pkt_scalar_store_s0, void, env, int) DEF_HELPER_2(probe_hvx_stores, void, env, int) DEF_HELPER_2(probe_pkt_scalar_hvx_stores, void, env, int) +DEF_HELPER_4(probe_and_commit, void, env, i32, s32, i64)#if !defined(CONFIG_USER_ONLY )DEF_HELPER_3 (raise_stack_overflow, void, env, i32, i32)
target/hexagon/op_helper.c +14 -3
@@ -488,7 +488,8 @@ void HELPER(probe_pkt_scalar_hvx_stores)(CPUHexagonState *env, int mask) } } -#if !defined(CONFIG_HEXAGON_IDEF_PARSER ) || defined(CONFIG_HELPER_TO_TCG ) +#if !defined(CONFIG_HEXAGON_IDEF_PARSER ) || defined(TARGET_HELPER_TO_TCG ) +/* * mem_noshuf * Section 5.5 of the Hexagon V67 Programmer's Reference Manual
@@ -500,8 +501,18 @@ static void check_noshuf(CPUHexagonState *env, bool pkt_has_scalar_store_s1, uint32_t slot, target_ulong vaddr, int size, uintptr_t ra) { - if (slot == 0 && pkt_has_scalar_store_s1 && - ((env->slot_cancelled & (1 << 1)) == 0)) { + if (slot == 0 && pkt_has_scalar_store_s1) { +helper_probe_and_commit (env ,vaddr ,size ,ra ); + } +} + +/* +* +*/ +void HELPER (probe_and_commit )(CPUHexagonState *env,target_ulong vaddr, +int size,uintptr_t ra) +{ +if ((env->slot_cancelled & (1 <<1 )) ==0 ) {probe_read (env, vaddr, size,MMU_USER_IDX , ra);commit_store (env,1 , ra); }
Signature · 1 line
2.52.0
Re: [PATCH v2 46/50] target/hexagon: Split probe_and_commit helper
Quoted 39 lines · Anton Johansson via qemu development
Split function into a slowpath such that the faster function can be translated by helper-to-tcg.
Signed-off-by Anton Johansson <[email protected]>--- target/hexagon/helper.h | 1 + target/hexagon/op_helper.c | 17 ++++++++++++++--- 2 files changed, 15 insertions(+), 3 deletions(-)target/hexagon/helper.h +1
-0@@ -107,6 +107,7 @@ DEF_HELPER_4(probe_noshuf_load, void, env, i32, int, int) DEF_HELPER_2(probe_pkt_scalar_store_s0, void, env, int) DEF_HELPER_2(probe_hvx_stores, void, env, int) DEF_HELPER_2(probe_pkt_scalar_hvx_stores, void, env, int) +DEF_HELPER_4(probe_and_commit, void, env, i32, s32, i64)#if !defined(CONFIG_USER_ONLY )DEF_HELPER_3 (raise_stack_overflow, void, env, i32, i32)target/hexagon/op_helper.c +2
-1@@ -488,7 +488,8 @@ void HELPER(probe_pkt_scalar_hvx_stores)(CPUHexagonState *env, int mask) } } -#if !defined(CONFIG_HEXAGON_IDEF_PARSER ) || defined(CONFIG_HELPER_TO_TCG ) +#if !defined(CONFIG_HEXAGON_IDEF_PARSER ) || defined(TARGET_HELPER_TO_TCG ) +/* * mem_noshuf * Section 5.5 of the Hexagon V67 Programmer's Reference Manual @@ -500,8 +501,18 @@ static void check_noshuf(CPUHexagonState *env, bool pkt_has_scalar_store_s1, uint32_t slot, target_ulong vaddr, int size, uintptr_t ra) {
target/hexagon/op_helper.c static void check_noshuf(CPUHexagonState *env, bool pkt_has_scalar_store_s1, · line 511
Quoted 10 lines
target/hexagon/op_helper.c +8
-2-if (slot ==0 && pkt_has_scalar_store_s1 && - ((env->slot_cancelled & (1 <<1 )) ==0 )) { +if (slot ==0 && pkt_has_scalar_store_s1) { +helper_probe_and_commit (env, vaddr, size, ra); + } +} + +/* +* +*/
Either fill the comment block, or drop it :) Otherwise:
Quoted 7 lines
target/hexagon/op_helper.c +4
-0+void HELPER (probe_and_commit )(CPUHexagonState *env,target_ulong vaddr, +int size,uintptr_t ra) +{ +if ((env->slot_cancelled & (1 <<1 )) ==0 ) {probe_read (env, vaddr, size,MMU_USER_IDX , ra);commit_store (env,1 , ra); }
PATCH v2 47/50 target/hexagon: Use helper-to-tcg helper calls
In reply to Anton Johansson via qemu developmentReplaces the default "helper-gen.h" header with those provided by helper-to-tcg. gen_helper_*() definitions are generated for all helpers in the input module, regardless of whether they successfully translated or not. The generated gen_helper_*() calls into either emit_*() or tcg_gen_callN depending on whether translation of that helper was successful or not.
--- target/hexagon/genptr.c | 4 ++++ target/hexagon/helper.h | 8 ++++++++ target/hexagon/translate.c | 8 ++++++++ 3 files changed, 20 insertions(+)
target/hexagon/genptr.c +4 -0
@@ -20,7 +20,11 @@#include "internal.h" #include "tcg/tcg-op.h" #include "tcg/tcg-op-gvec.h" +#if defined(TARGET_HELPER_TO_TCG ) +#include "helper-to-tcg-emitted.h" +#else #include "exec/helper-gen.h" +#endif #include "insn.h" #include "opcodes.h" #include "sys_macros.h"
target/hexagon/helper.h +8 -0
@@ -133,3 +133,11 @@ DEF_HELPER_1(resched, void, env) DEF_HELPER_3(modify_ssr, void, env, i32, i32) DEF_HELPER_1(pending_interrupt, void, env)#endif + +/* +* Include generated helper-to-tcg support helpers, but not during IR +* generation since they won't have been emitted at that point. +*/ +#if defined(TARGET_HELPER_TO_TCG ) && !defined(HELPER_TO_TCG_IR_GEN ) +# include "helper-to-tcg-support-helpers.h" +#endif
target/hexagon/translate.c +8 -0
@@ -20,7 +20,15 @@#include "cpu.h" #include "tcg/tcg-op.h" #include "tcg/tcg-op-gvec.h" +/* +* Include generated gen_helper_*() definitions if using helper-to-tcg, but +* only when not generating input IR since the header won't exist at that point. +*/ +#if defined(TARGET_HELPER_TO_TCG ) && !defined(HELPER_TO_TCG_IR_GEN ) +#include "helper-to-tcg-emitted.h" +#else #include "exec/helper-gen.h" +#endif #include "exec/helper-proto.h" #include "exec/translation-block.h" #include "accel/tcg/cpu-ldst.h"
Signature · 1 line
2.52.0
PATCH v2 48/50 target/hexagon: Manually call generated HVX instructions
In reply to Anton Johansson via qemu developmentFor HVX instructions that were successfully translated by helper-to-tcg, emit calls to emit_*() "manually" from generate_*(). Recall that scalar instructions translated by helper-to-tcg are automatically called by a hook in tcg_gen_callN.
--- target/hexagon/gen_tcg_funcs.py | 14 ++++++++ target/hexagon/hex_common.py | 58 +++++++++++++++++++++++++++++++++ 2 files changed, 72 insertions(+)
target/hexagon/gen_tcg_funcs.py +14 -0
@@ -86,7 +86,21 @@ def gen_tcg_func(f, tag, regs, imms): arguments = ", ".join(["ctx", "ctx->insn", "&ctx->pkt"] + declared) f.write(f" emit_{tag}({arguments});\n") + elif hex_common.is_helper_to_tcg_enabled(tag) and tag.startswith("V6_"): + ## For vector functions translated by helper-to-tcg we need to + ## manually call the emitted code. All other instructions translated + ## are automatically called by the helper-functions dispatcher in + ## tcg_gen_callN. + declared = [] + ## Handle registers + ret_type = hex_common.helper_ret_type(tag, regs).call_arg + if ret_type != "void": + declared.append(ret_type) + for arg in hex_common.helper_to_tcg_hvx_call_args(tag, regs, imms): + declared.append(arg) + arguments = ", ".join(declared) + f.write(f" emit_{tag}({arguments});\n") elif hex_common.skip_qemu_helper(tag): if "A_FPOP" in hex_common.attribdict[tag]: f.write(" TCGv pkt_need_commit = ")
target/hexagon/hex_common.py +58 -0
76 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v2 49/50 target/hexagon: Use idef-parser as a fallback
In reply to Anton Johansson via qemu developmentOnly generate input functions to idef-parser for instructions which failed to be translated by helper-to-tcg, and which have no overrides.
--- target/hexagon/gen_idef_parser_funcs.py | 9 +++++++++ 1 file changed, 9 insertions(+)
target/hexagon/gen_idef_parser_funcs.py +9 -0
@@ -49,10 +49,13 @@ def main(): ) parser.add_argument("semantics", help="semantics file") parser.add_argument("out", help="output file") + parser.add_argument("--helper-to-tcg", help="file of instructions translated by helper-to-tcg") args = parser.parse_args() hex_common.read_semantics_file(args.semantics) hex_common.calculate_attribs() hex_common.init_registers() + if args.helper_to_tcg: + hex_common.read_helper_to_tcg_enabled_file(args.helper_to_tcg) tagregs = hex_common.get_tagregs() tagimms = hex_common.get_tagimms()
@@ -62,6 +65,12 @@ def main(): for tag in hex_common.tags: if hex_common.tag_ignore(tag): continue + ## Skip instructions with overrides + if hex_common.skip_qemu_helper(tag): + continue + ## Skip instructions translated by helper-to-tcg + if hex_common.is_helper_to_tcg_enabled(tag): + continue ## Skip the priv instructions if "A_PRIV" in hex_common.attribdict[tag]: continue
Signature · 1 line
2.52.0
PATCH v2 50/50 target/hexagon: Use helper-to-tcg
In reply to Anton Johansson via qemu developmentModifies meson.build to use helper-to-tcg for automatic translation of helper functions. Any helper functions with the "helper-to-tcg" attribute will be automatically translated to TCG.
Order of code generation is changed, and helper functions are always generated first, for all instructions. Helper functions are needed as input helper-to-tcg. Next, input to idef-parser is generated for all instructions that were not successfully translated by helper-to-tcg.
As such, a majority of instructions will be translated by helper-to-tcg, and the remaining instructions fed through idef-parser can be reduced moving forward.
--- configs/targets/hexagon-linux-user.mak | 1 + configs/targets/hexagon-softmmu.mak | 1 + target/hexagon/meson.build | 148 ++++++++++++++++++------- 3 files changed, 110 insertions(+), 40 deletions(-)
configs/targets/hexagon-linux-user.mak +1 -0
@@ -4,3 +4,4 @@ TARGET_SYSTBL=syscall.tbl TARGET_SYSTBL_ABI=common,32,hexagon,time32,stat64,rlimit,renameat TARGET_LONG_BITS=32 TARGET_NOT_USING_LEGACY_NATIVE_ENDIAN_API=y +TARGET_HELPER_TO_TCG=y
configs/targets/hexagon-softmmu.mak +1 -0
@@ -6,3 +6,4 @@ TARGET_LONG_BITS=32 TARGET_NOT_USING_LEGACY_LDST_PHYS_API=y TARGET_NOT_USING_LEGACY_NATIVE_ENDIAN_API=y TARGET_NEED_FDT=y +TARGET_HELPER_TO_TCG=y
target/hexagon/meson.build +108 -40
185 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v2 50/50] target/hexagon: Use helper-to-tcg
target/hexagon/meson.build hexagon_softmmu_ss.add(files( · line 330
Quoted 6 lines · On Thu, 30 Jul 2026 05:10:24 +0200 Anton Johansson via qemu development…
target/hexagon/meson.build +6
-0+ ll = custom_target('to-ll', + input: helper_to_tcg_input_files, + output:'helper-to-tcg-input.ll', + depends: [helper_funcs_generated, helper_protos_generated, libqemuutil], + command: helper_to_tcg_get_llvm_ir_cmd + ['-o', '@OUTPUT@', '@INPUT@', '--target-path', 'target/hexagon'] + )
Currently we're not tracking dependencies correctly.
For instance, if you change target/hexagon/macros.h and run ninja target/hexagon/helper-to-tcg-input.ll, the target is not rebuilt.
This can be fixed in get-llvm-ir.py by passing -MD -MF output.ll.d to clang, merging all the .d files you get into a single one and then passing it as depfile: in the custom_target invocation.
We should also depend on
meson.current_build_dir() / 'compile_commands.json'
Signature · 2 lines
Alessandro Di Federico rev.ng Labs