target/arm: ISV=0 data abort emulation library
v5 cover target/arm: ISV=0 data abort emulation library
Add a shared emulation library for AArch64 load/store instructions that cause ISV=0 data aborts under hardware virtualization, and wire it into HVF (macOS) and WHPX (Windows).
When the Instruction Syndrome Valid bit is clear, the hypervisor cannot determine the faulting instruction's target register or access size from the syndrome alone. This previously hit an assert(isv) and killed the VM. The library fetches and decodes the faulting instruction using a decodetree-generated decoder, then emulates it directly against the vCPU register file and memory.
The library uses its own a64-ldst.decode rather than sharing target/arm/tcg/a64.decode — TCG's trans_* functions emit IR into a translation block, while this library's execute directly. Decode patterns are kept consistent with TCG's where possible; differences are noted in the relevant commit messages.
Changes since v4:
- Rebased onto current master
- Add SPDX license identifier to new meson.build
- Resent as new top-level thread (Alex Bennée)
Changes since v3:
- Document decodetree pattern differences from TCG in commit messages for patches 1/6 and 5/6.
Changes since v2:
- Inject synchronous external abort (matching kvm_inject_arm_sea() syndrome) on unhandled instruction or memory error, instead of silently advancing PC or returning an error.
- Fix WHPX advance_pc bug: error paths no longer advance PC.
- Add page-crossing guard in mem_read/mem_write to prevent partial side effects from cpu_memory_rw_debug().
Changes since v1:
- Split monolithic patch into 6 incremental patches: framework, then one patch per coherent instruction group (Peter)
- Removed per-backend callback ops; library uses CPUArchState directly with cpu_memory_rw_debug() for memory access (Mohamed)
- Removed mock unit tests (Mohamed; kvm-unit-tests is the right vehicle for decoder validation)
- Added architectural justification for separate decode file
Lucas Amaral (6):
target/arm/emulate: add ISV=0 emulation library with load/store
immediate
target/arm/emulate: add load/store register offset target/arm/emulate: add load/store pair target/arm/emulate: add load/store exclusive target/arm/emulate: add atomic, compare-and-swap, and PAC load target/arm/hvf,whpx: wire ISV=0 emulation for data aborts
target/arm/emulate/a64-ldst.decode | 293 +++++++++++ target/arm/emulate/arm_emulate.c | 758 +++++++++++++++++++++++++++++ target/arm/emulate/arm_emulate.h | 30 ++ target/arm/emulate/meson.build | 8 + target/arm/hvf/hvf.c | 46 +- target/arm/meson.build | 1 + target/arm/whpx/whpx-all.c | 61 ++- 7 files changed, 1193 insertions(+), 4 deletions(-) create mode 100644 target/arm/emulate/a64-ldst.decode create mode 100644 target/arm/emulate/arm_emulate.c create mode 100644 target/arm/emulate/arm_emulate.h create mode 100644 target/arm/emulate/meson.build
Signature · 1 line
2.52.0
PATCH v5 1/6 target/arm/emulate: add ISV=0 emulation library with load/store immediate
Add a shared emulation library for AArch64 load/store instructions that cause ISV=0 data aborts under hardware virtualization (HVF, WHPX).
When the Instruction Syndrome Valid bit is clear, the hypervisor cannot determine the faulting instruction's target register or access size from the syndrome alone. This library fetches and decodes the instruction using a decodetree-generated decoder, then emulates it by accessing the vCPU's register file (CPUARMState) and memory (cpu_memory_rw_debug) directly.
This patch establishes the framework and adds load/store single with immediate addressing — the most common ISV=0 trigger. Subsequent patches add register-offset, pair, exclusive, and atomic instructions.
Instruction coverage:
- STR/LDR (GPR): unscaled, post-indexed, unprivileged, pre-indexed, unsigned offset — all sizes (8/16/32/64-bit), sign/zero extension
- STR/LDR (SIMD/FP): same addressing modes, 8-128 bit elements
- PRFM: prefetch treated as NOP
- DC cache maintenance (SYS CRn=C7): NOP on MMIO
This library uses its own a64-ldst.decode rather than sharing target/arm/tcg/a64.decode. TCG's trans_* functions are a compiler:
they emit IR ops into a translation block for later execution. This library's trans_* functions are an interpreter: they execute directly against the vCPU register file and memory. The decodetree-generated dispatcher calls trans_* by name, so both cannot coexist in the same translation unit. Decode patterns are kept consistent with TCG's where possible.
Decodetree differences from TCG:
- &ldst_imm adds a 'u' flag to distinguish 9-bit signed vs 12-bit unsigned immediate forms. TCG uses %uimm_scaled to pre-scale the unsigned immediate at decode time; here imm:12 is extracted raw and the handler scales it.
--- target/arm/emulate/a64-ldst.decode | 129 ++++++++++++++++ target/arm/emulate/arm_emulate.c | 237 +++++++++++++++++++++++++++++ target/arm/emulate/arm_emulate.h | 30 ++++ target/arm/emulate/meson.build | 8 + target/arm/meson.build | 1 + 5 files changed, 405 insertions(+) create mode 100644 target/arm/emulate/a64-ldst.decode create mode 100644 target/arm/emulate/arm_emulate.c create mode 100644 target/arm/emulate/arm_emulate.h create mode 100644 target/arm/emulate/meson.build
target/arm/emulate/a64-ldst.decode new file +129 -0
@@ -0,0 +1,129 @@ +# AArch64 load/store instruction patterns for ISV=0 emulation +# +# Copyright (c) 2026 Lucas Amaral <[email protected]> +# +# SPDX-License-Identifier: GPL-2.0-or-later + +### Argument sets + +# Load/store immediate (unscaled, pre/post-index, unprivileged, unsigned offset) +# 'u' flag: 0 = 9-bit signed immediate (byte offset), 1 = 12-bit unsigned (needs << sz) +&ldst_imm rt rn imm sz sign w p unpriv ext u + +### Format templates + +# Load/store immediate (9-bit signed) +@ldst_imm .. ... . .. .. . imm:s9 .. rn:5 rt:5 &ldst_imm u=0 unpriv=0 p=0 w=0 +@ldst_imm_pre .. ... . .. .. . imm:s9 .. rn:5 rt:5 &ldst_imm u=0 unpriv=0 p=0 w=1 +@ldst_imm_post .. ... . .. .. . imm:s9 .. rn:5 rt:5 &ldst_imm u=0 unpriv=0 p=1 w=1 +@ldst_imm_user .. ... . .. .. . imm:s9 .. rn:5 rt:5 &ldst_imm u=0 unpriv=1 p=0 w=0 + +# Load/store unsigned offset (12-bit, handler scales by << sz) +@ldst_uimm .. ... . .. .. imm:12 rn:5 rt:5 &ldst_imm u=1 unpriv=0 p=0 w=0 + +### Load/store register — unscaled immediate (LDUR/STUR) + +# GPR +STR_i sz:2 111 0 00 00 0 ......... 00 ..... ..... @ldst_imm sign=0 ext=0 +LDR_i 00 111 0 00 01 0 ......... 00 ..... ..... @ldst_imm sign=0 ext=1 sz=0 +LDR_i 01 111 0 00 01 0 ......... 00 ..... ..... @ldst_imm sign=0 ext=1 sz=1 +LDR_i 10 111 0 00 01 0 ......... 00 ..... ..... @ldst_imm sign=0 ext=1 sz=2 +LDR_i 11 111 0 00 01 0 ......... 00 ..... ..... @ldst_imm sign=0 ext=0 sz=3 +LDR_i 00 111 0 00 10 0 ......... 00 ..... ..... @ldst_imm sign=1 ext=0 sz=0 +LDR_i 01 111 0 00 10 0 ......... 00 ..... ..... @ldst_imm sign=1 ext=0 sz=1 +LDR_i 10 111 0 00 10 0 ......... 00 ..... ..... @ldst_imm sign=1 ext=0 sz=2 +LDR_i 00 111 0 00 11 0 ......... 00 ..... ..... @ldst_imm sign=1 ext=1 sz=0 +LDR_i 01 111 0 00 11 0 ......... 00 ..... ..... @ldst_imm sign=1 ext=1 sz=1 + +# SIMD/FP +STR_v_i sz:2 111 1 00 00 0 ......... 00 ..... ..... @ldst_imm sign=0 ext=0 +STR_v_i 00 111 1 00 10 0 ......... 00 ..... ..... @ldst_imm sign=0 ext=0 sz=4 +LDR_v_i sz:2 111 1 00 01 0 ......... 00 ..... ..... @ldst_imm sign=0 ext=0 +LDR_v_i 00 111 1 00 11 0 ......... 00 ..... ..... @ldst_imm sign=0 ext=0 sz=4 + +### Load/store register — post-indexed + +# GPR +STR_i sz:2 111 0 00 00 0 ......... 01 ..... ..... @ldst_imm_post sign=0 ext=0 +LDR_i 00 111 0 00 01 0 ......... 01 ..... ..... @ldst_imm_post sign=0 ext=1 sz=0 +LDR_i 01 111 0 00 01 0 ......... 01 ..... ..... @ldst_imm_post sign=0 ext=1 sz=1 +LDR_i 10 111 0 00 01 0 ......... 01 ..... ..... @ldst_imm_post sign=0 ext=1 sz=2 +LDR_i 11 111 0 00 01 0 ......... 01 ..... ..... @ldst_imm_post sign=0 ext=0 sz=3 +LDR_i 00 111 0 00 10 0 ......... 01 ..... ..... @ldst_imm_post sign=1 ext=0 sz=0 +LDR_i 01 111 0 00 10 0 ......... 01 ..... ..... @ldst_imm_post sign=1 ext=0 sz=1 +LDR_i 10 111 0 00 10 0 ......... 01 ..... ..... @ldst_imm_post sign=1 ext=0 sz=2 +LDR_i 00 111 0 00 11 0 ......... 01 ..... ..... @ldst_imm_post sign=1 ext=1 sz=0 +LDR_i 01 111 0 00 11 0 ......... 01 ..... ..... @ldst_imm_post sign=1 ext=1 sz=1 + +# SIMD/FP +STR_v_i sz:2 111 1 00 00 0 ......... 01 ..... ..... @ldst_imm_post sign=0 ext=0 +STR_v_i 00 111 1 00 10 0 ......... 01 ..... ..... @ldst_imm_post sign=0 ext=0 sz=4 +LDR_v_i sz:2 111 1 00 01 0 ......... 01 ..... ..... @ldst_imm_post sign=0 ext=0 +LDR_v_i 00 111 1 00 11 0 ......... 01 ..... ..... @ldst_imm_post sign=0 ext=0 sz=4 + +### Load/store register — unprivileged + +# GPR only (no SIMD/FP unprivileged forms) +STR_i sz:2 111 0 00 00 0 ......... 10 ..... ..... @ldst_imm_user sign=0 ext=0 +LDR_i 00 111 0 00 01 0 ......... 10 ..... ..... @ldst_imm_user sign=0 ext=1 sz=0 +LDR_i 01 111 0 00 01 0 ......... 10 ..... ..... @ldst_imm_user sign=0 ext=1 sz=1 +LDR_i 10 111 0 00 01 0 ......... 10 ..... ..... @ldst_imm_user sign=0 ext=1 sz=2 +LDR_i 11 111 0 00 01 0 ......... 10 ..... ..... @ldst_imm_user sign=0 ext=0 sz=3 +LDR_i 00 111 0 00 10 0 ......... 10 ..... ..... @ldst_imm_user sign=1 ext=0 sz=0 +LDR_i 01 111 0 00 10 0 ......... 10 ..... ..... @ldst_imm_user sign=1 ext=0 sz=1 +LDR_i 10 111 0 00 10 0 ......... 10 ..... ..... @ldst_imm_user sign=1 ext=0 sz=2 +LDR_i 00 111 0 00 11 0 ......... 10 ..... ..... @ldst_imm_user sign=1 ext=1 sz=0 +LDR_i 01 111 0 00 11 0 ......... 10 ..... ..... @ldst_imm_user sign=1 ext=1 sz=1 + +### Load/store register — pre-indexed + +# GPR +STR_i sz:2 111 0 00 00 0 ......... 11 ..... ..... @ldst_imm_pre sign=0 ext=0 +LDR_i 00 111 0 00 01 0 ......... 11 ..... ..... @ldst_imm_pre sign=0 ext=1 sz=0 +LDR_i 01 111 0 00 01 0 ......... 11 ..... ..... @ldst_imm_pre sign=0 ext=1 sz=1 +LDR_i 10 111 0 00 01 0 ......... 11 ..... ..... @ldst_imm_pre sign=0 ext=1 sz=2 +LDR_i 11 111 0 00 01 0 ......... 11 ..... ..... @ldst_imm_pre sign=0 ext=0 sz=3 +LDR_i 00 111 0 00 10 0 ......... 11 ..... ..... @ldst_imm_pre sign=1 ext=0 sz=0 +LDR_i 01 111 0 00 10 0 ......... 11 ..... ..... @ldst_imm_pre sign=1 ext=0 sz=1 +LDR_i 10 111 0 00 10 0 ......... 11 ..... ..... @ldst_imm_pre sign=1 ext=0 sz=2 +LDR_i 00 111 0 00 11 0 ......... 11 ..... ..... @ldst_imm_pre sign=1 ext=1 sz=0 +LDR_i 01 111 0 00 11 0 ......... 11 ..... ..... @ldst_imm_pre sign=1 ext=1 sz=1 + +# SIMD/FP +STR_v_i sz:2 111 1 00 00 0 ......... 11 ..... ..... @ldst_imm_pre sign=0 ext=0 +STR_v_i 00 111 1 00 10 0 ......... 11 ..... ..... @ldst_imm_pre sign=0 ext=0 sz=4 +LDR_v_i sz:2 111 1 00 01 0 ......... 11 ..... ..... @ldst_imm_pre sign=0 ext=0 +LDR_v_i 00 111 1 00 11 0 ......... 11 ..... ..... @ldst_imm_pre sign=0 ext=0 sz=4 + +### PRFM — unscaled immediate: prefetch is a NOP + +NOP 11 111 0 00 10 0 --------- 00 ----- ----- + +### Load/store register — unsigned offset + +# GPR +STR_i sz:2 111 0 01 00 ............ ..... ..... @ldst_uimm sign=0 ext=0 +LDR_i 00 111 0 01 01 ............ ..... ..... @ldst_uimm sign=0 ext=1 sz=0 +LDR_i 01 111 0 01 01 ............ ..... ..... @ldst_uimm sign=0 ext=1 sz=1 +LDR_i 10 111 0 01 01 ............ ..... ..... @ldst_uimm sign=0 ext=1 sz=2 +LDR_i 11 111 0 01 01 ............ ..... ..... @ldst_uimm sign=0 ext=0 sz=3 +LDR_i 00 111 0 01 10 ............ ..... ..... @ldst_uimm sign=1 ext=0 sz=0 +LDR_i 01 111 0 01 10 ............ ..... ..... @ldst_uimm sign=1 ext=0 sz=1 +LDR_i 10 111 0 01 10 ............ ..... ..... @ldst_uimm sign=1 ext=0 sz=2 +LDR_i 00 111 0 01 11 ............ ..... ..... @ldst_uimm sign=1 ext=1 sz=0 +LDR_i 01 111 0 01 11 ............ ..... ..... @ldst_uimm sign=1 ext=1 sz=1 + +# PRFM — unsigned offset +NOP 11 111 0 01 10 ------------ ----- ----- + +# SIMD/FP +STR_v_i sz:2 111 1 01 00 ............ ..... ..... @ldst_uimm sign=0 ext=0 +STR_v_i 00 111 1 01 10 ............ ..... ..... @ldst_uimm sign=0 ext=0 sz=4 +LDR_v_i sz:2 111 1 01 01 ............ ..... ..... @ldst_uimm sign=0 ext=0 +LDR_v_i 00 111 1 01 11 ............ ..... ..... @ldst_uimm sign=0 ext=0 sz=4 + +### System instructions — DC cache maintenance + +# SYS with CRn=C7 covers all data cache operations (DC CIVAC, CVAC, etc.). +# On MMIO regions, cache maintenance is a harmless no-op. +NOP 1101 0101 0000 1 --- 0111 ---- --- -----
target/arm/emulate/arm_emulate.c new file +237 -0
237 lines not on this page · raw message
target/arm/emulate/arm_emulate.h new file +30 -0
@@ -0,0 +1,30 @@ +/* +* AArch64 instruction emulation library +* +* Copyright (c) 2026 Lucas Amaral <[email protected]> +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#ifndef ARM_EMULATE_H +#define ARM_EMULATE_H + +#include "qemu/osdep.h" + +/** +* ArmEmulResult - return status from arm_emul_insn() +*/ +typedef enum { +ARM_EMUL_OK ,/* Instruction emulated successfully */ +ARM_EMUL_UNHANDLED ,/* Instruction not recognized by decoder */ +ARM_EMUL_ERR_MEM ,/* Memory access failed */ +}ArmEmulResult ; + +/** +* arm_emul_insn - decode and emulate one AArch64 instruction +* +* Caller must synchronize CPU state and fetch @insn before calling. +*/ +ArmEmulResult arm_emul_insn (CPUArchState *env,uint32_t insn); + +#endif /* ARM_EMULATE_H */
target/arm/emulate/meson.build new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-or-later + +gen_a64_ldst = decodetree.process('a64-ldst.decode', + extra_args: ['--static-decode=decode_a64_ldst']) + +arm_common_system_ss.add(when: 'TARGET_AARCH64', if_true: [ + gen_a64_ldst, files('arm_emulate.c') +])
target/arm/meson.build +1 -0
@@ -57,6 +57,7 @@ arm_common_system_ss.add(files( 'vfp_fpscr.c', )) +subdir('emulate') subdir('hvf') subdir('whpx')
Signature · 1 line
2.52.0
Re: [PATCH v5 1/6] target/arm/emulate: add ISV=0 emulation library with load/store immediate
Quoted 4 lines · Lucas Amaral
target/arm/emulate/arm_emulate.c new file +4
-0+/* Memory access wrappers */ + +static int mem_read (DisasContext *ctx,uint64_t va,void *buf,int size) +{
target/arm/emulate/arm_emulate.c line 84
Quoted 10 lines
target/arm/emulate/arm_emulate.c new file +10
-0+if (((va & ~TARGET_PAGE_MASK ) + size) >TARGET_PAGE_SIZE ) { + ctx->result =ARM_EMUL_ERR_MEM ; +return -1 ; + } +int ret =cpu_memory_rw_debug (ctx->cpu, va, buf, size, false); +if (ret !=0 ) { + ctx->result =ARM_EMUL_ERR_MEM ; + } +return ret; +}
This is not implementing access for a debugger, but emulating an insn. Thus *_debug is the wrong interface to use.
There is no direct interface for you to use here, because the ones we have at present will raise an exception and jump back to an emulation loop that doesn't exist here. You'd need to translate the virtual address manually, recognize any exception raised, perform the access to the physical address, and recognize any hw exception raised.
target/arm/emulate/arm_emulate.c line 105
Quoted 7 lines
target/arm/emulate/arm_emulate.c new file +7
-0+/* Sign/zero extension helpers */ + +static uint64_t sign_extend (uint64_t val,int from_bits) +{ +int shift =64 - from_bits; +return (int64_t )(val << shift) >> shift; +}
This is sextract64.
target/arm/emulate/arm_emulate.c line 220
Quoted 7 lines
target/arm/emulate/arm_emulate.c new file +7
-0+/* PRFM, DC cache maintenance -- treated as NOP */ +static bool trans_NOP (DisasContext *ctx,arg_NOP *a) +{ + (void )ctx; + (void )a; +return true; +}
You don't need the (void) expressions.
r~
PATCH v5 2/6 target/arm/emulate: add load/store register offset
In reply to Lucas AmaralAdd emulation for load/store register offset addressing mode (DDI 0487 C3.3.9). The offset register value is extended via UXTB/UXTH/UXTW/UXTX/SXTB/SXTH/SXTW/SXTX and optionally shifted by the element size.
Instruction coverage:
- STR/LDR (GPR): register offset with extend, all sizes
- STR/LDR (SIMD/FP): register offset with extend, 8-128 bit
- PRFM register offset: NOP
--- target/arm/emulate/a64-ldst.decode | 29 ++++++++ target/arm/emulate/arm_emulate.c | 103 +++++++++++++++++++++++++++++ 2 files changed, 132 insertions(+)
target/arm/emulate/a64-ldst.decode +29 -0
@@ -10,6 +10,9 @@ # 'u' flag: 0 = 9-bit signed immediate (byte offset), 1 = 12-bit unsigned (needs << sz) &ldst_imm rt rn imm sz sign w p unpriv ext u +# Load/store register offset +&ldst rm rn rt sign ext sz opt s + ### Format templates # Load/store immediate (9-bit signed)
@@ -21,6 +24,9 @@ # Load/store unsigned offset (12-bit, handler scales by << sz) @ldst_uimm .. ... . .. .. imm:12 rn:5 rt:5 &ldst_imm u=1 unpriv=0 p=0 w=0 +# Load/store register offset +@ldst .. ... . .. .. . rm:5 opt:3 s:1 .. rn:5 rt:5 &ldst + ### Load/store register — unscaled immediate (LDUR/STUR) # GPR
@@ -122,6 +128,29 @@ STR_v_i 00 111 1 01 10 ............ ..... ..... @ldst_uimm sign= LDR_v_i sz:2 111 1 01 01 ............ ..... ..... @ldst_uimm sign=0 ext=0 LDR_v_i 00 111 1 01 11 ............ ..... ..... @ldst_uimm sign=0 ext=0 sz=4 +### Load/store register — register offset + +# GPR +STR sz:2 111 0 00 00 1 ..... ... . 10 ..... ..... @ldst sign=0 ext=0 +LDR 00 111 0 00 01 1 ..... ... . 10 ..... ..... @ldst sign=0 ext=1 sz=0 +LDR 01 111 0 00 01 1 ..... ... . 10 ..... ..... @ldst sign=0 ext=1 sz=1 +LDR 10 111 0 00 01 1 ..... ... . 10 ..... ..... @ldst sign=0 ext=1 sz=2 +LDR 11 111 0 00 01 1 ..... ... . 10 ..... ..... @ldst sign=0 ext=0 sz=3 +LDR 00 111 0 00 10 1 ..... ... . 10 ..... ..... @ldst sign=1 ext=0 sz=0 +LDR 01 111 0 00 10 1 ..... ... . 10 ..... ..... @ldst sign=1 ext=0 sz=1 +LDR 10 111 0 00 10 1 ..... ... . 10 ..... ..... @ldst sign=1 ext=0 sz=2 +LDR 00 111 0 00 11 1 ..... ... . 10 ..... ..... @ldst sign=1 ext=1 sz=0 +LDR 01 111 0 00 11 1 ..... ... . 10 ..... ..... @ldst sign=1 ext=1 sz=1 + +# PRFM — register offset +NOP 11 111 0 00 10 1 ----- -1- - 10 ----- ----- + +# SIMD/FP +STR_v sz:2 111 1 00 00 1 ..... ... . 10 ..... ..... @ldst sign=0 ext=0 +STR_v 00 111 1 00 10 1 ..... ... . 10 ..... ..... @ldst sign=0 ext=0 sz=4 +LDR_v sz:2 111 1 00 01 1 ..... ... . 10 ..... ..... @ldst sign=0 ext=0 +LDR_v 00 111 1 00 11 1 ..... ... . 10 ..... ..... @ldst sign=0 ext=0 sz=4 + ### System instructions — DC cache maintenance # SYS with CRn=C7 covers all data cache operations (DC CIVAC, CVAC, etc.).
target/arm/emulate/arm_emulate.c +103 -0
109 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v5 3/6 target/arm/emulate: add load/store pair
In reply to Lucas AmaralAdd emulation for load/store pair instructions (DDI 0487 C3.3.14 -- C3.3.16). All addressing modes are covered: non-temporal (STNP/LDNP), post-indexed, signed offset, and pre-indexed.
Instruction coverage:
- STP/LDP (GPR): 32/64-bit pairs, all addressing modes
- STP/LDP (SIMD/FP): 32/64/128-bit pairs, all addressing modes
- LDPSW: sign-extending 32-bit pair load
- STGP: store allocation tag pair (tag operation is NOP for MMIO)
--- target/arm/emulate/a64-ldst.decode | 68 ++++++++++++++++++ target/arm/emulate/arm_emulate.c | 111 +++++++++++++++++++++++++++++ 2 files changed, 179 insertions(+)
target/arm/emulate/a64-ldst.decode +68 -0
@@ -10,6 +10,9 @@ # 'u' flag: 0 = 9-bit signed immediate (byte offset), 1 = 12-bit unsigned (needs << sz) &ldst_imm rt rn imm sz sign w p unpriv ext u +# Load/store pair (GPR and SIMD/FP) +&ldstpair rt2 rt rn imm sz sign w p + # Load/store register offset &ldst rm rn rt sign ext sz opt s
@@ -24,6 +27,9 @@ # Load/store unsigned offset (12-bit, handler scales by << sz) @ldst_uimm .. ... . .. .. imm:12 rn:5 rt:5 &ldst_imm u=1 unpriv=0 p=0 w=0 +# Load/store pair: imm7 is signed, scaled by element size in handler +@ldstpair .. ... . ... . imm:s7 rt2:5 rn:5 rt:5 &ldstpair + # Load/store register offset @ldst .. ... . .. .. . rm:5 opt:3 s:1 .. rn:5 rt:5 &ldst
@@ -128,6 +134,68 @@ STR_v_i 00 111 1 01 10 ............ ..... ..... @ldst_uimm sign= LDR_v_i sz:2 111 1 01 01 ............ ..... ..... @ldst_uimm sign=0 ext=0 LDR_v_i 00 111 1 01 11 ............ ..... ..... @ldst_uimm sign=0 ext=0 sz=4 +### Load/store pair — non-temporal (STNP/LDNP) + +# STNP/LDNP: offset only, no writeback. Non-temporal hint ignored. +STP 00 101 0 000 0 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=0 w=0 +LDP 00 101 0 000 1 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=0 w=0 +STP 10 101 0 000 0 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=0 +LDP 10 101 0 000 1 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=0 +STP_v 00 101 1 000 0 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=0 w=0 +LDP_v 00 101 1 000 1 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=0 w=0 +STP_v 01 101 1 000 0 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=0 +LDP_v 01 101 1 000 1 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=0 +STP_v 10 101 1 000 0 ....... ..... ..... ..... @ldstpair sz=4 sign=0 p=0 w=0 +LDP_v 10 101 1 000 1 ....... ..... ..... ..... @ldstpair sz=4 sign=0 p=0 w=0 + +### Load/store pair — post-indexed + +STP 00 101 0 001 0 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=1 w=1 +LDP 00 101 0 001 1 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=1 w=1 +LDP 01 101 0 001 1 ....... ..... ..... ..... @ldstpair sz=2 sign=1 p=1 w=1 +STP 10 101 0 001 0 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=1 w=1 +LDP 10 101 0 001 1 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=1 w=1 +STP_v 00 101 1 001 0 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=1 w=1 +LDP_v 00 101 1 001 1 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=1 w=1 +STP_v 01 101 1 001 0 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=1 w=1 +LDP_v 01 101 1 001 1 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=1 w=1 +STP_v 10 101 1 001 0 ....... ..... ..... ..... @ldstpair sz=4 sign=0 p=1 w=1 +LDP_v 10 101 1 001 1 ....... ..... ..... ..... @ldstpair sz=4 sign=0 p=1 w=1 + +### Load/store pair — signed offset + +STP 00 101 0 010 0 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=0 w=0 +LDP 00 101 0 010 1 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=0 w=0 +LDP 01 101 0 010 1 ....... ..... ..... ..... @ldstpair sz=2 sign=1 p=0 w=0 +STP 10 101 0 010 0 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=0 +LDP 10 101 0 010 1 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=0 +STP_v 00 101 1 010 0 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=0 w=0 +LDP_v 00 101 1 010 1 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=0 w=0 +STP_v 01 101 1 010 0 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=0 +LDP_v 01 101 1 010 1 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=0 +STP_v 10 101 1 010 0 ....... ..... ..... ..... @ldstpair sz=4 sign=0 p=0 w=0 +LDP_v 10 101 1 010 1 ....... ..... ..... ..... @ldstpair sz=4 sign=0 p=0 w=0 + +### Load/store pair — pre-indexed + +STP 00 101 0 011 0 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=0 w=1 +LDP 00 101 0 011 1 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=0 w=1 +LDP 01 101 0 011 1 ....... ..... ..... ..... @ldstpair sz=2 sign=1 p=0 w=1 +STP 10 101 0 011 0 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=1 +LDP 10 101 0 011 1 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=1 +STP_v 00 101 1 011 0 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=0 w=1 +LDP_v 00 101 1 011 1 ....... ..... ..... ..... @ldstpair sz=2 sign=0 p=0 w=1 +STP_v 01 101 1 011 0 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=1 +LDP_v 01 101 1 011 1 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=1 +STP_v 10 101 1 011 0 ....... ..... ..... ..... @ldstpair sz=4 sign=0 p=0 w=1 +LDP_v 10 101 1 011 1 ....... ..... ..... ..... @ldstpair sz=4 sign=0 p=0 w=1 + +### Load/store pair — STGP (store allocation tag + pair) + +STGP 01 101 0 001 0 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=1 w=1 +STGP 01 101 0 010 0 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=0 +STGP 01 101 0 011 0 ....... ..... ..... ..... @ldstpair sz=3 sign=0 p=0 w=1 + ### Load/store register — register offset # GPR
target/arm/emulate/arm_emulate.c +111 -0
117 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v5 3/6] target/arm/emulate: add load/store pair
Quoted 47 lines · Lucas Amaral
target/arm/emulate/arm_emulate.c +47
-0+/* +* Load/store pair: STP, LDP, STNP, LDNP, STGP, LDPSW +* (DDI 0487 C3.3.14 -- C3.3.16) +*/ + +static bool trans_STP (DisasContext *ctx,arg_ldstpair *a) +{ +int esize =1 << a->sz;/* 4 or 8 bytes */ +int64_t offset = (int64_t )a->imm << a->sz; +uint64_t base =base_read (ctx, a->rn); +uint64_t va = a->p ? base : base + offset;/* post-index: unmodified base */ +uint8_t buf[16 ];/* max 2 x 8 bytes */ + +uint64_t v1 =gpr_read (ctx, a->rt); +uint64_t v2 =gpr_read (ctx, a->rt2); +memcpy (buf, &v1, esize); +memcpy (buf + esize, &v2, esize); + +if (mem_write (ctx, va, buf,2 * esize) !=0 ) { +return true; + } + +if (a->w) { +base_write (ctx, a->rn, base + offset); + } +return true; +} + +static bool trans_LDP (DisasContext *ctx,arg_ldstpair *a) +{ +int esize =1 << a->sz; +int64_t offset = (int64_t )a->imm << a->sz; +uint64_t base =base_read (ctx, a->rn); +uint64_t va = a->p ? base : base + offset; +uint8_t buf[16 ]; +uint64_t v1 =0 , v2 =0 ; + +if (mem_read (ctx, va, buf,2 * esize) !=0 ) { +return true; + } +memcpy (&v1, buf, esize); +memcpy (&v2, buf + esize, esize); + +/* LDPSW: sign-extend 32-bit values to 64-bit (sign=1, sz=2) */ +if (a->sign) { + v1 =sign_extend (v1,8 * esize); + v2 =sign_extend (v2,8 * esize);
target/arm/emulate/arm_emulate.c static uint64_t load_extend(uint64_t val, int sz, int sign, int ext) · line 181
Quoted 10 lines
target/arm/emulate/arm_emulate.c +10
-0+ } + +gpr_write (ctx, a->rt, v1); +gpr_write (ctx, a->rt2, v2); + +if (a->w) { +base_write (ctx, a->rn, base + offset); + } +return true; +}
The copy into v1 and v2 is dodgy. While I expect that we'll never support big-endian aarch64 as a host, this can be written in a non-dodgy way.
The assignment into rt and rt2 must take the *guest* endianness into consideration, and that is something that we support.
r~
PATCH v5 4/6 target/arm/emulate: add load/store exclusive
In reply to Lucas AmaralAdd emulation for load/store exclusive instructions (DDI 0487 C3.3.6). Exclusive monitors have no meaning on emulated MMIO accesses, so STXR always reports success (Rs=0) and LDXR does not set a monitor.
Instruction coverage:
- STXR/STLXR: exclusive store, 8/16/32/64-bit
- LDXR/LDAXR: exclusive load, 8/16/32/64-bit
- STXP/STLXP: exclusive store pair, 32/64-bit
- LDXP/LDAXP: exclusive load pair, 32/64-bit
STXP/LDXP use two explicit decode patterns (sz=2, sz=3) for the 32/64-bit size variants.
--- target/arm/emulate/a64-ldst.decode | 22 +++++++++ target/arm/emulate/arm_emulate.c | 74 ++++++++++++++++++++++++++++++ 2 files changed, 96 insertions(+)
target/arm/emulate/a64-ldst.decode +22 -0
@@ -10,6 +10,9 @@ # 'u' flag: 0 = 9-bit signed immediate (byte offset), 1 = 12-bit unsigned (needs << sz) &ldst_imm rt rn imm sz sign w p unpriv ext u +# Load/store exclusive +&stxr rn rt rt2 rs sz lasr + # Load/store pair (GPR and SIMD/FP) &ldstpair rt2 rt rn imm sz sign w p
@@ -18,6 +21,9 @@ ### Format templates +# Exclusives +@stxr sz:2 ...... ... rs:5 lasr:1 rt2:5 rn:5 rt:5 &stxr + # Load/store immediate (9-bit signed) @ldst_imm .. ... . .. .. . imm:s9 .. rn:5 rt:5 &ldst_imm u=0 unpriv=0 p=0 w=0 @ldst_imm_pre .. ... . .. .. . imm:s9 .. rn:5 rt:5 &ldst_imm u=0 unpriv=0 p=0 w=1
@@ -134,6 +140,22 @@ STR_v_i 00 111 1 01 10 ............ ..... ..... @ldst_uimm sign= LDR_v_i sz:2 111 1 01 01 ............ ..... ..... @ldst_uimm sign=0 ext=0 LDR_v_i 00 111 1 01 11 ............ ..... ..... @ldst_uimm sign=0 ext=0 sz=4 +### Load/store exclusive + +# STXR / STLXR (sz encodes 8/16/32/64-bit) +STXR .. 001000 000 ..... . ..... ..... ..... @stxr + +# LDXR / LDAXR +LDXR .. 001000 010 ..... . ..... ..... ..... @stxr + +# STXP / STLXP (bit[31]=1, bit[30]=sf → sz=2 for 32-bit, sz=3 for 64-bit) +STXP 10 001000 001 rs:5 lasr:1 rt2:5 rn:5 rt:5 &stxr sz=2 +STXP 11 001000 001 rs:5 lasr:1 rt2:5 rn:5 rt:5 &stxr sz=3 + +# LDXP / LDAXP +LDXP 10 001000 011 rs:5 lasr:1 rt2:5 rn:5 rt:5 &stxr sz=2 +LDXP 11 001000 011 rs:5 lasr:1 rt2:5 rn:5 rt:5 &stxr sz=3 + ### Load/store pair — non-temporal (STNP/LDNP) # STNP/LDNP: offset only, no writeback. Non-temporal hint ignored.
target/arm/emulate/arm_emulate.c +74 -0
80 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v5 5/6 target/arm/emulate: add atomic, compare-and-swap, and PAC load
In reply to Lucas AmaralAdd emulation for remaining ISV=0 load/store instruction classes.
Atomic memory operations (DDI 0487 C3.3.2):
- LDADD, LDCLR, LDEOR, LDSET: arithmetic/logic atomics
- LDSMAX, LDSMIN, LDUMAX, LDUMIN: signed/unsigned min/max
- SWP: atomic swap
Non-atomic read-modify-write, sufficient for MMIO where concurrent access is not a concern. Acquire/release semantics are ignored.
Compare-and-swap (DDI 0487 C3.3.1):
- CAS/CASA/CASAL/CASL: single-register compare-and-swap
- CASP/CASPA/CASPAL/CASPL: register-pair compare-and-swap CASP validates even register pairs; odd or r31 returns UNHANDLED.
Load with PAC (DDI 0487 C6.2.121):
- LDRAA/LDRAB: pointer-authenticated load, offset/pre-indexed Pointer authentication is not emulated (equivalent to auth always succeeding), which is correct for MMIO since PAC is a software security mechanism, not a memory access semantic.
Decodetree differences from TCG:
- %ldra_imm extracts the raw S:imm9 field; the handler scales by << 3. TCG applies !function=times_8 in the formatter.
- @ldra uses wildcards for fixed opcode bits that TCG locks down (bits 31:30, bit 20, bit 11); the fixed bits are matched by the instruction pattern instead.
- @cas is an explicit format template; TCG uses inline field extraction.
CASP uses two explicit decode patterns for the 32/64-bit size variants. LDRA's offset immediate is stored raw in the decode; the handler scales by << 3.
--- target/arm/emulate/a64-ldst.decode | 45 ++++++ target/arm/emulate/arm_emulate.c | 233 +++++++++++++++++++++++++++++ 2 files changed, 278 insertions(+)
target/arm/emulate/a64-ldst.decode +45 -0
@@ -16,6 +16,16 @@ # Load/store pair (GPR and SIMD/FP) &ldstpair rt2 rt rn imm sz sign w p +# Atomic memory operations +&atomic rs rn rt a r sz + +# Compare-and-swap +&cas rs rn rt sz a r + +# Load with PAC (LDRAA/LDRAB, FEAT_PAuth) +%ldra_imm 22:s1 12:9 +&ldra rt rn imm m w + # Load/store register offset &ldst rm rn rt sign ext sz opt s
@@ -36,6 +46,15 @@ # Load/store pair: imm7 is signed, scaled by element size in handler @ldstpair .. ... . ... . imm:s7 rt2:5 rn:5 rt:5 &ldstpair +# Atomics +@atomic sz:2 ... . .. a:1 r:1 . rs:5 . ... .. rn:5 rt:5 &atomic + +# Compare-and-swap: sz extracted by pattern (CAS) or set constant (CASP) +@cas .. ...... . a:1 . rs:5 r:1 ..... rn:5 rt:5 &cas + +# Load with PAC +@ldra .. ... . .. m:1 . . ......... w:1 . rn:5 rt:5 &ldra imm=%ldra_imm + # Load/store register offset @ldst .. ... . .. .. . rm:5 opt:3 s:1 .. rn:5 rt:5 &ldst
@@ -241,6 +260,32 @@ STR_v 00 111 1 00 10 1 ..... ... . 10 ..... ..... @ldst sign=0 ext= LDR_v sz:2 111 1 00 01 1 ..... ... . 10 ..... ..... @ldst sign=0 ext=0 LDR_v 00 111 1 00 11 1 ..... ... . 10 ..... ..... @ldst sign=0 ext=0 sz=4 +### Compare-and-swap + +# CAS / CASA / CASAL / CASL +CAS sz:2 001000 1 . 1 ..... . 11111 ..... ..... @cas + +# CASP / CASPA / CASPAL / CASPL (pair: Rt,Rt+1 and Rs,Rs+1) +CASP 00 001000 0 . 1 ..... . 11111 ..... ..... @cas sz=2 +CASP 01 001000 0 . 1 ..... . 11111 ..... ..... @cas sz=3 + +### Atomic memory operations + +LDADD .. 111 0 00 . . 1 ..... 0000 00 ..... ..... @atomic +LDCLR .. 111 0 00 . . 1 ..... 0001 00 ..... ..... @atomic +LDEOR .. 111 0 00 . . 1 ..... 0010 00 ..... ..... @atomic +LDSET .. 111 0 00 . . 1 ..... 0011 00 ..... ..... @atomic +LDSMAX .. 111 0 00 . . 1 ..... 0100 00 ..... ..... @atomic +LDSMIN .. 111 0 00 . . 1 ..... 0101 00 ..... ..... @atomic +LDUMAX .. 111 0 00 . . 1 ..... 0110 00 ..... ..... @atomic +LDUMIN .. 111 0 00 . . 1 ..... 0111 00 ..... ..... @atomic +SWP .. 111 0 00 . . 1 ..... 1000 00 ..... ..... @atomic + +### Load with PAC (FEAT_PAuth) + +# LDRAA (M=0) / LDRAB (M=1), offset (W=0) / pre-indexed (W=1) +LDRA 11 111 0 00 . . 1 ......... . 1 ..... ..... @ldra + ### System instructions — DC cache maintenance # SYS with CRn=C7 covers all data cache operations (DC CIVAC, CVAC, etc.).
target/arm/emulate/arm_emulate.c +233 -0
239 lines not on this page · raw message
Signature · 1 line
2.52.0
PATCH v5 6/6 target/arm/hvf, whpx: wire ISV=0 emulation for data aborts
In reply to Lucas AmaralWhen a data abort with ISV=0 occurs during MMIO emulation, the syndrome register does not carry the access size or target register. Previously this hit an assert(isv) and killed the VM.
Replace the assert with instruction fetch + decode + emulate using the shared library in target/arm/emulate/. The faulting instruction is read from guest memory via cpu_memory_rw_debug(), decoded by the decodetree- generated decoder, and emulated against the vCPU register file.
Both HVF (macOS) and WHPX (Windows Hyper-V) use the same pattern:
1. cpu_synchronize_state() to flush hypervisor registers
2. Fetch 4-byte instruction at env->pc
3. arm_emul_insn(env, insn)
4. On success, advance PC past the emulated instruction
If the instruction is unhandled or a memory error occurs, a synchronous external abort is injected into the guest via syn_data_abort_no_iss() with fnv=1 and fsc=0x10, matching the syndrome that KVM uses in kvm_inject_arm_sea(). The guest kernel's fault handler then reports the error through its normal data abort path.
WHPX adds a whpx_inject_data_abort() helper and adjusts the whpx_handle_mmio() return convention so the caller skips PC advancement when an exception has been injected.
--- target/arm/hvf/hvf.c | 46 ++++++++++++++++++++++++++-- target/arm/whpx/whpx-all.c | 61 +++++++++++++++++++++++++++++++++++++- 2 files changed, 103 insertions(+), 4 deletions(-)
target/arm/hvf/hvf.c +43 -3
@@ -32,6 +32,7 @@#include "arm-powerctl.h" #include "target/arm/cpu.h" #include "target/arm/internals.h" +#include "emulate/arm_emulate.h" #include "target/arm/multiprocessing.h" #include "target/arm/gtimer.h" #include "target/arm/trace.h"
@@ -2175,10 +2176,49 @@ static int hvf_handle_exception(CPUState *cpu, hv_vcpu_exit_exception_t *excp)assert (!s1ptw);/* -* TODO: ISV will be 0 for SIMD or SVE accesses. -* Inject the exception into the guest. +* ISV=0: syndrome doesn't carry access size/register info. +* Fetch and emulate via target/arm/emulate/. */ -assert (isv); +if (!isv) { +ARMCPU *arm_cpu =ARM_CPU (cpu); +CPUARMState *env = &arm_cpu->env; +uint32_t insn; +ArmEmulResult r; + +cpu_synchronize_state (cpu); + +if (cpu_memory_rw_debug (cpu, env->pc, + (uint8_t *)&insn,4 , false) !=0 ) { +bool same_el =arm_current_el (env) ==1 ; +uint32_t esr =syn_data_abort_no_iss (same_el, +1 ,0 ,0 ,0 , iswrite,0x10 ); + +error_report ("HVF: cannot read insn at pc=0x%" PRIx64, + (uint64_t )env->pc); + env->exception.vaddress = excp->virtual_address; +hvf_raise_exception (cpu,EXCP_DATA_ABORT , esr,1 ); +break ; + } + + r =arm_emul_insn (env, insn); +if (r ==ARM_EMUL_UNHANDLED || r ==ARM_EMUL_ERR_MEM ) { +bool same_el =arm_current_el (env) ==1 ; +uint32_t esr =syn_data_abort_no_iss (same_el, +1 ,0 ,0 ,0 , iswrite,0x10 ); + +error_report ("HVF: ISV=0 %s insn 0x%08x at " +"pc=0x%" PRIx64", injecting data abort" , + r ==ARM_EMUL_UNHANDLED ?"unhandled" + :"memory error" , + insn, (uint64_t )env->pc); + env->exception.vaddress = excp->virtual_address; +hvf_raise_exception (cpu,EXCP_DATA_ABORT , esr,1 ); +break ; + } + + advance_pc = true; +break ; + }/* * Emulate MMIO.
target/arm/whpx/whpx-all.c +60 -1
85 lines not on this page · raw message
Signature · 1 line
2.52.0