system/memory: Make ram device region directly accessible
v4 cover system/memory: Make ram device region directly accessible
All ram device regions was turned to be indirectly accessible by commit 4a2e242bbb ("memory: Don't use memcpy for ram_device regions"). This leads to a frozen guest where a NVidia GH100 GPU is passed from host. The memory in its PCI BAR#4 can be allocated as DMA target buffer. qemu has to take DMA bounce buffer in address_space_map() to cover the DMA request. However, the bounce buffer size is 4096 bytes only and it's exhaused very quickly when the guest has significant disk activities on compiling 'cuda-samples'. The full log and problem description can be found from PATCH[2/3]'s commit log.
Fix the issue handled in commit 4a2e242bbb by replacing memmove() with newly added qemu_ram_move() where the aligned and small-sized accesses are handled by qatomics, and fall back to memmove() otherwise, for the directly accessible regions. With this, we can revert 4a2e242bbb to make ram device region directly accessible again and bypass the bounce buffer in address_space_map() where the guest hang happens.
PATCH[1] replaces memcpy() with memomve() for directly accessible regions PATCH[2] uses qemu_ram_move() for directly accessible regions PATCH[3] makes ram device region directly accessible again
Changelog
=========
v3 -> v4:
* https://lore.kernel.org/qemu-arm/[email protected]/
* New PATCH[1/3] to replace memcpy() with memmove() (Peterx)
* Unified qemu_ram_move() for all architectures (Peterx/Michael)
v2 -> v3:
* https://lore.kernel.org/qemu-arm/[email protected]/
* Documentation for qemu_ram_{copy, move} (Peter/Michael)
* Support qemu_ram_move() for overlapped src/dest (Richard)
* Use {memcpy, memmove} if step is 16-bytes or more (Michael)
* Code improvements (Richard/Michael)
v1 -> v2:
* https://lore.kernel.org/qemu-arm/[email protected]/
* Rename address_space_{memcpy, memmove}() to qemu_ram_{copy, move}()
and move them to physmem.c and memory.h (Philippe)
* Use memcpy() and memmove() in qemu_ram_{copy, move}() for the variable
length case (Miachel)
* Handle unaligned access in qemu_ram_{copy, move}() for all archs
except i386 and x86_64 (Richard/Michael)
RFCv1 -> v1:
* https://lists.nongnu.org/archive/html/qemu-arm/2026-06/msg00307.html
* Reworked solution based on suggestions from Peter Xu, Peter Maydell and Michael S. Tsirkin
Gavin Shan (3):
system/memory: Use memmove() for directly accessible regions system/memory: Use qemu_ram_move() for directly accessible regions system/memory: Make ram device region directly accessible
hw/remote/vfio-user-obj.c | 4 ++-- include/system/memory.h | 28 ++++++++++++++++---------- system/memory.c | 41 +-------------------------------------- system/physmem.c | 39 +++++++++++++++++++++++++++++++++++-- system/trace-events | 2 -- 5 files changed, 58 insertions(+), 56 deletions(-)
Signature · 1 line
2.55.0
PATCH v4 1/3 system/memory: Use memmove() for directly accessible regions
Similar to what's done in commit 4a73aee88140 ("softmmu: Use memmove in flatview_write_continue"), there are more sites where the overlapping source and destination buffer are allowed for the directly accessible regions. Use memmove() in those sites, listed as below.
hw/remote/vfio-user-obj.c::vfu_object_mr_rw
include/system/memory.h::address_space_read
system/physmem.c::flatview_read_continue_step
---
v4: New patch
--- hw/remote/vfio-user-obj.c | 4 ++-- include/system/memory.h | 2 +- system/physmem.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)
hw/remote/vfio-user-obj.c +2 -2
@@ -375,9 +375,9 @@ static int vfu_object_mr_rw(MemoryRegion *mr, uint8_t *buf, hwaddr offset, ram_ptr =memory_region_get_ram_ptr (mr);if (is_write) { -memcpy ((ram_ptr + offset), buf, size); +memmove ((ram_ptr + offset), buf, size); }else { -memcpy (buf, (ram_ptr + offset), size); +memmove (buf, (ram_ptr + offset), size); }return 0 ;
include/system/memory.h +1 -1
@@ -2739,7 +2739,7 @@ MemTxResult address_space_read(const AddressSpace *as, hwaddr addr, mr =flatview_translate (fv, addr, &addr1, &l, false, attrs);if (len == l &&memory_access_is_direct (mr, false, attrs)) { ptr =qemu_map_ram_ptr (mr->ram_block, addr1); -memcpy (buf, ptr, len); +memmove (buf, ptr, len); }else { result =flatview_read_continue (fv, addr, attrs, buf, len, addr1, l, mr);
system/physmem.c +1 -1
@@ -3363,7 +3363,7 @@ static MemTxResult flatview_read_continue_step(MemTxAttrs attrs, uint8_t *buf,uint8_t *ram_ptr =qemu_ram_ptr_length (mr->ram_block, mr_addr, l, false, false); -memcpy (buf, ram_ptr, *l); +memmove (buf, ram_ptr, *l);return MEMTX_OK ; }
Signature · 1 line
2.55.0
Re: [PATCH v4 1/3] system/memory: Use memmove() for directly accessible regions
Quoted 11 lines · Gavin Shan
Similar to what's done in commit
4a73aee88140("softmmu: Use memmove inflatview_write_continue"), there are more sites where the overlapping source and destination buffer are allowed for the directly accessible regions. Usememmove()in those sites, listed as below.hw/remote/vfio-user-obj.c::vfu_object_mr_rw
include/system/memory.h::address_space_read
system/physmem.c::flatview_read_continue_stepSigned-off-by Gavin Shan <[email protected]>
Even if the overlap isn't possible, it's definitely better to use memmove() than to have to think hard about whether there's some situation where it might happen :-)
thanks
-- PMM
Re: [PATCH v4 1/3] system/memory: Use memmove() for directly accessible regions
In reply to Gavin ShanQuoted 10 lines · Gavin Shan
Similar to what's done in commit
4a73aee88140("softmmu: Use memmove inflatview_write_continue"), there are more sites where the overlapping source and destination buffer are allowed for the directly accessible regions. Usememmove()in those sites, listed as below.hw/remote/vfio-user-obj.c::vfu_object_mr_rw
include/system/memory.h::address_space_read
system/physmem.c::flatview_read_continue_stepSigned-off-by Gavin Shan <[email protected]>
Signature · 1 line
Peter Xu
PATCH v4 2/3 system/memory: Use qemu_ram_move() for directly accessible regions
In reply to Gavin ShanAll ram device regions were turned to be indirectly accessible by commit 4a2e242bbb ("memory: Don't use memcpy for ram_device regions"). This leads to guest hang on attempt to build 'cuda-samples' as reported by Julia. The guest is started by the following command lines, with GH100 GPU card passed from the host.
host$ lspci | grep GH100
0009:01:00.0 3D controller: NVIDIA Corporation GH100 [GH200 120GB / 480GB] (rev a1)
host$ /home/sandbox/gavin/qemu.main/build/qemu-system-aarch64 \
-machine virt,gic-version=host,ras=on,highmem-mmio-size=4T \
-accel kvm -cpu host -smp cpus=48 -m size=8G \
-drive file=/home/gavin/sandbox/images/disk.qcow2,if=none,id=d0 \
-device virtio-blk-pci,id=vb0,bus=pcie.0,drive=d0,num-queues=4 \
-device vfio-pci-nohotplug,host=0009:01:00.0,bus=pcie.1.0
:
guest$ cd cuda-samples/build
guest$ make -j 20 clean
guest$ make -j 20
:
[ 54%] Linking CUDA executable graphMemoryNodes
[ 54%] Built target graphMemoryNodes
<no more output afterwards, guest becomes frozen here>guest$ qemu-system-aarch64: virtio: bogus descriptor or out of resources
[ 555.814025] virtio_blk virtio0: [vda] new size: 268435456 512-byte logical blocks (137 GB/128 GiB)
When the GPU's driver (NVidia open driver) is loaded on guest bootup, the memory blocks residing in the PCI BAR#4 of the GH100 GPU card can be presented to the guest through memory hot-add. The page cache can then be allocated from the hot added memory blocks when cuda-samples is being built. Afterwards, the page cache is sent to QEMU's virtio-blk device as part of the DMA request, the bounce buffer has to be used to accomodate the request as the corresponding memory region (MemoryRegion) is an indirectly accessible ram device region in qemu. However, the max bounce bufer size is only 4096 bytes by default and that is exhausted quickly, leading to a reset on the virtio-blk device and frozen guest eventually.
QEMU
====
virtio_blk_handle_output
virtio_blk_handle_vq
virtio_blk_get_request
virtqueue_pop
virtqueue_split_pop
virtqueue_map_desc
address_space_map
memory_access_is_direct # Return false
memory_region_supports_direct_access
(qemu) info mtree
memory-region: pci_bridge_pci
0000000000000000-ffffffffffffffff (prio 0, container): pci_bridge_pci
0000042000000000-0000043fffffffff (prio 1, i/o): 0009:01:00.0 base BAR 4
0000042000000000-0000043fffffffff (prio 0, i/o): 0009:01:00.0 BAR 4
0000042000000000-000004379fffffff (prio 0, ramd): 0009:01:00.0 BAR 4 mmaps[0]This adds qemu_ram_move() where the aligned and small-sized accesses are handled by qatomics, and fall back to memmove() otherwise. The memove() for the directly accessible regions is replaced by qemu_ram_move() so that the issue covered by commit 4a2e242bbb (MMIO access instructions were optimized to SSE instructions) is fixed. This makes 'ram_device_mem_ops' redundant, paving the way to revert that commit to make the ram device region directly accessible again in the next patch.
---
v4: Unified qemu_ram_move() for all architectures where the qatomics are
used for aligned and small-sized accesses, and fall back to memmove() otherwise. system/physmem.c::address_space_write_rom is removed from the listed sites.
--- hw/remote/vfio-user-obj.c | 4 ++-- include/system/memory.h | 17 ++++++++++++++++- system/physmem.c | 39 +++++++++++++++++++++++++++++++++++++-- 3 files changed, 55 insertions(+), 5 deletions(-)
hw/remote/vfio-user-obj.c +2 -2
@@ -375,9 +375,9 @@ static int vfu_object_mr_rw(MemoryRegion *mr, uint8_t *buf, hwaddr offset, ram_ptr =memory_region_get_ram_ptr (mr);if (is_write) { -memmove ((ram_ptr + offset), buf, size); +qemu_ram_move ((ram_ptr + offset), buf, size); }else { -memmove (buf, (ram_ptr + offset), size); +qemu_ram_move (buf, (ram_ptr + offset), size); }return 0 ;
include/system/memory.h +16 -1
@@ -2666,6 +2666,21 @@ void address_space_register_map_client(AddressSpace *as, QEMUBH *bh);void address_space_unregister_map_client (AddressSpace *as,QEMUBH *bh);/* Internal functions, part of the implementation of address_space_read. */ + +/** +* qemu_ram_move: move data from or to ramblock +* +* @dst: destination where the data is moved to +* @src: source where the data is moved from +* @n: length of data to be moved +* +* Move @n bytes from @src to @dst with the assumption that @src and @dst +* can overlap. The access is atomic if the source and destination buffer +* aren't overlapped for a well aligned and small-sized access. Otherwise, +* fall back to the standard memmove(). +*/ +void qemu_ram_move (void *dst,const void *src,size_t n); +MemTxResult address_space_read_full (const AddressSpace *as,hwaddr addr,MemTxAttrs attrs,void *buf,hwaddr len); MemTxResult flatview_read_continue(FlatView *fv, hwaddr addr,
@@ -2739,7 +2754,7 @@ MemTxResult address_space_read(const AddressSpace *as, hwaddr addr, mr =flatview_translate (fv, addr, &addr1, &l, false, attrs);if (len == l &&memory_access_is_direct (mr, false, attrs)) { ptr =qemu_map_ram_ptr (mr->ram_block, addr1); -memmove (buf, ptr, len); +qemu_ram_move (buf, ptr, len); }else { result =flatview_read_continue (fv, addr, attrs, buf, len, addr1, l, mr);
system/physmem.c +37 -2
@@ -3158,6 +3158,41 @@ void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size)invalidate_and_set_dirty (mr, addr, size); } +void qemu_ram_move (void *dst,const void *src,size_t n) +{ +uintptr_t test, len; + +if (src == dst || n ==0 ) { +return ; + } + + test = (uintptr_t )src | (uintptr_t )dst | n; + len = test & -test; + +/* Overlapping buffers, unaligned or oversized access */ +if (n >8 || len != n) { +memmove (dst, src, n); +return ; + } + +switch (len) { +case 1 : +qatomic_set ((uint8_t *)dst,qatomic_read ((uint8_t *)src)); +break ; +case 2 : +qatomic_set ((uint16_t *)dst,qatomic_read ((uint16_t *)src)); +break ; +case 4 : +qatomic_set ((uint32_t *)dst,qatomic_read ((uint32_t *)src)); +break ; +case 8 : +qatomic_set ((uint64_t *)dst,qatomic_read ((uint64_t *)src)); +break ; +default : +g_assert_not_reached (); + } +} +int memory_access_size (MemoryRegion *mr,unsigned l,hwaddr addr) {unsigned access_size_max = mr->ops->valid.max_access_size;
@@ -3270,7 +3305,7 @@ static MemTxResult flatview_write_continue_step(MemTxAttrs attrs,uint8_t *ram_ptr =qemu_ram_ptr_length (mr->ram_block, mr_addr, l, false, true); -memmove (ram_ptr, buf, *l); +qemu_ram_move (ram_ptr, buf, *l);invalidate_and_set_dirty (mr, mr_addr, *l);return MEMTX_OK ;
@@ -3363,7 +3398,7 @@ static MemTxResult flatview_read_continue_step(MemTxAttrs attrs, uint8_t *buf,uint8_t *ram_ptr =qemu_ram_ptr_length (mr->ram_block, mr_addr, l, false, false); -memmove (buf, ram_ptr, *l); +qemu_ram_move (buf, ram_ptr, *l);return MEMTX_OK ; }
Signature · 1 line
2.55.0
Re: [PATCH v4 2/3] system/memory: Use qemu_ram_move() for directly accessible regions
On Mon, 27 Jul 2026 at 04:27, Gavin Shan <[email protected]> wrote:
Initial note: I think this is basically the right thing; I have some suggestions for beefing up the doc comment and some minor other things below.
Quoted 57 lines
All ram device regions were turned to be indirectly accessible by commit
4a2e242bbb("memory: Don't use memcpy forram_deviceregions"). This leads to guest hang on attempt to build 'cuda-samples' as reported by Julia. The guest is started by the following command lines, with GH100 GPU card passed from the host.host$ lspci | grep GH100 0009:01:00.0 3D controller: NVIDIA Corporation GH100 [GH200 120GB / 480GB] (rev a1) host$ /home/sandbox/gavin/qemu.main/build/qemu-system-aarch64 \ -machine virt,gic-version=host,ras=on,highmem-mmio-size=4T \ -accel kvm -cpu host -smp cpus=48 -m size=8G \ -drive file=/home/gavin/sandbox/images/disk.qcow2,if=none,id=d0 \ -device virtio-blk-pci,id=vb0,bus=pcie.0,drive=d0,num-queues=4 \ -device vfio-pci-nohotplug,host=0009:01:00.0,bus=pcie.1.0 : guest$ cd cuda-samples/build guest$ make -j 20 clean guest$ make -j 20 : [ 54%] Linking CUDA executable graphMemoryNodes [ 54%] Built target graphMemoryNodes <no more output afterwards, guest becomes frozen here>guest$ qemu-system-aarch64: virtio: bogus descriptor or out of resources
[ 555.814025] virtio_blk virtio0: [vda] new size: 268435456 512-byte logical blocks (137 GB/128 GiB)When the GPU's driver (NVidia open driver) is loaded on guest bootup, the memory blocks residing in the PCI BAR#4 of the GH100 GPU card can be presented to the guest through memory hot-add. The page cache can then be allocated from the hot added memory blocks when cuda-samples is being built. Afterwards, the page cache is sent to QEMU's virtio-blk device as part of the DMA request, the bounce buffer has to be used to accomodate the request as the corresponding memory region (MemoryRegion) is an indirectly accessible ram device region in qemu. However, the max bounce bufer size is only 4096 bytes by default and that is exhausted quickly, leading to a reset on the virtio-blk device and frozen guest eventually.
QEMU
====
virtio_blk_handle_output
virtio_blk_handle_vq
virtio_blk_get_request
virtqueue_pop
virtqueue_split_pop
virtqueue_map_desc
address_space_map
memory_access_is_direct# Return false
memory_region_supports_direct_access(qemu) info mtree
memory-region:pci_bridge_pci0000000000000000-ffffffffffffffff (prio 0, container): pci_bridge_pci 0000042000000000-0000043fffffffff (prio 1, i/o): 0009:01:00.0 base BAR 4 0000042000000000-0000043fffffffff (prio 0, i/o): 0009:01:00.0 BAR 4 0000042000000000-000004379fffffff (prio 0, ramd): 0009:01:00.0 BAR 4 mmaps[0]
Quoted 7 lines
This adds
qemu_ram_move()where the aligned and small-sized accesses are handled by qatomics, and fall back tomemmove()otherwise. Thememove()for the directly accessible regions is replaced byqemu_ram_move()so that the issue covered by commit4a2e242bbb(MMIO access instructions were optimized to SSE instructions) is fixed. This makes 'ram_device_mem_ops' redundant, paving the way to revert that commit to make the ram device region directly accessible again in the next patch.
I think this commit message should also describe the second category of bug that we intend it to fix: the one where a device does e.g. address_space_stb() to a data structure in guest memory and requires it to write exactly that byte exactly once.
Quoted 15 lines
Reported-by Julia Graham <[email protected]>Suggested-by Michael S. Tsirkin <[email protected]>Suggested-by Peter Xu <[email protected]>Suggested-by Richard Henderson <[email protected]>Suggested-by Peter Maydell <[email protected]>Signed-off-by Gavin Shan <[email protected]>include/system/memory.h +3
-0@@ -2666,6 +2666,21 @@ void address_space_register_map_client(AddressSpace *as, QEMUBH *bh);void address_space_unregister_map_client (AddressSpace *as,QEMUBH *bh);/* Internal functions, part of the implementation of address_space_read. */ + +/** +* qemu_ram_move: move data from or to ramblock
include/system/memory.h void address_space_register_map_client(AddressSpace *as, QEMUBH *bh); · line 2681
Quoted 10 lines
include/system/memory.h +10
-0+* +* @dst: destination where the data is moved to +* @src: source where the data is moved from +* @n: length of data to be moved +* +* Move @n bytes from @src to @dst with the assumption that @src and @dst +* can overlap. The access is atomic if the source and destination buffer +* aren't overlapped for a well aligned and small-sized access. Otherwise, +* fall back to the standard memmove(). +*/
I think we could usefully expand this comment, because the reasons we need it are not immediately obvious. How about:
===begin===
Move @n bytes from @src to @dst; the memory areas may overlap. This provides the same semantics as memmove(), plus an additional stronger guarantee: if @n is 1, 2 or 4 or 8 bytes, and @src and @dst are both naturally aligned for that access size, and the memory areas do not overlap, then both the load and the store will be done as a single atomic access (with the semantics of qatomic_read() and qatomic_set()).
This is the underlying function that we use to implement accesses by a guest vCPU or a device DMA operation to a ram block. The atomic guarantee is needed for two major cases:
- when the ram block is backed by a PCI BAR passed through from a host device (and so it might be hardware registers that must be accessed exactly once at the right width)
- when an emulated device updates a data structure shared in guest memory with guest software (e.g. a network device's set of tx and rx descriptor blocks), if a write to memory is accidentally performed multiple times then it can break the guest code.
We don't attempt to perform the exact access when it would be unaligned, because this can't necessarily be done on all host architectures; although this is strictly speaking not doing what would happen on real hardware, we don't think there are going to be situations where that matters in practice. ===endit===
?
(if you take my suggestion below, then delete "and the memory areas do not overlap", because the only case where the accesses are naturally aligned and they overlap is the case of src == dst)
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3165
Quoted 13 lines
system/physmem.c +5
-0@@ -3158,6 +3158,41 @@ void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size)invalidate_and_set_dirty (mr, addr, size); } +void qemu_ram_move (void *dst,const void *src,size_t n) +{ +uintptr_t test, len; + +if (src == dst || n ==0 ) {
I think we should not bother testing for src == dst. It's vanishingly unlikely to actually happen, and if it does happen then the code will work fine, and it's probably better to actually do the access in that case than to skip it.
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3170
Quoted 5 lines
system/physmem.c +5
-0+return ; + } + + test = (uintptr_t )src | (uintptr_t )dst | n; + len = test & -test;
What is this doing? I am not a fan of clever bit twiddling that isn't commented to explain itself. Readers of the code shouldn't have to go off and search for an explanation of what is going on.
Quoted 14 lines
system/physmem.c +14
-0+ +/* Overlapping buffers, unaligned or oversized access */ +if (n >8 || len != n) { +memmove (dst, src, n); +return ; + } + +switch (len) { +case 1 : +qatomic_set ((uint8_t *)dst,qatomic_read ((uint8_t *)src)); +break ; +case 2 : +qatomic_set ((uint16_t *)dst,qatomic_read ((uint16_t *)src)); +break ;
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3194
Quoted 10 lines
system/physmem.c +10
-0+case 4 : +qatomic_set ((uint32_t *)dst,qatomic_read ((uint32_t *)src)); +break ; +case 8 : +qatomic_set ((uint64_t *)dst,qatomic_read ((uint64_t *)src)); +break ; +default : +g_assert_not_reached (); + } +}
thanks
-- PMM
Re: [PATCH v4 2/3] system/memory: Use qemu_ram_move() for directly accessible regions
Quoted 71 lines · Peter Maydell
On Mon, 27 Jul 2026 at 04:27, Gavin Shan <[email protected]> wrote:
Initial note: I think this is basically the right thing; I have some suggestions for beefing up the doc comment and some minor other things below.
Quoted 57 lines
All ram device regions were turned to be indirectly accessible by commit
4a2e242bbb("memory: Don't use memcpy forram_deviceregions"). This leads to guest hang on attempt to build 'cuda-samples' as reported by Julia. The guest is started by the following command lines, with GH100 GPU card passed from the host.host$ lspci | grep GH100 0009:01:00.0 3D controller: NVIDIA Corporation GH100 [GH200 120GB / 480GB] (rev a1) host$ /home/sandbox/gavin/qemu.main/build/qemu-system-aarch64 \ -machine virt,gic-version=host,ras=on,highmem-mmio-size=4T \ -accel kvm -cpu host -smp cpus=48 -m size=8G \ -drive file=/home/gavin/sandbox/images/disk.qcow2,if=none,id=d0 \ -device virtio-blk-pci,id=vb0,bus=pcie.0,drive=d0,num-queues=4 \ -device vfio-pci-nohotplug,host=0009:01:00.0,bus=pcie.1.0 : guest$ cd cuda-samples/build guest$ make -j 20 clean guest$ make -j 20 : [ 54%] Linking CUDA executable graphMemoryNodes [ 54%] Built target graphMemoryNodes <no more output afterwards, guest becomes frozen here> guest$ qemu-system-aarch64: virtio: bogus descriptor or out of resources [ 555.814025] virtio_blk virtio0: [vda] new size: 268435456 512-byte logical blocks (137 GB/128 GiB)When the GPU's driver (NVidia open driver) is loaded on guest bootup, the memory blocks residing in the PCI BAR#4 of the GH100 GPU card can be presented to the guest through memory hot-add. The page cache can then be allocated from the hot added memory blocks when cuda-samples is being built. Afterwards, the page cache is sent to QEMU's virtio-blk device as part of the DMA request, the bounce buffer has to be used to accomodate the request as the corresponding memory region (MemoryRegion) is an indirectly accessible ram device region in qemu. However, the max bounce bufer size is only 4096 bytes by default and that is exhausted quickly, leading to a reset on the virtio-blk device and frozen guest eventually.
QEMU ==== virtio_blk_handle_output virtio_blk_handle_vq virtio_blk_get_request virtqueue_pop virtqueue_split_pop virtqueue_map_desc address_space_map memory_access_is_direct # Return false memory_region_supports_direct_access (qemu) info mtree memory-region: pci_bridge_pci 0000000000000000-ffffffffffffffff (prio 0, container): pci_bridge_pci 0000042000000000-0000043fffffffff (prio 1, i/o): 0009:01:00.0 base BAR 4 0000042000000000-0000043fffffffff (prio 0, i/o): 0009:01:00.0 BAR 4 0000042000000000-000004379fffffff (prio 0, ramd): 0009:01:00.0 BAR 4 mmaps[0]Quoted 7 lines
This adds
qemu_ram_move()where the aligned and small-sized accesses are handled by qatomics, and fall back tomemmove()otherwise. Thememove()for the directly accessible regions is replaced byqemu_ram_move()so that the issue covered by commit4a2e242bbb(MMIO access instructions were optimized to SSE instructions) is fixed. This makes 'ram_device_mem_ops' redundant, paving the way to revert that commit to make the ram device region directly accessible again in the next patch.
Quoted 12 lines
I think this commit message should also describe the second category of bug that we intend it to fix: the one where a device does e.g.
address_space_stb()to a data structure in guest memory and requires it to write exactly that byte exactly once.Quoted 6 lines
Reported-by Julia Graham <[email protected]>Suggested-by Michael S. Tsirkin <[email protected]>Suggested-by Peter Xu <[email protected]>Suggested-by Richard Henderson <[email protected]>Suggested-by Peter Maydell <[email protected]>Signed-off-by Gavin Shan <[email protected]>
---
Quoted 17 lines
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3165
Quoted 5 lines
system/physmem.c +5
-0+void qemu_ram_move (void *dst,const void *src,size_t n) +{ +uintptr_t test, len; + +if (src == dst || n ==0 ) {I think we should not bother testing for src == dst. It's vanishingly unlikely to actually happen, and if it does happen then the code will work fine, and it's probably better to actually do the access in that case than to skip it.
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3170
Quoted 5 lines
system/physmem.c +5
-0+return ; + } + + test = (uintptr_t )src | (uintptr_t )dst | n; + len = test & -test;
Quoted 4 lines
What is this doing? I am not a fan of clever bit twiddling that isn't commented to explain itself. Readers of the code shouldn't have to go off and search for an explanation of what is going on.
"max alignment of the 3 values"?
Quoted 17 lines
Quoted 17 lines
system/physmem.c +16
-0+ +/* Overlapping buffers, unaligned or oversized access */ +if (n >8 || len != n) { +memmove (dst, src, n); +return ; + } + +switch (len) { +case 1 : +qatomic_set ((uint8_t *)dst,qatomic_read ((uint8_t *)src)); +break ; +case 2 : +qatomic_set ((uint16_t *)dst,qatomic_read ((uint16_t *)src)); +break ; +case 4 : +qatomic_set ((uint32_t *)dst,qatomic_read ((uint32_t *)src));
Quoted 8 lines
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3194
Quoted 8 lines
system/physmem.c +8
-0+break ; +case 8 : +qatomic_set ((uint64_t *)dst,qatomic_read ((uint64_t *)src)); +break ; +default : +g_assert_not_reached (); + } +}
Maybe more readable (untested):
-- >8 --
void qemu_ram_move(void *dst, const void *src, size_t len)
{
if (unlikely(n == 0)) {
return;
}
if (len == 1) {
qatomic_set((uint8_t *)dst, qatomic_read((uint8_t *)src));
return;
} else if (QEMU_PTR_IS_ALIGNED(dst, len) &&
QEMU_PTR_IS_ALIGNED(src, len)) {
switch (len) {
case 2:
qatomic_set((uint16_t *)dst, qatomic_read((uint16_t *)src));
return;
case 4:
qatomic_set((uint32_t *)dst, qatomic_read((uint32_t *)src));
return;
case 8:
qatomic_set((uint64_t *)dst, qatomic_read((uint64_t *)src));
return;
default:
break;
}
}
/* Overlapping buffers, unaligned or oversized access */
memmove(dst, src, len); }
---
Re: [PATCH v4 2/3] system/memory: Use qemu_ram_move() for directly accessible regions
Quoted 103 lines · Philippe Mathieu-Daudé
Quoted 71 lines · Peter Maydell
On Mon, 27 Jul 2026 at 04:27, Gavin Shan <[email protected]> wrote:
Initial note: I think this is basically the right thing; I have some suggestions for beefing up the doc comment and some minor other things below.
Quoted 57 lines
All ram device regions were turned to be indirectly accessible by commit
4a2e242bbb("memory: Don't use memcpy forram_deviceregions"). This leads to guest hang on attempt to build 'cuda-samples' as reported by Julia. The guest is started by the following command lines, with GH100 GPU card passed from the host.host$ lspci | grep GH100
0009:01:00.0 3D controller: NVIDIA Corporation GH100 [GH200 120GB / 480GB] (rev a1) host$ /home/sandbox/gavin/qemu.main/build/qemu-system-aarch64 \
-machine virt,gic-version=host,ras=on,highmem-mmio-size=4T \
-accel kvm -cpu host -smp cpus=48 -m size=8G \
-drive file=/home/gavin/sandbox/images/disk.qcow2,if=none,id=d0 \
-device virtio-blk-pci,id=vb0,bus=pcie.0,drive=d0,num-queues=4 \
-device vfio-pci-nohotplug,host=0009:01:00.0,bus=pcie.1.0
:
guest$ cd cuda-samples/build
guest$ make -j 20 clean
guest$ make -j 20
:
[ 54%] Linking CUDA executable graphMemoryNodes
[ 54%] Built target graphMemoryNodes
<no more output afterwards, guest becomes frozen here>guest$ qemu-system-aarch64: virtio: bogus descriptor or out of resources
[ 555.814025]virtio_blkvirtio0: [vda] new size: 268435456 512-byte logical blocks (137 GB/128 GiB)When the GPU's driver (NVidia open driver) is loaded on guest bootup, the memory blocks residing in the PCI BAR#4 of the GH100 GPU card can be presented to the guest through memory hot-add. The page cache can then be allocated from the hot added memory blocks when cuda-samples is being built. Afterwards, the page cache is sent to QEMU's virtio-blk device as part of the DMA request, the bounce buffer has to be used to accomodate the request as the corresponding memory region (MemoryRegion) is an indirectly accessible ram device region in qemu. However, the max bounce bufer size is only 4096 bytes by default and that is exhausted quickly, leading to a reset on the virtio-blk device and frozen guest eventually.
QEMU
====
virtio_blk_handle_output
virtio_blk_handle_vq
virtio_blk_get_request
virtqueue_pop
virtqueue_split_pop
virtqueue_map_desc
address_space_mapmemory_access_is_direct # Return falsememory_region_supports_direct_access(qemu) info mtree
memory-region:pci_bridge_pci
0000000000000000-ffffffffffffffff (prio 0, container):pci_bridge_pci0000042000000000-0000043fffffffff (prio 1, i/o): 0009:01:00.0 base BAR 4 0000042000000000-0000043fffffffff (prio 0, i/o): 0009:01:00.0 BAR 4 0000042000000000-000004379fffffff (prio 0, ramd): 0009:01:00.0 BAR 4 mmaps[0]Quoted 7 lines
This adds
qemu_ram_move()where the aligned and small-sized accesses are handled by qatomics, and fall back tomemmove()otherwise. Thememove()for the directly accessible regions is replaced byqemu_ram_move()so that the issue covered by commit4a2e242bbb(MMIO access instructions were optimized to SSE instructions) is fixed. This makes 'ram_device_mem_ops' redundant, paving the way to revert that commit to make the ram device region directly accessible again in the next patch.Quoted 12 lines
I think this commit message should also describe the second category of bug that we intend it to fix: the one where a device does e.g.
address_space_stb()to a data structure in guest memory and requires it to write exactly that byte exactly once.Quoted 6 lines
Reported-by Julia Graham <[email protected]>Suggested-by Michael S. Tsirkin <[email protected]>Suggested-by Peter Xu <[email protected]>Suggested-by Richard Henderson <[email protected]>Suggested-by Peter Maydell <[email protected]>Signed-off-by Gavin Shan <[email protected]>---
Quoted 17 lines
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3165
Quoted 5 lines
system/physmem.c +5
-0+void qemu_ram_move (void *dst,const void *src,size_t n) +{ + uintptr_t test, len; + + if (src == dst || n == 0) {I think we should not bother testing for src == dst. It's vanishingly unlikely to actually happen, and if it does happen then the code will work fine, and it's probably better to actually do the access in that case than to skip it.
Quoted 5 lines
+ return;
+ }
+
+ test = (uintptr_t)src | (uintptr_t)dst | n;
+ len = test & -test;
Quoted 7 lines
Quoted 4 lines
What is this doing? I am not a fan of clever bit twiddling that isn't commented to explain itself. Readers of the code shouldn't have to go off and search for an explanation of what is going on.
"max alignment of the 3 values"?
Yes, it's the maximal length of the aligned access that is determined by @src, @dst and @n. I've put a comments for (v5).
/*
* Maximal length of aligned access that are determined by @src,
* @dst and @n
*/
Quoted 48 lines
Quoted 17 lines
Quoted 17 lines
system/physmem.c +16
-0+ + /* Overlapping buffers, unaligned or oversized access */ + if (n > 8 || len != n) { + memmove(dst, src, n); + return; + } + + switch (len) { + case 1: + qatomic_set((uint8_t *)dst, qatomic_read((uint8_t *)src)); + break; + case 2: + qatomic_set((uint16_t *)dst, qatomic_read((uint16_t *)src)); + break; + case 4: + qatomic_set((uint32_t *)dst, qatomic_read((uint32_t *)src));Quoted 8 lines
Quoted 8 lines
+ break;
+ case 8:
+qatomic_set((uint64_t *)dst, qatomic_read((uint64_t *)src)); + break;
+ default:
+g_assert_not_reached();
+ }
+}Maybe more readable (untested):
-- >8 --
voidqemu_ram_move(void *dst, const void *src, size_t len)
{
if (unlikely(n == 0)) {
return;
}
if (len == 1) {
qatomic_set((uint8_t *)dst, qatomic_read((uint8_t *)src)); return;
} else if (QEMU_PTR_IS_ALIGNED(dst, len)&&QEMU_PTR_IS_ALIGNED(src, len)) { switch (len) {
case 2:
qatomic_set((uint16_t *)dst, qatomic_read((uint16_t *)src)); return;
case 4:
qatomic_set((uint32_t *)dst, qatomic_read((uint32_t *)src)); return;
case 8:
qatomic_set((uint64_t *)dst, qatomic_read((uint64_t *)src)); return;
Quoted 9 lines
default:
break;
}
}
/* Overlapping buffers, unaligned or oversized access */memmove(dst, src, len);
}
---
Thanks for the proposed code, but I'd like to keep what we already had. One thing I try to avoid from the beginning is the unnecessary nested if statements. Besides, one irrelevant question is that QEMU_PTR_IS_ALIGNED() looks a bit strange as it uses % operator, meaning it works even the operand isn't power of 2.
#define QEMU_IS_ALIGNED(n, m) (((n) % (m)) == 0)
#define QEMU_PTR_IS_ALIGNED(p, n) QEMU_IS_ALIGNED((uintptr_t)(p), (n))
QEMU_PTR_IS_ALIGNED(3, 3) => true
Thanks,
Gavin
Re: [PATCH v4 2/3] system/memory: Use qemu_ram_move() for directly accessible regions
In reply to Peter MaydellQuoted 12 lines · Peter Maydell
I think we could usefully expand this comment, because the reasons we need it are not immediately obvious. How about:
===begin===
Move @n bytes from @src to @dst; the memory areas may overlap. This provides the same semantics asmemmove(), plus an additional stronger guarantee: if @n is 1, 2 or 4 or 8 bytes, and @src and @dst are both naturally aligned for that access size, and the memory areas do not overlap, then both the load and the store will be done as a single atomic access (with the semantics ofqatomic_read()andqatomic_set()).
Quoted 11 lines
This is the underlying function that we use to implement accesses by a guest vCPU or a device DMA operation to a ram block. The atomic guarantee is needed for two major cases:
- when the ram block is backed by a PCI BAR passed through from a host device (and so it might be hardware registers that must be accessed exactly once at the right width)
- when an emulated device updates a data structure shared in guest memory with guest software (e.g. a network device's set of tx and rx descriptor blocks), if a write to memory is accidentally performed multiple times then it can break the guest code.
Maybe also append it with "the guest code when it busy polls the guest memory"; I just found that the polling model isn't something obvious too when reading it first.
Quoted 6 lines
We don't attempt to perform the exact access when it would be unaligned, because this can't necessarily be done on all host architectures; although this is strictly speaking not doing what would happen on real hardware, we don't think there are going to be situations where that matters in practice.
Oh yes, mentioning the unaligned part of discussion would also be nice, I forgot it when replying.
Thanks,
Quoted 1 line
===endit===
Signature · 1 line
Peter Xu
Re: [PATCH v4 2/3] system/memory: Use qemu_ram_move() for directly accessible regions
Quoted 25 lines · Peter Xu
Quoted 12 lines · Peter Maydell
I think we could usefully expand this comment, because the reasons we need it are not immediately obvious. How about:
===begin===
Move @n bytes from @src to @dst; the memory areas may overlap. This provides the same semantics asmemmove(), plus an additional stronger guarantee: if @n is 1, 2 or 4 or 8 bytes, and @src and @dst are both naturally aligned for that access size, and the memory areas do not overlap, then both the load and the store will be done as a single atomic access (with the semantics ofqatomic_read()andqatomic_set()).Quoted 11 lines
This is the underlying function that we use to implement accesses by a guest vCPU or a device DMA operation to a ram block. The atomic guarantee is needed for two major cases:
- when the ram block is backed by a PCI BAR passed through from a host device (and so it might be hardware registers that must be accessed exactly once at the right width)
- when an emulated device updates a data structure shared in guest memory with guest software (e.g. a network device's set of tx and rx descriptor blocks), if a write to memory is accidentally performed multiple times then it can break the guest code.
Quoted 4 lines
Maybe also append it with "the guest code when it busy polls the guest memory"; I just found that the polling model isn't something obvious too when reading it first.
The extended context has been included for (v5).
Quoted 10 lines
Quoted 6 lines
We don't attempt to perform the exact access when it would be unaligned, because this can't necessarily be done on all host architectures; although this is strictly speaking not doing what would happen on real hardware, we don't think there are going to be situations where that matters in practice.
Oh yes, mentioning the unaligned part of discussion would also be nice, I forgot it when replying.
+1. The context has been included for (v5) either.
Thanks,
Gavin
Re: [PATCH v4 2/3] system/memory: Use qemu_ram_move() for directly accessible regions
In reply to Peter MaydellQuoted 6 lines · Peter Maydell
On Mon, 27 Jul 2026 at 04:27, Gavin Shan <[email protected]> wrote:
Initial note: I think this is basically the right thing; I have some suggestions for beefing up the doc comment and some minor other things below.
Thanks for your review and comments.
Quoted 65 lines
Quoted 57 lines
All ram device regions were turned to be indirectly accessible by commit
4a2e242bbb("memory: Don't use memcpy forram_deviceregions"). This leads to guest hang on attempt to build 'cuda-samples' as reported by Julia. The guest is started by the following command lines, with GH100 GPU card passed from the host.host$ lspci | grep GH100 0009:01:00.0 3D controller: NVIDIA Corporation GH100 [GH200 120GB / 480GB] (rev a1) host$ /home/sandbox/gavin/qemu.main/build/qemu-system-aarch64 \ -machine virt,gic-version=host,ras=on,highmem-mmio-size=4T \ -accel kvm -cpu host -smp cpus=48 -m size=8G \ -drive file=/home/gavin/sandbox/images/disk.qcow2,if=none,id=d0 \ -device virtio-blk-pci,id=vb0,bus=pcie.0,drive=d0,num-queues=4 \ -device vfio-pci-nohotplug,host=0009:01:00.0,bus=pcie.1.0 : guest$ cd cuda-samples/build guest$ make -j 20 clean guest$ make -j 20 : [ 54%] Linking CUDA executable graphMemoryNodes [ 54%] Built target graphMemoryNodes <no more output afterwards, guest becomes frozen here> guest$ qemu-system-aarch64: virtio: bogus descriptor or out of resources [ 555.814025] virtio_blk virtio0: [vda] new size: 268435456 512-byte logical blocks (137 GB/128 GiB)When the GPU's driver (NVidia open driver) is loaded on guest bootup, the memory blocks residing in the PCI BAR#4 of the GH100 GPU card can be presented to the guest through memory hot-add. The page cache can then be allocated from the hot added memory blocks when cuda-samples is being built. Afterwards, the page cache is sent to QEMU's virtio-blk device as part of the DMA request, the bounce buffer has to be used to accomodate the request as the corresponding memory region (MemoryRegion) is an indirectly accessible ram device region in qemu. However, the max bounce bufer size is only 4096 bytes by default and that is exhausted quickly, leading to a reset on the virtio-blk device and frozen guest eventually.
QEMU ==== virtio_blk_handle_output virtio_blk_handle_vq virtio_blk_get_request virtqueue_pop virtqueue_split_pop virtqueue_map_desc address_space_map memory_access_is_direct # Return false memory_region_supports_direct_access (qemu) info mtree memory-region: pci_bridge_pci 0000000000000000-ffffffffffffffff (prio 0, container): pci_bridge_pci 0000042000000000-0000043fffffffff (prio 1, i/o): 0009:01:00.0 base BAR 4 0000042000000000-0000043fffffffff (prio 0, i/o): 0009:01:00.0 BAR 4 0000042000000000-000004379fffffff (prio 0, ramd): 0009:01:00.0 BAR 4 mmaps[0]Quoted 7 lines
This adds
qemu_ram_move()where the aligned and small-sized accesses are handled by qatomics, and fall back tomemmove()otherwise. Thememove()for the directly accessible regions is replaced byqemu_ram_move()so that the issue covered by commit4a2e242bbb(MMIO access instructions were optimized to SSE instructions) is fixed. This makes 'ram_device_mem_ops' redundant, paving the way to revert that commit to make the ram device region directly accessible again in the next patch.
Quoted 5 lines
I think this commit message should also describe the second category of bug that we intend it to fix: the one where a device does e.g.
address_space_stb()to a data structure in guest memory and requires it to write exactly that byte exactly once.
Yes, making sense. I've added below context for (v5), which will be posted shortly.
---
Besides, this also fixes the issue of the unexpected frozen reception on e1000 NIC in the scenario of DPDK due to the wrong Rx queue full indication caused by the following memcpy(), which is turned to 3 consective 'strb' instructions to the same location by glibc-2.24+ for aarch64. With this applied, the syntax of one-byte store is strictly ensured by a one-byte qatomic set.
QEMU
====
e1000_receive_iov
pci_dma_write
pci_dma_rw
dma_memory_rw
dma_memory_rw_relaxed
address_space_rw
address_space_write
flatview_write
flatview_write_continue
flatview_write_continue_step
memcpy # 3 consective 'strb' instructions
Quoted 50 lines
Quoted 15 lines
Reported-by Julia Graham <[email protected]>Suggested-by Michael S. Tsirkin <[email protected]>Suggested-by Peter Xu <[email protected]>Suggested-by Richard Henderson <[email protected]>Suggested-by Peter Maydell <[email protected]>Signed-off-by Gavin Shan <[email protected]>include/system/memory.h +3
-0@@ -2666,6 +2666,21 @@ void address_space_register_map_client(AddressSpace *as, QEMUBH *bh);void address_space_unregister_map_client (AddressSpace *as,QEMUBH *bh);/* Internal functions, part of the implementation of address_space_read. */ + +/** +* qemu_ram_move: move data from or to ramblock include/system/memory.h void address_space_register_map_client(AddressSpace *as, QEMUBH *bh); · line 2681
Quoted 10 lines
include/system/memory.h +10
-0+* +* @dst: destination where the data is moved to +* @src: source where the data is moved from +* @n: length of data to be moved +* +* Move @n bytes from @src to @dst with the assumption that @src and @dst +* can overlap. The access is atomic if the source and destination buffer +* aren't overlapped for a well aligned and small-sized access. Otherwise, +* fall back to the standard memmove(). +*/ I think we could usefully expand this comment, because the reasons we need it are not immediately obvious. How about:
===begin===
Move @n bytes from @src to @dst; the memory areas may overlap. This provides the same semantics asmemmove(), plus an additional stronger guarantee: if @n is 1, 2 or 4 or 8 bytes, and @src and @dst are both naturally aligned for that access size, and the memory areas do not overlap, then both the load and the store will be done as a single atomic access (with the semantics ofqatomic_read()andqatomic_set()).This is the underlying function that we use to implement accesses by a guest vCPU or a device DMA operation to a ram block. The atomic guarantee is needed for two major cases:
- when the ram block is backed by a PCI BAR passed through from a host device (and so it might be hardware registers that must be accessed exactly once at the right width)
- when an emulated device updates a data structure shared in guest memory with guest software (e.g. a network device's set of tx and rx descriptor blocks), if a write to memory is accidentally performed multiple times then it can break the guest code.
Quoted 9 lines
We don't attempt to perform the exact access when it would be unaligned, because this can't necessarily be done on all host architectures; although this is strictly speaking not doing what would happen on real hardware, we don't think there are going to be situations where that matters in practice. ===endit===
?
Thanks for the suggested comments to qemu_ram_move(), which looks much clearer than what we had. I've integrated this for (v5) with minor format adjustment.
Quoted 5 lines
(if you take my suggestion below, then delete "and the memory areas do not overlap", because the only case where the accesses are naturally aligned and they overlap is the case of src == dst)
I would keep the check (src == dst), see the explanation below.
Quoted 9 lines
Quoted 9 lines
system/physmem.c +0
-0@@ -3158,6 +3158,41 @@ void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size)invalidate_and_set_dirty (mr, addr, size); }
Quoted 11 lines
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3165
Quoted 5 lines
system/physmem.c +5
-0+void qemu_ram_move (void *dst,const void *src,size_t n) +{ +uintptr_t test, len; + +if (src == dst || n ==0 ) {I think we should not bother testing for src == dst. It's vanishingly unlikely to actually happen, and if it does happen then the code will work fine, and it's probably better to actually do the access in that case than to skip it.
We can drop the check of (src == dst), but it will introduce inconsistent behaviors. For example, qemu_ram_move(0x1, 0x1, 0x2) is finally turned to memmove(0x1, 0x1,0x2) where no memory movement happens in glibc::memmove(), but qemu_ram_move(0x0, 0x0, 0x2) is turned to qatomic_set((uint16_t *)dst, qatomic_read((uint16_t *)src)) where we do have memory movement happening. So I would like to keep the check of (src == dst) in order for the consistent behaviors.
Quoted 11 lines
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3170
Quoted 5 lines
system/physmem.c +5
-0+return ; + } + + test = (uintptr_t )src | (uintptr_t )dst | n; + len = test & -test;What is this doing? I am not a fan of clever bit twiddling that isn't commented to explain itself. Readers of the code shouldn't have to go off and search for an explanation of what is going on.
The following comments will be added for (v5).
/*
* Maximal length of aligned access that are determined by @src,
* @dst and @n
*/
Quoted 16 lines
Quoted 16 lines
system/physmem.c +16
-0+ +/* Overlapping buffers, unaligned or oversized access */ +if (n >8 || len != n) { +memmove (dst, src, n); +return ; + } + +switch (len) { +case 1 : +qatomic_set ((uint8_t *)dst,qatomic_read ((uint8_t *)src)); +break ; +case 2 : +qatomic_set ((uint16_t *)dst,qatomic_read ((uint16_t *)src)); +break ; +case 4 : +qatomic_set ((uint32_t *)dst,qatomic_read ((uint32_t *)src));
Quoted 8 lines
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3194
Quoted 8 lines
system/physmem.c +8
-0+break ; +case 8 : +qatomic_set ((uint64_t *)dst,qatomic_read ((uint64_t *)src)); +break ; +default : +g_assert_not_reached (); + } +}
Thanks,
Gavin
Re: [PATCH v4 2/3] system/memory: Use qemu_ram_move() for directly accessible regions
Quoted 15 lines · Gavin Shan
Quoted 11 lines · Peter Maydell
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3165
Quoted 5 lines
system/physmem.c +5
-0+void qemu_ram_move (void *dst,const void *src,size_t n) +{ +uintptr_t test, len; + +if (src == dst || n ==0 ) {I think we should not bother testing for src == dst. It's vanishingly unlikely to actually happen, and if it does happen then the code will work fine, and it's probably better to actually do the access in that case than to skip it.
Quoted 7 lines
We can drop the check of (src == dst), but it will introduce inconsistent behaviors. For example,
qemu_ram_move(0x1, 0x1, 0x2)is finally turned tomemmove(0x1, 0x1,0x2)where no memory movement happens inglibc::memmove(), butqemu_ram_move(0x0, 0x0, 0x2)is turned toqatomic_set((uint16_t *)dst, qatomic_read((uint16_t *)src))where we do have memory movement happening. So I would like to keep the check of (src == dst) in order for the consistent behaviors.
Well, it depends on which kind of consistency you want. If we do the src == dst check, then we have the inconsistency that a small aligned access always happens exactly once, *unless* it happens that src == dst, in which case it doesn't happen even though the caller asked for it.
You can see this in the documentation comment I suggested above:
if you don't check 'src == dst' then you can document it as "memmove, plus for a small access where src and dst are naturally aligned we guarantee it to happen exactly once atomically". If you do check src == dst then it gets more complicated, because you have to say "memmove, plus for a small access where src and dest are naturally aligned *and* src != dst then ...".
I think that because it's the small sized accesses where we know that it might be necessary to really actually do the access, that we should do that also for src == dst.
Quoted 9 lines
Quoted 8 lines
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3170
Quoted 2 lines
system/physmem.c +2
-0+ test = (uintptr_t )src | (uintptr_t )dst | n; + len = test & -test;What is this doing? I am not a fan of clever bit twiddling that isn't commented to explain itself. Readers of the code shouldn't have to go off and search for an explanation of what is going on.
Quoted 6 lines
The following comments will be added for (v5).
/*
* Maximal length of aligned access that are determined by @src,
* @dst and @n
*/
What I mean is more that it ought to say why "x & -x" achieves that goal.
-- PMM
Re: [PATCH v4 2/3] system/memory: Use qemu_ram_move() for directly accessible regions
Quoted 53 lines · Peter Maydell
Quoted 15 lines · Gavin Shan
Quoted 11 lines · Peter Maydell
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3165
Quoted 5 lines
system/physmem.c +5
-0+void qemu_ram_move (void *dst,const void *src,size_t n) +{ +uintptr_t test, len; + +if (src == dst || n ==0 ) {I think we should not bother testing for src == dst. It's vanishingly unlikely to actually happen, and if it does happen then the code will work fine, and it's probably better to actually do the access in that case than to skip it.
Quoted 7 lines
We can drop the check of (src == dst), but it will introduce inconsistent behaviors. For example,
qemu_ram_move(0x1, 0x1, 0x2)is finally turned tomemmove(0x1, 0x1,0x2)where no memory movement happens inglibc::memmove(), butqemu_ram_move(0x0, 0x0, 0x2)is turned toqatomic_set((uint16_t *)dst, qatomic_read((uint16_t *)src))where we do have memory movement happening. So I would like to keep the check of (src == dst) in order for the consistent behaviors.Well, it depends on which kind of consistency you want. If we do the src == dst check, then we have the inconsistency that a small aligned access always happens exactly once, *unless* it happens that src == dst, in which case it doesn't happen even though the caller asked for it.
You can see this in the documentation comment I suggested above:
if you don't check 'src == dst' then you can document it as "memmove, plus for a small access where src and dst are naturally aligned we guarantee it to happen exactly once atomically". If you do check src == dst then it gets more complicated, because you have to say "memmove, plus for a small access where src and dest are naturally aligned *and* src != dst then ...".I think that because it's the small sized accesses where we know that it might be necessary to really actually do the access, that we should do that also for src == dst.
Quoted 11 lines
Quoted 8 lines
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3170
Quoted 2 lines
system/physmem.c +2
-0+ test = (uintptr_t )src | (uintptr_t )dst | n; + len = test & -test;What is this doing? I am not a fan of clever bit twiddling that isn't commented to explain itself. Readers of the code shouldn't have to go off and search for an explanation of what is going on.
The following comments will be added for (v5).
Quoted 7 lines
Quoted 4 lines
/*
* Maximal length of aligned access that are determined by @src,
* @dst and @n
*/What I mean is more that it ought to say why "x & -x" achieves that goal.
Hmm isn't "x & -x" restricted to power of 2? Here we have no garanty @test is, because we have no garanty @n is. Ah, this is catched by the 'len != n' check and returns after calling memmove(). Indeed not very clear notation. Again:
...
/* Overlapping buffers, unaligned or oversized access */
if (!QEMU_IS_ALIGNED((uintptr_t)src | (uintptr_t)dst, n)) {
memmove(dst, src, n);
return;
}
switch (n) {
...Re: [PATCH v4 2/3] system/memory: Use qemu_ram_move() for directly accessible regions
Quoted 31 lines · Philippe Mathieu-Daudé
Quoted 16 lines · Peter Maydell
Quoted 15 lines · Gavin Shan
Quoted 8 lines · Peter Maydell
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3170
Quoted 2 lines
system/physmem.c +2
-0+ test = (uintptr_t )src | (uintptr_t )dst | n; + len = test & -test;What is this doing? I am not a fan of clever bit twiddling that isn't commented to explain itself. Readers of the code shouldn't have to go off and search for an explanation of what is going on.
The following comments will be added for (v5).
Quoted 7 lines
Quoted 4 lines
/*
* Maximal length of aligned access that are determined by @src,
* @dst and @n
*/What I mean is more that it ought to say why "x & -x" achieves that goal.
Hmm isn't "x & -x" restricted to power of 2? Here we have no garanty @test is, because we have no garanty @n is. Ah, this is catched by the 'len != n' check and returns after calling
memmove(). Indeed not very clear notation. Again:
Quoted 10 lines
...
/* Overlapping buffers, unaligned or oversized access */
if (!QEMU_IS_ALIGNED((uintptr_t)src | (uintptr_t)dst, n)) {
memmove(dst, src, n);
return;
}
switch (n) {
...I'm not inherently against the bit twiddling (QEMU_IS_ALIGNED on a variable length will do a division, maybe we care?); I just want that if we do use a bit-twiddling trick that we explain why it does the thing we want it to do.
-- PMM
Re: [PATCH v4 2/3] system/memory: Use qemu_ram_move() for directly accessible regions
Quoted 41 lines · Peter Maydell
Quoted 39 lines · Philippe Mathieu-Daudé
Quoted 16 lines · Peter Maydell
Quoted 15 lines · Gavin Shan
Quoted 8 lines · Peter Maydell
system/physmem.c void memory_region_flush_rom_device(MemoryRegion *mr, hwaddr addr, hwaddr size) · line 3170
Quoted 2 lines
system/physmem.c +2
-0+ test = (uintptr_t )src | (uintptr_t )dst | n; + len = test & -test;What is this doing? I am not a fan of clever bit twiddling that isn't commented to explain itself. Readers of the code shouldn't have to go off and search for an explanation of what is going on.
The following comments will be added for (v5).
Quoted 7 lines
Quoted 4 lines
/*
* Maximal length of aligned access that are determined by @src,
* @dst and @n
*/What I mean is more that it ought to say why "x & -x" achieves that goal.
Hmm isn't "x & -x" restricted to power of 2? Here we have no garanty @test is, because we have no garanty @n is. Ah, this is catched by the 'len != n' check and returns after calling
memmove(). Indeed not very clear notation. Again:... /* Overlapping buffers, unaligned or oversized access */ if (!QEMU_IS_ALIGNED((uintptr_t)src | (uintptr_t)dst, n)) { memmove(dst, src, n); return; }
Quoted 8 lines
Quoted 2 lines
switch (n) { ...I'm not inherently against the bit twiddling (
QEMU_IS_ALIGNEDon a variable length will do a division, maybe we care?); I just want that if we do use a bit-twiddling trick that we explain why it does the thing we want it to do.
Ok, PeterM and Philippe, Could you please help to check if below code looks good to you? Thanks a lot :-)
void qemu_ram_move(void *dst, const void *src, size_t n)
{
if (n == 0) {
return;
}
/* Overlapping areas, unaligned or oversized access */
if (!is_power_of_2(n) || n > 8 ||
!QEMU_IS_ALIGNED((uintptr_t)src | (uintptr_t)dst, n)) {
memmove(dst, src, n);
return;
}
switch (n) {
case 1:
qatomic_set((uint8_t *)dst, qatomic_read((uint8_t *)src));
break;
case 2:
qatomic_set((uint16_t *)dst, qatomic_read((uint16_t *)src));
break;
case 4:
qatomic_set((uint32_t *)dst, qatomic_read((uint32_t *)src));
break;
case 8:
qatomic_set((uint64_t *)dst, qatomic_read((uint64_t *)src));
break;
default:
g_assert_not_reached();
}
}QEMU_IS_ALIGNED() doesn't require the arguments are power-of-two values.
#define QEMU_IS_ALIGNED(n, m) (((n) % (m)) == 0) /* QEMU_IS_ALIGNED(3, 3) => true */
I also need to drop "and the memory areas do not overlap" in (v6) from the comments for include/system/memory.h::qemu_ram_move() as the check 'src == dst' check has been dropped. Sorry that I sent (v5) too quick because our downstream need a stabilized version to integrate.
Thanks,
Gavin
Re: [PATCH v4 2/3] system/memory: Use qemu_ram_move() for directly accessible regions
Quoted 42 lines · Gavin Shan
Quoted 41 lines · Peter Maydell
Quoted 39 lines · Philippe Mathieu-Daudé
Quoted 16 lines · Peter Maydell
Quoted 15 lines · Gavin Shan
Quoted 8 lines · Peter Maydell
Review
Quoted 2 lines
(quoted diff) +2
-0+ test = (uintptr_t)src | (uintptr_t)dst | n; + len = test & -test;What is this doing? I am not a fan of clever bit twiddling that isn't commented to explain itself. Readers of the code shouldn't have to go off and search for an explanation of what is going on.
The following comments will be added for (v5).
Quoted 7 lines
Quoted 4 lines
/*
* Maximal length of aligned access that are determined by @src,
* @dst and @n
*/What I mean is more that it ought to say why "x & -x" achieves that goal.
Hmm isn't "x & -x" restricted to power of 2? Here we have no garanty @test is, because we have no garanty @n is. Ah, this is catched by the 'len != n' check and returns after calling
memmove(). Indeed not very clear notation. Again:... /* Overlapping buffers, unaligned or oversized access */ if (!QEMU_IS_ALIGNED((uintptr_t)src | (uintptr_t)dst, n)) { memmove(dst, src, n); return; }
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
switch (n) { ...I'm not inherently against the bit twiddling (
QEMU_IS_ALIGNEDon a variable length will do a division, maybe we care?); I just want that if we do use a bit-twiddling trick that we explain why it does the thing we want it to do.
I think the bit ops should be more efficient. I plan to queue v5 with a fixup, v5 here:
https://lore.kernel.org/qemu-devel/[email protected]/
Fixup:
system/physmem.c +8 -3
@@ -3162,13 +3162,18 @@ void qemu_ram_move(void *dst, const void *src, size_t n) {uintptr_t test, len; -if (src == dst || n ==0 ) { +if (n ==0 ) {return ; }/* -* Maximal length of aligned access that are determined by @src, -* @dst and @n +* Calculate "the lowest set bit" over @src, @dst and @n, result put +* into @len (which guarantees a power-of-two). With that and the +* later check (len!=n), it makes sure that we will only do the atomic +* ops when: +* +* (1) @n is a power-of-two +* (2) @src and @dst addresses are both aligned to @n */ test = (uintptr_t )src | (uintptr_t )dst | n; len = test & -test;
Does it look ok to all of you?
Thanks,
Signature · 1 line
Peter Xu
Re: [PATCH v4 2/3] system/memory: Use qemu_ram_move() for directly accessible regions
Quoted 12 lines · Peter Xu
I think the bit ops should be more efficient. I plan to queue v5 with a fixup, v5 here:
https://lore.kernel.org/qemu-devel/[email protected]/
Fixup:
system/physmem.c +0
-0
system/physmem.c void qemu_ram_move(void *dst, const void *src, size_t n) · line 3167
Quoted 7 lines
system/physmem.c +1
-1@@ -3162,13 +3162,18 @@ void qemu_ram_move(void *dst, const void *src, size_t n) {uintptr_t test, len; -if (src == dst || n ==0 ) { +if (n ==0 ) {return ; }
As we're dropping the src==dst check here, we also need to drop the "and the memory areas do not overlap" clause from the documentation comment.
Otherwise I'm happy with the v5 with that fixup applied, and you can apply my Reviewed-by: tag.
thanks
-- PMM
Re: [PATCH v4 2/3] system/memory: Use qemu_ram_move() for directly accessible regions
Quoted 25 lines · Peter Maydell
Quoted 12 lines · Peter Xu
I think the bit ops should be more efficient. I plan to queue v5 with a fixup, v5 here:
https://lore.kernel.org/qemu-devel/[email protected]/
Fixup:
system/physmem.c +0
-0system/physmem.c void qemu_ram_move(void *dst, const void *src, size_t n) · line 3167
Quoted 7 lines
system/physmem.c +1
-1@@ -3162,13 +3162,18 @@ void qemu_ram_move(void *dst, const void *src, size_t n) {uintptr_t test, len; -if (src == dst || n ==0 ) { +if (n ==0 ) {return ; }As we're dropping the src==dst check here, we also need to drop the "and the memory areas do not overlap" clause from the documentation comment.
Quoted 2 lines
Otherwise I'm happy with the v5 with that fixup applied, and you can apply my Reviewed-by: tag.
Will do, thanks.
Signature · 1 line
Peter Xu
PATCH v4 3/3 system/memory: Make ram device region directly accessible
In reply to Gavin ShanThis basically reverts 4a2e242bbb30 ("memory: Don't use memcpy for ram_device regions") to make ram device region directly accessible again. With this, the bounce buffer is bypassed in address_space_map() when a ram device region is involved, potentially avoid to overrun the (small) bounce buffer.
--- include/system/memory.h | 11 ++--------- system/memory.c | 41 +---------------------------------------- system/trace-events | 2 -- 3 files changed, 3 insertions(+), 51 deletions(-)
include/system/memory.h +2 -9
@@ -2698,15 +2698,8 @@ static inline bool memory_region_supports_direct_access(const MemoryRegion *mr)if (memory_region_is_romd (mr)) {return true; } -if (!memory_region_is_ram (mr)) { -return false; - } -/* -* RAM DEVICE regions can be accessed directly using memcpy, but it might -* be MMIO and access using mempy can be wrong (e.g., using instructions not -* intended for MMIO access). So we treat this as IO. -*/ -return !memory_region_is_ram_device (mr); + +return memory_region_is_ram (mr); } static inline bool memory_access_is_direct(const MemoryRegion *mr,
system/memory.c +1 -40
54 lines not on this page · raw message
system/trace-events +0 -2
@@ -20,8 +20,6 @@ memory_region_ops_read(int cpu_index, void *mr, uint64_t addr, uint64_t value, u memory_region_ops_write(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size, const char *name) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u name '%s'" memory_region_subpage_read(int cpu_index, void *mr, uint64_t offset, uint64_t value, unsigned size) "cpu %d mr %p offset 0x%"PRIx64" value 0x%"PRIx64" size %u" memory_region_subpage_write(int cpu_index, void *mr, uint64_t offset, uint64_t value, unsigned size) "cpu %d mr %p offset 0x%"PRIx64" value 0x%"PRIx64" size %u" -memory_region_ram_device_read(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u" -memory_region_ram_device_write(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u" memory_region_sync_dirty(const char *mr, const char *listener, int global) "mr '%s' listener '%s' synced (global=%d)" flatview_new(void *view, void *root) "%p (root %p)" flatview_destroy(void *view, void *root) "%p (root %p)"
Signature · 1 line
2.55.0
Quoted 7 lines · Gavin Shan
This basically reverts
4a2e242bbb30("memory: Don't use memcpy forram_deviceregions") to make ram device region directly accessible again. With this, the bounce buffer is bypassed inaddress_space_map()when a ram device region is involved, potentially avoid to overrun the (small) bounce buffer.
Quoted 7 lines
Reported-by Julia Graham <[email protected]>Suggested-by Michael S. Tsirkin <[email protected]>Suggested-by Peter Xu <[email protected]>Suggested-by Richard Henderson <[email protected]>Suggested-by Peter Maydell <[email protected]>Signed-off-by Gavin Shan <[email protected]>---
thanks
-- PMM