drm/virtio: Support partial maps of GEM objects
Subsystems (4)
PATCH 1/1 drm/virtio: Support partial maps of GEM objects
Those are useful to implement coherent cross-vm mmap.
--- drivers/gpu/drm/virtio/virtgpu_vram.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/gpu/drm/virtio/virtgpu_vram.c +2 -3
@@ -56,12 +56,11 @@ static int virtio_gpu_vram_mmap(struct drm_gem_object *obj,else if (vram->map_info ==VIRTIO_GPU_MAP_CACHE_UNCACHED ) vma->vm_page_prot =pgprot_noncached (vma->vm_page_prot); -/* Partial mappings of GEM buffers don't happen much in practice. */ -if (vm_size != vram->vram_node.size) +if (vm_size > vram->vram_node.size)return -EINVAL ; ret =io_remap_pfn_range (vma, vma->vm_start, - vram->vram_node.start >>PAGE_SHIFT , + (vram->vram_node.start >>PAGE_SHIFT ) + vma->vm_pgoff, vm_size, vma->vm_page_prot);return ret; }
---
base-commit: 643e2e259c2b25a2af0ae4c23c6e16586d9fd19c
change-id: 20250109-virtgpu-gem-partial-map-335ec40656d1
Dmitry Osipenko
maintainerHi,
Quoted 18 lines · On 1/10/25 00:45, Sasha Finkelstein via B4 Relay wrote:
From: Sasha Finkelstein <[email protected]>
Those are useful to implement coherent cross-vm mmap.
Signed-off-by Sasha Finkelstein <[email protected]>--- drivers/gpu/drm/virtio/virtgpu_vram.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)drivers/gpu/drm/virtio/virtgpu_vram.c +0
-1@@ -56,12 +56,11 @@ static int virtio_gpu_vram_mmap(struct drm_gem_object *obj,else if (vram->map_info ==VIRTIO_GPU_MAP_CACHE_UNCACHED ) vma->vm_page_prot =pgprot_noncached (vma->vm_page_prot); -/* Partial mappings of GEM buffers don't happen much in practice. */
drivers/gpu/drm/virtio/virtgpu_vram.c static int virtio_gpu_vram_mmap(struct drm_gem_object *obj, · line 66
Quoted 10 lines
drivers/gpu/drm/virtio/virtgpu_vram.c +2
-2-if (vm_size != vram->vram_node.size) +if (vm_size > vram->vram_node.size)return -EINVAL ; ret =io_remap_pfn_range (vma, vma->vm_start, - vram->vram_node.start >>PAGE_SHIFT , + (vram->vram_node.start >>PAGE_SHIFT ) + vma->vm_pgoff, vm_size, vma->vm_page_prot);return ret; }
The vma->vm_pgoff is fake in DRM, it's used for looking up DRM GEM object based on the vma->vm_pgoff value when mmap is invoked. The vma->vm_pgoff should be treated as zero here. Hence we can map a part of GEM, but only from its start. See drm_gem_mmap().
BTW, partial mapping is wanted by Intel native context that I work on. The intent of this patch is correct. There are practical uses for partial mappings.
Please correct vma->vm_pgoff in v2.
Signature · 2 lines
Best regards, Dmitry
Quoted 9 lines · Dmitry Osipenko
drivers/gpu/drm/virtio/virtgpu_vram.c static int virtio_gpu_vram_mmap(struct drm_gem_object *obj, · line 66
Quoted 6 lines
ret = io_remap_pfn_range(vma, vma->vm_start,drivers/gpu/drm/virtio/virtgpu_vram.c +1
-1- vram->vram_node.start >>PAGE_SHIFT , + (vram->vram_node.start >>PAGE_SHIFT ) + vma->vm_pgoff, vm_size, vma->vm_page_prot);return ret; }The
vma->vm_pgoffis fake in DRM, it's used for looking up DRM GEM object based on thevma->vm_pgoffvalue when mmap is invoked.
If my understanding is correct, vm_pgoff gets "unfaked" by
https://elixir.bootlin.com/linux/v6.12.6/source/drivers/gpu/drm/virtio/virtgpu_vram.c#L48
Quoted 2 lines
vma->vm_pgoffshould be treated as zero here. Hence we can map a part of GEM, but only from its start. Seedrm_gem_mmap().
I've had a "v0" (not on ml) of this patch that always treated vma->vm_pgoff as zero. This broke when anything tried to mmap with a non-zero offset. Adding vm_pgoff made it work correctly.
Quoted 1 line
Please correct
vma->vm_pgoffin v2.
I need apps to be able to mmap with a non-zero offset for my usecase. While the correct value may be something else other than what is in the current patch, 0 is definitely incorrect for at least some workloads.
Dmitry Osipenko
maintainerQuoted 15 lines · Sasha Finkelstein
Quoted 9 lines · Dmitry Osipenko
drivers/gpu/drm/virtio/virtgpu_vram.c static int virtio_gpu_vram_mmap(struct drm_gem_object *obj, · line 66
Quoted 6 lines
ret = io_remap_pfn_range(vma, vma->vm_start,drivers/gpu/drm/virtio/virtgpu_vram.c +1
-1- vram->vram_node.start >>PAGE_SHIFT , + (vram->vram_node.start >>PAGE_SHIFT ) + vma->vm_pgoff, vm_size, vma->vm_page_prot);return ret; }The
vma->vm_pgoffis fake in DRM, it's used for looking up DRM GEM object based on thevma->vm_pgoffvalue when mmap is invoked.If my understanding is correct,
vm_pgoffgets "unfaked" by
https://elixir.bootlin.com/linux/v6.12.6/source/drivers/gpu/drm/virtio/virtgpu_vram.c#L48
Quoted 6 lines
Quoted 2 lines
vma->vm_pgoffshould be treated as zero here. Hence we can map a part of GEM, but only from its start. Seedrm_gem_mmap().I've had a "v0" (not on ml) of this patch that always treated
vma->vm_pgoffas zero. This broke when anything tried to mmap with a non-zero offset. Addingvm_pgoffmade it work correctly.
I've tested this patch. Partial mapping with a non-zero offset doesn't work because drm_gem_mmap() rejects it. I'd want to see your sample code that performs mmaping, maybe I'm missing something.
Quoted 5 lines
Quoted 1 line
Please correct
vma->vm_pgoffin v2.I need apps to be able to mmap with a non-zero offset for my usecase. While the correct value may be something else other than what is in the current patch, 0 is definitely incorrect for at least some workloads.
drm_gem_mmap() uses drm_vma_offset_exact_lookup_locked() that doesn't allow vma->vm_pgoff != node.start. AFAICT, no one driver supports mapping with a non-zero offset, perhaps for a reason that I don't know about.
Signature · 2 lines
Best regards, Dmitry
Dmitry Osipenko
maintainerQuoted 29 lines · Dmitry Osipenko
Quoted 15 lines · Sasha Finkelstein
Quoted 9 lines · Dmitry Osipenko
drivers/gpu/drm/virtio/virtgpu_vram.c static int virtio_gpu_vram_mmap(struct drm_gem_object *obj, · line 66
Quoted 6 lines
ret = io_remap_pfn_range(vma, vma->vm_start,drivers/gpu/drm/virtio/virtgpu_vram.c +1
-1- vram->vram_node.start >>PAGE_SHIFT , + (vram->vram_node.start >>PAGE_SHIFT ) + vma->vm_pgoff, vm_size, vma->vm_page_prot);return ret; }The
vma->vm_pgoffis fake in DRM, it's used for looking up DRM GEM object based on thevma->vm_pgoffvalue when mmap is invoked.If my understanding is correct,
vm_pgoffgets "unfaked" by
https://elixir.bootlin.com/linux/v6.12.6/source/drivers/gpu/drm/virtio/virtgpu_vram.c#L48Quoted 6 lines
Quoted 2 lines
vma->vm_pgoffshould be treated as zero here. Hence we can map a part of GEM, but only from its start. Seedrm_gem_mmap().I've had a "v0" (not on ml) of this patch that always treated
vma->vm_pgoffas zero. This broke when anything tried to mmap with a non-zero offset. Addingvm_pgoffmade it work correctly.I've tested this patch. Partial mapping with a non-zero offset doesn't work because
drm_gem_mmap()rejects it. I'd want to see your sample code that performs mmaping, maybe I'm missing something.Quoted 2 lines
Quoted 1 line
Please correct
vma->vm_pgoffin v2.
Quoted 8 lines
Quoted 3 lines
I need apps to be able to mmap with a non-zero offset for my usecase. While the correct value may be something else other than what is in the current patch, 0 is definitely incorrect for at least some workloads.
drm_gem_mmap()usesdrm_vma_offset_exact_lookup_locked()that doesn't allowvma->vm_pgoff!= node.start. AFAICT, no one driver supports mapping with a non-zero offset, perhaps for a reason that I don't know about.
See now that a non-zero mapping of a dmabuf might work. Will test it.
Signature · 2 lines
Best regards, Dmitry
Quoted 3 lines · Dmitry Osipenko
I've tested this patch. Partial mapping with a non-zero offset doesn't work because
drm_gem_mmap()rejects it. I'd want to see your sample code that performs mmaping, maybe I'm missing something.
The entire setup is a huge pile of moving parts, but here is the relevant function that creates a fd from the cross domain info (this is a part of the vm agent)
https://github.com/WhatAmISupposedToPutHere/krun/blob/pipewire/crates/muvm/src/guest/bridge/common.rs#L745
After the fd is created, it is passed to a pipewire client running inside the vm,
which in turn does a simple mmap() (with a non-zero offset) on the relevant fd. Not sure where exactly pipewire's code does that, the syscall was observed via stracing the relevant process.
Dmitry Osipenko
maintainerQuoted 34 lines · Dmitry Osipenko
Quoted 33 lines · Dmitry Osipenko
Quoted 15 lines · Sasha Finkelstein
Quoted 9 lines · Dmitry Osipenko
drivers/gpu/drm/virtio/virtgpu_vram.c static int virtio_gpu_vram_mmap(struct drm_gem_object *obj, · line 66
Quoted 6 lines
ret = io_remap_pfn_range(vma, vma->vm_start,drivers/gpu/drm/virtio/virtgpu_vram.c +1
-1- vram->vram_node.start >>PAGE_SHIFT , + (vram->vram_node.start >>PAGE_SHIFT ) + vma->vm_pgoff, vm_size, vma->vm_page_prot);return ret; }The
vma->vm_pgoffis fake in DRM, it's used for looking up DRM GEM object based on thevma->vm_pgoffvalue when mmap is invoked.If my understanding is correct,
vm_pgoffgets "unfaked" by
https://elixir.bootlin.com/linux/v6.12.6/source/drivers/gpu/drm/virtio/virtgpu_vram.c#L48Quoted 6 lines
Quoted 2 lines
vma->vm_pgoffshould be treated as zero here. Hence we can map a part of GEM, but only from its start. Seedrm_gem_mmap().I've had a "v0" (not on ml) of this patch that always treated
vma->vm_pgoffas zero. This broke when anything tried to mmap with a non-zero offset. Addingvm_pgoffmade it work correctly.I've tested this patch. Partial mapping with a non-zero offset doesn't work because
drm_gem_mmap()rejects it. I'd want to see your sample code that performs mmaping, maybe I'm missing something.Quoted 5 lines
Quoted 1 line
Please correct
vma->vm_pgoffin v2.I need apps to be able to mmap with a non-zero offset for my usecase. While the correct value may be something else other than what is in the current patch, 0 is definitely incorrect for at least some workloads.
Quoted 6 lines
Quoted 4 lines
drm_gem_mmap()usesdrm_vma_offset_exact_lookup_locked()that doesn't allowvma->vm_pgoff!= node.start. AFAICT, no one driver supports mapping with a non-zero offset, perhaps for a reason that I don't know about.See now that a non-zero mapping of a dmabuf might work. Will test it.
Works for dmabuf
drivers/gpu/drm/virtio/virtgpu_vram.c static int virtio_gpu_vram_mmap(struct drm_gem_object *obj, · line 60
Quoted 5 lines
drivers/gpu/drm/virtio/virtgpu_vram.c +1
-2-/* Partial mappings of GEM buffers don't happen much in practice. */ -if (vm_size != vram->vram_node.size) +if (vm_size > vram->vram_node.size)return -EINVAL ;
This check should include the vm_pgoff, like that:
if (check_add_overflow(vma->vm_pgoff << PAGE_SHIFT, vm_size, &vm_end))
return -EINVAL;
if (vm_end > vram->vram_node.size)
return -EINVAL;
The size and offset are actually validated before this code is reached, but doesn't hurt to keep the check around.
I corrected the check and applied patch to misc-next, thanks!
Signature · 2 lines
Best regards, Dmitry