Skip to contentSkip to conversation

drm/virtio: Support partial maps of GEM objects

PATCH In review Sasha Finkelstein 2 participants 7 messages Jan 9, 2025 – Jan 24, 2025

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/virtio: Support partial maps of GEM objects

Those are useful to implement coherent cross-vm mmap.

---
 drivers/gpu/drm/virtio/virtgpu_vram.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/gpu/drm/virtio/virtgpu_vram.c +2 -3
index 25df81c027837c248a746e41856b5aa7e216b8d5..64e2c6dbdd678ac4c0da89fdd4c9dbf937c2c335 100644
@@ -56,12 +56,11 @@ static int virtio_gpu_vram_mmap(struct drm_gem_object *obj,
 	else if (vram->map_info == VIRTIO_GPU_MAP_CACHE_UNCACHED)
 		vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
 
-	/* Partial mappings of GEM buffers don't happen much in practice. */
-	if (vm_size != vram->vram_node.size)
+	if (vm_size > vram->vram_node.size)
 		return -EINVAL;
 
 	ret = io_remap_pfn_range(vma, vma->vm_start,
-				 vram->vram_node.start >> PAGE_SHIFT,
+				 (vram->vram_node.start >> PAGE_SHIFT) + vma->vm_pgoff,
 				 vm_size, vma->vm_page_prot);
 	return ret;
 }

---
base-commit: 643e2e259c2b25a2af0ae4c23c6e16586d9fd19c
change-id: 20250109-virtgpu-gem-partial-map-335ec40656d1

maintainer
#1 lore raw

Hi,

Quoted 18 lines · On 1/10/25 00:45, Sasha Finkelstein via B4 Relay wrote:

From: Sasha Finkelstein <[email protected]>

Those are useful to implement coherent cross-vm mmap.

---
 drivers/gpu/drm/virtio/virtgpu_vram.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/gpu/drm/virtio/virtgpu_vram.c +0 -1
index 25df81c027837c248a746e41856b5aa7e216b8d5..64e2c6dbdd678ac4c0da89fdd4c9dbf937c2c335 100644
@@ -56,12 +56,11 @@ static int virtio_gpu_vram_mmap(struct drm_gem_object *obj,
 	else if (vram->map_info == VIRTIO_GPU_MAP_CACHE_UNCACHED)
 		vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
 
-	/* Partial mappings of GEM buffers don't happen much in practice. */

drivers/gpu/drm/virtio/virtgpu_vram.c static int virtio_gpu_vram_mmap(struct drm_gem_object *obj, · line 66

Quoted 10 lines
drivers/gpu/drm/virtio/virtgpu_vram.c +2 -2
-	if (vm_size != vram->vram_node.size)
+	if (vm_size > vram->vram_node.size)
 		return -EINVAL;
 
 	ret = io_remap_pfn_range(vma, vma->vm_start,
-				 vram->vram_node.start >> PAGE_SHIFT,
+				 (vram->vram_node.start >> PAGE_SHIFT) + vma->vm_pgoff,
 				 vm_size, vma->vm_page_prot);
 	return ret;
 }

The vma->vm_pgoff is fake in DRM, it's used for looking up DRM GEM object based on the vma->vm_pgoff value when mmap is invoked. The vma->vm_pgoff should be treated as zero here. Hence we can map a part of GEM, but only from its start. See drm_gem_mmap().

BTW, partial mapping is wanted by Intel native context that I work on. The intent of this patch is correct. There are practical uses for partial mappings.

Please correct vma->vm_pgoff in v2.

Signature · 2 lines
Best regards,
Dmitry
#2 lore raw
Quoted 9 lines · Dmitry Osipenko

drivers/gpu/drm/virtio/virtgpu_vram.c static int virtio_gpu_vram_mmap(struct drm_gem_object *obj, · line 66

Quoted 6 lines
      ret = io_remap_pfn_range(vma, vma->vm_start,
drivers/gpu/drm/virtio/virtgpu_vram.c +1 -1
-                              vram->vram_node.start >> PAGE_SHIFT,
+                              (vram->vram_node.start >> PAGE_SHIFT) + vma->vm_pgoff,
                               vm_size, vma->vm_page_prot);
      return ret;
 }

The vma->vm_pgoff is fake in DRM, it's used for looking up DRM GEM object based on the vma->vm_pgoff value when mmap is invoked.

If my understanding is correct, vm_pgoff gets "unfaked" by
https://elixir.bootlin.com/linux/v6.12.6/source/drivers/gpu/drm/virtio/virtgpu_vram.c#L48

Quoted 2 lines

vma->vm_pgoff should be treated as zero here. Hence we can map a part of GEM, but only from its start. See drm_gem_mmap().

I've had a "v0" (not on ml) of this patch that always treated vma->vm_pgoff as zero. This broke when anything tried to mmap with a non-zero offset. Adding vm_pgoff made it work correctly.

Quoted 1 line

Please correct vma->vm_pgoff in v2.

I need apps to be able to mmap with a non-zero offset for my usecase. While the correct value may be something else other than what is in the current patch, 0 is definitely incorrect for at least some workloads.

maintainer
#3 lore raw
Quoted 15 lines · Sasha Finkelstein
Quoted 9 lines · Dmitry Osipenko

drivers/gpu/drm/virtio/virtgpu_vram.c static int virtio_gpu_vram_mmap(struct drm_gem_object *obj, · line 66

Quoted 6 lines
      ret = io_remap_pfn_range(vma, vma->vm_start,
drivers/gpu/drm/virtio/virtgpu_vram.c +1 -1
-                              vram->vram_node.start >> PAGE_SHIFT,
+                              (vram->vram_node.start >> PAGE_SHIFT) + vma->vm_pgoff,
                               vm_size, vma->vm_page_prot);
      return ret;
 }

The vma->vm_pgoff is fake in DRM, it's used for looking up DRM GEM object based on the vma->vm_pgoff value when mmap is invoked.

If my understanding is correct, vm_pgoff gets "unfaked" by
https://elixir.bootlin.com/linux/v6.12.6/source/drivers/gpu/drm/virtio/virtgpu_vram.c#L48

Quoted 6 lines
Quoted 2 lines

vma->vm_pgoff should be treated as zero here. Hence we can map a part of GEM, but only from its start. See drm_gem_mmap().

I've had a "v0" (not on ml) of this patch that always treated vma->vm_pgoff as zero. This broke when anything tried to mmap with a non-zero offset. Adding vm_pgoff made it work correctly.

I've tested this patch. Partial mapping with a non-zero offset doesn't work because drm_gem_mmap() rejects it. I'd want to see your sample code that performs mmaping, maybe I'm missing something.

Quoted 5 lines
Quoted 1 line

Please correct vma->vm_pgoff in v2.

I need apps to be able to mmap with a non-zero offset for my usecase. While the correct value may be something else other than what is in the current patch, 0 is definitely incorrect for at least some workloads.

drm_gem_mmap() uses drm_vma_offset_exact_lookup_locked() that doesn't allow vma->vm_pgoff != node.start. AFAICT, no one driver supports mapping with a non-zero offset, perhaps for a reason that I don't know about.

Signature · 2 lines
Best regards,
Dmitry
maintainer
#4 lore raw
Quoted 29 lines · Dmitry Osipenko
Quoted 15 lines · Sasha Finkelstein
Quoted 9 lines · Dmitry Osipenko

drivers/gpu/drm/virtio/virtgpu_vram.c static int virtio_gpu_vram_mmap(struct drm_gem_object *obj, · line 66

Quoted 6 lines
      ret = io_remap_pfn_range(vma, vma->vm_start,
drivers/gpu/drm/virtio/virtgpu_vram.c +1 -1
-                              vram->vram_node.start >> PAGE_SHIFT,
+                              (vram->vram_node.start >> PAGE_SHIFT) + vma->vm_pgoff,
                               vm_size, vma->vm_page_prot);
      return ret;
 }

The vma->vm_pgoff is fake in DRM, it's used for looking up DRM GEM object based on the vma->vm_pgoff value when mmap is invoked.

If my understanding is correct, vm_pgoff gets "unfaked" by
https://elixir.bootlin.com/linux/v6.12.6/source/drivers/gpu/drm/virtio/virtgpu_vram.c#L48

Quoted 6 lines
Quoted 2 lines

vma->vm_pgoff should be treated as zero here. Hence we can map a part of GEM, but only from its start. See drm_gem_mmap().

I've had a "v0" (not on ml) of this patch that always treated vma->vm_pgoff as zero. This broke when anything tried to mmap with a non-zero offset. Adding vm_pgoff made it work correctly.

I've tested this patch. Partial mapping with a non-zero offset doesn't work because drm_gem_mmap() rejects it. I'd want to see your sample code that performs mmaping, maybe I'm missing something.

Quoted 2 lines
Quoted 1 line

Please correct vma->vm_pgoff in v2.

Quoted 8 lines
Quoted 3 lines

I need apps to be able to mmap with a non-zero offset for my usecase. While the correct value may be something else other than what is in the current patch, 0 is definitely incorrect for at least some workloads.

drm_gem_mmap() uses drm_vma_offset_exact_lookup_locked() that doesn't allow vma->vm_pgoff != node.start. AFAICT, no one driver supports mapping with a non-zero offset, perhaps for a reason that I don't know about.

See now that a non-zero mapping of a dmabuf might work. Will test it.

Signature · 2 lines
Best regards,
Dmitry
maintainer
#5 lore raw
Quoted 34 lines · Dmitry Osipenko
Quoted 33 lines · Dmitry Osipenko
Quoted 15 lines · Sasha Finkelstein
Quoted 9 lines · Dmitry Osipenko

drivers/gpu/drm/virtio/virtgpu_vram.c static int virtio_gpu_vram_mmap(struct drm_gem_object *obj, · line 66

Quoted 6 lines
      ret = io_remap_pfn_range(vma, vma->vm_start,
drivers/gpu/drm/virtio/virtgpu_vram.c +1 -1
-                              vram->vram_node.start >> PAGE_SHIFT,
+                              (vram->vram_node.start >> PAGE_SHIFT) + vma->vm_pgoff,
                               vm_size, vma->vm_page_prot);
      return ret;
 }

The vma->vm_pgoff is fake in DRM, it's used for looking up DRM GEM object based on the vma->vm_pgoff value when mmap is invoked.

If my understanding is correct, vm_pgoff gets "unfaked" by
https://elixir.bootlin.com/linux/v6.12.6/source/drivers/gpu/drm/virtio/virtgpu_vram.c#L48

Quoted 6 lines
Quoted 2 lines

vma->vm_pgoff should be treated as zero here. Hence we can map a part of GEM, but only from its start. See drm_gem_mmap().

I've had a "v0" (not on ml) of this patch that always treated vma->vm_pgoff as zero. This broke when anything tried to mmap with a non-zero offset. Adding vm_pgoff made it work correctly.

I've tested this patch. Partial mapping with a non-zero offset doesn't work because drm_gem_mmap() rejects it. I'd want to see your sample code that performs mmaping, maybe I'm missing something.

Quoted 5 lines
Quoted 1 line

Please correct vma->vm_pgoff in v2.

I need apps to be able to mmap with a non-zero offset for my usecase. While the correct value may be something else other than what is in the current patch, 0 is definitely incorrect for at least some workloads.

Quoted 6 lines
Quoted 4 lines

drm_gem_mmap() uses drm_vma_offset_exact_lookup_locked() that doesn't allow vma->vm_pgoff != node.start. AFAICT, no one driver supports mapping with a non-zero offset, perhaps for a reason that I don't know about.

See now that a non-zero mapping of a dmabuf might work. Will test it.

Works for dmabuf

drivers/gpu/drm/virtio/virtgpu_vram.c static int virtio_gpu_vram_mmap(struct drm_gem_object *obj, · line 60

Quoted 5 lines
drivers/gpu/drm/virtio/virtgpu_vram.c +1 -2
-	/* Partial mappings of GEM buffers don't happen much in practice. */
-	if (vm_size != vram->vram_node.size)
+	if (vm_size > vram->vram_node.size)
 		return -EINVAL;

This check should include the vm_pgoff, like that:

if (check_add_overflow(vma->vm_pgoff << PAGE_SHIFT, vm_size, &vm_end))

	return -EINVAL;

if (vm_end > vram->vram_node.size)

	return -EINVAL;

The size and offset are actually validated before this code is reached, but doesn't hurt to keep the check around.

I corrected the check and applied patch to misc-next, thanks!

Signature · 2 lines
Best regards,
Dmitry
#6 lore raw
In reply to Dmitry Osipenko
Quoted 3 lines · Dmitry Osipenko

I've tested this patch. Partial mapping with a non-zero offset doesn't work because drm_gem_mmap() rejects it. I'd want to see your sample code that performs mmaping, maybe I'm missing something.

The entire setup is a huge pile of moving parts, but here is the relevant function that creates a fd from the cross domain info (this is a part of the vm agent)
https://github.com/WhatAmISupposedToPutHere/krun/blob/pipewire/crates/muvm/src/guest/bridge/common.rs#L745
After the fd is created, it is passed to a pipewire client running inside the vm,
which in turn does a simple mmap() (with a non-zero offset) on the relevant fd. Not sure where exactly pipewire's code does that, the syscall was observed via stracing the relevant process.