drm/gem: Fix mmap fake offset handling for drm_gem_object_funcs.mmap
Subsystems (3)
PATCH v2 1/1 drm/gem: Fix mmap fake offset handling for drm_gem_object_funcs.mmap
Commit c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs") introduced a GEM object mmap() hook which is expected to subtract the fake offset from vm_pgoff. However, for mmap() on dmabufs, there is not a fake offset.
To fix this, let's always call mmap() object callback with an offset of 0, and leave it up to drm_gem_mmap_obj() to remove the fake offset.
TTM still needs the fake offset, so we have to add it back until that's fixed.
---
v2:
- Move subtracting the fake offset out of mmap() obj callbacks.
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
Rob
drivers/gpu/drm/drm_gem.c | 3 +++ drivers/gpu/drm/drm_gem_shmem_helper.c | 3 --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 7 +++++++ include/drm/drm_gem.h | 4 +++- 4 files changed, 13 insertions(+), 4 deletions(-)
drivers/gpu/drm/drm_gem.c +3 -0
@@ -1106,6 +1106,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,return -EINVAL ;if (obj->funcs && obj->funcs->mmap) { +/* Remove the fake offset */ + vma->vm_pgoff -=drm_vma_node_start (&obj->vma_node); + ret = obj->funcs->mmap (obj, vma);if (ret)return ret;
drivers/gpu/drm/drm_gem_shmem_helper.c +0 -3
@@ -542,9 +542,6 @@ int drm_gem_shmem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) vma->vm_page_prot =pgprot_decrypted (vma->vm_page_prot); vma->vm_ops = &drm_gem_shmem_vm_ops; -/* Remove the fake offset */ - vma->vm_pgoff -=drm_vma_node_start (&shmem->base.vma_node); -return 0 ; }EXPORT_SYMBOL_GPL (drm_gem_shmem_mmap);
drivers/gpu/drm/ttm/ttm_bo_vm.c +7 -0
@@ -482,6 +482,13 @@ EXPORT_SYMBOL(ttm_bo_mmap);int ttm_bo_mmap_obj (struct vm_area_struct *vma,struct ttm_buffer_object *bo) {ttm_bo_get (bo); + +/* +* FIXME: &drm_gem_object_funcs.mmap is called with the fake offset +* removed. Add it back here until the rest of TTM works without it. +*/ + vma->vm_pgoff +=drm_vma_node_start (&bo->base.vma_node); +ttm_bo_mmap_vma_setup (bo, vma);return 0 ; }
include/drm/drm_gem.h +3 -1
@@ -159,7 +159,9 @@ struct drm_gem_object_funcs {* * The callback is used by by both drm_gem_mmap_obj() and * drm_gem_prime_mmap(). When @mmap is present @vm_ops is not -* used, the @mmap callback must set vma->vm_ops instead. +* used, the @mmap callback must set vma->vm_ops instead. The @mmap +* callback is always called with a 0 offset. The caller will remove +* the fake offset as necessary. * */ int (*mmap)(struct drm_gem_object *obj,struct vm_area_struct *vma);
Signature · 6 lines
2.20.1 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 10 lines · Rob Herring
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixing ttm looks easy, there are not many calls to drm_vma_node_start() the ttm code. Can look into this when I'm back from kvm forum @ lyon.
drivers/gpu/drm/ttm/ttm_bo_vm.c EXPORT_SYMBOL(ttm_bo_mmap); · line 494
Quoted 13 lines
int
ttm_bo_mmap_obj(struct vm_area_struct *vma, struct ttm_buffer_object *bo){drivers/gpu/drm/ttm/ttm_bo_vm.c +7
-0ttm_bo_get (bo); + +/* +* FIXME: &drm_gem_object_funcs.mmap is called with the fake offset +* removed. Add it back here until the rest of TTM works without it. +*/ + vma->vm_pgoff +=drm_vma_node_start (&bo->base.vma_node); +ttm_bo_mmap_vma_setup (bo, vma);return 0 ; }
Yes, that should keep ttm happy for now. Survived a quick smoke test with qemu and bochs.
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 81 lines · Rob Herring
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
Rob
drivers/gpu/drm/drm_gem.c | 3 +++ drivers/gpu/drm/drm_gem_shmem_helper.c | 3 --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 7 +++++++ include/drm/drm_gem.h | 4 +++- 4 files changed, 13 insertions(+), 4 deletions(-)drivers/gpu/drm/drm_gem.c +3
-0@@ -1106,6 +1106,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,return -EINVAL ;if (obj->funcs && obj->funcs->mmap) { +/* Remove the fake offset */ + vma->vm_pgoff -=drm_vma_node_start (&obj->vma_node); + ret = obj->funcs->mmap (obj, vma);if (ret)return ret;drivers/gpu/drm/drm_gem_shmem_helper.c +0
-3@@ -542,9 +542,6 @@ int drm_gem_shmem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) vma->vm_page_prot =pgprot_decrypted (vma->vm_page_prot); vma->vm_ops = &drm_gem_shmem_vm_ops; -/* Remove the fake offset */ - vma->vm_pgoff -=drm_vma_node_start (&shmem->base.vma_node); -return 0 ; }EXPORT_SYMBOL_GPL (drm_gem_shmem_mmap);drivers/gpu/drm/ttm/ttm_bo_vm.c +7
-0@@ -482,6 +482,13 @@ EXPORT_SYMBOL(ttm_bo_mmap);int ttm_bo_mmap_obj (struct vm_area_struct *vma,struct ttm_buffer_object *bo) {ttm_bo_get (bo); + +/* +* FIXME: &drm_gem_object_funcs.mmap is called with the fake offset +* removed. Add it back here until the rest of TTM works without it. +*/ + vma->vm_pgoff +=drm_vma_node_start (&bo->base.vma_node); +ttm_bo_mmap_vma_setup (bo, vma);return 0 ; }include/drm/drm_gem.h +0
-0
include/drm/drm_gem.h struct drm_gem_object_funcs { · line 165
Quoted 8 lines
include/drm/drm_gem.h +3
-1@@ -159,7 +159,9 @@ struct drm_gem_object_funcs {* * The callback is used by by both drm_gem_mmap_obj() and * drm_gem_prime_mmap(). When @mmap is present @vm_ops is not -* used, the @mmap callback must set vma->vm_ops instead. +* used, the @mmap callback must set vma->vm_ops instead. The @mmap +* callback is always called with a 0 offset. The caller will remove +* the fake offset as necessary. *
Maybe remove this empty comment line here while at it. With that
I think I'll follow up with a patch to annotate drm_gem_mmap_obj as deprecated and that instead this here should be used. -Daniel
Quoted 5 lines
include/drm/drm_gem.h +0
-0*/ int (*mmap)(struct drm_gem_object *obj,struct vm_area_struct *vma);Signature · 1 line
2.20.1
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 20 lines · Daniel Vetter
Quoted 19 lines · Rob Herring
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.
Quoted 2 lines
Quoted 2 lines
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
So unfortunately I'm already having regrets on this. We might even have broken some of the ttm drivers here.
Trouble is, if you need to shoot down userspace ptes of a bo (because it's getting moved or whatever), then we do that with unmap_mapping_range. Which means each bo needs to be mapping with a unique (offset, adress_space), or it won't work. By remapping all the bo to 0 we've broken this. We've also had this broken this for a while for the simplistic dma-buf mmap, since without any further action we'll reuse the address space of the dma-buf inode, not of the drm_device.
Strangely both etnaviv and msm have a comment to that effect - grep for unmap_mapping_range. But neither actually uses it.
Not exactly sure what's the best course of action here now.
Also adding Thomas Zimmermann, who's worked on all the vram helpers. -Daniel
Quoted 87 lines
Quoted 60 lines
Rob
drivers/gpu/drm/drm_gem.c | 3 +++ drivers/gpu/drm/drm_gem_shmem_helper.c | 3 --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 7 +++++++ include/drm/drm_gem.h | 4 +++- 4 files changed, 13 insertions(+), 4 deletions(-)drivers/gpu/drm/drm_gem.c +3
-0@@ -1106,6 +1106,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,return -EINVAL ;if (obj->funcs && obj->funcs->mmap) { +/* Remove the fake offset */ + vma->vm_pgoff -=drm_vma_node_start (&obj->vma_node); + ret = obj->funcs->mmap (obj, vma);if (ret)return ret;drivers/gpu/drm/drm_gem_shmem_helper.c +0
-3@@ -542,9 +542,6 @@ int drm_gem_shmem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) vma->vm_page_prot =pgprot_decrypted (vma->vm_page_prot); vma->vm_ops = &drm_gem_shmem_vm_ops; -/* Remove the fake offset */ - vma->vm_pgoff -=drm_vma_node_start (&shmem->base.vma_node); -return 0 ; }EXPORT_SYMBOL_GPL (drm_gem_shmem_mmap);drivers/gpu/drm/ttm/ttm_bo_vm.c +7
-0@@ -482,6 +482,13 @@ EXPORT_SYMBOL(ttm_bo_mmap);int ttm_bo_mmap_obj (struct vm_area_struct *vma,struct ttm_buffer_object *bo) {ttm_bo_get (bo); + +/* +* FIXME: &drm_gem_object_funcs.mmap is called with the fake offset +* removed. Add it back here until the rest of TTM works without it. +*/ + vma->vm_pgoff +=drm_vma_node_start (&bo->base.vma_node); +ttm_bo_mmap_vma_setup (bo, vma);return 0 ; }include/drm/drm_gem.h +0
-0include/drm/drm_gem.h struct drm_gem_object_funcs { · line 165
Quoted 8 lines
include/drm/drm_gem.h +3
-1@@ -159,7 +159,9 @@ struct drm_gem_object_funcs {* * The callback is used by by both drm_gem_mmap_obj() and * drm_gem_prime_mmap(). When @mmap is present @vm_ops is not -* used, the @mmap callback must set vma->vm_ops instead. +* used, the @mmap callback must set vma->vm_ops instead. The @mmap +* callback is always called with a 0 offset. The caller will remove +* the fake offset as necessary. * Maybe remove this empty comment line here while at it. With that
Reviewed-by Daniel Vetter <[email protected]>I think I'll follow up with a patch to annotate
drm_gem_mmap_objas deprecated and that instead this here should be used. -DanielQuoted 5 lines
include/drm/drm_gem.h +0
-0*/ int (*mmap)(struct drm_gem_object *obj,struct vm_area_struct *vma);Signature · 1 line
2.20.1Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 35 lines · Daniel Vetter
Quoted 20 lines · Daniel Vetter
Quoted 19 lines · Rob Herring
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.Quoted 2 lines
Quoted 2 lines
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
So unfortunately I'm already having regrets on this. We might even have broken some of the ttm drivers here.
Trouble is, if you need to shoot down userspace ptes of a bo (because it's getting moved or whatever), then we do that with
unmap_mapping_range. Which means each bo needs to be mapping with a unique (offset,adress_space), or it won't work. By remapping all the bo to 0 we've broken this. We've also had this broken this for a while for the simplistic dma-buf mmap, since without any further action we'll reuse the address space of the dma-buf inode, not of thedrm_device.
Quoted 6 lines
Strangely both etnaviv and msm have a comment to that effect - grep for
unmap_mapping_range. But neither actually uses it.Not exactly sure what's the best course of action here now.
Also adding Thomas Zimmermann, who's worked on all the vram helpers.
Correction, I missed the unmap_mapping_range in the vma node manager header, so didn't spot the drivers using drm_vma_node_unmap. We did break all the ttm stuff :-/
Plus panfrost, which is using drm_gem_shmem_purge_locked. -Daniel
Quoted 94 lines
-Daniel
Quoted 87 lines
Quoted 60 lines
Rob
drivers/gpu/drm/drm_gem.c | 3 +++ drivers/gpu/drm/drm_gem_shmem_helper.c | 3 --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 7 +++++++ include/drm/drm_gem.h | 4 +++- 4 files changed, 13 insertions(+), 4 deletions(-)drivers/gpu/drm/drm_gem.c +3
-0@@ -1106,6 +1106,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,return -EINVAL ;if (obj->funcs && obj->funcs->mmap) { +/* Remove the fake offset */ + vma->vm_pgoff -=drm_vma_node_start (&obj->vma_node); + ret = obj->funcs->mmap (obj, vma);if (ret)return ret;drivers/gpu/drm/drm_gem_shmem_helper.c +0
-3@@ -542,9 +542,6 @@ int drm_gem_shmem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) vma->vm_page_prot =pgprot_decrypted (vma->vm_page_prot); vma->vm_ops = &drm_gem_shmem_vm_ops; -/* Remove the fake offset */ - vma->vm_pgoff -=drm_vma_node_start (&shmem->base.vma_node); -return 0 ; }EXPORT_SYMBOL_GPL (drm_gem_shmem_mmap);drivers/gpu/drm/ttm/ttm_bo_vm.c +7
-0@@ -482,6 +482,13 @@ EXPORT_SYMBOL(ttm_bo_mmap);int ttm_bo_mmap_obj (struct vm_area_struct *vma,struct ttm_buffer_object *bo) {ttm_bo_get (bo); + +/* +* FIXME: &drm_gem_object_funcs.mmap is called with the fake offset +* removed. Add it back here until the rest of TTM works without it. +*/ + vma->vm_pgoff +=drm_vma_node_start (&bo->base.vma_node); +ttm_bo_mmap_vma_setup (bo, vma);return 0 ; }include/drm/drm_gem.h +0
-0include/drm/drm_gem.h struct drm_gem_object_funcs { · line 165
Quoted 8 lines
include/drm/drm_gem.h +3
-1@@ -159,7 +159,9 @@ struct drm_gem_object_funcs {* * The callback is used by by both drm_gem_mmap_obj() and * drm_gem_prime_mmap(). When @mmap is present @vm_ops is not -* used, the @mmap callback must set vma->vm_ops instead. +* used, the @mmap callback must set vma->vm_ops instead. The @mmap +* callback is always called with a 0 offset. The caller will remove +* the fake offset as necessary. * Maybe remove this empty comment line here while at it. With that
Reviewed-by Daniel Vetter <[email protected]>I think I'll follow up with a patch to annotate
drm_gem_mmap_objas deprecated and that instead this here should be used. -DanielQuoted 5 lines
include/drm/drm_gem.h +0
-0*/ int (*mmap)(struct drm_gem_object *obj,struct vm_area_struct *vma);Signature · 1 line
2.20.1Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.chSignature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 39 lines · Daniel Vetter
Quoted 38 lines · Daniel Vetter
Quoted 20 lines · Daniel Vetter
Quoted 19 lines · Rob Herring
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.Quoted 2 lines
Quoted 2 lines
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
So unfortunately I'm already having regrets on this. We might even have broken some of the ttm drivers here.
Trouble is, if you need to shoot down userspace ptes of a bo (because it's getting moved or whatever), then we do that with
unmap_mapping_range. Which means each bo needs to be mapping with a unique (offset,adress_space), or it won't work. By remapping all the bo to 0 we've broken this. We've also had this broken this for a while for the simplistic dma-buf mmap, since without any further action we'll reuse the address space of the dma-buf inode, not of thedrm_device.Strangely both etnaviv and msm have a comment to that effect - grep for
unmap_mapping_range. But neither actually uses it.
Quoted 7 lines
Quoted 3 lines
Not exactly sure what's the best course of action here now.
Also adding Thomas Zimmermann, who's worked on all the vram helpers.
Correction, I missed the
unmap_mapping_rangein the vma node manager header, so didn't spot the drivers usingdrm_vma_node_unmap. We did break all the ttm stuff :-/
ttm still uses the offset, now added in ttm_bo_mmap_obj(). So, for normal mmap behavior did not change I think. The simplistic dma-buf mmap did change, it now uses the offset because it goes through ttm_bo_mmap_obj() too.
Not fully sure which address space is used for dma-bufs though. As far I can see neither the old nor the new dma-buf mmap code path touch vma->vm_file (unless the driver does explicitly care, like msm does in msm_gem_mmap_obj).
Quoted 1 line
Plus panfrost, which is using
drm_gem_shmem_purge_locked.
Hmm, looking at drm_gem_shmem_purge_locked I'm wondering why it uses a mix of dev->anon_inode->i_mapping and file_inode(obj->filp)->i_mapping.
Also shmem helpers used a zero vm_pgoff before, only difference is the change is applied in drm_gem_mmap_obj() now instead of somewhere in the shmem helper code.
slightly confused,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 60 lines · Gerd Hoffmann
Quoted 39 lines · Daniel Vetter
Quoted 38 lines · Daniel Vetter
Quoted 20 lines · Daniel Vetter
Quoted 19 lines · Rob Herring
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.Quoted 2 lines
Quoted 2 lines
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
So unfortunately I'm already having regrets on this. We might even have broken some of the ttm drivers here.
Trouble is, if you need to shoot down userspace ptes of a bo (because it's getting moved or whatever), then we do that with
unmap_mapping_range. Which means each bo needs to be mapping with a unique (offset,adress_space), or it won't work. By remapping all the bo to 0 we've broken this. We've also had this broken this for a while for the simplistic dma-buf mmap, since without any further action we'll reuse the address space of the dma-buf inode, not of thedrm_device.Strangely both etnaviv and msm have a comment to that effect - grep for
unmap_mapping_range. But neither actually uses it.Quoted 7 lines
Quoted 3 lines
Not exactly sure what's the best course of action here now.
Also adding Thomas Zimmermann, who's worked on all the vram helpers.
Correction, I missed the
unmap_mapping_rangein the vma node manager header, so didn't spot the drivers usingdrm_vma_node_unmap. We did break all the ttm stuff :-/ttm still uses the offset, now added in
ttm_bo_mmap_obj(). So, for normal mmap behavior did not change I think. The simplistic dma-buf mmap did change, it now uses the offset because it goes throughttm_bo_mmap_obj()too.Not fully sure which address space is used for dma-bufs though. As far I can see neither the old nor the new dma-buf mmap code path touch
vma->vm_file(unless the driver does explicitly care, like msm does inmsm_gem_mmap_obj).Quoted 1 line
Plus panfrost, which is using
drm_gem_shmem_purge_locked.
Quoted 8 lines
Hmm, looking at
drm_gem_shmem_purge_lockedI'm wondering why it uses a mix ofdev->anon_inode->i_mappingand file_inode(obj->filp)->i_mapping.Also shmem helpers used a zero
vm_pgoffbefore, only difference is the change is applied indrm_gem_mmap_obj()now instead of somewhere in the shmem helper code.slightly confused,
I think summary is:
- shmem helper pte shotdown is broken for all cases now with obj->funcs->mmap
- ttm/vram-helpers only for dma-buf mmap redirection (because of the wrong f/i_mapping).
Rob, are you looking into this? I guess there's two options:
- Go back to fake offset everywhere, and weep.
- Add a per-bo mapping struct, consistently use that everywhere (probably more work).
If we go with weeping maybe note the 2nd option as a todo item in todo.rst?
-Daniel
Quoted 2 lines
Gerd
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 60 lines · Daniel Vetter
Quoted 58 lines · Gerd Hoffmann
Quoted 39 lines · Daniel Vetter
Quoted 38 lines · Daniel Vetter
Quoted 20 lines · Daniel Vetter
Quoted 19 lines · Rob Herring
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.Quoted 2 lines
Quoted 2 lines
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
So unfortunately I'm already having regrets on this. We might even have broken some of the ttm drivers here.
Trouble is, if you need to shoot down userspace ptes of a bo (because it's getting moved or whatever), then we do that with
unmap_mapping_range. Which means each bo needs to be mapping with a unique (offset,adress_space), or it won't work. By remapping all the bo to 0 we've broken this. We've also had this broken this for a while for the simplistic dma-buf mmap, since without any further action we'll reuse the address space of the dma-buf inode, not of thedrm_device.Strangely both etnaviv and msm have a comment to that effect - grep for
unmap_mapping_range. But neither actually uses it.Quoted 7 lines
Quoted 3 lines
Not exactly sure what's the best course of action here now.
Also adding Thomas Zimmermann, who's worked on all the vram helpers.
Correction, I missed the
unmap_mapping_rangein the vma node manager header, so didn't spot the drivers usingdrm_vma_node_unmap. We did break all the ttm stuff :-/ttm still uses the offset, now added in
ttm_bo_mmap_obj(). So, for normal mmap behavior did not change I think. The simplistic dma-buf mmap did change, it now uses the offset because it goes throughttm_bo_mmap_obj()too.Not fully sure which address space is used for dma-bufs though. As far I can see neither the old nor the new dma-buf mmap code path touch
vma->vm_file(unless the driver does explicitly care, like msm does inmsm_gem_mmap_obj).
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Plus panfrost, which is using
drm_gem_shmem_purge_locked.Hmm, looking at
drm_gem_shmem_purge_lockedI'm wondering why it uses a mix ofdev->anon_inode->i_mappingand file_inode(obj->filp)->i_mapping.
Probably my copy-n-paste from other implementations...
Quoted 5 lines
Quoted 5 lines
Also shmem helpers used a zero
vm_pgoffbefore, only difference is the change is applied indrm_gem_mmap_obj()now instead of somewhere in the shmem helper code.slightly confused,
Me too.
Quoted 3 lines
I think summary is:
- shmem helper pte shotdown is broken for all cases now withobj->funcs->mmap
Does it help that userspace always does a munmap before making pages purgeable?
Quoted 4 lines
- ttm/vram-helpers only for dma-buf mmap redirection (because of the wrong f/i_mapping).
Rob, are you looking into this?
Still trying to understand all this...
Quoted 9 lines
I guess there's two options:
- Go back to fake offset everywhere, and weep.
- Add a per-bo mapping struct, consistently use that everywhere (probably more work).If we go with weeping maybe note the 2nd option as a todo item in
todo.rst?
-Daniel
Quoted 7 lines
Quoted 2 lines
Gerd
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 74 lines · Rob Herring
Quoted 60 lines · Daniel Vetter
Quoted 58 lines · Gerd Hoffmann
Quoted 39 lines · Daniel Vetter
Quoted 38 lines · Daniel Vetter
Quoted 20 lines · Daniel Vetter
Quoted 19 lines · Rob Herring
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.Quoted 2 lines
Quoted 2 lines
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
So unfortunately I'm already having regrets on this. We might even have broken some of the ttm drivers here.
Trouble is, if you need to shoot down userspace ptes of a bo (because it's getting moved or whatever), then we do that with
unmap_mapping_range. Which means each bo needs to be mapping with a unique (offset,adress_space), or it won't work. By remapping all the bo to 0 we've broken this. We've also had this broken this for a while for the simplistic dma-buf mmap, since without any further action we'll reuse the address space of the dma-buf inode, not of thedrm_device.Strangely both etnaviv and msm have a comment to that effect - grep for
unmap_mapping_range. But neither actually uses it.Quoted 7 lines
Quoted 3 lines
Not exactly sure what's the best course of action here now.
Also adding Thomas Zimmermann, who's worked on all the vram helpers.
Correction, I missed the
unmap_mapping_rangein the vma node manager header, so didn't spot the drivers usingdrm_vma_node_unmap. We did break all the ttm stuff :-/ttm still uses the offset, now added in
ttm_bo_mmap_obj(). So, for normal mmap behavior did not change I think. The simplistic dma-buf mmap did change, it now uses the offset because it goes throughttm_bo_mmap_obj()too.Not fully sure which address space is used for dma-bufs though. As far I can see neither the old nor the new dma-buf mmap code path touch
vma->vm_file(unless the driver does explicitly care, like msm does inmsm_gem_mmap_obj).Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Plus panfrost, which is using
drm_gem_shmem_purge_locked.Hmm, looking at
drm_gem_shmem_purge_lockedI'm wondering why it uses a mix ofdev->anon_inode->i_mappingand file_inode(obj->filp)->i_mapping.Probably my copy-n-paste from other implementations...
Quoted 5 lines
Quoted 5 lines
Also shmem helpers used a zero
vm_pgoffbefore, only difference is the change is applied indrm_gem_mmap_obj()now instead of somewhere in the shmem helper code.slightly confused,
Quoted 7 lines
Me too.
Quoted 3 lines
I think summary is:
- shmem helper pte shotdown is broken for all cases now withobj->funcs->mmapDoes it help that userspace always does a munmap before making pages purgeable?
I guess ... but kinda awkward to leave this issue in here, it's really surprising if you call the pte shootdown function, and it doesn't work as advertised.
-Daniel
Quoted 24 lines
Quoted 5 lines
- ttm/vram-helpers only for dma-buf mmap redirection (because of the wrong f/i_mapping).
Rob, are you looking into this?
Still trying to understand all this...
Quoted 16 lines
I guess there's two options:
- Go back to fake offset everywhere, and weep.
- Add a per-bo mapping struct, consistently use that everywhere (probably more work).If we go with weeping maybe note the 2nd option as a todo item in
todo.rst?
-DanielQuoted 2 lines
Gerd
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 63 lines · Daniel Vetter
Quoted 61 lines · Rob Herring
Quoted 60 lines · Daniel Vetter
Quoted 58 lines · Gerd Hoffmann
Quoted 39 lines · Daniel Vetter
Quoted 38 lines · Daniel Vetter
Quoted 20 lines · Daniel Vetter
Quoted 19 lines · Rob Herring
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.Quoted 2 lines
Quoted 2 lines
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
So unfortunately I'm already having regrets on this. We might even have broken some of the ttm drivers here.
Trouble is, if you need to shoot down userspace ptes of a bo (because it's getting moved or whatever), then we do that with
unmap_mapping_range. Which means each bo needs to be mapping with a unique (offset,adress_space), or it won't work. By remapping all the bo to 0 we've broken this. We've also had this broken this for a while for the simplistic dma-buf mmap, since without any further action we'll reuse the address space of the dma-buf inode, not of thedrm_device.Strangely both etnaviv and msm have a comment to that effect - grep for
unmap_mapping_range. But neither actually uses it.Quoted 7 lines
Quoted 3 lines
Not exactly sure what's the best course of action here now.
Also adding Thomas Zimmermann, who's worked on all the vram helpers.
Correction, I missed the
unmap_mapping_rangein the vma node manager header, so didn't spot the drivers usingdrm_vma_node_unmap. We did break all the ttm stuff :-/ttm still uses the offset, now added in
ttm_bo_mmap_obj(). So, for normal mmap behavior did not change I think. The simplistic dma-buf mmap did change, it now uses the offset because it goes throughttm_bo_mmap_obj()too.Not fully sure which address space is used for dma-bufs though. As far I can see neither the old nor the new dma-buf mmap code path touch
vma->vm_file(unless the driver does explicitly care, like msm does inmsm_gem_mmap_obj).
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Plus panfrost, which is using
drm_gem_shmem_purge_locked.Hmm, looking at
drm_gem_shmem_purge_lockedI'm wondering why it uses a mix ofdev->anon_inode->i_mappingand file_inode(obj->filp)->i_mapping.Probably my copy-n-paste from other implementations...
I checked and yes, this is just copy-n-paste from msm. BTW, the code with the unmap_mapping_range comment mentioned above doesn't apply for shmem helpers because it applies to cacheable mappings which are (yet) supported.
Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Quoted 6 lines
Also shmem helpers used a zero
vm_pgoffbefore, only difference is the change is applied indrm_gem_mmap_obj()now instead of somewhere in the shmem helper code.slightly confused,
Me too.
Quoted 3 lines
I think summary is:
- shmem helper pte shotdown is broken for all cases now withobj->funcs->mmap
Quoted 5 lines
Quoted 1 line
Does it help that userspace always does a munmap before making pages purgeable?
I guess ... but kinda awkward to leave this issue in here, it's really surprising if you call the pte shootdown function, and it doesn't work as advertised.
I was mainly wondering how this worked for us and how to hit it not working to test fixing it.
Is there a simple way to check if a BO is mmapped or not? I'd assume so, just don't know the answer off hand. A simple fix would be to either require buffer is not mapped before madvise or skip purging if it is mapped.
Rob
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 96 lines · Rob Herring
Quoted 63 lines · Daniel Vetter
Quoted 61 lines · Rob Herring
Quoted 60 lines · Daniel Vetter
Quoted 58 lines · Gerd Hoffmann
Quoted 39 lines · Daniel Vetter
Quoted 38 lines · Daniel Vetter
Quoted 20 lines · Daniel Vetter
Quoted 19 lines · Rob Herring
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.Quoted 2 lines
Quoted 2 lines
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
So unfortunately I'm already having regrets on this. We might even have broken some of the ttm drivers here.
Trouble is, if you need to shoot down userspace ptes of a bo (because it's getting moved or whatever), then we do that with
unmap_mapping_range. Which means each bo needs to be mapping with a unique (offset,adress_space), or it won't work. By remapping all the bo to 0 we've broken this. We've also had this broken this for a while for the simplistic dma-buf mmap, since without any further action we'll reuse the address space of the dma-buf inode, not of thedrm_device.Strangely both etnaviv and msm have a comment to that effect - grep for
unmap_mapping_range. But neither actually uses it.Quoted 7 lines
Quoted 3 lines
Not exactly sure what's the best course of action here now.
Also adding Thomas Zimmermann, who's worked on all the vram helpers.
Correction, I missed the
unmap_mapping_rangein the vma node manager header, so didn't spot the drivers usingdrm_vma_node_unmap. We did break all the ttm stuff :-/ttm still uses the offset, now added in
ttm_bo_mmap_obj(). So, for normal mmap behavior did not change I think. The simplistic dma-buf mmap did change, it now uses the offset because it goes throughttm_bo_mmap_obj()too.Not fully sure which address space is used for dma-bufs though. As far I can see neither the old nor the new dma-buf mmap code path touch
vma->vm_file(unless the driver does explicitly care, like msm does inmsm_gem_mmap_obj).Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Plus panfrost, which is using
drm_gem_shmem_purge_locked.Hmm, looking at
drm_gem_shmem_purge_lockedI'm wondering why it uses a mix ofdev->anon_inode->i_mappingand file_inode(obj->filp)->i_mapping.Probably my copy-n-paste from other implementations...
I checked and yes, this is just copy-n-paste from msm. BTW, the code with the
unmap_mapping_rangecomment mentioned above doesn't apply for shmem helpers because it applies to cacheable mappings which are (yet) supported.Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Quoted 6 lines
Also shmem helpers used a zero
vm_pgoffbefore, only difference is the change is applied indrm_gem_mmap_obj()now instead of somewhere in the shmem helper code.slightly confused,
Me too.
Quoted 3 lines
I think summary is:
- shmem helper pte shotdown is broken for all cases now withobj->funcs->mmapQuoted 5 lines
Quoted 1 line
Does it help that userspace always does a munmap before making pages purgeable?
I guess ... but kinda awkward to leave this issue in here, it's really surprising if you call the pte shootdown function, and it doesn't work as advertised.
Quoted 7 lines
I was mainly wondering how this worked for us and how to hit it not working to test fixing it.
Is there a simple way to check if a BO is mmapped or not? I'd assume so, just don't know the answer off hand. A simple fix would be to either require buffer is not mapped before madvise or skip purging if it is mapped.
Imo simplest fix is to just go back and readd the fake offset. Ugly, but works at least. That leaves exported dma-bufs, and maybe those shouldn't be quite so funny in their behaviour (or we just exchange the mapping in the vma, should be a one-liner to address that). Workarounds and hacks in drivers definitely sounds like the wrong approach here to me.
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi,
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
I guess ... but kinda awkward to leave this issue in here, it's really surprising if you call the pte shootdown function, and it doesn't work as advertised.
I was mainly wondering how this worked for us and how to hit it not working to test fixing it.
Quoted 7 lines
Quoted 4 lines
Is there a simple way to check if a BO is mmapped or not? I'd assume so, just don't know the answer off hand. A simple fix would be to either require buffer is not mapped before madvise or skip purging if it is mapped.
Imo simplest fix is to just go back and readd the fake offset. Ugly, but works at least.
Well, shmem helpers removed the fake offset before, so I'm likewise starting to wonder what exactly broke by removing the offset elsewhere.
Maybe shmem helpers where already broken before that patch. I can see that removing the fake offset and continuing to use a shared address space isn't going to fly. Unlike ttm the shmem helpers should have no problems using obj->filp, but I can't see the shmem helper code switching vma->vm_file over to obj->filp anywhere ...
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Thomas Zimmermann
maintainerHi
Am 08.11.19 um 17:27 schrieb Daniel Vetter:
Quoted 35 lines · Daniel Vetter
Quoted 20 lines · Rob Herring
Quoted 19 lines
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.Quoted 2 lines
Quoted 2 lines
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
So unfortunately I'm already having regrets on this. We might even have broken some of the ttm drivers here.
Trouble is, if you need to shoot down userspace ptes of a bo (because it's getting moved or whatever), then we do that with
unmap_mapping_range. Which means each bo needs to be mapping with a unique (offset,adress_space), or it won't work. By remapping all the bo to 0 we've broken this. We've also had this broken this for a while for the simplistic dma-buf mmap, since without any further action we'll reuse the address space of the dma-buf inode, not of thedrm_device.
Quoted 6 lines
Strangely both etnaviv and msm have a comment to that effect - grep for
unmap_mapping_range. But neither actually uses it.Not exactly sure what's the best course of action here now.
Also adding Thomas Zimmermann, who's worked on all the vram helpers.
VRAM helpers use drm_gem_ttm_mmap(), which wraps ttm_bo_mmap_obj(). These changes should be transparent.
Best regards
Thomas
Quoted 90 lines
-Daniel
Quoted 87 lines
Quoted 60 lines
Rob
drivers/gpu/drm/drm_gem.c | 3 +++ drivers/gpu/drm/drm_gem_shmem_helper.c | 3 --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 7 +++++++ include/drm/drm_gem.h | 4 +++- 4 files changed, 13 insertions(+), 4 deletions(-)drivers/gpu/drm/drm_gem.c +3
-0@@ -1106,6 +1106,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,return -EINVAL ;if (obj->funcs && obj->funcs->mmap) { +/* Remove the fake offset */ + vma->vm_pgoff -=drm_vma_node_start (&obj->vma_node); + ret = obj->funcs->mmap (obj, vma);if (ret)return ret;drivers/gpu/drm/drm_gem_shmem_helper.c +0
-3@@ -542,9 +542,6 @@ int drm_gem_shmem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) vma->vm_page_prot =pgprot_decrypted (vma->vm_page_prot); vma->vm_ops = &drm_gem_shmem_vm_ops; -/* Remove the fake offset */ - vma->vm_pgoff -=drm_vma_node_start (&shmem->base.vma_node); -return 0 ; }EXPORT_SYMBOL_GPL (drm_gem_shmem_mmap);drivers/gpu/drm/ttm/ttm_bo_vm.c +7
-0@@ -482,6 +482,13 @@ EXPORT_SYMBOL(ttm_bo_mmap);int ttm_bo_mmap_obj (struct vm_area_struct *vma,struct ttm_buffer_object *bo) {ttm_bo_get (bo); + +/* +* FIXME: &drm_gem_object_funcs.mmap is called with the fake offset +* removed. Add it back here until the rest of TTM works without it. +*/ + vma->vm_pgoff +=drm_vma_node_start (&bo->base.vma_node); +ttm_bo_mmap_vma_setup (bo, vma);return 0 ; }include/drm/drm_gem.h +0
-0include/drm/drm_gem.h struct drm_gem_object_funcs { · line 165
Quoted 8 lines
include/drm/drm_gem.h +3
-1@@ -159,7 +159,9 @@ struct drm_gem_object_funcs {* * The callback is used by by both drm_gem_mmap_obj() and * drm_gem_prime_mmap(). When @mmap is present @vm_ops is not -* used, the @mmap callback must set vma->vm_ops instead. +* used, the @mmap callback must set vma->vm_ops instead. The @mmap +* callback is always called with a 0 offset. The caller will remove +* the fake offset as necessary. * Maybe remove this empty comment line here while at it. With that
Reviewed-by Daniel Vetter <[email protected]>I think I'll follow up with a patch to annotate
drm_gem_mmap_objas deprecated and that instead this here should be used. -DanielQuoted 5 lines
include/drm/drm_gem.h +0
-0*/ int (*mmap)(struct drm_gem_object *obj,struct vm_area_struct *vma);Signature · 1 line
2.20.1Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Signature · 6 lines
Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 41 lines · Thomas Zimmermann
Hi
Am 08.11.19 um 17:27 schrieb Daniel Vetter:
Quoted 38 lines · Daniel Vetter
Quoted 20 lines · Rob Herring
Quoted 19 lines
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.Quoted 2 lines
Quoted 2 lines
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
So unfortunately I'm already having regrets on this. We might even have broken some of the ttm drivers here.
Trouble is, if you need to shoot down userspace ptes of a bo (because it's getting moved or whatever), then we do that with
unmap_mapping_range. Which means each bo needs to be mapping with a unique (offset,adress_space), or it won't work. By remapping all the bo to 0 we've broken this. We've also had this broken this for a while for the simplistic dma-buf mmap, since without any further action we'll reuse the address space of the dma-buf inode, not of thedrm_device.Strangely both etnaviv and msm have a comment to that effect - grep for
unmap_mapping_range. But neither actually uses it.
Quoted 6 lines
Quoted 3 lines
Not exactly sure what's the best course of action here now.
Also adding Thomas Zimmermann, who's worked on all the vram helpers.
VRAM helpers use
drm_gem_ttm_mmap(), which wrapsttm_bo_mmap_obj(). These changes should be transparent.
There's still the issue that for dma-buf mmap vs drm mmap you use different f_mapping, which means ttm's pte shootdown won't work correctly for dma-buf mmaps. But yeah normal operation for ttm/vram helpers should be fine.
-Daniel
Quoted 103 lines
Best regards
ThomasQuoted 90 lines
-Daniel
Quoted 87 lines
Quoted 60 lines
Rob
drivers/gpu/drm/drm_gem.c | 3 +++ drivers/gpu/drm/drm_gem_shmem_helper.c | 3 --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 7 +++++++ include/drm/drm_gem.h | 4 +++- 4 files changed, 13 insertions(+), 4 deletions(-)drivers/gpu/drm/drm_gem.c +3
-0@@ -1106,6 +1106,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,return -EINVAL ;if (obj->funcs && obj->funcs->mmap) { +/* Remove the fake offset */ + vma->vm_pgoff -=drm_vma_node_start (&obj->vma_node); + ret = obj->funcs->mmap (obj, vma);if (ret)return ret;drivers/gpu/drm/drm_gem_shmem_helper.c +0
-3@@ -542,9 +542,6 @@ int drm_gem_shmem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) vma->vm_page_prot =pgprot_decrypted (vma->vm_page_prot); vma->vm_ops = &drm_gem_shmem_vm_ops; -/* Remove the fake offset */ - vma->vm_pgoff -=drm_vma_node_start (&shmem->base.vma_node); -return 0 ; }EXPORT_SYMBOL_GPL (drm_gem_shmem_mmap);drivers/gpu/drm/ttm/ttm_bo_vm.c +7
-0@@ -482,6 +482,13 @@ EXPORT_SYMBOL(ttm_bo_mmap);int ttm_bo_mmap_obj (struct vm_area_struct *vma,struct ttm_buffer_object *bo) {ttm_bo_get (bo); + +/* +* FIXME: &drm_gem_object_funcs.mmap is called with the fake offset +* removed. Add it back here until the rest of TTM works without it. +*/ + vma->vm_pgoff +=drm_vma_node_start (&bo->base.vma_node); +ttm_bo_mmap_vma_setup (bo, vma);return 0 ; }include/drm/drm_gem.h +0
-0include/drm/drm_gem.h struct drm_gem_object_funcs { · line 165
Quoted 8 lines
include/drm/drm_gem.h +3
-1@@ -159,7 +159,9 @@ struct drm_gem_object_funcs {* * The callback is used by by both drm_gem_mmap_obj() and * drm_gem_prime_mmap(). When @mmap is present @vm_ops is not -* used, the @mmap callback must set vma->vm_ops instead. +* used, the @mmap callback must set vma->vm_ops instead. The @mmap +* callback is always called with a 0 offset. The caller will remove +* the fake offset as necessary. * Maybe remove this empty comment line here while at it. With that
Reviewed-by Daniel Vetter <[email protected]>I think I'll follow up with a patch to annotate
drm_gem_mmap_objas deprecated and that instead this here should be used. -DanielQuoted 5 lines
include/drm/drm_gem.h +0
-0*/ int (*mmap)(struct drm_gem_object *obj,struct vm_area_struct *vma);Signature · 1 line
2.20.1Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.chSignature · 6 lines
Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Thomas Zimmermann
maintainerHi
Am 12.11.19 um 10:32 schrieb Daniel Vetter:
Quoted 46 lines · Thomas Zimmermann
Quoted 45 lines
Hi
Am 08.11.19 um 17:27 schrieb Daniel Vetter:
Quoted 41 lines · Daniel Vetter
Quoted 20 lines · Rob Herring
Quoted 19 lines
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.Quoted 2 lines
Quoted 2 lines
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
So unfortunately I'm already having regrets on this. We might even have broken some of the ttm drivers here.
Trouble is, if you need to shoot down userspace ptes of a bo (because it's getting moved or whatever), then we do that with
unmap_mapping_range. Which means each bo needs to be mapping with a unique (offset,adress_space), or it won't work. By remapping all the bo to 0 we've broken this. We've also had this broken this for a while for the simplistic dma-buf mmap, since without any further action we'll reuse the address space of the dma-buf inode, not of thedrm_device.Strangely both etnaviv and msm have a comment to that effect - grep for
unmap_mapping_range. But neither actually uses it.Not exactly sure what's the best course of action here now.
Also adding Thomas Zimmermann, who's worked on all the vram helpers.
Quoted 7 lines
Quoted 2 lines
VRAM helpers use
drm_gem_ttm_mmap(), which wrapsttm_bo_mmap_obj(). These changes should be transparent.There's still the issue that for dma-buf mmap vs drm mmap you use different
f_mapping, which means ttm's pte shootdown won't work correctly for dma-buf mmaps. But yeah normal operation for ttm/vram helpers should be fine.
VRAM helpers don't support dmabufs.
Best regards
Thomas
Quoted 109 lines
-Daniel
Quoted 103 lines
Best regards
ThomasQuoted 90 lines
-Daniel
Quoted 87 lines
Quoted 60 lines
Rob
drivers/gpu/drm/drm_gem.c | 3 +++ drivers/gpu/drm/drm_gem_shmem_helper.c | 3 --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 7 +++++++ include/drm/drm_gem.h | 4 +++- 4 files changed, 13 insertions(+), 4 deletions(-)drivers/gpu/drm/drm_gem.c +3
-0@@ -1106,6 +1106,9 @@ int drm_gem_mmap_obj(struct drm_gem_object *obj, unsigned long obj_size,return -EINVAL ;if (obj->funcs && obj->funcs->mmap) { +/* Remove the fake offset */ + vma->vm_pgoff -=drm_vma_node_start (&obj->vma_node); + ret = obj->funcs->mmap (obj, vma);if (ret)return ret;drivers/gpu/drm/drm_gem_shmem_helper.c +0
-3@@ -542,9 +542,6 @@ int drm_gem_shmem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) vma->vm_page_prot =pgprot_decrypted (vma->vm_page_prot); vma->vm_ops = &drm_gem_shmem_vm_ops; -/* Remove the fake offset */ - vma->vm_pgoff -=drm_vma_node_start (&shmem->base.vma_node); -return 0 ; }EXPORT_SYMBOL_GPL (drm_gem_shmem_mmap);drivers/gpu/drm/ttm/ttm_bo_vm.c +7
-0@@ -482,6 +482,13 @@ EXPORT_SYMBOL(ttm_bo_mmap);int ttm_bo_mmap_obj (struct vm_area_struct *vma,struct ttm_buffer_object *bo) {ttm_bo_get (bo); + +/* +* FIXME: &drm_gem_object_funcs.mmap is called with the fake offset +* removed. Add it back here until the rest of TTM works without it. +*/ + vma->vm_pgoff +=drm_vma_node_start (&bo->base.vma_node); +ttm_bo_mmap_vma_setup (bo, vma);return 0 ; }include/drm/drm_gem.h +0
-0include/drm/drm_gem.h struct drm_gem_object_funcs { · line 165
Quoted 8 lines
include/drm/drm_gem.h +3
-1@@ -159,7 +159,9 @@ struct drm_gem_object_funcs {* * The callback is used by by both drm_gem_mmap_obj() and * drm_gem_prime_mmap(). When @mmap is present @vm_ops is not -* used, the @mmap callback must set vma->vm_ops instead. +* used, the @mmap callback must set vma->vm_ops instead. The @mmap +* callback is always called with a 0 offset. The caller will remove +* the fake offset as necessary. * Maybe remove this empty comment line here while at it. With that
Reviewed-by Daniel Vetter <[email protected]>I think I'll follow up with a patch to annotate
drm_gem_mmap_objas deprecated and that instead this here should be used. -DanielQuoted 5 lines
include/drm/drm_gem.h +0
-0*/ int (*mmap)(struct drm_gem_object *obj,struct vm_area_struct *vma);Signature · 1 line
2.20.1Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.chSignature · 6 lines
Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Felix Imendörffer
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 52 lines · Thomas Zimmermann
Hi
Am 12.11.19 um 10:32 schrieb Daniel Vetter:
Quoted 49 lines · Thomas Zimmermann
Quoted 47 lines
Hi
Am 08.11.19 um 17:27 schrieb Daniel Vetter:
Quoted 35 lines · Daniel Vetter
Quoted 20 lines · Rob Herring
Quoted 19 lines
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.Quoted 2 lines
Quoted 2 lines
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
So unfortunately I'm already having regrets on this. We might even have broken some of the ttm drivers here.
Trouble is, if you need to shoot down userspace ptes of a bo (because it's getting moved or whatever), then we do that with
unmap_mapping_range. Which means each bo needs to be mapping with a unique (offset,adress_space), or it won't work. By remapping all the bo to 0 we've broken this. We've also had this broken this for a while for the simplistic dma-buf mmap, since without any further action we'll reuse the address space of the dma-buf inode, not of thedrm_device.Quoted 6 lines
Strangely both etnaviv and msm have a comment to that effect - grep for
unmap_mapping_range. But neither actually uses it.Not exactly sure what's the best course of action here now.
Also adding Thomas Zimmermann, who's worked on all the vram helpers.
VRAM helpers use
drm_gem_ttm_mmap(), which wrapsttm_bo_mmap_obj(). These changes should be transparent.
Quoted 6 lines
Quoted 4 lines
There's still the issue that for dma-buf mmap vs drm mmap you use different
f_mapping, which means ttm's pte shootdown won't work correctly for dma-buf mmaps. But yeah normal operation for ttm/vram helpers should be fine.VRAM helpers don't support dmabufs.
It's not that simple. Even when not supporting dma-buf export and import it is still possible to create dma-bufs, import them into the same driver (which doesn't actually export+import but just grabs a gem object reference instead) and also to mmap them via prime/dma-buf code path ...
Can ttm use the same trick msm uses? Now that ttm bo's are a gem object superclass I think we should be able to switch vma->vm_file to bo->base.filp, simliar to msm_gem_mmap_obj(), probably best done in drm_gem_ttm_mmap().
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 66 lines · Gerd Hoffmann
Quoted 52 lines · Thomas Zimmermann
Hi
Am 12.11.19 um 10:32 schrieb Daniel Vetter:
Quoted 49 lines · Thomas Zimmermann
Quoted 47 lines
Hi
Am 08.11.19 um 17:27 schrieb Daniel Vetter:
Quoted 35 lines · Daniel Vetter
Quoted 20 lines · Rob Herring
Quoted 19 lines
Commit
c40069cb7bd6("drm: addmmap()todrm_gem_object_funcs") introduced a GEM objectmmap()hook which is expected to subtract the fake offset fromvm_pgoff. However, formmap()on dmabufs, there is not a fake offset.To fix this, let's always call
mmap()object callback with an offset of 0, and leave it up todrm_gem_mmap_obj()to remove the fake offset.TTM still needs the fake offset, so we have to add it back until that's fixed.
Fixes c40069cb7bd6 ("drm: add mmap() to drm_gem_object_funcs")Cc Gerd Hoffmann <[email protected]>Cc Daniel Vetter <[email protected]>Signed-off-by Rob Herring <[email protected]>---
v2:
- Move subtracting the fake offset out ofmmap()obj callbacks.Quoted 2 lines
Quoted 2 lines
I've tested shmem, but not ttm. Hopefully, I understood what's needed for TTM.
So unfortunately I'm already having regrets on this. We might even have broken some of the ttm drivers here.
Trouble is, if you need to shoot down userspace ptes of a bo (because it's getting moved or whatever), then we do that with
unmap_mapping_range. Which means each bo needs to be mapping with a unique (offset,adress_space), or it won't work. By remapping all the bo to 0 we've broken this. We've also had this broken this for a while for the simplistic dma-buf mmap, since without any further action we'll reuse the address space of the dma-buf inode, not of thedrm_device.Quoted 6 lines
Strangely both etnaviv and msm have a comment to that effect - grep for
unmap_mapping_range. But neither actually uses it.Not exactly sure what's the best course of action here now.
Also adding Thomas Zimmermann, who's worked on all the vram helpers.
VRAM helpers use
drm_gem_ttm_mmap(), which wrapsttm_bo_mmap_obj(). These changes should be transparent.Quoted 6 lines
Quoted 4 lines
There's still the issue that for dma-buf mmap vs drm mmap you use different
f_mapping, which means ttm's pte shootdown won't work correctly for dma-buf mmaps. But yeah normal operation for ttm/vram helpers should be fine.VRAM helpers don't support dmabufs.
It's not that simple. Even when not supporting dma-buf export and import it is still possible to create dma-bufs, import them into the same driver (which doesn't actually export+import but just grabs a gem object reference instead) and also to mmap them via prime/dma-buf code path ...
Quoted 4 lines
Can ttm use the same trick msm uses? Now that ttm bo's are a gem object superclass I think we should be able to switch
vma->vm_filetobo->base.filp, simliar tomsm_gem_mmap_obj(), probably best done indrm_gem_ttm_mmap().
bo->base.filp is the shmem inode file, and I'm not too sure how much shmem approves of us misappropriating the mapping. For shmem only objects it probably doesn't matter (since both gem mmaps and shmem mmaps will point at the same underlying struct page), but for vram/ttm/anything else the gem mmap might point into iomem, and shmem then might go boom trying to do stuff with that. I think having our own mapping would be the cleanest long-term approach ...
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi,
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines
VRAM helpers use
drm_gem_ttm_mmap(), which wrapsttm_bo_mmap_obj(). These changes should be transparent.There's still the issue that for dma-buf mmap vs drm mmap you use different
f_mapping, which means ttm's pte shootdown won't work correctly for dma-buf mmaps. But yeah normal operation for ttm/vram helpers should be fine.
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
VRAM helpers don't support dmabufs.
It's not that simple. Even when not supporting dma-buf export and import it is still possible to create dma-bufs, import them into the same driver (which doesn't actually export+import but just grabs a gem object reference instead) and also to mmap them via prime/dma-buf code path ...
... but after looking again I think we are all green here. Given that only self-import works we'll only see vram gem objects in the mmap code path, which should have everything set up correctly. Same goes for qxl.
All other ttm drivers still use the old mmap code path, so all green there too I think. Also I somehow doubt dma-buf mmap vs. drm mmap ends up using different f_mapping, ttm code has a WARN_ON in ttm_bo_vm_open() which would fire should that be the case.
Do imported dma-bufs hit the drm mmap code path in the first place? Wouldn't mmap be handled by the exporting driver?
Quoted 11 lines
Quoted 4 lines
Can ttm use the same trick msm uses? Now that ttm bo's are a gem object superclass I think we should be able to switch
vma->vm_filetobo->base.filp, simliar tomsm_gem_mmap_obj(), probably best done indrm_gem_ttm_mmap().
bo->base.filpis the shmem inode file, and I'm not too sure how much shmem approves of us misappropriating the mapping. For shmem only objects it probably doesn't matter (since both gem mmaps and shmem mmaps will point at the same underlying struct page), but for vram/ttm/anything else the gem mmap might point into iomem, and shmem then might go boom trying to do stuff with that.
Yes, agreeing here after wading through the code ...
Quoted 2 lines
I think having our own mapping would be the cleanest long-term approach ...
You mean using per drm object (instead of per drm device) address spaces?
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 23 lines · Gerd Hoffmann
Hi,
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines
VRAM helpers use
drm_gem_ttm_mmap(), which wrapsttm_bo_mmap_obj(). These changes should be transparent.There's still the issue that for dma-buf mmap vs drm mmap you use different
f_mapping, which means ttm's pte shootdown won't work correctly for dma-buf mmaps. But yeah normal operation for ttm/vram helpers should be fine.Quoted 7 lines
Quoted 7 lines
Quoted 1 line
VRAM helpers don't support dmabufs.
It's not that simple. Even when not supporting dma-buf export and import it is still possible to create dma-bufs, import them into the same driver (which doesn't actually export+import but just grabs a gem object reference instead) and also to mmap them via prime/dma-buf code path ...
... but after looking again I think we are all green here. Given that only self-import works we'll only see vram gem objects in the mmap code path, which should have everything set up correctly. Same goes for qxl.
Quoted 7 lines
All other ttm drivers still use the old mmap code path, so all green there too I think. Also I somehow doubt dma-buf mmap vs. drm mmap ends up using different
f_mapping, ttm code has aWARN_ONinttm_bo_vm_open()which would fire should that be the case.Do imported dma-bufs hit the drm mmap code path in the first place? Wouldn't mmap be handled by the exporting driver?
drm_gem_dmabuf_mmap -> obj->funcs->mmap -> ttm_bo_mmap_obj
And I'm not seeing anyone adjusting vm_file->f_mapping anywhere here at all.
Note to hit this you need userspace to
- handle2fd on a buffer to create a dma-buf fd
- call mmap directly on that dma-buf fd
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines
Can ttm use the same trick msm uses? Now that ttm bo's are a gem object superclass I think we should be able to switch
vma->vm_filetobo->base.filp, simliar tomsm_gem_mmap_obj(), probably best done indrm_gem_ttm_mmap().
bo->base.filpis the shmem inode file, and I'm not too sure how much shmem approves of us misappropriating the mapping. For shmem only objects it probably doesn't matter (since both gem mmaps and shmem mmaps will point at the same underlying struct page), but for vram/ttm/anything else the gem mmap might point into iomem, and shmem then might go boom trying to do stuff with that.
Quoted 6 lines
Yes, agreeing here after wading through the code ...
Quoted 2 lines
I think having our own mapping would be the cleanest long-term approach ...
You mean using per drm object (instead of per drm device) address spaces?
Yup. But I think that would be quite a pile of work, since we need an anon inode for each gem bo.
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi,
Quoted 9 lines
Quoted 9 lines
... but after looking again I think we are all green here. Given that only self-import works we'll only see vram gem objects in the mmap code path, which should have everything set up correctly. Same goes for qxl.
All other ttm drivers still use the old mmap code path, so all green there too I think. Also I somehow doubt dma-buf mmap vs. drm mmap ends up using different
f_mapping, ttm code has aWARN_ONinttm_bo_vm_open()which would fire should that be the case.
Quoted 6 lines
Quoted 2 lines
Do imported dma-bufs hit the drm mmap code path in the first place? Wouldn't mmap be handled by the exporting driver?
drm_gem_dmabuf_mmap->obj->funcs->mmap->ttm_bo_mmap_objAnd I'm not seeing anyone adjusting
vm_file->f_mappinganywhere here at all.
[ some more code browsing ]
Ok, I see. dma-bufs get their own file, their own anon inode and thereby their own address space. So that it used when mmaping the dma-buf.
drm filehandle's get the shared address space instead, drm_open() sets it.
So, yes, I see the problem. It's not new though, as far I can see the old dma-buf mmap code path doesn't adjust f_mapping anywhere either ...
Quoted 3 lines
Note to hit this you need userspace to
- handle2fd on a buffer to create a dma-buf fd
- call mmap directly on that dma-buf fd
Hmm, seems for handle2fd I need a dummy gem_prime_get_sg_table function wired up even when not actually exporting/importing anything. So I think neither qxl nor any of the vram drivers allow to trigger that (and no other ttm driver uses the new ttm mmap code yet).
So, $subject patch should not make things worse in ttm land.
When hacking the bochs driver to have export callbacks (without supporting actual exports) handle2fd + mmap() callback works fine. Didn't verify yet I actually get the correct pages mapped. But maybe mmap() isn't the problem when the correct address space is important for unmap only.
Is there a good test case?
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 41 lines · Gerd Hoffmann
Hi,
Quoted 9 lines
Quoted 9 lines
... but after looking again I think we are all green here. Given that only self-import works we'll only see vram gem objects in the mmap code path, which should have everything set up correctly. Same goes for qxl.
All other ttm drivers still use the old mmap code path, so all green there too I think. Also I somehow doubt dma-buf mmap vs. drm mmap ends up using different
f_mapping, ttm code has aWARN_ONinttm_bo_vm_open()which would fire should that be the case.Quoted 6 lines
Quoted 2 lines
Do imported dma-bufs hit the drm mmap code path in the first place? Wouldn't mmap be handled by the exporting driver?
drm_gem_dmabuf_mmap->obj->funcs->mmap->ttm_bo_mmap_objAnd I'm not seeing anyone adjusting
vm_file->f_mappinganywhere here at all.[ some more code browsing ]
Ok, I see. dma-bufs get their own file, their own anon inode and thereby their own address space. So that it used when mmaping the dma-buf.
drm filehandle's get the shared address space instead,
drm_open()sets it.So, yes, I see the problem. It's not new though, as far I can see the old dma-buf mmap code path doesn't adjust
f_mappinganywhere either ...Quoted 3 lines
Note to hit this you need userspace to
- handle2fd on a buffer to create a dma-buf fd
- call mmap directly on that dma-buf fdHmm, seems for handle2fd I need a dummy
gem_prime_get_sg_tablefunction wired up even when not actually exporting/importing anything. So I think neither qxl nor any of the vram drivers allow to trigger that (and no other ttm driver uses the new ttm mmap code yet).So, $subject patch should not make things worse in ttm land.
Quoted 7 lines
When hacking the bochs driver to have export callbacks (without supporting actual exports) handle2fd +
mmap()callback works fine. Didn't verify yet I actually get the correct pages mapped. But maybemmap()isn't the problem when the correct address space is important for unmap only.Is there a good test case?
You need memory pressure, to force ttm to unmap the bo, not userspace. So roughly
1. create bo
2. mmap it through drm fd, write some stuff
3. export to dma-buf, mmap it, verify stuff is there
4. create a pile more bo, mmap them write to them
5. once you've thrashed all of vram enough, recheck your original bo. If I'm right you should get the following:
- drm fd mmap still show right content
- dma-buf fd mmap shows random crap that you've written into other buffers
Ofc you need to make sure that an mmap actually forces the buffer into vram. So might need a combo of modeset+mmap, to make that happen. Plain mmap might just give you ptes that point into system memory, which is not managed by ttm like vram.
munmap called by userspace isn't a problem, since that starts from the right struct mm_struct. It's when you start with the object (i.e. in the driver), and need to figure out where all the various vma that mmap it are where this matters.
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 12 lines
You need memory pressure, to force ttm to unmap the bo, not userspace. So roughly
1. create bo
2. mmap it through drm fd, write some stuff
3. export to dma-buf, mmap it, verify stuff is there
4. create a pile more bo, mmap them write to them
5. once you've thrashed all of vram enough, recheck your original bo. If I'm right you should get the following:
- drm fd mmap still show right content
- dma-buf fd mmap shows random crap that you've written into other buffers
Quoted 4 lines
Ofc you need to make sure that an mmap actually forces the buffer into vram. So might need a combo of modeset+mmap, to make that happen. Plain mmap might just give you ptes that point into system memory, which is not managed by ttm like vram.
Is any move buffer good enough to trigger this, i.e. will SYSTEM -> VRAM work too? That'll be easier because all I need to do is map the buffer to a crtc to force pinning to vram, then check if the mappings are intact still ...
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 18 lines · Gerd Hoffmann
Quoted 13 lines
You need memory pressure, to force ttm to unmap the bo, not userspace. So roughly
1. create bo
2. mmap it through drm fd, write some stuff
3. export to dma-buf, mmap it, verify stuff is there
4. create a pile more bo, mmap them write to them
5. once you've thrashed all of vram enough, recheck your original bo. If I'm right you should get the following:
- drm fd mmap still show right content
- dma-buf fd mmap shows random crap that you've written into other buffersQuoted 4 lines
Ofc you need to make sure that an mmap actually forces the buffer into vram. So might need a combo of modeset+mmap, to make that happen. Plain mmap might just give you ptes that point into system memory, which is not managed by ttm like vram.
Quoted 4 lines
Is any move buffer good enough to trigger this, i.e. will SYSTEM -> VRAM work too? That'll be easier because all I need to do is map the buffer to a crtc to force pinning to vram, then check if the mappings are intact still ...
I think that should work too. Just make sure you actually write through SYSTEM first (maybe with some printk or whatever). -Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 19 lines · Daniel Vetter
Quoted 18 lines · Gerd Hoffmann
Quoted 17 lines
You need memory pressure, to force ttm to unmap the bo, not userspace. So roughly
1. create bo
2. mmap it through drm fd, write some stuff
3. export to dma-buf, mmap it, verify stuff is there
4. create a pile more bo, mmap them write to them
5. once you've thrashed all of vram enough, recheck your original bo. If I'm right you should get the following:
- drm fd mmap still show right content
- dma-buf fd mmap shows random crap that you've written into other buffersOfc you need to make sure that an mmap actually forces the buffer into vram. So might need a combo of modeset+mmap, to make that happen. Plain mmap might just give you ptes that point into system memory, which is not managed by ttm like vram.
Quoted 6 lines
Quoted 4 lines
Is any move buffer good enough to trigger this, i.e. will SYSTEM -> VRAM work too? That'll be easier because all I need to do is map the buffer to a crtc to force pinning to vram, then check if the mappings are intact still ...
I think that should work too.
Seems to work ok for ttm/vram.
Test code:
https://git.kraxel.org/cgit/drminfo/commit/?id=a9eb96504dc17376e07b5c6edf5296b41a48bbfe
Quoted 2 lines
Just make sure you actually write
through SYSTEM first (maybe with some printk or whatever).
That works fine. Sprinkled ...
system("cat /sys/kernel/debug/dri/0/vram-mm"); ... into the test code, and drmModeSetCrtc() indeed makes the gem object show up there.
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 26 lines · Gerd Hoffmann
Quoted 24 lines · Daniel Vetter
Quoted 22 lines · Gerd Hoffmann
Quoted 13 lines
You need memory pressure, to force ttm to unmap the bo, not userspace. So roughly
1. create bo
2. mmap it through drm fd, write some stuff
3. export to dma-buf, mmap it, verify stuff is there
4. create a pile more bo, mmap them write to them
5. once you've thrashed all of vram enough, recheck your original bo. If I'm right you should get the following:
- drm fd mmap still show right content
- dma-buf fd mmap shows random crap that you've written into other buffersQuoted 4 lines
Ofc you need to make sure that an mmap actually forces the buffer into vram. So might need a combo of modeset+mmap, to make that happen. Plain mmap might just give you ptes that point into system memory, which is not managed by ttm like vram.
Is any move buffer good enough to trigger this, i.e. will SYSTEM -> VRAM work too? That'll be easier because all I need to do is map the buffer to a crtc to force pinning to vram, then check if the mappings are intact still ...
Quoted 6 lines
Quoted 1 line
I think that should work too.
Seems to work ok for ttm/vram.
Test code:
https://git.kraxel.org/cgit/drminfo/commit/?id=a9eb96504dc17376e07b5c6edf5296b41a48bbfe
I think that's not nasty enough. If the mappings aren't updated, then you'll still see the the same old pages with the right contents. You need to change them somehow after they migrated (with vram eviction that happens automatically since there'll b another object in the same spot, for system memory I think you need the shrinker to kick in and free the pages first). Easiest probably to wait for a key press so you can appreciate the color, then write a different color (full screen) when the buffer is in vram.
Quoted 9 lines
Quoted 2 lines
Just make sure you actually write
through SYSTEM first (maybe with some printk or whatever).That works fine. Sprinkled ...
system("cat /sys/kernel/debug/dri/0/vram-mm");... into the test code, and
drmModeSetCrtc()indeed makes the gem object show up there.
You'd need to check the ptes themselves. Which I think not even proc shows by default (only the file that's supposed to be mapped). But good to confirm that the buffer moved at least.
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi,
Quoted 12 lines
Quoted 11 lines
Quoted 6 lines
Quoted 4 lines
Is any move buffer good enough to trigger this, i.e. will SYSTEM -> VRAM work too? That'll be easier because all I need to do is map the buffer to a crtc to force pinning to vram, then check if the mappings are intact still ...
I think that should work too.
Seems to work ok for ttm/vram.
Test code:
https://git.kraxel.org/cgit/drminfo/commit/?id=a9eb96504dc17376e07b5c6edf5296b41a48bbfe
Quoted 8 lines
I think that's not nasty enough. If the mappings aren't updated, then you'll still see the the same old pages with the right contents. You need to change them somehow after they migrated (with vram eviction that happens automatically since there'll b another object in the same spot, for system memory I think you need the shrinker to kick in and free the pages first). Easiest probably to wait for a key press so you can appreciate the color, then write a different color (full screen) when the buffer is in vram.
update-object-after-move works fine.
try zap mappings with madvise(dontneed) has no bad effects (after vram move, trying to force re-creating the ptes).
didn't try the memory pressure thing yet.
Quoted 3 lines
You'd need to check the ptes themselves. Which I think not even proc shows by default (only the file that's supposed to be mapped). But good to confirm that the buffer moved at least.
One reproducable glitch found: fork() while having a dma-buf mapped triggers the WARN_ON in ttm_bo_vm_open(). Both old & new mmap code paths, so this isn't something new coming from the drm_gem_object_funcs.mmap switch. Instead it is an old issue coming from the address space handling being different in drm mmap and dmabuf mmap code paths.
I can see now why you want a private address space for each object. Does that imply we need an anon_inode for each gem object? Or is there some more lightweight way to do this?
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 23 lines · Gerd Hoffmann
Hi,
Quoted 12 lines
Quoted 11 lines
Quoted 6 lines
Quoted 4 lines
Is any move buffer good enough to trigger this, i.e. will SYSTEM -> VRAM work too? That'll be easier because all I need to do is map the buffer to a crtc to force pinning to vram, then check if the mappings are intact still ...
I think that should work too.
Seems to work ok for ttm/vram.
Test code:
https://git.kraxel.org/cgit/drminfo/commit/?id=a9eb96504dc17376e07b5c6edf5296b41a48bbfeQuoted 8 lines
I think that's not nasty enough. If the mappings aren't updated, then you'll still see the the same old pages with the right contents. You need to change them somehow after they migrated (with vram eviction that happens automatically since there'll b another object in the same spot, for system memory I think you need the shrinker to kick in and free the pages first). Easiest probably to wait for a key press so you can appreciate the color, then write a different color (full screen) when the buffer is in vram.
Quoted 4 lines
update-object-after-move works fine.
try zap mappings with madvise(dontneed) has no bad effects (after vram move, trying to force re-creating the ptes).
Well if it's broken the zapping wouldn't work :-)
Quoted 1 line
didn't try the memory pressure thing yet.
I'm surprised ... and I have no idea how/why it keeps working.
For my paranoia, can you instrument the ttm page fault handler, and double check that we get new faults after the move, and set up new ptes for the same old mapping, but now pointing at the new place in vram?
Quoted 11 lines
Quoted 3 lines
You'd need to check the ptes themselves. Which I think not even proc shows by default (only the file that's supposed to be mapped). But good to confirm that the buffer moved at least.
One reproducable glitch found:
fork()while having a dma-buf mapped triggers theWARN_ONinttm_bo_vm_open(). Both old & new mmap code paths, so this isn't something new coming from the drm_gem_object_funcs.mmap switch. Instead it is an old issue coming from the address space handling being different in drm mmap and dmabuf mmap code paths.
Quoted 3 lines
I can see now why you want a private address space for each object. Does that imply we need an
anon_inodefor each gem object? Or is there some more lightweight way to do this?
I have no idea whether we can get a address_space struct without an inode (and no disasters).
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi,
Quoted 6 lines
Quoted 3 lines
I can see now why you want a private address space for each object. Does that imply we need an
anon_inodefor each gem object? Or is there some more lightweight way to do this?I have no idea whether we can get a
address_spacestruct without an inode (and no disasters).
Anything building on shmem helpers should be able to use obj->filp->f_mapping, right? So allocating an inode unconditionally doesn't look like a good plan.
Guess I'll go look at ttm-local changes for starters and see how it goes.
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 6 lines · Gerd Hoffmann
Hi,
Quoted 4 lines
Quoted 3 lines
I can see now why you want a private address space for each object. Does that imply we need an
anon_inodefor each gem object? Or is there some more lightweight way to do this?
Quoted 6 lines
Quoted 2 lines
I have no idea whether we can get a
address_spacestruct without an inode (and no disasters).Anything building on shmem helpers should be able to use
obj->filp->f_mapping, right? So allocating an inode unconditionally doesn't look like a good plan.
Not sure the shmem helpers forward the mmap to the underlying shmem file, so not sure this is the greatest way either.
Quoted 2 lines
Guess I'll go look at ttm-local changes for starters and see how it goes.
I think for ttm just consistently using the one per-device mapping for everything, with all the fake offsets, is probably the quickest route. Maybe also for for shmem helpers.
I'm kinda leaning towards that we just restore the fake offset everywhere, and document how it works. Until someone comes up with a really bright idea.
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi,
Quoted 6 lines
Quoted 3 lines
Anything building on shmem helpers should be able to use
obj->filp->f_mapping, right? So allocating an inode unconditionally doesn't look like a good plan.Not sure the shmem helpers forward the mmap to the underlying shmem file, so not sure this is the greatest way either.
I think so, shmem helpers already zap the fake offset, and this would not work without per-object address space I think.
Quoted 5 lines
Quoted 2 lines
Guess I'll go look at ttm-local changes for starters and see how it goes.
I think for ttm just consistently using the one per-device mapping for everything, with all the fake offsets, is probably the quickest route.
Hmm, not clear how to fit dmabufs into this. dmabufs already have their own file, inode and address space. I'm not sure you can switch that over to the per-device mapping in the first place, and even if you can I have my doubts this is a good idea from a security point of view ...
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 7 lines · Gerd Hoffmann
Hi,
Quoted 4 lines
Quoted 3 lines
Anything building on shmem helpers should be able to use
obj->filp->f_mapping, right? So allocating an inode unconditionally doesn't look like a good plan.
Quoted 5 lines
Quoted 2 lines
Not sure the shmem helpers forward the mmap to the underlying shmem file, so not sure this is the greatest way either.
I think so, shmem helpers already zap the fake offset, and this would not work without per-object address space I think.
That's why I think shmem helpers have a problem right now, and why it's probably best to go back to the fake offset for everything. But we seem to have lost Rob Herring in all this thread, so readding him.
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines
Guess I'll go look at ttm-local changes for starters and see how it goes.
I think for ttm just consistently using the one per-device mapping for everything, with all the fake offsets, is probably the quickest route.
Hmm, not clear how to fit dmabufs into this. dmabufs already have their own file, inode and address space. I'm not sure you can switch that over to the per-device mapping in the first place, and even if you can I have my doubts this is a good idea from a security point of view ...
You can (plenty drivers do that already), and not sure how that breaks security? Imo it's more consistent, since everything ends up pointing to the same underlying resource.
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi,
Quoted 9 lines
Quoted 7 lines
Quoted 2 lines
I think for ttm just consistently using the one per-device mapping for everything, with all the fake offsets, is probably the quickest route.
Hmm, not clear how to fit dmabufs into this. dmabufs already have their own file, inode and address space. I'm not sure you can switch that over to the per-device mapping in the first place, and even if you can I have my doubts this is a good idea from a security point of view ...
You can (plenty drivers do that already),
Have pointer(s) to code?
Quoted 2 lines
and not sure how that breaks
security?
Go try mmap(fake-offset) on the dma-buf file handle to access other buffers of the drm device? Hmm, thinking again, I guess the verify-access restrictions should prevent that.
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 6 lines · Gerd Hoffmann
Hi,
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
I think for ttm just consistently using the one per-device mapping for everything, with all the fake offsets, is probably the quickest route.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Hmm, not clear how to fit dmabufs into this. dmabufs already have their own file, inode and address space. I'm not sure you can switch that over to the per-device mapping in the first place, and even if you can I have my doubts this is a good idea from a security point of view ...
You can (plenty drivers do that already),
Have pointer(s) to code?
dma_buf_mmap() does the same trick, but the other way round: It converts from gem mapping and fake offset to dma-buf mapping and 0 offset. I think we'd simply need the inverse.
Quoted 6 lines
Quoted 2 lines
and not sure how that breaks
security?Go try mmap(fake-offset) on the dma-buf file handle to access other buffers of the drm device? Hmm, thinking again, I guess the verify-access restrictions should prevent that.
Ah, we're not going to replace the mapping on the dma-buf file. Only the file of the vma structure. Doing the former would indeed be pretty bad from a security pov. So don't do what amdgpu_gem_prime_export does, that's the bad stuff :-)
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi,
Quoted 3 lines
Ah, we're not going to replace the mapping on the dma-buf file. Only the file of the vma structure. Doing the former would indeed be pretty bad from a security pov.
Now where do I get a filp from? Can I just call drm_open?
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 5 lines · Gerd Hoffmann
Quoted 3 lines
Ah, we're not going to replace the mapping on the dma-buf file. Only the file of the vma structure. Doing the former would indeed be pretty bad from a security pov.
Now where do I get a filp from? Can I just call
drm_open?
Hm, now I wonder whether it's maybe ok to just exchange the filp->f_mapping. As long as we don't mix up the kinds of mapping and page-cache management that can happon on a given address_space structure (that's why I'm not keeon the shmem mapping reused, since shmem uses the same address_space structure internally to manage the page allocations - address_space both contains the page cache for a file, and also the reverse mapping information). So kinda what drm_open does, except we do that to the dma-buf file. So exactly what amdgpu is doing and that I just complained about :-)
Aside: the amdgpu isn't great because it's racy, userspace could have guessed the fd and already started an mmap before we managed to update stuff. But aside from that maybe rolling out the amdgpu trick for everyone is the right way?
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi,
Quoted 3 lines
Aside: the amdgpu isn't great because it's racy, userspace could have guessed the fd and already started an mmap before we managed to update stuff.
Can't see that race. When I read the code correctly the fd is created and installed (using dma_buf_fd) only after dmabuf setup is finished.
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 8 lines · Gerd Hoffmann
Hi,
Quoted 3 lines
Aside: the amdgpu isn't great because it's racy, userspace could have guessed the fd and already started an mmap before we managed to update stuff.
Can't see that race. When I read the code correctly the fd is created and installed (using
dma_buf_fd) only after dmabuf setup is finished.
Right, I mixed things up. Looks all good.
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi,
Quoted 7 lines
Quoted 4 lines
update-object-after-move works fine.
try zap mappings with madvise(dontneed) has no bad effects (after vram move, trying to force re-creating the ptes).
Well if it's broken the zapping wouldn't work :-)
Quoted 7 lines
Quoted 1 line
didn't try the memory pressure thing yet.
I'm surprised ... and I have no idea how/why it keeps working.
For my paranoia, can you instrument the ttm page fault handler, and double check that we get new faults after the move, and set up new ptes for the same old mapping, but now pointing at the new place in vram?
Hmm, only the drm device mapping is faulted in after moving it, the dma-buf mapping is not. Fixed by:
-------------------------- cut here ------------------------ From 3a7f6c6dbf3b1e4b412c2b283b2ea4edff9f33b5 Mon Sep 17 00:00:00 2001
From: Gerd Hoffmann <[email protected]>
Date: Thu, 21 Nov 2019 08:39:17 +0100
Subject: [PATCH] drm: share address space for dma bufs
--- drivers/gpu/drm/drm_prime.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/gpu/drm/drm_prime.c +3 -1
@@ -240,6 +240,7 @@ void drm_prime_destroy_file_private(struct drm_prime_file_private *prime_fpriv)struct dma_buf *drm_gem_dmabuf_export (struct drm_device *dev,struct dma_buf_export_info *exp_info) { +struct drm_gem_object *obj = exp_info->priv;struct dma_buf *dma_buf; dma_buf =dma_buf_export (exp_info);
@@ -247,7 +248,8 @@ struct dma_buf *drm_gem_dmabuf_export(struct drm_device *dev,return dma_buf;drm_dev_get (dev); -drm_gem_object_get (exp_info->priv); +drm_gem_object_get (obj); + dma_buf->file->f_mapping = obj->dev->anon_inode->i_mapping;return dma_buf; }
Signature · 13 lines
2.18.1 -------------------------- cut here ------------------------ git branch: https://git.kraxel.org/cgit/linux/log/?h=drm-mmap-debug cheers, Gerd _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 11 lines · Gerd Hoffmann
Hi,
Quoted 9 lines
Quoted 4 lines
update-object-after-move works fine.
try zap mappings with madvise(dontneed) has no bad effects (after vram move, trying to force re-creating the ptes).
Well if it's broken the zapping wouldn't work :-)
Quoted 1 line
didn't try the memory pressure thing yet.
Quoted 8 lines
Quoted 5 lines
I'm surprised ... and I have no idea how/why it keeps working.
For my paranoia, can you instrument the ttm page fault handler, and double check that we get new faults after the move, and set up new ptes for the same old mapping, but now pointing at the new place in vram?
Hmm, only the drm device mapping is faulted in after moving it, the dma-buf mapping is not. Fixed by:
Ah yes, that's more what I'd expect to happen, and the below is what I'd expect to fix things up. I think we should move it up ahead of the device callback (so that drivers can overwrite) and then push as a fix. Separate from a possible patch to undo the fake offset removal. -Daniel
Quoted 43 lines
-------------------------- cut here ------------------------ From
3a7f6c6dbf3b1e4b412c2b283b2ea4edff9f33b5Mon Sep 17 00:00:00 2001
From: Gerd Hoffmann <[email protected]>
Date: Thu, 21 Nov 2019 08:39:17 +0100
Subject: [PATCH] drm: share address space for dma bufs--- drivers/gpu/drm/drm_prime.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)drivers/gpu/drm/drm_prime.c +3
-1@@ -240,6 +240,7 @@ void drm_prime_destroy_file_private(struct drm_prime_file_private *prime_fpriv)struct dma_buf *drm_gem_dmabuf_export (struct drm_device *dev,struct dma_buf_export_info *exp_info) { +struct drm_gem_object *obj = exp_info->priv;struct dma_buf *dma_buf; dma_buf =dma_buf_export (exp_info);@@ -247,7 +248,8 @@ struct dma_buf *drm_gem_dmabuf_export(struct drm_device *dev,return dma_buf;drm_dev_get (dev); -drm_gem_object_get (exp_info->priv); +drm_gem_object_get (obj); + dma_buf->file->f_mapping = obj->dev->anon_inode->i_mapping;return dma_buf; }Signature · 8 lines
2.18.1 -------------------------- cut here ------------------------ git branch: https://git.kraxel.org/cgit/linux/log/?h=drm-mmap-debug cheers, Gerd
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 18 lines · Daniel Vetter
Quoted 17 lines · Gerd Hoffmann
Hi,
Quoted 14 lines
Quoted 4 lines
update-object-after-move works fine.
try zap mappings with madvise(dontneed) has no bad effects (after vram move, trying to force re-creating the ptes).
Well if it's broken the zapping wouldn't work :-)
Quoted 1 line
didn't try the memory pressure thing yet.
I'm surprised ... and I have no idea how/why it keeps working.
For my paranoia, can you instrument the ttm page fault handler, and double check that we get new faults after the move, and set up new ptes for the same old mapping, but now pointing at the new place in vram?
Quoted 8 lines
Quoted 2 lines
Hmm, only the drm device mapping is faulted in after moving it, the dma-buf mapping is not. Fixed by:
Ah yes, that's more what I'd expect to happen, and the below is what I'd expect to fix things up. I think we should move it up ahead of the device callback (so that drivers can overwrite) and then push as a fix. Separate from a possible patch to undo the fake offset removal. -Daniel
Is there a more elegant way than copying the intel list on non-intel patches to kick intel ci?
cheers,
Gerd
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 23 lines · Gerd Hoffmann
Quoted 21 lines · Daniel Vetter
Quoted 19 lines · Gerd Hoffmann
Hi,
Quoted 7 lines
Quoted 4 lines
update-object-after-move works fine.
try zap mappings with madvise(dontneed) has no bad effects (after vram move, trying to force re-creating the ptes).
Well if it's broken the zapping wouldn't work :-)
Quoted 7 lines
Quoted 1 line
didn't try the memory pressure thing yet.
I'm surprised ... and I have no idea how/why it keeps working.
For my paranoia, can you instrument the ttm page fault handler, and double check that we get new faults after the move, and set up new ptes for the same old mapping, but now pointing at the new place in vram?
Hmm, only the drm device mapping is faulted in after moving it, the dma-buf mapping is not. Fixed by:
Quoted 8 lines
Quoted 5 lines
Ah yes, that's more what I'd expect to happen, and the below is what I'd expect to fix things up. I think we should move it up ahead of the device callback (so that drivers can overwrite) and then push as a fix. Separate from a possible patch to undo the fake offset removal. -Daniel
Is there a more elegant way than copying the intel list on non-intel patches to kick intel ci?
Nope, not really.
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 33 lines · Daniel Vetter
Quoted 23 lines · Gerd Hoffmann
Hi,
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines
VRAM helpers use
drm_gem_ttm_mmap(), which wrapsttm_bo_mmap_obj(). These changes should be transparent.There's still the issue that for dma-buf mmap vs drm mmap you use different
f_mapping, which means ttm's pte shootdown won't work correctly for dma-buf mmaps. But yeah normal operation for ttm/vram helpers should be fine.Quoted 7 lines
Quoted 7 lines
Quoted 1 line
VRAM helpers don't support dmabufs.
It's not that simple. Even when not supporting dma-buf export and import it is still possible to create dma-bufs, import them into the same driver (which doesn't actually export+import but just grabs a gem object reference instead) and also to mmap them via prime/dma-buf code path ...
... but after looking again I think we are all green here. Given that only self-import works we'll only see vram gem objects in the mmap code path, which should have everything set up correctly. Same goes for qxl.
Quoted 7 lines
All other ttm drivers still use the old mmap code path, so all green there too I think. Also I somehow doubt dma-buf mmap vs. drm mmap ends up using different
f_mapping, ttm code has aWARN_ONinttm_bo_vm_open()which would fire should that be the case.Do imported dma-bufs hit the drm mmap code path in the first place? Wouldn't mmap be handled by the exporting driver?
Quoted 7 lines
drm_gem_dmabuf_mmap->obj->funcs->mmap->ttm_bo_mmap_objAnd I'm not seeing anyone adjusting
vm_file->f_mappinganywhere here at all.Note to hit this you need userspace to
- handle2fd on a buffer to create a dma-buf fd
- call mmap directly on that dma-buf fd
That was exactly the vgem IGT test that prompted $subject fix. (With vgem modified to use shmem helpers)
Rob
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 36 lines · Rob Herring
Quoted 35 lines · Daniel Vetter
Quoted 23 lines · Gerd Hoffmann
Hi,
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines
VRAM helpers use
drm_gem_ttm_mmap(), which wrapsttm_bo_mmap_obj(). These changes should be transparent.There's still the issue that for dma-buf mmap vs drm mmap you use different
f_mapping, which means ttm's pte shootdown won't work correctly for dma-buf mmaps. But yeah normal operation for ttm/vram helpers should be fine.Quoted 7 lines
Quoted 7 lines
Quoted 1 line
VRAM helpers don't support dmabufs.
It's not that simple. Even when not supporting dma-buf export and import it is still possible to create dma-bufs, import them into the same driver (which doesn't actually export+import but just grabs a gem object reference instead) and also to mmap them via prime/dma-buf code path ...
... but after looking again I think we are all green here. Given that only self-import works we'll only see vram gem objects in the mmap code path, which should have everything set up correctly. Same goes for qxl.
Quoted 7 lines
All other ttm drivers still use the old mmap code path, so all green there too I think. Also I somehow doubt dma-buf mmap vs. drm mmap ends up using different
f_mapping, ttm code has aWARN_ONinttm_bo_vm_open()which would fire should that be the case.Do imported dma-bufs hit the drm mmap code path in the first place? Wouldn't mmap be handled by the exporting driver?
drm_gem_dmabuf_mmap->obj->funcs->mmap->ttm_bo_mmap_obj
Quoted 8 lines
Quoted 5 lines
And I'm not seeing anyone adjusting
vm_file->f_mappinganywhere here at all.Note to hit this you need userspace to
- handle2fd on a buffer to create a dma-buf fd
- call mmap directly on that dma-buf fdThat was exactly the vgem IGT test that prompted $subject fix. (With vgem modified to use shmem helpers)
See my other mail, this only hits the right mmap paths. For the unmap side you need even more (and that part is driver dependent, and vgem would need some debug tricks to expose that for testing).
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel