drm/nouveau: fix takedown in move_notify
Subsystems (3)
PATCH 1/1 drm/nouveau: fix takedown in move_notify
move_notify is called by ttm after the the object is idle and about to be destroyed. Clean up the vm list properly in that case.
This is not a problem right now, since the list should already be empty, but if it wasn't empty, vm_put was not called which leads to random corruption later.
With this fix, nouveau_gem_object_close can be safely changed to a noop, forcing the vm bindings to be removed when the original object is. This is not done in this patch since it may lead to the object staying mapped in the vm space until the gem object refcount drops to 0. This shouldn't be a big issue however.
If we choose to do so does allow us to use ttm's delayed destruction mechanism to unmap vm after object is idle, resulting in ioread32 no longer taking up 30% of cpu in Team Fortress 2 if I don't do the vma unmap in nouveau_gem_object_close.
---
drivers/gpu/drm/nouveau/nouveau_bo.c +14 -6
@@ -1139,12 +1139,22 @@ nouveau_bo_move_ntfy(struct ttm_buffer_object *bo, struct ttm_mem_reg *new_mem)if (bo->destroy != nouveau_bo_del_ttm)return ; +if (!new_mem) { +while (!list_empty (&nvbo->vma_list)) { + vma = list_first_entry(&nvbo->vma_list, struct nouveau_vma, head); + +nouveau_vm_unmap (vma); +nouveau_vm_put (vma); +list_del (&vma->head); + } +return ; + } +list_for_each_entry (vma, &nvbo->vma_list, head) { -if (new_mem && new_mem->mem_type ==TTM_PL_VRAM ) { +if (new_mem->mem_type ==TTM_PL_VRAM ) {nouveau_vm_map (vma, new_mem->mm_node); - }else -if (new_mem && new_mem->mem_type ==TTM_PL_TT && - nvbo->page_shift == vma->vm->vmm->spg_shift) { + }else if (new_mem->mem_type ==TTM_PL_TT && + nvbo->page_shift == vma->vm->vmm->spg_shift) {if (((struct nouveau_mem *)new_mem->mm_node)->sg) nouveau_vm_map_sg_table(vma, 0, new_mem-> num_pages << PAGE_SHIFT,
@@ -1153,8 +1163,6 @@ nouveau_bo_move_ntfy(struct ttm_buffer_object *bo, struct ttm_mem_reg *new_mem)nouveau_vm_map_sg (vma,0 , new_mem-> num_pages <<PAGE_SHIFT , new_mem->mm_node); - }else { -nouveau_vm_unmap (vma); } } }
Op 21-11-12 14:15, Maarten Lankhorst schreef:
Quoted 10 lines
With this fix,
nouveau_gem_object_closecan be safely changed to a noop, forcing the vm bindings to be removed when the original object is. This is not done in this patch since it may lead to the object staying mapped in the vm space until the gem object refcount drops to 0. This shouldn't be a big issue however.If we choose to do so does allow us to use ttm's delayed destruction mechanism to unmap vm after object is idle, resulting in ioread32 no longer taking up 30% of cpu in Team Fortress 2 if I don't do the vma unmap in
nouveau_gem_object_close.
And for those crazy enough to try, deliberately in the most hacky form possible:
drm/nouveau: use ttm delayed destroy for unmapping vm
According to perf top, drops ioread32 from taking 30% cpu to 3% cpu..
drivers/gpu/drm/nouveau/nouveau_gem.c +1 -1
@@ -111,7 +111,7 @@ nouveau_gem_object_close(struct drm_gem_object *gem, struct drm_file *file_priv)struct nouveau_vma *vma;int ret; -if (!cli->base.vm) +// if (!cli->base.vm) return ; ret =ttm_bo_reserve (&nvbo->bo, false, false, false,0 );
A message that is not in the archive (sent off-list or not yet received): [email protected]
Emil Velikov
Tested-byQuoted 13 lines · Maarten Lankhorst
move_notifyis called by ttm after the the object is idle and about to be destroyed. Clean up the vm list properly in that case.This is not a problem right now, since the list should already be empty, but if it wasn't empty,
vm_putwas not called which leads to random corruption later.With this fix,
nouveau_gem_object_closecan be safely changed to a noop, forcing the vm bindings to be removed when the original object is. This is not done in this patch since it may lead to the object staying mapped in the vm space until the gem object refcount drops to 0. This shouldn't be a big issue however.
Quoted 7 lines
If we choose to do so does allow us to use ttm's delayed destruction mechanism to unmap vm after object is idle, resulting in ioread32 no longer taking up 30% of cpu in Team Fortress 2 if I don't do the vma unmap in
nouveau_gem_object_close.Signed-off-by Maarten Lankhorst <[email protected]>
Hi Maarten
I've noticed ioread32 of up-to 40% on of cpu my system. With your patch if goes down to ~3% with no side effects. Frame-rate appears to be slightly improved, although it's hard to say.
Is there any reason for holding this patch ?
For what it's worth you can add
Quoted 47 lines
---
drivers/gpu/drm/nouveau/nouveau_bo.c +14
-6@@ -1139,12 +1139,22 @@ nouveau_bo_move_ntfy(struct ttm_buffer_object *bo, struct ttm_mem_reg *new_mem)if (bo->destroy != nouveau_bo_del_ttm)return ; +if (!new_mem) { +while (!list_empty (&nvbo->vma_list)) { + vma = list_first_entry(&nvbo->vma_list, struct nouveau_vma, head); + +nouveau_vm_unmap (vma); +nouveau_vm_put (vma); +list_del (&vma->head); + } +return ; + } +list_for_each_entry (vma, &nvbo->vma_list, head) { -if (new_mem && new_mem->mem_type ==TTM_PL_VRAM ) { +if (new_mem->mem_type ==TTM_PL_VRAM ) {nouveau_vm_map (vma, new_mem->mm_node); - }else -if (new_mem && new_mem->mem_type ==TTM_PL_TT && - nvbo->page_shift == vma->vm->vmm->spg_shift) { + }else if (new_mem->mem_type ==TTM_PL_TT && + nvbo->page_shift == vma->vm->vmm->spg_shift) {if (((struct nouveau_mem *)new_mem->mm_node)->sg) nouveau_vm_map_sg_table(vma, 0, new_mem-> num_pages << PAGE_SHIFT,@@ -1153,8 +1163,6 @@ nouveau_bo_move_ntfy(struct ttm_buffer_object *bo, struct ttm_mem_reg *new_mem)nouveau_vm_map_sg (vma,0 , new_mem-> num_pages <<PAGE_SHIFT , new_mem->mm_node); - }else { -nouveau_vm_unmap (vma); } } }_______________________________________________
Nouveau mailing list [email protected]
http://lists.freedesktop.org/mailman/listinfo/nouveau
Op 10-12-12 17:39, Emil Velikov schreef:
Quoted 23 lines · Maarten Lankhorst
Quoted 13 lines
move_notifyis called by ttm after the the object is idle and about to be destroyed. Clean up the vm list properly in that case.This is not a problem right now, since the list should already be empty, but if it wasn't empty,
vm_putwas not called which leads to random corruption later.With this fix,
nouveau_gem_object_closecan be safely changed to a noop, forcing the vm bindings to be removed when the original object is. This is not done in this patch since it may lead to the object staying mapped in the vm space until the gem object refcount drops to 0. This shouldn't be a big issue however.Quoted 7 lines
If we choose to do so does allow us to use ttm's delayed destruction mechanism to unmap vm after object is idle, resulting in ioread32 no longer taking up 30% of cpu in Team Fortress 2 if I don't do the vma unmap in
nouveau_gem_object_close.Signed-off-by Maarten Lankhorst <[email protected]>Hi Maarten
Quoted 8 lines
I've noticed ioread32 of up-to 40% on of cpu my system. With your patch if goes down to ~3% with no side effects. Frame-rate appears to be slightly improved, although it's hard to say.
Is there any reason for holding this patch ?
For what it's worth you can addTested-by Emil Velikov <[email protected]>
Well 2 reasons..
1) In the current form, you're guaranteed to cause memory corruption on forced client takedown. Locking needs more thought. The warns you get are very real errors. ;-)
Quoted 9 lines
Quoted 9 lines
---
drivers/gpu/drm/nouveau/nouveau_bo.c +0
-0@@ -1139,12 +1139,22 @@ nouveau_bo_move_ntfy(struct ttm_buffer_object *bo, struct ttm_mem_reg *new_mem)if (bo->destroy != nouveau_bo_del_ttm)return ;
Quoted 7 lines
drivers/gpu/drm/nouveau/nouveau_bo.c nouveau_bo_move_ntfy(struct ttm_buffer_object *bo, struct ttm_mem_reg *new_mem) · line 1148
Quoted 7 lines
drivers/gpu/drm/nouveau/nouveau_bo.c +7
-0+if (!new_mem) { +while (!list_empty (&nvbo->vma_list)) { + vma = list_first_entry(&nvbo->vma_list, struct nouveau_vma, head); + +nouveau_vm_unmap (vma); +nouveau_vm_put (vma); +list_del (&vma->head);
2. I missed a kfree here ;-)
~Maarten