Skip to contentSkip to conversation

drm/amdgpu: Need to set moved to true when evict bo

PATCH In review Emily Deng 4 participants 7 messages Aug 28, 2018 – Aug 29, 2018 5 messages not in the archive

Subsystems (2)
lore
Share & tools
lore

Emily Deng

author
#0 lore raw

PATCH 1/1 drm/amdgpu: Need to set moved to true when evict bo

Fix the VMC page fault when the running sequence is as below:
1.amdgpu_gem_create_ioctl 2.ttm_bo_swapout->amdgpu_vm_bo_invalidate, as not called amdgpu_vm_bo_base_init, so won't called list_add_tail(&base->bo_list, &bo->va). Even the bo was evicted, it won't set the bo_base->moved. 3.drm_gem_open_ioctl->amdgpu_vm_bo_base_init, here only called list_move_tail(&base->vm_status, &vm->evicted), but not set the bo_base->moved. 4.amdgpu_vm_bo_map->amdgpu_vm_bo_insert_map, as the bo_base->moved is not set true, the function amdgpu_vm_bo_insert_map will call list_move(&bo_va->base.vm_status, &vm->moved) 5.amdgpu_cs_ioctl won't validate the swapout bo, as it is only in the moved list, not in the evict list. So VMC page fault occurs.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 1 +
 1 file changed, 1 insertion(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -0
index 1f4b8df..015e20e 100644
@@ -172,6 +172,7 @@ static void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base,
 	 * is validated on next vm use to avoid fault.
 	 * */
 	list_move_tail(&base->vm_status, &vm->evicted);
+	base->moved = true;
 }
 
 /**
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1535460039-11302-1-git-send-email-Emily.Deng-5C7GfCeVMHo@public.gmane.org

#2 lore raw

Am 28.08.2018 um 14:40 schrieb Emily Deng:

Quoted 15 lines

Fix the VMC page fault when the running sequence is as below:
1.amdgpu_gem_create_ioctl 2.ttm_bo_swapout->amdgpu_vm_bo_invalidate, as not called amdgpu_vm_bo_base_init, so won't called list_add_tail(&base->bo_list, &bo->va). Even the bo was evicted, it won't set the bo_base->moved. 3.drm_gem_open_ioctl->amdgpu_vm_bo_base_init, here only called list_move_tail(&base->vm_status, &vm->evicted), but not set the bo_base->moved. 4.amdgpu_vm_bo_map->amdgpu_vm_bo_insert_map, as the bo_base->moved is not set true, the function amdgpu_vm_bo_insert_map will call list_move(&bo_va->base.vm_status, &vm->moved) 5.amdgpu_cs_ioctl won't validate the swapout bo, as it is only in the moved list, not in the evict list. So VMC page fault occurs.

Quoted 1 line

Good catch, patch is Reviewed-by: Christian König <[email protected]>

Quoted 9 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 1 +
  1 file changed, 1 insertion(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0 -0
index 1f4b8df..015e20e 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c static void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, · line 178

Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -0
@@ -172,6 +172,7 @@ static void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base,
  	 * is validated on next vm use to avoid fault.
  	 * */
  	list_move_tail(&base->vm_status, &vm->evicted);
+	base->moved = true;
  }
 
  /**

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): ec7e701a-8df6-5699-65e1-78d4c4336cdb-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#4 lore raw
Quoted 16 lines · Christian König

Am 28.08.2018 um 14:40 schrieb Emily Deng:

Quoted 15 lines

Fix the VMC page fault when the running sequence is as below:
1.amdgpu_gem_create_ioctl 2.ttm_bo_swapout->amdgpu_vm_bo_invalidate, as not called amdgpu_vm_bo_base_init, so won't called list_add_tail(&base->bo_list, &bo->va). Even the bo was evicted, it won't set the bo_base->moved. 3.drm_gem_open_ioctl->amdgpu_vm_bo_base_init, here only called list_move_tail(&base->vm_status, &vm->evicted), but not set the bo_base->moved. 4.amdgpu_vm_bo_map->amdgpu_vm_bo_insert_map, as the bo_base->moved is not set true, the function amdgpu_vm_bo_insert_map will call list_move(&bo_va->base.vm_status, &vm->moved) 5.amdgpu_cs_ioctl won't validate the swapout bo, as it is only in the moved list, not in the evict list. So VMC page fault occurs.

Quoted 3 lines
Quoted 1 line

Good catch, patch is Reviewed-by: Christian König <[email protected]>

Really good debug, Emily, you can add my Reviewed-by: Chunming Zhou <[email protected]> as well if you still don't push it yet.

Regards,
David Zhou

Quoted 14 lines
Quoted 14 lines

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 1 +

  1 file changed, 1 insertion(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0 -0
index 1f4b8df..015e20e 100644
@@ -172,6 +172,7 @@ static void amdgpu_vm_bo_base_init(struct 
 amdgpu_vm_bo_base *base,
        * is validated on next vm use to avoid fault.
        * */
       list_move_tail(&base->vm_status, &vm->evicted);
Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c static void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, · line 178

Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -0
+    base->moved = true;
   }
     /**

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 584f9507-9f95-d00c-ff59-0920a08fef88-5C7GfCeVMHo@public.gmane.org

#6 lore raw
Quoted 27 lines

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of zhoucm1
Sent: Wednesday, August 29, 2018 10:12 AM
To: Koenig, Christian <[email protected]>; Deng, Emily <[email protected]>; [email protected]
Subject: Re: [PATCH] drm/amdgpu: Need to set moved to true when evict bo

Quoted 16 lines · Christian König

Am 28.08.2018 um 14:40 schrieb Emily Deng:

Quoted 9 lines

Fix the VMC page fault when the running sequence is as below:
1.amdgpu_gem_create_ioctl 2.ttm_bo_swapout->amdgpu_vm_bo_invalidate, as not called amdgpu_vm_bo_base_init, so won't called list_add_tail(&base->bo_list, &bo->va). Even the bo was evicted, it won't set the bo_base->moved. 3.drm_gem_open_ioctl->amdgpu_vm_bo_base_init, here only called list_move_tail(&base->vm_status, &vm->evicted), but not set the bo_base->moved. 4.amdgpu_vm_bo_map->amdgpu_vm_bo_insert_map, as the bo_base-

moved is

Quoted 5 lines

not set true, the function amdgpu_vm_bo_insert_map will call list_move(&bo_va->base.vm_status, &vm->moved) 5.amdgpu_cs_ioctl won't validate the swapout bo, as it is only in the moved list, not in the evict list. So VMC page fault occurs.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

Good catch, patch is Reviewed-by: Christian König
<[email protected]>

Really good debug, Emily, you can add my Reviewed-by: Chunming Zhou <[email protected]> as well if you still don't push it yet.

David, thanks you for helping me to find out the root cause, already pushed the patch.

Quoted 26 lines

Regards,
David Zhou

Quoted 15 lines
Quoted 15 lines

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 1 +

  1 file changed, 1 insertion(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0 -0
index 1f4b8df..015e20e 100644
@@ -172,6 +172,7 @@ static void amdgpu_vm_bo_base_init(struct
 amdgpu_vm_bo_base *base,
        * is validated on next vm use to avoid fault.
        * */
       list_move_tail(&base->vm_status, &vm->evicted);
Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c static void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, · line 178

Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -0
+    base->moved = true;
   }
     /**

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#7 lore raw
In reply to a message not in the archive
Quoted 6 lines · Emily Deng

Fix the VMC page fault when the running sequence is as below:
1.amdgpu_gem_create_ioctl 2.ttm_bo_swapout->amdgpu_vm_bo_invalidate, as not called amdgpu_vm_bo_base_init, so won't called list_add_tail(&base->bo_list, &bo->va). Even the bo was evicted, it won't set the bo_base->moved.

IMO, the evicted bo should be created previously. On BO creation we will add it to the bo->va as below:

amdgpu_gem_create_ioctl
drm_gem_handle_create

    amdgpu_gem_object_open
      amdgpu_vm_bo_add
        amdgpu_vm_bo_base_init
          list_add_tail(&base->bo_list, &bo->va)

Then it could be set moved in bo invalidate when evicting.

could you provide a bit more backgroud about the issue? looks a per vm bo is evicted and a new same bo created.

Jerry

Quoted 20 lines

3.drm_gem_open_ioctl->amdgpu_vm_bo_base_init, here only called list_move_tail(&base->vm_status, &vm->evicted), but not set the bo_base->moved. 4.amdgpu_vm_bo_map->amdgpu_vm_bo_insert_map, as the bo_base->moved is not set true, the function amdgpu_vm_bo_insert_map will call list_move(&bo_va->base.vm_status, &vm->moved) 5.amdgpu_cs_ioctl won't validate the swapout bo, as it is only in the moved list, not in the evict list. So VMC page fault occurs.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 1 +
  1 file changed, 1 insertion(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0 -0
index 1f4b8df..015e20e 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c static void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, · line 178

Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -0
@@ -172,6 +172,7 @@ static void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base,
  	 * is validated on next vm use to avoid fault.
  	 * */
  	list_move_tail(&base->vm_status, &vm->evicted);
+	base->moved = true;
  }
 
  /**

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): [email protected]

#9 lore raw

Am 29.08.2018 um 04:52 schrieb Zhang, Jerry (Junwei):

Quoted 8 lines · Emily Deng
Quoted 6 lines

Fix the VMC page fault when the running sequence is as below:
1.amdgpu_gem_create_ioctl 2.ttm_bo_swapout->amdgpu_vm_bo_invalidate, as not called amdgpu_vm_bo_base_init, so won't called list_add_tail(&base->bo_list, &bo->va). Even the bo was evicted, it won't set the bo_base->moved.

Quoted 6 lines

IMO, the evicted bo should be created previously. On BO creation we will add it to the bo->va as below:

amdgpu_gem_create_ioctl
  drm_gem_handle_create
    amdgpu_gem_object_open

And here is the problem. Between creating the BO and opening it in the client the BO can be evicted.

That's what Emily's patch is handling here.

Christian.

Quoted 37 lines

amdgpu_vm_bo_add
        amdgpu_vm_bo_base_init
          list_add_tail(&base->bo_list, &bo->va)

Then it could be set moved in bo invalidate when evicting.

could you provide a bit more backgroud about the issue? looks a per vm bo is evicted and a new same bo created.

Jerry

Quoted 26 lines

3.drm_gem_open_ioctl->amdgpu_vm_bo_base_init, here only called list_move_tail(&base->vm_status, &vm->evicted), but not set the bo_base->moved. 4.amdgpu_vm_bo_map->amdgpu_vm_bo_insert_map, as the bo_base->moved is not set true, the function amdgpu_vm_bo_insert_map will call list_move(&bo_va->base.vm_status, &vm->moved) 5.amdgpu_cs_ioctl won't validate the swapout bo, as it is only in the moved list, not in the evict list. So VMC page fault occurs.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 1 +

  1 file changed, 1 insertion(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -0
index 1f4b8df..015e20e 100644
@@ -172,6 +172,7 @@ static void amdgpu_vm_bo_base_init(struct 
 amdgpu_vm_bo_base *base,
        * is validated on next vm use to avoid fault.
        * */
       list_move_tail(&base->vm_status, &vm->evicted);
+    base->moved = true;
   }
 
Quoted 6 lines
Quoted 2 lines

  /**

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): fe2eec30-a4f7-e877-26c1-32a7b8982f31-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#11 lore raw
Quoted 9 lines · Christian König

Am 29.08.2018 um 04:52 schrieb Zhang, Jerry (Junwei):

Quoted 8 lines · Emily Deng
Quoted 6 lines

Fix the VMC page fault when the running sequence is as below:
1.amdgpu_gem_create_ioctl 2.ttm_bo_swapout->amdgpu_vm_bo_invalidate, as not called amdgpu_vm_bo_base_init, so won't called list_add_tail(&base->bo_list, &bo->va). Even the bo was evicted, it won't set the bo_base->moved.

Quoted 8 lines
Quoted 6 lines

IMO, the evicted bo should be created previously. On BO creation we will add it to the bo->va as below:

amdgpu_gem_create_ioctl
drm_gem_handle_create
amdgpu_gem_object_open

And here is the problem. Between creating the BO and opening it in the client the BO can be evicted.

Thanks to explain that.

It's the key point, falling in Murphy's law as well.

Jerry

Quoted 48 lines

That's what Emily's patch is handling here.

Christian.

Quoted 42 lines

amdgpu_vm_bo_add

        amdgpu_vm_bo_base_init
          list_add_tail(&base->bo_list, &bo->va)

Then it could be set moved in bo invalidate when evicting.

could you provide a bit more backgroud about the issue? looks a per vm bo is evicted and a new same bo created.

Jerry

Quoted 20 lines

3.drm_gem_open_ioctl->amdgpu_vm_bo_base_init, here only called list_move_tail(&base->vm_status, &vm->evicted), but not set the bo_base->moved. 4.amdgpu_vm_bo_map->amdgpu_vm_bo_insert_map, as the bo_base->moved is not set true, the function amdgpu_vm_bo_insert_map will call list_move(&bo_va->base.vm_status, &vm->moved) 5.amdgpu_cs_ioctl won't validate the swapout bo, as it is only in the moved list, not in the evict list. So VMC page fault occurs.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 1 +
  1 file changed, 1 insertion(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0 -0
index 1f4b8df..015e20e 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c static void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, · line 178

Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -0
@@ -172,6 +172,7 @@ static void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base,
       * is validated on next vm use to avoid fault.
       * */
      list_move_tail(&base->vm_status, &vm->evicted);
+    base->moved = true;
  }
 
  /**

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx