Skip to contentSkip to conversation

drm/amdgpu: user pages array memory leak fix

PATCH In review Yang, Philip 3 participants 4 messages Oct 11, 2019 – Oct 12, 2019 2 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 drm/amdgpu: user pages array memory leak fix

user_pages array should always be freed after validation regardless if user pages are changed after bo is created because with HMM change parse bo always allocate user pages array to get user pages for userptr bo.

v2: remove unused local variable and amend commit

v3: add back get user pages in gem_userptr_ioctl, to detect application bug where an userptr VMA is not ananymous memory and reject it.

Bugzilla: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1844962

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -3
index c18a153b3d2a..e7b39daa22f6 100644
@@ -476,7 +476,6 @@ static int amdgpu_cs_list_validate(struct amdgpu_cs_parser *p,
 
 	list_for_each_entry(lobj, validated, tv.head) {
 		struct amdgpu_bo *bo = ttm_to_amdgpu_bo(lobj->tv.bo);
-		bool binding_userptr = false;
 		struct mm_struct *usermm;
 
 		usermm = amdgpu_ttm_tt_get_usermm(bo->tbo.ttm);
@@ -493,14 +492,13 @@ static int amdgpu_cs_list_validate(struct amdgpu_cs_parser *p,
 
 			amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm,
 						     lobj->user_pages);
-			binding_userptr = true;
 		}
 
 		r = amdgpu_cs_validate(p, bo);
 		if (r)
 			return r;
 
-		if (binding_userptr) {
+		if (lobj->user_pages) {
 			kvfree(lobj->user_pages);
 			lobj->user_pages = NULL;
 		}
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): [email protected]

Reviewed-by Felix Kuehling
#2 lore raw
Quoted 33 lines · Philip

user_pages array should always be freed after validation regardless if user pages are changed after bo is created because with HMM change parse bo always allocate user pages array to get user pages for userptr bo.

v2: remove unused local variable and amend commit

v3: add back get user pages in gem_userptr_ioctl, to detect application bug where an userptr VMA is not ananymous memory and reject it.

Bugzilla: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1844962

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 4 +---
  1 file changed, 1 insertion(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +0 -1
index c18a153b3d2a..e7b39daa22f6 100644
@@ -476,7 +476,6 @@ static int amdgpu_cs_list_validate(struct amdgpu_cs_parser *p,
 
  	list_for_each_entry(lobj, validated, tv.head) {
  		struct amdgpu_bo *bo = ttm_to_amdgpu_bo(lobj->tv.bo);
-		bool binding_userptr = false;
  		struct mm_struct *usermm;
 
  		usermm = amdgpu_ttm_tt_get_usermm(bo->tbo.ttm);
@@ -493,14 +492,13 @@ static int amdgpu_cs_list_validate(struct amdgpu_cs_parser *p,
 
  			amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm,

drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c static int amdgpu_cs_list_validate(struct amdgpu_cs_parser *p, · line 501

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -2
  						     lobj->user_pages);
-			binding_userptr = true;
  		}
 
  		r = amdgpu_cs_validate(p, bo);
  		if (r)
  			return r;
 
-		if (binding_userptr) {
+		if (lobj->user_pages) {

This if is not needed. kvfree should be able to handle NULL pointers, and unconditionally setting the pointer to NULL afterwards is not a problem either. With that fixed, this commit is

However, I don't think this should be the final solution. My concern with this solution is, that you end up freeing and regenerating the user_pages arrays more frequently than necessary: On every command submission, even if there was no MMU notifier since the last command submission. I was hoping we could get back to a solution where we can maintain the same user_pages array across command submissions, since MMU notifiers are rare. That should reduce overhead from doing all thos page table walks in HMM on every command submissions when using userptrs.

Regards,
  Felix

drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c static int amdgpu_cs_list_validate(struct amdgpu_cs_parser *p, · line 504

Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +0 -0
  			kvfree(lobj->user_pages);
  			lobj->user_pages = NULL;
  		}

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 9abf4c92-a30c-ae3d-5544-6428c3aec2ce-5C7GfCeVMHo@public.gmane.org

#4 lore raw
Quoted 51 lines · Felix
Quoted 33 lines · Philip

user_pages array should always be freed after validation regardless if user pages are changed after bo is created because with HMM change parse bo always allocate user pages array to get user pages for userptr bo.

v2: remove unused local variable and amend commit

v3: add back get user pages in gem_userptr_ioctl, to detect application bug where an userptr VMA is not ananymous memory and reject it.

Bugzilla: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1844962

---
   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 4 +---
   1 file changed, 1 insertion(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +0 -1
index c18a153b3d2a..e7b39daa22f6 100644
@@ -476,7 +476,6 @@ static int amdgpu_cs_list_validate(struct amdgpu_cs_parser *p,
 
   	list_for_each_entry(lobj, validated, tv.head) {
   		struct amdgpu_bo *bo = ttm_to_amdgpu_bo(lobj->tv.bo);
-		bool binding_userptr = false;
   		struct mm_struct *usermm;
 
   		usermm = amdgpu_ttm_tt_get_usermm(bo->tbo.ttm);
@@ -493,14 +492,13 @@ static int amdgpu_cs_list_validate(struct amdgpu_cs_parser *p,
 
   			amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm,

drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c static int amdgpu_cs_list_validate(struct amdgpu_cs_parser *p, · line 501

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -2
   						     lobj->user_pages);
-			binding_userptr = true;
   		}
 
   		r = amdgpu_cs_validate(p, bo);
   		if (r)
   			return r;
 
-		if (binding_userptr) {
+		if (lobj->user_pages) {

This if is not needed. kvfree should be able to handle NULL pointers, and unconditionally setting the pointer to NULL afterwards is not a problem either. With that fixed, this commit is

Quoted 9 lines

However, I don't think this should be the final solution. My concern with this solution is, that you end up freeing and regenerating the user_pages arrays more frequently than necessary: On every command submission, even if there was no MMU notifier since the last command submission. I was hoping we could get back to a solution where we can maintain the same user_pages array across command submissions, since MMU notifiers are rare. That should reduce overhead from doing all thos page table walks in HMM on every command submissions when using userptrs.

Yes, I will have another patch to address this using hmm_range_valid, the idea is to allow hmm range tracking cross gem_userptr_ioctl and cs_ioctl.

Thanks,
Philip

Quoted 7 lines

Regards,
  Felix

Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +0 -0
   			kvfree(lobj->user_pages);
   			lobj->user_pages = NULL;
   		}

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#5 lore raw
In reply to a message not in the archive

Confirming that v3 patch still fixes the bug.

Thanks,
-Joe

Quoted 49 lines · Philip

user_pages array should always be freed after validation regardless if user pages are changed after bo is created because with HMM change parse bo always allocate user pages array to get user pages for userptr bo.

v2: remove unused local variable and amend commit

v3: add back get user pages in gem_userptr_ioctl, to detect application bug where an userptr VMA is not ananymous memory and reject it.

Bugzilla: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1844962

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +0 -1
index c18a153b3d2a..e7b39daa22f6 100644
@@ -476,7 +476,6 @@ static int amdgpu_cs_list_validate(struct
 amdgpu_cs_parser *p,
 
        list_for_each_entry(lobj, validated, tv.head) {
                struct amdgpu_bo *bo = ttm_to_amdgpu_bo(lobj->tv.bo);
-               bool binding_userptr = false;
                struct mm_struct *usermm;
 
                usermm = amdgpu_ttm_tt_get_usermm(bo->tbo.ttm);
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -2
@@ -493,14 +492,13 @@ static int amdgpu_cs_list_validate(struct
 amdgpu_cs_parser *p,
 
                        amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm,
                                                     lobj->user_pages);
-                       binding_userptr = true;
                }
 
                r = amdgpu_cs_validate(p, bo);
                if (r)
                        return r;
 
-               if (binding_userptr) {
+               if (lobj->user_pages) {
                        kvfree(lobj->user_pages);
                        lobj->user_pages = NULL;
                }
Quoted 4 lines

--
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx