Skip to contentSkip to conversation

drm/amdgpu: fix error handling in amdgpu_bo_do_create

PATCH In review Christian König 3 participants 7 messages Oct 31, 2017 5 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/amdgpu: fix error handling in amdgpu_bo_do_create

From: Christian König <[email protected]>

The bo structure is freed up in case of an error, so we can't do any accounting if that happens.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +3 -3
index e44b880eefdd..ff6f842655d1 100644
@@ -369,6 +369,9 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
 	r = ttm_bo_init_reserved(&adev->mman.bdev, &bo->tbo, size, type,
 				 &bo->placement, page_align, !kernel, NULL,
 				 acc_size, sg, resv, &amdgpu_ttm_bo_destroy);
+	if (unlikely(r != 0))
+		return r;
+
 	bytes_moved = atomic64_read(&adev->num_bytes_moved) -
 		      initial_bytes_moved;
 	if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
@@ -378,9 +381,6 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
 	else
 		amdgpu_cs_report_moved_bytes(adev, bytes_moved, 0);
 
-	if (unlikely(r != 0))
-		return r;
-
 	if (kernel)
 		bo->tbo.priority = 1;
 
Signature · 6 lines
2.11.0

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20171031083725.1687-1-deathsimple-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org

Reviewed-by
#2 lore raw
Quoted 26 lines · Christian König

From: Christian König <[email protected]>

The bo structure is freed up in case of an error, so we can't do any accounting if that happens.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +3 -0
index e44b880eefdd..ff6f842655d1 100644
@@ -369,6 +369,9 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
 	r = ttm_bo_init_reserved(&adev->mman.bdev, &bo->tbo, size, type,
 				 &bo->placement, page_align, !kernel, NULL,
 				 acc_size, sg, resv, &amdgpu_ttm_bo_destroy);
+	if (unlikely(r != 0))
+		return r;
+
 	bytes_moved = atomic64_read(&adev->num_bytes_moved) -
 		      initial_bytes_moved;
 	if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c static int amdgpu_bo_do_create(struct amdgpu_device *adev, · line 386

Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -3
@@ -378,9 +381,6 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
 	else
 		amdgpu_cs_report_moved_bytes(adev, bytes_moved, 0);
 
-	if (unlikely(r != 0))
-		return r;
-
 	if (kernel)
 		bo->tbo.priority = 1;
 

Hmm. If it's at least theoretically possible that ttm_bo_init_reserved moves some BOs before returning an error, we should instead make the accounting safe WRT bo having been freed. But assuming it's not possible,

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 32a42bfb-45ef-fcf9-a4b2-701eff867b94-otUistvHUpPR7s880joybQ@public.gmane.org

#4 lore raw

Am 31.10.2017 um 09:43 schrieb Michel Dänzer:

Quoted 29 lines · Christian König
Quoted 28 lines

From: Christian König <[email protected]>

The bo structure is freed up in case of an error, so we can't do any accounting if that happens.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +3 -0
index e44b880eefdd..ff6f842655d1 100644
@@ -369,6 +369,9 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
  	r = ttm_bo_init_reserved(&adev->mman.bdev, &bo->tbo, size, type,
  				 &bo->placement, page_align, !kernel, NULL,
  				 acc_size, sg, resv, &amdgpu_ttm_bo_destroy);
+	if (unlikely(r != 0))
+		return r;
+
  	bytes_moved = atomic64_read(&adev->num_bytes_moved) -
  		      initial_bytes_moved;
  	if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
@@ -378,9 +381,6 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
  	else
  		amdgpu_cs_report_moved_bytes(adev, bytes_moved, 0);
 
Quoted 10 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c static int amdgpu_bo_do_create(struct amdgpu_device *adev, · line 386

Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -3
-	if (unlikely(r != 0))
-		return r;
-
  	if (kernel)
  		bo->tbo.priority = 1;
 

Hmm. If it's at least theoretically possible that ttm_bo_init_reserved moves some BOs before returning an error, we should instead make the accounting safe WRT bo having been freed. But assuming it's not possible,

I have made patches for this back in April and send them to the list, but the value is rather low and you need to change a lot of places in other drivers as well.

So I've gave up on that set for now because of lack of time.

Quoted 1 line

Thanks for the review,
Christian.

Quoted 2 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 72441e7f-836a-f298-5b0e-e12e4d317c2a-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#6 lore raw
Quoted 42 lines · Christian König

Am 31.10.2017 um 09:43 schrieb Michel Dänzer:

Quoted 32 lines · Christian König
Quoted 31 lines

From: Christian König <[email protected]>

The bo structure is freed up in case of an error, so we can't do any accounting if that happens.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 6 +++---

  1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +3 -0
index e44b880eefdd..ff6f842655d1 100644
@@ -369,6 +369,9 @@ static int amdgpu_bo_do_create(struct
 amdgpu_device *adev,
       r = ttm_bo_init_reserved(&adev->mman.bdev, &bo->tbo, size, type,
                    &bo->placement, page_align, !kernel, NULL,
                    acc_size, sg, resv, &amdgpu_ttm_bo_destroy);
+    if (unlikely(r != 0))
+        return r;
+

      bytes_moved = atomic64_read(&adev->num_bytes_moved) - initial_bytes_moved;
      if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -0
@@ -378,9 +381,6 @@ static int amdgpu_bo_do_create(struct
 amdgpu_device *adev,

      else
          amdgpu_cs_report_moved_bytes(adev, bytes_moved, 0); -    if (unlikely(r != 0))

Quoted 8 lines
Quoted 5 lines

-        return r;
-
      if (kernel)
          bo->tbo.priority = 1;

Hmm. If it's at least theoretically possible that ttm_bo_init_reserved moves some BOs before returning an error, we should instead make the accounting safe WRT bo having been freed. But assuming it's not possible,

Quoted 3 lines

I have made patches for this back in April and send them to the list, but the value is rather low and you need to change a lot of places in other drivers as well.

At least in this case, simply moving the

	if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
	    bo->tbo.mem.mem_type == TTM_PL_VRAM &&
	    bo->tbo.mem.start < adev->mc.visible_vram_size >> PAGE_SHIFT)

test before the ttm_bo_init_reserved call, and then passing bytes_moved or 0 for the third parameter of amdgpu_cs_report_moved_bytes depending on its result, would work as well, wouldn't it?

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1a5b12d5-3d2c-a308-a954-7d45d9b93d22-otUistvHUpPR7s880joybQ@public.gmane.org

#8 lore raw
Quoted 47 lines · Michel Dänzer
Quoted 42 lines · Christian König

Am 31.10.2017 um 09:43 schrieb Michel Dänzer:

Quoted 32 lines · Christian König
Quoted 31 lines

From: Christian König <[email protected]>

The bo structure is freed up in case of an error, so we can't do any accounting if that happens.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 6 +++---

  1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +3 -0
index e44b880eefdd..ff6f842655d1 100644
@@ -369,6 +369,9 @@ static int amdgpu_bo_do_create(struct
 amdgpu_device *adev,
       r = ttm_bo_init_reserved(&adev->mman.bdev, &bo->tbo, size, type,
                    &bo->placement, page_align, !kernel, NULL,
                    acc_size, sg, resv, &amdgpu_ttm_bo_destroy);
+    if (unlikely(r != 0))
+        return r;
+

      bytes_moved = atomic64_read(&adev->num_bytes_moved) - initial_bytes_moved;
      if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -0
@@ -378,9 +381,6 @@ static int amdgpu_bo_do_create(struct
 amdgpu_device *adev,

      else
          amdgpu_cs_report_moved_bytes(adev, bytes_moved, 0); -    if (unlikely(r != 0))

Quoted 8 lines
Quoted 5 lines

-        return r;
-
      if (kernel)
          bo->tbo.priority = 1;

Hmm. If it's at least theoretically possible that ttm_bo_init_reserved moves some BOs before returning an error, we should instead make the accounting safe WRT bo having been freed. But assuming it's not possible,

Quoted 3 lines

I have made patches for this back in April and send them to the list, but the value is rather low and you need to change a lot of places in other drivers as well.

Quoted 7 lines

At least in this case, simply moving the

	if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
	    bo->tbo.mem.mem_type == TTM_PL_VRAM &&
	    bo->tbo.mem.start < adev->mc.visible_vram_size >> PAGE_SHIFT)

test before the ttm_bo_init_reserved call, [...]

... except that's not possible, because at least bo->tbo.mem.start isn't decided yet before calling ttm_bo_init_reserved. Never mind the noise.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#9 lore raw
In reply to a message not in the archive

Am 31.10.2017 um 16:14 schrieb Michel Dänzer:

Quoted 47 lines · Christian König
Quoted 41 lines

Am 31.10.2017 um 09:43 schrieb Michel Dänzer:

Quoted 32 lines · Christian König
Quoted 31 lines

From: Christian König <[email protected]>

The bo structure is freed up in case of an error, so we can't do any accounting if that happens.

---
   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 6 +++---
   1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +3 -0
index e44b880eefdd..ff6f842655d1 100644
@@ -369,6 +369,9 @@ static int amdgpu_bo_do_create(struct
 amdgpu_device *adev,
       r = ttm_bo_init_reserved(&adev->mman.bdev, &bo->tbo, size, type,
                    &bo->placement, page_align, !kernel, NULL,
                    acc_size, sg, resv, &amdgpu_ttm_bo_destroy);
+    if (unlikely(r != 0))
+        return r;
+
       bytes_moved = atomic64_read(&adev->num_bytes_moved) -
                 initial_bytes_moved;

      if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -0
@@ -378,9 +381,6 @@ static int amdgpu_bo_do_create(struct
 amdgpu_device *adev,
       else
           amdgpu_cs_report_moved_bytes(adev, bytes_moved, 0);
   -    if (unlikely(r != 0))
Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c static int amdgpu_bo_do_create(struct · line 386

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -2
-        return r;
-
       if (kernel)
           bo->tbo.priority = 1;

Hmm. If it's at least theoretically possible that ttm_bo_init_reserved moves some BOs before returning an error, we should instead make the accounting safe WRT bo having been freed. But assuming it's not possible,

Quoted 3 lines

I have made patches for this back in April and send them to the list, but the value is rather low and you need to change a lot of places in other drivers as well.

At least in this case, simply moving the

Quoted 7 lines
	if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
	    bo->tbo.mem.mem_type == TTM_PL_VRAM &&
	    bo->tbo.mem.start < adev->mc.visible_vram_size >> PAGE_SHIFT)

test before the ttm_bo_init_reserved call, and then passing bytes_moved or 0 for the third parameter of amdgpu_cs_report_moved_bytes depending on its result, would work as well, wouldn't it?

No, that won't work. Neither bo->tbo.mem.mem_type nor bo->tbo.mem.start are determined yet before the call to ttm_bo_init().

The root problem is that we have a mis-assumption about how visible VRAM works here (e.g. it doesn't handle the case where a BO is partially in visible VRAM) and that we use the global bytes moved counter as indicator for the local operation (e.g. to know how many bytes moved the current client).

The only real solution is to give ttm_bo_init_* and ttm_bo_validate a context parameter where we can note how much work we had to do to fulfill the request.

Regards,
Christian.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 02b789ad-754e-b241-5742-77f99327ca88-5C7GfCeVMHo@public.gmane.org

#11 lore raw
Quoted 66 lines · Christian König

Am 31.10.2017 um 16:14 schrieb Michel Dänzer:

Quoted 49 lines · Christian König
Quoted 43 lines

Am 31.10.2017 um 09:43 schrieb Michel Dänzer:

Quoted 34 lines · Christian König
Quoted 33 lines

From: Christian König <[email protected]>

The bo structure is freed up in case of an error, so we can't do any accounting if that happens.

---
   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 6 +++---

   1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +3 -0
index e44b880eefdd..ff6f842655d1 100644
@@ -369,6 +369,9 @@ static int amdgpu_bo_do_create(struct
 amdgpu_device *adev,
        r = ttm_bo_init_reserved(&adev->mman.bdev, &bo->tbo, size,
 type,
                     &bo->placement, page_align, !kernel, NULL,
                     acc_size, sg, resv, &amdgpu_ttm_bo_destroy);
+    if (unlikely(r != 0))
+        return r;
+

       bytes_moved = atomic64_read(&adev->num_bytes_moved) - initial_bytes_moved;
       if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -0
@@ -378,9 +381,6 @@ static int amdgpu_bo_do_create(struct
 amdgpu_device *adev,

       else
           amdgpu_cs_report_moved_bytes(adev, bytes_moved, 0); -    if (unlikely(r != 0))
-        return r;

Quoted 8 lines
Quoted 4 lines

-
       if (kernel)
           bo->tbo.priority = 1;

Hmm. If it's at least theoretically possible that ttm_bo_init_reserved moves some BOs before returning an error, we should instead make the accounting safe WRT bo having been freed. But assuming it's not possible,

Quoted 3 lines

I have made patches for this back in April and send them to the list, but the value is rather low and you need to change a lot of places in other drivers as well.

At least in this case, simply moving the

Quoted 7 lines

    if (adev->mc.visible_vram_size < adev->mc.real_vram_size && bo->tbo.mem.mem_type == TTM_PL_VRAM && bo->tbo.mem.start < adev->mc.visible_vram_size >> PAGE_SHIFT)

test before the ttm_bo_init_reserved call, and then passing bytes_moved or 0 for the third parameter of amdgpu_cs_report_moved_bytes depending on its result, would work as well, wouldn't it?

No, that won't work. Neither bo->tbo.mem.mem_type nor bo->tbo.mem.start are determined yet before the call to ttm_bo_init().

The root problem is that we have a mis-assumption about how visible VRAM works here (e.g. it doesn't handle the case where a BO is partially in visible VRAM) and that we use the global bytes moved counter as indicator for the local operation (e.g. to know how many bytes moved the current client).

Quoted 3 lines

The only real solution is to give ttm_bo_init_* and ttm_bo_validate a context parameter where we can note how much work we had to do to fulfill the request.

Right, though in the meantime I wonder if over-accounting (i.e. calling amdgpu_cs_report_moved_bytes(adev, bytes_moved, bytes_moved)) in the failure case wouldn't be better than not accounting at all.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx