Skip to contentSkip to conversation

drm/amdgpu: Fix the dead lock issue.

PATCH In review Emily Deng 4 participants 10 messages Sep 11, 2018 8 messages not in the archive

Subsystems (2)
lore
Share & tools
lore

Emily Deng

author
#0 lore raw

PATCH 1/1 drm/amdgpu: Fix the dead lock issue.

It will ramdomly have the dead lock issue when test TDR:
1. amdgpu_device_handle_vram_lost gets the lock shadow_list_lock
2. amdgpu_bo_create locked the bo's resv lock
3. amdgpu_bo_create_shadow is waiting for the shadow_list_lock
4. amdgpu_device_recover_vram_from_shadow is waiting for the bo's resv lock.

v2:
Make a local copy of the list

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +20 -1
index 2a21267..8c81404 100644
@@ -3105,6 +3105,9 @@ static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
 	long r = 1;
 	int i = 0;
 	long tmo;
+	struct list_head local_shadow_list;
+
+	INIT_LIST_HEAD(&local_shadow_list);
 
 	if (amdgpu_sriov_runtime(adev))
 		tmo = msecs_to_jiffies(8000);
@@ -3112,8 +3115,19 @@ static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
 		tmo = msecs_to_jiffies(100);
 
 	DRM_INFO("recover vram bo from shadow start\n");
+
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&adev->shadow_list, &local_shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
+
 	mutex_lock(&adev->shadow_list_lock);
-	list_for_each_entry_safe(bo, tmp, &adev->shadow_list, shadow_list) {
+	list_for_each_entry_safe(bo, tmp, &local_shadow_list, shadow_list) {
+		mutex_unlock(&adev->shadow_list_lock);
+
+		if (!bo)
+			continue;
+
 		next = NULL;
 		amdgpu_device_recover_vram_from_shadow(adev, ring, bo, &next);
 		if (fence) {
@@ -3132,9 +3146,14 @@ static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
 
 		dma_fence_put(fence);
 		fence = next;
+		mutex_lock(&adev->shadow_list_lock);
 	}
 	mutex_unlock(&adev->shadow_list_lock);
 
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&local_shadow_list, &adev->shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
 	if (fence) {
 		r = dma_fence_wait_timeout(fence, false, tmo);
 		if (r == 0)
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1536634293-26099-1-git-send-email-Emily.Deng-5C7GfCeVMHo@public.gmane.org

#2 lore raw
Quoted 33 lines · Emily Deng

It will ramdomly have the dead lock issue when test TDR:
1. amdgpu_device_handle_vram_lost gets the lock shadow_list_lock
2. amdgpu_bo_create locked the bo's resv lock
3. amdgpu_bo_create_shadow is waiting for the shadow_list_lock
4. amdgpu_device_recover_vram_from_shadow is waiting for the bo's resv lock.

v2:
Make a local copy of the list

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 21 ++++++++++++++++++++-
  1 file changed, 20 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +3 -0
index 2a21267..8c81404 100644
@@ -3105,6 +3105,9 @@ static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
  	long r = 1;
  	int i = 0;
  	long tmo;
+	struct list_head local_shadow_list;
+
+	INIT_LIST_HEAD(&local_shadow_list);
 
  	if (amdgpu_sriov_runtime(adev))
  		tmo = msecs_to_jiffies(8000);
@@ -3112,8 +3115,19 @@ static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
  		tmo = msecs_to_jiffies(100);
 
  	DRM_INFO("recover vram bo from shadow start\n");

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev) · line 3126

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +8 -1
+
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&adev->shadow_list, &local_shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
+
  	mutex_lock(&adev->shadow_list_lock);
-	list_for_each_entry_safe(bo, tmp, &adev->shadow_list, shadow_list) {
+	list_for_each_entry_safe(bo, tmp, &local_shadow_list, shadow_list) {
+		mutex_unlock(&adev->shadow_list_lock);

We may not to use shadow_list_lock when traverse the local shadow list.

Regards,
Jerry

Quoted 13 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +5 -0
+
+		if (!bo)
+			continue;
+
  		next = NULL;
  		amdgpu_device_recover_vram_from_shadow(adev, ring, bo, &next);
  		if (fence) {
@@ -3132,9 +3146,14 @@ static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
 
  		dma_fence_put(fence);
  		fence = next;
+		mutex_lock(&adev->shadow_list_lock);

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev) · line 3159

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +4 -0
  	}
  	mutex_unlock(&adev->shadow_list_lock);
 
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&local_shadow_list, &adev->shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
  	if (fence) {
  		r = dma_fence_wait_timeout(fence, false, tmo);
  		if (r == 0)

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#3 lore raw
In reply to a message not in the archive
Quoted 30 lines · Emily Deng

It will ramdomly have the dead lock issue when test TDR:
1. amdgpu_device_handle_vram_lost gets the lock shadow_list_lock
2. amdgpu_bo_create locked the bo's resv lock
3. amdgpu_bo_create_shadow is waiting for the shadow_list_lock
4. amdgpu_device_recover_vram_from_shadow is waiting for the bo's resv lock.

v2:
Make a local copy of the list

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 21 ++++++++++++++++++++-
  1 file changed, 20 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +3 -0
index 2a21267..8c81404 100644
@@ -3105,6 +3105,9 @@ static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
  	long r = 1;
  	int i = 0;
  	long tmo;
+	struct list_head local_shadow_list;
+
+	INIT_LIST_HEAD(&local_shadow_list);
 
  	if (amdgpu_sriov_runtime(adev))
  		tmo = msecs_to_jiffies(8000);

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev) · line 3124

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +6 -0
@@ -3112,8 +3115,19 @@ static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
  		tmo = msecs_to_jiffies(100);
 
  	DRM_INFO("recover vram bo from shadow start\n");
+
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&adev->shadow_list, &local_shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
+
  	mutex_lock(&adev->shadow_list_lock);

local_shadow_list is a local variable, I think it doesn't need lock at all, no one change it. Otherwise looks good to me.

Thanks,
David Zhou

Quoted 15 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +7 -1
-	list_for_each_entry_safe(bo, tmp, &adev->shadow_list, shadow_list) {
+	list_for_each_entry_safe(bo, tmp, &local_shadow_list, shadow_list) {
+		mutex_unlock(&adev->shadow_list_lock);
+
+		if (!bo)
+			continue;
+
  		next = NULL;
  		amdgpu_device_recover_vram_from_shadow(adev, ring, bo, &next);
  		if (fence) {
@@ -3132,9 +3146,14 @@ static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
 
  		dma_fence_put(fence);
  		fence = next;
+		mutex_lock(&adev->shadow_list_lock);

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev) · line 3159

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +4 -0
  	}
  	mutex_unlock(&adev->shadow_list_lock);
 
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&local_shadow_list, &adev->shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
  	if (fence) {
  		r = dma_fence_wait_timeout(fence, false, tmo);
  		if (r == 0)

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): bf049e4b-321f-33b4-dad2-708269515e90-5C7GfCeVMHo@public.gmane.org

#5 lore raw
Quoted 45 lines

-----Original Message-----
From: Zhou, David(ChunMing)
Sent: Tuesday, September 11, 2018 11:03 AM
To: Deng, Emily <[email protected]>; [email protected]
Subject: Re: [PATCH] drm/amdgpu: Fix the dead lock issue.

Quoted 7 lines · Emily Deng

It will ramdomly have the dead lock issue when test TDR:
1. amdgpu_device_handle_vram_lost gets the lock shadow_list_lock 2. amdgpu_bo_create locked the bo's resv lock 3. amdgpu_bo_create_shadow is waiting for the shadow_list_lock 4. amdgpu_device_recover_vram_from_shadow is waiting for the bo's resv lock.

Quoted 6 lines

v2:
Make a local copy of the list

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 21

++++++++++++++++++++-

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int

Quoted 8 lines
  1 file changed, 20 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
index 2a21267..8c81404 100644
@@ -3105,6 +3105,9 @@ static int

amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +3 -0
  	long r = 1;
  	int i = 0;
  	long tmo;
+	struct list_head local_shadow_list;
+
+	INIT_LIST_HEAD(&local_shadow_list);
 
  	if (amdgpu_sriov_runtime(adev))
  		tmo = msecs_to_jiffies(8000);
@@ -3112,8 +3115,19 @@ static int

amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
  		tmo = msecs_to_jiffies(100);
 
Quoted 10 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev) · line 3124

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +6 -0
  	DRM_INFO("recover vram bo from shadow start\n");
+
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&adev->shadow_list, &local_shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
+
  	mutex_lock(&adev->shadow_list_lock);

local_shadow_list is a local variable, I think it doesn't need lock at all, no one change it. Otherwise looks good to me.

The bo->shadow_list which now is in local_shadow_list maybe destroy in case that it already in amdgpu_bo_destroy, then it will change local_shadow_list, so need lock the shadow_list_lock.
Best wishes
Emily Deng

Quoted 22 lines

Thanks,
David Zhou

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +6 -1
-	list_for_each_entry_safe(bo, tmp, &adev->shadow_list, shadow_list) {
+	list_for_each_entry_safe(bo, tmp, &local_shadow_list, shadow_list) {
+		mutex_unlock(&adev->shadow_list_lock);
+
+		if (!bo)
+			continue;
+
  		next = NULL;
  		amdgpu_device_recover_vram_from_shadow(adev, ring, bo,

&next);

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
  		if (fence) {
@@ -3132,9 +3146,14 @@ static int
 amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
 
  		dma_fence_put(fence);
  		fence = next;
+		mutex_lock(&adev->shadow_list_lock);
  	}
  	mutex_unlock(&adev->shadow_list_lock);
 
Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +4 -0
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&local_shadow_list, &adev->shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
  	if (fence) {
  		r = dma_fence_wait_timeout(fence, false, tmo);

if (r == 0)

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): BN7PR12MB2644CEE78E8E6D658D1A686E8F040-Zx/IyJUqfGKoYeSiBV3SvgdYzm3356FpvxpqHgZTriW3zl9H0oFU5g@public.gmane.org

#7 lore raw
Quoted 55 lines · Emily
Quoted 45 lines

-----Original Message-----
From: Zhou, David(ChunMing)
Sent: Tuesday, September 11, 2018 11:03 AM
To: Deng, Emily <[email protected]>; [email protected]
Subject: Re: [PATCH] drm/amdgpu: Fix the dead lock issue.

Quoted 7 lines · Emily Deng

It will ramdomly have the dead lock issue when test TDR:
1. amdgpu_device_handle_vram_lost gets the lock shadow_list_lock 2. amdgpu_bo_create locked the bo's resv lock 3. amdgpu_bo_create_shadow is waiting for the shadow_list_lock 4. amdgpu_device_recover_vram_from_shadow is waiting for the bo's resv lock.

Quoted 6 lines

v2:
Make a local copy of the list

---
   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 21

++++++++++++++++++++-

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int

Quoted 8 lines
   1 file changed, 20 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
index 2a21267..8c81404 100644
@@ -3105,6 +3105,9 @@ static int

amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +3 -0
   	long r = 1;
   	int i = 0;
   	long tmo;
+	struct list_head local_shadow_list;
+
+	INIT_LIST_HEAD(&local_shadow_list);
 
   	if (amdgpu_sriov_runtime(adev))
   		tmo = msecs_to_jiffies(8000);
@@ -3112,8 +3115,19 @@ static int

amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
   		tmo = msecs_to_jiffies(100);
 
Quoted 10 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev) · line 3124

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +6 -0
   	DRM_INFO("recover vram bo from shadow start\n");
+
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&adev->shadow_list, &local_shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
+
   	mutex_lock(&adev->shadow_list_lock);

local_shadow_list is a local variable, I think it doesn't need lock at all, no one change it. Otherwise looks good to me.

Quoted 2 lines

The bo->shadow_list which now is in local_shadow_list maybe destroy in case that it already in amdgpu_bo_destroy, then it will change local_shadow_list, so need lock the shadow_list_lock.

Ah, sorry for noise, I forget you don't reference these BOs.

Thanks,
David Zhou

Quoted 24 lines

Best wishes
Emily Deng

Quoted 22 lines

Thanks,
David Zhou

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +6 -1
-	list_for_each_entry_safe(bo, tmp, &adev->shadow_list, shadow_list) {
+	list_for_each_entry_safe(bo, tmp, &local_shadow_list, shadow_list) {
+		mutex_unlock(&adev->shadow_list_lock);
+
+		if (!bo)
+			continue;
+
   		next = NULL;
   		amdgpu_device_recover_vram_from_shadow(adev, ring, bo,

&next);

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
   		if (fence) {
@@ -3132,9 +3146,14 @@ static int
 amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
 
   		dma_fence_put(fence);
   		fence = next;
+		mutex_lock(&adev->shadow_list_lock);
   	}
   	mutex_unlock(&adev->shadow_list_lock);
 
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +4 -0
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&local_shadow_list, &adev->shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
   	if (fence) {
   		r = dma_fence_wait_timeout(fence, false, tmo);
   		if (r == 0)

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): fcbc7544-d08d-4051-c554-f1f9cecb87e1-5C7GfCeVMHo@public.gmane.org

#9 lore raw
Quoted 66 lines

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of zhoucm1
Sent: Tuesday, September 11, 2018 11:28 AM
To: Deng, Emily <[email protected]>; Zhou, David(ChunMing) <[email protected]>; [email protected]
Subject: Re: [PATCH] drm/amdgpu: Fix the dead lock issue.

Quoted 55 lines · Emily
Quoted 55 lines

-----Original Message-----
From: Zhou, David(ChunMing)
Sent: Tuesday, September 11, 2018 11:03 AM
To: Deng, Emily <[email protected]>; [email protected]
Subject: Re: [PATCH] drm/amdgpu: Fix the dead lock issue.

Quoted 7 lines · Emily Deng

It will ramdomly have the dead lock issue when test TDR:
1. amdgpu_device_handle_vram_lost gets the lock shadow_list_lock 2. amdgpu_bo_create locked the bo's resv lock 3. amdgpu_bo_create_shadow is waiting for the shadow_list_lock 4. amdgpu_device_recover_vram_from_shadow is waiting for the bo's resv lock.

Quoted 6 lines

v2:
Make a local copy of the list

---
   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 21

++++++++++++++++++++-

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int

Quoted 8 lines
   1 file changed, 20 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
index 2a21267..8c81404 100644
@@ -3105,6 +3105,9 @@ static int

amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +3 -0
   	long r = 1;
   	int i = 0;
   	long tmo;
+	struct list_head local_shadow_list;
+
+	INIT_LIST_HEAD(&local_shadow_list);
 
   	if (amdgpu_sriov_runtime(adev))
   		tmo = msecs_to_jiffies(8000);
@@ -3112,8 +3115,19 @@ static int

amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int · line 3124

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +6 -0
   		tmo = msecs_to_jiffies(100);
 
   	DRM_INFO("recover vram bo from shadow start\n");
+
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&adev->shadow_list, &local_shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
+
   	mutex_lock(&adev->shadow_list_lock);

local_shadow_list is a local variable, I think it doesn't need lock at all, no one change it. Otherwise looks good to me.

Quoted 4 lines
Quoted 2 lines

The bo->shadow_list which now is in local_shadow_list maybe destroy in case that it already in amdgpu_bo_destroy, then it will change

local_shadow_list, so need lock the shadow_list_lock. Ah, sorry for noise, I forget you don't reference these BOs.

Yes, I don't reference these Bos, as I found even reference these Bos, it still couldn't avoid the case that another process is already in amdgpu_bo_destroy.

Quoted 27 lines

Thanks,
David Zhou

Quoted 24 lines

Best wishes
Emily Deng

Quoted 22 lines

Thanks,
David Zhou

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int · line 3132

Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +6 -1
-	list_for_each_entry_safe(bo, tmp, &adev->shadow_list, shadow_list) {
+	list_for_each_entry_safe(bo, tmp, &local_shadow_list, shadow_list) {
+		mutex_unlock(&adev->shadow_list_lock);
+
+		if (!bo)
+			continue;
+
   		next = NULL;
   		amdgpu_device_recover_vram_from_shadow(adev, ring, bo,

&next);

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
   		if (fence) {
@@ -3132,9 +3146,14 @@ static int
 amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
 
   		dma_fence_put(fence);
   		fence = next;
+		mutex_lock(&adev->shadow_list_lock);
   	}
   	mutex_unlock(&adev->shadow_list_lock);
 
Quoted 12 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +4 -0
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&local_shadow_list, &adev->shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
   	if (fence) {
   		r = dma_fence_wait_timeout(fence, false, tmo);
   		if (r == 0)

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): BN7PR12MB2644E374B512DC7EF10CA6D08F040-Zx/IyJUqfGKoYeSiBV3SvgdYzm3356FpvxpqHgZTriW3zl9H0oFU5g@public.gmane.org

#11 lore raw
Quoted 66 lines · Emily
Quoted 66 lines

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of zhoucm1
Sent: Tuesday, September 11, 2018 11:28 AM
To: Deng, Emily <[email protected]>; Zhou, David(ChunMing) <[email protected]>; [email protected]
Subject: Re: [PATCH] drm/amdgpu: Fix the dead lock issue.

Quoted 55 lines · Emily
Quoted 55 lines

-----Original Message-----
From: Zhou, David(ChunMing)
Sent: Tuesday, September 11, 2018 11:03 AM
To: Deng, Emily <[email protected]>; [email protected]
Subject: Re: [PATCH] drm/amdgpu: Fix the dead lock issue.

Quoted 7 lines · Emily Deng

It will ramdomly have the dead lock issue when test TDR:
1. amdgpu_device_handle_vram_lost gets the lock shadow_list_lock 2. amdgpu_bo_create locked the bo's resv lock 3. amdgpu_bo_create_shadow is waiting for the shadow_list_lock 4. amdgpu_device_recover_vram_from_shadow is waiting for the bo's resv lock.

Quoted 6 lines

v2:
Make a local copy of the list

---
    drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 21

++++++++++++++++++++-

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int

Quoted 8 lines
    1 file changed, 20 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
index 2a21267..8c81404 100644
@@ -3105,6 +3105,9 @@ static int

amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +3 -0
    	long r = 1;
    	int i = 0;
    	long tmo;
+	struct list_head local_shadow_list;
+
+	INIT_LIST_HEAD(&local_shadow_list);
 
    	if (amdgpu_sriov_runtime(adev))
    		tmo = msecs_to_jiffies(8000);
@@ -3112,8 +3115,19 @@ static int

amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int · line 3124

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +6 -0
    		tmo = msecs_to_jiffies(100);
 
    	DRM_INFO("recover vram bo from shadow start\n");
+
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&adev->shadow_list, &local_shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
+
    	mutex_lock(&adev->shadow_list_lock);

local_shadow_list is a local variable, I think it doesn't need lock at all, no one change it. Otherwise looks good to me.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

The bo->shadow_list which now is in local_shadow_list maybe destroy in case that it already in amdgpu_bo_destroy, then it will change

local_shadow_list, so need lock the shadow_list_lock. Ah, sorry for noise, I forget you don't reference these BOs.

Yes, I don't reference these Bos, as I found even reference these Bos, it still couldn't avoid the case that another process is already in amdgpu_bo_destroy.

??? that shouldn't happen, the reference is belonged to list. But back to here, we don't need reference them.
And since no shadow BO is added to local after splice, we'd better to use list_next_entry to iterate the local shadow list instead of list_for_each_entry_safe.

Thanks,
David Zhou

Quoted 26 lines
Quoted 26 lines

Thanks,
David Zhou

Quoted 24 lines

Best wishes
Emily Deng

Quoted 22 lines

Thanks,
David Zhou

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int · line 3132

Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +6 -1
-	list_for_each_entry_safe(bo, tmp, &adev->shadow_list, shadow_list) {
+	list_for_each_entry_safe(bo, tmp, &local_shadow_list, shadow_list) {
+		mutex_unlock(&adev->shadow_list_lock);
+
+		if (!bo)
+			continue;
+
    		next = NULL;
    		amdgpu_device_recover_vram_from_shadow(adev, ring, bo,

&next);

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
    		if (fence) {
@@ -3132,9 +3146,14 @@ static int
 amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
 
    		dma_fence_put(fence);
    		fence = next;
+		mutex_lock(&adev->shadow_list_lock);
    	}
    	mutex_unlock(&adev->shadow_list_lock);
 
Quoted 11 lines
Quoted 11 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +4 -0
+	mutex_lock(&adev->shadow_list_lock);
+	list_splice_init(&local_shadow_list, &adev->shadow_list);
+	mutex_unlock(&adev->shadow_list_lock);
+
    	if (fence) {
    		r = dma_fence_wait_timeout(fence, false, tmo);
    		if (r == 0)

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): be67e50d-f161-c7da-256e-149ed1b56f26-5C7GfCeVMHo@public.gmane.org

#13 lore raw
Quoted 73 lines · On 2018年09月11日 11:37, zhoucm1 wrote:
Quoted 70 lines · Emily
Quoted 70 lines

-----Original Message-----
From: amd-gfx <amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org> On Behalf Of zhoucm1
Sent: Tuesday, September 11, 2018 11:28 AM
To: Deng, Emily <[email protected]>; Zhou, David(ChunMing) <[email protected]>; [email protected]
Subject: Re: [PATCH] drm/amdgpu: Fix the dead lock issue.

Quoted 55 lines · Emily
Quoted 45 lines

-----Original Message-----
From: Zhou, David(ChunMing)
Sent: Tuesday, September 11, 2018 11:03 AM
To: Deng, Emily <[email protected]>; [email protected]
Subject: Re: [PATCH] drm/amdgpu: Fix the dead lock issue.

Quoted 7 lines · Emily Deng

It will ramdomly have the dead lock issue when test TDR:
1. amdgpu_device_handle_vram_lost gets the lock shadow_list_lock 2. amdgpu_bo_create locked the bo's resv lock 3. amdgpu_bo_create_shadow is waiting for the shadow_list_lock 4. amdgpu_device_recover_vram_from_shadow is waiting for the bo's resv lock.

Quoted 6 lines

v2:
      Make a local copy of the list

---
    drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 21

++++++++++++++++++++-

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int

Quoted 8 lines
    1 file changed, 20 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
index 2a21267..8c81404 100644
@@ -3105,6 +3105,9 @@ static int

amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +3 -0
         long r = 1;
         int i = 0;
         long tmo;
+    struct list_head local_shadow_list;
+
+    INIT_LIST_HEAD(&local_shadow_list);
 
         if (amdgpu_sriov_runtime(adev))
             tmo = msecs_to_jiffies(8000);
@@ -3112,8 +3115,19 @@ static int

amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
             tmo = msecs_to_jiffies(100);
 
Quoted 10 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int · line 3124

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +6 -0
         DRM_INFO("recover vram bo from shadow start\n");
+
+    mutex_lock(&adev->shadow_list_lock);
+    list_splice_init(&adev->shadow_list, &local_shadow_list);
+    mutex_unlock(&adev->shadow_list_lock);
+
+
         mutex_lock(&adev->shadow_list_lock);

local_shadow_list is a local variable, I think it doesn't need lock at all, no one change it. Otherwise looks good to me.

Quoted 2 lines

The bo->shadow_list which now is in local_shadow_list maybe destroy in case that it already in amdgpu_bo_destroy, then it will change

local_shadow_list, so need lock the shadow_list_lock. Ah, sorry for noise, I forget you don't reference these BOs.

Quoted 17 lines
Quoted 2 lines

Yes, I don't reference these Bos, as I found even reference these Bos, it still couldn't avoid the case that another process is already in amdgpu_bo_destroy.

??? that shouldn't happen, the reference is belonged to list. But back to here, we don't need reference them.
And since no shadow BO is added to local after splice, we'd better to use list_next_entry to iterate the local shadow list instead of list_for_each_entry_safe.

Thanks,
David Zhou

Quoted 10 lines
Quoted 10 lines

Thanks,
David Zhou

Quoted 8 lines

Best wishes
Emily Deng

Quoted 6 lines

Thanks,
David Zhou

Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -1
-    list_for_each_entry_safe(bo, tmp, &adev->shadow_list, 
 shadow_list) {
+    list_for_each_entry_safe(bo, tmp, &local_shadow_list, 
 shadow_list) {

because shadow list doesn't take bo reference, we should give a amdgpu_bo_ref(bo) with attached patch before unlock. You can have a try.

Thanks,
David Zhou

Quoted 30 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +5 -0
+ mutex_unlock(&adev->shadow_list_lock);
+
+        if (!bo)
+            continue;
+
             next = NULL;

            amdgpu_device_recover_vram_from_shadow(adev, ring, bo,

&next);

Quoted 10 lines

            if (fence) {

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
@@ -3132,9 +3146,14 @@ static int
 amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
 
             dma_fence_put(fence);
             fence = next;
+        mutex_lock(&adev->shadow_list_lock);
         }
         mutex_unlock(&adev->shadow_list_lock);
 
Quoted 11 lines
Quoted 11 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +4 -0
+    mutex_lock(&adev->shadow_list_lock);
+    list_splice_init(&local_shadow_list, &adev->shadow_list);
+    mutex_unlock(&adev->shadow_list_lock);
+
         if (fence) {
             r = dma_fence_wait_timeout(fence, false, tmo);

            if (r == 0)

_______________________________________________
amd-gfx mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Quoted 6 lines

_______________________________________________
amd-gfx mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

From 7676fc70738699dddca5627e21be0d82e91aea05 Mon Sep 17 00:00:00 2001

From: Chunming Zhou <[email protected]>
Date: Tue, 11 Sep 2018 13:37:31 +0800
Subject: [PATCH] drm/amdgpu: changing of shadow bo reference should take
shadow lock

because shadow list doesn't reference shadow bo.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +10 -4
index b0e14a3d54ef..50651157203b 100644
@@ -101,11 +101,9 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo)
 		drm_prime_gem_destroy(&bo->gem_base, bo->tbo.sg);
 	drm_gem_object_release(&bo->gem_base);
 	amdgpu_bo_unref(&bo->parent);
-	if (!list_empty(&bo->shadow_list)) {
-		mutex_lock(&adev->shadow_list_lock);
+	if (!list_empty(&bo->shadow_list))
 		list_del_init(&bo->shadow_list);
-		mutex_unlock(&adev->shadow_list_lock);
-	}
+
 	kfree(bo->metadata);
 	kfree(bo);
 }
@@ -838,13 +836,21 @@ struct amdgpu_bo *amdgpu_bo_ref(struct amdgpu_bo *bo)
 void amdgpu_bo_unref(struct amdgpu_bo **bo)
 {
 	struct ttm_buffer_object *tbo;
+	bool locked = false;
 
 	if ((*bo) == NULL)
 		return;
 
+	if (bo->parent && bo->parent->shadow == bo &&
+	    !list_empty(&bo->parent->shadow_list)) {
+		mutex_lock(&adev->shadow_list_lock);
+		locked = true;
+	}
 	tbo = &((*bo)->tbo);
 	ttm_bo_put(tbo);
 	*bo = NULL;
+	if (locked)
+		mutex_unlock(&adev->shadow_list_lock);
 }
 
 /**
Signature · 1 line
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 83f7a45f-1aee-a2a8-bc82-f3433157c6cb-5C7GfCeVMHo@public.gmane.org

#15 lore raw

That won't work correctly. The TTM BO is unreferenced in a couple of more places which we don't have control over.

To make it even worse we actually can't take the reservation lock during GPU reset because the reservation object might already be destroyed when we remove the BO from the list.

I will take a look at this myself today to find a solution which should work.

Regards,
Christian.

Am 11.09.2018 um 07:41 schrieb zhoucm1:

Quoted 135 lines · On 2018年09月11日 11:37, zhoucm1 wrote:
Quoted 73 lines · Emily
Quoted 70 lines
Quoted 70 lines

-----Original Message-----
From: amd-gfx <amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org> On Behalf Of zhoucm1
Sent: Tuesday, September 11, 2018 11:28 AM
To: Deng, Emily <[email protected]>; Zhou, David(ChunMing) <[email protected]>; [email protected]
Subject: Re: [PATCH] drm/amdgpu: Fix the dead lock issue.

Quoted 55 lines · Emily
Quoted 45 lines

-----Original Message-----
From: Zhou, David(ChunMing)
Sent: Tuesday, September 11, 2018 11:03 AM
To: Deng, Emily <[email protected]>; [email protected]
Subject: Re: [PATCH] drm/amdgpu: Fix the dead lock issue.

Quoted 7 lines · Emily Deng

It will ramdomly have the dead lock issue when test TDR:
1. amdgpu_device_handle_vram_lost gets the lock shadow_list_lock 2. amdgpu_bo_create locked the bo's resv lock 3. amdgpu_bo_create_shadow is waiting for the shadow_list_lock 4. amdgpu_device_recover_vram_from_shadow is waiting for the bo's resv lock.

Quoted 6 lines

v2:
      Make a local copy of the list

---
    drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 21

++++++++++++++++++++-

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int

Quoted 8 lines
    1 file changed, 20 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
index 2a21267..8c81404 100644
@@ -3105,6 +3105,9 @@ static int

amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +3 -0
         long r = 1;
         int i = 0;
         long tmo;
+    struct list_head local_shadow_list;
+
+    INIT_LIST_HEAD(&local_shadow_list);
 
         if (amdgpu_sriov_runtime(adev))
             tmo = msecs_to_jiffies(8000);
@@ -3112,8 +3115,19 @@ static int

amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
             tmo = msecs_to_jiffies(100);
 
Quoted 10 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int · line 3124

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +6 -0
         DRM_INFO("recover vram bo from shadow start\n");
+
+    mutex_lock(&adev->shadow_list_lock);
+    list_splice_init(&adev->shadow_list, &local_shadow_list);
+    mutex_unlock(&adev->shadow_list_lock);
+
+
         mutex_lock(&adev->shadow_list_lock);

local_shadow_list is a local variable, I think it doesn't need lock at all, no one change it. Otherwise looks good to me.

Quoted 2 lines

The bo->shadow_list which now is in local_shadow_list maybe destroy in case that it already in amdgpu_bo_destroy, then it will change

local_shadow_list, so need lock the shadow_list_lock. Ah, sorry for noise, I forget you don't reference these BOs.

Quoted 17 lines
Quoted 2 lines

Yes, I don't reference these Bos, as I found even reference these Bos, it still couldn't avoid the case that another process is already in amdgpu_bo_destroy.

??? that shouldn't happen, the reference is belonged to list. But back to here, we don't need reference them.
And since no shadow BO is added to local after splice, we'd better to use list_next_entry to iterate the local shadow list instead of list_for_each_entry_safe.

Thanks,
David Zhou

Quoted 10 lines
Quoted 10 lines

Thanks,
David Zhou

Quoted 8 lines

Best wishes
Emily Deng

Quoted 6 lines

Thanks,
David Zhou

Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -1
- list_for_each_entry_safe(bo, tmp, &adev->shadow_list, 
 shadow_list) {
+    list_for_each_entry_safe(bo, tmp, &local_shadow_list, 
 shadow_list) {

because shadow list doesn't take bo reference, we should give a amdgpu_bo_ref(bo) with attached patch before unlock. You can have a try.

Thanks,
David Zhou

Quoted 30 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +5 -0
+ mutex_unlock(&adev->shadow_list_lock);
+
+        if (!bo)
+            continue;
+
             next = NULL;

amdgpu_device_recover_vram_from_shadow(adev, ring, bo,

&next);

Quoted 10 lines

            if (fence) {

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
@@ -3132,9 +3146,14 @@ static int
 amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)
 
             dma_fence_put(fence);
             fence = next;
+        mutex_lock(&adev->shadow_list_lock);
         }
         mutex_unlock(&adev->shadow_list_lock);
 
Quoted 11 lines
Quoted 11 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +4 -0
+    mutex_lock(&adev->shadow_list_lock);
+    list_splice_init(&local_shadow_list, &adev->shadow_list);
+    mutex_unlock(&adev->shadow_list_lock);
+
         if (fence) {
             r = dma_fence_wait_timeout(fence, false, tmo);

            if (r == 0)

_______________________________________________
amd-gfx mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Quoted 4 lines

_______________________________________________
amd-gfx mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Quoted 4 lines

_______________________________________________
amd-gfx mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): c67be83d-aecd-a291-a564-15f1fe501680-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#17 lore raw

From: Christian König <[email protected]>
Sent: Tuesday, September 11, 2018 2:40 PM
To: Zhou, David(ChunMing) <[email protected]>; Deng, Emily <[email protected]>; Zhou, David(ChunMing) <[email protected]>; [email protected]
Subject: Re: [PATCH] drm/amdgpu: Fix the dead lock issue.

That won't work correctly. The TTM BO is unreferenced in a couple of more places which we don't have control over.

To make it even worse we actually can't take the reservation lock during GPU reset because the reservation object might already be destroyed when we remove the BO from the list.

I will take a look at this myself today to find a solution which should work. Ok, thanks very much.

Best wishes
Emily Deng

Regards,
Christian.

Am 11.09.2018 um 07:41 schrieb zhoucm1:

On 2018年09月11日 11:37, zhoucm1 wrote:

On 2018年09月11日 11:32, Deng, Emily wrote:

-----Original Message-----
From: amd-gfx <[email protected]><mailto:[email protected]> On Behalf Of zhoucm1
Sent: Tuesday, September 11, 2018 11:28 AM
To: Deng, Emily <[email protected]><mailto:[email protected]>; Zhou, David(ChunMing) <[email protected]><mailto:[email protected]>; [email protected]<mailto:[email protected]>
Subject: Re: [PATCH] drm/amdgpu: Fix the dead lock issue.

On 2018年09月11日 11:23, Deng, Emily wrote:

-----Original Message-----
From: Zhou, David(ChunMing)
Sent: Tuesday, September 11, 2018 11:03 AM
To: Deng, Emily <[email protected]><mailto:[email protected]>; [email protected]<mailto:[email protected]>
Subject: Re: [PATCH] drm/amdgpu: Fix the dead lock issue.

On 2018年09月11日 10:51, Emily Deng wrote:

It will ramdomly have the dead lock issue when test TDR:
1. amdgpu_device_handle_vram_lost gets the lock shadow_list_lock 2. amdgpu_bo_create locked the bo's resv lock 3. amdgpu_bo_create_shadow is waiting for the shadow_list_lock 4. amdgpu_device_recover_vram_from_shadow is waiting for the bo's resv lock.

v2:
Make a local copy of the list

---
    drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 21

++++++++++++++++++++-

    1 file changed, 20 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 2a21267..8c81404 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
@@ -3105,6 +3105,9 @@ static int

amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

        long r = 1;
        int i = 0;
        long tmo;

+ struct list_head local_shadow_list;
+
+ INIT_LIST_HEAD(&local_shadow_list);

        if (amdgpu_sriov_runtime(adev))
            tmo = msecs_to_jiffies(8000);

@@ -3112,8 +3115,19 @@ static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

            tmo = msecs_to_jiffies(100);

        DRM_INFO("recover vram bo from shadow start\n");
+
+    mutex_lock(&adev->shadow_list_lock);
+    list_splice_init(&adev->shadow_list, &local_shadow_list);
+    mutex_unlock(&adev->shadow_list_lock);
+
+
        mutex_lock(&adev->shadow_list_lock);

local_shadow_list is a local variable, I think it doesn't need lock
at all, no one change it. Otherwise looks good to me.
The bo->shadow_list which now is in local_shadow_list maybe destroy in
case that it already in amdgpu_bo_destroy, then it will change
local_shadow_list, so need lock the shadow_list_lock.
Ah, sorry for noise, I forget you don't reference these BOs.
Yes, I don't reference these Bos, as I found even reference these Bos, it still couldn't avoid the case that another process is already in amdgpu_bo_destroy.
??? that shouldn't happen, the reference is belonged to list. But back to here, we don't need reference them.
And since no shadow BO is added to local after splice, we'd better to use list_next_entry to iterate the local shadow list instead of list_for_each_entry_safe.

Thanks,
David Zhou

Thanks,
David Zhou

Best wishes
Emily Deng

Thanks,
David Zhou

- list_for_each_entry_safe(bo, tmp, &adev->shadow_list, shadow_list) {
+ list_for_each_entry_safe(bo, tmp, &local_shadow_list, shadow_list) {
because shadow list doesn't take bo reference, we should give a amdgpu_bo_ref(bo) with attached patch before unlock. You can have a try.

Thanks,
David Zhou

+ mutex_unlock(&adev->shadow_list_lock);
+
+ if (!bo)
+ continue;
+
next = NULL; amdgpu_device_recover_vram_from_shadow(adev, ring, bo,
&next);

            if (fence) {

@@ -3132,9 +3146,14 @@ static int amdgpu_device_handle_vram_lost(struct amdgpu_device *adev)

            dma_fence_put(fence);
            fence = next;
+        mutex_lock(&adev->shadow_list_lock);
        }
        mutex_unlock(&adev->shadow_list_lock);
+    mutex_lock(&adev->shadow_list_lock);
+    list_splice_init(&local_shadow_list, &adev->shadow_list);
+    mutex_unlock(&adev->shadow_list_lock);
+
        if (fence) {
            r = dma_fence_wait_timeout(fence, false, tmo);
            if (r == 0)

_______________________________________________
amd-gfx mailing list [email protected]<mailto:[email protected]>
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list [email protected]<mailto:[email protected]>
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________

amd-gfx mailing list

[email protected]<mailto:[email protected]>

https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx