Skip to contentSkip to conversation

drm/amdgpu: Make sure ttm delayed work finished

PATCH In review xinhui pan 2 participants 2 messages Apr 13, 2022

Subsystems (2)
lore
Share & tools
lore

xinhui pan

author
#0 lore raw

PATCH v2 1/1 drm/amdgpu: Make sure ttm delayed work finished

ttm_device_delayed_workqueue would reschedule itself if there is pending BO to be destroyed. So just one flush + cancel_sync is not enough. We still see lru_list not empty warnging.

Fix it by waiting all BO to be destroyed.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +8 -2
index 6f47726f1765..56dcf8c3a3cd 100644
@@ -3957,11 +3957,17 @@ static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev)
  */
 void amdgpu_device_fini_hw(struct amdgpu_device *adev)
 {
+	int pending = 1;
+
 	dev_info(adev->dev, "amdgpu: finishing device.\n");
 	flush_delayed_work(&adev->delayed_init_work);
-	if (adev->mman.initialized) {
+	while (adev->mman.initialized && pending) {
 		flush_delayed_work(&adev->mman.bdev.wq);
-		ttm_bo_lock_delayed_workqueue(&adev->mman.bdev);
+		pending = ttm_bo_lock_delayed_workqueue(&adev->mman.bdev);
+		if (pending) {
+			ttm_bo_unlock_delayed_workqueue(&adev->mman.bdev, true);
+			msleep(((HZ / 100) < 1) ? 1 : HZ / 100);
+		}
 	}
 	adev->shutdown = true;
 
Signature · 1 line
2.25.1
#1 lore raw

Dear xinhui,

Am 13.04.22 um 08:46 schrieb xinhui pan:

Quoted 3 lines

ttm_device_delayed_workqueue would reschedule itself if there is pending BO to be destroyed. So just one flush + cancel_sync is not enough. We still see lru_list not empty warnging.

warning

(scripts/checkpatch.pl --codespell should have found this.)

Quoted 2 lines

Fix it by waiting all BO to be destroyed.

Please explain the waiting algorithm. Why at least one millisecond?

Do you have a reproducer for this issue?

Quoted 11 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 10 ++++++++--
  1 file changed, 8 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
index 6f47726f1765..56dcf8c3a3cd 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) · line 3960

Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
@@ -3957,11 +3957,17 @@ static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev)
   */
  void amdgpu_device_fini_hw(struct amdgpu_device *adev)
  {
+	int pending = 1;

Could this be bool?

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) · line 3968

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +5 -2
+
  	dev_info(adev->dev, "amdgpu: finishing device.\n");
  	flush_delayed_work(&adev->delayed_init_work);
-	if (adev->mman.initialized) {
+	while (adev->mman.initialized && pending) {
  		flush_delayed_work(&adev->mman.bdev.wq);
-		ttm_bo_lock_delayed_workqueue(&adev->mman.bdev);
+		pending = ttm_bo_lock_delayed_workqueue(&adev->mman.bdev);
+		if (pending) {
+			ttm_bo_unlock_delayed_workqueue(&adev->mman.bdev, true);

Does this call affect adev->mman.initialized? If not a do-while loop might be better suited, so pending is only checked once.

if (adev->mman.initialized) {
	do {
		flush_delayed_work(&adev->mman.bdev.wq);
		ttm_bo_unlock_delayed_workqueue(&adev->mman.bdev, true);
		msleep(((HZ / 100) < 1) ? 1 : HZ / 100);
	} while (ttm_bo_lock_delayed_workqueue(&adev->mman.bdev));
}

The logic is slightly different, as ttm_bo_unlock_delayed_workqueue(&adev->mman.bdev, true); and the sleep are run at least once, so the suggestion might be unsuitable.

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) · line 3969

Quoted 1 line
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
+			msleep(((HZ / 100) < 1) ? 1 : HZ / 100);

Maybe add a comment for that formula? (No idea, if common knowledge, but why is a delay needed, and the loop cannot be run as fast as possible?)

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) · line 3973

Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
+		}
  	}
  	adev->shutdown = true;
 

Kind regards,

Paul