drm/amdgpu: Make sure ttm delayed work finished
Subsystems (2)
PATCH 1/1 drm/amdgpu: Make sure ttm delayed work finished
ttm_device_delayed_workqueue would reschedule itself if there is pending BO to be destroyed. So just one flush + cancel_sync is not enough. We still see lru_list not empty warnging.
Fix it by waiting all BO to be destroyed.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +8 -2
@@ -3957,11 +3957,17 @@ static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev)*/ void amdgpu_device_fini_hw (struct amdgpu_device *adev) { +int pending =1 ; +dev_info (adev->dev,"amdgpu: finishing device.\n" );flush_delayed_work (&adev->delayed_init_work); -if (adev->mman.initialized) { +while (adev->mman.initialized && pending) {flush_delayed_work (&adev->mman.bdev.wq); -ttm_bo_lock_delayed_workqueue (&adev->mman.bdev); + pending =ttm_bo_lock_delayed_workqueue (&adev->mman.bdev); +if (pending) { +ttm_bo_unlock_delayed_workqueue (&adev->mman.bdev, true); + msleep((HZ / 100) < 1) ? 1 : HZ / 100); + } } adev->shutdown = true;
Signature · 1 line
2.25.1
+ msleep((HZ / 100) < 1) ? 1 : HZ / 100);
a "(" is missed. With it fixed, this patch is:
Regards,
Guchun
-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of xinhui pan
Sent: Wednesday, April 13, 2022 11:09 AM
To: [email protected]
Cc: Deucher, Alexander <[email protected]>; Pan, Xinhui <[email protected]>; Koenig, Christian <[email protected]>
Subject: [PATCH] drm/amdgpu: Make sure ttm delayed work finished
ttm_device_delayed_workqueue would reschedule itself if there is pending BO to be destroyed. So just one flush + cancel_sync is not enough. We still see lru_list not empty warnging.
Fix it by waiting all BO to be destroyed.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +8 -3
@@ -3957,11 +3957,17 @@ static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev)*/ void amdgpu_device_fini_hw (struct amdgpu_device *adev) { +int pending =1 ; +dev_info (adev->dev,"amdgpu: finishing device.\n" );flush_delayed_work (&adev->delayed_init_work); -if (adev->mman.initialized) { +while (adev->mman.initialized && pending) {flush_delayed_work (&adev->mman.bdev.wq); -ttm_bo_lock_delayed_workqueue (&adev->mman.bdev); + pending =ttm_bo_lock_delayed_workqueue (&adev->mman.bdev); +if (pending) { +ttm_bo_unlock_delayed_workqueue (&adev->mman.bdev, true); + msleep((HZ / 100) < 1) ? 1 : HZ / 100); + } } adev->shutdown = true; --
2.25.1
Koenig, Christian
maintainerWe don't need that.
TTM only reschedules when the BOs are still busy.
And if the BOs are still busy when you unload the driver we have much bigger problems that this TTM worker :)
Regards,
Christian
________________________________
Von: Pan, Xinhui <[email protected]>
Gesendet: Mittwoch, 13. April 2022 05:08
An: [email protected] <[email protected]>
Cc: Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Pan, Xinhui <[email protected]>
Betreff: [PATCH] drm/amdgpu: Make sure ttm delayed work finished
ttm_device_delayed_workqueue would reschedule itself if there is pending BO to be destroyed. So just one flush + cancel_sync is not enough. We still see lru_list not empty warnging.
Fix it by waiting all BO to be destroyed.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +8 -2
@@ -3957,11 +3957,17 @@ static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev)*/ void amdgpu_device_fini_hw (struct amdgpu_device *adev) { +int pending =1 ; +dev_info (adev->dev,"amdgpu: finishing device.\n" );flush_delayed_work (&adev->delayed_init_work); -if (adev->mman.initialized) { +while (adev->mman.initialized && pending) {flush_delayed_work (&adev->mman.bdev.wq); -ttm_bo_lock_delayed_workqueue (&adev->mman.bdev); + pending =ttm_bo_lock_delayed_workqueue (&adev->mman.bdev); +if (pending) { +ttm_bo_unlock_delayed_workqueue (&adev->mman.bdev, true); + msleep((HZ / 100) < 1) ? 1 : HZ / 100); + } } adev->shutdown = true;
--
2.25.1
[AMD Official Use Only]
The log from tester says it is the drm framebuffer BO being busy.
I just feel there is lack of time for its fence to be signaled. As a delay works too in my test.
But the warning is a little annoying.
________________________________________
发件人: Koenig, Christian <[email protected]> 发送时间: 2022年4月13日 15:30
收件人: Pan, Xinhui; [email protected]
抄送: Deucher, Alexander
主题: AW: [PATCH] drm/amdgpu: Make sure ttm delayed work finished
We don't need that.
TTM only reschedules when the BOs are still busy.
And if the BOs are still busy when you unload the driver we have much bigger problems that this TTM worker :)
Regards,
Christian
________________________________
Von: Pan, Xinhui <[email protected]>
Gesendet: Mittwoch, 13. April 2022 05:08
An: [email protected] <[email protected]>
Cc: Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Pan, Xinhui <[email protected]>
Betreff: [PATCH] drm/amdgpu: Make sure ttm delayed work finished
ttm_device_delayed_workqueue would reschedule itself if there is pending BO to be destroyed. So just one flush + cancel_sync is not enough. We still see lru_list not empty warnging.
Fix it by waiting all BO to be destroyed.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +8 -2
@@ -3957,11 +3957,17 @@ static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev)*/ void amdgpu_device_fini_hw (struct amdgpu_device *adev) { +int pending =1 ; +dev_info (adev->dev,"amdgpu: finishing device.\n" );flush_delayed_work (&adev->delayed_init_work); -if (adev->mman.initialized) { +while (adev->mman.initialized && pending) {flush_delayed_work (&adev->mman.bdev.wq); -ttm_bo_lock_delayed_workqueue (&adev->mman.bdev); + pending =ttm_bo_lock_delayed_workqueue (&adev->mman.bdev); +if (pending) { +ttm_bo_unlock_delayed_workqueue (&adev->mman.bdev, true); + msleep((HZ / 100) < 1) ? 1 : HZ / 100); + } } adev->shutdown = true;
--
2.25.1
Christian König
maintainerThat warning is a bit more than a little annoying.
Before we stop the delayed delete worker we *must* absolutely make sure that there is nothing going on the hardware any more. Otherwise we could easily run into use after free issues.
There should somewhere be a amdgpu_fence_wait_empty() before the flush_delayed_work() call. If that isn't there we do have a problem elsewhere.
Thanks for investigating this,
Christian.
Am 13.04.22 um 09:47 schrieb Pan, Xinhui:
Quoted 69 lines
[AMD Official Use Only]
The log from tester says it is the drm framebuffer BO being busy.
I just feel there is lack of time for its fence to be signaled. As a delay works too in my test.
But the warning is a little annoying.________________________________________
发件人: Koenig, Christian <[email protected]> 发送时间: 2022年4月13日 15:30
收件人: Pan, Xinhui; [email protected]
抄送: Deucher, Alexander
主题: AW: [PATCH] drm/amdgpu: Make sure ttm delayed work finishedWe don't need that.
TTM only reschedules when the BOs are still busy.
And if the BOs are still busy when you unload the driver we have much bigger problems that this TTM worker :)
Regards,
Christian________________________________
Von: Pan, Xinhui <[email protected]>
Gesendet: Mittwoch, 13. April 2022 05:08
An: [email protected] <[email protected]>
Cc: Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Pan, Xinhui <[email protected]>
Betreff: [PATCH] drm/amdgpu: Make sure ttm delayed work finished
ttm_device_delayed_workqueuewould reschedule itself if there is pending BO to be destroyed. So just one flush +cancel_syncis not enough. We still seelru_listnot empty warnging.Fix it by waiting all BO to be destroyed.
Signed-off-by xinhui pan <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +8
-3@@ -3957,11 +3957,17 @@ static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev)*/ void amdgpu_device_fini_hw (struct amdgpu_device *adev) { +int pending =1 ; +dev_info (adev->dev,"amdgpu: finishing device.\n" );flush_delayed_work (&adev->delayed_init_work); -if (adev->mman.initialized) { +while (adev->mman.initialized && pending) {flush_delayed_work (&adev->mman.bdev.wq); -ttm_bo_lock_delayed_workqueue (&adev->mman.bdev); + pending =ttm_bo_lock_delayed_workqueue (&adev->mman.bdev); +if (pending) { +ttm_bo_unlock_delayed_workqueue (&adev->mman.bdev, true); + msleep((HZ / 100) < 1) ? 1 : HZ / 100); + } } adev->shutdown = true; --2.25.1