Skip to contentSkip to conversation

drm/amdgpu: properly powergate Polaris12 UVD/VCE on suspend

PATCH In review Evan Quan 2 participants 7 messages Aug 17, 2021 – Aug 19, 2021

Subsystems (2)
lore
Share & tools
lore

Evan Quan

author
#0 lore raw

PATCH 1/1 drm/amdgpu: properly powergate Polaris12 UVD/VCE on suspend

If the powergating of UVD/VCE is in process, wait for its completion before proceeding(suspending). This can fix some hangs observed on suspending when UVD/VCE still using(e.g. issue "pm-suspend" when video is still playing).

Change-Id I36f39d9731e0a9638b52d5d92558b0ee9c23a9ed
---
 drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c | 5 +++++
 drivers/gpu/drm/amd/amdgpu/vce_v3_0.c | 5 +++++
 2 files changed, 10 insertions(+)
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +5 -0
index 4eebf973a065..2fdce572baeb 100644
@@ -554,6 +554,11 @@ static int uvd_v6_0_suspend(void *handle)
 	int r;
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->uvd.idle_work);
+
 	r = uvd_v6_0_hw_fini(adev);
 	if (r)
 		return r;
drivers/gpu/drm/amd/amdgpu/vce_v3_0.c +5 -0
index 6d9108fa22e0..f0adecd5ec0b 100644
@@ -503,6 +503,11 @@ static int vce_v3_0_suspend(void *handle)
 	int r;
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->vce.idle_work);
+
 	r = vce_v3_0_hw_fini(adev);
 	if (r)
 		return r;
Signature · 1 line
2.29.0
#1 lore raw
Quoted 18 lines · Evan Quan

If the powergating of UVD/VCE is in process, wait for its completion before proceeding(suspending). This can fix some hangs observed on suspending when UVD/VCE still using(e.g. issue "pm-suspend" when video is still playing).

Change-Id I36f39d9731e0a9638b52d5d92558b0ee9c23a9ed
---
  drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c | 5 +++++
  drivers/gpu/drm/amd/amdgpu/vce_v3_0.c | 5 +++++
  2 files changed, 10 insertions(+)
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +0 -0
index 4eebf973a065..2fdce572baeb 100644

drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c static int uvd_v6_0_suspend(void *handle) · line 561

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +5 -0
@@ -554,6 +554,11 @@ static int uvd_v6_0_suspend(void *handle)
  	int r;
  	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->uvd.idle_work);
+

If running idle is a prerequisite before going to suspend, then something else is missing here.

Otherwise, the hang looks more like a pending work launched after hardware is suspended and trying to access hardware. As the hardware is going to be suspended anyway, doesn't it work with cancel_delayed_work_sync - making sure that nothing is going to be launched later to access hardware?

Then this may be a potential issue for other suspend calls also where work is pending to be launched when hardware is suspended.

Thanks,
Lijo

Quoted 20 lines
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +0 -0
  	r = uvd_v6_0_hw_fini(adev);
  	if (r)
  		return r;
drivers/gpu/drm/amd/amdgpu/vce_v3_0.c +5 -0
index 6d9108fa22e0..f0adecd5ec0b 100644
@@ -503,6 +503,11 @@ static int vce_v3_0_suspend(void *handle)
  	int r;
  	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->vce.idle_work);
+
  	r = vce_v3_0_hw_fini(adev);
  	if (r)
  		return r;
author
#2 lore raw

[AMD Official Use Only]

+Leo to share his insights

Quoted 42 lines

-----Original Message-----
From: Lazar, Lijo <[email protected]>
Sent: Tuesday, August 17, 2021 3:28 PM
To: Quan, Evan <[email protected]>; [email protected]
Cc: Deucher, Alexander <[email protected]>; Chen, Guchun <[email protected]>; Pan, Xinhui <[email protected]>
Subject: Re: [PATCH] drm/amdgpu: properly powergate Polaris12 UVD/VCE on suspend

Quoted 19 lines · Evan Quan

If the powergating of UVD/VCE is in process, wait for its completion before proceeding(suspending). This can fix some hangs observed on suspending when UVD/VCE still using(e.g. issue "pm-suspend" when video is still playing).

Change-Id I36f39d9731e0a9638b52d5d92558b0ee9c23a9ed
---
  drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c | 5 +++++
  drivers/gpu/drm/amd/amdgpu/vce_v3_0.c | 5 +++++
  2 files changed, 10 insertions(+)
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +0 -0
index 4eebf973a065..2fdce572baeb 100644

drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c static int uvd_v6_0_suspend(void *handle) · line 561

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +5 -0
@@ -554,6 +554,11 @@ static int uvd_v6_0_suspend(void *handle)
  	int r;
  	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->uvd.idle_work);
+

If running idle is a prerequisite before going to suspend, then something else is missing here.

Quoted 5 lines

Otherwise, the hang looks more like a pending work launched after hardware is suspended and trying to access hardware. As the hardware is going to be suspended anyway, doesn't it work with cancel_delayed_work_sync - making sure that nothing is going to be launched later to access hardware?

[Quan, Evan] The reason we chose flush_delayed_work instead of cancel_delayed_work_sync is we think those operations performed in idle_work(dpm disablement, powergating) seems needed considering the action is 'suspend'. So, instead of "cancel", maybe waiting for them completion is more proper.

Quoted 3 lines

Then this may be a potential issue for other suspend calls also where work is pending to be launched when hardware is suspended.

[Quan, Evan] Do you mean we need to check whether there is similar issue for other IPs?

BR
Evan

Quoted 25 lines

Thanks,
Lijo

Quoted 21 lines
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +0 -0
  	r = uvd_v6_0_hw_fini(adev);
  	if (r)
  		return r;
drivers/gpu/drm/amd/amdgpu/vce_v3_0.c +5 -0
index 6d9108fa22e0..f0adecd5ec0b 100644
@@ -503,6 +503,11 @@ static int vce_v3_0_suspend(void *handle)
  	int r;
  	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->vce.idle_work);
+
  	r = vce_v3_0_hw_fini(adev);
  	if (r)
  		return r;
#3 lore raw
Quoted 46 lines · Evan

[AMD Official Use Only]

+Leo to share his insights

Quoted 42 lines

-----Original Message-----
From: Lazar, Lijo <[email protected]>
Sent: Tuesday, August 17, 2021 3:28 PM
To: Quan, Evan <[email protected]>; [email protected]
Cc: Deucher, Alexander <[email protected]>; Chen, Guchun <[email protected]>; Pan, Xinhui <[email protected]>
Subject: Re: [PATCH] drm/amdgpu: properly powergate Polaris12 UVD/VCE on suspend

Quoted 19 lines · Evan Quan

If the powergating of UVD/VCE is in process, wait for its completion before proceeding(suspending). This can fix some hangs observed on suspending when UVD/VCE still using(e.g. issue "pm-suspend" when video is still playing).

Change-Id I36f39d9731e0a9638b52d5d92558b0ee9c23a9ed
---
   drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c | 5 +++++
   drivers/gpu/drm/amd/amdgpu/vce_v3_0.c | 5 +++++
   2 files changed, 10 insertions(+)
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +0 -0
index 4eebf973a065..2fdce572baeb 100644

drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c static int uvd_v6_0_suspend(void *handle) · line 561

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +5 -0
@@ -554,6 +554,11 @@ static int uvd_v6_0_suspend(void *handle)
   	int r;
   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->uvd.idle_work);
+

If running idle is a prerequisite before going to suspend, then something else is missing here.

Quoted 6 lines
Quoted 5 lines

Otherwise, the hang looks more like a pending work launched after hardware is suspended and trying to access hardware. As the hardware is going to be suspended anyway, doesn't it work with cancel_delayed_work_sync - making sure that nothing is going to be launched later to access hardware?

[Quan, Evan] The reason we chose flush_delayed_work instead of cancel_delayed_work_sync is we think those operations performed in idle_work(dpm disablement, powergating) seems needed considering the action is 'suspend'. So, instead of "cancel", maybe waiting for them completion is more proper.

But it will do so only if the work is scheduled - so it doesn't seem to be a prerequisite for suspend. If it was a prerequisite, then the existing code is missing that (so that it gets done for all cases).

Quoted 5 lines
Quoted 3 lines

Then this may be a potential issue for other suspend calls also where work is pending to be launched when hardware is suspended.

[Quan, Evan] Do you mean we need to check whether there is similar issue for other IPs?

Yes, if there are cases where other IPs may schedule a delayed work and call hw_fini without cancelling the work.

Thanks,
Lijo

Quoted 27 lines

BR
Evan

Quoted 25 lines

Thanks,
Lijo

Quoted 21 lines
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +0 -0
   	r = uvd_v6_0_hw_fini(adev);
   	if (r)
   		return r;
drivers/gpu/drm/amd/amdgpu/vce_v3_0.c +5 -0
index 6d9108fa22e0..f0adecd5ec0b 100644
@@ -503,6 +503,11 @@ static int vce_v3_0_suspend(void *handle)
   	int r;
   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->vce.idle_work);
+
   	r = vce_v3_0_hw_fini(adev);
   	if (r)
   		return r;
author
#4 lore raw

[AMD Official Use Only]

Quoted 66 lines

-----Original Message-----
From: Lazar, Lijo <[email protected]>
Sent: Tuesday, August 17, 2021 6:09 PM
To: Quan, Evan <[email protected]>; [email protected]; Liu, Leo <[email protected]>
Cc: Deucher, Alexander <[email protected]>; Chen, Guchun <[email protected]>; Pan, Xinhui <[email protected]>
Subject: Re: [PATCH] drm/amdgpu: properly powergate Polaris12 UVD/VCE on suspend

Quoted 11 lines · Evan

[AMD Official Use Only]

+Leo to share his insights

Quoted 7 lines

-----Original Message-----
From: Lazar, Lijo <[email protected]>
Sent: Tuesday, August 17, 2021 3:28 PM
To: Quan, Evan <[email protected]>; [email protected]
Cc: Deucher, Alexander <[email protected]>; Chen, Guchun <[email protected]>; Pan, Xinhui <[email protected]>
Subject: Re: [PATCH] drm/amdgpu: properly powergate Polaris12

UVD/VCE

Quoted 35 lines
Quoted 35 lines

on suspend

Quoted 19 lines · Evan Quan

If the powergating of UVD/VCE is in process, wait for its completion before proceeding(suspending). This can fix some hangs observed on suspending when UVD/VCE still using(e.g. issue "pm-suspend" when video is still playing).

Change-Id I36f39d9731e0a9638b52d5d92558b0ee9c23a9ed
---
   drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c | 5 +++++
   drivers/gpu/drm/amd/amdgpu/vce_v3_0.c | 5 +++++
   2 files changed, 10 insertions(+)
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +0 -0
index 4eebf973a065..2fdce572baeb 100644

drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c static int uvd_v6_0_suspend(void *handle) · line 561

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +5 -0
@@ -554,6 +554,11 @@ static int uvd_v6_0_suspend(void *handle)
   	int r;
   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->uvd.idle_work);
+

If running idle is a prerequisite before going to suspend, then something else is missing here.

Quoted 6 lines
Quoted 5 lines

Otherwise, the hang looks more like a pending work launched after hardware is suspended and trying to access hardware. As the hardware is going to be suspended anyway, doesn't it work with cancel_delayed_work_sync - making sure that nothing is going to be launched later to access hardware?

[Quan, Evan] The reason we chose flush_delayed_work instead of

Quoted 8 lines

cancel_delayed_work_sync is we think those operations performed in idle_work(dpm disablement, powergating) seems needed considering the action is 'suspend'. So, instead of "cancel", maybe waiting for them completion is more proper.

But it will do so only if the work is scheduled - so it doesn't seem to be a prerequisite for suspend. If it was a prerequisite, then the existing code is missing that (so that it gets done for all cases).

[Quan, Evan] Just confirmed that cancel_delayed_work_sync() alone cannot work.
In fact, our current driver already get cancel_delayed_work_sync() called(e.g. in amdgpu_uvd_suspend() on the path of uvd_v6_0_suspend). To get the issue fixed, it has to be either:
1. flush_delayed_work()
Or
2. cancel_delayed_work_sync + amdgpu_dpm_enable_uvd/vce(adev, false) (the job performed in idle work)

Btw, I do not think flush_delayed_work() is an incomplete fix. Since the UVD/VCE idle work is appended on ring->funcs->end_use. So, as long as the UVD/VCE ring used and ended(it will since at least there is ring/ib test), there will be a chance to get the work waited and completed.

BR
Evan

Quoted 41 lines
Quoted 5 lines
Quoted 4 lines

Then this may be a potential issue for other suspend calls also where work is pending to be launched when hardware is suspended.

[Quan, Evan] Do you mean we need to check whether there is similar issue

for other IPs?

Quoted 1 line

Yes, if there are cases where other IPs may schedule a delayed work and call hw_fini without cancelling the work.

Thanks,
Lijo

Quoted 27 lines

BR
Evan

Quoted 25 lines

Thanks,
Lijo

Quoted 21 lines
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +0 -0
   	r = uvd_v6_0_hw_fini(adev);
   	if (r)
   		return r;
drivers/gpu/drm/amd/amdgpu/vce_v3_0.c +5 -0
index 6d9108fa22e0..f0adecd5ec0b 100644
@@ -503,6 +503,11 @@ static int vce_v3_0_suspend(void *handle)
   	int r;
   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->vce.idle_work);
+
   	r = vce_v3_0_hw_fini(adev);
   	if (r)
   		return r;
#5 lore raw
Quoted 75 lines · Evan

[AMD Official Use Only]

Quoted 71 lines

-----Original Message-----
From: Lazar, Lijo <[email protected]>
Sent: Tuesday, August 17, 2021 6:09 PM
To: Quan, Evan <[email protected]>; [email protected]; Liu, Leo <[email protected]>
Cc: Deucher, Alexander <[email protected]>; Chen, Guchun <[email protected]>; Pan, Xinhui <[email protected]>
Subject: Re: [PATCH] drm/amdgpu: properly powergate Polaris12 UVD/VCE on suspend

Quoted 11 lines · Evan

[AMD Official Use Only]

+Leo to share his insights

Quoted 7 lines

-----Original Message-----
From: Lazar, Lijo <[email protected]>
Sent: Tuesday, August 17, 2021 3:28 PM
To: Quan, Evan <[email protected]>; [email protected]
Cc: Deucher, Alexander <[email protected]>; Chen, Guchun <[email protected]>; Pan, Xinhui <[email protected]>
Subject: Re: [PATCH] drm/amdgpu: properly powergate Polaris12

UVD/VCE

Quoted 35 lines
Quoted 35 lines

on suspend

Quoted 19 lines · Evan Quan

If the powergating of UVD/VCE is in process, wait for its completion before proceeding(suspending). This can fix some hangs observed on suspending when UVD/VCE still using(e.g. issue "pm-suspend" when video is still playing).

Change-Id I36f39d9731e0a9638b52d5d92558b0ee9c23a9ed
---
    drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c | 5 +++++
    drivers/gpu/drm/amd/amdgpu/vce_v3_0.c | 5 +++++
    2 files changed, 10 insertions(+)
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +0 -0
index 4eebf973a065..2fdce572baeb 100644

drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c static int uvd_v6_0_suspend(void *handle) · line 561

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +5 -0
@@ -554,6 +554,11 @@ static int uvd_v6_0_suspend(void *handle)
    	int r;
    	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->uvd.idle_work);
+

If running idle is a prerequisite before going to suspend, then something else is missing here.

Quoted 6 lines
Quoted 5 lines

Otherwise, the hang looks more like a pending work launched after hardware is suspended and trying to access hardware. As the hardware is going to be suspended anyway, doesn't it work with cancel_delayed_work_sync - making sure that nothing is going to be launched later to access hardware?

[Quan, Evan] The reason we chose flush_delayed_work instead of

cancel_delayed_work_sync is we think those operations performed in idle_work(dpm disablement, powergating) seems needed considering the action is 'suspend'. So, instead of "cancel", maybe waiting for them completion is more proper.

Quoted 5 lines
Quoted 3 lines

But it will do so only if the work is scheduled - so it doesn't seem to be a prerequisite for suspend. If it was a prerequisite, then the existing code is missing that (so that it gets done for all cases).

[Quan, Evan] Just confirmed that cancel_delayed_work_sync() alone cannot work.
In fact, our current driver already get cancel_delayed_work_sync() called(e.g. in amdgpu_uvd_suspend() on the path of uvd_v6_0_suspend).

Thanks Evan for checking this further.

If cancel_delayed_work_sync() is called in amdgpu_uvd_suspend(), then it means that originally executing idle_work was not considered as a prerequisite for suspending.

_uvd_suspend() is called "after" uvd_v6_0_hw_fini(adev), that maybe a little too late.

Quoted 4 lines

To get the issue fixed, it has to be either:
1. flush_delayed_work()
Or
2. cancel_delayed_work_sync + amdgpu_dpm_enable_uvd/vce(adev, false) (the job performed in idle work)

At minimum, it proves that disabling dpm is required for suspend.

Quoted 2 lines

Btw, I do not think flush_delayed_work() is an incomplete fix. Since the UVD/VCE idle work is appended on ring->funcs->end_use. So, as long as the UVD/VCE ring used and ended(it will since at least there is ring/ib test), there will be a chance to get the work waited and completed.

I agree that it fixes the issue, only thing is someone should look further to provide the right sequence of uvd suspend.

It doesn't give a root cause/right sequence - it only tells that forcing to execute idle_work fixes the problem probably due to an extra delay added by increased execution time or it disables DPM or something else. Someone should confirm that all of idle_work or a part of it (as dpm disable) is required for proper suspend sequence.

That said, I don't have any objections to this fix either. If there are other things, probably fix it this way and move on.

Thanks,
Lijo

Quoted 44 lines

BR
Evan

Quoted 41 lines
Quoted 5 lines
Quoted 4 lines

Then this may be a potential issue for other suspend calls also where work is pending to be launched when hardware is suspended.

[Quan, Evan] Do you mean we need to check whether there is similar issue

for other IPs?

Quoted 1 line

Yes, if there are cases where other IPs may schedule a delayed work and call hw_fini without cancelling the work.

Thanks,
Lijo

Quoted 27 lines

BR
Evan

Quoted 25 lines

Thanks,
Lijo

Quoted 21 lines
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +0 -0
    	r = uvd_v6_0_hw_fini(adev);
    	if (r)
    		return r;
drivers/gpu/drm/amd/amdgpu/vce_v3_0.c +5 -0
index 6d9108fa22e0..f0adecd5ec0b 100644
@@ -503,6 +503,11 @@ static int vce_v3_0_suspend(void *handle)
    	int r;
    	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->vce.idle_work);
+
    	r = vce_v3_0_hw_fini(adev);
    	if (r)
    		return r;
author
#6 lore raw

[AMD Official Use Only]

Quoted 127 lines

-----Original Message-----
From: Lazar, Lijo <[email protected]>
Sent: Wednesday, August 18, 2021 5:35 PM
To: Quan, Evan <[email protected]>; [email protected]; Liu, Leo <[email protected]>
Cc: Deucher, Alexander <[email protected]>; Chen, Guchun <[email protected]>; Pan, Xinhui <[email protected]>
Subject: Re: [PATCH] drm/amdgpu: properly powergate Polaris12 UVD/VCE on suspend

Quoted 12 lines · Evan

[AMD Official Use Only]

Quoted 8 lines

-----Original Message-----
From: Lazar, Lijo <[email protected]>
Sent: Tuesday, August 17, 2021 6:09 PM
To: Quan, Evan <[email protected]>; [email protected]; Liu, Leo <[email protected]>
Cc: Deucher, Alexander <[email protected]>; Chen, Guchun <[email protected]>; Pan, Xinhui <[email protected]>
Subject: Re: [PATCH] drm/amdgpu: properly powergate Polaris12

UVD/VCE

Quoted 5 lines
Quoted 5 lines

on suspend

On 8/17/2021 3:13 PM, Quan, Evan wrote:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

[AMD Official Use Only]

+Leo to share his insights

Quoted 4 lines

-----Original Message-----
From: Lazar, Lijo <[email protected]>
Sent: Tuesday, August 17, 2021 3:28 PM
To: Quan, Evan <[email protected]>; amd-

[email protected]

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Cc Deucher, Alexander <[email protected]>; Chen,

Guchun

Quoted 21 lines
Quoted 21 lines
Quoted 2 lines
Quoted 2 lines

<[email protected]>; Pan, Xinhui <[email protected]>
Subject: Re: [PATCH] drm/amdgpu: properly powergate Polaris12

UVD/VCE

Quoted 18 lines
Quoted 18 lines

on suspend

Quoted 13 lines · Evan Quan

If the powergating of UVD/VCE is in process, wait for its completion before proceeding(suspending). This can fix some hangs observed on suspending when UVD/VCE still using(e.g. issue "pm-suspend" when video is still playing).

Change-Id I36f39d9731e0a9638b52d5d92558b0ee9c23a9ed
---
    drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c | 5 +++++
    drivers/gpu/drm/amd/amdgpu/vce_v3_0.c | 5 +++++
    2 files changed, 10 insertions(+)
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c static int uvd_v6_0_suspend(void *handle) · line 555

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +0 -0
index 4eebf973a065..2fdce572baeb 100644
@@ -554,6 +554,11 @@ static int uvd_v6_0_suspend(void *handle)
    	int r;
    	struct amdgpu_device *adev = (struct amdgpu_device

*)handle;

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c static int uvd_v6_0_suspend(void *handle) · line 561

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +5 -0
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->uvd.idle_work);
+

If running idle is a prerequisite before going to suspend, then something else is missing here.

Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 5 lines

Otherwise, the hang looks more like a pending work launched after hardware is suspended and trying to access hardware. As the hardware is going to be suspended anyway, doesn't it work with cancel_delayed_work_sync - making sure that nothing is going to be launched later to access hardware?

[Quan, Evan] The reason we chose flush_delayed_work instead of

cancel_delayed_work_sync is we think those operations performed in idle_work(dpm disablement, powergating) seems needed considering

the

Quoted 7 lines
Quoted 6 lines

action is 'suspend'. So, instead of "cancel", maybe waiting for them completion is more proper.

But it will do so only if the work is scheduled - so it doesn't seem to be a prerequisite for suspend. If it was a prerequisite, then the existing code is missing that (so that it gets done for all cases).

[Quan, Evan] Just confirmed that cancel_delayed_work_sync() alone

cannot work.

Quoted 1 line

In fact, our current driver already get cancel_delayed_work_sync()

called(e.g. in amdgpu_uvd_suspend() on the path of uvd_v6_0_suspend).

Thanks Evan for checking this further.

If cancel_delayed_work_sync() is called in amdgpu_uvd_suspend(), then it means that originally executing idle_work was not considered as a prerequisite for suspending.

_uvd_suspend() is called "after" uvd_v6_0_hw_fini(adev), that maybe a little too late.

Quoted 4 lines

To get the issue fixed, it has to be either:
1. flush_delayed_work()
Or
2. cancel_delayed_work_sync + amdgpu_dpm_enable_uvd/vce(adev,

false)

Quoted 1 line

(the job performed in idle work)

At minimum, it proves that disabling dpm is required for suspend.

Quoted 1 line

Btw, I do not think flush_delayed_work() is an incomplete fix. Since the

UVD/VCE idle work is appended on ring->funcs->end_use.

Quoted 1 line

So, as long as the UVD/VCE ring used and ended(it will since at least there is

ring/ib test), there will be a chance to get the work waited and completed.

I agree that it fixes the issue, only thing is someone should look further to provide the right sequence of uvd suspend.

Quoted 8 lines

It doesn't give a root cause/right sequence - it only tells that forcing to execute idle_work fixes the problem probably due to an extra delay added by increased execution time or it disables DPM or something else. Someone should confirm that all of idle_work or a part of it (as dpm disable) is required for proper suspend sequence.

That said, I don't have any objections to this fix either. If there are other things, probably fix it this way and move on.

[Quan, Evan] Thanks Lijo. I checked more yesterday.
I found the issue was not seen on sienna cichlid and I checked its related source code(vcn_v3_0.c).
I think I might got some clues what's the right sequence and the reason why adding amdgpu_dpm_enable_uvd/vce(adev,false) can address the issue. For vcn_v3_0.c, the cleanups performed on suspend:
- cancel_delayed_work_sync
- vcn_v3_0_set_powergating_state()
- There is actually more jobs performed than just powergating. The naming is a little confusing.
- what it does are: powergating enablement + clockgating enablement + dpm disablement

For uvd_v6_0.c(where the issue was found), the above cleanups are missing. But if "cancel_delayed_work_sync + amdgpu_dpm_enable_uvd/vce(adev," added, the sequence will be just like above. That might explain why it works.
- cancel_delayed_work_sync
- amdgpu_dpm_enable_uvd/vce()
- Again, the naming is confusing. It actually performs more than just dpm disablement/enablmenet.
- What it does are: powergating enablement + clockgating enablement + dpm disablement

So, to me it seems proper to introduce a quick fix by " cancel_delayed_work_sync + amdgpu_dpm_enable_uvd/vce()".
A more proper long-term solution may be as the latest VCN(to put all the necessary cleanups in ->set_powergating_state()). But that will affect more than S3. I will leave that to Leo or James who are experts on UVD/VCE/VCN relates.

BR
Evan

Quoted 51 lines

false)

Thanks,
Lijo

Quoted 24 lines

BR
Evan

Quoted 21 lines
Quoted 6 lines
Quoted 4 lines

Then this may be a potential issue for other suspend calls also where work is pending to be launched when hardware is suspended.

[Quan, Evan] Do you mean we need to check whether there is similar issue

for other IPs?

Quoted 1 line

Yes, if there are cases where other IPs may schedule a delayed work and call hw_fini without cancelling the work.

Thanks,
Lijo

Quoted 6 lines

BR
Evan

Quoted 4 lines

Thanks,
Lijo

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

drivers/gpu/drm/amd/amdgpu/vce_v3_0.c static int vce_v3_0_suspend(void *handle) · line 504

Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +0 -0
    	r = uvd_v6_0_hw_fini(adev);
    	if (r)
    		return r;
drivers/gpu/drm/amd/amdgpu/vce_v3_0.c +0 -0
index 6d9108fa22e0..f0adecd5ec0b 100644
@@ -503,6 +503,11 @@ static int vce_v3_0_suspend(void *handle)
    	int r;
    	struct amdgpu_device *adev = (struct amdgpu_device

*)handle;

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/vce_v3_0.c +5 -0
+	/*
+	 * If the powergating is in process, wait for its completion.
+	 */
+	flush_delayed_work(&adev->vce.idle_work);
+
    	r = vce_v3_0_hw_fini(adev);
    	if (r)
    		return r;