Skip to contentSkip to conversation

Pageflip fixes for the amd dal/dc staging tree.

PATCH In review Mario Kleiner 4 participants 6 messages Apr 21, 2017 – Apr 24, 2017 1 message not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi

These two patches were written and tested against Harry's hwentland amd-staging-dc-drm-next branch from a few days ago. Not sure if that is the right one to base dal/dc patches against, but it looks recent and active enough?

They fix pageflip scheduling to always flip at the right target vblank and pageflip completion events to send out the correct vblank count and timestamp of flip completion.

Successfully tested on AMD R9 380 Tonga Pro - VI/DCE-10. After applying these, the dal staging branch performs nicely with my tests :)

I've also tested amdgpu-pro 17.10.2 on the same setup against Linux 4.4/4.8 and found a couple of similar bugs and fixes, but that in a separate mail.

-mario

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#1 lore raw

PATCH 1/2 drm/amd/display: Fix race between vblank irq and pageflip irq.

Since DC now uses CRTC_VERTICAL_INTERRUPT0 as VBLANK irq trigger and vblank interrupts actually happen earliest at start of vblank, instead of a bit before vblank, we no longer need some of the fudging logic to deal with too early vblank irq handling (grep for lb_vblank_lead_lines). This itself fixes a pageflip scheduling bug in DC, caused by uninitialized use of lb_vblank_lead_lines, with a wrong startup value of 0. Thanks to the new vblank irq trigger this value of zero is now actually correct for DC :).

A new problem is that vblank irq's race against pflip irq's, and as both can fire at first line of vblank, it is no longer guaranteed that vblank irq handling (therefore -> drm_handle_vblank() -> drm_update_vblank_count()) executes before pflip irq handling for a given vblank interval when a pageflip completes. Therefore the vblank count and timestamps emitted to user-space as part of the pageflip completion event will be often stale and cause new timestamping and swap scheduling errors in user-space.

This was observed with large frequency on R9 380 Tonga Pro.

Fix this by enforcing a vblank count+timestamp update right before emitting the pageflip completion event from the pflip irq handler. The logic in core drm_update_vblank_count() makes sure that no redundant or conflicting updates happen, iow. the call turns into a no-op if it wasn't needed for that vblank, burning a few microseconds of cpu time though.

Successfully tested on AMD R9 380 "Tonga Pro" (VI/DCE 10) with DC enabled on the current DC staging branch. Independent measurement of pageflip completion timing with special hardware measurement equipment now confirms correct pageflip timestamps and counts in the pageflip completion events.

Note that there is another unresolved pageflip bug present in current dc staging, which causes pageflips to complete one vblank too early when the pageflip ioctl gets called while in vblank. Something seems to be amiss in the way amdgpu_dm_do_flip() handles 'target_vblank', or how amdgpu_dm_atomic_commit_tail() computes 'target' for calling amdgpu_dm_do_flip().

Cc Harry Wentland <[email protected]>
Cc Andrey Grodzovsky <[email protected]>
Cc Alex Deucher <[email protected]>
Cc Michel Dänzer <[email protected]>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +3 -0
index 794362e..0d77b0a 100644
@@ -208,6 +208,9 @@ static void dm_pflip_high_irq(void *interrupt_params)
 	if (amdgpu_crtc->event
 			&& amdgpu_crtc->event->event.base.type
 			== DRM_EVENT_FLIP_COMPLETE) {
+		/* Update to correct count/ts if racing with vblank irq */
+		drm_accurate_vblank_count(&amdgpu_crtc->base);
+
 		drm_crtc_send_vblank_event(&amdgpu_crtc->base, amdgpu_crtc->event);
 		/* page flip completed. clean up */
 		amdgpu_crtc->event = NULL;
Signature · 6 lines
2.7.4

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] drm/amd/display: Fix race between vblank irq and pageflip irq.

Quoted 33 lines · Mario Kleiner

Since DC now uses CRTC_VERTICAL_INTERRUPT0 as VBLANK irq trigger and vblank interrupts actually happen earliest at start of vblank, instead of a bit before vblank, we no longer need some of the fudging logic to deal with too early vblank irq handling (grep for lb_vblank_lead_lines). This itself fixes a pageflip scheduling bug in DC, caused by uninitialized use of lb_vblank_lead_lines, with a wrong startup value of 0. Thanks to the new vblank irq trigger this value of zero is now actually correct for DC :).

A new problem is that vblank irq's race against pflip irq's, and as both can fire at first line of vblank, it is no longer guaranteed that vblank irq handling (therefore -> drm_handle_vblank() -> drm_update_vblank_count()) executes before pflip irq handling for a given vblank interval when a pageflip completes. Therefore the vblank count and timestamps emitted to user-space as part of the pageflip completion event will be often stale and cause new timestamping and swap scheduling errors in user-space.

This was observed with large frequency on R9 380 Tonga Pro.

Fix this by enforcing a vblank count+timestamp update right before emitting the pageflip completion event from the pflip irq handler. The logic in core drm_update_vblank_count() makes sure that no redundant or conflicting updates happen, iow. the call turns into a no-op if it wasn't needed for that vblank, burning a few microseconds of cpu time though.

Successfully tested on AMD R9 380 "Tonga Pro" (VI/DCE 10) with DC enabled on the current DC staging branch. Independent measurement of pageflip completion timing with special hardware measurement equipment now confirms correct pageflip timestamps and counts in the pageflip completion events.

Quoted 6 lines

Note that there is another unresolved pageflip bug present in current dc staging, which causes pageflips to complete one vblank too early when the pageflip ioctl gets called while in vblank. Something seems to be amiss in the way amdgpu_dm_do_flip() handles 'target_vblank', or how amdgpu_dm_atomic_commit_tail() computes 'target' for calling amdgpu_dm_do_flip().

If the last paragraph refers to the problem fixed by patch 2, I'd drop that paragraph. With that,

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#3 lore raw

PATCH 2/2 drm/amd/display: Prevent premature pageflip when comitting in vblank.

In reply to Mario Kleiner

Make sure we do not program a hw pageflip inside vblank 'n' iff the atomic flip is comitted while inside the same vblank 'n'. We must defer such a flip by one refresh cycle to vblank 'n+1'.

Without this, pageflips programmed via X11 GLX_OML_sync_control extensions glXSwapBuffersMscOML(..., target_msc, ...); call and/or via DRI3/Present PresentPixmap(..., target_msc, ...); request will complete one vblank too early whenever target_msc > current_msc + 1, ie. more than 1 vblank in the future. In such a case, the call of the pageflip ioctl() would be triggered by a queued drmWaitVblank() vblank event, which itself gets dispatched inside the vblank one frame before the target_msc vblank.

Testing with this patch does no longer show any problems with OML_sync_control swap scheduling or flip completion timestamps. Tested on R9 380 Tonga.

Cc Harry Wentland <[email protected]>
Cc Alex Deucher <[email protected]>
Cc Michel Dänzer <[email protected]>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c +4 -5
index 086a842..19be2d9 100644
@@ -2460,6 +2460,9 @@ static void amdgpu_dm_do_flip(
 	struct amdgpu_device *adev = crtc->dev->dev_private;
 	bool async_flip = (acrtc->flip_flags & DRM_MODE_PAGE_FLIP_ASYNC) != 0;
 
+	/* Prepare wait for target vblank early - before the fence-waits */
+	target_vblank = target - drm_crtc_vblank_count(crtc) +
+			amdgpu_get_vblank_counter_kms(crtc->dev, acrtc->crtc_id);
 
 	/*TODO This might fail and hence better not used, wait
 	 * explicitly on fences instead
@@ -2478,13 +2481,9 @@ static void amdgpu_dm_do_flip(
 
 	amdgpu_bo_unreserve(abo);
 
-	/* Wait for target vblank */
 	/* Wait until we're out of the vertical blank period before the one
 	 * targeted by the flip
 	 */
-	target_vblank = target - drm_crtc_vblank_count(crtc) +
-			amdgpu_get_vblank_counter_kms(crtc->dev, acrtc->crtc_id);
-
 	while ((acrtc->enabled &&
 		(amdgpu_get_crtc_scanoutpos(adev->ddev, acrtc->crtc_id, 0,
 					&vpos, &hpos, NULL, NULL,
@@ -2763,7 +2762,7 @@ void amdgpu_dm_atomic_commit_tail(
 			amdgpu_dm_do_flip(
 					crtc,
 					fb,
-					drm_crtc_vblank_count(crtc));
+					drm_crtc_vblank_count(crtc) + 1);
 
 			wait_for_vblank =
 					acrtc->flip_flags & DRM_MODE_PAGE_FLIP_ASYNC ?
Signature · 6 lines
2.7.4

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

A message that is not in the archive (sent off-list or not yet received): 1492791786-24543-3-git-send-email-mario.kleiner.de-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

Reviewed-by
#5 lore raw

Re: [PATCH 2/2] drm/amd/display: Prevent premature pageflip when comitting in vblank.

Quoted 42 lines · Mario Kleiner

Make sure we do not program a hw pageflip inside vblank 'n' iff the atomic flip is comitted while inside the same vblank 'n'. We must defer such a flip by one refresh cycle to vblank 'n+1'.

Without this, pageflips programmed via X11 GLX_OML_sync_control extensions glXSwapBuffersMscOML(..., target_msc, ...); call and/or via DRI3/Present PresentPixmap(..., target_msc, ...); request will complete one vblank too early whenever target_msc > current_msc + 1, ie. more than 1 vblank in the future. In such a case, the call of the pageflip ioctl() would be triggered by a queued drmWaitVblank() vblank event, which itself gets dispatched inside the vblank one frame before the target_msc vblank.

Testing with this patch does no longer show any problems with OML_sync_control swap scheduling or flip completion timestamps. Tested on R9 380 Tonga.

Cc Harry Wentland <[email protected]>
Cc Alex Deucher <[email protected]>
Cc Michel Dänzer <[email protected]>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c +3 -0
index 086a842..19be2d9 100644
@@ -2460,6 +2460,9 @@ static void amdgpu_dm_do_flip(
 	struct amdgpu_device *adev = crtc->dev->dev_private;
 	bool async_flip = (acrtc->flip_flags & DRM_MODE_PAGE_FLIP_ASYNC) != 0;
 
+	/* Prepare wait for target vblank early - before the fence-waits */
+	target_vblank = target - drm_crtc_vblank_count(crtc) +
+			amdgpu_get_vblank_counter_kms(crtc->dev, acrtc->crtc_id);
 
 	/*TODO This might fail and hence better not used, wait
 	 * explicitly on fences instead
@@ -2478,13 +2481,9 @@ static void amdgpu_dm_do_flip(
 
 	amdgpu_bo_unreserve(abo);
 

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c static void amdgpu_dm_do_flip( · line 2489

Quoted 10 lines
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c +0 -4
-	/* Wait for target vblank */
 	/* Wait until we're out of the vertical blank period before the one
 	 * targeted by the flip
 	 */
-	target_vblank = target - drm_crtc_vblank_count(crtc) +
-			amdgpu_get_vblank_counter_kms(crtc->dev, acrtc->crtc_id);
-
 	while ((acrtc->enabled &&
 		(amdgpu_get_crtc_scanoutpos(adev->ddev, acrtc->crtc_id, 0,
 					&vpos, &hpos, NULL, NULL,

Makes sense.

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c void amdgpu_dm_atomic_commit_tail( · line 2768

Quoted 10 lines
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c +1 -1
@@ -2763,7 +2762,7 @@ void amdgpu_dm_atomic_commit_tail(
 			amdgpu_dm_do_flip(
 					crtc,
 					fb,
-					drm_crtc_vblank_count(crtc));
+					drm_crtc_vblank_count(crtc) + 1);
 
 			wait_for_vblank =
 					acrtc->flip_flags & DRM_MODE_PAGE_FLIP_ASYNC ?

I suspect this code runs relatively late, so if userspace calls DRM_IOCTL_MODE_PAGE_FLIP shortly before the start of vertical blank, it could result in the flip being unnecessarily delayed by one frame. I added drm_crtc_funcs::page_flip_target to address this.

Anyway, this is okay for now.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
maintainer Acked-by
#6 lore raw
In reply to Mario Kleiner
Quoted 6 lines · Mario Kleiner

Hi

These two patches were written and tested against Harry's hwentland amd-staging-dc-drm-next branch from a few days ago. Not sure if that is the right one to base dal/dc patches against, but it looks recent and active enough?

Yes, I suggest you use that one? It doesn't get extensive testing like Alex's amd-staging-4.9 but it's based on a more recent tree. Andrey and I use that for cleaning up DC.

Quoted 10 lines

They fix pageflip scheduling to always flip at the right target vblank and pageflip completion events to send out the correct vblank count and timestamp of flip completion.

Successfully tested on AMD R9 380 Tonga Pro - VI/DCE-10. After applying these, the dal staging branch performs nicely with my tests :)

I've also tested amdgpu-pro 17.10.2 on the same setup against Linux 4.4/4.8 and found a couple of similar bugs and fixes, but that in a separate mail.

Thanks for your extensive testing and the fixes. I'll let Andrey review this more thoroughly as he's spent quite a bit of time trying to get our flip/vblank model right. This series looks quite sensible to me, though, so feel free to add my

Harry

Quoted 3 lines

-mario

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel