Pageflip fixes for the amd dal/dc staging tree.
Subsystems (3)
Hi
These two patches were written and tested against Harry's hwentland amd-staging-dc-drm-next branch from a few days ago. Not sure if that is the right one to base dal/dc patches against, but it looks recent and active enough?
They fix pageflip scheduling to always flip at the right target vblank and pageflip completion events to send out the correct vblank count and timestamp of flip completion.
Successfully tested on AMD R9 380 Tonga Pro - VI/DCE-10. After applying these, the dal staging branch performs nicely with my tests :)
I've also tested amdgpu-pro 17.10.2 on the same setup against Linux 4.4/4.8 and found a couple of similar bugs and fixes, but that in a separate mail.
-mario
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
PATCH 1/2 drm/amd/display: Fix race between vblank irq and pageflip irq.
Since DC now uses CRTC_VERTICAL_INTERRUPT0 as VBLANK irq trigger and vblank interrupts actually happen earliest at start of vblank, instead of a bit before vblank, we no longer need some of the fudging logic to deal with too early vblank irq handling (grep for lb_vblank_lead_lines). This itself fixes a pageflip scheduling bug in DC, caused by uninitialized use of lb_vblank_lead_lines, with a wrong startup value of 0. Thanks to the new vblank irq trigger this value of zero is now actually correct for DC :).
A new problem is that vblank irq's race against pflip irq's, and as both can fire at first line of vblank, it is no longer guaranteed that vblank irq handling (therefore -> drm_handle_vblank() -> drm_update_vblank_count()) executes before pflip irq handling for a given vblank interval when a pageflip completes. Therefore the vblank count and timestamps emitted to user-space as part of the pageflip completion event will be often stale and cause new timestamping and swap scheduling errors in user-space.
This was observed with large frequency on R9 380 Tonga Pro.
Fix this by enforcing a vblank count+timestamp update right before emitting the pageflip completion event from the pflip irq handler. The logic in core drm_update_vblank_count() makes sure that no redundant or conflicting updates happen, iow. the call turns into a no-op if it wasn't needed for that vblank, burning a few microseconds of cpu time though.
Successfully tested on AMD R9 380 "Tonga Pro" (VI/DCE 10) with DC enabled on the current DC staging branch. Independent measurement of pageflip completion timing with special hardware measurement equipment now confirms correct pageflip timestamps and counts in the pageflip completion events.
Note that there is another unresolved pageflip bug present in current dc staging, which causes pageflips to complete one vblank too early when the pageflip ioctl gets called while in vblank. Something seems to be amiss in the way amdgpu_dm_do_flip() handles 'target_vblank', or how amdgpu_dm_atomic_commit_tail() computes 'target' for calling amdgpu_dm_do_flip().
--- drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +3 -0
@@ -208,6 +208,9 @@ static void dm_pflip_high_irq(void *interrupt_params)if (amdgpu_crtc->event && amdgpu_crtc->event->event.base.type ==DRM_EVENT_FLIP_COMPLETE ) { +/* Update to correct count/ts if racing with vblank irq */ +drm_accurate_vblank_count (&amdgpu_crtc->base); +drm_crtc_send_vblank_event (&amdgpu_crtc->base, amdgpu_crtc->event);/* page flip completed. clean up */ amdgpu_crtc->event =NULL ;
Signature · 6 lines
2.7.4 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH 1/2] drm/amd/display: Fix race between vblank irq and pageflip irq.
Quoted 33 lines · Mario Kleiner
Since DC now uses
CRTC_VERTICAL_INTERRUPT0as VBLANK irq trigger and vblank interrupts actually happen earliest at start of vblank, instead of a bit before vblank, we no longer need some of the fudging logic to deal with too early vblank irq handling (grep forlb_vblank_lead_lines). This itself fixes a pageflip scheduling bug in DC, caused by uninitialized use oflb_vblank_lead_lines, with a wrong startup value of 0. Thanks to the new vblank irq trigger this value of zero is now actually correct for DC :).A new problem is that vblank irq's race against pflip irq's, and as both can fire at first line of vblank, it is no longer guaranteed that vblank irq handling (therefore ->
drm_handle_vblank()->drm_update_vblank_count()) executes before pflip irq handling for a given vblank interval when a pageflip completes. Therefore the vblank count and timestamps emitted to user-space as part of the pageflip completion event will be often stale and cause new timestamping and swap scheduling errors in user-space.This was observed with large frequency on R9 380 Tonga Pro.
Fix this by enforcing a vblank count+timestamp update right before emitting the pageflip completion event from the pflip irq handler. The logic in core
drm_update_vblank_count()makes sure that no redundant or conflicting updates happen, iow. the call turns into a no-op if it wasn't needed for that vblank, burning a few microseconds of cpu time though.Successfully tested on AMD R9 380 "Tonga Pro" (VI/DCE 10) with DC enabled on the current DC staging branch. Independent measurement of pageflip completion timing with special hardware measurement equipment now confirms correct pageflip timestamps and counts in the pageflip completion events.
Quoted 6 lines
Note that there is another unresolved pageflip bug present in current dc staging, which causes pageflips to complete one vblank too early when the pageflip ioctl gets called while in vblank. Something seems to be amiss in the way
amdgpu_dm_do_flip()handles 'target_vblank', or howamdgpu_dm_atomic_commit_tail()computes 'target' for callingamdgpu_dm_do_flip().
If the last paragraph refers to the problem fixed by patch 2, I'd drop that paragraph. With that,
Signature · 6 lines
Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
PATCH 2/2 drm/amd/display: Prevent premature pageflip when comitting in vblank.
In reply to Mario KleinerMake sure we do not program a hw pageflip inside vblank 'n' iff the atomic flip is comitted while inside the same vblank 'n'. We must defer such a flip by one refresh cycle to vblank 'n+1'.
Without this, pageflips programmed via X11 GLX_OML_sync_control extensions glXSwapBuffersMscOML(..., target_msc, ...); call and/or via DRI3/Present PresentPixmap(..., target_msc, ...); request will complete one vblank too early whenever target_msc > current_msc + 1, ie. more than 1 vblank in the future. In such a case, the call of the pageflip ioctl() would be triggered by a queued drmWaitVblank() vblank event, which itself gets dispatched inside the vblank one frame before the target_msc vblank.
Testing with this patch does no longer show any problems with OML_sync_control swap scheduling or flip completion timestamps. Tested on R9 380 Tonga.
--- drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c +4 -5
@@ -2460,6 +2460,9 @@ static void amdgpu_dm_do_flip(struct amdgpu_device *adev = crtc->dev->dev_private;bool async_flip = (acrtc->flip_flags &DRM_MODE_PAGE_FLIP_ASYNC ) !=0 ; +/* Prepare wait for target vblank early - before the fence-waits */ + target_vblank = target -drm_crtc_vblank_count (crtc) + +amdgpu_get_vblank_counter_kms (crtc->dev, acrtc->crtc_id);/*TODO This might fail and hence better not used, wait * explicitly on fences instead
@@ -2478,13 +2481,9 @@ static void amdgpu_dm_do_flip(amdgpu_bo_unreserve (abo); -/* Wait for target vblank */ /* Wait until we're out of the vertical blank period before the one * targeted by the flip */ - target_vblank = target -drm_crtc_vblank_count (crtc) + -amdgpu_get_vblank_counter_kms (crtc->dev, acrtc->crtc_id); - while ((acrtc->enabled && (amdgpu_get_crtc_scanoutpos(adev->ddev, acrtc->crtc_id, 0, &vpos, &hpos, NULL, NULL,
@@ -2763,7 +2762,7 @@ void amdgpu_dm_atomic_commit_tail(amdgpu_dm_do_flip ( crtc, fb, -drm_crtc_vblank_count (crtc)); +drm_crtc_vblank_count (crtc) +1 ); wait_for_vblank = acrtc->flip_flags & DRM_MODE_PAGE_FLIP_ASYNC ?
Signature · 6 lines
2.7.4 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
A message that is not in the archive (sent off-list or not yet received): 1492791786-24543-3-git-send-email-mario.kleiner.de-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Michel Dänzer
Reviewed-byRe: [PATCH 2/2] drm/amd/display: Prevent premature pageflip when comitting in vblank.
Quoted 42 lines · Mario Kleiner
Make sure we do not program a hw pageflip inside vblank 'n' iff the atomic flip is comitted while inside the same vblank 'n'. We must defer such a flip by one refresh cycle to vblank 'n+1'.
Without this, pageflips programmed via X11
GLX_OML_sync_controlextensionsglXSwapBuffersMscOML(..., target_msc, ...); call and/or via DRI3/PresentPresentPixmap(..., target_msc, ...); request will complete one vblank too early whenevertarget_msc>current_msc+ 1, ie. more than 1 vblank in the future. In such a case, the call of the pageflipioctl()would be triggered by a queueddrmWaitVblank()vblank event, which itself gets dispatched inside the vblank one frame before thetarget_mscvblank.Testing with this patch does no longer show any problems with
OML_sync_controlswap scheduling or flip completion timestamps. Tested on R9 380 Tonga.Signed-off-by Mario Kleiner <[email protected]>Cc Harry Wentland <[email protected]>Cc Alex Deucher <[email protected]>Cc Michel Dänzer <[email protected]>--- drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c +3
-0@@ -2460,6 +2460,9 @@ static void amdgpu_dm_do_flip(struct amdgpu_device *adev = crtc->dev->dev_private;bool async_flip = (acrtc->flip_flags &DRM_MODE_PAGE_FLIP_ASYNC ) !=0 ; +/* Prepare wait for target vblank early - before the fence-waits */ + target_vblank = target -drm_crtc_vblank_count (crtc) + +amdgpu_get_vblank_counter_kms (crtc->dev, acrtc->crtc_id);/*TODO This might fail and hence better not used, wait * explicitly on fences instead @@ -2478,13 +2481,9 @@ static void amdgpu_dm_do_flip(amdgpu_bo_unreserve (abo);
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c static void amdgpu_dm_do_flip( · line 2489
Quoted 10 lines
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c +0
-4-/* Wait for target vblank */ /* Wait until we're out of the vertical blank period before the one * targeted by the flip */ - target_vblank = target -drm_crtc_vblank_count (crtc) + -amdgpu_get_vblank_counter_kms (crtc->dev, acrtc->crtc_id); - while ((acrtc->enabled && (amdgpu_get_crtc_scanoutpos(adev->ddev, acrtc->crtc_id, 0, &vpos, &hpos, NULL, NULL,
Makes sense.
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c void amdgpu_dm_atomic_commit_tail( · line 2768
Quoted 10 lines
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_types.c +1
-1@@ -2763,7 +2762,7 @@ void amdgpu_dm_atomic_commit_tail(amdgpu_dm_do_flip ( crtc, fb, -drm_crtc_vblank_count (crtc)); +drm_crtc_vblank_count (crtc) +1 ); wait_for_vblank = acrtc->flip_flags & DRM_MODE_PAGE_FLIP_ASYNC ?
I suspect this code runs relatively late, so if userspace calls DRM_IOCTL_MODE_PAGE_FLIP shortly before the start of vertical blank, it could result in the flip being unnecessarily delayed by one frame. I added drm_crtc_funcs::page_flip_target to address this.
Anyway, this is okay for now.
Signature · 6 lines
Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Quoted 6 lines · Mario Kleiner
Hi
These two patches were written and tested against Harry's hwentland amd-staging-dc-drm-next branch from a few days ago. Not sure if that is the right one to base dal/dc patches against, but it looks recent and active enough?
Yes, I suggest you use that one? It doesn't get extensive testing like Alex's amd-staging-4.9 but it's based on a more recent tree. Andrey and I use that for cleaning up DC.
Quoted 10 lines
They fix pageflip scheduling to always flip at the right target vblank and pageflip completion events to send out the correct vblank count and timestamp of flip completion.
Successfully tested on AMD R9 380 Tonga Pro - VI/DCE-10. After applying these, the dal staging branch performs nicely with my tests :)
I've also tested amdgpu-pro 17.10.2 on the same setup against Linux 4.4/4.8 and found a couple of similar bugs and fixes, but that in a separate mail.
Thanks for your extensive testing and the fixes. I'll let Andrey review this more thoroughly as he's spent quite a bit of time trying to get our flip/vblank model right. This series looks quite sensible to me, though, so feel free to add my
Harry
Quoted 3 lines
-mario
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel