Skip to contentSkip to conversation

RFC for a render API to support adaptive sync and VRR

DISCUSSION Harry Wentland 19 participants 43 messages Apr 9, 2018 – Aug 17, 2018 23 messages not in the archive

lore
Share & tools
lore
#0 lore raw

=== What is adaptive sync and VRR? ===

Adaptive sync has been part of the DisplayPort spec for a while now and allows graphics adapters to drive displays with varying frame timings. VRR (variable refresh rate) is essentially the same, but defined for HDMI.

=== Why allow variable frame timings? ===

Variable render times don't align with fixed refresh rates, leading to stuttering, tearing, and/or input lag.

e.g. (rc = render completion, dr = display refresh)

rc B C D E F
dr A B C C D E F

^ ^
frame missed
repeated display
twice refresh

=== Other use cases of adaptive sync ====

Beside the variable render case, adaptive sync also allows adjustment of refresh rates without a mode change. One such use case would be 24 Hz video.

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
* bool variable_refresh_compatible
* int target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
* drive monitor at timing's normal refresh rate

variable_refresh_compatible = 1, target_frame_duration_ns = 0
* send new frame to monitor as soon as it's available, if within min/max of monitor's reported capabilities

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
* send new frame to monitor with the specified target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

=== Previous discussions ===

https://lists.freedesktop.org/archives/dri-devel/2017-October/155207.html

=== Feedback and moving forward ===

I'm hoping to get some feedback on this or continue the discussion on how adaptive sync / VRR might look like in the DRM ecosystem. Once there are no major concerns or objections left we'll probably start creating some patches to sketch this out a bit better and see how it looks in practice.

Cheers,
Harry
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): c79421ce-9408-5cd6-5de3-0fa521146e4b-5C7GfCeVMHo@public.gmane.org

#2 lore raw

Adding dri-devel, which I should've included from the start.

Quoted 62 lines · Harry Wentland

=== What is adaptive sync and VRR? ===

Adaptive sync has been part of the DisplayPort spec for a while now and allows graphics adapters to drive displays with varying frame timings. VRR (variable refresh rate) is essentially the same, but defined for HDMI.

=== Why allow variable frame timings? ===

Variable render times don't align with fixed refresh rates, leading to stuttering, tearing, and/or input lag.

e.g. (rc = render completion, dr = display refresh)

rc B C D E F
dr A B C C D E F

^ ^
frame missed
repeated display
twice refresh

=== Other use cases of adaptive sync ====

Beside the variable render case, adaptive sync also allows adjustment of refresh rates without a mode change. One such use case would be 24 Hz video.

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
* bool variable_refresh_compatible
* int target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
* drive monitor at timing's normal refresh rate

variable_refresh_compatible = 1, target_frame_duration_ns = 0
* send new frame to monitor as soon as it's available, if within min/max of monitor's reported capabilities

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
* send new frame to monitor with the specified target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

=== Previous discussions ===

https://lists.freedesktop.org/archives/dri-devel/2017-October/155207.html

=== Feedback and moving forward ===

I'm hoping to get some feedback on this or continue the discussion on how adaptive sync / VRR might look like in the DRM ecosystem. Once there are no major concerns or objections left we'll probably start creating some patches to sketch this out a bit better and see how it looks in practice.

Quoted 7 lines

Cheers,
Harry
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 435ebd04-0435-5a6a-9f1e-e4c4fc629aa9-5C7GfCeVMHo@public.gmane.org

#4 lore raw

Thanks for initiating the discussion. Find my comments below:

Quoted 30 lines · Harry Wentland

Adding dri-devel, which I should've included from the start.

Quoted 27 lines · Harry Wentland

=== What is adaptive sync and VRR? ===

Adaptive sync has been part of the DisplayPort spec for a while now and allows graphics adapters to drive displays with varying frame timings. VRR (variable refresh rate) is essentially the same, but defined for HDMI.

=== Why allow variable frame timings? ===

Variable render times don't align with fixed refresh rates, leading to stuttering, tearing, and/or input lag.

e.g. (rc = render completion, dr = display refresh)

rc B C D E F
dr A B C C D E F

^ ^
frame missed
repeated display
twice refresh

=== Other use cases of adaptive sync ====

Beside the variable render case, adaptive sync also allows adjustment of refresh rates without a mode change. One such use case would be 24 Hz video.

One of the the advantages here when the render speed is slower than the display refresh rate, since we are stretching the vertical blanking interval the display adapters will follow "draw fast and then go idle" approach. This gives power savings when render rate is lower than the display refresh rate.

Quoted 13 lines
Quoted 13 lines

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
* bool variable_refresh_compatible
* int target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
* drive monitor at timing's normal refresh rate

Quoted 8 lines
Quoted 8 lines

variable_refresh_compatible = 1, target_frame_duration_ns = 0
* send new frame to monitor as soon as it's available, if within min/max of monitor's reported capabilities

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
* send new frame to monitor with the specified target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

What I would like is two sets of properties on a CRTC or preferably on a connector:

KMD properties that UMD can query:
* vrr_capable - This will be an immutable property for exposing hardware's capability of supporting VRR. This will be set by the kernel after reading the EDID mode information and monitor range capabilities.
* vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max refresh rates supported.
These properties are optional and will be created and attached to the DP/eDP connector when the connector
is getting intialized.

Properties that you mentioned above that the UMD can set before kernel can enable VRR functionality *bool vrr_enable or vrr_compatible
target_frame_duration_ns

The monitor only specifies the monitor range through EDID. Apart from this should we also need to scan the modes and check if there are modes that have the same pixel clock and horizontal timings but variable vertical totals?

I have RFC patches for all the above mentioned. If we get a concensus/agreement on the above properties and method to check monitor's VRR capability, I can submit those patches atleast as RFC.

Regards
Manasi

Quoted 14 lines
Quoted 14 lines

=== Previous discussions ===

https://lists.freedesktop.org/archives/dri-devel/2017-October/155207.html

=== Feedback and moving forward ===

I'm hoping to get some feedback on this or continue the discussion on how adaptive sync / VRR might look like in the DRM ecosystem. Once there are no major concerns or objections left we'll probably start creating some patches to sketch this out a bit better and see how it looks in practice.

Quoted 7 lines
Quoted 7 lines

Cheers,
Harry
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): [email protected]

#6 lore raw

Am 09.04.2018 um 23:45 schrieb Manasi Navare:

Quoted 56 lines

Thanks for initiating the discussion. Find my comments below:

Quoted 30 lines · Harry Wentland

Adding dri-devel, which I should've included from the start.

Quoted 27 lines · Harry Wentland

=== What is adaptive sync and VRR? ===

Adaptive sync has been part of the DisplayPort spec for a while now and allows graphics adapters to drive displays with varying frame timings. VRR (variable refresh rate) is essentially the same, but defined for HDMI.

=== Why allow variable frame timings? ===

Variable render times don't align with fixed refresh rates, leading to stuttering, tearing, and/or input lag.

e.g. (rc = render completion, dr = display refresh)

rc B C D E F
dr A B C C D E F

^ ^
frame missed
repeated display
twice refresh

=== Other use cases of adaptive sync ====

Beside the variable render case, adaptive sync also allows adjustment of refresh rates without a mode change. One such use case would be 24 Hz video.

One of the the advantages here when the render speed is slower than the display refresh rate, since we are stretching the vertical blanking interval the display adapters will follow "draw fast and then go idle" approach. This gives power savings when render rate is lower than the display refresh rate.

Quoted 12 lines
Quoted 12 lines

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
* bool variable_refresh_compatible
* int target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
* drive monitor at timing's normal refresh rate

Quoted 8 lines
Quoted 8 lines

variable_refresh_compatible = 1, target_frame_duration_ns = 0
* send new frame to monitor as soon as it's available, if within min/max of monitor's reported capabilities

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
* send new frame to monitor with the specified target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

Quoted 8 lines

What I would like is two sets of properties on a CRTC or preferably on a connector:

KMD properties that UMD can query:
* vrr_capable - This will be an immutable property for exposing hardware's capability of supporting VRR. This will be set by the kernel after reading the EDID mode information and monitor range capabilities.
* vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max refresh rates supported.
These properties are optional and will be created and attached to the DP/eDP connector when the connector
is getting intialized.

Mhm, aren't those properties actually per mode and not per CRTC/connector?

Quoted 3 lines

Properties that you mentioned above that the UMD can set before kernel can enable VRR functionality *bool vrr_enable or vrr_compatible
target_frame_duration_ns

Yeah, that certainly makes sense. But target_frame_duration_ns is a bad name/semantics.

We should use an absolute timestamp where the frame should be presented, otherwise you could run into a bunch of trouble with IOCTL restarts or missed blanks.

Regards,
Christian.

Quoted 30 lines

The monitor only specifies the monitor range through EDID. Apart from this should we also need to scan the modes and check if there are modes that have the same pixel clock and horizontal timings but variable vertical totals?

I have RFC patches for all the above mentioned. If we get a concensus/agreement on the above properties and method to check monitor's VRR capability, I can submit those patches atleast as RFC.

Regards
Manasi

Quoted 20 lines
Quoted 20 lines

=== Previous discussions ===

https://lists.freedesktop.org/archives/dri-devel/2017-October/155207.html

=== Feedback and moving forward ===

I'm hoping to get some feedback on this or continue the discussion on how adaptive sync / VRR might look like in the DRM ecosystem. Once there are no major concerns or objections left we'll probably start creating some patches to sketch this out a bit better and see how it looks in practice.

Cheers,
Harry
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#7 lore raw
Quoted 51 lines · Christian König

Am 09.04.2018 um 23:45 schrieb Manasi Navare:

Quoted 32 lines

Thanks for initiating the discussion. Find my comments below:

Quoted 20 lines · Harry Wentland
Quoted 19 lines · Harry Wentland

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
  * bool    variable_refresh_compatible
  * int    target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
  * drive monitor at timing's normal refresh rate

variable_refresh_compatible = 1, target_frame_duration_ns = 0
  * send new frame to monitor as soon as it's available, if within min/max of monitor's reported capabilities

Quoted 7 lines
Quoted 7 lines

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
  * send new frame to monitor with the specified target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

What I would like is two sets of properties on a CRTC or preferably on a connector:

Quoted 10 lines

KMD properties that UMD can query:
* vrr_capable -  This will be an immutable property for exposing hardware's capability of supporting VRR. This will be set by the kernel after
reading the EDID mode information and monitor range capabilities.
* vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max refresh rates supported.
These properties are optional and will be created and attached to the DP/eDP connector when the connector
is getting intialized.

Mhm, aren't those properties actually per mode and not per CRTC/connector?

Quoted 4 lines

Properties that you mentioned above that the UMD can set before kernel can enable VRR functionality
*bool vrr_enable or vrr_compatible
target_frame_duration_ns

Quoted 6 lines

Yeah, that certainly makes sense. But target_frame_duration_ns is a bad name/semantics.

We should use an absolute timestamp where the frame should be presented, otherwise you could run into a bunch of trouble with IOCTL restarts or missed blanks.

Also, a fixed target frame duration isn't suitable even for video playback, due to drift between the video and audio clocks.

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

A message that is not in the archive (sent off-list or not yet received): b75e31f7-ce59-de97-5445-cb4c03ff7336-otUistvHUpPR7s880joybQ@public.gmane.org

#9 lore raw
Quoted 59 lines · Michel Dänzer
Quoted 51 lines · Christian König

Am 09.04.2018 um 23:45 schrieb Manasi Navare:

Quoted 32 lines

Thanks for initiating the discussion. Find my comments below:

Quoted 20 lines · Harry Wentland
Quoted 19 lines · Harry Wentland

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
  * bool    variable_refresh_compatible
  * int    target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
  * drive monitor at timing's normal refresh rate

variable_refresh_compatible = 1, target_frame_duration_ns = 0
  * send new frame to monitor as soon as it's available, if within min/max of monitor's reported capabilities

Quoted 7 lines
Quoted 7 lines

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
  * send new frame to monitor with the specified target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

What I would like is two sets of properties on a CRTC or preferably on a connector:

Quoted 10 lines

KMD properties that UMD can query:
* vrr_capable -  This will be an immutable property for exposing hardware's capability of supporting VRR. This will be set by the kernel after
reading the EDID mode information and monitor range capabilities.
* vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max refresh rates supported.
These properties are optional and will be created and attached to the DP/eDP connector when the connector
is getting intialized.

Mhm, aren't those properties actually per mode and not per CRTC/connector?

Quoted 4 lines

Properties that you mentioned above that the UMD can set before kernel can enable VRR functionality
*bool vrr_enable or vrr_compatible
target_frame_duration_ns

Quoted 6 lines

Yeah, that certainly makes sense. But target_frame_duration_ns is a bad name/semantics.

We should use an absolute timestamp where the frame should be presented, otherwise you could run into a bunch of trouble with IOCTL restarts or missed blanks.

Quoted 7 lines

Also, a fixed target frame duration isn't suitable even for video playback, due to drift between the video and audio clocks.

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

I'm not sure if the driver can ever give a guarantee of the exact time a flip occurs. What we have control over with our HW is frame duration.

Are Croteam devs trying to predict render times? I'm not sure how that would work. We've had bad experience in the past with games that try to do framepacing as that's usually not accurate and tends to lead to more problems than benefits.

Harry

Quoted 1 line

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): ae9992e5-43ab-9636-13e1-8a5c692480b5-5C7GfCeVMHo@public.gmane.org

#11 lore raw

Am 10.04.2018 um 17:08 schrieb Harry Wentland:

Quoted 56 lines · Michel Dänzer
Quoted 55 lines · Christian König
Quoted 54 lines

Am 09.04.2018 um 23:45 schrieb Manasi Navare:

Quoted 31 lines

Thanks for initiating the discussion. Find my comments below:

Quoted 19 lines · Harry Wentland
Quoted 18 lines · Harry Wentland

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
  * bool    variable_refresh_compatible
  * int    target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
  * drive monitor at timing's normal refresh rate

variable_refresh_compatible = 1, target_frame_duration_ns = 0
  * send new frame to monitor as soon as it's available, if within
min/max of monitor's reported capabilities

Quoted 7 lines
Quoted 7 lines

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
  * send new frame to monitor with the specified
target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

What I would like is two sets of properties on a CRTC or preferably on a connector:

Quoted 10 lines

KMD properties that UMD can query:
* vrr_capable -  This will be an immutable property for exposing hardware's capability of supporting VRR. This will be set by the kernel after
reading the EDID mode information and monitor range capabilities.
* vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max refresh rates supported.
These properties are optional and will be created and attached to the DP/eDP connector when the connector
is getting intialized.

Mhm, aren't those properties actually per mode and not per CRTC/connector?

Quoted 4 lines

Properties that you mentioned above that the UMD can set before kernel can enable VRR functionality
*bool vrr_enable or vrr_compatible
target_frame_duration_ns

Yeah, that certainly makes sense. But target_frame_duration_ns is a bad name/semantics.

We should use an absolute timestamp where the frame should be presented, otherwise you could run into a bunch of trouble with IOCTL restarts or missed blanks.

Quoted 8 lines
Quoted 7 lines

Also, a fixed target frame duration isn't suitable even for video playback, due to drift between the video and audio clocks.

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

I'm not sure if the driver can ever give a guarantee of the exact time a flip occurs. What we have control over with our HW is frame duration.

Sounds like you misunderstood what we mean here.

The driver does not need to give an exact guarantee that a flip happens at that time. It should just not flip before that specific time.

E.g. when we missed a VBLANK your approach would still wait for the specific amount of time, while an absolute timestamp would mean to flip as soon as possible after that timestamp passed.

As Michel noted that is also exactly what video players need.

Quoted 2 lines

Are Croteam devs trying to predict render times? I'm not sure how that would work. We've had bad experience in the past with games that try to do framepacing as that's usually not accurate and tends to lead to more problems than benefits.

As far as I understand that is just a regulated feedback system, e.g. the application records the timestamps of the last three frames (or so) and then uses that + margin to as world time for the 3D rendering.

When the application has finished sending all rendering commands it sends the frame to be displayed exactly with that timestamp as well.

The timestamp when the frame was actually displayed is then used again as input to the algorithm.

Regards,
Christian.

Quoted 7 lines

Harry

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#12 lore raw
In reply to a message not in the archive
Quoted 10 lines

-----Original Message-----
From: Wentland, Harry
Sent: Tuesday, April 10, 2018 11:08
To: Michel Dänzer <[email protected]>; Koenig, Christian <[email protected]>; Manasi Navare <[email protected]>
Cc: Haehnle, Nicolai <[email protected]>; Daniel Vetter <[email protected]>; Daenzer, Michel <[email protected]>; dri-devel <[email protected]>; amd-gfx mailing list <[email protected]>; Deucher, Alexander <[email protected]>; Cyr, Aric <[email protected]>; Koo, Anthony <[email protected]>
Subject: Re: RFC for a render API to support adaptive sync and VRR

Quoted 62 lines
Quoted 61 lines · Michel Dänzer
Quoted 51 lines · Christian König

Am 09.04.2018 um 23:45 schrieb Manasi Navare:

Quoted 32 lines

Thanks for initiating the discussion. Find my comments below:

Quoted 20 lines · Harry Wentland
Quoted 19 lines · Harry Wentland

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
  * bool    variable_refresh_compatible
  * int    target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
  * drive monitor at timing's normal refresh rate

variable_refresh_compatible = 1, target_frame_duration_ns = 0
  * send new frame to monitor as soon as it's available, if within min/max of monitor's reported capabilities

Quoted 7 lines
Quoted 7 lines

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
  * send new frame to monitor with the specified target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

What I would like is two sets of properties on a CRTC or preferably on a connector:

Quoted 10 lines

KMD properties that UMD can query:
* vrr_capable -  This will be an immutable property for exposing hardware's capability of supporting VRR. This will be set by the kernel after
reading the EDID mode information and monitor range capabilities.
* vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max refresh rates supported.
These properties are optional and will be created and attached to the DP/eDP connector when the connector
is getting intialized.

Mhm, aren't those properties actually per mode and not per CRTC/connector?

Quoted 4 lines

Properties that you mentioned above that the UMD can set before kernel can enable VRR functionality
*bool vrr_enable or vrr_compatible
target_frame_duration_ns

Quoted 6 lines

Yeah, that certainly makes sense. But target_frame_duration_ns is a bad name/semantics.

We should use an absolute timestamp where the frame should be presented, otherwise you could run into a bunch of trouble with IOCTL restarts or missed blanks.

Also, a fixed target frame duration isn't suitable even for video playback, due to drift between the video and audio clocks.

Why? Even if they drift, you know you want to show your 24Hz video frame for 41.6666ms and adaptive sync can ensure that with reasonable accuracy. All we're doing is eliminating the need for frame rate converters from the application and offloading that to hardware.

Quoted 10 lines
Quoted 4 lines

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

I'm not sure if the driver can ever give a guarantee of the exact time a flip occurs. What we have control over with our HW is frame duration.

Are Croteam devs trying to predict render times? I'm not sure how that would work. We've had bad experience in the past with games that try to do framepacing as that's usually not accurate and tends to lead to more problems than benefits.

For gaming, it doesn't make sense nor is it feasible to know how exactly how long a render will take with microsecond precision, very coarse guesses at best. The point of adaptive sync is that it works *transparently* for the majority of cases, within the capability of the HW and driver. We don't want to have every game re-write their engine to support this, but we do want the majority to "just work".

The only exception is the video case where an application may want to request a fixed frame duration aligned to the video content. This requires an explicit interface for the video app, and our proposal is to keep it simple: app knows how long a frame should be presented for, and we try to honour that.

-Aric
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#13 lore raw

Am 10.04.2018 um 17:35 schrieb Cyr, Aric:

Quoted 75 lines
Quoted 10 lines

-----Original Message-----
From: Wentland, Harry
Sent: Tuesday, April 10, 2018 11:08
To: Michel Dänzer <[email protected]>; Koenig, Christian <[email protected]>; Manasi Navare <[email protected]>
Cc: Haehnle, Nicolai <[email protected]>; Daniel Vetter <[email protected]>; Daenzer, Michel <[email protected]>; dri-devel <[email protected]>; amd-gfx mailing list <[email protected]>; Deucher, Alexander <[email protected]>; Cyr, Aric <[email protected]>; Koo, Anthony <[email protected]>
Subject: Re: RFC for a render API to support adaptive sync and VRR

Quoted 58 lines
Quoted 57 lines · Michel Dänzer
Quoted 48 lines · Christian König

Am 09.04.2018 um 23:45 schrieb Manasi Navare:

Quoted 31 lines

Thanks for initiating the discussion. Find my comments below:

Quoted 19 lines · Harry Wentland
Quoted 18 lines · Harry Wentland

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
  * bool    variable_refresh_compatible
  * int    target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
  * drive monitor at timing's normal refresh rate

variable_refresh_compatible = 1, target_frame_duration_ns = 0
  * send new frame to monitor as soon as it's available, if within
min/max of monitor's reported capabilities

Quoted 7 lines
Quoted 7 lines

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
  * send new frame to monitor with the specified
target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

What I would like is two sets of properties on a CRTC or preferably on a connector:

Quoted 10 lines

KMD properties that UMD can query:
* vrr_capable -  This will be an immutable property for exposing hardware's capability of supporting VRR. This will be set by the kernel after
reading the EDID mode information and monitor range capabilities.
* vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max refresh rates supported.
These properties are optional and will be created and attached to the DP/eDP connector when the connector
is getting intialized.

Mhm, aren't those properties actually per mode and not per CRTC/connector?

Quoted 4 lines

Properties that you mentioned above that the UMD can set before kernel can enable VRR functionality
*bool vrr_enable or vrr_compatible
target_frame_duration_ns

Quoted 6 lines

Yeah, that certainly makes sense. But target_frame_duration_ns is a bad name/semantics.

We should use an absolute timestamp where the frame should be presented, otherwise you could run into a bunch of trouble with IOCTL restarts or missed blanks.

Also, a fixed target frame duration isn't suitable even for video playback, due to drift between the video and audio clocks.

Why? Even if they drift, you know you want to show your 24Hz video frame for 41.6666ms and adaptive sync can ensure that with reasonable accuracy. All we're doing is eliminating the need for frame rate converters from the application and offloading that to hardware.

Quoted 4 lines
Quoted 4 lines

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Quoted 8 lines
Quoted 5 lines

I'm not sure if the driver can ever give a guarantee of the exact time a flip occurs. What we have control over with our HW is frame duration.

Are Croteam devs trying to predict render times? I'm not sure how that would work. We've had bad experience in the past with games that try to do framepacing as that's usually not accurate and tends to lead to more problems than benefits.

For gaming, it doesn't make sense nor is it feasible to know how exactly how long a render will take with microsecond precision, very coarse guesses at best. The point of adaptive sync is that it works *transparently* for the majority of cases, within the capability of the HW and driver. We don't want to have every game re-write their engine to support this, but we do want the majority to "just work".

The only exception is the video case where an application may want to request a fixed frame duration aligned to the video content. This requires an explicit interface for the video app, and our proposal is to keep it simple: app knows how long a frame should be presented for, and we try to honour that.

Well I strongly disagree on that.

See VDPAU for example: https://http.download.nvidia.com/XFree86/vdpau/doxygen/html/group___vdp_presentation_queue.html#ga5bd61ca8ef5d1bc54ca6921aa57f835a

Quoted 5 lines

[in]

earliest_presentation_time The timestamp associated with the
surface. The presentation queue will not display the surface until the presentation queue's current time is at least this value.

Especially video players want an interface where they can specify when exactly a frame should show up on the display and then get the feedback when it actually was displayed.

For video games we have a similar situation where a frame is rendered for a certain world time and in the ideal case we would actually display the frame at this world time.

I mean we have the guys from Valve on this mailing list so I think we should just get the feedback from them and see what they prefer.

Regards,
Christian.

Quoted 2 lines

-Aric

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

A message that is not in the archive (sent off-list or not yet received): fc53ce53-f02c-891d-4aa5-e9a11cbe3007-5C7GfCeVMHo@public.gmane.org

#15 lore raw

From: Koenig, Christian
Sent: Tuesday, April 10, 2018 11:43

Am 10.04.2018 um 17:35 schrieb Cyr, Aric:

-----Original Message-----

From: Wentland, Harry

Sent: Tuesday, April 10, 2018 11:08

To: Michel Dänzer <[email protected]><mailto:[email protected]>; Koenig, Christian <[email protected]><mailto:[email protected]>; Manasi Navare

<[email protected]><mailto:[email protected]>

Cc Haehnle, Nicolai <[email protected]><mailto:[email protected]>; Daniel Vetter <[email protected]><mailto:[email protected]>; Daenzer, Michel

<[email protected]><mailto:[email protected]>; dri-devel <[email protected]><mailto:[email protected]>; amd-gfx mailing list <[email protected]><mailto:[email protected]>;

Deucher, Alexander <[email protected]><mailto:[email protected]>; Cyr, Aric <[email protected]><mailto:[email protected]>; Koo, Anthony <[email protected]><mailto:[email protected]>

Subject: Re: RFC for a render API to support adaptive sync and VRR

On 2018-04-10 03:37 AM, Michel Dänzer wrote:

On 2018-04-10 08:45 AM, Christian König wrote:

Am 09.04.2018 um 23:45 schrieb Manasi Navare:

Thanks for initiating the discussion. Find my comments below:

On Mon, Apr 09, 2018 at 04:00:21PM -0400, Harry Wentland wrote:

On 2018-04-09 03:56 PM, Harry Wentland wrote:

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way

to let us know whether to vary frame timings or to target a

different frame time. These can be provided as atomic properties on

a CRTC:

* bool variable_refresh_compatible

* int target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0

* drive monitor at timing's normal refresh rate

variable_refresh_compatible = 1, target_frame_duration_ns = 0

* send new frame to monitor as soon as it's available, if within

min/max of monitor's reported capabilities

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0

* send new frame to monitor with the specified

target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible

cannot be supported the atomic check will reject the commit.

What I would like is two sets of properties on a CRTC or preferably on

a connector:

KMD properties that UMD can query:

* vrr_capable - This will be an immutable property for exposing

hardware's capability of supporting VRR. This will be set by the

kernel after

reading the EDID mode information and monitor range capabilities.

* vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max

refresh rates supported.

These properties are optional and will be created and attached to the

DP/eDP connector when the connector

is getting intialized.

Mhm, aren't those properties actually per mode and not per CRTC/connector?

Properties that you mentioned above that the UMD can set before kernel

can enable VRR functionality

*bool vrr_enable or vrr_compatible

target_frame_duration_ns

Yeah, that certainly makes sense. But target_frame_duration_ns is a bad

name/semantics.

We should use an absolute timestamp where the frame should be presented,

otherwise you could run into a bunch of trouble with IOCTL restarts or

missed blanks.

Also, a fixed target frame duration isn't suitable even for video

playback, due to drift between the video and audio clocks.

Why? Even if they drift, you know you want to show your 24Hz video frame for 41.6666ms and adaptive sync can ensure that with reasonable accuracy.

All we're doing is eliminating the need for frame rate converters from the application and offloading that to hardware.

Time-based presentation seems to be the right approach for preventing

micro-stutter in games as well, Croteam developers have been researching

this.

I'm not sure if the driver can ever give a guarantee of the exact time a flip occurs. What we have control over with our HW is frame

duration.

Are Croteam devs trying to predict render times? I'm not sure how that would work. We've had bad experience in the past with

games that try to do framepacing as that's usually not accurate and tends to lead to more problems than benefits.

For gaming, it doesn't make sense nor is it feasible to know how exactly how long a render will take with microsecond precision, very coarse guesses at best. The point of adaptive sync is that it works *transparently* for the majority of cases, within the capability of the HW and driver. We don't want to have every game re-write their engine to support this, but we do want the majority to "just work".

The only exception is the video case where an application may want to request a fixed frame duration aligned to the video content. This requires an explicit interface for the video app, and our proposal is to keep it simple: app knows how long a frame should be presented for, and we try to honour that.

Well I strongly disagree on that.

See VDPAU for example: https://http.download.nvidia.com/XFree86/vdpau/doxygen/html/group___vdp_presentation_queue.html#ga5bd61ca8ef5d1bc54ca6921aa57f835a

[in]

earliest_presentation_time

The timestamp associated with the surface. The presentation queue will not display the surface until the presentation queue's current time is at least this value.

Especially video players want an interface where they can specify when exactly a frame should show up on the display and then get the feedback when it actually was displayed.

That presentation time doesn’t need to come to kernel as such and actually is fine as-is completely decoupled from adaptive sync. As long as the video player provides the new target_frame_duration_ns on the flip, then the driver/HW will target the correct refresh rate to match the source content. This simply means that more often than not the video presents will align very close to the monitor’s refresh rate, resulting in a smooth video experience. For example, if you have 24Hz content, and an adaptive sync monitor with a range of 40-60Hz, once the target_frame_duration_ns is provided, driver can configure the monitor to a fixed refresh rate of 48Hz causing all video presents to be frame-doubled in hardware without further application intervention.

For video games we have a similar situation where a frame is rendered for a certain world time and in the ideal case we would actually display the frame at this world time.

That seems like it would be a poorly written game that flips like that, unless they are explicitly trying to throttle the framerate for some reason. When a game presents a completed frame, they’d like that to happen as soon as possible. This is why non-VSYNC modes of flipping exist and many games leverage this. Adaptive sync gives you the lower latency of immediate flips without the tearing imposed by using non-VSYNC flipping.

I mean we have the guys from Valve on this mailing list so I think we should just get the feedback from them and see what they prefer.

We have thousands of Steam games on other OSes that work great already, but we’d certainly be interested in any additional feedback. My guess is they prefer to “do nothing” and let driver/HW manage it, otherwise you exempt all existing games from supporting adaptive sync without a rewrite or update.

Regards,
Christian.

-Aric

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): MWHPR12MB1837488A047C03E955282A4182BE0-Gy0DoCVfaSXe1Hf+mWOIywdYzm3356FpvxpqHgZTriW3zl9H0oFU5g@public.gmane.org

#17 lore raw
Quoted 1 line · Aric

That presentation time doesn’t need to come to kernel as such and actually is fine as-is completely decoupled from adaptive sync.  As long as the video player provides the new target_frame_duration_ns on the flip, then the driver/HW will target the correct refresh rate to match the source content.  This simply means that more often than not the video presents will  align very close to the monitor’s refresh rate, resulting in a smooth video experience.  For example, if you have 24Hz content, and an adaptive sync monitor with a range of 40-60Hz, once the target_frame_duration_ns is provided, driver can configure the monitor to a fixed refresh rate of 48Hz causing all video presents to be frame-doubled in hardware without further application intervention.

What about multi-monitor displays, where you want to play an animation that spans multiple monitors. You really want all monitors to flip at the same time.

I understand where you're coming from, but the perspective of refusing a target presentation time is a rather selfish one of "we're the display, we're the most important, everybody else has to adjust to us" (e.g. to get perfect sync between video and audio). I admit I'm phrasing it in a bit of an extreme way, but perhaps this phrasing helps to see why that's just not a very good attitude to have.

All devices (whether video or audio or whatever) should be able to receive a target presentation time.

If the application can make your life a bit easier by providing the targetted refresh rate as additional *hint-only* parameter (like in your 24 Hz --> 48 Hz doubling example), then maybe we should indeed consider that.

Cheers,
Nicolai

Quoted 9 lines

For video games we have a similar situation where a frame is rendered for a certain world time and in the ideal case we would actually display the frame at this world time.

That seems like it would be a poorly written game that flips like that, unless they are explicitly trying to throttle the framerate for some reason.  When a game presents a completed frame, they’d like that to happen as soon as possible.  This is why non-VSYNC modes of flipping exist and many games leverage this.  Adaptive sync gives you the lower latency of immediate flips without the tearing imposed by using non-VSYNC flipping.

I mean we have the guys from Valve on this mailing list so I think we should just get the feedback from them and see what they prefer.

Quoted 9 lines

We have thousands of Steam games on other OSes that work great already, but we’d certainly be interested in any additional feedback.  My guess is they prefer to “do nothing” and let driver/HW manage it, otherwise you exempt all existing games from supporting adaptive sync without a rewrite or update.

Regards,
Christian.

-Aric

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 54baa87e-33a9-bd69-a309-d61176eb4e8d-5C7GfCeVMHo@public.gmane.org

#19 lore raw
Quoted 5 lines · Nicolai Hähnle
Quoted 1 line · Aric

That presentation time doesn’t need to come to kernel as such and actually is fine as-is completely decoupled from adaptive sync.  As long as the video player provides the new target_frame_duration_ns on the flip, then the driver/HW will target the correct refresh rate to match the source content.  This simply means that more often than not the video presents will  align very close to the monitor’s refresh rate, resulting in a smooth video experience.  For example, if you have 24Hz content, and an adaptive sync monitor with a range of 40-60Hz, once the target_frame_duration_ns is provided, driver can configure the monitor to a fixed refresh rate of 48Hz causing all video presents to be frame-doubled in hardware without further application intervention.

What about multi-monitor displays, where you want to play an animation that spans multiple monitors. You really want all monitors to flip at the same time.

Syncing two monitors is what we currently do with our timing sync feature where we drive two monitors from the same clock source if they use the same timing. That, along with VSync, guarantees all monitors flip at the same time. I'm not sure if it works with adaptive sync.

Are you suggesting to use adaptive sync to do an in-SW sync of multiple displays?

Quoted 2 lines

I understand where you're coming from, but the perspective of refusing a target presentation time is a rather selfish one of "we're the display, we're the most important, everybody else has to adjust to us" (e.g. to get perfect sync between video and audio). I admit I'm phrasing it in a bit of an extreme way, but perhaps this phrasing helps to see why that's just not a very good attitude to have.

I really dislike arguing on an emotional basis and would rather not use words such as "selfish" in this discussion. I believe all of us want to come to the best possible solution based on technical merit.

Quoted 2 lines

All devices (whether video or audio or whatever) should be able to receive a target presentation time.

I'm not sure I understand the full extent of the problem as I'm not really familiar with how this is currently done, but isn't the problem the same without variable refresh rates (or targeted refresh rates)? A Video API would still have to somehow synchronize audio and video to 60Hz on most monitors today. What would change if we gave user mode the ability to suggest we flip at video frame rates (24/48Hz)?

Harry

Quoted 25 lines

If the application can make your life a bit easier by providing the targetted refresh rate as additional *hint-only* parameter (like in your 24 Hz --> 48 Hz doubling example), then maybe we should indeed consider that.

Cheers,
Nicolai

Quoted 18 lines

For video games we have a similar situation where a frame is rendered for a certain world time and in the ideal case we would actually display the frame at this world time.

That seems like it would be a poorly written game that flips like that, unless they are explicitly trying to throttle the framerate for some reason.  When a game presents a completed frame, they’d like that to happen as soon as possible.  This is why non-VSYNC modes of flipping exist and many games leverage this.  Adaptive sync gives you the lower latency of immediate flips without the tearing imposed by using non-VSYNC flipping.

I mean we have the guys from Valve on this mailing list so I think we should just get the feedback from them and see what they prefer.

We have thousands of Steam games on other OSes that work great already, but we’d certainly be interested in any additional feedback.  My guess is they prefer to “do nothing” and let driver/HW manage it, otherwise you exempt all existing games from supporting adaptive sync without a rewrite or update.

Regards,
Christian.

    -Aric

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): [email protected]

#21 lore raw
Quoted 15 lines

-----Original Message-----
From: Michel Dänzer [mailto:[email protected]]
Sent: Tuesday, April 10, 2018 13:06

Quoted 10 lines · Aric

From: Koenig, Christian Sent: Tuesday, April 10, 2018 11:43

Quoted 3 lines

For video games we have a similar situation where a frame is rendered for a certain world time and in the ideal case we would actually display the frame at this world time.

That seems like it would be a poorly written game that flips like that, unless they are explicitly trying to throttle the framerate for some reason. When a game presents a completed frame, they’d like that to happen as soon as possible.

Quoted 4 lines

What you're describing is what most games have been doing traditionally. Croteam's research shows that this results in micro-stuttering, because frames may be presented too early. To avoid that, they want to explicitly time each presentation as described by Christian.

Yes, I agree completely. However that's only truly relevant for fixed refreshed rate displays.
This is the primary reason for having Adaptive Sync.
There is no perfect way to solve this without Adaptive Sync, but yes they can come up with better algorithms to improve fixed refresh rate displays.

Quoted 3 lines

Maybe we should try getting the Croteam guys researching this involved directly here.

I'd be interested in any research they could share, for sure.
We also have years of experience and research here, but not distilled into any readily available format.

Quoted 5 lines

--
Earthling Michel Dänzer | http://www.amd.com
Libre software enthusiast | Mesa and X developer

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): 65006f34-39b9-5aca-e0c0-77a591b69c3a-otUistvHUpPR7s880joybQ@public.gmane.org

#24 lore raw
Quoted 25 lines

-----Original Message-----
From: Michel Dänzer [mailto:[email protected]]
Sent: Tuesday, April 10, 2018 13:16

Quoted 20 lines · Aric
Quoted 19 lines

-----Original Message-----
From: Michel Dänzer [mailto:[email protected]]
Sent: Tuesday, April 10, 2018 13:06

Quoted 10 lines · Aric

From: Koenig, Christian Sent: Tuesday, April 10, 2018 11:43

Quoted 3 lines

For video games we have a similar situation where a frame is rendered for a certain world time and in the ideal case we would actually display the frame at this world time.

That seems like it would be a poorly written game that flips like that, unless they are explicitly trying to throttle the framerate for some reason. When a game presents a completed frame, they’d like that to happen as soon as possible.

What you're describing is what most games have been doing traditionally. Croteam's research shows that this results in micro-stuttering, because frames may be presented too early. To avoid that, they want to explicitly time each presentation as described by Christian.

Quoted 5 lines
Quoted 2 lines

Yes, I agree completely. However that's only truly relevant for fixed refreshed rate displays.

No, it also affects variable refresh; possibly even more in some cases, because the presentation time is less predictable.

Yes, and that's why you don't want to do it when you have variable refresh. The hardware in the monitor and GPU will do it for you, so why bother? The input to their algorithms will be noisy causing worst estimations. If you just present as fast as you can, it'll just work (within reason).
The majority of gamers want maximum FPS for their games, and there's quite frequently outrage at a particular game when they are limited to something lower that what their monitor could otherwise support (i.e. I don't want my game limited to 30Hz if I have a shiny 144Hz gaming display I paid good money for). Of course, there's always exceptions... but in our experience those are few and far between.

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): [email protected]

#26 lore raw
Quoted 34 lines

From: Haehnle, Nicolai
Sent: Tuesday, April 10, 2018 13:48

Quoted 31 lines · Aric
Quoted 30 lines

-----Original Message-----
From: Michel Dänzer [mailto:[email protected]]
Sent: Tuesday, April 10, 2018 13:16

Quoted 15 lines · Aric
Quoted 15 lines

-----Original Message-----
From: Michel Dänzer [mailto:[email protected]]
Sent: Tuesday, April 10, 2018 13:06

Quoted 10 lines · Aric

From: Koenig, Christian Sent: Tuesday, April 10, 2018 11:43

Quoted 3 lines

For video games we have a similar situation where a frame is rendered for a certain world time and in the ideal case we would actually display the frame at this world time.

That seems like it would be a poorly written game that flips like that, unless they are explicitly trying to throttle the framerate for some reason. When a game presents a completed frame, they’d like that to happen as soon as possible.

Quoted 7 lines
Quoted 4 lines

What you're describing is what most games have been doing traditionally. Croteam's research shows that this results in micro-stuttering, because frames may be presented too early. To avoid that, they want to explicitly time each presentation as described by Christian.

Yes, I agree completely. However that's only truly relevant for fixed refreshed rate displays.

No, it also affects variable refresh; possibly even more in some cases, because the presentation time is less predictable.

Quoted 6 lines
Quoted 1 line

Yes, and that's why you don't want to do it when you have variable refresh. The hardware in the monitor and GPU will do it for you,

so why bother?

I think Michel's point is that the monitor and GPU hardware *cannot* really do this, because there's synchronization with audio to take into account, which the GPU or monitor don't know about.

How does it work fine today given that all kernel seems to know is 'current' or 'current+1' vsyncs. Presumably the applications somehow schedule all this just fine.
If this works without variable refresh for 60Hz, will it not work for a fixed-rate "48Hz" monitor (assuming a 24Hz video)?

Quoted 2 lines

Also, as I wrote separately, there's the case of synchronizing multiple monitors.

For multimonitor to work with VRR, they'll have to be timing and flip synchronized.
This is impossible for an application to manage, it needs driver/HW control or you end up with one display flipping before the other and it looks terrible. And definitely forget about multiGPU without professional workstation-type support needed to sync the displays across adapters.

Quoted 7 lines
Quoted 1 line

The input to their algorithms will be noisy causing worst estimations. If you just present as fast as you can, it'll just work (within

reason).

Quoted 1 line

The majority of gamers want maximum FPS for their games, and there's quite frequently outrage at a particular game when they are

limited to something lower that what their monitor could otherwise support (i.e. I don't want my game limited to 30Hz if I have a shiny
144Hz gaming display I paid good money for). Of course, there's always exceptions... but in our experience those are few and far
between.

Quoted 6 lines

I agree that games most likely shouldn't try to be smart. I'm curious about the Croteam findings, but even if they did a really clever thing that works better than just telling the display driver "display ASAP please", chances are that *most* developers won't do that. And they'll most likely get it wrong, so our guidance should really be "games should ask for ASAP presentation, and nothing else".

Right, I think this is the 'easy' case and is covered in Harry's initial proposal when target_frame_duration_ns = 0.

Quoted 5 lines

However, there *are* legitimate use cases for requesting a specific presentation time, and there *is* precedent of APIs that expose such features.

Are there any real problems with exposing an absolute target present time?

Realistically, how far into the future are you requesting a presentation time? Won't it almost always be something like current_time+1000/video_frame_rate? If so, why not just tell the driver to set 1000/video_frame_rate and have the GPU/monitor create nicely spaced VSYNCs for you that match the source content?

In fact, you probably wouldn't even need to change your video player at all, other than having it pass the target_frame_duration_ns. You could consider this a 'hint' as you suggested, since it's cannot be guaranteed in cases your driver or HW doesn't support variable refresh. If the target_frame_duration_ns hint is supported/applied, then the video app should have nothing extra to do that it wouldn't already do for any arbitrary fixed-refresh rate display. If not supported (say the drm_atomic_check fails with -EINVAL or something), the video app falls can stop requesting a fixed target_frame_duration_ns.

A fundamental problem I have with a target present time though is how to accommodate present times that are larger than one VSYNC time? If my monitor has a 40Hz-60Hz variable refresh, it's easy to translate "my content is 24Hz, repeat this next frame an integer multiple number of times so that it lands within the monitor range". Driver fixes display to an even 48Hz and everything good (no worse than a 30Hz clip on a traditional 60Hz display anyways). This frame-doubling is all hardware based and doesn't require any polling.

Now if you change that to "show my content in at least X nanoseconds" it can work on all displays, but the intent of the app is gone and driver/GPU/display cannot optimize. For example, the HDMI VRR spec defines a "CinemaVRR" mode where target refresh rate error is accounted for based on 0.1% deviation from requested and the v_total lines are incremented/decremented to compensate. If we don't know the target rate, we will not be able to comply to this industry standard specification.

Also, how would you manage an absolute target present time in kernel? I guess app and driver need to use a common system clock or tick count, but when would you know to 'wake up' and execute the flip? If you wait for VSYNC then you'll always timeout out on v_total_max (i.e. minimum refresh rate), check your time and see "yup, need to present now" and then flip. Now your monitor just jumped from lowest refresh rate to something else which can cause other problems. If you use some timer, then you're burning needless power polling some counter and still wouldn't have the same accuracy you could achieve with a fixed duration.

Regards,
Aric
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#27 lore raw
Quoted 23 lines · Aric
Quoted 23 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 8 lines
Quoted 3 lines

For video games we have a similar situation where a frame is rendered for a certain world time and in the ideal case we would actually display the frame at this world time.

That seems like it would be a poorly written game that flips like that, unless they are explicitly trying to throttle the framerate for some reason. When a game presents a completed frame, they’d like that to happen as soon as possible.

What you're describing is what most games have been doing traditionally. Croteam's research shows that this results in micro-stuttering, because frames may be presented too early. To avoid that, they want to explicitly time each presentation as described by Christian.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Yes, I agree completely. However that's only truly relevant for fixed refreshed rate displays.

No, it also affects variable refresh; possibly even more in some cases, because the presentation time is less predictable.

Yes, and that's why you don't want to do it when you have variable refresh. The hardware in the monitor and GPU will do it for you,

so why bother?

Quoted 7 lines
Quoted 3 lines

I think Michel's point is that the monitor and GPU hardware *cannot* really do this, because there's synchronization with audio to take into account, which the GPU or monitor don't know about.

How does it work fine today given that all kernel seems to know is 'current' or 'current+1' vsyncs. Presumably the applications somehow schedule all this just fine.
If this works without variable refresh for 60Hz, will it not work for a fixed-rate "48Hz" monitor (assuming a 24Hz video)?

You're right. I guess a better way to state the point is that it *doesn't* really work today with fixed refresh, but if we're going to introduce a new API, then why not do so in a way that can fix these additional problems as well?

Quoted 6 lines
Quoted 2 lines

Also, as I wrote separately, there's the case of synchronizing multiple monitors.

For multimonitor to work with VRR, they'll have to be timing and flip synchronized.
This is impossible for an application to manage, it needs driver/HW control or you end up with one display flipping before the other and it looks terrible. And definitely forget about multiGPU without professional workstation-type support needed to sync the displays across adapters.

I'm not a display expert, but I find it hard to believe that it's that difficult. Perhaps you can help us understand?

Say you have a multi-GPU system, and each GPU has multiple displays attached, and a single application is driving them all. The application queues flips for all displays with the same target_present_time_ns attribute. Starting at some time T, the application simply asks for the same present time T + i * 16666667 (or whatever) for frame i from all displays.

Of course it's to be expected that some (or all) of the displays will not be able to hit the target time on the first bunch of flips due to hardware limitations, but as long as the range of supported frame times is wide enough, I'd expect all of them to drift towards presenting at the correct time eventually, even across multiple GPUs, with this simple scheme.

Why would that not work to sync up all displays almost perfectly?

[snip]

Quoted 10 lines
Quoted 1 line

Are there any real problems with exposing an absolute target present time?

Realistically, how far into the future are you requesting a presentation time? Won't it almost always be something like current_time+1000/video_frame_rate? If so, why not just tell the driver to set 1000/video_frame_rate and have the GPU/monitor create nicely spaced VSYNCs for you that match the source content?

In fact, you probably wouldn't even need to change your video player at all, other than having it pass the target_frame_duration_ns. You could consider this a 'hint' as you suggested, since it's cannot be guaranteed in cases your driver or HW doesn't support variable refresh. If the target_frame_duration_ns hint is supported/applied, then the video app should have nothing extra to do that it wouldn't already do for any arbitrary fixed-refresh rate display. If not supported (say the drm_atomic_check fails with -EINVAL or something), the video app falls can stop requesting a fixed target_frame_duration_ns.

A fundamental problem I have with a target present time though is how to accommodate present times that are larger than one VSYNC time? If my monitor has a 40Hz-60Hz variable refresh, it's easy to translate "my content is 24Hz, repeat this next frame an integer multiple number of times so that it lands within the monitor range". Driver fixes display to an even 48Hz and everything good (no worse than a 30Hz clip on a traditional 60Hz display anyways). This frame-doubling is all hardware based and doesn't require any polling.

Now if you change that to "show my content in at least X nanoseconds" it can work on all displays, but the intent of the app is gone and driver/GPU/display cannot optimize. For example, the HDMI VRR spec defines a "CinemaVRR" mode where target refresh rate error is accounted for based on 0.1% deviation from requested and the v_total lines are incremented/decremented to compensate. If we don't know the target rate, we will not be able to comply to this industry standard specification.

Okay, that's interesting. Does this mean that the display driver still programs a refresh rate to some hardware register?

What if you want to initiate some CPU-controlled drift, i.e. you know you're targeting 2*24Hz, but you'd like to shift all flip times to be X ms later? Can you program hardware for that, and how does it work? Do have you twiddle the refresh rate, or can the hardware do it natively?

How about what I wrote in an earlier mail of having attributes:

- target_present_time_ns
- hint_frame_time_ns (optional)

... and if a video player set both, the driver could still do the optimizations you've explained?

Quoted 1 line

Also, how would you manage an absolute target present time in kernel? I guess app and driver need to use a common system clock or tick count, but when would you know to 'wake up' and execute the flip? If you wait for VSYNC then you'll always timeout out on v_total_max (i.e. minimum refresh rate), check your time and see "yup, need to present now" and then flip. Now your monitor just jumped from lowest refresh rate to something else which can cause other problems. If you use some timer, then you're burning needless power polling some counter and still wouldn't have the same accuracy you could achieve with a fixed duration.

For the clock, we just have to specify which one to take. I believe CLOCK_MONOTONIC makes the most sense for this kind of thing.

For your other questions, I'm afraid I just don't know enough about modern display hardware to give a really good answer, but with my naive understanding I would imagine something like the following:

1. When the atomic commit happens, the driver twiddles with the display timings to get the start of scanout for the next frame as close as possible to the specified target present time (I assume this is what v_total_max is about?)

2. The kernel then schedules a timer for the time when the display hardware is finished scanning out the previous frame and starts vblank.

3. In the handler for that timer, the kernel checks whether any fence associated to the new frame's surface has signaled. If yes, it changes the display hardware's framebuffer pointer to the new frame. Otherwise, it atomically registers for the handler to be run again when the fence is signaled.

3b. The handler should check if vblank has already ended (either due to extreme CPU overload or because the fence was signaled too late).

Actually, that last point makes me wonder how the case of "present ASAP" is actually implemented in hardware.

But again, all this is just from my naive understanding of the display hardware.

Cheers,
Nicolai

Quoted 4 lines

Regards,
Aric

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

A message that is not in the archive (sent off-list or not yet received): 72c1d357-db72-b181-9421-04f99b3bbf2f-5C7GfCeVMHo@public.gmane.org

#29 lore raw
Quoted 41 lines · Nicolai Hähnle
Quoted 34 lines · Aric
Quoted 26 lines
Quoted 26 lines
Quoted 25 lines
Quoted 18 lines
Quoted 11 lines
Quoted 10 lines
Quoted 4 lines

For video games we have a similar situation where a frame is rendered
for a certain world time and in the ideal case we would actually display the frame at this world time.

That seems like it would be a poorly written game that flips like that, unless they are explicitly trying to throttle the framerate for
some reason.  When a game presents a completed frame, they’d like that to happen as soon as possible.

Quoted 6 lines

What you're describing is what most games have been doing traditionally.
Croteam's research shows that this results in micro-stuttering, because
frames may be presented too early. To avoid that, they want to explicitly time each presentation as described by Christian.

Quoted 3 lines

Yes, I agree completely.  However that's only truly relevant for fixed
refreshed rate displays.

No, it also affects variable refresh; possibly even more in some cases,
because the presentation time is less predictable.

Quoted 7 lines
Quoted 2 lines

Yes, and that's why you don't want to do it when you have variable refresh.  The hardware in the monitor and GPU will do it for you,

so why bother?

I think Michel's point is that the monitor and GPU hardware *cannot* really do this, because there's synchronization with audio to take into account, which the GPU or monitor don't know about.

Quoted 5 lines

How does it work fine today given that all kernel seems to know is 'current' or 'current+1' vsyncs.
Presumably the applications somehow schedule all this just fine. If this works without variable refresh for 60Hz, will it not work for a fixed-rate "48Hz" monitor (assuming a 24Hz video)?

Quoted 4 lines

You're right. I guess a better way to state the point is that it *doesn't* really work today with fixed refresh, but if we're going to introduce a new API, then why not do so in a way that can fix these additional problems as well?

Exactly. With a fixed frame duration, we'll still have fundamentally the same issues as we currently do without variable refresh, not making use of the full potential of variable refresh.

Quoted 6 lines

Say you have a multi-GPU system, and each GPU has multiple displays attached, and a single application is driving them all. The application queues flips for all displays with the same target_present_time_ns attribute. Starting at some time T, the application simply asks for the same present time T + i * 16666667 (or whatever) for frame i from all displays.

BTW, this is an interesting side point I've wanted to make: Any applications / use cases which really do want a fixed refresh rate can trivially do it with time-based presentation like this.

Quoted 8 lines

Of course it's to be expected that some (or all) of the displays will not be able to hit the target time on the first bunch of flips due to hardware limitations, but as long as the range of supported frame times is wide enough, I'd expect all of them to drift towards presenting at the correct time eventually, even across multiple GPUs, with this simple scheme.

Why would that not work to sync up all displays almost perfectly?

Seconded.

Quoted 7 lines

How about what I wrote in an earlier mail of having attributes:

- target_present_time_ns
- hint_frame_time_ns (optional)

... and if a video player set both, the driver could still do the optimizations you've explained?

FWIW, I don't think a property would be a good mechanism for the target presentation time.

At least with VDPAU, video players are already explicitly specifying the target presentation time, so no changes should be required at that level. Don't know about other video APIs.

The X11 Present extension protocol is also prepared for specifying the target presentation time already, the support for it just needs to be implemented.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 2005817c-09c0-aebe-bb96-9e22d6f16df5-otUistvHUpPR7s880joybQ@public.gmane.org

#31 lore raw
Quoted 10 lines

From: Michel Dänzer [mailto:[email protected]]
Sent: Wednesday, April 11, 2018 05:50

Quoted 7 lines · Nicolai Hähnle
Quoted 5 lines · Aric

How does it work fine today given that all kernel seems to know is 'current' or 'current+1' vsyncs.
Presumably the applications somehow schedule all this just fine. If this works without variable refresh for 60Hz, will it not work for a fixed-rate "48Hz" monitor (assuming a 24Hz video)?

Quoted 8 lines
Quoted 4 lines

You're right. I guess a better way to state the point is that it *doesn't* really work today with fixed refresh, but if we're going to introduce a new API, then why not do so in a way that can fix these additional problems as well?

Exactly. With a fixed frame duration, we'll still have fundamentally the same issues as we currently do without variable refresh, not making use of the full potential of variable refresh.

I see. Well then, that's makes this sort of orthogonal to the discussion.
If you say that there are no media players on Linux today that can maintain audio/video sync with a 60Hz display, then that problem is much larger than the one we're trying to solve here. By the way, I don't believe that is a true statement :)

Quoted 6 lines
Quoted 6 lines

Say you have a multi-GPU system, and each GPU has multiple displays attached, and a single application is driving them all. The application queues flips for all displays with the same target_present_time_ns attribute. Starting at some time T, the application simply asks for the same present time T + i * 16666667 (or whatever) for frame i from all displays.

[snip]

Quoted 3 lines
Quoted 1 line

Why would that not work to sync up all displays almost perfectly?

Seconded.

It doesn't work that way unfortunately. In theory, sounds great, but if you ask anyone who's worked with framelock/genlock, it is a complicated problem.
Easiest explaination is that you need to be able to atomically program registers across multiple GPUs at the same time, this is not possible without hardware assist (see AMD's S400 module for example). We have enough to discuss without this, so let's leave mGPU for another day since we can't solve it here anyways.

Quoted 2 lines
Quoted 2 lines

Okay, that's interesting. Does this mean that the display driver still programs a refresh rate to some hardware register?

Yes, driver can, in some cases, update the minimum and maximum vertical total each flip.
In fixed rated example, you would set them equal to achieve your desired refresh rate.
We don't program refresh rate, just the vertical total min/max (which translates to affecting refresh rate, given constant pixel clock and horizontal total). Thus, our refresh rate granularity is one line-time, on the order of microsec accuracy.

Quoted 11 lines
Quoted 7 lines

How about what I wrote in an earlier mail of having attributes:

- target_present_time_ns
- hint_frame_time_ns (optional)

... and if a video player set both, the driver could still do the optimizations you've explained?

FWIW, I don't think a property would be a good mechanism for the target presentation time.

Quoted 7 lines

At least with VDPAU, video players are already explicitly specifying the target presentation time, so no changes should be required at that level. Don't know about other video APIs.

The X11 Present extension protocol is also prepared for specifying the target presentation time already, the support for it just needs to be implemented.

I'm perfectly OK with presentation time-based *API*. I get it from a user mode/app perspective, and that's fine. We need that feedback and would like help defining that portions of the stack.
However, I think it doesn't make as much sense as a *DDI* because it doesn't correspond to any hardware real or logical (i.e. no one would implement it in HW this way) and the industry specs aren't defined that way.
You can have libdrm or some other usermode component translate your presentation time into a frame duration and schedule it. What's the advantage of having this in kernel besides the fact we lose the intent of the application and could prevent features and optimizations. When it gets to kernel, I think it is much more elegant for the flip structure to contain a simple duration that says "hey, show this frame on the screen for this long". Then we don't need any clocks or timers just some simple math and program the hardware.

In short,
1) We can simplify media players' lives by helping them get really, really close to their content rate, so they wouldn't need any frame rate conversion. They'll still need A/V syncing though, and variable refresh cannot solve this and thus is way out of scope of what we're proposing.

2) For gaming, don't even try to guess a frame duration, the driver/hardware will do a better job every time, just specify duration=0 and flip as fast as you can.

Regards,
Aric

P.S. Thanks for the Croteam link. Interesting, but basically nullified by variable refresh rate displays. You won't have stuttering/microstuttering/juddering/tearing if your display's refresh rate matches the render/present rate of the game. Maybe I should grab The Talos Principle to see how well it works with FreeSync display :)

--
ARIC CYR
PMTS Software Engineer | SW – Display Technologies

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): MWHPR12MB1837F8120B126FA3A4CE899982BD0-Gy0DoCVfaSXe1Hf+mWOIywdYzm3356FpvxpqHgZTriW3zl9H0oFU5g@public.gmane.org

#33 lore raw
Quoted 15 lines · Aric
Quoted 15 lines

From: Michel Dänzer [mailto:[email protected]]
Sent: Wednesday, April 11, 2018 05:50

Quoted 11 lines · Nicolai Hähnle
Quoted 5 lines · Aric

How does it work fine today given that all kernel seems to know is 'current' or 'current+1' vsyncs.
Presumably the applications somehow schedule all this just fine. If this works without variable refresh for 60Hz, will it not work for a fixed-rate "48Hz" monitor (assuming a 24Hz video)?

You're right. I guess a better way to state the point is that it *doesn't* really work today with fixed refresh, but if we're going to introduce a new API, then why not do so in a way that can fix these additional problems as well?

Quoted 7 lines
Quoted 3 lines

Exactly. With a fixed frame duration, we'll still have fundamentally the same issues as we currently do without variable refresh, not making use of the full potential of variable refresh.

I see. Well then, that's makes this sort of orthogonal to the discussion.
If you say that there are no media players on Linux today that can maintain audio/video sync with a 60Hz display, then that problem is much larger than the one we're trying to solve here. By the way, I don't believe that is a true statement :)

Indeed, that's not what we're saying:

With fixed refresh rate, audio/video sync cannot be maintained without occasional visual artifacts, due to skipped / repeated frames.

Quoted 15 lines
Quoted 15 lines
Quoted 7 lines

How about what I wrote in an earlier mail of having attributes:

- target_present_time_ns
- hint_frame_time_ns (optional)

... and if a video player set both, the driver could still do the optimizations you've explained?

FWIW, I don't think a property would be a good mechanism for the target presentation time.

At least with VDPAU, video players are already explicitly specifying the target presentation time, so no changes should be required at that level. Don't know about other video APIs.

Quoted 6 lines
Quoted 3 lines

The X11 Present extension protocol is also prepared for specifying the target presentation time already, the support for it just needs to be implemented.

I'm perfectly OK with presentation time-based *API*. I get it from a user mode/app perspective, and that's fine. We need that feedback and would like help defining that portions of the stack. However, I think it doesn't make as much sense as a *DDI* because it doesn't correspond to any hardware real or logical (i.e. no one would implement it in HW this way) and the industry specs aren't defined that way.

Which specs are you referring to? There are at least two specs (VDPAU and VK_GOOGLE_display_timing) which are defined that way.

Quoted 1 line

You can have libdrm or some other usermode component translate your presentation time into a frame duration and schedule it.

This cuts both ways.

Quoted 1 line

What's the advantage of having this in kernel besides the fact we lose the intent of the application and could prevent features and optimizations.

To me, presentation time is much clearer as intent of the application. It can express all the same things frame duration can, but not the other way around.

Quoted 3 lines

When it gets to kernel, I think it is much more elegant for the flip structure to contain a simple duration that says "hey, show this frame on the screen for this long".

A game cannot know this in advance, can it? Per the Croteam presentation, it depends on when this frame is actually presented (among other things).

Quoted 1 line

1) We can simplify media players' lives by helping them get really, really close to their content rate, so they wouldn't need any frame rate conversion.

At least with VDPAU, media players shouldn't need any changes at all, as they're already explicitly specifying the presentation times.

Quoted 1 line

They'll still need A/V syncing though, and variable refresh cannot solve this

I've been trying to explain that it can, perfectly. Can you explain why you think it can't, or ask if something isn't clear about what I've been explaining?

Quoted 1 line

P.S. Thanks for the Croteam link. Interesting, but basically nullified by variable refresh rate displays.

According to whom / what? I don't see why it wouldn't apply to variable refresh as well. Without time-based presentation, the game cannot prevent a frame from being presented too early.

There is no doubt that the artifacts of not doing this properly will be less noticeable with variable refresh, but that doesn't mean they don't exist.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#34 lore raw
In reply to Cyr, Aric
Quoted 11 lines · Aric
Quoted 7 lines

At least with VDPAU, video players are already explicitly specifying the target presentation time, so no changes should be required at that level. Don't know about other video APIs.

The X11 Present extension protocol is also prepared for specifying the target presentation time already, the support for it just needs to be implemented.

I'm perfectly OK with presentation time-based *API*. I get it from a user mode/app perspective, and that's fine. We need that feedback and would like help defining that portions of the stack.
However, I think it doesn't make as much sense as a *DDI* because it doesn't correspond to any hardware real or logical (i.e. no one would implement it in HW this way) and the industry specs aren't defined that way.
You can have libdrm or some other usermode component translate your presentation time into a frame duration and schedule it. What's the advantage of having this in kernel besides the fact we lose the intent of the application and could prevent features and optimizations. When it gets to kernel, I think it is much more elegant for the flip structure to contain a simple duration that says "hey, show this frame on the screen for this long". Then we don't need any clocks or timers just some simple math and program the hardware.

There isn't necessarily an inherent advantage to having this translation in the kernel. However, we *must* do this translation in a place that is owned by display experts (i.e., you guys), because only you guys know how to actually do that translation reliably and correctly.

Since your work is currently limited to the kernel, it makes sense to do it in the kernel.

If the translation doesn't happen in a place that you feel comfortable working on, we're setting ourselves up for a future where this hypothetical future UMD component will get this wrong, and there'll be a lot of finger-pointing between you guys and whoever writes that UMD, with likely little willingness to actually go into the respective other codebase to fix what's wrong. And that's a pretty sucky future.

Cheers,
Nicolai

P.S.: I'm also a little surprised that you seem to be saying that requesting a target present time is basically impossible (at least, that's kind of implied by your statement about mGPUs), and yet there's precedent for such APIs in both Vulkan and VDPAU.

Quoted 7 lines

In short,
1) We can simplify media players' lives by helping them get really, really close to their content rate, so they wouldn't need any frame rate conversion. They'll still need A/V syncing though, and variable refresh cannot solve this and thus is way out of scope of what we're proposing.

2) For gaming, don't even try to guess a frame duration, the driver/hardware will do a better job every time, just specify duration=0 and flip as fast as you can.

Quoted 12 lines

Regards,
Aric

P.S. Thanks for the Croteam link. Interesting, but basically nullified by variable refresh rate displays. You won't have stuttering/microstuttering/juddering/tearing if your display's refresh rate matches the render/present rate of the game. Maybe I should grab The Talos Principle to see how well it works with FreeSync display :)

--
ARIC CYR
PMTS Software Engineer | SW – Display Technologies

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

A message that is not in the archive (sent off-list or not yet received): a5b5c31f-e7c5-8ae0-1239-06bfa86b939b-5C7GfCeVMHo@public.gmane.org

#36 lore raw
Quoted 25 lines · Nicolai Hähnle
Quoted 8 lines · Aric
Quoted 7 lines

At least with VDPAU, video players are already explicitly specifying the target presentation time, so no changes should be required at that level. Don't know about other video APIs.

The X11 Present extension protocol is also prepared for specifying the target presentation time already, the support for it just needs to be implemented.

Quoted 15 lines

I'm perfectly OK with presentation time-based *API*.  I get it from a user mode/app perspective, and that's fine.  We need that feedback and would like help defining that portions of the stack. However, I think it doesn't make as much sense as a *DDI* because it doesn't correspond to any hardware real or logical (i.e. no one would implement it in HW this way) and the industry specs aren't defined that way.
You can have libdrm or some other usermode component translate your presentation time into a frame duration and schedule it.  What's the advantage of having this in kernel besides the fact we lose the intent of the application and could prevent features and optimizations.  When it gets to kernel, I think it is much more elegant for the flip structure to contain a simple duration that says "hey, show this frame on the screen for this long".  Then we don't need any clocks or timers just some simple math and program the hardware.

Quoted 2 lines

There isn't necessarily an inherent advantage to having this translation in the kernel.

One such advantage is that it doesn't require userspace to predict the future, where at least in the Vulkan case there is no information to base the prediction on. I fail to see how that can work at all.

Quoted 4 lines

P.S.: I'm also a little surprised that you seem to be saying that requesting a target present time is basically impossible (at least, that's kind of implied by your statement about mGPUs), and yet there's precedent for such APIs in both Vulkan and VDPAU.

Keep in mind that the constraint is "present no earlier than", which can be satisfied e.g. by waiting for the target time to pass before programming the flip to the hardware.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#37 lore raw
In reply to a message not in the archive
Quoted 9 lines · Nicolai Hähnle
Quoted 8 lines · Aric
Quoted 7 lines

At least with VDPAU, video players are already explicitly specifying the target presentation time, so no changes should be required at that level. Don't know about other video APIs.

The X11 Present extension protocol is also prepared for specifying the target presentation time already, the support for it just needs to be implemented.

Quoted 7 lines
Quoted 3 lines

I'm perfectly OK with presentation time-based *API*.  I get it from a user mode/app perspective, and that's fine.  We need that feedback and would like help defining that portions of the stack.
However, I think it doesn't make as much sense as a *DDI* because it doesn't correspond to any hardware real or logical (i.e. no one would implement it in HW this way) and the industry specs aren't defined that way.
You can have libdrm or some other usermode component translate your presentation time into a frame duration and schedule it.  What's the advantage of having this in kernel besides the fact we lose the intent of the application and could prevent features and optimizations.  When it gets to kernel, I think it is much more elegant for the flip structure to contain a simple duration that says "hey, show this frame on the screen for this long".  Then we don't need any clocks or timers just some simple math and program the hardware.

There isn't necessarily an inherent advantage to having this translation in the kernel. However, we *must* do this translation in a place that is owned by display experts (i.e., you guys), because only you guys know how to actually do that translation reliably and correctly.

Since your work is currently limited to the kernel, it makes sense to do it in the kernel.

We're actively trying to change this. I want us (the display team) to eventually own anything display related across the stack, or at least closely work with the owners of those components.

Quoted 3 lines

If the translation doesn't happen in a place that you feel comfortable working on, we're setting ourselves up for a future where this hypothetical future UMD component will get this wrong, and there'll be a lot of finger-pointing between you guys and whoever writes that UMD, with likely little willingness to actually go into the respective other codebase to fix what's wrong. And that's a pretty sucky future.

If finger-pointing happened I'd like to apologize. Again, this is something we actively try to change.

Ultimately I'm looking for a solution that works for everyone and is not owned on a SW component basis, but rather expertise basis.

Harry

Quoted 26 lines

Cheers,
Nicolai

P.S.: I'm also a little surprised that you seem to be saying that requesting a target present time is basically impossible (at least, that's kind of implied by your statement about mGPUs), and yet there's precedent for such APIs in both Vulkan and VDPAU.

Quoted 19 lines

In short,
  1) We can simplify media players' lives by helping them get really, really close to their content rate, so they wouldn't need any frame rate conversion. They'll still need A/V syncing though, and variable refresh cannot solve this and thus is way out of scope of what we're proposing.

  2) For gaming, don't even try to guess a frame duration, the driver/hardware will do a better job every time, just specify duration=0 and flip as fast as you can.

Regards,
   Aric

P.S. Thanks for the Croteam link.  Interesting, but basically nullified by variable refresh rate displays.  You won't have stuttering/microstuttering/juddering/tearing if your display's refresh rate matches the render/present rate of the game.  Maybe I should grab The Talos Principle to see how well it works with FreeSync display :)

Signature · 2 lines
ARIC CYR
PMTS Software Engineer | SW – Display Technologies

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#38 lore raw
In reply to Cyr, Aric
Quoted 20 lines · Aric
Quoted 20 lines

From: Michel Dänzer [mailto:[email protected]]

Quoted 18 lines · Aric
Quoted 17 lines

From: Michel Dänzer [mailto:[email protected]]

Quoted 10 lines · Aric

From: Koenig, Christian Sent: Tuesday, April 10, 2018 11:43

Quoted 3 lines

For video games we have a similar situation where a frame is rendered for a certain world time and in the ideal case we would actually display the frame at this world time.

That seems like it would be a poorly written game that flips like that, unless they are explicitly trying to throttle the framerate for some reason. When a game presents a completed frame, they’d like that to happen as soon as possible.

What you're describing is what most games have been doing traditionally. Croteam's research shows that this results in micro-stuttering, because frames may be presented too early. To avoid that, they want to explicitly time each presentation as described by Christian.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Yes, I agree completely. However that's only truly relevant for fixed refreshed rate displays.

No, it also affects variable refresh; possibly even more in some cases, because the presentation time is less predictable.

Yes, and that's why you don't want to do it when you have variable refresh. The hardware in the monitor and GPU will do it for you, so why bother? The input to their algorithms will be noisy causing worst estimations. If you just present as fast as you can, it'll just work (within reason).

If a frame is presented earlier than the time corresponding to the state of the world as displayed in the frame, it results in stutter, just as when it's presented too late.

Quoted 1 line

The majority of gamers want maximum FPS for their games, and there's quite frequently outrage at a particular game when they are limited to something lower that what their monitor could otherwise support (i.e. I don't want my game limited to 30Hz if I have a shiny 144Hz gaming display I paid good money for).

That doesn't (have to) happen.

See
https://www.gdcvault.com/play/1025407/Advanced-Graphics-Techniques-Tutorial-The
for Croteam's talk about this at this year's GDC. It says the best API available so far is the Vulkan extension VK_GOOGLE_display_timing, which (among other things) allows specifying the earliest desired presentation time via VkPresentTimeGOOGLE::desiredPresentTime . (The talk also mentions that they previously experimented with VDPAU, because it allows specifying the target presentation time)

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#39 lore raw
In reply to Cyr, Aric

On 2018-04-10 06:26 PM, Cyr, Aric wrote:> > My guess is they prefer to “do nothing” and let driver/HW manage it,

Quoted 2 lines

otherwise you exempt all existing games from supporting adaptive sync without a rewrite or update.

Nobody is saying adaptive sync should only work with explicit target presentation times provided by the application. We're just arguing that target presentation time as a mechanism is superior to target refresh rate, both for video and game use cases. It also trivially allows emulating "as early as possible" (target presentation time = 0) and "fixed refresh rate" (target presentation time = start + i * target frame duration) behaviour, even transparently for the application.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

A message that is not in the archive (sent off-list or not yet received): MWHPR12MB1837C57016A470A5199E855282BE0-Gy0DoCVfaSXe1Hf+mWOIywdYzm3356FpvxpqHgZTriW3zl9H0oFU5g@public.gmane.org

#41 lore raw
Quoted 63 lines · Aric
Quoted 63 lines · Michel Dänzer
Quoted 62 lines · Christian König
Quoted 60 lines

Am 09.04.2018 um 23:45 schrieb Manasi Navare:

Quoted 36 lines

Thanks for initiating the discussion. Find my comments
below: On Mon, Apr 09, 2018 at 04:00:21PM -0400, Harry

Quoted 22 lines · Wentland
Quoted 21 lines · Harry Wentland

=== A DRM render API to support variable refresh rates
===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC: * bool variable_refresh_compatible * int target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0 * drive monitor at timing's normal refresh rate

variable_refresh_compatible = 1, target_frame_duration_ns = 0 * send new frame to monitor as soon as it's available, if within min/max of monitor's reported capabilities

Quoted 8 lines
Quoted 8 lines

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0 * send new frame to monitor with the specified target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

What I would like is two sets of properties on a CRTC or preferably on a connector:

Quoted 9 lines

KMD properties that UMD can query: * vrr_capable - This will be an immutable property for exposing hardware's capability of supporting VRR. This will be set by the kernel after reading the EDID mode information and monitor range capabilities. * vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max refresh rates supported. These properties are optional and will be created and attached to the DP/eDP connector when the connector is getting intialized.

Mhm, aren't those properties actually per mode and not per CRTC/connector?

Quoted 3 lines

Properties that you mentioned above that the UMD can set before kernel can enable VRR functionality *bool vrr_enable or vrr_compatible target_frame_duration_ns

Yeah, that certainly makes sense. But target_frame_duration_ns is a bad name/semantics.

We should use an absolute timestamp where the frame should be presented, otherwise you could run into a bunch of trouble with IOCTL restarts or missed blanks.

Quoted 6 lines
Quoted 2 lines
Quoted 2 lines

Also, a fixed target frame duration isn't suitable even for video playback, due to drift between the video and audio clocks.

Why? Even if they drift, you know you want to show your 24Hz video frame for 41.6666ms and adaptive sync can ensure that with reasonable accuracy.

Due to the drift, the video player has to occasionally either skip a frame or present it twice to prevent audio and video going out of sync, resulting in visual artifacts.

With time-based presentation and variable refresh rate, audio and video can stay in sync without occasional visual artifacts.

It would be a pity to create a "variable refresh rate API" which doesn't allow harnessing this strength of variable refresh rate.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#42 lore raw
In reply to a message not in the archive
Quoted 59 lines · Michel Dänzer
Quoted 51 lines · Christian König

Am 09.04.2018 um 23:45 schrieb Manasi Navare:

Quoted 32 lines

Thanks for initiating the discussion. Find my comments below:

Quoted 20 lines · Harry Wentland
Quoted 19 lines · Harry Wentland

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
* bool variable_refresh_compatible
* int target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
* drive monitor at timing's normal refresh rate

variable_refresh_compatible = 1, target_frame_duration_ns = 0
* send new frame to monitor as soon as it's available, if within
min/max of monitor's reported capabilities

Quoted 7 lines
Quoted 7 lines

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
* send new frame to monitor with the specified
target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

What I would like is two sets of properties on a CRTC or preferably on a connector:

Quoted 10 lines

KMD properties that UMD can query:
* vrr_capable - This will be an immutable property for exposing hardware's capability of supporting VRR. This will be set by the kernel after
reading the EDID mode information and monitor range capabilities.
* vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max refresh rates supported.
These properties are optional and will be created and attached to the DP/eDP connector when the connector
is getting intialized.

Mhm, aren't those properties actually per mode and not per CRTC/connector?

Quoted 4 lines

Properties that you mentioned above that the UMD can set before kernel can enable VRR functionality
*bool vrr_enable or vrr_compatible
target_frame_duration_ns

Quoted 6 lines

Yeah, that certainly makes sense. But target_frame_duration_ns is a bad name/semantics.

We should use an absolute timestamp where the frame should be presented, otherwise you could run into a bunch of trouble with IOCTL restarts or missed blanks.

Quoted 6 lines

Also, a fixed target frame duration isn't suitable even for video playback, due to drift between the video and audio clocks.

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Another case that you can handle with time-based presentation but not with refresh-based API is the use of per-scanline flips in conjunction with damage rects. For example if you know that the damage rect covers a certain Y range, you can flip when you're outside that range if the time that you were given allows it. That's even independent from VRR displays.

Stéphane

Quoted 9 lines

--
Earthling Michel Dänzer | http://www.amd.com
Libre software enthusiast | Mesa and X developer
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#43 lore raw
Quoted 67 lines · Stéphane Marchesin
Quoted 59 lines · Michel Dänzer
Quoted 51 lines · Christian König

Am 09.04.2018 um 23:45 schrieb Manasi Navare:

Quoted 32 lines

Thanks for initiating the discussion. Find my comments below:

Quoted 20 lines · Harry Wentland
Quoted 19 lines · Harry Wentland

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
* bool variable_refresh_compatible
* int target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
* drive monitor at timing's normal refresh rate

variable_refresh_compatible = 1, target_frame_duration_ns = 0
* send new frame to monitor as soon as it's available, if within
min/max of monitor's reported capabilities

Quoted 7 lines
Quoted 7 lines

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
* send new frame to monitor with the specified
target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

What I would like is two sets of properties on a CRTC or preferably on a connector:

Quoted 10 lines

KMD properties that UMD can query:
* vrr_capable - This will be an immutable property for exposing hardware's capability of supporting VRR. This will be set by the kernel after
reading the EDID mode information and monitor range capabilities.
* vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max refresh rates supported.
These properties are optional and will be created and attached to the DP/eDP connector when the connector
is getting intialized.

Mhm, aren't those properties actually per mode and not per CRTC/connector?

Quoted 4 lines

Properties that you mentioned above that the UMD can set before kernel can enable VRR functionality
*bool vrr_enable or vrr_compatible
target_frame_duration_ns

Quoted 6 lines

Yeah, that certainly makes sense. But target_frame_duration_ns is a bad name/semantics.

We should use an absolute timestamp where the frame should be presented, otherwise you could run into a bunch of trouble with IOCTL restarts or missed blanks.

Quoted 6 lines

Also, a fixed target frame duration isn't suitable even for video playback, due to drift between the video and audio clocks.

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Quoted 7 lines

Another case that you can handle with time-based presentation but not with refresh-based API is the use of per-scanline flips in conjunction with damage rects. For example if you know that the damage rect covers a certain Y range, you can flip when you're outside that range if the time that you were given allows it. That's even independent from VRR displays.

That's an interesting use-case. I don't think we have given much thought to damage rects before.

Harry

Quoted 12 lines

Stéphane

Quoted 9 lines

--
Earthling Michel Dänzer | http://www.amd.com
Libre software enthusiast | Mesa and X developer
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#44 lore raw
In reply to a message not in the archive
Quoted 3 lines · Michel Dänzer

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Both the Vulkan GOOGLE_display_timing extension and X11 Present extension offer the ability to specify the desired display time in seconds.

Similarly, I'd suggest that the min/max display refresh rate values be advertised as time between frames rather than frames per second.

I'd also encourage using a single unit for all of these values, preferably nanoseconds. Absolute times should all be referenced to CLOCK_MONOTONIC.

Signature · 1 line
-keith

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#45 lore raw
Quoted 9 lines · Keith Packard
Quoted 3 lines · Michel Dänzer

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Both the Vulkan GOOGLE_display_timing extension and X11 Present extension offer the ability to specify the desired display time in seconds.

Quoted 6 lines

Similarly, I'd suggest that the min/max display refresh rate values be advertised as time between frames rather than frames per second.

I'd also encourage using a single unit for all of these values, preferably nanoseconds. Absolute times should all be referenced to CLOCK_MONOTONIC.

+1 on everything Keith said. I got somehow dragged in khr vk discussions around preventing micro-stuttering, and consensus seems to be that timestamps for scheduling frames is the way to go, most likely absolute ones (not everything is running Linux unfortunately, so can't go outright and claim it's guaranteed to be CLOCK_MONOTONIC). -Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#46 lore raw
Quoted 6 lines · Daniel Vetter
Quoted 5 lines · Keith Packard
Quoted 3 lines · Michel Dänzer

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Quoted 6 lines
Quoted 6 lines

Both the Vulkan GOOGLE_display_timing extension and X11 Present extension offer the ability to specify the desired display time in seconds.

Similarly, I'd suggest that the min/max display refresh rate values be advertised as time between frames rather than frames per second.

So there is a global min and max refresh rate as advertised by the monitor range descriptor. That I guess can be exposed as a global range in terms of min and max time between frames as a global property of the connector.

We dont need the per mode min and max refresh rate to be exposed right?

Quoted 11 lines
Quoted 4 lines

I'd also encourage using a single unit for all of these values, preferably nanoseconds. Absolute times should all be referenced to CLOCK_MONOTONIC.

+1 on everything Keith said. I got somehow dragged in khr vk discussions around preventing micro-stuttering, and consensus seems to be that timestamps for scheduling frames is the way to go, most likely absolute ones (not everything is running Linux unfortunately, so can't go outright and claim it's guaranteed to be CLOCK_MONOTONIC). -Daniel

And yes I also got consensus from Mesa and media folks about using the absolute timestamp for scheduling the frames and then the driver will modify the vblank logic to "present no earlier than the timestamp"

Manasi

Quoted 8 lines
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

A message that is not in the archive (sent off-list or not yet received): [email protected]

#48 lore raw

Hi,

Quoted 12 lines · Manasi Navare
Quoted 10 lines · Daniel Vetter
Quoted 3 lines · Keith Packard

I'd also encourage using a single unit for all of these values, preferably nanoseconds. Absolute times should all be referenced to CLOCK_MONOTONIC.

+1 on everything Keith said. I got somehow dragged in khr vk discussions around preventing micro-stuttering, and consensus seems to be that timestamps for scheduling frames is the way to go, most likely absolute ones (not everything is running Linux unfortunately, so can't go outright and claim it's guaranteed to be CLOCK_MONOTONIC).

Quoted 3 lines

And yes I also got consensus from Mesa and media folks about using the absolute timestamp for scheduling the frames and then the driver will modify the vblank logic to "present no earlier than the timestamp"

Whilst we're all piling in, here's another AOL. We didn't yet implement this for Wayland because of the fun involved in adding a FIFO mode to a very mailbox window system, but at some point we'll have to suck it up and push it.

Cheers,
Daniel
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#49 lore raw
In reply to a message not in the archive
Quoted 11 lines · Manasi Navare
Quoted 10 lines · Daniel Vetter
Quoted 9 lines · Keith Packard
Quoted 3 lines · Michel Dänzer

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Both the Vulkan GOOGLE_display_timing extension and X11 Present extension offer the ability to specify the desired display time in seconds.

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

Similarly, I'd suggest that the min/max display refresh rate values be advertised as time between frames rather than frames per second.

So there is a global min and max refresh rate as advertised by the monitor range descriptor. That I guess can be exposed as a global range in terms of min and max time between frames as a global property of the connector.

We dont need the per mode min and max refresh rate to be exposed right?

If I understand VRR right, with CinemaVRR acceptable refresh rates might fall outside the range advertised by the monitor. Would we
1) advertise 24/1.001 as a lower bound,
2) expect media apps to use the lower bound simply for informational purposes,
3) or simply not support CinemaVRR?

(1) has the added caveat that not all reported rates would be supported.

Alternatively a bit could indicate that CinemaVRR is support, but I'm not sure if user mode would need all these details.

Harry

Quoted 27 lines
Quoted 6 lines
Quoted 5 lines

I'd also encourage using a single unit for all of these values, preferably nanoseconds. Absolute times should all be referenced to CLOCK_MONOTONIC.

Quoted 6 lines

+1 on everything Keith said. I got somehow dragged in khr vk discussions around preventing micro-stuttering, and consensus seems to be that timestamps for scheduling frames is the way to go, most likely absolute ones (not everything is running Linux unfortunately, so can't go outright and claim it's guaranteed to be CLOCK_MONOTONIC). -Daniel

And yes I also got consensus from Mesa and media folks about using the absolute timestamp for scheduling the frames and then the driver will modify the vblank logic to "present no earlier than the timestamp"

Manasi

Quoted 8 lines
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 5 lines

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#50 lore raw
Quoted 26 lines · Harry Wentland
Quoted 11 lines · Manasi Navare
Quoted 10 lines · Daniel Vetter
Quoted 9 lines · Keith Packard
Quoted 3 lines · Michel Dänzer

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Both the Vulkan GOOGLE_display_timing extension and X11 Present extension offer the ability to specify the desired display time in seconds.

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

Similarly, I'd suggest that the min/max display refresh rate values be advertised as time between frames rather than frames per second.

So there is a global min and max refresh rate as advertised by the monitor range descriptor. That I guess can be exposed as a global range in terms of min and max time between frames as a global property of the connector.

We dont need the per mode min and max refresh rate to be exposed right?

If I understand VRR right, with CinemaVRR acceptable refresh rates might fall outside the range advertised by the monitor. Would we
1) advertise 24/1.001 as a lower bound,
2) expect media apps to use the lower bound simply for informational purposes,
3) or simply not support CinemaVRR?

Quoted 5 lines

(1) has the added caveat that not all reported rates would be supported.

Alternatively a bit could indicate that CinemaVRR is support, but I'm not sure if user mode would need all these details.

Harry

Are there special CinemaVRR suported monitors? In that case we need to understand how those monitors advertise the monitor range and if they have a bit in EDID that indicate they are CinemaVRR capable as opposed to just the Adaptive Sync/VRR.
Harry, if you have one of those monitors, could you send the EDID dump for that?

Manasi

Quoted 28 lines
Quoted 28 lines
Quoted 7 lines
Quoted 6 lines

I'd also encourage using a single unit for all of these values, preferably nanoseconds. Absolute times should all be referenced to CLOCK_MONOTONIC.

Quoted 6 lines

+1 on everything Keith said. I got somehow dragged in khr vk discussions around preventing micro-stuttering, and consensus seems to be that timestamps for scheduling frames is the way to go, most likely absolute ones (not everything is running Linux unfortunately, so can't go outright and claim it's guaranteed to be CLOCK_MONOTONIC). -Daniel

And yes I also got consensus from Mesa and media folks about using the absolute timestamp for scheduling the frames and then the driver will modify the vblank logic to "present no earlier than the timestamp"

Manasi

Quoted 8 lines
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 5 lines
Quoted 5 lines

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

A message that is not in the archive (sent off-list or not yet received): [email protected]

#52 lore raw
Quoted 29 lines · Manasi Navare
Quoted 28 lines · Harry Wentland
Quoted 11 lines · Manasi Navare
Quoted 10 lines · Daniel Vetter
Quoted 9 lines · Keith Packard
Quoted 3 lines · Michel Dänzer

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Both the Vulkan GOOGLE_display_timing extension and X11 Present extension offer the ability to specify the desired display time in seconds.

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

Similarly, I'd suggest that the min/max display refresh rate values be advertised as time between frames rather than frames per second.

So there is a global min and max refresh rate as advertised by the monitor range descriptor. That I guess can be exposed as a global range in terms of min and max time between frames as a global property of the connector.

We dont need the per mode min and max refresh rate to be exposed right?

If I understand VRR right, with CinemaVRR acceptable refresh rates might fall outside the range advertised by the monitor. Would we
1) advertise 24/1.001 as a lower bound,
2) expect media apps to use the lower bound simply for informational purposes,
3) or simply not support CinemaVRR?

(1) has the added caveat that not all reported rates would be supported.

Quoted 8 lines
Quoted 3 lines

Alternatively a bit could indicate that CinemaVRR is support, but I'm not sure if user mode would need all these details.

Harry

Are there special CinemaVRR suported monitors? In that case we need to understand how those monitors advertise the monitor range and if they have a bit in EDID that indicate they are CinemaVRR capable as opposed to just the Adaptive Sync/VRR.
Harry, if you have one of those monitors, could you send the EDID dump for that?

As long as the any multiple of the 24/1.001 refresh rate is within the officially supported refresh range rate this should work out. Maybe we'll end up uploading 2x (to run at ~48Hz), maybe the kernel only uploads at 24Hz. But should all be fine.

Ofc if we have CinemaVRR screens which don't fit this, then maybe we need to figure out something ...
-Daniel

Quoted 36 lines

Manasi

Quoted 33 lines
Quoted 33 lines
Quoted 7 lines
Quoted 6 lines

I'd also encourage using a single unit for all of these values, preferably nanoseconds. Absolute times should all be referenced to CLOCK_MONOTONIC.

Quoted 6 lines

+1 on everything Keith said. I got somehow dragged in khr vk discussions around preventing micro-stuttering, and consensus seems to be that timestamps for scheduling frames is the way to go, most likely absolute ones (not everything is running Linux unfortunately, so can't go outright and claim it's guaranteed to be CLOCK_MONOTONIC). -Daniel

And yes I also got consensus from Mesa and media folks about using the absolute timestamp for scheduling the frames and then the driver will modify the vblank logic to "present no earlier than the timestamp"

Manasi

Quoted 8 lines
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#53 lore raw
Quoted 42 lines

From: Daniel Vetter [mailto:[email protected]] On Behalf Of Daniel Vetter
Sent: Tuesday, April 24, 2018 08:10

Quoted 16 lines · Manasi Navare
Quoted 15 lines · Harry Wentland
Quoted 14 lines · Manasi Navare
Quoted 12 lines · Daniel Vetter
Quoted 11 lines · Keith Packard
Quoted 3 lines · Michel Dänzer

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Both the Vulkan GOOGLE_display_timing extension and X11 Present extension offer the ability to specify the desired display time in seconds.

Similarly, I'd suggest that the min/max display refresh rate values be advertised as time between frames rather than frames per second.

Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

So there is a global min and max refresh rate as advertised by the monitor range descriptor. That I guess can be exposed as a global range in terms of min and max time between frames as a global property of the connector.

We dont need the per mode min and max refresh rate to be exposed right?

If I understand VRR right, with CinemaVRR acceptable refresh rates might fall outside the range advertised by the monitor. Would

we

Quoted 6 lines
Quoted 6 lines

1) advertise 24/1.001 as a lower bound,
2) expect media apps to use the lower bound simply for informational purposes,
3) or simply not support CinemaVRR?

(1) has the added caveat that not all reported rates would be supported.

Quoted 8 lines
Quoted 3 lines

Alternatively a bit could indicate that CinemaVRR is support, but I'm not sure if user mode would need all these details.

Harry

Are there special CinemaVRR suported monitors? In that case we need to understand how those monitors advertise the monitor range and if they have a bit in EDID that indicate they are CinemaVRR capable as opposed to just the Adaptive Sync/VRR.
Harry, if you have one of those monitors, could you send the EDID dump for that?

Quoted 4 lines

As long as the any multiple of the 24/1.001 refresh rate is within the officially supported refresh range rate this should work out. Maybe we'll end up uploading 2x (to run at ~48Hz), maybe the kernel only uploads at 24Hz. But should all be fine.

Ya, I think this makes most sense. An app can really only know when it ideally wants to present the next frame. We should let drivers figure out how best to accommodate that time, whether Adaptive Sync or not, frame doubling, etc. Various hardware will may have different capabilities whose complexity we really don't want to expose to app level.

All an app needs to know is when they want to present a frame (input to kernel), and at what time it was actually presented (feedback from kernel). Anything else is superfluous and likely overcomplicating things.

Regards,

--
ARIC CYR
PMTS Software Engineer | SW - Display Technologies

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

A message that is not in the archive (sent off-list or not yet received): 20180424120946.GW31310-dv86pmgwkMBes7Z6vYuT8azUEOm+Xw19@public.gmane.org

#55 lore raw
Quoted 39 lines · Daniel Vetter
Quoted 29 lines · Manasi Navare
Quoted 28 lines · Harry Wentland
Quoted 11 lines · Manasi Navare
Quoted 10 lines · Daniel Vetter
Quoted 9 lines · Keith Packard
Quoted 3 lines · Michel Dänzer

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Both the Vulkan GOOGLE_display_timing extension and X11 Present extension offer the ability to specify the desired display time in seconds.

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

Similarly, I'd suggest that the min/max display refresh rate values be advertised as time between frames rather than frames per second.

So there is a global min and max refresh rate as advertised by the monitor range descriptor. That I guess can be exposed as a global range in terms of min and max time between frames as a global property of the connector.

We dont need the per mode min and max refresh rate to be exposed right?

If I understand VRR right, with CinemaVRR acceptable refresh rates might fall outside the range advertised by the monitor. Would we
1) advertise 24/1.001 as a lower bound,
2) expect media apps to use the lower bound simply for informational purposes,
3) or simply not support CinemaVRR?

(1) has the added caveat that not all reported rates would be supported.

Quoted 8 lines
Quoted 3 lines

Alternatively a bit could indicate that CinemaVRR is support, but I'm not sure if user mode would need all these details.

Harry

Are there special CinemaVRR suported monitors? In that case we need to understand how those monitors advertise the monitor range and if they have a bit in EDID that indicate they are CinemaVRR capable as opposed to just the Adaptive Sync/VRR.
Harry, if you have one of those monitors, could you send the EDID dump for that?

Quoted 5 lines

As long as the any multiple of the 24/1.001 refresh rate is within the officially supported refresh range rate this should work out. Maybe we'll end up uploading 2x (to run at ~48Hz), maybe the kernel only uploads at 24Hz. But should all be fine.

Would kernel driver upload 48Hz when UMD asks for 24Hz or would UMD be expected to submit double frames?

If kernel driver supports frame doubling (like our DC driver) we would probably report half of monitor-reported min-refresh (or rather double of monitor-reported max frame time).

Harry

Quoted 41 lines

Ofc if we have CinemaVRR screens which don't fit this, then maybe we need to figure out something ...
-Daniel

Quoted 36 lines

Manasi

Quoted 33 lines
Quoted 33 lines
Quoted 7 lines
Quoted 6 lines

I'd also encourage using a single unit for all of these values, preferably nanoseconds. Absolute times should all be referenced to CLOCK_MONOTONIC.

Quoted 6 lines

+1 on everything Keith said. I got somehow dragged in khr vk discussions around preventing micro-stuttering, and consensus seems to be that timestamps for scheduling frames is the way to go, most likely absolute ones (not everything is running Linux unfortunately, so can't go outright and claim it's guaranteed to be CLOCK_MONOTONIC). -Daniel

And yes I also got consensus from Mesa and media folks about using the absolute timestamp for scheduling the frames and then the driver will modify the vblank logic to "present no earlier than the timestamp"

Manasi

Quoted 8 lines
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): b9c33eeb-1db8-f782-cfbe-411d94fecb67-5C7GfCeVMHo@public.gmane.org

#57 lore raw
Quoted 48 lines · Harry Wentland
Quoted 39 lines · Daniel Vetter
Quoted 29 lines · Manasi Navare
Quoted 28 lines · Harry Wentland
Quoted 11 lines · Manasi Navare
Quoted 10 lines · Daniel Vetter
Quoted 9 lines · Keith Packard
Quoted 3 lines · Michel Dänzer

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Both the Vulkan GOOGLE_display_timing extension and X11 Present extension offer the ability to specify the desired display time in seconds.

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

Similarly, I'd suggest that the min/max display refresh rate values be advertised as time between frames rather than frames per second.

So there is a global min and max refresh rate as advertised by the monitor range descriptor. That I guess can be exposed as a global range in terms of min and max time between frames as a global property of the connector.

We dont need the per mode min and max refresh rate to be exposed right?

If I understand VRR right, with CinemaVRR acceptable refresh rates might fall outside the range advertised by the monitor. Would we
1) advertise 24/1.001 as a lower bound,
2) expect media apps to use the lower bound simply for informational purposes,
3) or simply not support CinemaVRR?

(1) has the added caveat that not all reported rates would be supported.

Quoted 8 lines
Quoted 3 lines

Alternatively a bit could indicate that CinemaVRR is support, but I'm not sure if user mode would need all these details.

Harry

Are there special CinemaVRR suported monitors? In that case we need to understand how those monitors advertise the monitor range and if they have a bit in EDID that indicate they are CinemaVRR capable as opposed to just the Adaptive Sync/VRR.
Harry, if you have one of those monitors, could you send the EDID dump for that?

Quoted 5 lines

As long as the any multiple of the 24/1.001 refresh rate is within the officially supported refresh range rate this should work out. Maybe we'll end up uploading 2x (to run at ~48Hz), maybe the kernel only uploads at 24Hz. But should all be fine.

Quoted 3 lines

Would kernel driver upload 48Hz when UMD asks for 24Hz or would UMD be expected to submit double frames?

If kernel driver supports frame doubling (like our DC driver) we would probably report half of monitor-reported min-refresh (or rather double of monitor-reported max frame time).

Your driver (amdgpu) already supports frame doubling, except only for vblank seqno instead of timestamps. Whether VRR can get down to 24Hz or not is totally irrelevant from userspace's point of view. By default the kernel is expected to keep display the current frame for as long as userspace gives it a new one. There's no expectation that userspace provides a new buffer for every vblank (whether that's a fixed or variable refresh rate doesn't matter).
-Daniel

Quoted 44 lines

Harry

Quoted 41 lines

Ofc if we have CinemaVRR screens which don't fit this, then maybe we need to figure out something ...
-Daniel

Quoted 36 lines

Manasi

Quoted 33 lines
Quoted 33 lines
Quoted 7 lines
Quoted 6 lines

I'd also encourage using a single unit for all of these values, preferably nanoseconds. Absolute times should all be referenced to CLOCK_MONOTONIC.

Quoted 6 lines

+1 on everything Keith said. I got somehow dragged in khr vk discussions around preventing micro-stuttering, and consensus seems to be that timestamps for scheduling frames is the way to go, most likely absolute ones (not everything is running Linux unfortunately, so can't go outright and claim it's guaranteed to be CLOCK_MONOTONIC). -Daniel

And yes I also got consensus from Mesa and media folks about using the absolute timestamp for scheduling the frames and then the driver will modify the vblank logic to "present no earlier than the timestamp"

Manasi

Quoted 8 lines

--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): CAKMK7uHzLRUwZSFbMb4qDzXBODVSQVTTw6BxgmvJdES4gJmBDg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#59 lore raw

It would be really nice to have support for the automatic extension-less fullscreen game scenario. Maybe you don't have to solve everything in the first implementation...
So a friendly ping here!

Regards
//Ernst

Den tis 24 apr. 2018 kl 23:58 skrev Daniel Vetter <[email protected]>:

Quoted 110 lines · Harry Wentland
Quoted 48 lines · Daniel Vetter
Quoted 39 lines · Manasi Navare
Quoted 29 lines · Harry Wentland
Quoted 28 lines · Manasi Navare
Quoted 11 lines · Daniel Vetter
Quoted 10 lines · Keith Packard
Quoted 9 lines
Quoted 3 lines · Michel Dänzer

Time-based presentation seems to be the right approach for preventing micro-stutter in games as well, Croteam developers have been researching this.

Both the Vulkan GOOGLE_display_timing extension and X11 Present extension offer the ability to specify the desired display time in seconds.

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

Similarly, I'd suggest that the min/max display refresh rate values be advertised as time between frames rather than frames per second.

So there is a global min and max refresh rate as advertised by the monitor range descriptor. That I guess can be exposed as a global range in terms of min and max time between frames as a global property of the connector.

We dont need the per mode min and max refresh rate to be exposed right?

If I understand VRR right, with CinemaVRR acceptable refresh rates might fall outside the range advertised by the monitor. Would we
1) advertise 24/1.001 as a lower bound,
2) expect media apps to use the lower bound simply for informational purposes,
3) or simply not support CinemaVRR?

(1) has the added caveat that not all reported rates would be supported.

Quoted 8 lines
Quoted 3 lines

Alternatively a bit could indicate that CinemaVRR is support, but I'm not sure if user mode would need all these details.

Harry

Are there special CinemaVRR suported monitors? In that case we need to understand how those monitors advertise the monitor range and if they have a bit in EDID that indicate they are CinemaVRR capable as opposed to just the Adaptive Sync/VRR.
Harry, if you have one of those monitors, could you send the EDID dump for that?

Quoted 5 lines

As long as the any multiple of the 24/1.001 refresh rate is within the officially supported refresh range rate this should work out. Maybe we'll end up uploading 2x (to run at ~48Hz), maybe the kernel only uploads at 24Hz. But should all be fine.

Quoted 3 lines

Would kernel driver upload 48Hz when UMD asks for 24Hz or would UMD be expected to submit double frames?

If kernel driver supports frame doubling (like our DC driver) we would probably report half of monitor-reported min-refresh (or rather double of monitor-reported max frame time).

Your driver (amdgpu) already supports frame doubling, except only for vblank seqno instead of timestamps. Whether VRR can get down to 24Hz or not is totally irrelevant from userspace's point of view. By default the kernel is expected to keep display the current frame for as long as userspace gives it a new one. There's no expectation that userspace provides a new buffer for every vblank (whether that's a fixed or variable refresh rate doesn't matter).
-Daniel

Quoted 44 lines

Harry

Quoted 41 lines

Ofc if we have CinemaVRR screens which don't fit this, then maybe we need to figure out something ...
-Daniel

Quoted 36 lines

Manasi

Quoted 33 lines
Quoted 33 lines
Quoted 7 lines
Quoted 6 lines

I'd also encourage using a single unit for all of these values, preferably nanoseconds. Absolute times should all be referenced to CLOCK_MONOTONIC.

Quoted 6 lines

+1 on everything Keith said. I got somehow dragged in khr vk discussions around preventing micro-stuttering, and consensus seems to be that timestamps for scheduling frames is the way to go, most likely absolute ones (not everything is running Linux unfortunately, so can't go outright and claim it's guaranteed to be CLOCK_MONOTONIC). -Daniel

And yes I also got consensus from Mesa and media folks about using the absolute timestamp for scheduling the frames and then the driver will modify the vblank logic to "present no earlier than the timestamp"

Manasi

Quoted 8 lines

--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Quoted 8 lines

--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#60 lore raw
In reply to a message not in the archive

Adding Anthony and Aric who've been working on Freesync with DC on other OSes for a while.

Quoted 29 lines · Manasi Navare

Thanks for initiating the discussion. Find my comments below:

Quoted 26 lines · Harry Wentland

Adding dri-devel, which I should've included from the start.

Quoted 23 lines · Harry Wentland

=== What is adaptive sync and VRR? ===

Adaptive sync has been part of the DisplayPort spec for a while now and allows graphics adapters to drive displays with varying frame timings. VRR (variable refresh rate) is essentially the same, but defined for HDMI.

=== Why allow variable frame timings? ===

Variable render times don't align with fixed refresh rates, leading to stuttering, tearing, and/or input lag.

e.g. (rc = render completion, dr = display refresh)

rc B C D E F
dr A B C C D E F

^ ^
frame missed
repeated display
twice refresh

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines

=== Other use cases of adaptive sync ====

Beside the variable render case, adaptive sync also allows adjustment of refresh rates without a mode change. One such use case would be 24 Hz video.

One of the the advantages here when the render speed is slower than the display refresh rate, since we are stretching the vertical blanking interval the display adapters will follow "draw fast and then go idle" approach. This gives power savings when render rate is lower than the display refresh rate.

Are you talking about a use case, such as an idle desktop, where the renders are quite sporadic?

Quoted 23 lines
Quoted 14 lines
Quoted 14 lines

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
* bool variable_refresh_compatible
* int target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
* drive monitor at timing's normal refresh rate

Quoted 8 lines
Quoted 8 lines

variable_refresh_compatible = 1, target_frame_duration_ns = 0
* send new frame to monitor as soon as it's available, if within min/max of monitor's reported capabilities

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
* send new frame to monitor with the specified target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

Quoted 9 lines

What I would like is two sets of properties on a CRTC or preferably on a connector:

KMD properties that UMD can query:
* vrr_capable - This will be an immutable property for exposing hardware's capability of supporting VRR. This will be set by the kernel after reading the EDID mode information and monitor range capabilities.
* vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max refresh rates supported.
These properties are optional and will be created and attached to the DP/eDP connector when the connector
is getting intialized.

If we're talking about the properties from the EDID these might not necessarily align with a currently selected mode, which might have a refresh rate lower than the vrr_refresh_max, requiring us to cap it at that. In some scenarios we also might do low framerate compensation [1] where we do magic to allow the framerate to drop below the supported range.

I think if a vrr_refresh_max/min are exposed to UMD these should really be only for informational purposes, in which case it might make more sense to expose them through sysfs or even debugfs entries.

[1] https://www.amd.com/Documents/freesync-lfc.pdf

Quoted 7 lines

Properties that you mentioned above that the UMD can set before kernel can enable VRR functionality *bool vrr_enable or vrr_compatible
target_frame_duration_ns

The monitor only specifies the monitor range through EDID. Apart from this should we also need to scan the modes and check if there are modes that have the same pixel clock and horizontal timings but variable vertical totals?

I'm not sure about the VRR spec, but for adaptive sync we should only consider the range limits specified in the EDID and allow adaptive sync for modes within that range.

Quoted 3 lines

I have RFC patches for all the above mentioned. If we get a concensus/agreement on the above properties and method to check monitor's VRR capability, I can submit those patches atleast as RFC.

That sounds great. I wouldn't mind trying those patches and then working together to arrive at something that works for both Intel and AMD. Who knows, maybe other driver guys are interested in this as well.

Harry

Quoted 24 lines

Regards
Manasi

Quoted 21 lines
Quoted 21 lines

=== Previous discussions ===

https://lists.freedesktop.org/archives/dri-devel/2017-October/155207.html

=== Feedback and moving forward ===

I'm hoping to get some feedback on this or continue the discussion on how adaptive sync / VRR might look like in the DRM ecosystem. Once there are no major concerns or objections left we'll probably start creating some patches to sketch this out a bit better and see how it looks in practice.

Cheers,
Harry
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#61 lore raw
Quoted 34 lines · Harry Wentland

Adding Anthony and Aric who've been working on Freesync with DC on other OSes for a while.

Quoted 31 lines · Manasi Navare

Thanks for initiating the discussion. Find my comments below:

Quoted 28 lines · Harry Wentland

Adding dri-devel, which I should've included from the start.

Quoted 25 lines · Harry Wentland

=== What is adaptive sync and VRR? ===

Adaptive sync has been part of the DisplayPort spec for a while now and allows graphics adapters to drive displays with varying frame timings. VRR (variable refresh rate) is essentially the same, but defined for HDMI.

=== Why allow variable frame timings? ===

Variable render times don't align with fixed refresh rates, leading to stuttering, tearing, and/or input lag.

e.g. (rc = render completion, dr = display refresh)

rc B C D E F
dr A B C C D E F

^ ^
frame missed
repeated display
twice refresh

=== Other use cases of adaptive sync ====

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

Beside the variable render case, adaptive sync also allows adjustment of refresh rates without a mode change. One such use case would be 24 Hz video.

One of the the advantages here when the render speed is slower than the display refresh rate, since we are stretching the vertical blanking interval the display adapters will follow "draw fast and then go idle" approach. This gives power savings when render rate is lower than the display refresh rate.

Are you talking about a use case, such as an idle desktop, where the renders are quite sporadic?

I was refering to a case where the render rate is lower say 24Hz but the display rate is fixed 60Hz, that means we are pretty much displaying the same frame twice. But with Adaptive Sync, the display rate would be lowered to 24hz and the vertical blanking time will be stretched where instead of drawing the same frame twice, the system is now idle in that extra blanking time thus giving some power savings.

Quoted 33 lines
Quoted 23 lines
Quoted 14 lines
Quoted 14 lines

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
* bool variable_refresh_compatible
* int target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
* drive monitor at timing's normal refresh rate

Quoted 8 lines
Quoted 8 lines

variable_refresh_compatible = 1, target_frame_duration_ns = 0
* send new frame to monitor as soon as it's available, if within min/max of monitor's reported capabilities

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
* send new frame to monitor with the specified target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

Quoted 9 lines

What I would like is two sets of properties on a CRTC or preferably on a connector:

KMD properties that UMD can query:
* vrr_capable - This will be an immutable property for exposing hardware's capability of supporting VRR. This will be set by the kernel after reading the EDID mode information and monitor range capabilities.
* vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max refresh rates supported.
These properties are optional and will be created and attached to the DP/eDP connector when the connector
is getting intialized.

Quoted 1 line

If we're talking about the properties from the EDID these might not necessarily align with a currently selected mode, which might have a refresh rate lower than the vrr_refresh_max, requiring us to cap it at that. In some scenarios we also might do low framerate compensation [1] where we do magic to allow the framerate to drop below the supported range.

Actually the way I have coded that currently is span through all the EDID modes and for each mode with the same resolution but different refresh rates supported, create a VRR field part of drm_mode_config structure that would have refresh_max and min. So that way we store the max and min per mode as opposed to a per crtc/connector property.

Quoted 2 lines

I think if a vrr_refresh_max/min are exposed to UMD these should really be only for informational purposes, in which case it might make more sense to expose them through sysfs or even debugfs entries.

I think this range obtained from EDID monitor range descriptor should still be exposed through a property so that userspace does not request a target frame rate beyond whats supported.

Manasi

Quoted 13 lines

[1] https://www.amd.com/Documents/freesync-lfc.pdf

Quoted 7 lines

Properties that you mentioned above that the UMD can set before kernel can enable VRR functionality *bool vrr_enable or vrr_compatible
target_frame_duration_ns

The monitor only specifies the monitor range through EDID. Apart from this should we also need to scan the modes and check if there are modes that have the same pixel clock and horizontal timings but variable vertical totals?

I'm not sure about the VRR spec, but for adaptive sync we should only consider the range limits specified in the EDID and allow adaptive sync for modes within that range.

Quoted 32 lines
Quoted 3 lines

I have RFC patches for all the above mentioned. If we get a concensus/agreement on the above properties and method to check monitor's VRR capability, I can submit those patches atleast as RFC.

That sounds great. I wouldn't mind trying those patches and then working together to arrive at something that works for both Intel and AMD. Who knows, maybe other driver guys are interested in this as well.

Harry

Quoted 24 lines

Regards
Manasi

Quoted 21 lines
Quoted 21 lines

=== Previous discussions ===

https://lists.freedesktop.org/archives/dri-devel/2017-October/155207.html

=== Feedback and moving forward ===

I'm hoping to get some feedback on this or continue the discussion on how adaptive sync / VRR might look like in the DRM ecosystem. Once there are no major concerns or objections left we'll probably start creating some patches to sketch this out a bit better and see how it looks in practice.

Cheers,
Harry
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#62 lore raw
Quoted 54 lines

From: Manasi Navare [mailto:[email protected]]
Sent: Tuesday, April 10, 2018 17:37
To: Wentland, Harry <[email protected]>
Cc: amd-gfx mailing list <[email protected]>; Daniel Vetter <[email protected]>; Haehnle, Nicolai <[email protected]>; Daenzer, Michel <[email protected]>; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; dri-devel <[email protected]>; Cyr, Aric <[email protected]>; Koo, Anthony <[email protected]>
Subject: Re: RFC for a render API to support adaptive sync and VRR

Quoted 12 lines · Harry Wentland

Adding Anthony and Aric who've been working on Freesync with DC on other OSes for a while.

Quoted 9 lines · Manasi Navare

Thanks for initiating the discussion. Find my comments below:

Quoted 6 lines · Harry Wentland

Adding dri-devel, which I should've included from the start.

Quoted 3 lines · Harry Wentland

=== What is adaptive sync and VRR? ===

Adaptive sync has been part of the DisplayPort spec for a while now and allows graphics adapters to drive displays with varying

frame timings. VRR (variable refresh rate) is essentially the same, but defined for HDMI.

Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
Quoted 20 lines

=== Why allow variable frame timings? ===

Variable render times don't align with fixed refresh rates, leading to stuttering, tearing, and/or input lag.

e.g. (rc = render completion, dr = display refresh)

rc B C D E F
dr A B C C D E F

^ ^
frame missed
repeated display
twice refresh

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

=== Other use cases of adaptive sync ====

Beside the variable render case, adaptive sync also allows adjustment of refresh rates without a mode change. One such use

case would be 24 Hz video.

Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line

One of the the advantages here when the render speed is slower than the display refresh rate, since we are stretching the vertical

blanking interval

Quoted 1 line
Quoted 1 line

the display adapters will follow "draw fast and then go idle" approach. This gives power savings when render rate is lower than the

display refresh rate.

Quoted 1 line
Quoted 8 lines
Quoted 2 lines

Are you talking about a use case, such as an idle desktop, where the renders are quite sporadic?

I was refering to a case where the render rate is lower say 24Hz but the display rate is fixed 60Hz, that means we are pretty much displaying the same frame
twice. But with Adaptive Sync, the display rate would be lowered to 24hz and the vertical blanking time will be stretched where instead of drawing the
same frame twice, the system is now idle in that extra blanking time thus giving some power savings.

Hi Manasi,

Assuming the panel could go down to 24Hz, this would be possible.
If it was a game, it'd naturally do this since the refresh rate would track the render rate.

For a video where you have an adaptive sync capable player, it could request a fixed duration to achieve the same thing.
Most panels do not support as low as 24Hz however, so usually in the video case at least you'd end up with say 48Hz with the driver/HW providing automatic frame doubling.

Quoted 36 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target

a different frame time. These can be provided as atomic properties on a CRTC:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

* bool variable_refresh_compatible
* int target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
* drive monitor at timing's normal refresh rate

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

variable_refresh_compatible = 1, target_frame_duration_ns = 0
* send new frame to monitor as soon as it's available, if within min/max of monitor's reported capabilities

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
* send new frame to monitor with the specified target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the

commit.

Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line

What I would like is two sets of properties on a CRTC or preferably on a connector:

KMD properties that UMD can query:
* vrr_capable - This will be an immutable property for exposing hardware's capability of supporting VRR. This will be set by the

kernel after

Quoted 6 lines
Quoted 5 lines

reading the EDID mode information and monitor range capabilities.
* vrr_vrefresh_max, vrr_vrefresh_min - To expose the min and max refresh rates supported. These properties are optional and will be created and attached to the DP/eDP connector when the connector is getting intialized.

Quoted 7 lines
Quoted 1 line

If we're talking about the properties from the EDID these might not necessarily align with a currently selected mode, which might

have a refresh rate lower than the vrr_refresh_max, requiring us to cap it at that. In some scenarios we also might do low framerate compensation [1] where we do magic to allow the framerate to drop below the supported range.

Actually the way I have coded that currently is span through all the EDID modes and for each mode with the same resolution but different refresh rates supported, create a VRR field part of drm_mode_config structure that would have refresh_max and min. So that way we store the max and min per mode as opposed to a per crtc/connector property.

Adaptive Sync spec specifies that the refresh rate range comes from the Monitor Range Descriptor and that all modes in EDID must be supported at any of those refresh rates. I'd have to check the HDMI VRR spec, but I would assume it's similar, hence the per-connector property. Per-mode is probably overkill in this case although I think I know the scenario you are referring to but it is a small old proprietary solution that no one uses and could be emulated with a correct Monitor Range Descriptor EDID quirk.

Quoted 6 lines
Quoted 2 lines

I think if a vrr_refresh_max/min are exposed to UMD these should really be only for informational purposes, in which case it might

make more sense to expose them through sysfs or even debugfs entries.

I think this range obtained from EDID monitor range descriptor should still be exposed through a property so that userspace does not request a target frame rate beyond whats supported.

Or the drm_atomic_check could fail? The problem with user space knowing this is that it doesn't know what to do with it. If you say you want 24Hz but the monitor range is 40-120Hz, should user mode not even try to request? Should it request double? triple? I think it becomes difficult because the answer could very well depend on the driver and GPU and monitor combination. I think it has to be the driver, which knows the HW capabilities, verify what is possible or not, which is why I'd suggest an drm_atomic_check failure.

Quoted 1 line

Manasi

- Aric
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#63 lore raw
In reply to a message not in the archive
Quoted 1 line · Harry Wentland

Adding dri-devel, which I should've included from the start.

Top posting, because I'm lazy and was out sick ...

Few observations:
- Stéphane has a great point which seems to have been ignored thus far.
- Where's the VK extension for this - there must be one :-) Starting with a full implementation for that (based on radv or anv or something like that) might help.
- Imo if we do a conversion between the vk api and what we feed into the hw, then let's not do a midlayer mistake: That conversion should happen at the bottom, in the kernel driver, maybe assisted with some helpers. Not somewhere in-between, like in libdrm of all places!

Cheers, Daniel

Quoted 71 lines · Harry Wentland
Quoted 69 lines

=== What is adaptive sync and VRR? ===

Adaptive sync has been part of the DisplayPort spec for a while now and allows graphics adapters to drive displays with varying frame timings. VRR (variable refresh rate) is essentially the same, but defined for HDMI.

=== Why allow variable frame timings? ===

Variable render times don't align with fixed refresh rates, leading to stuttering, tearing, and/or input lag.

e.g. (rc = render completion, dr = display refresh)

rc B C D E F
dr A B C C D E F

^ ^
frame missed
repeated display
twice refresh

=== Other use cases of adaptive sync ====

Beside the variable render case, adaptive sync also allows adjustment of refresh rates without a mode change. One such use case would be 24 Hz video.

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
* bool variable_refresh_compatible
* int target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
* drive monitor at timing's normal refresh rate

variable_refresh_compatible = 1, target_frame_duration_ns = 0
* send new frame to monitor as soon as it's available, if within min/max of monitor's reported capabilities

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
* send new frame to monitor with the specified target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

=== Previous discussions ===

https://lists.freedesktop.org/archives/dri-devel/2017-October/155207.html

=== Feedback and moving forward ===

I'm hoping to get some feedback on this or continue the discussion on how adaptive sync / VRR might look like in the DRM ecosystem. Once there are no major concerns or objections left we'll probably start creating some patches to sketch this out a bit better and see how it looks in practice.

Cheers,
Harry
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20180413160453.GC31310-dv86pmgwkMBes7Z6vYuT8azUEOm+Xw19@public.gmane.org

#65 lore raw
Quoted 10 lines · Daniel Vetter
Quoted 1 line · Harry Wentland

Adding dri-devel, which I should've included from the start.

Top posting, because I'm lazy and was out sick ...

Few observations:
- Stéphane has a great point which seems to have been ignored thus far.
- Where's the VK extension for this - there must be one :-) Starting with a full implementation for that (based on radv or anv or something like that) might help.

Good point. The intention of this very early RFC was to understand if we're sort of thinking along the same lines as the rest of the community before going ahead and prototyping something that's not going to work well in the end. The focus here was on the kernel API. We haven't done any investigation of VK, GL, or MM APIs on this yet and were hoping for some guidance on that. That guidance seems to be that from VK and MM API perspectives frame_duration doesn't cut it and we should rather pursue an absolute presentation time.

Harry

Quoted 5 lines

- Imo if we do a conversion between the vk api and what we feed into the hw, then let's not do a midlayer mistake: That conversion should happen at the bottom, in the kernel driver, maybe assisted with some helpers. Not somewhere in-between, like in libdrm of all places!

Quoted 74 lines

Cheers, Daniel

Quoted 71 lines · Harry Wentland
Quoted 69 lines

=== What is adaptive sync and VRR? ===

Adaptive sync has been part of the DisplayPort spec for a while now and allows graphics adapters to drive displays with varying frame timings. VRR (variable refresh rate) is essentially the same, but defined for HDMI.

=== Why allow variable frame timings? ===

Variable render times don't align with fixed refresh rates, leading to stuttering, tearing, and/or input lag.

e.g. (rc = render completion, dr = display refresh)

rc B C D E F
dr A B C C D E F

^ ^
frame missed
repeated display
twice refresh

=== Other use cases of adaptive sync ====

Beside the variable render case, adaptive sync also allows adjustment of refresh rates without a mode change. One such use case would be 24 Hz video.

=== A DRM render API to support variable refresh rates ===

In order to benefit from adaptive sync and VRR userland needs a way to let us know whether to vary frame timings or to target a different frame time. These can be provided as atomic properties on a CRTC:
* bool variable_refresh_compatible
* int target_frame_duration_ns (nanosecond frame duration)

This gives us the following cases:

variable_refresh_compatible = 0, target_frame_duration_ns = 0
* drive monitor at timing's normal refresh rate

variable_refresh_compatible = 1, target_frame_duration_ns = 0
* send new frame to monitor as soon as it's available, if within min/max of monitor's reported capabilities

variable_refresh_compatible = 0/1, target_frame_duration_ns = > 0
* send new frame to monitor with the specified target_frame_duration_ns

When a target_frame_duration_ns or variable_refresh_compatible cannot be supported the atomic check will reject the commit.

=== Previous discussions ===

https://lists.freedesktop.org/archives/dri-devel/2017-October/155207.html

=== Feedback and moving forward ===

I'm hoping to get some feedback on this or continue the discussion on how adaptive sync / VRR might look like in the DRM ecosystem. Once there are no major concerns or objections left we'll probably start creating some patches to sketch this out a bit better and see how it looks in practice.

Cheers,
Harry
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx