Skip to contentSkip to conversation

amd/display: improve atomic cursor checks

PATCH In review Simon Ser 3 participants 9 messages Aug 17, 2023

Subsystems (3)
lore
Share & tools
lore

Simon Ser

author
#0 lore raw

v3 cover amd/display: improve atomic cursor checks

Changes in v3: rebase, split cursor rotation patch into 2.

Simon Ser (4):
amd/display: add cursor check for YUV underlying pipe amd/display: add cursor alpha and blend mode checks amd/display: add cursor rotation check amd/display: re-introduce cursor plane rotation prop

 .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 30 +++++++++++++++++++
 .../amd/display/amdgpu_dm/amdgpu_dm_plane.c   |  3 +-
 2 files changed, 31 insertions(+), 2 deletions(-)
Signature · 1 line
2.41.0
author
#1 lore raw

PATCH v3 1/4 amd/display: add cursor check for YUV underlying pipe

The cursor plane can't be displayed if the underlying pipe isn't using an RGB format. Reject such atomic commits so that user-space can have a fallback instead of an invisible cursor.

In theory we could support YUV if the cursor is also YUV, but at the moment only ARGB8888 cursors are supported.

Cc Alex Deucher <[email protected]>
Cc Harry Wentland <[email protected]>
Cc Nicholas Kazlauskas <[email protected]>
Cc Michel Dänzer <[email protected]>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +9 -0
index 268cb99a4c4b..f60858112d3d 100644
@@ -9844,6 +9844,15 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
 			return -EINVAL;
 		}
 
+		/* In theory we could probably support YUV cursors when the underlying
+		 * plane uses a YUV format, but there's no use-case for it yet. */
+		if (new_underlying_state->fb->format->is_yuv) {
+			drm_dbg_atomic(crtc->dev,
+				       "Cursor [PLANE:%d:%s] can't be used with YUV underlying [PLANE:%d:%s]\n",
+				       cursor->base.id, cursor->name, underlying->base.id, underlying->name);
+			return -EINVAL;
+		}
+
 		/* If this plane covers the whole CRTC, no need to check planes underneath */
 		if (new_underlying_state->crtc_x <= 0 &&
 		    new_underlying_state->crtc_y <= 0 &&
Signature · 1 line
2.41.0
author
#2 lore raw

PATCH v3 2/4 amd/display: add cursor alpha and blend mode checks

In reply to Simon Ser

We don't want a semi-transparent overlay to make the cursor plane semi-transparent as well. Same for the pixel blend mode.

Cc Alex Deucher <[email protected]>
Cc Harry Wentland <[email protected]>
Cc Nicholas Kazlauskas <[email protected]>
Cc Michel Dänzer <[email protected]>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +14 -0
index f60858112d3d..df40f74f5a15 100644
@@ -9853,6 +9853,20 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
 			return -EINVAL;
 		}
 
+		if (new_underlying_state->alpha != DRM_BLEND_ALPHA_OPAQUE) {
+			drm_dbg_atomic(crtc->dev,
+				       "Cursor [PLANE:%d:%s] can't be used with non-opaque underlying [PLANE:%d:%s]\n",
+				       cursor->base.id, cursor->name, underlying->base.id, underlying->name);
+			return -EINVAL;
+		}
+
+		if (new_underlying_state->pixel_blend_mode != DRM_MODE_BLEND_PREMULTI) {
+			drm_dbg_atomic(crtc->dev,
+				       "Cursor [PLANE:%d:%s] can't be used with non-premultiplied underlying [PLANE:%d:%s]\n",
+				       cursor->base.id, cursor->name, underlying->base.id, underlying->name);
+			return -EINVAL;
+		}
+
 		/* If this plane covers the whole CRTC, no need to check planes underneath */
 		if (new_underlying_state->crtc_x <= 0 &&
 		    new_underlying_state->crtc_y <= 0 &&
Signature · 1 line
2.41.0
author
#3 lore raw

PATCH v3 3/4 amd/display: add cursor rotation check

In reply to Simon Ser

The commit 1347385fe187 ("drm/amd/display: don't expose rotation prop for cursor plane") removed the rotation property for the cursor plane, assuming the cursor would always be displayed without any rotation. However the rotation is inherited from the underlying plane.

As a result, if the primary plane is rotated, then the cursor plane will incorrectly be rotated as well even though it doesn't have a rotation property.

To fix this, check that the underlying plane isn't rotated.

Cc Alex Deucher <[email protected]>
Cc Harry Wentland <[email protected]>
Cc Nicholas Kazlauskas <[email protected]>
Cc Michel Dänzer <[email protected]>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +7 -0
index df40f74f5a15..aba01255ba12 100644
@@ -9867,6 +9867,13 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
 			return -EINVAL;
 		}
 
+		if (new_underlying_state->rotation != DRM_MODE_ROTATE_0) {
+			drm_dbg_atomic(crtc->dev,
+				       "Cursor [PLANE:%d:%s] can't be used with rotated underlying [PLANE:%d:%s]\n",
+				       cursor->base.id, cursor->name, underlying->base.id, underlying->name);
+			return -EINVAL;
+		}
+
 		/* If this plane covers the whole CRTC, no need to check planes underneath */
 		if (new_underlying_state->crtc_x <= 0 &&
 		    new_underlying_state->crtc_y <= 0 &&
Signature · 1 line
2.41.0
maintainer
#4 lore raw

Re: [PATCH v3 3/4] amd/display: add cursor rotation check

Quoted 6 lines · Simon Ser

The commit 1347385fe187 ("drm/amd/display: don't expose rotation prop for cursor plane") removed the rotation property for the cursor plane, assuming the cursor would always be displayed without any rotation. However the rotation is inherited from the underlying plane.

Quoted 6 lines

As a result, if the primary plane is rotated, then the cursor plane will incorrectly be rotated as well even though it doesn't have a rotation property.

To fix this, check that the underlying plane isn't rotated.

HW cursor works with a rotated screen on KDE on AMD HW. This looks like it will break that.

Harry

Quoted 27 lines
Cc Alex Deucher <[email protected]>
Cc Harry Wentland <[email protected]>
Cc Nicholas Kazlauskas <[email protected]>
Cc Michel Dänzer <[email protected]>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +7 -0
index df40f74f5a15..aba01255ba12 100644
@@ -9867,6 +9867,13 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
 			return -EINVAL;
 		}
 
+		if (new_underlying_state->rotation != DRM_MODE_ROTATE_0) {
+			drm_dbg_atomic(crtc->dev,
+				       "Cursor [PLANE:%d:%s] can't be used with rotated underlying [PLANE:%d:%s]\n",
+				       cursor->base.id, cursor->name, underlying->base.id, underlying->name);
+			return -EINVAL;
+		}
+
 		/* If this plane covers the whole CRTC, no need to check planes underneath */
 		if (new_underlying_state->crtc_x <= 0 &&
 		    new_underlying_state->crtc_y <= 0 &&
maintainer
#5 lore raw

Re: [PATCH v3 3/4] amd/display: add cursor rotation check

Quoted 11 lines · Harry Wentland
Quoted 10 lines · Simon Ser

The commit 1347385fe187 ("drm/amd/display: don't expose rotation prop for cursor plane") removed the rotation property for the cursor plane, assuming the cursor would always be displayed without any rotation. However the rotation is inherited from the underlying plane.

As a result, if the primary plane is rotated, then the cursor plane will incorrectly be rotated as well even though it doesn't have a rotation property.

Quoted 6 lines
Quoted 2 lines

To fix this, check that the underlying plane isn't rotated.

HW cursor works with a rotated screen on KDE on AMD HW. This looks like it will break that.

Sorry, scrap that. KDE seems to do a SW rotation with HW cursor.

Harry

Quoted 30 lines

Harry

Quoted 27 lines
Cc Alex Deucher <[email protected]>
Cc Harry Wentland <[email protected]>
Cc Nicholas Kazlauskas <[email protected]>
Cc Michel Dänzer <[email protected]>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +7 -0
index df40f74f5a15..aba01255ba12 100644
@@ -9867,6 +9867,13 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
 			return -EINVAL;
 		}
 
+		if (new_underlying_state->rotation != DRM_MODE_ROTATE_0) {
+			drm_dbg_atomic(crtc->dev,
+				       "Cursor [PLANE:%d:%s] can't be used with rotated underlying [PLANE:%d:%s]\n",
+				       cursor->base.id, cursor->name, underlying->base.id, underlying->name);
+			return -EINVAL;
+		}
+
 		/* If this plane covers the whole CRTC, no need to check planes underneath */
 		if (new_underlying_state->crtc_x <= 0 &&
 		    new_underlying_state->crtc_y <= 0 &&
author
#6 lore raw

Re: [PATCH v3 3/4] amd/display: add cursor rotation check

Quoted 14 lines · Harry Wentland
Quoted 12 lines · Harry Wentland
Quoted 10 lines · Simon Ser

The commit 1347385fe187 ("drm/amd/display: don't expose rotation prop for cursor plane") removed the rotation property for the cursor plane, assuming the cursor would always be displayed without any rotation. However the rotation is inherited from the underlying plane.

As a result, if the primary plane is rotated, then the cursor plane will incorrectly be rotated as well even though it doesn't have a rotation property.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

To fix this, check that the underlying plane isn't rotated.

HW cursor works with a rotated screen on KDE on AMD HW. This looks like it will break that.

Sorry, scrap that. KDE seems to do a SW rotation with HW cursor.

Actually, you're right still: I've tested again with a rotate-90 primary plane and a cursor plane via the tentative tool, and the cursor FB is not rotated.

So please, disregard this patch and the next one.

#7 lore raw

Re: [PATCH v3 3/4] amd/display: add cursor rotation check

In reply to Harry Wentland
Quoted 12 lines · Harry Wentland
Quoted 11 lines · Harry Wentland
Quoted 10 lines · Simon Ser

The commit 1347385fe187 ("drm/amd/display: don't expose rotation prop for cursor plane") removed the rotation property for the cursor plane, assuming the cursor would always be displayed without any rotation. However the rotation is inherited from the underlying plane.

As a result, if the primary plane is rotated, then the cursor plane will incorrectly be rotated as well even though it doesn't have a rotation property.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

To fix this, check that the underlying plane isn't rotated.

HW cursor works with a rotated screen on KDE on AMD HW. This looks like it will break that.

Sorry, scrap that. KDE seems to do a SW rotation with HW cursor.

Hmm, it does work correctly though (with Navi 14) with 180° rotation in gnome-shell, which uses plane rotation for the primary plane and pre-rotated contents for the cursor plane.

This patch results in drmModeAtomicCommit failing and the monitor losing signal instead.

Simon, how did you determine that the HW cursor contents get rotated according to the underlying planes?

Signature · 2 lines
Earthling Michel Dänzer            |                  https://redhat.com
Libre software enthusiast          |         Mesa and Xwayland developer
author
#8 lore raw

PATCH v3 4/4 amd/display: re-introduce cursor plane rotation prop

In reply to Simon Ser

Allow user-space to use the cursor plane with a rotated underlying plane under the condition that both planes have the same rotation.

Cc Alex Deucher <[email protected]>
Cc Harry Wentland <[email protected]>
Cc Nicholas Kazlauskas <[email protected]>
Cc Michel Dänzer <[email protected]>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c       | 4 ++--
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c | 3 +--
 2 files changed, 3 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +2 -2
index aba01255ba12..23ff33c78879 100644
@@ -9867,9 +9867,9 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
 			return -EINVAL;
 		}
 
-		if (new_underlying_state->rotation != DRM_MODE_ROTATE_0) {
+		if (new_underlying_state->rotation != new_cursor_state->rotation) {
 			drm_dbg_atomic(crtc->dev,
-				       "Cursor [PLANE:%d:%s] can't be used with rotated underlying [PLANE:%d:%s]\n",
+				       "Cursor [PLANE:%d:%s] rotation doesn't match underlying [PLANE:%d:%s]\n",
 				       cursor->base.id, cursor->name, underlying->base.id, underlying->name);
 			return -EINVAL;
 		}
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_plane.c +1 -2
index 8eeca160d434..1b3ef68f5bc4 100644
@@ -1487,8 +1487,7 @@ int amdgpu_dm_plane_init(struct amdgpu_display_manager *dm,
 		DRM_MODE_ROTATE_0 | DRM_MODE_ROTATE_90 |
 		DRM_MODE_ROTATE_180 | DRM_MODE_ROTATE_270;
 
-	if (dm->adev->asic_type >= CHIP_BONAIRE &&
-	    plane->type != DRM_PLANE_TYPE_CURSOR)
+	if (dm->adev->asic_type >= CHIP_BONAIRE)
 		drm_plane_create_rotation_property(plane, DRM_MODE_ROTATE_0,
 						   supported_rotations);
 
Signature · 1 line
2.41.0