Skip to contentSkip to conversation

drm_hwcomposer: set CRTC background color when available

RFC PATCH In review Stefan Schake 5 participants 12 messages Feb 22, 2018 – Jul 2, 2018

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 drm_hwcomposer: set CRTC background color when available

Android assumes an implicit black background layer is always present behind all layers it specifies for composition. drm_hwcomposer currently punts responsibility for this to the kernel/DRM platform and puts layers with per-pixel alpha content on the primary plane when requested.

On some platforms (e.g. VC4) a background color fill has a cycle cost for the hardware composer and is not enabled by default. Instead, userland can request a background color through a CRTC property. Use this property to specify the implicit black background Android expects.

---
Kernel changes for this (background_color) are available here:

https://github.com/stschake/linux/commits/background-upstream

Sending as RFC because I'm not entirely clear on whose responsibility this should be, on most DRM drivers it seems to be implicit. I think a case could also be made that VC4 should not accept alpha formats on the lowest layer or enable background color fill when given one anyway. On the other hand, userland control over background color seems desirable regardless and is a feature of multiple hardware composers (i915, vc4, omap).

 drmcrtc.cpp              | 11 +++++++++++
 drmcrtc.h                |  2 ++
 drmdisplaycompositor.cpp | 15 +++++++++++++++
 3 files changed, 28 insertions(+)
drmcrtc.cpp +11 -0
index 1b354fe..d7bcd7a 100644
@@ -52,6 +52,13 @@ int DrmCrtc::Init() {
     ALOGE("Failed to get OUT_FENCE_PTR property");
     return ret;
   }
+
+  ret = drm_->GetCrtcProperty(*this, "background_color",
+                              &background_color_property_);
+  if (ret) {
+    ALOGI("Failed to get background_color property");
+    // This is an optional property
+  }
   return 0;
 }
 
@@ -86,4 +93,8 @@ const DrmProperty &DrmCrtc::mode_property() const {
 const DrmProperty &DrmCrtc::out_fence_ptr_property() const {
   return out_fence_ptr_property_;
 }
+
+const DrmProperty &DrmCrtc::background_color_property() const {
+  return background_color_property_;
+}
 }
drmcrtc.h +2 -0
index c5a5599..704573a 100644
@@ -46,6 +46,7 @@ class DrmCrtc {
   const DrmProperty &active_property() const;
   const DrmProperty &mode_property() const;
   const DrmProperty &out_fence_ptr_property() const;
+  const DrmProperty &background_color_property() const;
 
  private:
   DrmResources *drm_;
@@ -59,6 +60,7 @@ class DrmCrtc {
   DrmProperty active_property_;
   DrmProperty mode_property_;
   DrmProperty out_fence_ptr_property_;
+  DrmProperty background_color_property_;
 };
 }
 
drmdisplaycompositor.cpp +15 -0
index e556e86..69c52dd 100644
@@ -527,6 +527,21 @@ int DrmDisplayCompositor::CommitFrame(DrmDisplayComposition *display_comp,
       return ret;
     }
 
+    if (crtc->background_color_property().id() != 0) {
+      // Background color is specified as a 16 bit per channel RGBA value.
+      // Set a fully opaque black background as Android HWC expects.
+      const uint64_t background_color = 0xFFFF;
+
+      ret = drmModeAtomicAddProperty(pset, crtc->id(),
+                                     crtc->background_color_property().id(),
+                                     background_color) < 0;
+      if (ret) {
+        ALOGE("Failed to add CRTC background_color to pset: %d", ret);
+        drmModeAtomicFree(pset);
+        return ret;
+      }
+    }
+
     ret = drmModeAtomicAddProperty(pset, crtc->id(), crtc->mode_property().id(),
                                    mode_.blob_id) < 0 ||
           drmModeAtomicAddProperty(pset, connector->id(),
Signature · 6 lines
2.7.4

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#1 lore raw

Hey Stefan,

Quoted 4 lines · Stefan Schake

Android assumes an implicit black background layer is always present behind all layers it specifies for composition. drm_hwcomposer currently punts responsibility for this to the kernel/DRM platform and puts layers with per-pixel alpha content on the primary plane when requested.

I wasn't aware of this assumption, but given that it is the case, the patch looks like a good fix for the problem.

Unfortunately I don't have a hardware platform up and running to test the patch with at the moment.

Quoted 88 lines

On some platforms (e.g. VC4) a background color fill has a cycle cost for the hardware composer and is not enabled by default. Instead, userland can request a background color through a CRTC property. Use this property to specify the implicit black background Android expects.

---
Kernel changes for this (background_color) are available here:

https://github.com/stschake/linux/commits/background-upstream

Sending as RFC because I'm not entirely clear on whose responsibility this should be, on most DRM drivers it seems to be implicit. I think a case could also be made that VC4 should not accept alpha formats on the lowest layer or enable background color fill when given one anyway. On the other hand, userland control over background color seems desirable regardless and is a feature of multiple hardware composers (i915, vc4, omap).

  drmcrtc.cpp              | 11 +++++++++++
  drmcrtc.h                |  2 ++
  drmdisplaycompositor.cpp | 15 +++++++++++++++
  3 files changed, 28 insertions(+)
drmcrtc.cpp +11 -0
index 1b354fe..d7bcd7a 100644
@@ -52,6 +52,13 @@ int DrmCrtc::Init() {
      ALOGE("Failed to get OUT_FENCE_PTR property");
      return ret;
    }
+
+  ret = drm_->GetCrtcProperty(*this, "background_color",
+                              &background_color_property_);
+  if (ret) {
+    ALOGI("Failed to get background_color property");
+    // This is an optional property
+  }
    return 0;
  }
 
@@ -86,4 +93,8 @@ const DrmProperty &DrmCrtc::mode_property() const {
  const DrmProperty &DrmCrtc::out_fence_ptr_property() const {
    return out_fence_ptr_property_;
  }
+
+const DrmProperty &DrmCrtc::background_color_property() const {
+  return background_color_property_;
+}
  }
drmcrtc.h +2 -0
index c5a5599..704573a 100644
@@ -46,6 +46,7 @@ class DrmCrtc {
    const DrmProperty &active_property() const;
    const DrmProperty &mode_property() const;
    const DrmProperty &out_fence_ptr_property() const;
+  const DrmProperty &background_color_property() const;
 
   private:
    DrmResources *drm_;
@@ -59,6 +60,7 @@ class DrmCrtc {
    DrmProperty active_property_;
    DrmProperty mode_property_;
    DrmProperty out_fence_ptr_property_;
+  DrmProperty background_color_property_;
  };
  }
 
drmdisplaycompositor.cpp +8 -0
index e556e86..69c52dd 100644
@@ -527,6 +527,21 @@ int DrmDisplayCompositor::CommitFrame(DrmDisplayComposition *display_comp,
        return ret;
      }
 
+    if (crtc->background_color_property().id() != 0) {
+      // Background color is specified as a 16 bit per channel RGBA value.
+      // Set a fully opaque black background as Android HWC expects.
+      const uint64_t background_color = 0xFFFF;
+
+      ret = drmModeAtomicAddProperty(pset, crtc->id(),
+                                     crtc->background_color_property().id(),
+                                     background_color) < 0;

drmdisplaycompositor.cpp int DrmDisplayCompositor::CommitFrame(DrmDisplayComposition *display_comp, · line 547

Quoted 10 lines
drmdisplaycompositor.cpp +7 -0
+      if (ret) {
+        ALOGE("Failed to add CRTC background_color to pset: %d", ret);
+        drmModeAtomicFree(pset);
+        return ret;
+      }
+    }
+
      ret = drmModeAtomicAddProperty(pset, crtc->id(), crtc->mode_property().id(),
                                     mode_.blob_id) < 0 ||
            drmModeAtomicAddProperty(pset, connector->id(),

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#2 lore raw

Hey Robert,

Quoted 10 lines · Robert Foss

Hey Stefan,

Quoted 5 lines · Stefan Schake

Android assumes an implicit black background layer is always present behind all layers it specifies for composition. drm_hwcomposer currently punts responsibility for this to the kernel/DRM platform and puts layers with per-pixel alpha content on the primary plane when requested.

Quoted 6 lines

I wasn't aware of this assumption, but given that it is the case, the patch looks like a good fix for the problem.

Unfortunately I don't have a hardware platform up and running to test the patch with at the moment.

HWC1 has this ominous comment:

* IMPORTANT NOTE: There is an implicit layer containing opaque black
* pixels behind all the layers in the list. It is the responsibility of
* the hwcomposer module to make sure black pixels are output (or blended
* from).

It's not repeated in the HWC2 docs, but I think the assumption carried over given that Android prefers all things RGBA. Admittedly it's rare that you don't have a full-size opaque layer in the composition like a background picture, so the cases where this causes corrupted rendering on VC4 are limited to some time during boot after the animation finishes but the launcher isn't up yet.

Thanks,
Stefan
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#3 lore raw
In reply to Stefan Schake
Quoted 14 lines · Stefan Schake

Android assumes an implicit black background layer is always present behind all layers it specifies for composition. drm_hwcomposer currently punts responsibility for this to the kernel/DRM platform and puts layers with per-pixel alpha content on the primary plane when requested.

On some platforms (e.g. VC4) a background color fill has a cycle cost for the hardware composer and is not enabled by default. Instead, userland can request a background color through a CRTC property. Use this property to specify the implicit black background Android expects.

---
Kernel changes for this (background_color) are available here:

Quoted 8 lines

https://github.com/stschake/linux/commits/background-upstream

Sending as RFC because I'm not entirely clear on whose responsibility this should be, on most DRM drivers it seems to be implicit. I think a case could also be made that VC4 should not accept alpha formats on the lowest layer or enable background color fill when given one anyway. On the other hand, userland control over background color seems desirable regardless and is a feature of multiple hardware composers (i915, vc4, omap).

Couldn't we just ignore the alpha channel on the primary plane, on the assumption that it's supposed to be all zeroes below it? Or are we not premultiplied, so we do still need to multiply the primary plane's colors by the alpha value? I couldn't find any obvious DRM documentation about whether alpha is premultiplied.

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#4 lore raw

Hey Eric,

Quoted 25 lines · Eric Anholt
Quoted 14 lines · Stefan Schake

Android assumes an implicit black background layer is always present behind all layers it specifies for composition. drm_hwcomposer currently punts responsibility for this to the kernel/DRM platform and puts layers with per-pixel alpha content on the primary plane when requested.

On some platforms (e.g. VC4) a background color fill has a cycle cost for the hardware composer and is not enabled by default. Instead, userland can request a background color through a CRTC property. Use this property to specify the implicit black background Android expects.

---
Kernel changes for this (background_color) are available here:

Quoted 8 lines

https://github.com/stschake/linux/commits/background-upstream

Sending as RFC because I'm not entirely clear on whose responsibility this should be, on most DRM drivers it seems to be implicit. I think a case could also be made that VC4 should not accept alpha formats on the lowest layer or enable background color fill when given one anyway. On the other hand, userland control over background color seems desirable regardless and is a feature of multiple hardware composers (i915, vc4, omap).

Quoted 5 lines

Couldn't we just ignore the alpha channel on the primary plane, on the assumption that it's supposed to be all zeroes below it? Or are we not premultiplied, so we do still need to multiply the primary plane's colors by the alpha value? I couldn't find any obvious DRM documentation about whether alpha is premultiplied.

That would work, too (for a black background anyway). Though the easiest place to spoof this is presumably in the kernel. From what I have seen, everything in Android is already premultiplied, but technically it can specify either. Not sure if this
is correctly handled in drm_hwcomposer yet.

Thanks,
Stefan
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#5 lore raw

Hey,

Quoted 34 lines · Stefan Schake

Hey Eric,

Quoted 25 lines · Eric Anholt
Quoted 14 lines · Stefan Schake

Android assumes an implicit black background layer is always present behind all layers it specifies for composition. drm_hwcomposer currently punts responsibility for this to the kernel/DRM platform and puts layers with per-pixel alpha content on the primary plane when requested.

On some platforms (e.g. VC4) a background color fill has a cycle cost for the hardware composer and is not enabled by default. Instead, userland can request a background color through a CRTC property. Use this property to specify the implicit black background Android expects.

---
Kernel changes for this (background_color) are available here:

Quoted 8 lines

https://github.com/stschake/linux/commits/background-upstream

Sending as RFC because I'm not entirely clear on whose responsibility this should be, on most DRM drivers it seems to be implicit. I think a case could also be made that VC4 should not accept alpha formats on the lowest layer or enable background color fill when given one anyway. On the other hand, userland control over background color seems desirable regardless and is a feature of multiple hardware composers (i915, vc4, omap).

Quoted 5 lines

Couldn't we just ignore the alpha channel on the primary plane, on the assumption that it's supposed to be all zeroes below it? Or are we not premultiplied, so we do still need to multiply the primary plane's colors by the alpha value? I couldn't find any obvious DRM documentation about whether alpha is premultiplied.

Quoted 6 lines

That would work, too (for a black background anyway). Though the easiest place to spoof this is presumably in the kernel. From what I have seen, everything in Android is already premultiplied, but technically it can specify either. Not sure if this
is correctly handled in drm_hwcomposer yet.

To me Erics solution sounds pragmatic, and would be easy to reason about going forward. Making sure that all of the kernel drivers behave the same way might be the hardest part.

Stefan: Are you looking at an entirely kernel side fix now, or are you pushing this series forward?

Rob.
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#7 lore raw

Ack, thanks for the heads up!

Rob.

Quoted 12 lines · Stefan Schake

Hey Rob,

Quoted 4 lines · Robert Foss

Hey,

Stefan: Are you looking at an entirely kernel side fix now, or are you pushing this series forward?

I've sent out a kernel side fix for this:

https://patchwork.freedesktop.org/patch/207667/

Quoted 9 lines

So I guess for now this can be dropped, pending review.

Thanks,
Stefan
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#8 lore raw
In reply to Stefan Schake
Quoted 14 lines · Stefan Schake

Android assumes an implicit black background layer is always present behind all layers it specifies for composition. drm_hwcomposer currently punts responsibility for this to the kernel/DRM platform and puts layers with per-pixel alpha content on the primary plane when requested.

On some platforms (e.g. VC4) a background color fill has a cycle cost for the hardware composer and is not enabled by default. Instead, userland can request a background color through a CRTC property. Use this property to specify the implicit black background Android expects.

---
Kernel changes for this (background_color) are available here:

Quoted 8 lines

https://github.com/stschake/linux/commits/background-upstream

Sending as RFC because I'm not entirely clear on whose responsibility this should be, on most DRM drivers it seems to be implicit. I think a case could also be made that VC4 should not accept alpha formats on the lowest layer or enable background color fill when given one anyway. On the other hand, userland control over background color seems desirable regardless and is a feature of multiple hardware composers (i915, vc4, omap).

I see you've already typed vc4 patches to address, just wanted to clarify this here: The assumption is that all drivers already have a background color of black/0. The background color property would be to set a different background color than black.

If the driver has undefined behaviour for instead of an assumed background color of black, then I think that's a driver bug.

See https://dri.freedesktop.org/docs/drm/gpu/drm-kms.html#plane-composition-properties

"(e.g. for the background color, which currently is not exposed and assumed to be black)"

I'm still hoping for some artist to create a nice svg that explains the kms composition model with a nice graphic ...

Cheers, Daniel

Quoted 86 lines
 drmcrtc.cpp              | 11 +++++++++++
 drmcrtc.h                |  2 ++
 drmdisplaycompositor.cpp | 15 +++++++++++++++
 3 files changed, 28 insertions(+)
drmcrtc.cpp +11 -0
index 1b354fe..d7bcd7a 100644
@@ -52,6 +52,13 @@ int DrmCrtc::Init() {
     ALOGE("Failed to get OUT_FENCE_PTR property");
     return ret;
   }
+
+  ret = drm_->GetCrtcProperty(*this, "background_color",
+                              &background_color_property_);
+  if (ret) {
+    ALOGI("Failed to get background_color property");
+    // This is an optional property
+  }
   return 0;
 }
 
@@ -86,4 +93,8 @@ const DrmProperty &DrmCrtc::mode_property() const {
 const DrmProperty &DrmCrtc::out_fence_ptr_property() const {
   return out_fence_ptr_property_;
 }
+
+const DrmProperty &DrmCrtc::background_color_property() const {
+  return background_color_property_;
+}
 }
drmcrtc.h +2 -0
index c5a5599..704573a 100644
@@ -46,6 +46,7 @@ class DrmCrtc {
   const DrmProperty &active_property() const;
   const DrmProperty &mode_property() const;
   const DrmProperty &out_fence_ptr_property() const;
+  const DrmProperty &background_color_property() const;
 
  private:
   DrmResources *drm_;
@@ -59,6 +60,7 @@ class DrmCrtc {
   DrmProperty active_property_;
   DrmProperty mode_property_;
   DrmProperty out_fence_ptr_property_;
+  DrmProperty background_color_property_;
 };
 }
 
drmdisplaycompositor.cpp +15 -0
index e556e86..69c52dd 100644
@@ -527,6 +527,21 @@ int DrmDisplayCompositor::CommitFrame(DrmDisplayComposition *display_comp,
       return ret;
     }
 
+    if (crtc->background_color_property().id() != 0) {
+      // Background color is specified as a 16 bit per channel RGBA value.
+      // Set a fully opaque black background as Android HWC expects.
+      const uint64_t background_color = 0xFFFF;
+
+      ret = drmModeAtomicAddProperty(pset, crtc->id(),
+                                     crtc->background_color_property().id(),
+                                     background_color) < 0;
+      if (ret) {
+        ALOGE("Failed to add CRTC background_color to pset: %d", ret);
+        drmModeAtomicFree(pset);
+        return ret;
+      }
+    }
+
     ret = drmModeAtomicAddProperty(pset, crtc->id(), crtc->mode_property().id(),
                                    mode_.blob_id) < 0 ||
           drmModeAtomicAddProperty(pset, connector->id(),
Signature · 6 lines
2.7.4

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#9 lore raw
In reply to Stefan Schake

Op 22-02-18 om 04:54 schreef Stefan Schake:

Quoted 14 lines

Android assumes an implicit black background layer is always present behind all layers it specifies for composition. drm_hwcomposer currently punts responsibility for this to the kernel/DRM platform and puts layers with per-pixel alpha content on the primary plane when requested.

On some platforms (e.g. VC4) a background color fill has a cycle cost for the hardware composer and is not enabled by default. Instead, userland can request a background color through a CRTC property. Use this property to specify the implicit black background Android expects.

---
Kernel changes for this (background_color) are available here:

Quoted 8 lines

https://github.com/stschake/linux/commits/background-upstream

Sending as RFC because I'm not entirely clear on whose responsibility this should be, on most DRM drivers it seems to be implicit. I think a case could also be made that VC4 should not accept alpha formats on the lowest layer or enable background color fill when given one anyway. On the other hand, userland control over background color seems desirable regardless and is a feature of multiple hardware composers (i915, vc4, omap).

Ping? Would be nice if we were moving forward. :)
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#10 lore raw

Hey Maarten,

Quoted 17 lines · Maarten Lankhorst

Op 22-02-18 om 04:54 schreef Stefan Schake:

Quoted 16 lines

Android assumes an implicit black background layer is always present behind all layers it specifies for composition. drm_hwcomposer currently punts responsibility for this to the kernel/DRM platform and puts layers with per-pixel alpha content on the primary plane when requested.

On some platforms (e.g. VC4) a background color fill has a cycle cost for the hardware composer and is not enabled by default. Instead, userland can request a background color through a CRTC property. Use this property to specify the implicit black background Android expects.

---
Kernel changes for this (background_color) are available here:

https://github.com/stschake/linux/commits/background-upstream

Quoted 7 lines
Quoted 6 lines

Sending as RFC because I'm not entirely clear on whose responsibility this should be, on most DRM drivers it seems to be implicit. I think a case could also be made that VC4 should not accept alpha formats on the lowest layer or enable background color fill when given one anyway. On the other hand, userland control over background color seems desirable regardless and is a feature of multiple hardware composers (i915, vc4, omap).

Ping? Would be nice if we were moving forward. :)

I was unclear if DRM specified a black background when writing this. Someone pointed me towards the excerpt in the docs that explicitly mandates black background fill and so I ended up writing a VC4 patch that automatically sets it when required instead of the optional property used here.

Adding a background_color property would still be desirable, but I'm unclear on what the userspace would be at the moment. drm_hwc doesn't need any background color other than black and since that is the DRM default, it wouldn't need to use a property.

Thanks,
Stefan
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#11 lore raw

Op 29-06-18 om 12:27 schreef Stefan Schake:

Quoted 34 lines

Hey Maarten,

Quoted 17 lines · Maarten Lankhorst

Op 22-02-18 om 04:54 schreef Stefan Schake:

Quoted 16 lines

Android assumes an implicit black background layer is always present behind all layers it specifies for composition. drm_hwcomposer currently punts responsibility for this to the kernel/DRM platform and puts layers with per-pixel alpha content on the primary plane when requested.

On some platforms (e.g. VC4) a background color fill has a cycle cost for the hardware composer and is not enabled by default. Instead, userland can request a background color through a CRTC property. Use this property to specify the implicit black background Android expects.

---
Kernel changes for this (background_color) are available here:

https://github.com/stschake/linux/commits/background-upstream

Quoted 7 lines
Quoted 6 lines

Sending as RFC because I'm not entirely clear on whose responsibility this should be, on most DRM drivers it seems to be implicit. I think a case could also be made that VC4 should not accept alpha formats on the lowest layer or enable background color fill when given one anyway. On the other hand, userland control over background color seems desirable regardless and is a feature of multiple hardware composers (i915, vc4, omap).

Ping? Would be nice if we were moving forward. :)

I was unclear if DRM specified a black background when writing this. Someone pointed me towards the excerpt in the docs that explicitly mandates black background fill and so I ended up writing a VC4 patch that automatically sets it when required instead of the optional property used here.

Quoted 7 lines

Adding a background_color property would still be desirable, but I'm unclear on what the userspace would be at the moment. drm_hwc doesn't need any background color other than black and since that is the DRM default, it wouldn't need to use a property.

Thanks,
Stefan

It would still be useful for e-paper displays where the background color would be better suited as white, or when we want to blend a different color than black. But I fear for now there's no real userspace. :(

Which is unfortunate, because the series are easy to implement, but it's just a case where there's no need from userspace yet for the hw capability.

~Maarten

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel