Skip to contentSkip to conversation

drm/fourcc: Add a modifier for contiguous memory

RFC In review Hsia-Jun Li 4 participants 6 messages Nov 29, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

RFC 1/1 drm/fourcc: Add a modifier for contiguous memory

From: "Hsia-Jun(Randy) Li" <[email protected]>

Hello All

Currently, we assume all the pixel formats are multiple planes, devices could support each component has its own memory plane.
But that may not apply for any device in the world. We could have a device without IOMMU then this is not impossible.

Besides, when we export an handle through the PRIME, the upstream device(likes a capture card or camera) may not support non-contiguous memory. It would be better to allocate the handle in contiguous memory at the first time.

We may think the memory allocation is done in user space, we could do the trick there. But the dumb_create() sometimes is not the right API for that.

"Note that userspace is not allowed to use such objects for render acceleration - drivers must create their own private ioctls for such a use case."
"Note that dumb objects may not be used for gpu acceleration, as has been attempted on some ARM embedded platforms. Such drivers really must have a hardware-specific ioctl to allocate suitable buffer objects."

We need to relay on those device custom APIs then. It would be helpful for their library to calculate the right size for contiguous memory. It would be useful for the driver supports rendering dumb buffer as well.

---
 include/uapi/drm/drm_fourcc.h | 5 +++++
 1 file changed, 5 insertions(+)
include/uapi/drm/drm_fourcc.h +5 -0
index bc056f2d537d..ec039ced8257 100644
@@ -473,6 +473,11 @@ extern "C" {
  */
 #define DRM_FORMAT_MOD_LINEAR	fourcc_mod_code(NONE, 0)
 
+/*
+ * Contiguous memory
+ */
+#define DRM_FORMAT_MOD_CONTIG_MEM	fourcc_mod_code(NONE, 1)
+
 /*
  * Deprecated: use DRM_FORMAT_MOD_LINEAR instead
  *
Signature · 1 line
2.17.1
#1 lore raw

Format modifiers are for the buffer layout only, not for the allocation parameters, placement, etc. See the doc comment at the top of drm_fourcc.h.

#2 lore raw
Quoted 6 lines · Simon Ser

CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.

Format modifiers are for the buffer layout only, not for the allocation parameters, placement, etc. See the doc comment at the top of drm_fourcc.h.

In the v4l2 mail list, we have such proposal that dropping the pixel formats(not the codec formats) from v4l2 header completely, as the growing of tile pixel formats. But we can't get rid of those variants about non-contiguous(the same value FOURCC in v4l2 are all for the contiguous memory).

Before I solve this problem, I believe the support for tile formats in v4l2 would never be stable.

The most common way here is to hack the pixel format modifier, then userspace library could be aware this in allocation and get properties of the drm_planes.

Or another way, we could add a common plane property, indicated that whether the driver requests contiguous memory plane for a format?

Signature · 1 line
Hsia-Jun(Randy) Li
#3 lore raw
In reply to Hsia-Jun Li

Hi Randy,

Quoted 21 lines · Hsia-Jun Li

Currently, we assume all the pixel formats are multiple planes, devices could support each component has its own memory plane.
But that may not apply for any device in the world. We could have a device without IOMMU then this is not impossible.

Besides, when we export an handle through the PRIME, the upstream device(likes a capture card or camera) may not support non-contiguous memory. It would be better to allocate the handle in contiguous memory at the first time.

We may think the memory allocation is done in user space, we could do the trick there. But the dumb_create() sometimes is not the right API for that.

"Note that userspace is not allowed to use such objects for render acceleration - drivers must create their own private ioctls for such a use case."
"Note that dumb objects may not be used for gpu acceleration, as has been attempted on some ARM embedded platforms. Such drivers really must have a hardware-specific ioctl to allocate suitable buffer objects."

Quoted 3 lines

We need to relay on those device custom APIs then. It would be helpful for their library to calculate the right size for contiguous memory. It would be useful for the driver supports rendering dumb buffer as well.

As a buffer can only have a single modifier, this isn't practical. Contiguous needs to be negotiated separately and out of band. See e.g. dma-heaps for this.

Cheers,
Daniel

#4 lore raw
Quoted 27 lines · Daniel Stone

CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.

Hi Randy,

Quoted 21 lines · Hsia-Jun Li

Currently, we assume all the pixel formats are multiple planes, devices could support each component has its own memory plane.
But that may not apply for any device in the world. We could have a device without IOMMU then this is not impossible.

Besides, when we export an handle through the PRIME, the upstream device(likes a capture card or camera) may not support non-contiguous memory. It would be better to allocate the handle in contiguous memory at the first time.

We may think the memory allocation is done in user space, we could do the trick there. But the dumb_create() sometimes is not the right API for that.

"Note that userspace is not allowed to use such objects for render acceleration - drivers must create their own private ioctls for such a use case."
"Note that dumb objects may not be used for gpu acceleration, as has been attempted on some ARM embedded platforms. Such drivers really must have a hardware-specific ioctl to allocate suitable buffer objects."

Quoted 5 lines
Quoted 3 lines

We need to relay on those device custom APIs then. It would be helpful for their library to calculate the right size for contiguous memory. It would be useful for the driver supports rendering dumb buffer as well.

As a buffer can only have a single modifier, this isn't practical.

Usually only those legacy or low cost devices would request this modifier. Unlikely they would support tile format(or we would add support for them).

But yes, we would be better not set a trap for us.

Quoted 2 lines

Contiguous needs to be negotiated separately and out of band. See e.g. dma-heaps for this.

I don't really like the Android way here. If we are in a world of no hot-plug. That would be fine.

V4L2 has had a way to negotiate the memory layout it could support. Android gralloc would use the fixed platform sentences to decide the memory layout and buffer size. That is not flexible.

So would it be better that I add a common property(a list be the same length of the formats property) in drm_plane ?

Quoted 3 lines

Cheers,
Daniel

Signature · 1 line
Hsia-Jun(Randy) Li
#5 lore raw
In reply to Hsia-Jun Li
Quoted 5 lines · On Tue, 29 Nov 2022 18:10:30 +0800 Hsia-Jun Li <[email protected]> wrote:

From: "Hsia-Jun(Randy) Li" <[email protected]>

Hello All

Currently, we assume all the pixel formats are multiple planes,

Hi,

that's not true for any definition of "multiple planes" that I know of.

For example, DRM_FORMAT_XRGB8888 is a single-plane format by definition.

From below it sounds like you mean "physically non-contiguous". But no, pixel formats make no such assumption at all. Contiguous or not is independent of pixel formats.

Quoted 35 lines

devices
could support each component has its own memory plane. But that may not apply for any device in the world. We could have a device without IOMMU then this is not impossible.

Besides, when we export an handle through the PRIME, the upstream device(likes a capture card or camera) may not support non-contiguous memory. It would be better to allocate the handle in contiguous memory at the first time.

We may think the memory allocation is done in user space, we could do the trick there. But the dumb_create() sometimes is not the right API for that.

"Note that userspace is not allowed to use such objects for render acceleration - drivers must create their own private ioctls for such a use case."
"Note that dumb objects may not be used for gpu acceleration, as has been attempted on some ARM embedded platforms. Such drivers really must have a hardware-specific ioctl to allocate suitable buffer objects."

We need to relay on those device custom APIs then. It would be helpful for their library to calculate the right size for contiguous memory. It would be useful for the driver supports rendering dumb buffer as well.

---
 include/uapi/drm/drm_fourcc.h | 5 +++++
 1 file changed, 5 insertions(+)
include/uapi/drm/drm_fourcc.h +0 -0
index bc056f2d537d..ec039ced8257 100644

include/uapi/drm/drm_fourcc.h extern "C" { · line 479

Quoted 7 lines
include/uapi/drm/drm_fourcc.h +4 -0
@@ -473,6 +473,11 @@ extern "C" {
  */
 #define DRM_FORMAT_MOD_LINEAR	fourcc_mod_code(NONE, 0)
 
+/*
+ * Contiguous memory
+ */
+#define DRM_FORMAT_MOD_CONTIG_MEM	fourcc_mod_code(NONE, 1)

NAK. This is not what modifiers are for.

This also would not work in practise, because if this was a modifier, you would not be able to use the actual modifiers.

Thanks,
pq

Quoted 4 lines
include/uapi/drm/drm_fourcc.h +1 -0
+
 /*
  * Deprecated: use DRM_FORMAT_MOD_LINEAR instead
  *