Overlay support in the i.MX7 display
Hello,
I'm looking at the available options to support overlays in the display pipeline of the i.MX7. The LCDIF itself unfortunaltey doesn't support overlays, the feature being implemented in the PXP. A driver for the PXP is available but only supports older SoCs whose PXP doesn't support overlays. This driver is implemented as a V4L2 mem2mem driver, which makes support of additional input channels impossible.
Here are the options I can envision:
- Extend the existing PXP driver to support multiple channels. This is technically feasible, but will require moving away from the V4L2 mem2mem framework, which would break userspace. I don't think this path could lead anywhere.
- Write a new PXP driver for the i.MX7, still using V4L2, but with multiple video nodes. This would allow blending multiple layers, but would require writing the output to memory, while the PXP has support for direct connections to the LCDIF (through small SRAM buffers). Performances would thus be suboptimal. The API would also be awkward, as using the PXP for display would require usage of V4L2 in applications.
- Extend the mxsfb driver with PXP support, and expose the PXP inputs as KMS planes. The PXP would only be used when available, and would be transparent to applications. This would however prevent using it separately from the display (to perform multi-pass alpha blending for instance).
What would be the best option going forward ? Would any of you, by any chance, have already started work in this area ?
Signature · 7 lines
Regards, Laurent Pinchart _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi Laurent,
Quoted 8 lines · Laurent Pinchart
Hello,
I'm looking at the available options to support overlays in the display pipeline of the i.MX7. The LCDIF itself unfortunaltey doesn't support overlays, the feature being implemented in the PXP. A driver for the PXP is available but only supports older SoCs whose PXP doesn't support overlays. This driver is implemented as a V4L2 mem2mem driver, which makes support of additional input channels impossible.
Thanks for bringing this up, it is a topic I have wondered too:
Interaction between PXP and mxsfb.
I am not very familiar with the V4L2 subsystem so take my opinions with a grain of salt.
Quoted 8 lines
Here are the options I can envision:
- Extend the existing PXP driver to support multiple channels. This is technically feasible, but will require moving away from the V4L2 mem2mem framework, which would break userspace. I don't think this path could lead anywhere.
Quoted 7 lines
- Write a new PXP driver for the i.MX7, still using V4L2, but with multiple video nodes. This would allow blending multiple layers, but would require writing the output to memory, while the PXP has support for direct connections to the LCDIF (through small SRAM buffers). Performances would thus be suboptimal. The API would also be awkward, as using the PXP for display would require usage of V4L2 in applications.
So the video nodes would be sinks? I would expect overlays to be usable through KMS, I guess that would then not work, correct?
Quoted 6 lines
- Extend the mxsfb driver with PXP support, and expose the PXP inputs as KMS planes. The PXP would only be used when available, and would be transparent to applications. This would however prevent using it separately from the display (to perform multi-pass alpha blending for instance).
KMS planes are well defined and are well integrated with the KMS API, so I prefer this option. But is this compatible with the currently supported video use-case? E.g. could we make PXP available through V4L2 and through DRM/mxsfb?
Not sure what your use case is exactly, but when playing a video I wonder where is the higher value using PXP: Color conversion and scaling or compositing...? I would expect higher value in the former use case.
Quoted 3 lines
What would be the best option going forward ? Would any of you, by any chance, have already started work in this area ?
I have not worked in that area, so feel free!
--
Stefan
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 44 lines · On Sun, 03 Nov 2019 19:15:49 +0100 Stefan Agner <[email protected]> wrote:
Hi Laurent,
Quoted 8 lines · Laurent Pinchart
Hello,
I'm looking at the available options to support overlays in the display pipeline of the i.MX7. The LCDIF itself unfortunaltey doesn't support overlays, the feature being implemented in the PXP. A driver for the PXP is available but only supports older SoCs whose PXP doesn't support overlays. This driver is implemented as a V4L2 mem2mem driver, which makes support of additional input channels impossible.
Thanks for bringing this up, it is a topic I have wondered too:
Interaction between PXP and mxsfb.I am not very familiar with the V4L2 subsystem so take my opinions with a grain of salt.
Quoted 8 lines
Here are the options I can envision:
- Extend the existing PXP driver to support multiple channels. This is technically feasible, but will require moving away from the V4L2 mem2mem framework, which would break userspace. I don't think this path could lead anywhere.
Quoted 7 lines
- Write a new PXP driver for the i.MX7, still using V4L2, but with multiple video nodes. This would allow blending multiple layers, but would require writing the output to memory, while the PXP has support for direct connections to the LCDIF (through small SRAM buffers). Performances would thus be suboptimal. The API would also be awkward, as using the PXP for display would require usage of V4L2 in applications.
So the video nodes would be sinks? I would expect overlays to be usable through KMS, I guess that would then not work, correct?
Quoted 6 lines
- Extend the mxsfb driver with PXP support, and expose the PXP inputs as KMS planes. The PXP would only be used when available, and would be transparent to applications. This would however prevent using it separately from the display (to perform multi-pass alpha blending for instance).
Quoted 8 lines
KMS planes are well defined and are well integrated with the KMS API, so I prefer this option. But is this compatible with the currently supported video use-case? E.g. could we make PXP available through V4L2 and through DRM/mxsfb?
Not sure what your use case is exactly, but when playing a video I wonder where is the higher value using PXP: Color conversion and scaling or compositing...? I would expect higher value in the former use case.
Hi,
mind, with Wayland architecture, color conversion and scaling could be at the same level/step as compositing, in the display server instead of an application. Hence if the PXP capabilities were advertised as KMS planes, there should be nothing to patch in Wayland-designed applications to make use of them, assuming the applications did not already rely on V4L2 M2M devices.
Would it not be possible to expose PXP through both uAPI interfaces? At least KMS atomic's TEST_ONLY feature would make it easy to say "no" to userspace if another bit of userspace already reserved the device via e.g. V4L2.
Thanks,
pq
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hello,
Quoted 32 lines · Pekka Paalanen
Quoted 31 lines · On Sun, 03 Nov 2019 19:15:49 +0100 Stefan Agner wrote:
Quoted 8 lines · Laurent Pinchart
Hello,
I'm looking at the available options to support overlays in the display pipeline of the i.MX7. The LCDIF itself unfortunaltey doesn't support overlays, the feature being implemented in the PXP. A driver for the PXP is available but only supports older SoCs whose PXP doesn't support overlays. This driver is implemented as a V4L2 mem2mem driver, which makes support of additional input channels impossible.
Thanks for bringing this up, it is a topic I have wondered too:
Interaction between PXP and mxsfb.I am not very familiar with the V4L2 subsystem so take my opinions with a grain of salt.
Quoted 14 lines
Here are the options I can envision:
- Extend the existing PXP driver to support multiple channels. This is technically feasible, but will require moving away from the V4L2 mem2mem framework, which would break userspace. I don't think this path could lead anywhere.
- Write a new PXP driver for the i.MX7, still using V4L2, but with multiple video nodes. This would allow blending multiple layers, but would require writing the output to memory, while the PXP has support for direct connections to the LCDIF (through small SRAM buffers). Performances would thus be suboptimal. The API would also be awkward, as using the PXP for display would require usage of V4L2 in applications.
Quoted 2 lines
Quoted 2 lines
So the video nodes would be sinks? I would expect overlays to be usable through KMS, I guess that would then not work, correct?
There would be sink video nodes for the PXP inputs, and one source video node for the PXP output. The PXP can be used stand-alone, in memory-to-memory mode, and V4L2 is a good fit for that.
Quoted 11 lines
Quoted 11 lines
Quoted 6 lines
- Extend the mxsfb driver with PXP support, and expose the PXP inputs as KMS planes. The PXP would only be used when available, and would be transparent to applications. This would however prevent using it separately from the display (to perform multi-pass alpha blending for instance).
KMS planes are well defined and are well integrated with the KMS API, so I prefer this option. But is this compatible with the currently supported video use-case? E.g. could we make PXP available through V4L2 and through DRM/mxsfb?
That's the issue, it's not easily doable. I think we could do so, but how to ensure mutual exclusion between the two APIs needs to be researched. I fear it will result in an awkward solution with fuzzy semantics. A module parameter could be an option, but wouldn't be very flexible.
Quoted 3 lines
Quoted 3 lines
Not sure what your use case is exactly, but when playing a video I wonder where is the higher value using PXP: Color conversion and scaling or compositing...? I would expect higher value in the former use case.
I think it's highly use-case-dependent.
Quoted 11 lines
mind, with Wayland architecture, color conversion and scaling could be at the same level/step as compositing, in the display server instead of an application. Hence if the PXP capabilities were advertised as KMS planes, there should be nothing to patch in Wayland-designed applications to make use of them, assuming the applications did not already rely on V4L2 M2M devices.
Would it not be possible to expose PXP through both uAPI interfaces? At least KMS atomic's
TEST_ONLYfeature would make it easy to say "no" to userspace if another bit of userspace already reserved the device via e.g. V4L2.
We would also need to figure out how to do it the other way around, reporting properly through V4L2 that the device is busy. I think it's feasible, but I doubt it would result in anything usable for userspace. If the KMS device exposes multiple planes unconditionally and fails the atomic commit if the PXP is used through V4L2, I think it would be hard for Wayland to use this consistently. Given that I expect the PXP to be mostly used for display purpose I'm tempted to allocate it for display unconditionally, or, possibly, decide how to expose it through a module parameter.
We have a similar situation on Renesas R-Car Gen3 platforms, with a memory-to-memory compositor called VSP. Some VSP instances are connected to the display controller, and we allocate them for display unconditionally. Other VSP instances are exposed as V4L2 devices. We haven't heard of anyone who wanted to use the display VSP instances for unrelated purposes. If such a use case arose, exposing those instances through V4L2 would just be a matter of flipping one bit in the driver (all the infrastructure is in place), which we would likely expose as a module parameter.
Signature · 7 lines
Regards, Laurent Pinchart _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 78 lines · Laurent Pinchart
Hello,
Quoted 32 lines · Pekka Paalanen
Quoted 31 lines · On Sun, 03 Nov 2019 19:15:49 +0100 Stefan Agner wrote:
Quoted 8 lines · Laurent Pinchart
Hello,
I'm looking at the available options to support overlays in the display pipeline of the i.MX7. The LCDIF itself unfortunaltey doesn't support overlays, the feature being implemented in the PXP. A driver for the PXP is available but only supports older SoCs whose PXP doesn't support overlays. This driver is implemented as a V4L2 mem2mem driver, which makes support of additional input channels impossible.
Thanks for bringing this up, it is a topic I have wondered too:
Interaction between PXP and mxsfb.I am not very familiar with the V4L2 subsystem so take my opinions with a grain of salt.
Quoted 14 lines
Here are the options I can envision:
- Extend the existing PXP driver to support multiple channels. This is technically feasible, but will require moving away from the V4L2 mem2mem framework, which would break userspace. I don't think this path could lead anywhere.
- Write a new PXP driver for the i.MX7, still using V4L2, but with multiple video nodes. This would allow blending multiple layers, but would require writing the output to memory, while the PXP has support for direct connections to the LCDIF (through small SRAM buffers). Performances would thus be suboptimal. The API would also be awkward, as using the PXP for display would require usage of V4L2 in applications.
Quoted 2 lines
Quoted 2 lines
So the video nodes would be sinks? I would expect overlays to be usable through KMS, I guess that would then not work, correct?
There would be sink video nodes for the PXP inputs, and one source video node for the PXP output. The PXP can be used stand-alone, in memory-to-memory mode, and V4L2 is a good fit for that.
Quoted 11 lines
Quoted 11 lines
Quoted 6 lines
- Extend the mxsfb driver with PXP support, and expose the PXP inputs as KMS planes. The PXP would only be used when available, and would be transparent to applications. This would however prevent using it separately from the display (to perform multi-pass alpha blending for instance).
KMS planes are well defined and are well integrated with the KMS API, so I prefer this option. But is this compatible with the currently supported video use-case? E.g. could we make PXP available through V4L2 and through DRM/mxsfb?
That's the issue, it's not easily doable. I think we could do so, but how to ensure mutual exclusion between the two APIs needs to be researched. I fear it will result in an awkward solution with fuzzy semantics. A module parameter could be an option, but wouldn't be very flexible.
Quoted 3 lines
Quoted 3 lines
Not sure what your use case is exactly, but when playing a video I wonder where is the higher value using PXP: Color conversion and scaling or compositing...? I would expect higher value in the former use case.
I think it's highly use-case-dependent.
Quoted 11 lines
mind, with Wayland architecture, color conversion and scaling could be at the same level/step as compositing, in the display server instead of an application. Hence if the PXP capabilities were advertised as KMS planes, there should be nothing to patch in Wayland-designed applications to make use of them, assuming the applications did not already rely on V4L2 M2M devices.
Would it not be possible to expose PXP through both uAPI interfaces? At least KMS atomic's
TEST_ONLYfeature would make it easy to say "no" to userspace if another bit of userspace already reserved the device via e.g. V4L2.
Quoted 9 lines
We would also need to figure out how to do it the other way around, reporting properly through V4L2 that the device is busy. I think it's feasible, but I doubt it would result in anything usable for userspace. If the KMS device exposes multiple planes unconditionally and fails the atomic commit if the PXP is used through V4L2, I think it would be hard for Wayland to use this consistently. Given that I expect the PXP to be mostly used for display purpose I'm tempted to allocate it for display unconditionally, or, possibly, decide how to expose it through a module parameter.
KMS should be fine if planes are missing, userspace is supposed to be able to cope with that. Not all userspace does, but welp.
I figured the bigger issue will be on the v4l side, since "device temporarily gone" is not something v4l understands as a concept?
But even then having one device for userspace would be best I think, just a lot more reasonable (insert wish for unified video/display subsystem here).
Quoted 9 lines
We have a similar situation on Renesas R-Car Gen3 platforms, with a memory-to-memory compositor called VSP. Some VSP instances are connected to the display controller, and we allocate them for display unconditionally. Other VSP instances are exposed as V4L2 devices. We haven't heard of anyone who wanted to use the display VSP instances for unrelated purposes. If such a use case arose, exposing those instances through V4L2 would just be a matter of flipping one bit in the driver (all the infrastructure is in place), which we would likely expose as a module parameter.
Hm yeah I guess we could just assign them, if the use-cases are clear-cut enough. Are you thinking of doing these links with dt (so at least it would be patchable)?
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 88 lines · Daniel Vetter
Quoted 77 lines · Laurent Pinchart
Hello,
Quoted 32 lines · Pekka Paalanen
Quoted 31 lines · On Sun, 03 Nov 2019 19:15:49 +0100 Stefan Agner wrote:
Quoted 8 lines · Laurent Pinchart
Hello,
I'm looking at the available options to support overlays in the display pipeline of the i.MX7. The LCDIF itself unfortunaltey doesn't support overlays, the feature being implemented in the PXP. A driver for the PXP is available but only supports older SoCs whose PXP doesn't support overlays. This driver is implemented as a V4L2 mem2mem driver, which makes support of additional input channels impossible.
Thanks for bringing this up, it is a topic I have wondered too:
Interaction between PXP and mxsfb.I am not very familiar with the V4L2 subsystem so take my opinions with a grain of salt.
Quoted 14 lines
Here are the options I can envision:
- Extend the existing PXP driver to support multiple channels. This is technically feasible, but will require moving away from the V4L2 mem2mem framework, which would break userspace. I don't think this path could lead anywhere.
- Write a new PXP driver for the i.MX7, still using V4L2, but with multiple video nodes. This would allow blending multiple layers, but would require writing the output to memory, while the PXP has support for direct connections to the LCDIF (through small SRAM buffers). Performances would thus be suboptimal. The API would also be awkward, as using the PXP for display would require usage of V4L2 in applications.
Quoted 2 lines
Quoted 2 lines
So the video nodes would be sinks? I would expect overlays to be usable through KMS, I guess that would then not work, correct?
There would be sink video nodes for the PXP inputs, and one source video node for the PXP output. The PXP can be used stand-alone, in memory-to-memory mode, and V4L2 is a good fit for that.
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
- Extend the mxsfb driver with PXP support, and expose the PXP inputs as KMS planes. The PXP would only be used when available, and would be transparent to applications. This would however prevent using it separately from the display (to perform multi-pass alpha blending for instance).
KMS planes are well defined and are well integrated with the KMS API, so I prefer this option. But is this compatible with the currently supported video use-case? E.g. could we make PXP available through V4L2 and through DRM/mxsfb?
That's the issue, it's not easily doable. I think we could do so, but how to ensure mutual exclusion between the two APIs needs to be researched. I fear it will result in an awkward solution with fuzzy semantics. A module parameter could be an option, but wouldn't be very flexible.
Quoted 3 lines
Quoted 3 lines
Not sure what your use case is exactly, but when playing a video I wonder where is the higher value using PXP: Color conversion and scaling or compositing...? I would expect higher value in the former use case.
I think it's highly use-case-dependent.
Quoted 11 lines
mind, with Wayland architecture, color conversion and scaling could be at the same level/step as compositing, in the display server instead of an application. Hence if the PXP capabilities were advertised as KMS planes, there should be nothing to patch in Wayland-designed applications to make use of them, assuming the applications did not already rely on V4L2 M2M devices.
Would it not be possible to expose PXP through both uAPI interfaces? At least KMS atomic's
TEST_ONLYfeature would make it easy to say "no" to userspace if another bit of userspace already reserved the device via e.g. V4L2.Quoted 9 lines
We would also need to figure out how to do it the other way around, reporting properly through V4L2 that the device is busy. I think it's feasible, but I doubt it would result in anything usable for userspace. If the KMS device exposes multiple planes unconditionally and fails the atomic commit if the PXP is used through V4L2, I think it would be hard for Wayland to use this consistently. Given that I expect the PXP to be mostly used for display purpose I'm tempted to allocate it for display unconditionally, or, possibly, decide how to expose it through a module parameter.
Quoted 5 lines
KMS should be fine if planes are missing, userspace is supposed to be able to cope with that. Not all userspace does, but welp.
I figured the bigger issue will be on the v4l side, since "device temporarily gone" is not something v4l understands as a concept?
m2m are different, since each time you open the device you get a new instance. This is handled for decoders already in userspace, I really don't see the difference. I'm guessing application that wants to multiplex the PXP, will have to avoid the PXP planes for this HW, this might be a little tricky and seems HW specific.
Quoted 15 lines
But even then having one device for userspace would be best I think, just a lot more reasonable (insert wish for unified video/display subsystem here).
Quoted 9 lines
We have a similar situation on Renesas R-Car Gen3 platforms, with a memory-to-memory compositor called VSP. Some VSP instances are connected to the display controller, and we allocate them for display unconditionally. Other VSP instances are exposed as V4L2 devices. We haven't heard of anyone who wanted to use the display VSP instances for unrelated purposes. If such a use case arose, exposing those instances through V4L2 would just be a matter of flipping one bit in the driver (all the infrastructure is in place), which we would likely expose as a module parameter.
Quoted 4 lines
Hm yeah I guess we could just assign them, if the use-cases are clear-cut enough. Are you thinking of doing these links with dt (so at least it would be patchable)?
-Daniel
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 72 lines · Laurent Pinchart
Hello,
Quoted 32 lines · Pekka Paalanen
Quoted 31 lines · On Sun, 03 Nov 2019 19:15:49 +0100 Stefan Agner wrote:
Quoted 8 lines · Laurent Pinchart
Hello,
I'm looking at the available options to support overlays in the display pipeline of the i.MX7. The LCDIF itself unfortunaltey doesn't support overlays, the feature being implemented in the PXP. A driver for the PXP is available but only supports older SoCs whose PXP doesn't support overlays. This driver is implemented as a V4L2 mem2mem driver, which makes support of additional input channels impossible.
Thanks for bringing this up, it is a topic I have wondered too:
Interaction between PXP and mxsfb.I am not very familiar with the V4L2 subsystem so take my opinions with a grain of salt.
Quoted 14 lines
Here are the options I can envision:
- Extend the existing PXP driver to support multiple channels. This is technically feasible, but will require moving away from the V4L2 mem2mem framework, which would break userspace. I don't think this path could lead anywhere.
- Write a new PXP driver for the i.MX7, still using V4L2, but with multiple video nodes. This would allow blending multiple layers, but would require writing the output to memory, while the PXP has support for direct connections to the LCDIF (through small SRAM buffers). Performances would thus be suboptimal. The API would also be awkward, as using the PXP for display would require usage of V4L2 in applications.
Quoted 2 lines
Quoted 2 lines
So the video nodes would be sinks? I would expect overlays to be usable through KMS, I guess that would then not work, correct?
There would be sink video nodes for the PXP inputs, and one source video node for the PXP output. The PXP can be used stand-alone, in memory-to-memory mode, and V4L2 is a good fit for that.
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
- Extend the mxsfb driver with PXP support, and expose the PXP inputs as KMS planes. The PXP would only be used when available, and would be transparent to applications. This would however prevent using it separately from the display (to perform multi-pass alpha blending for instance).
KMS planes are well defined and are well integrated with the KMS API, so I prefer this option. But is this compatible with the currently supported video use-case? E.g. could we make PXP available through V4L2 and through DRM/mxsfb?
That's the issue, it's not easily doable. I think we could do so, but how to ensure mutual exclusion between the two APIs needs to be researched. I fear it will result in an awkward solution with fuzzy semantics. A module parameter could be an option, but wouldn't be very flexible.
Quoted 3 lines
Quoted 3 lines
Not sure what your use case is exactly, but when playing a video I wonder where is the higher value using PXP: Color conversion and scaling or compositing...? I would expect higher value in the former use case.
I think it's highly use-case-dependent.
Quoted 7 lines
mind, with Wayland architecture, color conversion and scaling could be at the same level/step as compositing, in the display server instead of an application. Hence if the PXP capabilities were advertised as KMS planes, there should be nothing to patch in Wayland-designed applications to make use of them, assuming the applications did not already rely on V4L2 M2M devices.
Quoted 8 lines
Quoted 4 lines
Would it not be possible to expose PXP through both uAPI interfaces? At least KMS atomic's
TEST_ONLYfeature would make it easy to say "no" to userspace if another bit of userspace already reserved the device via e.g. V4L2.We would also need to figure out how to do it the other way around, reporting properly through V4L2 that the device is busy. I think it's feasible, but I doubt it would result in anything usable for userspace.
We already have this needs for decoders with fixed number of streams.
Quoted 6 lines
If the KMS device exposes multiple planes unconditionally and fails the atomic commit if the PXP is used through V4L2, I think it would be hard for Wayland to use this consistently. Given that I expect the PXP to be mostly used for display purpose I'm tempted to allocate it for display unconditionally, or, possibly, decide how to expose it through a module parameter.
It's a strange statement "mostly used for display purpose", considering the upstream driver exist for video scaling and color conversion, and no one have yet implement the "display purpose" driver.
My impression is that the complication is kernel specific (the fact we very have two subsystems for the same IPs). Since software wise, sharing and allocating resources seems pretty simple.
Quoted 10 lines
We have a similar situation on Renesas R-Car Gen3 platforms, with a memory-to-memory compositor called VSP. Some VSP instances are connected to the display controller, and we allocate them for display unconditionally. Other VSP instances are exposed as V4L2 devices. We haven't heard of anyone who wanted to use the display VSP instances for unrelated purposes. If such a use case arose, exposing those instances through V4L2 would just be a matter of flipping one bit in the driver (all the infrastructure is in place), which we would likely expose as a module parameter.
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 54 lines · Pekka Paalanen
Quoted 42 lines · On Sun, 03 Nov 2019 19:15:49 +0100 Stefan Agner <[email protected]> wrote:
Hi Laurent,
Quoted 8 lines · Laurent Pinchart
Hello,
I'm looking at the available options to support overlays in the display pipeline of the i.MX7. The LCDIF itself unfortunaltey doesn't support overlays, the feature being implemented in the PXP. A driver for the PXP is available but only supports older SoCs whose PXP doesn't support overlays. This driver is implemented as a V4L2 mem2mem driver, which makes support of additional input channels impossible.
Thanks for bringing this up, it is a topic I have wondered too:
Interaction between PXP and mxsfb.I am not very familiar with the V4L2 subsystem so take my opinions with a grain of salt.
Quoted 7 lines
Here are the options I can envision:
- Extend the existing PXP driver to support multiple channels. This is technically feasible, but will require moving away from the V4L2 mem2mem framework, which would break userspace. I don't think this path could lead anywhere.
Quoted 7 lines
- Write a new PXP driver for the i.MX7, still using V4L2, but with multiple video nodes. This would allow blending multiple layers, but would require writing the output to memory, while the PXP has support for direct connections to the LCDIF (through small SRAM buffers). Performances would thus be suboptimal. The API would also be awkward, as using the PXP for display would require usage of V4L2 in applications.
So the video nodes would be sinks? I would expect overlays to be usable through KMS, I guess that would then not work, correct?
Quoted 5 lines
- Extend the mxsfb driver with PXP support, and expose the PXP inputs as KMS planes. The PXP would only be used when available, and would be transparent to applications. This would however prevent using it separately from the display (to perform multi-pass alpha blending for instance).
Quoted 8 lines
KMS planes are well defined and are well integrated with the KMS API, so I prefer this option. But is this compatible with the currently supported video use-case? E.g. could we make PXP available through V4L2 and through DRM/mxsfb?
Not sure what your use case is exactly, but when playing a video I wonder where is the higher value using PXP: Color conversion and scaling or compositing...? I would expect higher value in the former use case.
Quoted 8 lines
Hi,
mind, with Wayland architecture, color conversion and scaling could be at the same level/step as compositing, in the display server instead of an application. Hence if the PXP capabilities were advertised as KMS planes, there should be nothing to patch in Wayland-designed applications to make use of them, assuming the applications did not already rely on V4L2 M2M devices.
The PXP can already be used with GStreamer v4l2convert element, for CSC and scaling.
Quoted 5 lines
Would it not be possible to expose PXP through both uAPI interfaces? At least KMS atomic's
TEST_ONLYfeature would make it easy to say "no" to userspace if another bit of userspace already reserved the device via e.g. V4L2.
Same exist for decoders with fixed number of streams/instances I think.
Quoted 4 lines
Thanks,
pq
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Hi Laurent,
Quoted 9 lines · Laurent Pinchart
Hello,
I'm looking at the available options to support overlays in the display pipeline of the i.MX7. The LCDIF itself unfortunaltey doesn't support overlays, the feature being implemented in the PXP. A driver for the PXP is available but only supports older SoCs whose PXP doesn't support overlays. This driver is implemented as a V4L2 mem2mem driver, which makes support of additional input channels impossible.
Quoted 6 lines
Here are the options I can envision:
- Extend the existing PXP driver to support multiple channels. This is technically feasible, but will require moving away from the V4L2 mem2mem framework, which would break userspace. I don't think this path could lead anywhere.
I may be biased, but please don't break the V4L2 mem2mem usecase :)
Quoted 7 lines
- Write a new PXP driver for the i.MX7, still using V4L2, but with multiple video nodes. This would allow blending multiple layers, but would require writing the output to memory, while the PXP has support for direct connections to the LCDIF (through small SRAM buffers). Performances would thus be suboptimal. The API would also be awkward, as using the PXP for display would require usage of V4L2 in applications.
I'm not sure V4L2 is the best API for multi-pass 2D composition, especially as the PXP is able to blit an overlay onto a background in place.
Quoted 5 lines
- Extend the mxsfb driver with PXP support, and expose the PXP inputs as KMS planes. The PXP would only be used when available, and would be transparent to applications. This would however prevent using it separately from the display (to perform multi-pass alpha blending for instance).
For the SRAM block row buffer path to the LCDIF, I think the KMS plane abstraction is the way to go. The DRM and V4L2 drivers could be made to use a shared backend, such that only one of plane composition and V4L2 scaling/CSC functions can work at the same time.
Quoted 2 lines
What would be the best option going forward ? Would any of you, by any chance, have already started work in this area ?
I have not worked on this.
regards
Philipp
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 29 lines · Philipp Zabel
Hi Laurent,
Quoted 9 lines · Laurent Pinchart
Hello,
I'm looking at the available options to support overlays in the display pipeline of the i.MX7. The LCDIF itself unfortunaltey doesn't support overlays, the feature being implemented in the PXP. A driver for the PXP is available but only supports older SoCs whose PXP doesn't support overlays. This driver is implemented as a V4L2 mem2mem driver, which makes support of additional input channels impossible.
Quoted 6 lines
Here are the options I can envision:
- Extend the existing PXP driver to support multiple channels. This is technically feasible, but will require moving away from the V4L2 mem2mem framework, which would break userspace. I don't think this path could lead anywhere.
I may be biased, but please don't break the V4L2 mem2mem usecase :)
Quoted 7 lines
- Write a new PXP driver for the i.MX7, still using V4L2, but with multiple video nodes. This would allow blending multiple layers, but would require writing the output to memory, while the PXP has support for direct connections to the LCDIF (through small SRAM buffers). Performances would thus be suboptimal. The API would also be awkward, as using the PXP for display would require usage of V4L2 in applications.
Quoted 3 lines
I'm not sure V4L2 is the best API for multi-pass 2D composition, especially as the PXP is able to blit an overlay onto a background in place.
There was some userspace (GStreamer element) doing exactly that with v4l2 m2m using the imx6 driver. The API was fine, even though fences would have made programming it easier.
https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/308
(not merge as we don't have an agreement on kernel side, notably we don't even have a way to control the blend function, so the result is likely dependant on the use case the driver was written for)
The real limitation was that these IP usually supports more then just
blit/blend over another surface, and as you said, supports background.
And to support this use case, we'd need an m2m driver with multiple queues per direction. That was discussed in that last workshop at ELCE, and applies to other m2m IP like muxers and demuxers which exist on STB kind of SoC.
Quoted 12 lines
Quoted 6 lines
- Extend the mxsfb driver with PXP support, and expose the PXP inputs as KMS planes. The PXP would only be used when available, and would be transparent to applications. This would however prevent using it separately from the display (to perform multi-pass alpha blending for instance).
For the SRAM block row buffer path to the LCDIF, I think the KMS plane abstraction is the way to go. The DRM and V4L2 drivers could be made to use a shared backend, such that only one of plane composition and V4L2 scaling/CSC functions can work at the same time.
Quoted 8 lines
Quoted 2 lines
What would be the best option going forward ? Would any of you, by any chance, have already started work in this area ?
I have not worked on this.
regards
Philipp
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel