Documentation: gpu: Mention the requirements for new properties
Subsystems (3)
PATCH v3 1/1 Documentation: gpu: Mention the requirements for new properties
New KMS properties come with a bunch of requirements to avoid each driver from running their own, inconsistent, set of properties, eventually leading to issues like property conflicts, inconsistencies between drivers and semantics, etc.
Let's document what we expect.
---
Changes from v2:
- Take into account the feedback from Laurent and Lidiu to no longer force generic properties, but prefix vendor-specific properties with the vendor name
Changes from v1:
- Typos and wording reported by Daniel and Alex
--- Documentation/gpu/drm-kms.rst | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
Documentation/gpu/drm-kms.rst +27 -0
@@ -463,6 +463,33 @@ KMS Properties This section of the documentation is primarily aimed at user-space developers. For the driver APIs, see the other sections. +Requirements +------------ + +KMS drivers might need to add extra properties to support new features. +Each new property introduced in a driver need to meet a few +requirements, in addition to the one mentioned above.: + +- Before the introduction of any vendor-specific properties, they must + be first checked against the generic ones to avoid any conflict or + redundancy. + +- Vendor-specific properties must be prefixed by the vendor's name, + following the syntax "$vendor:$property". + +- Generic properties must be standardized, with some documentation to + describe how the property can be used. + +- Generic properties must provide a generic helper in the core code to + register that property on the object it attaches to. + +- Generic properties content must be decoded by the core and provided in + the object's associated state structure. That includes anything + drivers might want to precompute, like :c:type:`struct drm_clip_rect + <drm_clip_rect>` for planes. + +- An IGT test should be submitted. + Property Types and Blob Property Support ----------------------------------------
Signature · 1 line
2.31.1
Hi Maxime,
Quoted 138 lines · Maxime Ripard
New KMS properties come with a bunch of requirements to avoid each driver from running their own, inconsistent, set of properties, eventually leading to issues like property conflicts, inconsistencies between drivers and semantics, etc.
Let's document what we expect.
Cc Alexandre Belloni <[email protected]>Cc Alexandre Torgue <[email protected]>Cc Alex Deucher <[email protected]>Cc Alison Wang <[email protected]>Cc Alyssa Rosenzweig <[email protected]>Cc Andrew Jeffery <[email protected]>Cc Andrzej Hajda <[email protected]>Cc Anitha Chrisanthus <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Ben Skeggs <[email protected]>Cc Boris Brezillon <[email protected]>Cc Brian Starkey <[email protected]>Cc Chen Feng <[email protected]>Cc Chen-Yu Tsai <[email protected]>Cc Christian Gmeiner <[email protected]>Cc "Christian König" <[email protected]>Cc Chun-Kuang Hu <[email protected]>Cc Edmund Dea <[email protected]>Cc Eric Anholt <[email protected]>Cc Fabio Estevam <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Haneen Mohammed <[email protected]>Cc Hans de Goede <[email protected]>Cc "Heiko Stübner" <[email protected]>Cc Huang Rui <[email protected]>Cc Hyun Kwon <[email protected]>Cc Inki Dae <[email protected]>Cc Jani Nikula <[email protected]>Cc Jernej Skrabec <[email protected]>Cc Jerome Brunet <[email protected]>Cc Joel Stanley <[email protected]>Cc John Stultz <[email protected]>Cc Jonas Karlman <[email protected]>Cc Jonathan Hunter <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Joonyoung Shim <[email protected]>Cc Jyri Sarha <[email protected]>Cc Kevin Hilman <[email protected]>Cc Kieran Bingham <[email protected]>Cc Krzysztof Kozlowski <[email protected]>Cc Kyungmin Park <[email protected]>Cc Laurent Pinchart <[email protected]>Cc Linus Walleij <[email protected]>Cc Liviu Dudau <[email protected]>Cc Lucas Stach <[email protected]>Cc Ludovic Desroches <[email protected]>Cc Marek Vasut <[email protected]>Cc Martin Blumenstingl <[email protected]>Cc Matthias Brugger <[email protected]>Cc Maxime Coquelin <[email protected]>Cc Maxime Ripard <[email protected]>Cc Melissa Wen <[email protected]>Cc Neil Armstrong <[email protected]>Cc Nicolas Ferre <[email protected]>Cc "Noralf Trønnes" <[email protected]>Cc NXP Linux Team <[email protected]>Cc Oleksandr Andrushchenko <[email protected]>Cc Patrik Jakobsson <[email protected]>Cc Paul Cercueil <[email protected]>Cc Pengutronix Kernel Team <[email protected]>Cc Philippe Cornu <[email protected]>Cc Philipp Zabel <[email protected]>Cc Qiang Yu <[email protected]>Cc Rob Clark <[email protected]>Cc Robert Foss <[email protected]>Cc Rob Herring <[email protected]>Cc Rodrigo Siqueira <[email protected]>Cc Rodrigo Vivi <[email protected]>Cc Roland Scheidegger <[email protected]>Cc Russell King <[email protected]>Cc Sam Ravnborg <[email protected]>Cc Sandy Huang <[email protected]>Cc Sascha Hauer <[email protected]>Cc Sean Paul <[email protected]>Cc Seung-Woo Kim <[email protected]>Cc Shawn Guo <[email protected]>Cc Stefan Agner <[email protected]>Cc Steven Price <[email protected]>Cc Sumit Semwal <[email protected]>Cc Thierry Reding <[email protected]>Cc Tian Tao <[email protected]>Cc Tomeu Vizoso <[email protected]>Cc Tomi Valkeinen <[email protected]>Cc VMware Graphics <[email protected]>Cc Xinliang Liu <[email protected]>Cc Xinwei Kong <[email protected]>Cc Yannick Fertre <[email protected]>Cc Zack Rusin <[email protected]>Reviewed-by Daniel Vetter <[email protected]>Signed-off-by Maxime Ripard <[email protected]>---
Changes from v2:
- Take into account the feedback from Laurent and Lidiu to no longer force generic properties, but prefix vendor-specific properties with the vendor nameChanges from v1:
- Typos and wording reported by Daniel and Alex--- Documentation/gpu/drm-kms.rst | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)Documentation/gpu/drm-kms.rst +19
-0@@ -463,6 +463,33 @@ KMS Properties This section of the documentation is primarily aimed at user-space developers. For the driver APIs, see the other sections. +Requirements +------------ + +KMS drivers might need to add extra properties to support new features. +Each new property introduced in a driver need to meet a few +requirements, in addition to the one mentioned above.: + +- Before the introduction of any vendor-specific properties, they must + be first checked against the generic ones to avoid any conflict or + redundancy. + +- Vendor-specific properties must be prefixed by the vendor's name, + following the syntax "$vendor:$property". + +- Generic properties must be standardized, with some documentation to + describe how the property can be used. + +- Generic properties must provide a generic helper in the core code to + register that property on the object it attaches to.
Documentation/gpu/drm-kms.rst KMS Properties · line 494
Quoted 10 lines
Documentation/gpu/drm-kms.rst +8
-0+ +- Generic properties content must be decoded by the core and provided in + the object's associated state structure. That includes anything + drivers might want to precompute, like :c:type:`struct drm_clip_rect + <drm_clip_rect>` for planes. + +- An IGT test should be submitted. + Property Types and Blob Property Support ----------------------------------------
Looks nice, thanks for the work!
Best regards,
Liviu
Quoted 4 lines
Signature · 2 lines
-- 2.31.1
Signature · 8 lines
====================
| I would like to |
| fix the world, |
| but they're not |
| giving me the |
\ source code! /
---------------
¯\_(ツ)_/¯Quoted 138 lines · Maxime Ripard
New KMS properties come with a bunch of requirements to avoid each driver from running their own, inconsistent, set of properties, eventually leading to issues like property conflicts, inconsistencies between drivers and semantics, etc.
Let's document what we expect.
Cc Alexandre Belloni <[email protected]>Cc Alexandre Torgue <[email protected]>Cc Alex Deucher <[email protected]>Cc Alison Wang <[email protected]>Cc Alyssa Rosenzweig <[email protected]>Cc Andrew Jeffery <[email protected]>Cc Andrzej Hajda <[email protected]>Cc Anitha Chrisanthus <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Ben Skeggs <[email protected]>Cc Boris Brezillon <[email protected]>Cc Brian Starkey <[email protected]>Cc Chen Feng <[email protected]>Cc Chen-Yu Tsai <[email protected]>Cc Christian Gmeiner <[email protected]>Cc "Christian König" <[email protected]>Cc Chun-Kuang Hu <[email protected]>Cc Edmund Dea <[email protected]>Cc Eric Anholt <[email protected]>Cc Fabio Estevam <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Haneen Mohammed <[email protected]>Cc Hans de Goede <[email protected]>Cc "Heiko Stübner" <[email protected]>Cc Huang Rui <[email protected]>Cc Hyun Kwon <[email protected]>Cc Inki Dae <[email protected]>Cc Jani Nikula <[email protected]>Cc Jernej Skrabec <[email protected]>Cc Jerome Brunet <[email protected]>Cc Joel Stanley <[email protected]>Cc John Stultz <[email protected]>Cc Jonas Karlman <[email protected]>Cc Jonathan Hunter <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Joonyoung Shim <[email protected]>Cc Jyri Sarha <[email protected]>Cc Kevin Hilman <[email protected]>Cc Kieran Bingham <[email protected]>Cc Krzysztof Kozlowski <[email protected]>Cc Kyungmin Park <[email protected]>Cc Laurent Pinchart <[email protected]>Cc Linus Walleij <[email protected]>Cc Liviu Dudau <[email protected]>Cc Lucas Stach <[email protected]>Cc Ludovic Desroches <[email protected]>Cc Marek Vasut <[email protected]>Cc Martin Blumenstingl <[email protected]>Cc Matthias Brugger <[email protected]>Cc Maxime Coquelin <[email protected]>Cc Maxime Ripard <[email protected]>Cc Melissa Wen <[email protected]>Cc Neil Armstrong <[email protected]>Cc Nicolas Ferre <[email protected]>Cc "Noralf Trønnes" <[email protected]>Cc NXP Linux Team <[email protected]>Cc Oleksandr Andrushchenko <[email protected]>Cc Patrik Jakobsson <[email protected]>Cc Paul Cercueil <[email protected]>Cc Pengutronix Kernel Team <[email protected]>Cc Philippe Cornu <[email protected]>Cc Philipp Zabel <[email protected]>Cc Qiang Yu <[email protected]>Cc Rob Clark <[email protected]>Cc Robert Foss <[email protected]>Cc Rob Herring <[email protected]>Cc Rodrigo Siqueira <[email protected]>Cc Rodrigo Vivi <[email protected]>Cc Roland Scheidegger <[email protected]>Cc Russell King <[email protected]>Cc Sam Ravnborg <[email protected]>Cc Sandy Huang <[email protected]>Cc Sascha Hauer <[email protected]>Cc Sean Paul <[email protected]>Cc Seung-Woo Kim <[email protected]>Cc Shawn Guo <[email protected]>Cc Stefan Agner <[email protected]>Cc Steven Price <[email protected]>Cc Sumit Semwal <[email protected]>Cc Thierry Reding <[email protected]>Cc Tian Tao <[email protected]>Cc Tomeu Vizoso <[email protected]>Cc Tomi Valkeinen <[email protected]>Cc VMware Graphics <[email protected]>Cc Xinliang Liu <[email protected]>Cc Xinwei Kong <[email protected]>Cc Yannick Fertre <[email protected]>Cc Zack Rusin <[email protected]>Reviewed-by Daniel Vetter <[email protected]>Signed-off-by Maxime Ripard <[email protected]>---
Changes from v2:
- Take into account the feedback from Laurent and Lidiu to no longer force generic properties, but prefix vendor-specific properties with the vendor nameChanges from v1:
- Typos and wording reported by Daniel and Alex--- Documentation/gpu/drm-kms.rst | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)Documentation/gpu/drm-kms.rst +19
-0@@ -463,6 +463,33 @@ KMS Properties This section of the documentation is primarily aimed at user-space developers. For the driver APIs, see the other sections. +Requirements +------------ + +KMS drivers might need to add extra properties to support new features. +Each new property introduced in a driver need to meet a few +requirements, in addition to the one mentioned above.: + +- Before the introduction of any vendor-specific properties, they must + be first checked against the generic ones to avoid any conflict or + redundancy. + +- Vendor-specific properties must be prefixed by the vendor's name, + following the syntax "$vendor:$property". + +- Generic properties must be standardized, with some documentation to + describe how the property can be used. + +- Generic properties must provide a generic helper in the core code to + register that property on the object it attaches to.
Documentation/gpu/drm-kms.rst KMS Properties · line 494
Quoted 10 lines
Documentation/gpu/drm-kms.rst +8
-0+ +- Generic properties content must be decoded by the core and provided in + the object's associated state structure. That includes anything + drivers might want to precompute, like :c:type:`struct drm_clip_rect + <drm_clip_rect>` for planes. + +- An IGT test should be submitted. + Property Types and Blob Property Support ----------------------------------------
Quoted 4 lines
Signature · 2 lines
-- 2.31.1
Quoted 99 lines · Maxime Ripard
New KMS properties come with a bunch of requirements to avoid each driver from running their own, inconsistent, set of properties, eventually leading to issues like property conflicts, inconsistencies between drivers and semantics, etc.
Let's document what we expect.
Cc Alexandre Belloni <[email protected]>Cc Alexandre Torgue <[email protected]>Cc Alex Deucher <[email protected]>Cc Alison Wang <[email protected]>Cc Alyssa Rosenzweig <[email protected]>Cc Andrew Jeffery <[email protected]>Cc Andrzej Hajda <[email protected]>Cc Anitha Chrisanthus <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Ben Skeggs <[email protected]>Cc Boris Brezillon <[email protected]>Cc Brian Starkey <[email protected]>Cc Chen Feng <[email protected]>Cc Chen-Yu Tsai <[email protected]>Cc Christian Gmeiner <[email protected]>Cc "Christian König" <[email protected]>Cc Chun-Kuang Hu <[email protected]>Cc Edmund Dea <[email protected]>Cc Eric Anholt <[email protected]>Cc Fabio Estevam <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Haneen Mohammed <[email protected]>Cc Hans de Goede <[email protected]>Cc "Heiko Stübner" <[email protected]>Cc Huang Rui <[email protected]>Cc Hyun Kwon <[email protected]>Cc Inki Dae <[email protected]>Cc Jani Nikula <[email protected]>Cc Jernej Skrabec <[email protected]>Cc Jerome Brunet <[email protected]>Cc Joel Stanley <[email protected]>Cc John Stultz <[email protected]>Cc Jonas Karlman <[email protected]>Cc Jonathan Hunter <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Joonyoung Shim <[email protected]>Cc Jyri Sarha <[email protected]>Cc Kevin Hilman <[email protected]>Cc Kieran Bingham <[email protected]>Cc Krzysztof Kozlowski <[email protected]>Cc Kyungmin Park <[email protected]>Cc Laurent Pinchart <[email protected]>Cc Linus Walleij <[email protected]>Cc Liviu Dudau <[email protected]>Cc Lucas Stach <[email protected]>Cc Ludovic Desroches <[email protected]>Cc Marek Vasut <[email protected]>Cc Martin Blumenstingl <[email protected]>Cc Matthias Brugger <[email protected]>Cc Maxime Coquelin <[email protected]>Cc Maxime Ripard <[email protected]>Cc Melissa Wen <[email protected]>Cc Neil Armstrong <[email protected]>Cc Nicolas Ferre <[email protected]>Cc "Noralf Trønnes" <[email protected]>Cc NXP Linux Team <[email protected]>Cc Oleksandr Andrushchenko <[email protected]>Cc Patrik Jakobsson <[email protected]>Cc Paul Cercueil <[email protected]>Cc Pengutronix Kernel Team <[email protected]>Cc Philippe Cornu <[email protected]>Cc Philipp Zabel <[email protected]>Cc Qiang Yu <[email protected]>Cc Rob Clark <[email protected]>Cc Robert Foss <[email protected]>Cc Rob Herring <[email protected]>Cc Rodrigo Siqueira <[email protected]>Cc Rodrigo Vivi <[email protected]>Cc Roland Scheidegger <[email protected]>Cc Russell King <[email protected]>Cc Sam Ravnborg <[email protected]>Cc Sandy Huang <[email protected]>Cc Sascha Hauer <[email protected]>Cc Sean Paul <[email protected]>Cc Seung-Woo Kim <[email protected]>Cc Shawn Guo <[email protected]>Cc Stefan Agner <[email protected]>Cc Steven Price <[email protected]>Cc Sumit Semwal <[email protected]>Cc Thierry Reding <[email protected]>Cc Tian Tao <[email protected]>Cc Tomeu Vizoso <[email protected]>Cc Tomi Valkeinen <[email protected]>Cc VMware Graphics <[email protected]>Cc Xinliang Liu <[email protected]>Cc Xinwei Kong <[email protected]>Cc Yannick Fertre <[email protected]>Cc Zack Rusin <[email protected]>Reviewed-by Daniel Vetter <[email protected]>Signed-off-by Maxime Ripard <[email protected]>
Quoted 6 lines
---
Changes from v2:
- Take into account the feedback from Laurent and Lidiu to no longer force generic properties, but prefix vendor-specific properties with the vendor name
I'm pretty sure my r-b was without this ... Why exactly do we need this? KMS is meant to be fairly generic (bugs throw a wrench around here sometimes, and semantics can be tricky). If we open up the door to yolo vendor properties in upstream, then that goal is pretty much written off. And we've been there with vendor properties, it's a giantic mess.
Minimally drop my r-b, I'm definitely not in support of this idea.
If there's a strong consensus that we really need this then I'm not going to nack this, but this really needs a pile of acks from compositor folks that they're willing to live with the resulting fallout this will likely bring. Your cc list seems to have an absence of compositor folks, but instead every driver maintainer. That's backwards. We make uapi for userspace, not for kernel driver maintainers!
ltdr; I'd go back to v2. And then cc compositor folks on this to get their ack. -Daniel
Quoted 47 lines
Changes from v1:
- Typos and wording reported by Daniel and Alex--- Documentation/gpu/drm-kms.rst | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)Documentation/gpu/drm-kms.rst +27
-1@@ -463,6 +463,33 @@ KMS Properties This section of the documentation is primarily aimed at user-space developers. For the driver APIs, see the other sections. +Requirements +------------ + +KMS drivers might need to add extra properties to support new features. +Each new property introduced in a driver need to meet a few +requirements, in addition to the one mentioned above.: + +- Before the introduction of any vendor-specific properties, they must + be first checked against the generic ones to avoid any conflict or + redundancy. + +- Vendor-specific properties must be prefixed by the vendor's name, + following the syntax "$vendor:$property". + +- Generic properties must be standardized, with some documentation to + describe how the property can be used. + +- Generic properties must provide a generic helper in the core code to + register that property on the object it attaches to. + +- Generic properties content must be decoded by the core and provided in + the object's associated state structure. That includes anything + drivers might want to precompute, like :c:type:`struct drm_clip_rect + <drm_clip_rect>` for planes. + +- An IGT test should be submitted. + Property Types and Blob Property Support ---------------------------------------- --2.31.1
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Hi,
Quoted 100 lines · Daniel Vetter
Quoted 99 lines · Maxime Ripard
New KMS properties come with a bunch of requirements to avoid each driver from running their own, inconsistent, set of properties, eventually leading to issues like property conflicts, inconsistencies between drivers and semantics, etc.
Let's document what we expect.
Cc Alexandre Belloni <[email protected]>Cc Alexandre Torgue <[email protected]>Cc Alex Deucher <[email protected]>Cc Alison Wang <[email protected]>Cc Alyssa Rosenzweig <[email protected]>Cc Andrew Jeffery <[email protected]>Cc Andrzej Hajda <[email protected]>Cc Anitha Chrisanthus <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Ben Skeggs <[email protected]>Cc Boris Brezillon <[email protected]>Cc Brian Starkey <[email protected]>Cc Chen Feng <[email protected]>Cc Chen-Yu Tsai <[email protected]>Cc Christian Gmeiner <[email protected]>Cc "Christian König" <[email protected]>Cc Chun-Kuang Hu <[email protected]>Cc Edmund Dea <[email protected]>Cc Eric Anholt <[email protected]>Cc Fabio Estevam <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Haneen Mohammed <[email protected]>Cc Hans de Goede <[email protected]>Cc "Heiko Stübner" <[email protected]>Cc Huang Rui <[email protected]>Cc Hyun Kwon <[email protected]>Cc Inki Dae <[email protected]>Cc Jani Nikula <[email protected]>Cc Jernej Skrabec <[email protected]>Cc Jerome Brunet <[email protected]>Cc Joel Stanley <[email protected]>Cc John Stultz <[email protected]>Cc Jonas Karlman <[email protected]>Cc Jonathan Hunter <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Joonyoung Shim <[email protected]>Cc Jyri Sarha <[email protected]>Cc Kevin Hilman <[email protected]>Cc Kieran Bingham <[email protected]>Cc Krzysztof Kozlowski <[email protected]>Cc Kyungmin Park <[email protected]>Cc Laurent Pinchart <[email protected]>Cc Linus Walleij <[email protected]>Cc Liviu Dudau <[email protected]>Cc Lucas Stach <[email protected]>Cc Ludovic Desroches <[email protected]>Cc Marek Vasut <[email protected]>Cc Martin Blumenstingl <[email protected]>Cc Matthias Brugger <[email protected]>Cc Maxime Coquelin <[email protected]>Cc Maxime Ripard <[email protected]>Cc Melissa Wen <[email protected]>Cc Neil Armstrong <[email protected]>Cc Nicolas Ferre <[email protected]>Cc "Noralf Trønnes" <[email protected]>Cc NXP Linux Team <[email protected]>Cc Oleksandr Andrushchenko <[email protected]>Cc Patrik Jakobsson <[email protected]>Cc Paul Cercueil <[email protected]>Cc Pengutronix Kernel Team <[email protected]>Cc Philippe Cornu <[email protected]>Cc Philipp Zabel <[email protected]>Cc Qiang Yu <[email protected]>Cc Rob Clark <[email protected]>Cc Robert Foss <[email protected]>Cc Rob Herring <[email protected]>Cc Rodrigo Siqueira <[email protected]>Cc Rodrigo Vivi <[email protected]>Cc Roland Scheidegger <[email protected]>Cc Russell King <[email protected]>Cc Sam Ravnborg <[email protected]>Cc Sandy Huang <[email protected]>Cc Sascha Hauer <[email protected]>Cc Sean Paul <[email protected]>Cc Seung-Woo Kim <[email protected]>Cc Shawn Guo <[email protected]>Cc Stefan Agner <[email protected]>Cc Steven Price <[email protected]>Cc Sumit Semwal <[email protected]>Cc Thierry Reding <[email protected]>Cc Tian Tao <[email protected]>Cc Tomeu Vizoso <[email protected]>Cc Tomi Valkeinen <[email protected]>Cc VMware Graphics <[email protected]>Cc Xinliang Liu <[email protected]>Cc Xinwei Kong <[email protected]>Cc Yannick Fertre <[email protected]>Cc Zack Rusin <[email protected]>Reviewed-by Daniel Vetter <[email protected]>Signed-off-by Maxime Ripard <[email protected]>
Quoted 8 lines
Quoted 6 lines
---
Changes from v2:
- Take into account the feedback from Laurent and Lidiu to no longer force generic properties, but prefix vendor-specific properties with the vendor nameI'm pretty sure my r-b was without this ...
Yeah, sorry. I wanted to tell you on IRC that you wanted to have a second look, but I shouldn't have kept it and caught you by surprise indeed.
Quoted 7 lines
Why exactly do we need this? KMS is meant to be fairly generic (bugs throw a wrench around here sometimes, and semantics can be tricky). If we open up the door to yolo vendor properties in upstream, then that goal is pretty much written off. And we've been there with vendor properties, it's a giantic mess.
Minimally drop my r-b, I'm definitely not in support of this idea.
So the argument Lidiu and Laurent made was that in some cases, getting a generic property right with only a couple of users is hard. So they advocated for the right to keep non-generic properties. I can get the argument, and no-one else said that was wrong, so it felt like the consensus was there.
Quoted 7 lines
If there's a strong consensus that we really need this then I'm not going to nack this, but this really needs a pile of acks from compositor folks that they're willing to live with the resulting fallout this will likely bring. Your cc list seems to have an absence of compositor folks, but instead every driver maintainer. That's backwards. We make uapi for userspace, not for kernel driver maintainers!
Right, but it's mostly about in-kernel rules though? And you're the one who mentionned CC'ing the driver maintainers in the first iteration?
Quoted 2 lines
ltdr; I'd go back to v2. And then cc compositor folks on this to get their ack.
So, Pekka, Simon, is there anyone else I should Cc?
Thanks!
Maxime
Quoted 122 lines · Maxime Ripard
Hi,
Quoted 100 lines · Daniel Vetter
Quoted 99 lines · Maxime Ripard
New KMS properties come with a bunch of requirements to avoid each driver from running their own, inconsistent, set of properties, eventually leading to issues like property conflicts, inconsistencies between drivers and semantics, etc.
Let's document what we expect.
Cc Alexandre Belloni <[email protected]>Cc Alexandre Torgue <[email protected]>Cc Alex Deucher <[email protected]>Cc Alison Wang <[email protected]>Cc Alyssa Rosenzweig <[email protected]>Cc Andrew Jeffery <[email protected]>Cc Andrzej Hajda <[email protected]>Cc Anitha Chrisanthus <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Ben Skeggs <[email protected]>Cc Boris Brezillon <[email protected]>Cc Brian Starkey <[email protected]>Cc Chen Feng <[email protected]>Cc Chen-Yu Tsai <[email protected]>Cc Christian Gmeiner <[email protected]>Cc "Christian König" <[email protected]>Cc Chun-Kuang Hu <[email protected]>Cc Edmund Dea <[email protected]>Cc Eric Anholt <[email protected]>Cc Fabio Estevam <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Haneen Mohammed <[email protected]>Cc Hans de Goede <[email protected]>Cc "Heiko Stübner" <[email protected]>Cc Huang Rui <[email protected]>Cc Hyun Kwon <[email protected]>Cc Inki Dae <[email protected]>Cc Jani Nikula <[email protected]>Cc Jernej Skrabec <[email protected]>Cc Jerome Brunet <[email protected]>Cc Joel Stanley <[email protected]>Cc John Stultz <[email protected]>Cc Jonas Karlman <[email protected]>Cc Jonathan Hunter <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Joonyoung Shim <[email protected]>Cc Jyri Sarha <[email protected]>Cc Kevin Hilman <[email protected]>Cc Kieran Bingham <[email protected]>Cc Krzysztof Kozlowski <[email protected]>Cc Kyungmin Park <[email protected]>Cc Laurent Pinchart <[email protected]>Cc Linus Walleij <[email protected]>Cc Liviu Dudau <[email protected]>Cc Lucas Stach <[email protected]>Cc Ludovic Desroches <[email protected]>Cc Marek Vasut <[email protected]>Cc Martin Blumenstingl <[email protected]>Cc Matthias Brugger <[email protected]>Cc Maxime Coquelin <[email protected]>Cc Maxime Ripard <[email protected]>Cc Melissa Wen <[email protected]>Cc Neil Armstrong <[email protected]>Cc Nicolas Ferre <[email protected]>Cc "Noralf Trønnes" <[email protected]>Cc NXP Linux Team <[email protected]>Cc Oleksandr Andrushchenko <[email protected]>Cc Patrik Jakobsson <[email protected]>Cc Paul Cercueil <[email protected]>Cc Pengutronix Kernel Team <[email protected]>Cc Philippe Cornu <[email protected]>Cc Philipp Zabel <[email protected]>Cc Qiang Yu <[email protected]>Cc Rob Clark <[email protected]>Cc Robert Foss <[email protected]>Cc Rob Herring <[email protected]>Cc Rodrigo Siqueira <[email protected]>Cc Rodrigo Vivi <[email protected]>Cc Roland Scheidegger <[email protected]>Cc Russell King <[email protected]>Cc Sam Ravnborg <[email protected]>Cc Sandy Huang <[email protected]>Cc Sascha Hauer <[email protected]>Cc Sean Paul <[email protected]>Cc Seung-Woo Kim <[email protected]>Cc Shawn Guo <[email protected]>Cc Stefan Agner <[email protected]>Cc Steven Price <[email protected]>Cc Sumit Semwal <[email protected]>Cc Thierry Reding <[email protected]>Cc Tian Tao <[email protected]>Cc Tomeu Vizoso <[email protected]>Cc Tomi Valkeinen <[email protected]>Cc VMware Graphics <[email protected]>Cc Xinliang Liu <[email protected]>Cc Xinwei Kong <[email protected]>Cc Yannick Fertre <[email protected]>Cc Zack Rusin <[email protected]>Reviewed-by Daniel Vetter <[email protected]>Signed-off-by Maxime Ripard <[email protected]>Quoted 8 lines
Quoted 6 lines
---
Changes from v2:
- Take into account the feedback from Laurent and Lidiu to no longer force generic properties, but prefix vendor-specific properties with the vendor nameI'm pretty sure my r-b was without this ...
Yeah, sorry. I wanted to tell you on IRC that you wanted to have a second look, but I shouldn't have kept it and caught you by surprise indeed.
Quoted 6 lines
Why exactly do we need this? KMS is meant to be fairly generic (bugs throw a wrench around here sometimes, and semantics can be tricky). If we open up the door to yolo vendor properties in upstream, then that goal is pretty much written off. And we've been there with vendor properties, it's a giantic mess.
Quoted 7 lines
Quoted 1 line
Minimally drop my r-b, I'm definitely not in support of this idea.
So the argument Lidiu and Laurent made was that in some cases, getting a generic property right with only a couple of users is hard. So they advocated for the right to keep non-generic properties. I can get the argument, and no-one else said that was wrong, so it felt like the consensus was there.
I also think that (maybe mainly on embedded side) we may have 1) esoteric HW features which perhaps can't even be made generic, and 2) features which may or may not be generic, but for which support cannot be added to any common opensource userspace projects like X or Weston, as the only use cases for the features are specialized low level apps (often customer's closed-source apps).
While I agree with Daniel's "gigantic mess" problem, it would also be quite nice to have a way to support all the HW features upstream instead of carrying them in vendor trees.
Tomi
Quoted 138 lines · Tomi Valkeinen
Quoted 122 lines · Maxime Ripard
Hi,
Quoted 100 lines · Daniel Vetter
Quoted 99 lines · Maxime Ripard
New KMS properties come with a bunch of requirements to avoid each driver from running their own, inconsistent, set of properties, eventually leading to issues like property conflicts, inconsistencies between drivers and semantics, etc.
Let's document what we expect.
Cc Alexandre Belloni <[email protected]>Cc Alexandre Torgue <[email protected]>Cc Alex Deucher <[email protected]>Cc Alison Wang <[email protected]>Cc Alyssa Rosenzweig <[email protected]>Cc Andrew Jeffery <[email protected]>Cc Andrzej Hajda <[email protected]>Cc Anitha Chrisanthus <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Ben Skeggs <[email protected]>Cc Boris Brezillon <[email protected]>Cc Brian Starkey <[email protected]>Cc Chen Feng <[email protected]>Cc Chen-Yu Tsai <[email protected]>Cc Christian Gmeiner <[email protected]>Cc "Christian König" <[email protected]>Cc Chun-Kuang Hu <[email protected]>Cc Edmund Dea <[email protected]>Cc Eric Anholt <[email protected]>Cc Fabio Estevam <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Haneen Mohammed <[email protected]>Cc Hans de Goede <[email protected]>Cc "Heiko Stübner" <[email protected]>Cc Huang Rui <[email protected]>Cc Hyun Kwon <[email protected]>Cc Inki Dae <[email protected]>Cc Jani Nikula <[email protected]>Cc Jernej Skrabec <[email protected]>Cc Jerome Brunet <[email protected]>Cc Joel Stanley <[email protected]>Cc John Stultz <[email protected]>Cc Jonas Karlman <[email protected]>Cc Jonathan Hunter <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Joonyoung Shim <[email protected]>Cc Jyri Sarha <[email protected]>Cc Kevin Hilman <[email protected]>Cc Kieran Bingham <[email protected]>Cc Krzysztof Kozlowski <[email protected]>Cc Kyungmin Park <[email protected]>Cc Laurent Pinchart <[email protected]>Cc Linus Walleij <[email protected]>Cc Liviu Dudau <[email protected]>Cc Lucas Stach <[email protected]>Cc Ludovic Desroches <[email protected]>Cc Marek Vasut <[email protected]>Cc Martin Blumenstingl <[email protected]>Cc Matthias Brugger <[email protected]>Cc Maxime Coquelin <[email protected]>Cc Maxime Ripard <[email protected]>Cc Melissa Wen <[email protected]>Cc Neil Armstrong <[email protected]>Cc Nicolas Ferre <[email protected]>Cc "Noralf Trønnes" <[email protected]>Cc NXP Linux Team <[email protected]>Cc Oleksandr Andrushchenko <[email protected]>Cc Patrik Jakobsson <[email protected]>Cc Paul Cercueil <[email protected]>Cc Pengutronix Kernel Team <[email protected]>Cc Philippe Cornu <[email protected]>Cc Philipp Zabel <[email protected]>Cc Qiang Yu <[email protected]>Cc Rob Clark <[email protected]>Cc Robert Foss <[email protected]>Cc Rob Herring <[email protected]>Cc Rodrigo Siqueira <[email protected]>Cc Rodrigo Vivi <[email protected]>Cc Roland Scheidegger <[email protected]>Cc Russell King <[email protected]>Cc Sam Ravnborg <[email protected]>Cc Sandy Huang <[email protected]>Cc Sascha Hauer <[email protected]>Cc Sean Paul <[email protected]>Cc Seung-Woo Kim <[email protected]>Cc Shawn Guo <[email protected]>Cc Stefan Agner <[email protected]>Cc Steven Price <[email protected]>Cc Sumit Semwal <[email protected]>Cc Thierry Reding <[email protected]>Cc Tian Tao <[email protected]>Cc Tomeu Vizoso <[email protected]>Cc Tomi Valkeinen <[email protected]>Cc VMware Graphics <[email protected]>Cc Xinliang Liu <[email protected]>Cc Xinwei Kong <[email protected]>Cc Yannick Fertre <[email protected]>Cc Zack Rusin <[email protected]>Reviewed-by Daniel Vetter <[email protected]>Signed-off-by Maxime Ripard <[email protected]>Quoted 8 lines
Quoted 6 lines
---
Changes from v2:
- Take into account the feedback from Laurent and Lidiu to no longer force generic properties, but prefix vendor-specific properties with the vendor nameI'm pretty sure my r-b was without this ...
Yeah, sorry. I wanted to tell you on IRC that you wanted to have a second look, but I shouldn't have kept it and caught you by surprise indeed.
Quoted 6 lines
Why exactly do we need this? KMS is meant to be fairly generic (bugs throw a wrench around here sometimes, and semantics can be tricky). If we open up the door to yolo vendor properties in upstream, then that goal is pretty much written off. And we've been there with vendor properties, it's a giantic mess.
Quoted 7 lines
Quoted 1 line
Minimally drop my r-b, I'm definitely not in support of this idea.
So the argument Lidiu and Laurent made was that in some cases, getting a generic property right with only a couple of users is hard. So they advocated for the right to keep non-generic properties. I can get the argument, and no-one else said that was wrong, so it felt like the consensus was there.
I also think that (maybe mainly on embedded side) we may have 1) esoteric HW features which perhaps can't even be made generic, and 2) features which may or may not be generic, but for which support cannot be added to any common opensource userspace projects like X or Weston, as the only use cases for the features are specialized low level apps (often customer's closed-source apps).
Quoted 3 lines
While I agree with Daniel's "gigantic mess" problem, it would also be quite nice to have a way to support all the HW features upstream instead of carrying them in vendor trees.
So this means to be able to accomodate this "vendor properties are totally fine, go wild" exception we also need to throw "open source userspace user, fully reviewed and tested" into the drink?
At least my experience from what I've seen with funky vendor properties isn't so much that we can't figure out a reasonable way to expose them. The problem is that the userspace tends to be a (often closed source) vendor fork of a random compositor somewhere. So if that requirement is somehow a problem, we need to talk about _that_.
Not promising we'll totally merge some vendor properties without spelling out what exactly this means. All that ensures is that people submit patches and then get annoyed because they still can't be merged because the userspace situation is all the same.
-Daniel
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Quoted 11 lines · On Fri, 11 Jun 2021 07:54:07 +0200 Maxime Ripard <[email protected]> wrote:
Hi,
Quoted 8 lines · Daniel Vetter
Quoted 7 lines · Maxime Ripard
New KMS properties come with a bunch of requirements to avoid each driver from running their own, inconsistent, set of properties, eventually leading to issues like property conflicts, inconsistencies between drivers and semantics, etc.
Let's document what we expect.
Sorry for intentionally losing the whole CC list, but I'm not comfortable CC'ing such a huge list (of lists) with just this.
Quoted 4 lines
Quoted 2 lines
ltdr; I'd go back to v2. And then cc compositor folks on this to get their ack.
So, Pekka, Simon, is there anyone else I should Cc?
You can find some Wayland compositor contacts in the following. These are not all compositor projects listed there though. They also signed up for wayland-protocols rather than DRM UAPI discussions, but maybe taking a few won't mind:
https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/main/MEMBERS.md
Thanks,
pq
Quoted 120 lines · Maxime Ripard
Quoted 100 lines · Daniel Vetter
Quoted 99 lines · Maxime Ripard
New KMS properties come with a bunch of requirements to avoid each driver from running their own, inconsistent, set of properties, eventually leading to issues like property conflicts, inconsistencies between drivers and semantics, etc.
Let's document what we expect.
Cc Alexandre Belloni <[email protected]>Cc Alexandre Torgue <[email protected]>Cc Alex Deucher <[email protected]>Cc Alison Wang <[email protected]>Cc Alyssa Rosenzweig <[email protected]>Cc Andrew Jeffery <[email protected]>Cc Andrzej Hajda <[email protected]>Cc Anitha Chrisanthus <[email protected]>Cc Benjamin Gaignard <[email protected]>Cc Ben Skeggs <[email protected]>Cc Boris Brezillon <[email protected]>Cc Brian Starkey <[email protected]>Cc Chen Feng <[email protected]>Cc Chen-Yu Tsai <[email protected]>Cc Christian Gmeiner <[email protected]>Cc "Christian König" <[email protected]>Cc Chun-Kuang Hu <[email protected]>Cc Edmund Dea <[email protected]>Cc Eric Anholt <[email protected]>Cc Fabio Estevam <[email protected]>Cc Gerd Hoffmann <[email protected]>Cc Haneen Mohammed <[email protected]>Cc Hans de Goede <[email protected]>Cc "Heiko Stübner" <[email protected]>Cc Huang Rui <[email protected]>Cc Hyun Kwon <[email protected]>Cc Inki Dae <[email protected]>Cc Jani Nikula <[email protected]>Cc Jernej Skrabec <[email protected]>Cc Jerome Brunet <[email protected]>Cc Joel Stanley <[email protected]>Cc John Stultz <[email protected]>Cc Jonas Karlman <[email protected]>Cc Jonathan Hunter <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Joonyoung Shim <[email protected]>Cc Jyri Sarha <[email protected]>Cc Kevin Hilman <[email protected]>Cc Kieran Bingham <[email protected]>Cc Krzysztof Kozlowski <[email protected]>Cc Kyungmin Park <[email protected]>Cc Laurent Pinchart <[email protected]>Cc Linus Walleij <[email protected]>Cc Liviu Dudau <[email protected]>Cc Lucas Stach <[email protected]>Cc Ludovic Desroches <[email protected]>Cc Marek Vasut <[email protected]>Cc Martin Blumenstingl <[email protected]>Cc Matthias Brugger <[email protected]>Cc Maxime Coquelin <[email protected]>Cc Maxime Ripard <[email protected]>Cc Melissa Wen <[email protected]>Cc Neil Armstrong <[email protected]>Cc Nicolas Ferre <[email protected]>Cc "Noralf Trønnes" <[email protected]>Cc NXP Linux Team <[email protected]>Cc Oleksandr Andrushchenko <[email protected]>Cc Patrik Jakobsson <[email protected]>Cc Paul Cercueil <[email protected]>Cc Pengutronix Kernel Team <[email protected]>Cc Philippe Cornu <[email protected]>Cc Philipp Zabel <[email protected]>Cc Qiang Yu <[email protected]>Cc Rob Clark <[email protected]>Cc Robert Foss <[email protected]>Cc Rob Herring <[email protected]>Cc Rodrigo Siqueira <[email protected]>Cc Rodrigo Vivi <[email protected]>Cc Roland Scheidegger <[email protected]>Cc Russell King <[email protected]>Cc Sam Ravnborg <[email protected]>Cc Sandy Huang <[email protected]>Cc Sascha Hauer <[email protected]>Cc Sean Paul <[email protected]>Cc Seung-Woo Kim <[email protected]>Cc Shawn Guo <[email protected]>Cc Stefan Agner <[email protected]>Cc Steven Price <[email protected]>Cc Sumit Semwal <[email protected]>Cc Thierry Reding <[email protected]>Cc Tian Tao <[email protected]>Cc Tomeu Vizoso <[email protected]>Cc Tomi Valkeinen <[email protected]>Cc VMware Graphics <[email protected]>Cc Xinliang Liu <[email protected]>Cc Xinwei Kong <[email protected]>Cc Yannick Fertre <[email protected]>Cc Zack Rusin <[email protected]>Reviewed-by Daniel Vetter <[email protected]>Signed-off-by Maxime Ripard <[email protected]>Quoted 8 lines
Quoted 6 lines
---
Changes from v2:
- Take into account the feedback from Laurent and Lidiu to no longer force generic properties, but prefix vendor-specific properties with the vendor nameI'm pretty sure my r-b was without this ...
Yeah, sorry. I wanted to tell you on IRC that you wanted to have a second look, but I shouldn't have kept it and caught you by surprise indeed.
Quoted 6 lines
Why exactly do we need this? KMS is meant to be fairly generic (bugs throw a wrench around here sometimes, and semantics can be tricky). If we open up the door to yolo vendor properties in upstream, then that goal is pretty much written off. And we've been there with vendor properties, it's a giantic mess.
Quoted 7 lines
Quoted 1 line
Minimally drop my r-b, I'm definitely not in support of this idea.
So the argument Lidiu and Laurent made was that in some cases, getting a generic property right with only a couple of users is hard. So they advocated for the right to keep non-generic properties. I can get the argument, and no-one else said that was wrong, so it felt like the consensus was there.
My argument was two-fold. On one hand, there's the issue of standardizing properties when we have a single example. I don't have a good solution for that chicken-and-egg issue, when we tell vendor they should standardize properties, and when they ask how, we tell them we don't know as it hasn't been done before. The option for allowing a staging playground for vendor properties here isn't something I like, but we'll need something similar one way or another. Perhaps at the cost of not guaranteeing userspace ABI for those properties, and converting them later to something standard (it won't be received well by vendor of course, and will make a push for mainline more difficult to sell, so it's not a great solution).
On the other hand, there are esoteric vendor-specific features for which standardization really doesn't make sense. For those, I think vendor-specific properties are fine, as long as they're properly documented, with a design documentation merged at the same time as the property. A vendor prefix is really just a namespace clash handling tool here.
Quoted 15 lines
Quoted 7 lines
If there's a strong consensus that we really need this then I'm not going to nack this, but this really needs a pile of acks from compositor folks that they're willing to live with the resulting fallout this will likely bring. Your cc list seems to have an absence of compositor folks, but instead every driver maintainer. That's backwards. We make uapi for userspace, not for kernel driver maintainers!
Right, but it's mostly about in-kernel rules though? And you're the one who mentionned CC'ing the driver maintainers in the first iteration?
Quoted 2 lines
ltdr; I'd go back to v2. And then cc compositor folks on this to get their ack.
So, Pekka, Simon, is there anyone else I should Cc?
Signature · 3 lines
Regards, Laurent Pinchart
Quoted 7 lines · Daniel Vetter
If there's a strong consensus that we really need this then I'm not going to nack this, but this really needs a pile of acks from compositor folks that they're willing to live with the resulting fallout this will likely bring. Your cc list seems to have an absence of compositor folks, but instead every driver maintainer. That's backwards. We make uapi for userspace, not for kernel driver maintainers!
In wlroots we have a policy of only allowing standard KMS properties to be used. Any vendor-specific property is going to be less well-defined, less widely useful, potentially have more design issues, potentially overlap in functionality with other vendor-specific properties, likely have some hardware-specific assumptions, etc.
What matters here is discussing with other driver & user-space folks to make sure the new property's design is sound. Designing uAPI is hard.
If kernel folks are struggling with a user-space implementation, they should discuss with user-space folks to see which project would be interested. There's a chance a compositor will be interested in the new property and will just do the user-space part for you, if not we can suggest candidate projects.
tl;dr strong agree with Daniel here.
Quoted 19 lines · Simon Ser
Quoted 7 lines · Daniel Vetter
If there's a strong consensus that we really need this then I'm not going to nack this, but this really needs a pile of acks from compositor folks that they're willing to live with the resulting fallout this will likely bring. Your cc list seems to have an absence of compositor folks, but instead every driver maintainer. That's backwards. We make uapi for userspace, not for kernel driver maintainers!
In wlroots we have a policy of only allowing standard KMS properties to be used. Any vendor-specific property is going to be less well-defined, less widely useful, potentially have more design issues, potentially overlap in functionality with other vendor-specific properties, likely have some hardware-specific assumptions, etc.
What matters here is discussing with other driver & user-space folks to make sure the new property's design is sound. Designing uAPI is hard.
Quoted 7 lines
If kernel folks are struggling with a user-space implementation, they should discuss with user-space folks to see which project would be interested. There's a chance a compositor will be interested in the new property and will just do the user-space part for you, if not we can suggest candidate projects.
tl;dr strong agree with Daniel here.
I think the assumption you and Daniel are making is that the first implementation of a new KMS property can be made standard from day one and that it will work for any late comer driver as is, without having to make changes to its behaviour in a significant way. In my experience that is not the case.
I think we have moved from the times when we were trying to implement in the Linux world features that were available in the hardware but needed a kernel and userspace API. The set of properties that exist in KMS cover a lot of needed functionality and I don't expect to see new properties for stuff that is already supported by hardware.
What I'm expected to see in the future is new functionality that gets implemented by one hardware vendor and the kernel developers trying to enable that for userspace. It could be that the new property is generic, but there is no way of testing that on more than one implementation yet, so I'd say we are generous calling it "standard property". When the second or third hardware vendor comes along and starts supporting that property with their own set of extra requirements, then we can call it "standard". Then comes the effort cost: would it be easier to start with a vendor property that only the vendor needs to support (and can submit patches into the compositors to do so) and when the standard property gets added moves to that, or should we start with a generic property that gets implemented by the compositors (maybe, but then only one vendor supports it) and then later when we actually standardise the property we will have to carry backwards compatibility code in the kernel to handle the old behaviour for old userspace? My proposal to Maxime was for the former option to be reflected in the documentation, but I would like to hear your thoughts.
Best regards,
Liviu
Signature · 8 lines
====================
| I would like to |
| fix the world, |
| but they're not |
| giving me the |
\ source code! /
---------------
¯\_(ツ)_/¯Quoted 7 lines
What I'm expected to see in the future is new functionality that gets implemented by one hardware vendor and the kernel developers trying to enable that for userspace. It could be that the new property is generic, but there is no way of testing that on more than one implementation yet, so I'd say we are generous calling it "standard property". When the second or third hardware vendor comes along and starts supporting that property with their own set of extra requirements, then we can call it "standard". Then comes the effort cost: would it be easier to start with a vendor
Quoted 8 lines
property that only the vendor needs to support (and can submit patches into the compositors to do so) and when the standard property gets added moves to that, or should we start with a generic property that gets implemented by the compositors (maybe, but then only one vendor supports it) and then later when we actually standardise the property we will have to carry backwards compatibility code in the kernel to handle the old behaviour for old userspace? My proposal to Maxime was for the former option to be reflected in the documentation, but I would like to hear your thoughts.
Just my 2c - if the mainline kernel isn't willing to commit to a feature for upstream userspace to use, why does that feature belong in the kernel at all? I don't see much value in exposing hardware for the sake of exposing it when, practically, Linux userspace /can't/ use it as-is.
Might these vendor properties be used on downstream Android userspaces? That's not generally an upstream goal to support.
Quoted 16 lines · Alyssa Rosenzweig
Quoted 7 lines
What I'm expected to see in the future is new functionality that gets implemented by one hardware vendor and the kernel developers trying to enable that for userspace. It could be that the new property is generic, but there is no way of testing that on more than one implementation yet, so I'd say we are generous calling it "standard property". When the second or third hardware vendor comes along and starts supporting that property with their own set of extra requirements, then we can call it "standard". Then comes the effort cost: would it be easier to start with a vendor
Quoted 8 lines
property that only the vendor needs to support (and can submit patches into the compositors to do so) and when the standard property gets added moves to that, or should we start with a generic property that gets implemented by the compositors (maybe, but then only one vendor supports it) and then later when we actually standardise the property we will have to carry backwards compatibility code in the kernel to handle the old behaviour for old userspace? My proposal to Maxime was for the former option to be reflected in the documentation, but I would like to hear your thoughts.
Quoted 7 lines
Just my 2c - if the mainline kernel isn't willing to commit to a feature for upstream userspace to use, why does that feature belong in the kernel at all? I don't see much value in exposing hardware for the sake of exposing it when, practically, Linux userspace /can't/ use it as-is.
Might these vendor properties be used on downstream Android userspaces? That's not generally an upstream goal to support.
I think the assumption is that we are willing to commit to supporting a feature for userspace, just that (I personally) lack the confidence that I will be getting the feature right on the first attempt and using only one vendor hardware. And that supporting potential mistakes I might've made in the first version is harder if the feature was deemed "standard".
I'm talking from my experience with the writeback connector. We almost committed the feature twice before more people chipped in and asked us for changes, but that was lucky.
Best regards,
Liviu
Signature · 8 lines
====================
| I would like to |
| fix the world, |
| but they're not |
| giving me the |
\ source code! /
---------------
¯\_(ツ)_/¯Quoted 22 lines · Liviu Dudau
Quoted 21 lines · Alyssa Rosenzweig
Quoted 7 lines
What I'm expected to see in the future is new functionality that gets implemented by one hardware vendor and the kernel developers trying to enable that for userspace. It could be that the new property is generic, but there is no way of testing that on more than one implementation yet, so I'd say we are generous calling it "standard property". When the second or third hardware vendor comes along and starts supporting that property with their own set of extra requirements, then we can call it "standard". Then comes the effort cost: would it be easier to start with a vendor
Quoted 8 lines
property that only the vendor needs to support (and can submit patches into the compositors to do so) and when the standard property gets added moves to that, or should we start with a generic property that gets implemented by the compositors (maybe, but then only one vendor supports it) and then later when we actually standardise the property we will have to carry backwards compatibility code in the kernel to handle the old behaviour for old userspace? My proposal to Maxime was for the former option to be reflected in the documentation, but I would like to hear your thoughts.
Just my 2c - if the mainline kernel isn't willing to commit to a feature for upstream userspace to use, why does that feature belong in the kernel at all? I don't see much value in exposing hardware for the sake of exposing it when, practically, Linux userspace /can't/ use it as-is.
Quoted 8 lines
Quoted 2 lines
Might these vendor properties be used on downstream Android userspaces? That's not generally an upstream goal to support.
I think the assumption is that we are willing to commit to supporting a feature for userspace, just that (I personally) lack the confidence that I will be getting the feature right on the first attempt and using only one vendor hardware. And that supporting potential mistakes I might've made in the first version is harder if the feature was deemed "standard".
There's also the issue that the first developer to try to upstream a standard property for a new feature often gets ignored as nobody else has experience with that feature, and thus lack personal interest. That's not a pure technical issue, there's a management problem there too. The right solution may ne to figure out a good process to standardize new property without making it so difficult that everybody will give up and only use downstream kernels. I've heard too many times already from vendors that upstream isn't something they can target as the bar to entry is too high, and when I convince them to submit patches to extend APIs, those patches get ignored (I'm not talking about DRM/KMS only here).
Quoted 2 lines
I'm talking from my experience with the writeback connector. We almost committed the feature twice before more people chipped in and asked us for changes, but that was lucky.
Signature · 3 lines
Regards, Laurent Pinchart
Quoted 38 lines · On Fri, 11 Jun 2021 13:03:09 +0100 Liviu Dudau <[email protected]> wrote:
Quoted 19 lines · Simon Ser
Quoted 7 lines · Daniel Vetter
If there's a strong consensus that we really need this then I'm not going to nack this, but this really needs a pile of acks from compositor folks that they're willing to live with the resulting fallout this will likely bring. Your cc list seems to have an absence of compositor folks, but instead every driver maintainer. That's backwards. We make uapi for userspace, not for kernel driver maintainers!
In wlroots we have a policy of only allowing standard KMS properties to be used. Any vendor-specific property is going to be less well-defined, less widely useful, potentially have more design issues, potentially overlap in functionality with other vendor-specific properties, likely have some hardware-specific assumptions, etc.
What matters here is discussing with other driver & user-space folks to make sure the new property's design is sound. Designing uAPI is hard.
Quoted 7 lines
If kernel folks are struggling with a user-space implementation, they should discuss with user-space folks to see which project would be interested. There's a chance a compositor will be interested in the new property and will just do the user-space part for you, if not we can suggest candidate projects.
tl;dr strong agree with Daniel here.
I think the assumption you and Daniel are making is that the first implementation of a new KMS property can be made standard from day one and that it will work for any late comer driver as is, without having to make changes to its behaviour in a significant way. In my experience that is not the case.
I think we have moved from the times when we were trying to implement in the Linux world features that were available in the hardware but needed a kernel and userspace API. The set of properties that exist in KMS cover a lot of needed functionality and I don't expect to see new properties for stuff that is already supported by hardware.
Quoted 7 lines
What I'm expected to see in the future is new functionality that gets implemented by one hardware vendor and the kernel developers trying to enable that for userspace. It could be that the new property is generic, but there is no way of testing that on more than one implementation yet, so I'd say we are generous calling it "standard property". When the second or third hardware vendor comes along and starts supporting that property with their own set of extra requirements, then we can call it "standard".
I agree that is a problem with trying to make generic anything. But it does not mean you should not even try. Maybe trying really hard saves a couple revisions.
What I think should be planned for is revisions. How to add new properties that do the same thing but better, while documenting that a userspace KMS client can use only one revision at a time. You never remove old revisions, unless maybe with a DRM client cap they could disappear from that file description if that is necessary for seeing the new revision.
While designing this, one also needs to take into account that KMS clients need to be able to save and restore properties *they do not understand*. So exposing two revisions of the same feature simultaneously would break save/restore is that's an error.
Quoted 3 lines
Then comes the effort cost: would it be easier to start with a vendor property that only the vendor needs to support (and can submit patches into the compositors to do so) and when the standard property gets added moves to that, or
But you can't move, you can only add? You can't delete the old property in kernel if it was ever released with a kernel and anyone used it. In the same sentence you also imply that there is a user of it, so removing it will break that user. Then you'll have to track the userspace lifetime to figure out which decade you can try removing it.
Quoted 6 lines
should we start with a generic property that gets implemented by the compositors (maybe, but then only one vendor supports it) and then later when we actually standardise the property we will have to carry backwards compatibility code in the kernel to handle the old behaviour for old userspace? My proposal to Maxime was for the former option to be reflected in the documentation, but I would like to hear your thoughts.
You have to carry the backward compatibility in all cases, right?
Userspace OTOH can drop support for older less supported KMS properties while taking advantage of a new revision. Userspace is not required to support old kernels forever.
Here's a wild counter-proposal off a tangent:
How about we make "implemented in and testable with VKMS" the rule, instead of "is generic" for new properties?
VKMS is what compositors (will) use in CI. I would feel hugely less bad about using a property that only one hardware driver ever implements, if also VKMS implements it in a way that compositor CI can observe it working.
I don't expect this proposal to be accepted, but it's food for thought. The major problem for compositor projects is testing as you usually don't have the hardware, IMO. CI tends to not have any hardware.
Thanks,
pq
Quoted 49 lines · Pekka Paalanen
Quoted 38 lines · On Fri, 11 Jun 2021 13:03:09 +0100 Liviu Dudau <[email protected]> wrote:
Quoted 19 lines · Simon Ser
Quoted 7 lines · Daniel Vetter
If there's a strong consensus that we really need this then I'm not going to nack this, but this really needs a pile of acks from compositor folks that they're willing to live with the resulting fallout this will likely bring. Your cc list seems to have an absence of compositor folks, but instead every driver maintainer. That's backwards. We make uapi for userspace, not for kernel driver maintainers!
In wlroots we have a policy of only allowing standard KMS properties to be used. Any vendor-specific property is going to be less well-defined, less widely useful, potentially have more design issues, potentially overlap in functionality with other vendor-specific properties, likely have some hardware-specific assumptions, etc.
What matters here is discussing with other driver & user-space folks to make sure the new property's design is sound. Designing uAPI is hard.
Quoted 7 lines
If kernel folks are struggling with a user-space implementation, they should discuss with user-space folks to see which project would be interested. There's a chance a compositor will be interested in the new property and will just do the user-space part for you, if not we can suggest candidate projects.
tl;dr strong agree with Daniel here.
I think the assumption you and Daniel are making is that the first implementation of a new KMS property can be made standard from day one and that it will work for any late comer driver as is, without having to make changes to its behaviour in a significant way. In my experience that is not the case.
I think we have moved from the times when we were trying to implement in the Linux world features that were available in the hardware but needed a kernel and userspace API. The set of properties that exist in KMS cover a lot of needed functionality and I don't expect to see new properties for stuff that is already supported by hardware.
Quoted 7 lines
What I'm expected to see in the future is new functionality that gets implemented by one hardware vendor and the kernel developers trying to enable that for userspace. It could be that the new property is generic, but there is no way of testing that on more than one implementation yet, so I'd say we are generous calling it "standard property". When the second or third hardware vendor comes along and starts supporting that property with their own set of extra requirements, then we can call it "standard".
Quoted 3 lines
I agree that is a problem with trying to make generic anything. But it does not mean you should not even try. Maybe trying really hard saves a couple revisions.
Agree.
Quoted 8 lines
What I think should be planned for is revisions. How to add new properties that do the same thing but better, while documenting that a userspace KMS client can use only one revision at a time. You never remove old revisions, unless maybe with a DRM client cap they could disappear from that file description if that is necessary for seeing the new revision.
Quoted 4 lines
While designing this, one also needs to take into account that KMS clients need to be able to save and restore properties *they do not understand*. So exposing two revisions of the same feature simultaneously would break save/restore is that's an error.
I quite like the idea of having versions for properties.
Quoted 10 lines
Quoted 4 lines
Then comes the effort cost: would it be easier to start with a vendor property that only the vendor needs to support (and can submit patches into the compositors to do so) and when the standard property gets added moves to that, or
But you can't move, you can only add? You can't delete the old property in kernel if it was ever released with a kernel and anyone used it. In the same sentence you also imply that there is a user of it, so removing it will break that user. Then you'll have to track the userspace lifetime to figure out which decade you can try removing it.
Not that I am supporting the workflow, but I was trying to address the comments that vendors are going to push their own userspace implementation for their vendor properties. If that is the case, when they switch to the standard ones they can drop the support in userspace for their changes. With the implied assumption that you will have fewer vendor implementations hence easier to make changes, KMS properties can be deleted if you know there is no user of them (e.g. the vendor has upgraded all their software to the standard property).
Quoted 20 lines
Quoted 7 lines
should we start with a generic property that gets implemented by the compositors (maybe, but then only one vendor supports it) and then later when we actually standardise the property we will have to carry backwards compatibility code in the kernel to handle the old behaviour for old userspace? My proposal to Maxime was for the former option to be reflected in the documentation, but I would like to hear your thoughts.
You have to carry the backward compatibility in all cases, right?
Userspace OTOH can drop support for older less supported KMS properties while taking advantage of a new revision. Userspace is not required to support old kernels forever.
Here's a wild counter-proposal off a tangent:
How about we make "implemented in and testable with VKMS" the rule, instead of "is generic" for new properties?
Quoted 8 lines
VKMS is what compositors (will) use in CI. I would feel hugely less bad about using a property that only one hardware driver ever implements, if also VKMS implements it in a way that compositor CI can observe it working.
I don't expect this proposal to be accepted, but it's food for thought. The major problem for compositor projects is testing as you usually don't have the hardware, IMO. CI tends to not have any hardware.
While I don't dislike the proposal (I think it is quite sensible), I am worried that for some behaviours VKMS will implement them in a quirky way. To pick (again) the example of writeback, real hardware will have a way to tell if the buffer has been sent successfully to memory and it might take more than one refresh period, while VKMS (if I remember correctly) fakes it and signals the fence at the next vblank. If you code your compositor based on VKMS you might get unexpected artifacts on real hardware.
Best regards,
Liviu
Quoted 4 lines
Thanks,
pq
Signature · 8 lines
====================
| I would like to |
| fix the world, |
| but they're not |
| giving me the |
\ source code! /
---------------
¯\_(ツ)_/¯Quoted 63 lines · Liviu Dudau
Quoted 49 lines · Pekka Paalanen
Quoted 38 lines · On Fri, 11 Jun 2021 13:03:09 +0100 Liviu Dudau <[email protected]> wrote:
Quoted 19 lines · Simon Ser
Quoted 7 lines · Daniel Vetter
If there's a strong consensus that we really need this then I'm not going to nack this, but this really needs a pile of acks from compositor folks that they're willing to live with the resulting fallout this will likely bring. Your cc list seems to have an absence of compositor folks, but instead every driver maintainer. That's backwards. We make uapi for userspace, not for kernel driver maintainers!
In wlroots we have a policy of only allowing standard KMS properties to be used. Any vendor-specific property is going to be less well-defined, less widely useful, potentially have more design issues, potentially overlap in functionality with other vendor-specific properties, likely have some hardware-specific assumptions, etc.
What matters here is discussing with other driver & user-space folks to make sure the new property's design is sound. Designing uAPI is hard.
Quoted 7 lines
If kernel folks are struggling with a user-space implementation, they should discuss with user-space folks to see which project would be interested. There's a chance a compositor will be interested in the new property and will just do the user-space part for you, if not we can suggest candidate projects.
tl;dr strong agree with Daniel here.
I think the assumption you and Daniel are making is that the first implementation of a new KMS property can be made standard from day one and that it will work for any late comer driver as is, without having to make changes to its behaviour in a significant way. In my experience that is not the case.
I think we have moved from the times when we were trying to implement in the Linux world features that were available in the hardware but needed a kernel and userspace API. The set of properties that exist in KMS cover a lot of needed functionality and I don't expect to see new properties for stuff that is already supported by hardware.
Quoted 7 lines
What I'm expected to see in the future is new functionality that gets implemented by one hardware vendor and the kernel developers trying to enable that for userspace. It could be that the new property is generic, but there is no way of testing that on more than one implementation yet, so I'd say we are generous calling it "standard property". When the second or third hardware vendor comes along and starts supporting that property with their own set of extra requirements, then we can call it "standard".
Quoted 3 lines
I agree that is a problem with trying to make generic anything. But it does not mean you should not even try. Maybe trying really hard saves a couple revisions.
Agree.
Quoted 7 lines
What I think should be planned for is revisions. How to add new properties that do the same thing but better, while documenting that a userspace KMS client can use only one revision at a time. You never remove old revisions, unless maybe with a DRM client cap they could disappear from that file description if that is necessary for seeing the new revision.
Quoted 6 lines
Quoted 4 lines
While designing this, one also needs to take into account that KMS clients need to be able to save and restore properties *they do not understand*. So exposing two revisions of the same feature simultaneously would break save/restore is that's an error.
I quite like the idea of having versions for properties.
It's an interesting idea. We'll need to consider bundles of properties in that case I think, as in a v1 a feature could be implemented by properties A, B and C, while in v2 C could be replaced by D and E (beside A and B themselves also being changed in v2).
Quoted 46 lines
Quoted 9 lines
Quoted 3 lines
Then comes the effort cost: would it be easier to start with a vendor property that only the vendor needs to support (and can submit patches into the compositors to do so) and when the standard property gets added moves to that, or
But you can't move, you can only add? You can't delete the old property in kernel if it was ever released with a kernel and anyone used it. In the same sentence you also imply that there is a user of it, so removing it will break that user. Then you'll have to track the userspace lifetime to figure out which decade you can try removing it.
Not that I am supporting the workflow, but I was trying to address the comments that vendors are going to push their own userspace implementation for their vendor properties. If that is the case, when they switch to the standard ones they can drop the support in userspace for their changes. With the implied assumption that you will have fewer vendor implementations hence easier to make changes, KMS properties can be deleted if you know there is no user of them (e.g. the vendor has upgraded all their software to the standard property).
Quoted 19 lines
Quoted 6 lines
should we start with a generic property that gets implemented by the compositors (maybe, but then only one vendor supports it) and then later when we actually standardise the property we will have to carry backwards compatibility code in the kernel to handle the old behaviour for old userspace? My proposal to Maxime was for the former option to be reflected in the documentation, but I would like to hear your thoughts.
You have to carry the backward compatibility in all cases, right?
Userspace OTOH can drop support for older less supported KMS properties while taking advantage of a new revision. Userspace is not required to support old kernels forever.
Here's a wild counter-proposal off a tangent:
How about we make "implemented in and testable with VKMS" the rule, instead of "is generic" for new properties?
Quoted 8 lines
VKMS is what compositors (will) use in CI. I would feel hugely less bad about using a property that only one hardware driver ever implements, if also VKMS implements it in a way that compositor CI can observe it working.
I don't expect this proposal to be accepted, but it's food for thought. The major problem for compositor projects is testing as you usually don't have the hardware, IMO. CI tends to not have any hardware.
Quoted 7 lines
While I don't dislike the proposal (I think it is quite sensible), I am worried that for some behaviours VKMS will implement them in a quirky way. To pick (again) the example of writeback, real hardware will have a way to tell if the buffer has been sent successfully to memory and it might take more than one refresh period, while VKMS (if I remember correctly) fakes it and signals the fence at the next vblank. If you code your compositor based on VKMS you might get unexpected artifacts on real hardware.
I'm also worried about properties that related to hardware processing of frames, for instance related to colour spaces, or even scaling. VKMS has a software composer implemented in the kernel, it may get fairly complicated if we need to implement all kind of processing.
Signature · 3 lines
Regards, Laurent Pinchart
Quoted 130 lines · On Mon, 14 Jun 2021 19:33:47 +0300 Laurent Pinchart…
Quoted 63 lines · Liviu Dudau
Quoted 49 lines · Pekka Paalanen
Quoted 38 lines · On Fri, 11 Jun 2021 13:03:09 +0100 Liviu Dudau <[email protected]> wrote:
Quoted 19 lines · Simon Ser
Quoted 7 lines · Daniel Vetter
If there's a strong consensus that we really need this then I'm not going to nack this, but this really needs a pile of acks from compositor folks that they're willing to live with the resulting fallout this will likely bring. Your cc list seems to have an absence of compositor folks, but instead every driver maintainer. That's backwards. We make uapi for userspace, not for kernel driver maintainers!
In wlroots we have a policy of only allowing standard KMS properties to be used. Any vendor-specific property is going to be less well-defined, less widely useful, potentially have more design issues, potentially overlap in functionality with other vendor-specific properties, likely have some hardware-specific assumptions, etc.
What matters here is discussing with other driver & user-space folks to make sure the new property's design is sound. Designing uAPI is hard.
Quoted 7 lines
If kernel folks are struggling with a user-space implementation, they should discuss with user-space folks to see which project would be interested. There's a chance a compositor will be interested in the new property and will just do the user-space part for you, if not we can suggest candidate projects.
tl;dr strong agree with Daniel here.
I think the assumption you and Daniel are making is that the first implementation of a new KMS property can be made standard from day one and that it will work for any late comer driver as is, without having to make changes to its behaviour in a significant way. In my experience that is not the case.
I think we have moved from the times when we were trying to implement in the Linux world features that were available in the hardware but needed a kernel and userspace API. The set of properties that exist in KMS cover a lot of needed functionality and I don't expect to see new properties for stuff that is already supported by hardware.
Quoted 7 lines
What I'm expected to see in the future is new functionality that gets implemented by one hardware vendor and the kernel developers trying to enable that for userspace. It could be that the new property is generic, but there is no way of testing that on more than one implementation yet, so I'd say we are generous calling it "standard property". When the second or third hardware vendor comes along and starts supporting that property with their own set of extra requirements, then we can call it "standard".
Quoted 3 lines
I agree that is a problem with trying to make generic anything. But it does not mean you should not even try. Maybe trying really hard saves a couple revisions.
Agree.
Quoted 7 lines
What I think should be planned for is revisions. How to add new properties that do the same thing but better, while documenting that a userspace KMS client can use only one revision at a time. You never remove old revisions, unless maybe with a DRM client cap they could disappear from that file description if that is necessary for seeing the new revision.
Quoted 6 lines
Quoted 4 lines
While designing this, one also needs to take into account that KMS clients need to be able to save and restore properties *they do not understand*. So exposing two revisions of the same feature simultaneously would break save/restore is that's an error.
I quite like the idea of having versions for properties.
It's an interesting idea. We'll need to consider bundles of properties in that case I think, as in a v1 a feature could be implemented by properties A, B and C, while in v2 C could be replaced by D and E (beside A and B themselves also being changed in v2).
Quoted 46 lines
Quoted 9 lines
Quoted 3 lines
Then comes the effort cost: would it be easier to start with a vendor property that only the vendor needs to support (and can submit patches into the compositors to do so) and when the standard property gets added moves to that, or
But you can't move, you can only add? You can't delete the old property in kernel if it was ever released with a kernel and anyone used it. In the same sentence you also imply that there is a user of it, so removing it will break that user. Then you'll have to track the userspace lifetime to figure out which decade you can try removing it.
Not that I am supporting the workflow, but I was trying to address the comments that vendors are going to push their own userspace implementation for their vendor properties. If that is the case, when they switch to the standard ones they can drop the support in userspace for their changes. With the implied assumption that you will have fewer vendor implementations hence easier to make changes, KMS properties can be deleted if you know there is no user of them (e.g. the vendor has upgraded all their software to the standard property).
Quoted 19 lines
Quoted 6 lines
should we start with a generic property that gets implemented by the compositors (maybe, but then only one vendor supports it) and then later when we actually standardise the property we will have to carry backwards compatibility code in the kernel to handle the old behaviour for old userspace? My proposal to Maxime was for the former option to be reflected in the documentation, but I would like to hear your thoughts.
You have to carry the backward compatibility in all cases, right?
Userspace OTOH can drop support for older less supported KMS properties while taking advantage of a new revision. Userspace is not required to support old kernels forever.
Here's a wild counter-proposal off a tangent:
How about we make "implemented in and testable with VKMS" the rule, instead of "is generic" for new properties?
Quoted 8 lines
VKMS is what compositors (will) use in CI. I would feel hugely less bad about using a property that only one hardware driver ever implements, if also VKMS implements it in a way that compositor CI can observe it working.
I don't expect this proposal to be accepted, but it's food for thought. The major problem for compositor projects is testing as you usually don't have the hardware, IMO. CI tends to not have any hardware.
Quoted 7 lines
While I don't dislike the proposal (I think it is quite sensible), I am worried that for some behaviours VKMS will implement them in a quirky way. To pick (again) the example of writeback, real hardware will have a way to tell if the buffer has been sent successfully to memory and it might take more than one refresh period, while VKMS (if I remember correctly) fakes it and signals the fence at the next vblank. If you code your compositor based on VKMS you might get unexpected artifacts on real hardware.
Quoted 4 lines
I'm also worried about properties that related to hardware processing of frames, for instance related to colour spaces, or even scaling. VKMS has a software composer implemented in the kernel, it may get fairly complicated if we need to implement all kind of processing.
Hi,
indeed it will, but what else could one do to test userspace KMS clients in generic CI where all you can have is virtual hardware? Maybe in the long run VKMS needs to loop back to a userspace daemon that implements all the complex processing and returns the writeback result via VKMS again? That daemon would then need a single upstream, like the kernel, where it is maintained and correctness verified.
Or an LD_PRELOAD that hijacks all KMS ioctls and implements virtual stuff in userspace? Didn't someone already have something like that? It would need to be lifted to be a required part of kernel UAPI submissions, I suppose like IGT is nowadays.
Requiring a software implementation for testing has another huge benefit (or disadvantage, depending on who you ask): "hardware does secret magic" would not fly anymore. The software implementation would need to deliver the same results, or at least one would need to define the tolerances for e.g. color conversions. Otherwise userspace compositor test suites might pass on the software implementation and fail on hardware implementations. For compositor developers like me knowing the exact formulas would be a huge benefit as it would allow me to use KMS to off-load precision-sensitive operations (e.g. professional color management). Otherwise, compositors probably need a
switch: "high quality color management? Then do not use KMS features."
Thanks,
pq
Quoted 6 lines · Pekka Paalanen
indeed it will, but what else could one do to test userspace KMS clients in generic CI where all you can have is virtual hardware? Maybe in the long run VKMS needs to loop back to a userspace daemon that implements all the complex processing and returns the writeback result via VKMS again? That daemon would then need a single upstream, like the kernel, where it is maintained and correctness verified.
The complex processing must be implemented even without write-back, because user-space can ask for CRCs of the CRTC.
Quoted 4 lines
Or an
LD_PRELOADthat hijacks all KMS ioctls and implements virtual stuff in userspace? Didn't someone already have something like that? It would need to be lifted to be a required part of kernel UAPI submissions, I suppose like IGT is nowadays.
FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full software implementation with write-back connectors, but allows to expose virtual planes and check atomic commits in CI.
[1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
Quoted 5 lines
For compositor developers like me knowing the exact formulas would be a huge benefit as it would allow me to use KMS to off-load precision-sensitive operations (e.g. professional color management). Otherwise, compositors probably need a switch: "high quality color management? Then do not use KMS features."
I think for alpha blending there are already rounding issues depending on the hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the exact same formulae for color management stuff.
Quoted 29 lines · Simon Ser
Quoted 6 lines · Pekka Paalanen
indeed it will, but what else could one do to test userspace KMS clients in generic CI where all you can have is virtual hardware? Maybe in the long run VKMS needs to loop back to a userspace daemon that implements all the complex processing and returns the writeback result via VKMS again? That daemon would then need a single upstream, like the kernel, where it is maintained and correctness verified.
The complex processing must be implemented even without write-back, because user-space can ask for CRCs of the CRTC.
Quoted 4 lines
Or an
LD_PRELOADthat hijacks all KMS ioctls and implements virtual stuff in userspace? Didn't someone already have something like that? It would need to be lifted to be a required part of kernel UAPI submissions, I suppose like IGT is nowadays.FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full software implementation with write-back connectors, but allows to expose virtual planes and check atomic commits in CI.
[1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
Quoted 5 lines
For compositor developers like me knowing the exact formulas would be a huge benefit as it would allow me to use KMS to off-load precision-sensitive operations (e.g. professional color management). Otherwise, compositors probably need a switch: "high quality color management? Then do not use KMS features."
Quoted 3 lines
I think for alpha blending there are already rounding issues depending on the hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the exact same formulae for color management stuff.
Good, because otherwise you would be very quickly disappointed :-)
For scaling we would also need to replicate the exact same filter taps, which are often not documented.
Signature · 3 lines
Regards, Laurent Pinchart
Quoted 30 lines · On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart…
Quoted 29 lines · Simon Ser
Quoted 6 lines · Pekka Paalanen
indeed it will, but what else could one do to test userspace KMS clients in generic CI where all you can have is virtual hardware? Maybe in the long run VKMS needs to loop back to a userspace daemon that implements all the complex processing and returns the writeback result via VKMS again? That daemon would then need a single upstream, like the kernel, where it is maintained and correctness verified.
The complex processing must be implemented even without write-back, because user-space can ask for CRCs of the CRTC.
Quoted 4 lines
Or an
LD_PRELOADthat hijacks all KMS ioctls and implements virtual stuff in userspace? Didn't someone already have something like that? It would need to be lifted to be a required part of kernel UAPI submissions, I suppose like IGT is nowadays.FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full software implementation with write-back connectors, but allows to expose virtual planes and check atomic commits in CI.
[1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
Quoted 5 lines
For compositor developers like me knowing the exact formulas would be a huge benefit as it would allow me to use KMS to off-load precision-sensitive operations (e.g. professional color management). Otherwise, compositors probably need a switch: "high quality color management? Then do not use KMS features."
Quoted 8 lines
Quoted 3 lines
I think for alpha blending there are already rounding issues depending on the hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the exact same formulae for color management stuff.
Good, because otherwise you would be very quickly disappointed :-)
For scaling we would also need to replicate the exact same filter taps, which are often not documented.
That is where the documented tolerances come into play.
Userspace projects need screenshot-based testing, and we need to know how much tolerance we should allow or expect.
Good reminder about CRCs. CRCs have zero tolerance, so they are not useful for testing properties that have any leeway, are they?
Thanks,
pq
Quoted 2 lines · Pekka Paalanen
Good reminder about CRCs. CRCs have zero tolerance, so they are not useful for testing properties that have any leeway, are they?
IIRC, IGT's alpha blending test currently computes the CRC for all possible roundings, then checks that the hw returns one of the acceptable CRCs.
With more complex color management properties, this approach might not be possible and write-back support in hw drivers would really help.
Quoted 6 lines · Simon Ser
Quoted 2 lines · Pekka Paalanen
Good reminder about CRCs. CRCs have zero tolerance, so they are not useful for testing properties that have any leeway, are they?
Quoted 6 lines
IIRC, IGT's alpha blending test currently computes the CRC for all possible roundings, then checks that the hw returns one of the acceptable CRCs.
With more complex color management properties, this approach might not be possible and write-back support in hw drivers would really help.
Yeah CRC based tests have severe limits, and even if you only try to test the extreme stuff there's enough busted hw out there that they will fail. E.g. when scaling I'm sure there's hw that bleeds in pixels from outside the bounding box and can't be fixed, or there's some intel hw where the alpha blending gets the mapping between [0, 0xff] <-> [0.0, 1.0] and you get a nice faint ghost instead of full transparency or opaqueness.
I think for those broken hw is just broken, nothing we can do.
Also writeback isn't supported by enough display hw, and not everyone has access to chamelium or similar.
I think best we can do is that relevant igt have an interactive mode (built-in for crc tests, so you can pause and look at the screen) and then perhaps compare with the vkms reference implementation manually. That's not great, and vkms is also not anywhere near close enough for this either (and like currently you can't even look at it, we'd first need a v4l output mode).
-Daniel
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Quoted 35 lines · Pekka Paalanen
Quoted 34 lines · On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:
Quoted 32 lines · Simon Ser
Quoted 6 lines · Pekka Paalanen
indeed it will, but what else could one do to test userspace KMS clients in generic CI where all you can have is virtual hardware? Maybe in the long run VKMS needs to loop back to a userspace daemon that implements all the complex processing and returns the writeback result via VKMS again? That daemon would then need a single upstream, like the kernel, where it is maintained and correctness verified.
The complex processing must be implemented even without write-back, because user-space can ask for CRCs of the CRTC.
Quoted 4 lines
Or an
LD_PRELOADthat hijacks all KMS ioctls and implements virtual stuff in userspace? Didn't someone already have something like that? It would need to be lifted to be a required part of kernel UAPI submissions, I suppose like IGT is nowadays.FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full software implementation with write-back connectors, but allows to expose virtual planes and check atomic commits in CI.
[1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
Quoted 5 lines
For compositor developers like me knowing the exact formulas would be a huge benefit as it would allow me to use KMS to off-load precision-sensitive operations (e.g. professional color management). Otherwise, compositors probably need a switch: "high quality color management? Then do not use KMS features."
I think for alpha blending there are already rounding issues depending on the hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the exact same formulae for color management stuff.
Quoted 6 lines
Quoted 4 lines
Good, because otherwise you would be very quickly disappointed :-)
For scaling we would also need to replicate the exact same filter taps, which are often not documented.
That is where the documented tolerances come into play.
This is something I've experimented with a while ago, when developing automated tests for the rcar-du driver. When playing with different input images we had to constantly increases tolerances, up to a point where the tests started to miss real problems :-(
Quoted 5 lines
Userspace projects need screenshot-based testing, and we need to know how much tolerance we should allow or expect.
Good reminder about CRCs. CRCs have zero tolerance, so they are not useful for testing properties that have any leeway, are they?
Signature · 3 lines
Regards, Laurent Pinchart
Quoted 41 lines · On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart…
Quoted 40 lines · Pekka Paalanen
Quoted 38 lines · On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:
Quoted 29 lines · Simon Ser
Quoted 6 lines · Pekka Paalanen
indeed it will, but what else could one do to test userspace KMS clients in generic CI where all you can have is virtual hardware? Maybe in the long run VKMS needs to loop back to a userspace daemon that implements all the complex processing and returns the writeback result via VKMS again? That daemon would then need a single upstream, like the kernel, where it is maintained and correctness verified.
The complex processing must be implemented even without write-back, because user-space can ask for CRCs of the CRTC.
Quoted 4 lines
Or an
LD_PRELOADthat hijacks all KMS ioctls and implements virtual stuff in userspace? Didn't someone already have something like that? It would need to be lifted to be a required part of kernel UAPI submissions, I suppose like IGT is nowadays.FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full software implementation with write-back connectors, but allows to expose virtual planes and check atomic commits in CI.
[1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
Quoted 5 lines
For compositor developers like me knowing the exact formulas would be a huge benefit as it would allow me to use KMS to off-load precision-sensitive operations (e.g. professional color management). Otherwise, compositors probably need a switch: "high quality color management? Then do not use KMS features."
Quoted 3 lines
I think for alpha blending there are already rounding issues depending on the hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the exact same formulae for color management stuff.
Good, because otherwise you would be very quickly disappointed :-)
For scaling we would also need to replicate the exact same filter taps, which are often not documented.
Quoted 6 lines
Quoted 1 line
That is where the documented tolerances come into play.
This is something I've experimented with a while ago, when developing automated tests for the rcar-du driver. When playing with different input images we had to constantly increases tolerances, up to a point where the tests started to miss real problems :-(
What should we infer from that? That the hardware is broken and exposing those KMS properties is a false promise?
If a driver on certain hardware cannot correctly implement a KMS property over the full domain of the input space, should that driver then simply not expose the KMS property at all?
But I would assume that the vendor still wants to expose the features in upstream kernels, yet they cannot use the standard KMS properties for that. Should the driver then expose vendor-specific properties with the disclaimer that the result is not always what one would expect, so that userspace written and tested explicitly for that hardware can still work?
That is, a sufficient justification for a vendor-specific KMS property would be that a standard property already exists, but the hardware is too buggy to make it work. IOW, give up trying to make sense.
I would like to move towards a direction where *hardware* design and testing is eventually guided by Linux KMS property definitions and their tests. If we could have a rule that if a driver cannot correctly implement a property then it must not expose the property, maybe in the long term that might start having an effect?
My underlying assumption is that generic userspace will not use vendor-specific properties.
Or, since we have atomic commits with TEST_ONLY, should it be driver's responsibility to carefully inspect the full state and reject the commit if the hardware is incapable of implementing it correctly? Vendor-specific userspace would know to avoid failing configurations to begin with. I suppose that might put an endless whack-a-mole game on drivers though.
Thanks,
pq
Hi Pekka,
Quoted 44 lines · Pekka Paalanen
Quoted 43 lines · On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:
Quoted 41 lines · Pekka Paalanen
Quoted 30 lines · On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:
Quoted 29 lines · Simon Ser
Quoted 6 lines · Pekka Paalanen
indeed it will, but what else could one do to test userspace KMS clients in generic CI where all you can have is virtual hardware? Maybe in the long run VKMS needs to loop back to a userspace daemon that implements all the complex processing and returns the writeback result via VKMS again? That daemon would then need a single upstream, like the kernel, where it is maintained and correctness verified.
The complex processing must be implemented even without write-back, because user-space can ask for CRCs of the CRTC.
Quoted 4 lines
Or an
LD_PRELOADthat hijacks all KMS ioctls and implements virtual stuff in userspace? Didn't someone already have something like that? It would need to be lifted to be a required part of kernel UAPI submissions, I suppose like IGT is nowadays.FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full software implementation with write-back connectors, but allows to expose virtual planes and check atomic commits in CI.
[1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
Quoted 5 lines
For compositor developers like me knowing the exact formulas would be a huge benefit as it would allow me to use KMS to off-load precision-sensitive operations (e.g. professional color management). Otherwise, compositors probably need a switch: "high quality color management? Then do not use KMS features."
Quoted 8 lines
Quoted 3 lines
I think for alpha blending there are already rounding issues depending on the hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the exact same formulae for color management stuff.
Good, because otherwise you would be very quickly disappointed :-)
For scaling we would also need to replicate the exact same filter taps, which are often not documented.
That is where the documented tolerances come into play.
Quoted 7 lines
Quoted 4 lines
This is something I've experimented with a while ago, when developing automated tests for the rcar-du driver. When playing with different input images we had to constantly increases tolerances, up to a point where the tests started to miss real problems :-(
What should we infer from that? That the hardware is broken and exposing those KMS properties is a false promise?
No, just that the scaler doesn't document the internal hardware implementation (number of taps in the filters, coefficients, rounding, ...). That's the rule, not the exception, and it doesn't prevent correct operation, images get scaled in a reproducible way (the same input produces the same output).
Quoted 3 lines
If a driver on certain hardware cannot correctly implement a KMS property over the full domain of the input space, should that driver then simply not expose the KMS property at all?
The properties involved here would the the SRC and CRTC rectangles for the planes. They don't document pixel-perfect scaling :-)
Quoted 10 lines
But I would assume that the vendor still wants to expose the features in upstream kernels, yet they cannot use the standard KMS properties for that. Should the driver then expose vendor-specific properties with the disclaimer that the result is not always what one would expect, so that userspace written and tested explicitly for that hardware can still work?
That is, a sufficient justification for a vendor-specific KMS property would be that a standard property already exists, but the hardware is too buggy to make it work. IOW, give up trying to make sense.
It's not just about buggy hardware, it's also about implementation specificities, such as rounding, filters, order of operations in the color management pipeline (it's relatively easy when you only have two LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping features in the mix, not all hardware will implement the same pipeline), or various types of image compression (this device implements a "near-lossless" compression scheme that reduces the memory bandwidth by 50% for instance).
Quoted 5 lines
I would like to move towards a direction where *hardware* design and testing is eventually guided by Linux KMS property definitions and their tests. If we could have a rule that if a driver cannot correctly implement a property then it must not expose the property, maybe in the long term that might start having an effect?
I didn't even know a 3D LUT existed before getting the datasheet for this device. It's a long shot before the community will acquire enough knowledge to guide hardware design, especially if you consider the feature of tomorrow's devices that nobody knows about yet. A worthwhile goal, but not something we can fully achieve, and not in a short time frame.
Quoted 2 lines
My underlying assumption is that generic userspace will not use vendor-specific properties.
I expect some amount of device-specific code in userspace, yes.
There are usually large variations in how the hardware exposes access to a given feature, which leads to code having to convert the standard API parameters to hardware parameters. To a large extend this can be done in drivers, but for some more complex features, it may put too much burden on the kernel. There's a reason mesa is a userspace stack :-)
Quoted 6 lines
Or, since we have atomic commits with
TEST_ONLY, should it be driver's responsibility to carefully inspect the full state and reject the commit if the hardware is incapable of implementing it correctly? Vendor-specific userspace would know to avoid failing configurations to begin with. I suppose that might put an endless whack-a-mole game on drivers though.
Signature · 3 lines
Regards, Laurent Pinchart
Quoted 79 lines · On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart…
Hi Pekka,
Quoted 44 lines · Pekka Paalanen
Quoted 43 lines · On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:
Quoted 41 lines · Pekka Paalanen
Quoted 30 lines · On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:
Quoted 29 lines · Simon Ser
Quoted 6 lines · Pekka Paalanen
indeed it will, but what else could one do to test userspace KMS clients in generic CI where all you can have is virtual hardware? Maybe in the long run VKMS needs to loop back to a userspace daemon that implements all the complex processing and returns the writeback result via VKMS again? That daemon would then need a single upstream, like the kernel, where it is maintained and correctness verified.
The complex processing must be implemented even without write-back, because user-space can ask for CRCs of the CRTC.
Quoted 4 lines
Or an
LD_PRELOADthat hijacks all KMS ioctls and implements virtual stuff in userspace? Didn't someone already have something like that? It would need to be lifted to be a required part of kernel UAPI submissions, I suppose like IGT is nowadays.FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full software implementation with write-back connectors, but allows to expose virtual planes and check atomic commits in CI.
[1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
Quoted 5 lines
For compositor developers like me knowing the exact formulas would be a huge benefit as it would allow me to use KMS to off-load precision-sensitive operations (e.g. professional color management). Otherwise, compositors probably need a switch: "high quality color management? Then do not use KMS features."
Quoted 8 lines
Quoted 3 lines
I think for alpha blending there are already rounding issues depending on the hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the exact same formulae for color management stuff.
Good, because otherwise you would be very quickly disappointed :-)
For scaling we would also need to replicate the exact same filter taps, which are often not documented.
That is where the documented tolerances come into play.
Quoted 7 lines
Quoted 4 lines
This is something I've experimented with a while ago, when developing automated tests for the rcar-du driver. When playing with different input images we had to constantly increases tolerances, up to a point where the tests started to miss real problems :-(
What should we infer from that? That the hardware is broken and exposing those KMS properties is a false promise?
No, just that the scaler doesn't document the internal hardware implementation (number of taps in the filters, coefficients, rounding, ...). That's the rule, not the exception, and it doesn't prevent correct operation, images get scaled in a reproducible way (the same input produces the same output).
Quoted 3 lines
If a driver on certain hardware cannot correctly implement a KMS property over the full domain of the input space, should that driver then simply not expose the KMS property at all?
The properties involved here would the the SRC and CRTC rectangles for the planes. They don't document pixel-perfect scaling :-)
Quoted 10 lines
But I would assume that the vendor still wants to expose the features in upstream kernels, yet they cannot use the standard KMS properties for that. Should the driver then expose vendor-specific properties with the disclaimer that the result is not always what one would expect, so that userspace written and tested explicitly for that hardware can still work?
That is, a sufficient justification for a vendor-specific KMS property would be that a standard property already exists, but the hardware is too buggy to make it work. IOW, give up trying to make sense.
Quoted 8 lines
It's not just about buggy hardware, it's also about implementation specificities, such as rounding, filters, order of operations in the color management pipeline (it's relatively easy when you only have two LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping features in the mix, not all hardware will implement the same pipeline), or various types of image compression (this device implements a "near-lossless" compression scheme that reduces the memory bandwidth by 50% for instance).
Rounding shouldn't result in needing wide tolerances.
Filters are more difficult, but at least we can factor them out when testing other things. Filters could be tested in isolation with some image difference metrics rather than per-pixel independent comparisons.
The order of operations in the color management pipeline is very important. We can't work with "whatever". All the variability in hardware is exactly why I have been calling out for defined abstract color pipeline in the DRM UAPI docs, so that drivers would know which properties to map to their elements, so that userspace can have any possibility of using them correctly. If the hardware has a block that doesn't fit in the abstract pipeline, you get to add things to the abstract pipeline, or invent a whole another abstract pipeline and document that.
Lossy compression needs its own KMS properties to ensure it can be disabled when necessary. Near-lossless is not lossless, if a difference can be measured. The driver or hardware cannot guess if the end user is ok with "near-lossless" or not, so you have to be conservative and assume not ok, offering an opt-in for lossy.
...
Quoted 4 lines
Quoted 2 lines
My underlying assumption is that generic userspace will not use vendor-specific properties.
I expect some amount of device-specific code in userspace, yes.
If we had a reliable way to test device-specific code without the hardware and automatically in CI, then maybe.
Quoted 5 lines
There are usually large variations in how the hardware exposes access to a given feature, which leads to code having to convert the standard API parameters to hardware parameters. To a large extend this can be done in drivers, but for some more complex features, it may put too much burden on the kernel. There's a reason mesa is a userspace stack :-)
If we get a Khronos standardised 2D composition API... oh wait.
Nothing wrong with userspace libraries, but it does mean that driver developers need to contribute to them, like they do to Mesa. Is there any of that going on for KMS?
In the mean time, DRM UAPI basically must define a 2D composition API, or the new KMS properties will not see use outside of vendor trees.
Quoted 6 lines
Quoted 6 lines
Or, since we have atomic commits with
TEST_ONLY, should it be driver's responsibility to carefully inspect the full state and reject the commit if the hardware is incapable of implementing it correctly? Vendor-specific userspace would know to avoid failing configurations to begin with. I suppose that might put an endless whack-a-mole game on drivers though.
Thanks,
pq
Hi Pekka,
Quoted 87 lines · Pekka Paalanen
Quoted 77 lines · On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:
Quoted 44 lines · Pekka Paalanen
Quoted 43 lines · On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:
Quoted 41 lines · Pekka Paalanen
Quoted 30 lines · On Tue, 15 Jun 2021 12:45:57 +0300 Laurent Pinchart wrote:
Quoted 29 lines · Simon Ser
Quoted 6 lines · Pekka Paalanen
indeed it will, but what else could one do to test userspace KMS clients in generic CI where all you can have is virtual hardware? Maybe in the long run VKMS needs to loop back to a userspace daemon that implements all the complex processing and returns the writeback result via VKMS again? That daemon would then need a single upstream, like the kernel, where it is maintained and correctness verified.
The complex processing must be implemented even without write-back, because user-space can ask for CRCs of the CRTC.
Quoted 4 lines
Or an
LD_PRELOADthat hijacks all KMS ioctls and implements virtual stuff in userspace? Didn't someone already have something like that? It would need to be lifted to be a required part of kernel UAPI submissions, I suppose like IGT is nowadays.FWIW, I have a mock libdrm [1] for libliftoff. This is nowhere near a full software implementation with write-back connectors, but allows to expose virtual planes and check atomic commits in CI.
[1]: https://github.com/emersion/libliftoff/blob/master/test/libdrm_mock.c
Quoted 5 lines
For compositor developers like me knowing the exact formulas would be a huge benefit as it would allow me to use KMS to off-load precision-sensitive operations (e.g. professional color management). Otherwise, compositors probably need a switch: "high quality color management? Then do not use KMS features."
Quoted 8 lines
Quoted 3 lines
I think for alpha blending there are already rounding issues depending on the hardware. I wouldn't keep my hopes up for any guarantee that all hw uses the exact same formulae for color management stuff.
Good, because otherwise you would be very quickly disappointed :-)
For scaling we would also need to replicate the exact same filter taps, which are often not documented.
That is where the documented tolerances come into play.
Quoted 7 lines
Quoted 4 lines
This is something I've experimented with a while ago, when developing automated tests for the rcar-du driver. When playing with different input images we had to constantly increases tolerances, up to a point where the tests started to miss real problems :-(
What should we infer from that? That the hardware is broken and exposing those KMS properties is a false promise?
No, just that the scaler doesn't document the internal hardware implementation (number of taps in the filters, coefficients, rounding, ...). That's the rule, not the exception, and it doesn't prevent correct operation, images get scaled in a reproducible way (the same input produces the same output).
Quoted 3 lines
If a driver on certain hardware cannot correctly implement a KMS property over the full domain of the input space, should that driver then simply not expose the KMS property at all?
The properties involved here would the the SRC and CRTC rectangles for the planes. They don't document pixel-perfect scaling :-)
Quoted 10 lines
But I would assume that the vendor still wants to expose the features in upstream kernels, yet they cannot use the standard KMS properties for that. Should the driver then expose vendor-specific properties with the disclaimer that the result is not always what one would expect, so that userspace written and tested explicitly for that hardware can still work?
That is, a sufficient justification for a vendor-specific KMS property would be that a standard property already exists, but the hardware is too buggy to make it work. IOW, give up trying to make sense.
Quoted 8 lines
It's not just about buggy hardware, it's also about implementation specificities, such as rounding, filters, order of operations in the color management pipeline (it's relatively easy when you only have two LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping features in the mix, not all hardware will implement the same pipeline), or various types of image compression (this device implements a "near-lossless" compression scheme that reduces the memory bandwidth by 50% for instance).
Quoted 5 lines
Rounding shouldn't result in needing wide tolerances.
Filters are more difficult, but at least we can factor them out when testing other things. Filters could be tested in isolation with some image difference metrics rather than per-pixel independent comparisons.
The metrics I was using had both a tolerance on the pixel value and on the number of pixels accepted outside of the value tolerance. I'm sure we can improve it (perhaps taking locality into account), but that's heuristics, and keeping heuristics working across a wide variety of use cases is hard.
The filter I mentioned, by the way, is the scaler filter. Out of curiosity, do any of the devices you work on document with pixel-perfect precision how the hardware scaler is implemented ?
Quoted 9 lines
The order of operations in the color management pipeline is very important. We can't work with "whatever". All the variability in hardware is exactly why I have been calling out for defined abstract color pipeline in the DRM UAPI docs, so that drivers would know which properties to map to their elements, so that userspace can have any possibility of using them correctly. If the hardware has a block that doesn't fit in the abstract pipeline, you get to add things to the abstract pipeline, or invent a whole another abstract pipeline and document that.
One very typical difference between devices is the order of the processing blocks. By modelling the KMS pipeline as degamma -> ccm -> gamma, we can accommodate hardware that have any combination of [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at different points in the pipeline depending on the device, and it will start getting complicated, even if the use case is quite simple and common. This is getting a bit out of topic, but how would you solve this one in particular ?
Quoted 5 lines
Lossy compression needs its own KMS properties to ensure it can be disabled when necessary. Near-lossless is not lossless, if a difference can be measured. The driver or hardware cannot guess if the end user is ok with "near-lossless" or not, so you have to be conservative and assume not ok, offering an opt-in for lossy.
Sure, but what would be the barrier to entry for such a property that would enable the compression (it could actually be a pixel format modifier) ? Would it only need to be documented ? Would we need a software implementation in VKMS and/or in IGT ? The compression algorithm is proprietary and not documented, so the latter can't be done.
Quoted 16 lines
...
Quoted 4 lines
Quoted 2 lines
My underlying assumption is that generic userspace will not use vendor-specific properties.
I expect some amount of device-specific code in userspace, yes.
If we had a reliable way to test device-specific code without the hardware and automatically in CI, then maybe.
Quoted 5 lines
There are usually large variations in how the hardware exposes access to a given feature, which leads to code having to convert the standard API parameters to hardware parameters. To a large extend this can be done in drivers, but for some more complex features, it may put too much burden on the kernel. There's a reason mesa is a userspace stack :-)
Quoted 5 lines
If we get a Khronos standardised 2D composition API... oh wait.
Nothing wrong with userspace libraries, but it does mean that driver developers need to contribute to them, like they do to Mesa. Is there any of that going on for KMS?
Not that I'm aware of, but I think that's a direction we can consider seriously.
Quoted 9 lines
In the mean time, DRM UAPI basically must define a 2D composition API, or the new KMS properties will not see use outside of vendor trees.
Quoted 6 lines
Quoted 6 lines
Or, since we have atomic commits with
TEST_ONLY, should it be driver's responsibility to carefully inspect the full state and reject the commit if the hardware is incapable of implementing it correctly? Vendor-specific userspace would know to avoid failing configurations to begin with. I suppose that might put an endless whack-a-mole game on drivers though.
Signature · 3 lines
Regards, Laurent Pinchart
Quoted 7 lines · On Thu, 17 Jun 2021 16:37:14 +0300 Laurent Pinchart…
Hi Pekka,
Quoted 4 lines · Pekka Paalanen
Quoted 3 lines · On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:
Quoted 2 lines · Pekka Paalanen
Quoted 1 line · On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:
On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:
...
Quoted 49 lines
Quoted 43 lines
Quoted 34 lines
Quoted 9 lines
Quoted 6 lines
Quoted 1 line
That is where the documented tolerances come into play.
This is something I've experimented with a while ago, when developing automated tests for the rcar-du driver. When playing with different input images we had to constantly increases tolerances, up to a point where the tests started to miss real problems :-(
What should we infer from that? That the hardware is broken and exposing those KMS properties is a false promise?
No, just that the scaler doesn't document the internal hardware implementation (number of taps in the filters, coefficients, rounding, ...). That's the rule, not the exception, and it doesn't prevent correct operation, images get scaled in a reproducible way (the same input produces the same output).
Quoted 3 lines
If a driver on certain hardware cannot correctly implement a KMS property over the full domain of the input space, should that driver then simply not expose the KMS property at all?
The properties involved here would the the SRC and CRTC rectangles for the planes. They don't document pixel-perfect scaling :-)
Quoted 10 lines
But I would assume that the vendor still wants to expose the features in upstream kernels, yet they cannot use the standard KMS properties for that. Should the driver then expose vendor-specific properties with the disclaimer that the result is not always what one would expect, so that userspace written and tested explicitly for that hardware can still work?
That is, a sufficient justification for a vendor-specific KMS property would be that a standard property already exists, but the hardware is too buggy to make it work. IOW, give up trying to make sense.
Quoted 8 lines
It's not just about buggy hardware, it's also about implementation specificities, such as rounding, filters, order of operations in the color management pipeline (it's relatively easy when you only have two LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping features in the mix, not all hardware will implement the same pipeline), or various types of image compression (this device implements a "near-lossless" compression scheme that reduces the memory bandwidth by 50% for instance).
Quoted 5 lines
Rounding shouldn't result in needing wide tolerances.
Filters are more difficult, but at least we can factor them out when testing other things. Filters could be tested in isolation with some image difference metrics rather than per-pixel independent comparisons.
Quoted 5 lines
The metrics I was using had both a tolerance on the pixel value and on the number of pixels accepted outside of the value tolerance. I'm sure we can improve it (perhaps taking locality into account), but that's heuristics, and keeping heuristics working across a wide variety of use cases is hard.
Hi Laurent,
I was thinking of using a more, um, scientific error measures, e.g. sum or squared errors (SSE) or average SSE over the whole re-scaled/filtered image result, ignoring pixels outside of that.
What one normally uses when matching images in computer vision, for example. Could even add a threshold such that simple rounding-level errors would not even register in SSE. I'm sure there is plenty of literature on that, but it may be behind a paywall like IEEE.
SSE may or may not need to computed from light-linear pixel values, too. If one wanted to go even further, I'm sure there are computational models about human color and brightness difference sensitivity that could be used to weigh the errors.
Quoted 3 lines
The filter I mentioned, by the way, is the scaler filter. Out of curiosity, do any of the devices you work on document with pixel-perfect precision how the hardware scaler is implemented ?
I don't work on drivers, so wouldn't even look for hardware docs. I go by what KMS UAPI documents because that is the API I work with and nothing else. And yes, I ignore all the scaling filter issues for now.
Because I don't have a way to get feedback (writeback connectors maybe not existing and not hooked up in Weston quite yet), testing scaling/filtering precision has not been on-topic yet. Right now I'm interested in color correctness rather than geometrical filtering. For traditional color management, the expected pixel values are quite precise.
Quoted 10 lines
Quoted 9 lines
The order of operations in the color management pipeline is very important. We can't work with "whatever". All the variability in hardware is exactly why I have been calling out for defined abstract color pipeline in the DRM UAPI docs, so that drivers would know which properties to map to their elements, so that userspace can have any possibility of using them correctly. If the hardware has a block that doesn't fit in the abstract pipeline, you get to add things to the abstract pipeline, or invent a whole another abstract pipeline and document that.
Quoted 4 lines
One very typical difference between devices is the order of the processing blocks. By modelling the KMS pipeline as degamma -> ccm -> gamma, we can accommodate hardware that have any combination of [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at
But you cannot represent pipelines like
1D LUT -> 1D LUT -> CCM
because the abstract pipeline just doesn't have the elements for that. OTOH, maybe that ordering does not even make sense to have in hardware? So maybe not all combinations are actually needed.
Quoted 4 lines
different points in the pipeline depending on the device, and it will start getting complicated, even if the use case is quite simple and common. This is getting a bit out of topic, but how would you solve this one in particular ?
By defining all the points in the abstract color pipeline where a 3D LUT could exist. Then each point would probably need its own KMS property.
We already have the KMS pipeline exactly as degamma -> ctm -> gamma and drivers need to respect that order.
If the combinatorial explosion gets out of hand, maybe we need a KMS property to switch to whole another abstract pipeline which defines a different ordering on the same and/or different KMS properties.
From what I've learnt recently, if you have a 3D LUT, you want a 1D LUT on each side of it for memory vs. precision optimization. And after the degamma -> ctm -> gamma pipeline you may want one more ctm for RGB-to-YCbCr conversion. So I have hope that the abstract pipeline with all actually implemented hardware features might not go totally out of hand.
Quoted 6 lines
Quoted 5 lines
Lossy compression needs its own KMS properties to ensure it can be disabled when necessary. Near-lossless is not lossless, if a difference can be measured. The driver or hardware cannot guess if the end user is ok with "near-lossless" or not, so you have to be conservative and assume not ok, offering an opt-in for lossy.
Quoted 6 lines
Sure, but what would be the barrier to entry for such a property that would enable the compression (it could actually be a pixel format modifier) ? Would it only need to be documented ? Would we need a software implementation in VKMS and/or in IGT ? The compression algorithm is proprietary and not documented, so the latter can't be done.
Good questions. Shows that the idea of strictly requiring a VKMS implementation won't fly, which is what I expected.
Saying it could be a pixel format modifier is a really good point. A modifier cannot be the only thing to control it. Userspace does not decode modifiers, so it cannot filter it out when it wants lossless pixels. There must be something else to control it.
As a userspace dev, I would be ok with documenting a KMS property as "improves blah blah, but also does significant violence to your pixels", so I would know that this is something I need to consider.
You could argue that all KMS properties do violence to pixels, but that's not a useful definition. It would just mean that in some use cases I would never off-load anything to KMS. Depending on the use case that might still be true even if the errors were limited to reasonable rounding errors. I need an idea of how much error does KMS processing do, and ultimately I expect compositors to also need a last resort button for "do not trust KMS processing at all" which then makes the display server always use the exact same simplest possible KMS configuration and let the end user deal with the KMS errors via color-profiling his monitor.
It's quite a different thing to have color processing elements in an unexpected order in the pipeline than it is to have a scaling filter doing slightly unknown operations.
Quoted 16 lines
Quoted 16 lines
...
Quoted 4 lines
Quoted 2 lines
My underlying assumption is that generic userspace will not use vendor-specific properties.
I expect some amount of device-specific code in userspace, yes.
If we had a reliable way to test device-specific code without the hardware and automatically in CI, then maybe.
Quoted 5 lines
There are usually large variations in how the hardware exposes access to a given feature, which leads to code having to convert the standard API parameters to hardware parameters. To a large extend this can be done in drivers, but for some more complex features, it may put too much burden on the kernel. There's a reason mesa is a userspace stack :-)
Quoted 8 lines
Quoted 5 lines
If we get a Khronos standardised 2D composition API... oh wait.
Nothing wrong with userspace libraries, but it does mean that driver developers need to contribute to them, like they do to Mesa. Is there any of that going on for KMS?
Not that I'm aware of, but I think that's a direction we can consider seriously.
That would be awesome if the API is generic.
Quoted 2 lines
Quoted 2 lines
In the mean time, DRM UAPI basically must define a 2D composition API, or the new KMS properties will not see use outside of vendor trees.
Thanks,
pq
Hi Pekka,
Quoted 64 lines · Pekka Paalanen
Quoted 5 lines · On Thu, 17 Jun 2021 16:37:14 +0300 Laurent Pinchart wrote:
Quoted 4 lines · Pekka Paalanen
Quoted 3 lines · On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:
Quoted 2 lines · Pekka Paalanen
Quoted 1 line · On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:
On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:
...
Quoted 49 lines
Quoted 43 lines
Quoted 34 lines
Quoted 9 lines
Quoted 6 lines
Quoted 1 line
That is where the documented tolerances come into play.
This is something I've experimented with a while ago, when developing automated tests for the rcar-du driver. When playing with different input images we had to constantly increases tolerances, up to a point where the tests started to miss real problems :-(
What should we infer from that? That the hardware is broken and exposing those KMS properties is a false promise?
No, just that the scaler doesn't document the internal hardware implementation (number of taps in the filters, coefficients, rounding, ...). That's the rule, not the exception, and it doesn't prevent correct operation, images get scaled in a reproducible way (the same input produces the same output).
Quoted 3 lines
If a driver on certain hardware cannot correctly implement a KMS property over the full domain of the input space, should that driver then simply not expose the KMS property at all?
The properties involved here would the the SRC and CRTC rectangles for the planes. They don't document pixel-perfect scaling :-)
Quoted 10 lines
But I would assume that the vendor still wants to expose the features in upstream kernels, yet they cannot use the standard KMS properties for that. Should the driver then expose vendor-specific properties with the disclaimer that the result is not always what one would expect, so that userspace written and tested explicitly for that hardware can still work?
That is, a sufficient justification for a vendor-specific KMS property would be that a standard property already exists, but the hardware is too buggy to make it work. IOW, give up trying to make sense.
Quoted 8 lines
It's not just about buggy hardware, it's also about implementation specificities, such as rounding, filters, order of operations in the color management pipeline (it's relatively easy when you only have two LUTs and a CCM matrix, but if you through 3D LUTs and other tonemapping features in the mix, not all hardware will implement the same pipeline), or various types of image compression (this device implements a "near-lossless" compression scheme that reduces the memory bandwidth by 50% for instance).
Quoted 5 lines
Rounding shouldn't result in needing wide tolerances.
Filters are more difficult, but at least we can factor them out when testing other things. Filters could be tested in isolation with some image difference metrics rather than per-pixel independent comparisons.
Quoted 5 lines
The metrics I was using had both a tolerance on the pixel value and on the number of pixels accepted outside of the value tolerance. I'm sure we can improve it (perhaps taking locality into account), but that's heuristics, and keeping heuristics working across a wide variety of use cases is hard.
Quoted 5 lines
Hi Laurent,
I was thinking of using a more, um, scientific error measures, e.g. sum or squared errors (SSE) or average SSE over the whole re-scaled/filtered image result, ignoring pixels outside of that.
There are quite a few matching algorithms implement in imagemagick, we have tried them all, and they all suffered for the issue I initially described; we had to push the thresholds high enough to avoid false positives, and we then saw false negatives.
Quoted 9 lines
What one normally uses when matching images in computer vision, for example. Could even add a threshold such that simple rounding-level errors would not even register in SSE. I'm sure there is plenty of literature on that, but it may be behind a paywall like IEEE.
SSE may or may not need to computed from light-linear pixel values, too. If one wanted to go even further, I'm sure there are computational models about human color and brightness difference sensitivity that could be used to weigh the errors.
This isn't about human perception, but about the ability to check if the driver correctly configures the device :-) For driver testing, I don't care if the colours look right or not if userspace misconfigures the colour pipeline. What I need is to catch regressions when my driver messes up the hardware configuration. This particular device processes the images in stripes, and it's very easy for a stripe to be one pixel off (or actually a fraction of a pixel when the scaler is involved).
Testing if a device correctly implements colour processing is of course also useful, but is a distinct goal. Both need to be considered, when stating requirements such as fully documented properties with a VKMS software implementation and pixel-perfect tests in IGT.
Quoted 30 lines
Quoted 3 lines
The filter I mentioned, by the way, is the scaler filter. Out of curiosity, do any of the devices you work on document with pixel-perfect precision how the hardware scaler is implemented ?
I don't work on drivers, so wouldn't even look for hardware docs. I go by what KMS UAPI documents because that is the API I work with and nothing else. And yes, I ignore all the scaling filter issues for now.
Because I don't have a way to get feedback (writeback connectors maybe not existing and not hooked up in Weston quite yet), testing scaling/filtering precision has not been on-topic yet. Right now I'm interested in color correctness rather than geometrical filtering. For traditional color management, the expected pixel values are quite precise.
Quoted 10 lines
Quoted 9 lines
The order of operations in the color management pipeline is very important. We can't work with "whatever". All the variability in hardware is exactly why I have been calling out for defined abstract color pipeline in the DRM UAPI docs, so that drivers would know which properties to map to their elements, so that userspace can have any possibility of using them correctly. If the hardware has a block that doesn't fit in the abstract pipeline, you get to add things to the abstract pipeline, or invent a whole another abstract pipeline and document that.
Quoted 4 lines
One very typical difference between devices is the order of the processing blocks. By modelling the KMS pipeline as degamma -> ccm -> gamma, we can accommodate hardware that have any combination of [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at
Quoted 5 lines
But you cannot represent pipelines like
1D LUT -> 1D LUT -> CCM
because the abstract pipeline just doesn't have the elements for that. OTOH, maybe that ordering does not even make sense to have in hardware? So maybe not all combinations are actually needed.
If userspace wanted such a pipeline (I'm not sure why it would), then it could just combine the two LUTs in one.
Quoted 8 lines
Quoted 4 lines
different points in the pipeline depending on the device, and it will start getting complicated, even if the use case is quite simple and common. This is getting a bit out of topic, but how would you solve this one in particular ?
By defining all the points in the abstract color pipeline where a 3D LUT could exist. Then each point would probably need its own KMS property.
And when we'll add the next object ? This can't scale I'm afraid. You'll have a spec that tells you that things can be in any order, and may well be able to expose the order to userspace, but you won't be able to implement a generic userspace that makes use of it.
BTW, if we want to expose a finer-grained topology of processing blocks, I'd recommend looking at the Media Controller API, it was designed just for that purpose.
Quoted 7 lines
We already have the KMS pipeline exactly as degamma -> ctm -> gamma and drivers need to respect that order.
If the combinatorial explosion gets out of hand, maybe we need a KMS property to switch to whole another abstract pipeline which defines a different ordering on the same and/or different KMS properties.
Quoted 6 lines
From what I've learnt recently, if you have a 3D LUT, you want a 1D LUT on each side of it for memory vs. precision optimization. And after the degamma -> ctm -> gamma pipeline you may want one more ctm for RGB-to-YCbCr conversion. So I have hope that the abstract pipeline with all actually implemented hardware features might not go totally out of hand.
On the output of the CRTC, I have, in this order
- RGB to YUV conversion (can only use presets for BT.601 and BT.709, with limited or full range)
- 3D LUT
- 1D LUT
- YUV to RGB conversion (presets only)
The RGB to YUV and YUV to RGB conversions can be bypassed. That's it.
There's also a histogram engine, to allow implementation of dynamic gamma correction. This would require userspace to read the histogram data for every frame, and update the LUTs accordingly.
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines
Lossy compression needs its own KMS properties to ensure it can be disabled when necessary. Near-lossless is not lossless, if a difference can be measured. The driver or hardware cannot guess if the end user is ok with "near-lossless" or not, so you have to be conservative and assume not ok, offering an opt-in for lossy.
Quoted 6 lines
Sure, but what would be the barrier to entry for such a property that would enable the compression (it could actually be a pixel format modifier) ? Would it only need to be documented ? Would we need a software implementation in VKMS and/or in IGT ? The compression algorithm is proprietary and not documented, so the latter can't be done.
Quoted 7 lines
Good questions. Shows that the idea of strictly requiring a VKMS implementation won't fly, which is what I expected.
Saying it could be a pixel format modifier is a really good point. A modifier cannot be the only thing to control it. Userspace does not decode modifiers, so it cannot filter it out when it wants lossless pixels. There must be something else to control it.
The hardware implementation is such that the decompression is implemented ahead of the display DMA engine, which means that it depends on how the framebuffer memory is mapped. The display controller itself isn't aware of the compression. The driver needs to map framebuffers to the device through a specific memory window to decompress them.
Quoted 4 lines
As a userspace dev, I would be ok with documenting a KMS property as "improves blah blah, but also does significant violence to your pixels", so I would know that this is something I need to consider.
Quoted 10 lines
You could argue that all KMS properties do violence to pixels, but that's not a useful definition. It would just mean that in some use cases I would never off-load anything to KMS. Depending on the use case that might still be true even if the errors were limited to reasonable rounding errors. I need an idea of how much error does KMS processing do, and ultimately I expect compositors to also need a last resort button for "do not trust KMS processing at all" which then makes the display server always use the exact same simplest possible KMS configuration and let the end user deal with the KMS errors via color-profiling his monitor.
I really think we'll end up needing device-specific userspace components. Given the example above, with histogram calculation and dynamic gamma correction, there's no way we'll be able to have a single userspace implementation that would support these kind of features for all devices. It doesn't have to be solved now, but we probably want to start thinking about an API to plug device-specific components in compositors. This is the kind of problem that the libcamera project is tackling on the camera side, I believe a similar approach will be needed for displays too.
Quoted 3 lines
It's quite a different thing to have color processing elements in an unexpected order in the pipeline than it is to have a scaling filter doing slightly unknown operations.
That I agree with, those are two different issues. We all have our cross to bear :-)
Quoted 29 lines
Quoted 16 lines
Quoted 16 lines
...
Quoted 4 lines
Quoted 2 lines
My underlying assumption is that generic userspace will not use vendor-specific properties.
I expect some amount of device-specific code in userspace, yes.
If we had a reliable way to test device-specific code without the hardware and automatically in CI, then maybe.
Quoted 5 lines
There are usually large variations in how the hardware exposes access to a given feature, which leads to code having to convert the standard API parameters to hardware parameters. To a large extend this can be done in drivers, but for some more complex features, it may put too much burden on the kernel. There's a reason mesa is a userspace stack :-)
Quoted 8 lines
Quoted 5 lines
If we get a Khronos standardised 2D composition API... oh wait.
Nothing wrong with userspace libraries, but it does mean that driver developers need to contribute to them, like they do to Mesa. Is there any of that going on for KMS?
Not that I'm aware of, but I think that's a direction we can consider seriously.
That would be awesome if the API is generic.
Quoted 2 lines
Quoted 2 lines
In the mean time, DRM UAPI basically must define a 2D composition API, or the new KMS properties will not see use outside of vendor trees.
Signature · 3 lines
Regards, Laurent Pinchart
Quoted 9 lines · On Fri, 18 Jun 2021 12:58:49 +0300 Laurent Pinchart…
Hi Pekka,
Quoted 6 lines · Pekka Paalanen
Quoted 5 lines · On Thu, 17 Jun 2021 16:37:14 +0300 Laurent Pinchart wrote:
Quoted 4 lines · Pekka Paalanen
Quoted 3 lines · On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:
Quoted 2 lines · Pekka Paalanen
Quoted 1 line · On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:
On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:
...
Quoted 8 lines
This isn't about human perception, but about the ability to check if the driver correctly configures the device :-) For driver testing, I don't care if the colours look right or not if userspace misconfigures the colour pipeline. What I need is to catch regressions when my driver messes up the hardware configuration. This particular device processes the images in stripes, and it's very easy for a stripe to be one pixel off (or actually a fraction of a pixel when the scaler is involved).
Quoted 4 lines
Testing if a device correctly implements colour processing is of course also useful, but is a distinct goal. Both need to be considered, when stating requirements such as fully documented properties with a VKMS software implementation and pixel-perfect tests in IGT.
Right. Your driver testing requirements are different. You know how your hardware works, so you can build your tests to suit.
What this thread is about is KMS UAPI, and verifying that a KMS implementation matches the KMS UAPI specification. The UAPI specification is the only thing I can look at when writing userspace. As a userspace and compositor developer, I am specifically concerned about how humans will perceive the final image.
If you use your hardware-specific tests to ensure that your driver programs the hardware correctly, and IGT tests ensure that the KMS UAPI is implemented to the KMS UAPI spec, then I would hope that everything is fine. By reading the KMS UAPI spec I can decide to use or not use specific KMS features.
...in an ideal world.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
One very typical difference between devices is the order of the processing blocks. By modelling the KMS pipeline as degamma -> ccm -> gamma, we can accommodate hardware that have any combination of [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at
Quoted 8 lines
Quoted 5 lines
But you cannot represent pipelines like
1D LUT -> 1D LUT -> CCM
because the abstract pipeline just doesn't have the elements for that. OTOH, maybe that ordering does not even make sense to have in hardware? So maybe not all combinations are actually needed.If userspace wanted such a pipeline (I'm not sure why it would), then it could just combine the two LUTs in one.
Maybe? You can also combine the 1D LUTs into the 3D LUT in the middle, too, but the result is not generally the same as using them separately when the 3D LUT size is limited.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
different points in the pipeline depending on the device, and it will start getting complicated, even if the use case is quite simple and common. This is getting a bit out of topic, but how would you solve this one in particular ?
Quoted 8 lines
Quoted 3 lines
By defining all the points in the abstract color pipeline where a 3D LUT could exist. Then each point would probably need its own KMS property.
And when we'll add the next object ? This can't scale I'm afraid. You'll have a spec that tells you that things can be in any order, and may well be able to expose the order to userspace, but you won't be able to implement a generic userspace that makes use of it.
Are you saying that generic userspace cannot happen? On top of KMS UAPI, that is.
Why would a generic userspace library API be a more feasible effort?
Quoted 3 lines
BTW, if we want to expose a finer-grained topology of processing blocks, I'd recommend looking at the Media Controller API, it was designed just for that purpose.
Sure. However, right now we have KMS properties instead. Or maybe they should just go all unused?
Maybe the rule should be to not add any more KMS properties and tell people to design something completely new based on Media Controller API instead?
I've never looked at Media Controller API, so I have no idea how it would suit display servers.
I very much think that I cannot use processing blocks exposed by KMS if I cannot have guarantees that they work the same across all Linux drivers.
Quoted 22 lines
Quoted 7 lines
We already have the KMS pipeline exactly as degamma -> ctm -> gamma and drivers need to respect that order.
If the combinatorial explosion gets out of hand, maybe we need a KMS property to switch to whole another abstract pipeline which defines a different ordering on the same and/or different KMS properties.
Quoted 6 lines
From what I've learnt recently, if you have a 3D LUT, you want a 1D LUT on each side of it for memory vs. precision optimization. And after the degamma -> ctm -> gamma pipeline you may want one more ctm for RGB-to-YCbCr conversion. So I have hope that the abstract pipeline with all actually implemented hardware features might not go totally out of hand.
On the output of the CRTC, I have, in this order
- RGB to YUV conversion (can only use presets for BT.601 and BT.709, with limited or full range)
- 3D LUT
- 1D LUT
- YUV to RGB conversion (presets only)
Quoted 5 lines
The RGB to YUV and YUV to RGB conversions can be bypassed. That's it.
There's also a histogram engine, to allow implementation of dynamic gamma correction. This would require userspace to read the histogram data for every frame, and update the LUTs accordingly.
Interesting, but almost none of that would be used by the color management pipeline model I'm familiar with. I wonder what kind of use cases doing LUTs in YUV have been designed for, and *after* blending, even.
...
Quoted 9 lines
I really think we'll end up needing device-specific userspace components. Given the example above, with histogram calculation and dynamic gamma correction, there's no way we'll be able to have a single userspace implementation that would support these kind of features for all devices. It doesn't have to be solved now, but we probably want to start thinking about an API to plug device-specific components in compositors. This is the kind of problem that the libcamera project is tackling on the camera side, I believe a similar approach will be needed for displays too.
To be able to plug device-specific components in compositors, we need a driver-agnostic API through which compositors can use those device specific components. Still the same problem, now just in userspace than at the UAPI level.
Thanks,
pq
Hi Pekka,
Quoted 24 lines · Pekka Paalanen
Quoted 7 lines · On Fri, 18 Jun 2021 12:58:49 +0300 Laurent Pinchart wrote:
Quoted 6 lines · Pekka Paalanen
Quoted 5 lines · On Thu, 17 Jun 2021 16:37:14 +0300 Laurent Pinchart wrote:
Quoted 4 lines · Pekka Paalanen
Quoted 3 lines · On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:
Quoted 2 lines · Pekka Paalanen
Quoted 1 line · On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:
On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:
...
Quoted 8 lines
This isn't about human perception, but about the ability to check if the driver correctly configures the device :-) For driver testing, I don't care if the colours look right or not if userspace misconfigures the colour pipeline. What I need is to catch regressions when my driver messes up the hardware configuration. This particular device processes the images in stripes, and it's very easy for a stripe to be one pixel off (or actually a fraction of a pixel when the scaler is involved).
Quoted 4 lines
Testing if a device correctly implements colour processing is of course also useful, but is a distinct goal. Both need to be considered, when stating requirements such as fully documented properties with a VKMS software implementation and pixel-perfect tests in IGT.
Quoted 8 lines
Right. Your driver testing requirements are different. You know how your hardware works, so you can build your tests to suit.
What this thread is about is KMS UAPI, and verifying that a KMS implementation matches the KMS UAPI specification. The UAPI specification is the only thing I can look at when writing userspace. As a userspace and compositor developer, I am specifically concerned about how humans will perceive the final image.
Yes, the thread has gone in a few different directions.
Quoted 7 lines
If you use your hardware-specific tests to ensure that your driver programs the hardware correctly, and IGT tests ensure that the KMS UAPI is implemented to the KMS UAPI spec, then I would hope that everything is fine. By reading the KMS UAPI spec I can decide to use or not use specific KMS features.
...in an ideal world.
I wish I knew where that world was ;-)
I do agree with you though, you need predictability in userspace, otherwise it's quite useless (or not very useful). This brings back many memories (so unfortunately not too distant) with cameras. No latter than yesterday, I heard from a person with lots of involvement in GPU API design that GPUs seem easy compared to cameras. Hopefully this means we'll be able to solve this on the display side :-)
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Quoted 4 lines
One very typical difference between devices is the order of the processing blocks. By modelling the KMS pipeline as degamma -> ccm -> gamma, we can accommodate hardware that have any combination of [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at
But you cannot represent pipelines like
1D LUT -> 1D LUT -> CCM
because the abstract pipeline just doesn't have the elements for that. OTOH, maybe that ordering does not even make sense to have in hardware? So maybe not all combinations are actually needed.
Quoted 6 lines
Quoted 2 lines
If userspace wanted such a pipeline (I'm not sure why it would), then it could just combine the two LUTs in one.
Maybe? You can also combine the 1D LUTs into the 3D LUT in the middle, too, but the result is not generally the same as using them separately when the 3D LUT size is limited.
Yes, my 3D LUT has 17 points, I've heard about LUTs up to 33 points (with bilinear interpolation), that's quite little if used as a 1D LUT. There's also the fact that interpolation will result in cross-talk between the three colour channels.
Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
different points in the pipeline depending on the device, and it will start getting complicated, even if the use case is quite simple and common. This is getting a bit out of topic, but how would you solve this one in particular ?
Quoted 8 lines
Quoted 3 lines
By defining all the points in the abstract color pipeline where a 3D LUT could exist. Then each point would probably need its own KMS property.
And when we'll add the next object ? This can't scale I'm afraid. You'll have a spec that tells you that things can be in any order, and may well be able to expose the order to userspace, but you won't be able to implement a generic userspace that makes use of it.
Quoted 4 lines
Are you saying that generic userspace cannot happen? On top of KMS UAPI, that is.
Why would a generic userspace library API be a more feasible effort?
I'll make a parallel with the camera world again, because I think display hardware is taking a similar path, just later.
We used to have capture devices that offered a fairly high-level interface at the hardware or firmware level, with a set of features that were more or less standard. Devices could be exposed with an abstract model, as done in KMS today, with a set of standardized controls (V4L2 uses the word control when KMS uses property). We had way less automated testing, and didn't enough attention to consistency of the userspace API (we're talking about code from the beginning of the century, if not earlier, the world was different), so different drivers were not consistent in how the API was implemented (for instance the API didn't specify gain units, so drivers would use different units), but more or less, you could write a standard V4L2 application that would work with most devices, with a few quirks here and there.
Then the world changed. Cameras become more complex, with the raw sensor and the ISP exposed to Linux in the OMAP3 that Nokia used in the N900 and N9 phones. For a while we tried to stick with V4L2, and implement ISP control within the kernel, with the help of a userspace daemon behind the scenes to which the kernel driver called back. The daemon was required, as control of the camera required complicated computation that really benefited from floating point math. It was horrible, and the daemon was closed-source, but from an application point of view, it was still high-level V4L2 as before. This hit a big wall though, to expose a high-level API on top of a low-level hardware interface, we had to bridge the gap in the kernel. This meant hardcoding use cases on the kernel side, which clearly wouldn't scale, as not everybody had the same use cases (an industrial vision application and a smartphone camera application really don't see the world the same way). We were facing a dead end.
We then decided to expose a lower-level API from kernel to userspace, giving userspace the ability to control everything. That's how the Media Controller API was born (its purpose is to expose the internal topology of the device as a graph of connected entities, that's about it), as well as the V4L2 subdev userspace API (in KMS term, that would mean more or less exposing DRM bridges to userspace, with the ability to control them individually). This simplified the kernel implementation (no more hardcoded use cases, no more daemon behind the scenes), but at the same time resulted in an API that wasn't a good fit for applications to consume directly. One step closer to the hardware, one step further from the user. You can think of this as the API exposed by GPU kernel drivers, it's not enough, a userspace component to translate to a higher-level API is needed.
This was followed by 10 years of limbo as Nokia pulled the plug from Linux (and then smartphone) development. Fast forward, at the end of 2018, we created libcamera as the mesa of the camera world to solve this issue. It offers a standard camera stack with a high-level API, where device-specific code can be added (and kept to the minimum, as a centralized framework means we can share code that is common between devices).
I see lots of parallel with display pipelines getting more complex. It wouldn't be fair to expose a very low-level API and ask every single compositor to know about all the devices out there, but at the same time, we're reaching a level of hardware complexity that I believe makes it a dead end to try to abstract and standardize everything at the kernel level. This doesn't mean we should allow all vendors to throw in any kind of crappy properties or other API extensions, we can still enforce some ground rules, such as proper documentation, reasonable effort for standardization, and reference open-source userspace implementation in a project we would consider to be the equalivant of mesa for KMS.
Quoted 10 lines
Quoted 3 lines
BTW, if we want to expose a finer-grained topology of processing blocks, I'd recommend looking at the Media Controller API, it was designed just for that purpose.
Sure. However, right now we have KMS properties instead. Or maybe they should just go all unused?
Maybe the rule should be to not add any more KMS properties and tell people to design something completely new based on Media Controller API instead?
No, I think that would be too harsh, my point was that if we want to expose how the building blocks are layed out, the MC API is meant to expose a graph of connected entities, so it's a good match. Controlling the device should still go through KMS properties I believe.
Quoted 31 lines
I've never looked at Media Controller API, so I have no idea how it would suit display servers.
I very much think that I cannot use processing blocks exposed by KMS if I cannot have guarantees that they work the same across all Linux drivers.
Quoted 24 lines
Quoted 7 lines
We already have the KMS pipeline exactly as degamma -> ctm -> gamma and drivers need to respect that order.
If the combinatorial explosion gets out of hand, maybe we need a KMS property to switch to whole another abstract pipeline which defines a different ordering on the same and/or different KMS properties.
Quoted 6 lines
From what I've learnt recently, if you have a 3D LUT, you want a 1D LUT on each side of it for memory vs. precision optimization. And after the degamma -> ctm -> gamma pipeline you may want one more ctm for RGB-to-YCbCr conversion. So I have hope that the abstract pipeline with all actually implemented hardware features might not go totally out of hand.
On the output of the CRTC, I have, in this order
- RGB to YUV conversion (can only use presets for BT.601 and BT.709, with limited or full range)
- 3D LUT
- 1D LUT
- YUV to RGB conversion (presets only)The RGB to YUV and YUV to RGB conversions can be bypassed. That's it.
Quoted 8 lines
Quoted 3 lines
There's also a histogram engine, to allow implementation of dynamic gamma correction. This would require userspace to read the histogram data for every frame, and update the LUTs accordingly.
Interesting, but almost none of that would be used by the color management pipeline model I'm familiar with. I wonder what kind of use cases doing LUTs in YUV have been designed for, and *after* blending, even.
I'm afraid I can't tell as I don't know. This reaches the limit of my display colour management knowledge.
Quoted 12 lines
...
Quoted 9 lines
I really think we'll end up needing device-specific userspace components. Given the example above, with histogram calculation and dynamic gamma correction, there's no way we'll be able to have a single userspace implementation that would support these kind of features for all devices. It doesn't have to be solved now, but we probably want to start thinking about an API to plug device-specific components in compositors. This is the kind of problem that the libcamera project is tackling on the camera side, I believe a similar approach will be needed for displays too.
Quoted 4 lines
To be able to plug device-specific components in compositors, we need a driver-agnostic API through which compositors can use those device specific components. Still the same problem, now just in userspace than at the UAPI level.
Agreed, it pushes the problem to a different place :-) I however think it may be easier to solve it in userspace, as it's a better place to hardcode use cases. A general-purpose compositor may not want to make use of all hardware features, while a specific-purpose compositor may want to exercise the hardware features differently. This however opens the door to introducing badly-designed and un(der-)documented properties, which is not something I want, so we would still need to set ground rules, such as documentation and reference implementation in a given userspace compositor/stack.
On that note, I think we need to do a bit more hand-holding. I'm sitting on a patch series that is several years old to add 3D LUT support to my driver, and while I believe I'm quite experienced with KMS development, adding support for this in a reference compositor in userspace is a really big task that I don't know how to tackle. I also know very little about colour management, and have nobody in the team who has that knowledge and could help me. I'm thus facing a bit of a dead-end, not because I don't want to spend the time required for this, but because the bar is so high that I just can't do it. I would expect many developers to be facing the same challenges. If the community doesn't make an effort in fostering collaboration between developers (most kernel developers won't even know who to ask for help if they have to step out of the kernel to userspace) and providing good documentation, we'll just scare developers away. It took me weeks to figure out how to run Weston on this device, as it has no GPU that I can use, and falling back to a software compositor required finding out that lots of documentation is outdated and referenced command line options that were not accurate anymore (or I could mention painfully finding out that an issue I was blocked with for days was solved by plugging in a USB mouse to the board, as Weston didn't want to start without an input device). Another developer from my team tried before me and gave up, thinking it was impossible. That's the reality that many kernel developers are facing, the same way I suppose than me saying saying "you just need to recompile the kernel with this option enabled" is not very helpful for lots of userspace developers.
Signature · 3 lines
Regards, Laurent Pinchart
Quoted 11 lines · On Fri, 18 Jun 2021 15:19:15 +0300 Laurent Pinchart…
Hi Pekka,
Quoted 8 lines · Pekka Paalanen
Quoted 7 lines · On Fri, 18 Jun 2021 12:58:49 +0300 Laurent Pinchart wrote:
Quoted 6 lines · Pekka Paalanen
Quoted 5 lines · On Thu, 17 Jun 2021 16:37:14 +0300 Laurent Pinchart wrote:
Quoted 4 lines · Pekka Paalanen
Quoted 3 lines · On Thu, 17 Jun 2021 13:29:48 +0300 Laurent Pinchart wrote:
Quoted 2 lines · Pekka Paalanen
Quoted 1 line · On Thu, 17 Jun 2021 00:05:24 +0300 Laurent Pinchart wrote:
On Tue, Jun 15, 2021 at 01:16:56PM +0300, Pekka Paalanen wrote:
...
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 10 lines
Quoted 4 lines
One very typical difference between devices is the order of the processing blocks. By modelling the KMS pipeline as degamma -> ccm -> gamma, we can accommodate hardware that have any combination of [1-2] * 1D LUTs + 1 * CCM. Now, throw one 3D LUT into the mix, at
But you cannot represent pipelines like
1D LUT -> 1D LUT -> CCM
because the abstract pipeline just doesn't have the elements for that. OTOH, maybe that ordering does not even make sense to have in hardware? So maybe not all combinations are actually needed.If userspace wanted such a pipeline (I'm not sure why it would), then it could just combine the two LUTs in one.
Quoted 8 lines
Quoted 3 lines
Maybe? You can also combine the 1D LUTs into the 3D LUT in the middle, too, but the result is not generally the same as using them separately when the 3D LUT size is limited.
Yes, my 3D LUT has 17 points, I've heard about LUTs up to 33 points (with bilinear interpolation), that's quite little if used as a 1D LUT. There's also the fact that interpolation will result in cross-talk between the three colour channels.
Hi Laurent,
that is not what I meant, but that's true too.
What I meant was this:
https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/582#note_951271
pre_curve and post_curve are the 1D LUTs on each side of a size-limited 3D LUT. pre_curve controls the 3D LUT tap positioning to concentrate on the areas where the approximated function needs more taps.
Quoted 56 lines
Quoted 1 line
Why would a generic userspace library API be a more feasible effort?
I'll make a parallel with the camera world again, because I think display hardware is taking a similar path, just later.
We used to have capture devices that offered a fairly high-level interface at the hardware or firmware level, with a set of features that were more or less standard. Devices could be exposed with an abstract model, as done in KMS today, with a set of standardized controls (V4L2 uses the word control when KMS uses property). We had way less automated testing, and didn't enough attention to consistency of the userspace API (we're talking about code from the beginning of the century, if not earlier, the world was different), so different drivers were not consistent in how the API was implemented (for instance the API didn't specify gain units, so drivers would use different units), but more or less, you could write a standard V4L2 application that would work with most devices, with a few quirks here and there.
Then the world changed. Cameras become more complex, with the raw sensor and the ISP exposed to Linux in the OMAP3 that Nokia used in the N900 and N9 phones. For a while we tried to stick with V4L2, and implement ISP control within the kernel, with the help of a userspace daemon behind the scenes to which the kernel driver called back. The daemon was required, as control of the camera required complicated computation that really benefited from floating point math. It was horrible, and the daemon was closed-source, but from an application point of view, it was still high-level V4L2 as before. This hit a big wall though, to expose a high-level API on top of a low-level hardware interface, we had to bridge the gap in the kernel. This meant hardcoding use cases on the kernel side, which clearly wouldn't scale, as not everybody had the same use cases (an industrial vision application and a smartphone camera application really don't see the world the same way). We were facing a dead end.
We then decided to expose a lower-level API from kernel to userspace, giving userspace the ability to control everything. That's how the Media Controller API was born (its purpose is to expose the internal topology of the device as a graph of connected entities, that's about it), as well as the V4L2 subdev userspace API (in KMS term, that would mean more or less exposing DRM bridges to userspace, with the ability to control them individually). This simplified the kernel implementation (no more hardcoded use cases, no more daemon behind the scenes), but at the same time resulted in an API that wasn't a good fit for applications to consume directly. One step closer to the hardware, one step further from the user. You can think of this as the API exposed by GPU kernel drivers, it's not enough, a userspace component to translate to a higher-level API is needed.
This was followed by 10 years of limbo as Nokia pulled the plug from Linux (and then smartphone) development. Fast forward, at the end of 2018, we created libcamera as the mesa of the camera world to solve this issue. It offers a standard camera stack with a high-level API, where device-specific code can be added (and kept to the minimum, as a centralized framework means we can share code that is common between devices).
Quoted 11 lines
I see lots of parallel with display pipelines getting more complex. It wouldn't be fair to expose a very low-level API and ask every single compositor to know about all the devices out there, but at the same time, we're reaching a level of hardware complexity that I believe makes it a dead end to try to abstract and standardize everything at the kernel level. This doesn't mean we should allow all vendors to throw in any kind of crappy properties or other API extensions, we can still enforce some ground rules, such as proper documentation, reasonable effort for standardization, and reference open-source userspace implementation in a project we would consider to be the equalivant of mesa for KMS.
Thank you very much for explaining this! It makes perfect sense to me. It also sounds like a ¤&!%¤# amount of work, but really does sound like the best plan in the long term.
GPUs have OpenGL and Vulkan, but I suppose with cameras and display, there is nothing standard to aim for, for either hardware designers nor software designers? Chicken-and-egg.
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
BTW, if we want to expose a finer-grained topology of processing blocks, I'd recommend looking at the Media Controller API, it was designed just for that purpose.
Sure. However, right now we have KMS properties instead. Or maybe they should just go all unused?
Quoted 8 lines
Quoted 3 lines
Maybe the rule should be to not add any more KMS properties and tell people to design something completely new based on Media Controller API instead?
No, I think that would be too harsh, my point was that if we want to expose how the building blocks are layed out, the MC API is meant to expose a graph of connected entities, so it's a good match. Controlling the device should still go through KMS properties I believe.
Ok.
...
Quoted 5 lines
Quoted 4 lines
To be able to plug device-specific components in compositors, we need a driver-agnostic API through which compositors can use those device specific components. Still the same problem, now just in userspace than at the UAPI level.
Quoted 9 lines
Agreed, it pushes the problem to a different place :-) I however think it may be easier to solve it in userspace, as it's a better place to hardcode use cases. A general-purpose compositor may not want to make use of all hardware features, while a specific-purpose compositor may want to exercise the hardware features differently. This however opens the door to introducing badly-designed and un(der-)documented properties, which is not something I want, so we would still need to set ground rules, such as documentation and reference implementation in a given userspace compositor/stack.
Right, sounds fine, but I think it also shifts the KMS UAPI paradigm a little, away from generic and towards driver-specific. If the DRM maintainers agree to that, I'm fine with it. It's just different from what I have understood so far, which is why I'm so keen on getting UAPI documented precisely.
Quoted 6 lines
On that note, I think we need to do a bit more hand-holding. I'm sitting on a patch series that is several years old to add 3D LUT support to my driver, and while I believe I'm quite experienced with KMS development, adding support for this in a reference compositor in userspace is a really big task that I don't know how to tackle. I also know very little about colour management, and have nobody in the team who has that
Quoted 8 lines
knowledge and could help me. I'm thus facing a bit of a dead-end, not because I don't want to spend the time required for this, but because the bar is so high that I just can't do it. I would expect many developers to be facing the same challenges. If the community doesn't make an effort in fostering collaboration between developers (most kernel developers won't even know who to ask for help if they have to step out of the kernel to userspace) and providing good documentation, we'll just scare developers away.
That's why I'm screaming here, to make contact. :-D
And Simor Ser is another, and I have seen him even offer to implement userspace bits for new KMS properties in development.
But a 3D LUT in display is a really tiny piece in the landscape of color management. Color management on Wayland has been talked about, oh I don't remember, for a decade? Weston code has been experimented on for the past few years at least. But only last week, the very first piece of actual color management exclusive code was merged in Weston upstream.
Getting a display server project into a phase where it would actually start experimenting on KMS features can take a decade - not just work but finding the funding. Weston is not quite there yet either, not even with all the currently open merge requests.
Then, if your 3D LUT is after blending, I'm not sure I would have use for it in Weston! If it was on a KMS plane before blending, then yes, I would have use for it. All this depends on how the blending color space is chosen, and the idea we have right now does not have use for a 3D LUT after blending - except maybe for fullscreen single-buffer scenegraph, e.g. video and games.
Quoted 4 lines
It took me weeks to figure out how to
run Weston on this device, as it has no GPU that I can use, and falling back to a software compositor required finding out that lots of documentation is outdated and referenced command line options that were
Quoted 8 lines
not accurate anymore (or I could mention painfully finding out that an issue I was blocked with for days was solved by plugging in a USB mouse to the board, as Weston didn't want to start without an input device). Another developer from my team tried before me and gave up, thinking it was impossible. That's the reality that many kernel developers are facing, the same way I suppose than me saying saying "you just need to recompile the kernel with this option enabled" is not very helpful for lots of userspace developers.
Why have I not heard of your problems with Weston?!
We have Gitlab issues, mailing list and IRC. You're warmly welcome to ask when you hit a wall. :-)
I even read #dri-devel very often, so if you mentioned Weston there, it would have had a good chance to reach me.
Thanks,
pq