Add Support for Plane Color Pipeline
Subsystems (5)
cover Add Support for Plane Color Pipeline
Introduction
============
Modern hardwares have various color processing capabilities both at pre-blending and post-blending phases in the color pipeline. The current drm implementation exposes only the post-blending color hardware blocks. Support for pre-blending hardware is missing. There are multiple use cases where pre-blending color hardware will be useful:
a) Linearization of input buffers encoded in various transfer functions.
b) Color Space conversion
c) Tone mapping
d) Frame buffer format conversion
e) Non-linearization of buffer(apply transfer function)
f) 3D Luts
and other miscellaneous color operations.
Hence, there is a need to expose the color capabilities of the hardware to user-space. This will help userspace/middleware to use display hardware for color processing and blending instead of doing it through GPU shaders.
Work done so far and relevant references
========================================
Some implementation is done by Intel and AMD/Igalia to address the same. Broad consensus is there that we need a generic API at drm core to suffice the use case of various HW vendors. Below are the links capturing the discussion so far.
Intel's Plane Color Implementation: https://patchwork.freedesktop.org/series/90825/
AMD's Plane Color Implementation: https://patchwork.freedesktop.org/series/116862/
Hackfest conclusions
====================
HDR/Color Hackfest was organised by Redhat to bring all the industry stakeholders together and converge on a common uapi expectations. Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia and other prominent user-space developers and maintainers.
Discussions happened on the uapi expectations, opens, nature of hardware of multiple hardware vendors, challenges in generalizing the same and the path forward. Consensus was made that drm core should implement descriptive APIs and not go with prescriptive APIs. DRM core should just expose the hardware capabilities; enabling, customizing and programming the same should be done by the user-space. Driver should just honor the user space request without doing any operations internally.
Thanks to Simon Ser, for nicely documenting the design consensus and an UAPI RFC which can be referred to here:
Design considerations
=====================
Following are the important aspects taken into account while designing the current RFC proposal:
1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used)
2. Position of the HW block in the pipeline can be programmable
3. LUTs can be one dimentional or three dimentional
4. Number of LUT entries can vary across platforms
5. Precision of LUT entries can vary across platforms
6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic, Piece-Wise Linear(PWL) etc
7. There can be parameterized/non-parameterized fixed function HW blocks. e.g. Just a hardware bit, to convert from one color space to another.
8. Custom non-standard HW implementation.
9. Leaving scope for some vendor defined pescriptive implementation if required. 10.Scope to handle any modification in hardware as technology evolves
The current proposal takes into account the above considerations while keeping the implementation as generic as possible leaving scope for future additions or modifications.
This proposal is also in line to the details mentioned by Simon's RFC covering all the aspects discussed in hackfest.
Outline of the implementation
============================
Each Color Hardware block will be represented by a data structure drm_color_op. These color operations will form the building blocks of a color pipeline which best represents the underlying Hardware. Color operations can be re-arranged, substracted or added to create distinct color pipelines to accurately describe the Hardware blocks present in the display engine.
In this proposal, a color pipeline is represented as an array of struct drm_color_op. For individual color operation, we add blobs to advertise the capability of the particular Hardware block.
This color pipeline is then packaged within a blob for the user space to retrieve it.
To advertise the available color pipelines, an immutable ENUM property "GET_COLOR_PIPELINE" is introduced. This enum property has blob id's as values. With each blob id representing a distinct color pipeline based on underlying HW capabilities and their respective combinations.
Once the user space decides on a color pipeline, it can set the pipeline and the corresponding data for the hardware blocks within the pipeline with the BLOB property "SET_COLOR_PIPELINE".
Refer to Documentation/gpu/rfc/plane_color_pipeline.rst added in the patch
IGT and test details
====================
A set of IGT tests is written to demonstrate the usage of the proposed UAPIs along with some sanity validation.
Details of the IGT test can be found here:
https://patchwork.freedesktop.org/series/123018/
Opens
=====
a. To come up with a list of common HW blocks which can be defined generically by the DRM
core in agreement with all the stakeholders
b. To enhance/finalize the data structure to define segmented LUTs generically.
Out of Scope
============
a. The coefficients for CTM and LUT value calculations are out of scope of the proposal. b. The onus of programming the HW blocks and their values is on user-space. Driver will
just provide the interface for the same.
c. In order to compute LUT values and coefficients, a helper library can be created in
user-space. However, it is out of scope for the current proposal.
Acknowledgements and credits
============================
There are multiple contributors who have helped us to reach to this proposal. Special mention to Ville Syrjala<[email protected]>, Pekka Paalanen<[email protected]>, Simon Ser<[email protected]>, Harry Wentland<[email protected]>,
Melissa Wen<[email protected]>, Jonas<[email protected]>, Sebastian Wick<[email protected]>, Bhanu<[email protected]> and Shashank<[email protected]>.
Also, thanks to Carlos <[email protected]> and the Redhat team for organizing the HDR hackfest.
UAPI dependency and Usermode development
========================================
The current KMS implementation requires a user space consumer for it to be accepted upstream. Work is ongoing in weston and mutter community to get color management and HDR support implemented in the respective stacks.
=================================================================================
We have tried to take care of all the scenarios and use-cases which possibly could exists in the current proposal. Thanks to everyone who has contributed in all color management discussions so far. Let's work together to improve the current proposal and get this thing implemented in upstream linux. All the feedback and suggestions to enhance the design are welcome.
Regards,
Uma Shankar
Chaitanya Kumar Borah
Chaitanya Kumar Borah (14):
drm: Add color operation structure
drm: Add plane get color pipeline property
drm: Add helper to add color pipeline
drm: Manage color blob states
drm: Replace individual color blobs
drm: Reset pipeline when user sends NULL blob
drm: Reset plane color state on pipeline switch request drm/i915/color: Add HDR plane LUT range data to color pipeline drm/i915/color: Add SDR plane LUT range data to color pipeline drm/i915/color: Add color pipelines to plane drm/i915/color: Create and attach set color pipeline property drm/i915/color: Enable plane color features drm/i915/color: Add a dummy pipeline with 3D LUT drm/i915/color: Add example implementation for vendor specific color
operation
Uma Shankar (19):
drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
drm: Add structures for setting color pipeline
drm: Add set colorpipeline property
drm: Add Enhanced Gamma LUT precision structure
drm: Add color lut range structure
drm: Add color information to plane state drm/i915/color: Add lut range for SDR planes drm/i915/color: Add lut range for HDR planes drm/i915/color: Add color pipeline for HDR planes drm/i915/color: Add color pipeline for SDR planes drm/i915/color: Add plane color callbacks drm/i915/color: Load plane color luts from atomic flip drm/i915/xelpd: Add plane color check to glk_plane_color_ctl drm/i915/xelpd: Add register definitions for Plane Degamma drm/i915/color: Add color functions for ADL drm/i915/color: Program Plane Pre-CSC Registers drm/i915/xelpd: Add register definitions for Plane Post CSC drm/i915/xelpd: Program Plane Post CSC Registers drm/i915/color: Enable Plane CSC
.../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++ drivers/gpu/drm/drm_atomic_state_helper.c | 21 + drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++ drivers/gpu/drm/drm_color_mgmt.c | 130 ++++ drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++- drivers/gpu/drm/i915/display/intel_color.h | 7 +- .../drm/i915/display/skl_universal_plane.c | 21 +- drivers/gpu/drm/i915/i915_reg.h | 124 ++++ include/drm/drm_plane.h | 82 +++ include/uapi/drm/drm_mode.h | 196 +++++ include/uapi/drm/i915_drm.h | 25 + 11 files changed, 1899 insertions(+), 3 deletions(-) create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
Signature · 1 line
2.38.1
RFC 1/33 drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
Add the documentation for the new proposed Plane Color Pipeline.
--- .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++ 1 file changed, 394 insertions(+) create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
Documentation/gpu/rfc/plane_color_pipeline.rst new file +394 -0
@@ -0,0 +1,394 @@ +======================================= + Plane Color Pipeline: A UAPI proposal +======================================= + +To build the proposal on, lets take the premise of a color pipeline as shown +below. + + +-------------------------------------------+ + | RAM | + | +------+ +---------+ +---------+ | + | | FB 1 | | FB 2 | | FB N | | + | +------+ +---------+ +---------+ | + +-------------------------------------------+ + | Plane Color Hardware Block | + +--------------------------------------------+ + | +---v-----+ +---v-------+ +---v------+ | + | | Plane A | | Plane B | | Plane N | | + | | Pre-CSC | | Pre-CSC | | Pre-CSC | | + | +---+-----+ +---+-------+ +---+------+ | + | | | | | + | +---v-----+ +---v-------+ +---v------+ | + | |Plane A | | Plane B | | Plane N | | + | |CSC/CTM | | CSC/CTM | | CSC/CTM | | + | +---+-----+ +----+------+ +----+-----+ | + | | | | | + | +---v-----+ +----v------+ +----v-----+ | + | | Plane A | | Plane B | | Plane N | | + | |Post-CSC | | Post-CSC | | Post-CSC | | + | +---+-----+ +----+------+ +----+-----+ | + | | | | | + +--------------------------------------------+ ++------v--------------v---------------v-------| +|| || +|| Pipe Blender || ++--------------------+------------------------+ +| | | +| +-----------v----------+ | +| | Pipe Pre-CSC | | +| | | | +| +-----------+----------+ | +| | Pipe Color | +| +-----------v----------+ Hardware | +| | Pipe CSC/CTM | | +| | | | +| +-----------+----------+ | +| | | +| +-----------v----------+ | +| | Pipe Post-CSC | | +| | | | +| +-----------+----------+ | +| | | ++---------------------------------------------+ + | + v + Pipe Output + +Each plane consists of the following color blocks + * Pre-CSC : This block can used to linearize the input frame buffer data. + The linear data then can be further acted on by the following + color hardware blocks in the display hardware pipeline + + * CSC/CTM: Used to program color transformation matrix, this block is used + to perform color space conversions like BT2020 to BT709 or BT601 + etc. This block acts on the linearized data coming from the + Pre-CSC HW block. + + * Post-CSC: This HW block can be used to non-linearize frame buffer data to + match the sink. Another use case of it could be to perform Tone + mapping for HDR use-cases. + +Data from multiple planes will then be fed to pipe/crtc where it will get blended. +There is a similar set of HW blocks available at pipe/crtc level which acts on +this blended data. + +Below is a sample usecase fo video playback with sub-titles and playback +controls + +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ +│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ +│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │ +│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐ +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ +(subtitles) │ + │ +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├───┐ │ +│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │ +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ +(Playback controls UI) │ │ + │ │ +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │ +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐ │ │ +│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │ +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │ +(video frame) │ │ │ + │ │ │ +┌────────────────────────────────────────────────────────────────────────┴─┴─┘ +│ +│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐ +│ │ CRTC │ │ CRTC │ │ CRTC │ +└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────► TO Port + │ │ │ │ │ │ + └─────────────┘ └─────────────┘ └───────────────┘ + +This RFC is intended to propose an uAPI for the pre-blending color pipeline +(however, can be also extended to post blending pipeline). + +Below are the design considerations while designing the uAPI. + +1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used) +2. Position of the HW block can be programmable +3. LUTs can be one dimentional or three dimentional +4. Number of LUT entries can vary across platforms +5. Precision of LUT entries can vary across platforms +6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic, + Piece-Wise Linear(PWL) etc +7. There can be parameterized/non-parameterized fixed function HW blocks. + e.g. Just a hardware bit, to convert from one color space to another. + +Plane Color Pipeline: Design Proposal +===================================== +Each Color Hardware block will be represented by the structure drm_color_op. + +struct drm_color_op { + enum color_op_block name; + enum color_op_type type; + u32 blob_id; + u32 private_flags; +}; + +The members of which will constitute: +1. name: A standardised enum for the color hardware block +2. type: The type of mathematical operation done by the hardware block. + e.g. 1D Curve, 3D Curve, Matrix etc. +3. blob id: Id pointing to a blob containing information about the hardware + block advertising the respective capabilities to the userspace. + It can be an optional field depending on the members "name" and "type". +4. private_flags: This can be used to provide vendor specific hints + to user space + + + For example to represent LUTs, we introduce the drm_color_lut_range + structure. It can represent LUTs with varied number of entries and + distributions (Multi segmented, Logarithmic etc). + + struct drm_color_lut_range { + /* DRM_MODE_LUT_* */ + __u32 flags; + /* number of points on the curve */ + __u16 count; + /* input/output bits per component */ + __u8 input_bpc, output_bpc; + /* input start/end values */ + __s32 start, end; + /* output min/max values */ + __s32 min, max; + }; + +Note: More details on exact usage and implementation of this structure can be + found in the RFC. This structure is taken as is from the series. + https://patchwork.freedesktop.org/series/90825/ + However, we can add more members to it to encompass all use-cases. + For example. we can add a precision field to advertise the + bitdepth of the LUTs. Similarly, we can reserve some bits in the flag + field for vendor specific use cases. + + At the same time, we don't need to pass any additional information for the + CSC block as userspace and driver already agrees struct drm_color_ctm as + a uAPI to pass on data. + +These color operations will form the building blocks of a color pipeline which +best represents the underlying Hardware. Color operations can be re-arranged, +substracted or added to create distinct color pipelines to accurately describe +the Hardware blocks present in the display engine. + +In this proposal, a color pipeline is represented as an array of +struct drm_color_op. + +struct drm_color_op color_pipeline_1[] + +For example to represent the pre-blending color pipeline as described above + +We can create a color pipeline like this. + +struct drm_color_op color_pipeline_1[] = { + { + .name = DRM_CB_PRE_CSC, + .type = CURVE_1D, + .blob_id = 0; /* actual value to be populated during plane + initialization*/ + }, + { + .name = DRM_CB_CSC, + .type = MATRIX, + .blob_id = 0; + }, + { + .name = DRM_CB_POST_CSC, + .type = CURVE_1D, + .blob_id = 0; + }, +}; + +Then, for individual color operation, we add blobs to advertise the capability +of the particular Hardware block. In case of the example pipeline, we add +blobs of type drm_color_lut_range for the "pre-csc" and "post-csc". +For the "csc" block we pass no blob id to user space as it is known to both +user space and driver that drm_color_ctm structure is to be used for such +operation. + +To represent, this in a diagram. + + struct drm_color_op color_pipeline_1[] + +---------------------------+ + | | drm_color_op + | +---------------------+--+-----------+---------------------+ + | | | | | | + | | | | | +-----------------+ | + | | | | | | name | | + | | | | | +-----------------+ | + | | | | | | type | | + | | color_op_1 | | | +-----------------+ | + | | | | | | blob id | +--------+ + | | | | | +-----------------+ | | + | | | | | | private | | | + | | | | | | flags | | | + | | | | | +-----------------+ | | + | | | | | | | + | +---------------------+--+-----------+---------------------+ | + | | | + | | | + | +---------------------+ | | + | | | | drm_color_lut_range | + | | color_op_2 | | +-------------------------+ | + | | | | | | | + | | | | | +---------------------+ | | + | +---------------------+ | | | segment 1 { | |<---+ + | | | | ... | | + | +---------------------+ | | | .input_bpc = 16, | | + | | | | | | .output_bpc = 16, | | + | | color_op_3 | | | | ... | | + | | | | | | } | | + | | | | | +---------------------+ | + | +---------------------+ | | | + | . | | +---------------------+ | + | . | | | segment 2 { | | + | . | | | ... | | + +---------------------------+ | | } | | + | | | | + | | | | + | | | | + | +---------------------+ | + | . | + | . | + | . | + +-------------------------+ + + + +This color pipeline is then packaged within a blob for the user space to +retrieve it. Details can be found in the next section + +Exposing a color pipeline to user space +======================================= + +To advertise the available color pipelines, an immutable ENUM property +"GET_COLOR_PIPELINE" is introduced. +This enum property has blob id's as values. With each blob id representing +a distinct color pipeline based on underlying HW capabilities and their +respective combinations. + +The following output of drm_info [1], shows how color pipelines are visible +to userspace. + +├───Plane 0 + │ ├───Object ID: 31 + │ ├───CRTCs: {0} + │ ├───Legacy info + ... + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline, + color pipeline 1, color pipeline 2}= no color pipeline + +To understand the capabilities of individual pipelines, first the userspace +need to retrieve the pipeline blob with the blob ids retrieved by reading the +enum property. + +Once the color pipeline is retrieved, user can then parse through +individual drm_color_op blocks to understand the capabilities of each +hardware block. + +Check IGT series to see how user space can parse through color pipelines. +Refer the IGT series here: https://patchwork.freedesktop.org/series/123018/ + +Setting up a color pipeline +=========================== + +Once the user space decides on a color pipeline, it can set the pipeline and +the corresponding data for the hardware blocks within the pipeline with +the BLOB property "SET_COLOR_PIPELINE". + +To achieve this two structures are introduced + +1. struct drm_color_op_data: It represents data to be passed onto individual + color hardware blocks. It contains three members + a) name: to identify the color operation block + b) blob_id: pointing to the blob with data for the + corresponding hardware block + + struct drm_color_op_data { + enum color_op_block name; + u32 blob_id; + }; + +2. struct drm_color_pipeline: This structure represents the aggregate + pipeline to be set. it contains the following members + a) num: pipeline number to be selected + b) size: size of the data to be passed onto the driver + c) data: array of struct drm_color_op_data with data for + the hardware block/s that userspace wants to + set values for. + + struct drm_color_pipeline { + int num; + int size; + struct drm_color_op_data *data; + }; + + User can either: + 1. send data for all the color operations in a pipeline as shown in [2]. + The color operation data need not be in order that the pipeline advertises + however, it should not contain data for any + color operation that is not present in the pipeline. + + Note: This check for pipeline is not yet implemented but if the + wider proposal is accepted we have few solutions in mind. + + 2. send data for a subset of color operations as shown in [3].For the + color operation that userspace does not send data will retain their + older state. + + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob + property to NULL as shown in both [2] and [3] + + 4. change the color pipeline as demonstrated in [3]. + On the new pipeline, the user is expected to setup all the color hardware block + Once the user requests a pipeline change, the driver will provide it a clean slate + which means that all the data previously set by the user will be discarded even if + there are common hardware blocks between the two(previous and current) pipelines. + +IGT implementation can be found here [4] + +Representing Fixed function hardware +==================================== + +To provide support for fixed function hardware, the driver could expose vendor +specific struct drm_color_op with parameters that both the userspace and +driver agrees on. To demonstrate, let's consider a hyphothetical fixed +function hardware block that converts BT601 to BT2020. +The driver can choose to advertise the block as such. + +struct drm_color_op color_pipeline_X[] = { + { + .name = DRM_CB_PRIVATE, + .type = FIXED_FUNCTION, + .blob_id = 45; + }, +} + +Where the blob represents some vendor specific enums, strings or any other +appropriate data types which both the user-space and drivers are aligned on. + +blob:45 { + VENDORXXX_BT602_TO_BT2020, +}; + +For enabling or passing parameters to such blocks, the user can send data +to the driver wrapped within a blob like any other color operation block. + + struct drm_color_op_data { + .name = DRM_CB_PRIVATE; + .blob_id = 46; + } ; + +where blob with id 46 contains data to enable the fixed function hardware(s). + +References +========== + +[1] https://gitlab.freedesktop.org/emersion/drm_info +[2] https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1 +[3] https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1 +[4] https://patchwork.freedesktop.org/series/123018/
Signature · 1 line
2.38.1
Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
Quoted 183 lines · Uma Shankar
Add the documentation for the new proposed Plane Color Pipeline.
Co-developed-by Chaitanya Kumar Borah <[email protected]>Signed-off-by Chaitanya Kumar Borah <[email protected]>Signed-off-by Uma Shankar <[email protected]>--- .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++ 1 file changed, 394 insertions(+)create mode 100644
Documentation/gpu/rfc/plane_color_pipeline.rstDocumentation/gpu/rfc/plane_color_pipeline.rst new file +167
-0@@ -0,0 +1,394 @@ +======================================= + Plane Color Pipeline: A UAPI proposal +======================================= + +To build the proposal on, lets take the premise of a color pipeline as shown +below. + + +-------------------------------------------+ + | RAM | + | +------+ +---------+ +---------+ | + | | FB 1 | | FB 2 | | FB N | | + | +------+ +---------+ +---------+ | + +-------------------------------------------+ + | Plane Color Hardware Block | + +--------------------------------------------+ + | +---v-----+ +---v-------+ +---v------+ | + | | Plane A | | Plane B | | Plane N | | + | | Pre-CSC | | Pre-CSC | | Pre-CSC | | + | +---+-----+ +---+-------+ +---+------+ | + | | | | | + | +---v-----+ +---v-------+ +---v------+ | + | |Plane A | | Plane B | | Plane N | | + | |CSC/CTM | | CSC/CTM | | CSC/CTM | | + | +---+-----+ +----+------+ +----+-----+ | + | | | | | + | +---v-----+ +----v------+ +----v-----+ | + | | Plane A | | Plane B | | Plane N | | + | |Post-CSC | | Post-CSC | | Post-CSC | | + | +---+-----+ +----+------+ +----+-----+ | + | | | | | + +--------------------------------------------+ ++------v--------------v---------------v-------| +|| || +|| Pipe Blender || ++--------------------+------------------------+ +| | | +| +-----------v----------+ | +| | Pipe Pre-CSC | | +| | | | +| +-----------+----------+ | +| | Pipe Color | +| +-----------v----------+ Hardware | +| | Pipe CSC/CTM | | +| | | | +| +-----------+----------+ | +| | | +| +-----------v----------+ | +| | Pipe Post-CSC | | +| | | | +| +-----------+----------+ | +| | | ++---------------------------------------------+ + | + v + Pipe Output + +Each plane consists of the following color blocks + * Pre-CSC : This block can used to linearize the input frame buffer data. + The linear data then can be further acted on by the following + color hardware blocks in the display hardware pipeline + + * CSC/CTM: Used to program color transformation matrix, this block is used + to perform color space conversions like BT2020 to BT709 or BT601 + etc. This block acts on the linearized data coming from the + Pre-CSC HW block. + + * Post-CSC: This HW block can be used to non-linearize frame buffer data to + match the sink. Another use case of it could be to perform Tone + mapping for HDR use-cases. + +Data from multiple planes will then be fed to pipe/crtc where it will get blended. +There is a similar set of HW blocks available at pipe/crtc level which acts on +this blended data. + +Below is a sample usecase fo video playback with sub-titles and playback +controls + +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ +│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ +│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │ +│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐ +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ +(subtitles) │ + │ +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├───┐ │ +│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │ +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ +(Playback controls UI) │ │ + │ │ +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │ +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐ │ │ +│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │ +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │ +(video frame) │ │ │ + │ │ │ +┌────────────────────────────────────────────────────────────────────────┴─┴─┘ +│ +│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐ +│ │ CRTC │ │ CRTC │ │ CRTC │ +└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────► TO Port + │ │ │ │ │ │ + └─────────────┘ └─────────────┘ └───────────────┘ + +This RFC is intended to propose an uAPI for the pre-blending color pipeline +(however, can be also extended to post blending pipeline). + +Below are the design considerations while designing the uAPI. + +1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used) +2. Position of the HW block can be programmable +3. LUTs can be one dimentional or three dimentional +4. Number of LUT entries can vary across platforms +5. Precision of LUT entries can vary across platforms +6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic, + Piece-Wise Linear(PWL) etc +7. There can be parameterized/non-parameterized fixed function HW blocks. + e.g. Just a hardware bit, to convert from one color space to another. + +Plane Color Pipeline: Design Proposal +===================================== +Each Color Hardware block will be represented by the structure drm_color_op. + +struct drm_color_op { + enum color_op_block name; + enum color_op_type type; + u32 blob_id; + u32 private_flags; +}; + +The members of which will constitute: +1. name: A standardised enum for the color hardware block +2. type: The type of mathematical operation done by the hardware block. + e.g. 1D Curve, 3D Curve, Matrix etc. +3. blob id: Id pointing to a blob containing information about the hardware + block advertising the respective capabilities to the userspace. + It can be an optional field depending on the members "name" and "type". +4. private_flags: This can be used to provide vendor specific hints + to user space + + + For example to represent LUTs, we introduce the drm_color_lut_range + structure. It can represent LUTs with varied number of entries and + distributions (Multi segmented, Logarithmic etc). + + struct drm_color_lut_range { + /* DRM_MODE_LUT_* */ + __u32 flags; + /* number of points on the curve */ + __u16 count; + /* input/output bits per component */ + __u8 input_bpc, output_bpc; + /* input start/end values */ + __s32 start, end; + /* output min/max values */ + __s32 min, max; + }; + +Note: More details on exact usage and implementation of this structure can be + found in the RFC. This structure is taken as is from the series. + https://patchwork.freedesktop.org/series/90825/ + However, we can add more members to it to encompass all use-cases. + For example. we can add a precision field to advertise the + bitdepth of the LUTs. Similarly, we can reserve some bits in the flag + field for vendor specific use cases.
Documentation/gpu/rfc/plane_color_pipeline.rst line 177
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ + At the same time, we don't need to pass any additional information for the + CSC block as userspace and driver already agrees struct drm_color_ctm as + a uAPI to pass on data. + +These color operations will form the building blocks of a color pipeline which +best represents the underlying Hardware. Color operations can be re-arranged, +substracted or added to create distinct color pipelines to accurately describe +the Hardware blocks present in the display engine. +
What do you mean when you're talking of re-arranging, or subtracting color operations?
Quoted 77 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +77
-0+In this proposal, a color pipeline is represented as an array of +struct drm_color_op. + +struct drm_color_op color_pipeline_1[] + +For example to represent the pre-blending color pipeline as described above + +We can create a color pipeline like this. + +struct drm_color_op color_pipeline_1[] = { + { + .name = DRM_CB_PRE_CSC, + .type = CURVE_1D, + .blob_id = 0; /* actual value to be populated during plane + initialization*/ + }, + { + .name = DRM_CB_CSC, + .type = MATRIX, + .blob_id = 0; + }, + { + .name = DRM_CB_POST_CSC, + .type = CURVE_1D, + .blob_id = 0; + }, +}; + +Then, for individual color operation, we add blobs to advertise the capability +of the particular Hardware block. In case of the example pipeline, we add +blobs of type drm_color_lut_range for the "pre-csc" and "post-csc". +For the "csc" block we pass no blob id to user space as it is known to both +user space and driver that drm_color_ctm structure is to be used for such +operation. + +To represent, this in a diagram. + + struct drm_color_op color_pipeline_1[] + +---------------------------+ + | | drm_color_op + | +---------------------+--+-----------+---------------------+ + | | | | | | + | | | | | +-----------------+ | + | | | | | | name | | + | | | | | +-----------------+ | + | | | | | | type | | + | | color_op_1 | | | +-----------------+ | + | | | | | | blob id | +--------+ + | | | | | +-----------------+ | | + | | | | | | private | | | + | | | | | | flags | | | + | | | | | +-----------------+ | | + | | | | | | | + | +---------------------+--+-----------+---------------------+ | + | | | + | | | + | +---------------------+ | | + | | | | drm_color_lut_range | + | | color_op_2 | | +-------------------------+ | + | | | | | | | + | | | | | +---------------------+ | | + | +---------------------+ | | | segment 1 { | |<---+ + | | | | ... | | + | +---------------------+ | | | .input_bpc = 16, | | + | | | | | | .output_bpc = 16, | | + | | color_op_3 | | | | ... | | + | | | | | | } | | + | | | | | +---------------------+ | + | +---------------------+ | | | + | . | | +---------------------+ | + | . | | | segment 2 { | | + | . | | | ... | | + +---------------------------+ | | } | | + | | | | + | | | | + | | | | + | +---------------------+ |
Documentation/gpu/rfc/plane_color_pipeline.rst line 264
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ | . | + | . | + | . | + +-------------------------+ + + + +This color pipeline is then packaged within a blob for the user space to +retrieve it. Details can be found in the next section +
Not sure I like blobs that contain other blob ids.
Quoted 113 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +113
-0+Exposing a color pipeline to user space +======================================= + +To advertise the available color pipelines, an immutable ENUM property +"GET_COLOR_PIPELINE" is introduced. +This enum property has blob id's as values. With each blob id representing +a distinct color pipeline based on underlying HW capabilities and their +respective combinations. + +The following output of drm_info [1], shows how color pipelines are visible +to userspace. + +├───Plane 0 + │ ├───Object ID: 31 + │ ├───CRTCs: {0} + │ ├───Legacy info + ... + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline, + color pipeline 1, color pipeline 2}= no color pipeline + +To understand the capabilities of individual pipelines, first the userspace +need to retrieve the pipeline blob with the blob ids retrieved by reading the +enum property. + +Once the color pipeline is retrieved, user can then parse through +individual drm_color_op blocks to understand the capabilities of each +hardware block. + +Check IGT series to see how user space can parse through color pipelines. +Refer the IGT series here: https://patchwork.freedesktop.org/series/123018/ + +Setting up a color pipeline +=========================== + +Once the user space decides on a color pipeline, it can set the pipeline and +the corresponding data for the hardware blocks within the pipeline with +the BLOB property "SET_COLOR_PIPELINE". + +To achieve this two structures are introduced + +1. struct drm_color_op_data: It represents data to be passed onto individual + color hardware blocks. It contains three members + a) name: to identify the color operation block + b) blob_id: pointing to the blob with data for the + corresponding hardware block + + struct drm_color_op_data { + enum color_op_block name; + u32 blob_id; + }; + +2. struct drm_color_pipeline: This structure represents the aggregate + pipeline to be set. it contains the following members + a) num: pipeline number to be selected + b) size: size of the data to be passed onto the driver + c) data: array of struct drm_color_op_data with data for + the hardware block/s that userspace wants to + set values for. + + struct drm_color_pipeline { + int num; + int size; + struct drm_color_op_data *data; + }; + + User can either: + 1. send data for all the color operations in a pipeline as shown in [2]. + The color operation data need not be in order that the pipeline advertises + however, it should not contain data for any + color operation that is not present in the pipeline. + + Note: This check for pipeline is not yet implemented but if the + wider proposal is accepted we have few solutions in mind. + + 2. send data for a subset of color operations as shown in [3].For the + color operation that userspace does not send data will retain their + older state. + + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob + property to NULL as shown in both [2] and [3] + + 4. change the color pipeline as demonstrated in [3]. + On the new pipeline, the user is expected to setup all the color hardware block + Once the user requests a pipeline change, the driver will provide it a clean slate + which means that all the data previously set by the user will be discarded even if + there are common hardware blocks between the two(previous and current) pipelines. + +IGT implementation can be found here [4] + +Representing Fixed function hardware +==================================== + +To provide support for fixed function hardware, the driver could expose vendor +specific struct drm_color_op with parameters that both the userspace and +driver agrees on. To demonstrate, let's consider a hyphothetical fixed +function hardware block that converts BT601 to BT2020. +The driver can choose to advertise the block as such. + +struct drm_color_op color_pipeline_X[] = { + { + .name = DRM_CB_PRIVATE, + .type = FIXED_FUNCTION, + .blob_id = 45; + }, +} + +Where the blob represents some vendor specific enums, strings or any other +appropriate data types which both the user-space and drivers are aligned on. + +blob:45 { + VENDORXXX_BT602_TO_BT2020, +}; +
Documentation/gpu/rfc/plane_color_pipeline.rst line 387
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+For enabling or passing parameters to such blocks, the user can send data +to the driver wrapped within a blob like any other color operation block. + + struct drm_color_op_data { + .name = DRM_CB_PRIVATE; + .blob_id = 46; + } ; + +where blob with id 46 contains data to enable the fixed function hardware(s). +
One major thing missing from the RFC is an explanation on why we want to go with a prescriptive model, rather than a descriptive model. This question came up in Dave's response to Simon's RFC:
https://lore.kernel.org/dri-devel/CAPM=9tz54Jc1HSjdh5A7iG4X8Gvgg46qu7Ezvgnmj4N6gbY+Kw@mail.gmail.com/
This is a rough attempt at such an explanation:
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/diffs?commit_id=bc99737623796b39925767d6f8cbc097ad0b4d8d
Harry
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +7
-0+References +========== + +[1] https://gitlab.freedesktop.org/emersion/drm_info +[2] https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1 +[3] https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1 +[4] https://patchwork.freedesktop.org/series/123018/
RE: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
Quoted 220 lines
-----Original Message-----
From: Harry Wentland <[email protected]>
Sent: Wednesday, August 30, 2023 1:10 AM
To: Shankar, Uma <[email protected]>; [email protected]; dri- [email protected]
Cc: Borah, Chaitanya Kumar <[email protected]>; wayland- [email protected]
Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color PipelineQuoted 90 lines · Uma Shankar
Add the documentation for the new proposed Plane Color Pipeline.
Co-developed-by: Chaitanya Kumar Borah
<[email protected]>Signed-off-by Chaitanya Kumar Borah <[email protected]>Signed-off-by Uma Shankar <[email protected]>--- .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++ 1 file changed, 394 insertions(+)create mode 100644
Documentation/gpu/rfc/plane_color_pipeline.rstDocumentation/gpu/rfc/plane_color_pipeline.rst new file +72
-0@@ -0,0 +1,394 @@ +======================================= + Plane Color Pipeline: A UAPI proposal +======================================= + +To build the proposal on, lets take the premise of a color pipeline +as shown below. + + +-------------------------------------------+ + | RAM | + | +------+ +---------+ +---------+ | + | | FB 1 | | FB 2 | | FB N | | + | +------+ +---------+ +---------+ | + +-------------------------------------------+ + | Plane Color Hardware Block | + +--------------------------------------------+ + | +---v-----+ +---v-------+ +---v------+ | + | | Plane A | | Plane B | | Plane N | | + | | Pre-CSC | | Pre-CSC | | Pre-CSC | | + | +---+-----+ +---+-------+ +---+------+ | + | | | | | + | +---v-----+ +---v-------+ +---v------+ | + | |Plane A | | Plane B | | Plane N | | + | |CSC/CTM | | CSC/CTM | | CSC/CTM | | + | +---+-----+ +----+------+ +----+-----+ | + | | | | | + | +---v-----+ +----v------+ +----v-----+ | + | | Plane A | | Plane B | | Plane N | | + | |Post-CSC | | Post-CSC | | Post-CSC | | + | +---+-----+ +----+------+ +----+-----+ | + | | | | | + +--------------------------------------------+ ++------v--------------v---------------v-------| +|| || +|| Pipe Blender || ++--------------------+------------------------+ +| | | +| +-----------v----------+ | +| | Pipe Pre-CSC | | +| | | | +| +-----------+----------+ | +| | Pipe Color | +| +-----------v----------+ Hardware | +| | Pipe CSC/CTM | | +| | | | +| +-----------+----------+ | +| | | +| +-----------v----------+ | +| | Pipe Post-CSC | | +| | | | +| +-----------+----------+ | +| | | ++---------------------------------------------+ + | + v + Pipe Output + +Each plane consists of the following color blocks + * Pre-CSC : This block can used to linearize the input frame buffer data. + The linear data then can be further acted on by the following + color hardware blocks in the display hardware pipeline + + * CSC/CTM: Used to program color transformation matrix, this block is used + to perform color space conversions like BT2020 to BT709 or BT601 + etc. This block acts on the linearized data coming from the + Pre-CSC HW block. + + * Post-CSC: This HW block can be used to non-linearize frame buffer data to + match the sink. Another use case of it could be to perform Tone + mapping for HDR use-cases. + +Data from multiple planes will then be fed to pipe/crtc where it will get blended. +There is a similar set of HW blocks available at pipe/crtc levelDocumentation/gpu/rfc/plane_color_pipeline.rst line 82
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+which acts on this blended data. + +Below is a sample usecase fo video playback with sub-titles and +playback controls + +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ +│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ +│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │ +│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐ +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘│ > +(subtitles) │ > + │ > +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ > +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ > +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├───┐ │ > +│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │ > +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ > +(Playback controls UI) │ │ > + │ │ > +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ > +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │ > +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐ │ │ > +│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │ > +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │ > +(video frame) │ │ │Documentation/gpu/rfc/plane_color_pipeline.rst line 84
Quoted 3 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +2
-0+ +│ │ │+┌────────────────────────────────────────────────────────────────── ── > +────┴─┴─┘ > +│ > +│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐ > +│ │ CRTC │ │ CRTC │ │ CRTC │ > +└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────► TOPort
Quoted 63 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +63
-0+ │ │ │ │ │ │ + └─────────────┘ └─────────────┘ └───────────────┘ + +This RFC is intended to propose an uAPI for the pre-blending color +pipeline (however, can be also extended to post blending pipeline). + +Below are the design considerations while designing the uAPI. + +1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only +one can be used) 2. Position of the HW block can be programmable 3. +LUTs can be one dimentional or three dimentional 4. Number of LUT +entries can vary across platforms 5. Precision of LUT entries can +vary across platforms 6. Distribution of LUT entries may vary. e.g +Mutli-segmented, Logarithmic, + Piece-Wise Linear(PWL) etc +7. There can be parameterized/non-parameterized fixed function HW blocks. + e.g. Just a hardware bit, to convert from one color space to another. + +Plane Color Pipeline: Design Proposal +===================================== +Each Color Hardware block will be represented by the structure drm_color_op. + +struct drm_color_op { + enum color_op_block name; + enum color_op_type type; + u32 blob_id; + u32 private_flags; +}; + +The members of which will constitute: +1. name: A standardised enum for the color hardware block 2. type: +The type of mathematical operation done by the hardware block. + e.g. 1D Curve, 3D Curve, Matrix etc. +3. blob id: Id pointing to a blob containing information about the hardware + block advertising the respective capabilities to the userspace. + It can be an optional field depending on the members "name" and "type". +4. private_flags: This can be used to provide vendor specific hints + to user space + + + For example to represent LUTs, we introduce the drm_color_lut_range + structure. It can represent LUTs with varied number of entries and + distributions (Multi segmented, Logarithmic etc). + + struct drm_color_lut_range { + /* DRM_MODE_LUT_* */ + __u32 flags; + /* number of points on the curve */ + __u16 count; + /* input/output bits per component */ + __u8 input_bpc, output_bpc; + /* input start/end values */ + __s32 start, end; + /* output min/max values */ + __s32 min, max; + }; + +Note: More details on exact usage and implementation of this structure can be + found in the RFC. This structure is taken as is from the series. + https://patchwork.freedesktop.org/series/90825/ + However, we can add more members to it to encompass all use-cases. + For example. we can add a precision field to advertise the + bitdepth of the LUTs. Similarly, we can reserve some bits in the flagDocumentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ field for vendor specific use cases. + + At the same time, we don't need to pass any additional information for the + CSC block as userspace and driver already agrees struct drm_color_ctm as + a uAPI to pass on data. + +These color operations will form the building blocks of a color +pipeline which best represents the underlying Hardware. Color +operations can be re-arranged, substracted or added to create +distinct color pipelines to accurately describe the Hardware blocks present in thedisplay engine.
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 158
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+What do you mean when you're talking of re-arranging, or subtracting color operations?
I will re-phrase this to avoid confusion. The pipeline can be defined by driver by Putting together the hardware blocks in some order supported by respective hardware. If any new generation changes the behaviour, a new pipeline can be defined and exposed for that particular platform. So pipeline definition is fully flexible but done by driver. Userspace will just be able to use that and cannot alter the same once exposed by driver.
Quoted 84 lines
Quoted 22 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +22
-0+In this proposal, a color pipeline is represented as an array of +struct drm_color_op. + +struct drm_color_op color_pipeline_1[] + +For example to represent the pre-blending color pipeline as described +above + +We can create a color pipeline like this. + +struct drm_color_op color_pipeline_1[] = { + { + .name = DRM_CB_PRE_CSC, + .type = CURVE_1D, + .blob_id = 0; /* actual value to be populated during plane + initialization*/ + }, + { + .name = DRM_CB_CSC, + .type = MATRIX, + .blob_id = 0; + },Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ { + .name = DRM_CB_POST_CSC, + .type = CURVE_1D, + .blob_id = 0; + }, +}; + +Then, for individual color operation, we add blobs to advertise the +capability of the particular Hardware block. In case of the example +pipeline, we add blobs of type drm_color_lut_range for the "pre-csc" and "post-csc".
Quoted 51 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +51
-0+For the "csc" block we pass no blob id to user space as it is known +to both user space and driver that drm_color_ctm structure is to be +used for such operation. + +To represent, this in a diagram. + + struct drm_color_op color_pipeline_1[] + +---------------------------+ + | | drm_color_op + | +---------------------+--+-----------+---------------------+ + | | | | | | + | | | | | +-----------------+ | + | | | | | | name | | + | | | | | +-----------------+ | + | | | | | | type | | + | | color_op_1 | | | +-----------------+ | + | | | | | | blob id | +--------+ + | | | | | +-----------------+ | | + | | | | | | private | | | + | | | | | | flags | | | + | | | | | +-----------------+ | | + | | | | | | | + | +---------------------+--+-----------+---------------------+ | + | | | + | | | + | +---------------------+ | | + | | | | drm_color_lut_range | + | | color_op_2 | | +-------------------------+ | + | | | | | | | + | | | | | +---------------------+ | | + | +---------------------+ | | | segment 1 { | |<---+ + | | | | ... | | + | +---------------------+ | | | .input_bpc = 16, | | + | | | | | | .output_bpc = 16, | | + | | color_op_3 | | | | ... | | + | | | | | | } | | + | | | | | +---------------------+ | + | +---------------------+ | | | + | . | | +---------------------+ | + | . | | | segment 2 { | | + | . | | | ... | | + +---------------------------+ | | } | | + | | | | + | | | | + | | | | + | +---------------------+ | + | . | + | . | + | . | + + +-------------------------+
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 61
Quoted 6 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +6
-0+ + + +This color pipeline is then packaged within a blob for the user space +to retrieve it. Details can be found in the next section +Not sure I like blobs that contain other blob ids.
It provides flexibility and helps with just one interface to userspace. Its easy to handle and manage once we get the hang of it 😊.
We can clearly define the steps of parsing and data structures to be used while interpreting and parsing the blobs.
Quoted 141 lines
Quoted 9 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +9
-0+Exposing a color pipeline to user space +======================================= + +To advertise the available color pipelines, an immutable ENUM +property "GET_COLOR_PIPELINE" is introduced. +This enum property has blob id's as values. With each blob id +representing a distinct color pipeline based on underlying HW +capabilities and their respective combinations. +Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+The following output of drm_info [1], shows how color pipelines are +visible to userspace. + +├───Plane 0 + │ ├───Object ID: 31 + │ ├───CRTCs: {0} + │ ├───Legacy info + ... + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline, + color pipeline 1, color pipeline 2}=no color pipeline
Quoted 14 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +14
-0+ +To understand the capabilities of individual pipelines, first the +userspace need to retrieve the pipeline blob with the blob ids +retrieved by reading the enum property. + +Once the color pipeline is retrieved, user can then parse through +individual drm_color_op blocks to understand the capabilities of each +hardware block. + +Check IGT series to see how user space can parse through color pipelines. +Refer the IGT series here: +https://patchwork.freedesktop.org/series/123018/ + +Setting up a color pipelineDocumentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+=========================== + +Once the user space decides on a color pipeline, it can set the +pipeline and the corresponding data for the hardware blocks within +the pipeline with the BLOB property "SET_COLOR_PIPELINE". + +To achieve this two structures are introduced + +1. struct drm_color_op_data: It represents data to be passed onto individual + color hardware blocks. Itcontains three members
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 13 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +13
-0+ a) name: to identify the color operation block + b) blob_id: pointing to the blob with data for the + corresponding hardware block + + struct drm_color_op_data { + enum color_op_block name; + u32 blob_id; + }; + +2. struct drm_color_pipeline: This structure represents the aggregate + pipeline to be set. it contains the following members + a) num: pipeline number to be selected + b) size: size of the data to be passed ontothe driver
Documentation/gpu/rfc/plane_color_pipeline.rst line 304
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ c) data: array of struct
drm_color_op_datawith data forQuoted 17 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +17
-0+ the hardware block/s that userspace wants to + set values for. + + struct drm_color_pipeline { + int num; + int size; + struct drm_color_op_data *data; + }; + + User can either: + 1. send data for all the color operations in a pipeline as shown in [2]. + The color operation data need not be in order that the pipeline advertises + however, it should not contain data for any + color operation that is not present in the pipeline. + + Note: This check for pipeline is not yet implemented but if the + wider proposal is accepted we have few solutions in mind.Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ + 2. send data for a subset of color operations as shown in [3].For the + color operation that userspace does not send data will retain their + older state. + + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob + property to NULL as shown in both [2] and [3] + + 4. change the color pipeline as demonstrated in [3]. + On the new pipeline, the user is expected to setup all the color hardwareblock
Documentation/gpu/rfc/plane_color_pipeline.rst line 332
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ Once the user requests a pipeline change, the driver will provide it a cleanslate
Documentation/gpu/rfc/plane_color_pipeline.rst line 333
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ which means that all the data previously set by the user will be discardedeven if
Documentation/gpu/rfc/plane_color_pipeline.rst line 334
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ there are common hardware blocks between the two(previous and current)pipelines.
Quoted 12 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +12
-0+ +IGT implementation can be found here [4] + +Representing Fixed function hardware +==================================== + +To provide support for fixed function hardware, the driver could +expose vendor specific struct drm_color_op with parameters that both +the userspace and driver agrees on. To demonstrate, let's consider a +hyphothetical fixed function hardware block that converts BT601 to BT2020. +The driver can choose to advertise the block as such. +Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+struct drm_color_op color_pipeline_X[] = { + { + .name = DRM_CB_PRIVATE, + .type = FIXED_FUNCTION, + .blob_id = 45; + }, +} + +Where the blob represents some vendor specific enums, strings or any +other appropriate data types which both the user-space and drivers are alignedon.
Quoted 5 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +5
-0+ +blob:45 { + VENDORXXX_BT602_TO_BT2020, +}; +Documentation/gpu/rfc/plane_color_pipeline.rst line 387
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+For enabling or passing parameters to such blocks, the user can send +data to the driver wrapped within a blob like any other color operation block. + + struct drm_color_op_data { + .name = DRM_CB_PRIVATE; + .blob_id = 46; + } ; + +where blob with id 46 contains data to enable the fixed function hardware(s). +One major thing missing from the RFC is an explanation on why we want to go with a prescriptive model, rather than a descriptive model. This question came up in Dave's response to Simon's RFC:
https://lore.kernel.org/dri-
devel/CAPM=9tz54Jc1HSjdh5A7iG4X8Gvgg46qu7Ezvgnmj4N6gbY+Kw@mail.gmail.co m/
Quoted 4 lines
This is a rough attempt at such an explanation:
https://gitlab.freedesktop.org/hwentland/linux/-
/merge_requests/5/diffs?commit_id=bc99737623796b39925767d6f8cbc097ad0b4d 8d
Sure Harry, totally agree to this and will include in documentation to highlight the rationale of going with prescriptive model.
Regards,
Uma Shankar
Quoted 11 lines
Harry
Quoted 9 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +9
-0+References +========== + +[1] https://gitlab.freedesktop.org/emersion/drm_info +[2] +https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1 +[3] +https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1 +[4] https://patchwork.freedesktop.org/series/123018/
Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
Quoted 24 lines · Uma
Quoted 24 lines
-----Original Message-----
From: Harry Wentland <[email protected]>
Sent: Wednesday, August 30, 2023 1:10 AM
To: Shankar, Uma <[email protected]>; [email protected]; dri- [email protected]
Cc: Borah, Chaitanya Kumar <[email protected]>; wayland- [email protected]
Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color PipelineQuoted 11 lines · Uma Shankar
Add the documentation for the new proposed Plane Color Pipeline.
Co-developed-by: Chaitanya Kumar Borah
<[email protected]>Signed-off-by Chaitanya Kumar Borah <[email protected]>Signed-off-by Uma Shankar <[email protected]>--- .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++ 1 file changed, 394 insertions(+)create mode 100644
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 6 lines
Quoted 6 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 6 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +0
-0
...
Hi Uma!
Quoted 11 lines
Quoted 5 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 3 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +3
-0+This color pipeline is then packaged within a blob for the user space +to retrieve it. Details can be found in the next section +Not sure I like blobs that contain other blob ids.
It provides flexibility and helps with just one interface to userspace. Its easy to handle and manage once we get the hang of it 😊.
We can clearly define the steps of parsing and data structures to be used while interpreting and parsing the blobs.
Don't forget extendability. Possibly every single struct will need some kind of versioning, and then it's not simple to parse anymore. Add to that new/old kernel vs. old/new userspace, and it seems a bit nightmarish to design.
Also since it's records inside a single blob, it's like a new file
format: every record needs a standard header that allows skipping it appropriately if userspace does not understand it, or you need a standard index telling where everything is. Making all records the same size would waste space, and extendability requires variable size.
I also would not assume that we can declare a standard set of blocks and that nothing else will be needed. The existing hardware is too diverse for that from what I have understood. I assume that some hardware have blocks unique to them, and they want to at least expose that functionality through a UAPI that allows at least generic enumeration of functionality, even if it needs specialized userspace code to actually make use of.
If we go with
+struct drm_color_op {
+ enum color_op_block name;
+ enum color_op_type type;
+ u32 blob_id;
+ u32 private_flags;
+};
as in your proposal, I believe it can work (sorry, looking further down, I have assumed too much of 'type'), but the enumerations will become long, and the details blob_id is still specific to 'type'. This is unavoidable, but we can still choose the form between blobs and properties, integers and strings.
I have a feeling that introspection will be valuable here, to help people understand what their hardware could do if they had the code to use it. 'name' and 'type' being integers require a translation table to strings before they are readable, and it would be best if the kernel itself provided that translation.
I don't understand how 'private_flags' could be useful. There must not be any "hidden" features. Everything a block can be programmed to do via this UAPI must be clearly documented, there cannot be anything private. If two hardware versions of a block differ in a meaningful or significant way, they need to be exposed as different types of blocks.
OTOH, if one goes with a (new) DRM object with string named properties model, all that struct versioning and file format hassle has mostly a clear and well-understood solution. We only need to define the rules of how userspace needs to deal with properties or values it does not understand, so that the kernel can keep adding more.
Therefore, I'm not yet convinced with the "all blobs" design.
Quoted 45 lines
Quoted 45 lines
Quoted 9 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +9
-0+Exposing a color pipeline to user space +======================================= + +To advertise the available color pipelines, an immutable ENUM +property "GET_COLOR_PIPELINE" is introduced. +This enum property has blob id's as values. With each blob id +representing a distinct color pipeline based on underlying HW +capabilities and their respective combinations. +Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+The following output of drm_info [1], shows how color pipelines are +visible to userspace. + +├───Plane 0 + │ ├───Object ID: 31 + │ ├───CRTCs: {0} + │ ├───Legacy info + ... + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline, + color pipeline 1, color pipeline 2}=no color pipeline
Quoted 14 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +14
-0+ +To understand the capabilities of individual pipelines, first the +userspace need to retrieve the pipeline blob with the blob ids +retrieved by reading the enum property. + +Once the color pipeline is retrieved, user can then parse through +individual drm_color_op blocks to understand the capabilities of each +hardware block. + +Check IGT series to see how user space can parse through color pipelines. +Refer the IGT series here: +https://patchwork.freedesktop.org/series/123018/ + +Setting up a color pipelineDocumentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+=========================== + +Once the user space decides on a color pipeline, it can set the +pipeline and the corresponding data for the hardware blocks within +the pipeline with the BLOB property "SET_COLOR_PIPELINE". + +To achieve this two structures are introduced + +1. struct drm_color_op_data: It represents data to be passed onto individual + color hardware blocks. Itcontains three members
Quoted 6 lines
Quoted 6 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 312
Quoted 6 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +6
-0+ a) name: to identify the color operation block + b) blob_id: pointing to the blob with data for the + corresponding hardware block + + struct drm_color_op_data { + enum color_op_block name;
Why is this a global fixed enum rather than a pipeline specific ordinal or a unique-per-device ID?
There is no reason to believe that a 'name' always matches a hardware block 1:1. When drivers accumulate multiple different alternative pipelines due to backwards-compatibility reasons, the same 'name' could be implemented by different hardware blocks, or the same hardware block could implement different 'name's from different pipelines.
The names have also a problem. If you name something "pre-csc", then how do you name the thing that the next hardware version adds between "pre-csc" and "csc"?
Quoted 23 lines
Quoted 23 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +7
-0+ u32 blob_id; + }; + +2. struct drm_color_pipeline: This structure represents the aggregate + pipeline to be set. it contains the following members + a) num: pipeline number to be selected + b) size: size of the data to be passed ontothe driver
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ c) data: array of struct
drm_color_op_datawith data forQuoted 13 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +13
-0+ the hardware block/s that userspace wants to + set values for. + + struct drm_color_pipeline { + int num; + int size; + struct drm_color_op_data *data; + }; + + User can either: + 1. send data for all the color operations in a pipeline as shown in [2]. + The color operation data need not be in order that the pipeline advertises + however, it should not contain data for any
Quoted 8 lines
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 341
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +8
-0+ color operation that is not present in the pipeline. + + Note: This check for pipeline is not yet implemented but if the + wider proposal is accepted we have few solutions in mind. + + 2. send data for a subset of color operations as shown in [3].For the + color operation that userspace does not send data will retain their + older state.
Retaining existing state, especially for operations that userspace does not understand, can lead to incorrect and unexpected results. That's why I say that userspace must understand all operations in a pipeline, and all parameters of all used operations before it can actually use that pipeline.
Otherwise we have the same problem as KMS properties have in general
today: when new things are added that userspace does not understand, how will the userspace be able to maintain its old behaviour?
That question has two answers today:
- userspace learns to program everything, and accidentally (mal)functions until then
- you do not switch between KMS clients that might leave incorrect state in not-understood properties
Neither is a good answer, and the problem does not seem to have any practical traction either.
For color pipelines I hope we can, and believe that we must, do better to maintain correct behaviour while extending functionality.
Quoted 4 lines
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 345
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +4
-0+ + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob + property to NULL as shown in both [2] and [3] +
Is it a reset and disable, or only disable?
How is the reset state defined, if that state becomes active when the pipeline is next enabled and data not set for the operations?
Quoted 10 lines
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 2 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +2
-0+ 4. change the color pipeline as demonstrated in [3]. + On the new pipeline, the user is expected to setup all the color hardwareblock
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ Once the user requests a pipeline change, the driver will provide it a cleanslate
Quoted 1 line
+ which means that all the data previously set by the user will be discarded
even if
Quoted 1 line
+ there are common hardware blocks between the two(previous and current)
pipelines.
Quoted 1 line
+
Yes, alternative pipelines need to be completely independent.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +6
-0+IGT implementation can be found here [4] + +Representing Fixed function hardware +==================================== + +To provide support for fixed function hardware, the driver could
Quoted 8 lines
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 365
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +8
-0+expose vendor specific struct drm_color_op with parameters that both +the userspace and driver agrees on. To demonstrate, let's consider a +hyphothetical fixed function hardware block that converts BT601 to BT2020. +The driver can choose to advertise the block as such. + +struct drm_color_op color_pipeline_X[] = { + { + .name = DRM_CB_PRIVATE,
What if the hardware has 5 different custom blocks like this, multiple in the same pipeline. How do you 'name' them?
Quoted 1 line
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst line 366
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ .type = FIXED_FUNCTION,
I have been assuming that 'type' uniquely defines both the operation and the contents of the parameter blob, but this does not look like it. What defines the operation and the parameters?
Quoted 7 lines
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 6 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +6
-0+ .blob_id = 45; + }, +} + +Where the blob represents some vendor specific enums, strings or any +other appropriate data types which both the user-space and drivers are alignedon.
We have a word for that "data ... aligned on": UAPI.
Quoted 3 lines
Quoted 3 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 375
Quoted 3 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +3
-0+ +blob:45 { + VENDORXXX_BT602_TO_BT2020,
Repeating the question from above, how will userspace know that the contents of this blob need to be VENDORXXX_BT602_TO_BT2020 and not something else?
Quoted 20 lines
Quoted 20 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 387
Quoted 12 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +12
-0+}; + +For enabling or passing parameters to such blocks, the user can send +data to the driver wrapped within a blob like any other color operation block. + + struct drm_color_op_data { + .name = DRM_CB_PRIVATE; + .blob_id = 46; + } ; + +where blob with id 46 contains data to enable the fixed function hardware(s). +One major thing missing from the RFC is an explanation on why we want to go with a prescriptive model, rather than a descriptive model. This question came up in Dave's response to Simon's RFC:
https://lore.kernel.org/dri-
devel/CAPM=9tz54Jc1HSjdh5A7iG4X8Gvgg46qu7Ezvgnmj4N6gbY+Kw@mail.gmail.co m/
Quoted 4 lines
Quoted 4 lines
This is a rough attempt at such an explanation:
https://gitlab.freedesktop.org/hwentland/linux/-
/merge_requests/5/diffs?commit_id=bc99737623796b39925767d6f8cbc097ad0b4d 8d
Hey Harry, that's a good piece!
Quoted 3 lines
Sure Harry, totally agree to this and will include in documentation to highlight the rationale of going with prescriptive model.
Uma, the cover letter had descriptive and prescriptive mixed up.
Thanks,
pq
Quoted 11 lines
Quoted 11 lines
Harry
Quoted 9 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +9
-0+References +========== + +[1] https://gitlab.freedesktop.org/emersion/drm_info +[2] +https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1 +[3] +https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1 +[4] https://patchwork.freedesktop.org/series/123018/
RE: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
Quoted 51 lines
-----Original Message-----
From: dri-devel <[email protected]> On Behalf Of Pekka Paalanen
Sent: Wednesday, August 30, 2023 5:59 PM
To: Shankar, Uma <[email protected]>
Cc: [email protected]; Borah, Chaitanya Kumar <[email protected]>; [email protected]; wayland- [email protected]
Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color PipelineQuoted 32 lines · Uma
Quoted 32 lines
-----Original Message-----
From: Harry Wentland <[email protected]>
Sent: Wednesday, August 30, 2023 1:10 AM
To: Shankar, Uma <[email protected]>; [email protected]; dri- [email protected]
Cc: Borah, Chaitanya Kumar <[email protected]>; wayland- [email protected]
Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color PipelineQuoted 19 lines · Uma Shankar
Add the documentation for the new proposed Plane Color Pipeline.
Co-developed-by: Chaitanya Kumar Borah
<[email protected]>
Signed-off-by: Chaitanya Kumar Borah
<[email protected]>Signed-off-by Uma Shankar <[email protected]>--- .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++ 1 file changed, 394 insertions(+)create mode 100644
Documentation/gpu/rfc/plane_color_pipeline.rstDocumentation/gpu/rfc/plane_color_pipeline.rst new file +0
-0...
Quoted 1 line
Hi Uma!
Thanks Pekka for the feedback and useful inputs.
Quoted 9 lines
Quoted 9 lines
Quoted 5 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 3 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +3
-0+This color pipeline is then packaged within a blob for the user +space to retrieve it. Details can be found in the next section +Not sure I like blobs that contain other blob ids.
It provides flexibility and helps with just one interface to userspace. Its easy to handle and manage once we get the hang of it 😊.
Quoted 6 lines
Quoted 2 lines
We can clearly define the steps of parsing and data structures to be used while interpreting and parsing the blobs.
Don't forget extendability. Possibly every single struct will need some kind of versioning, and then it's not simple to parse anymore. Add to that new/old kernel vs. old/new userspace, and it seems a bit nightmarish to design.
Structure to be used to interpret the blob should be defined as UAPI only and is not expected to change once agreed upon. It should be interpreted like a standard property. So structure to be used, say for 3dLut or 1dlut or CTM operations should be standardized and fixed. No versioning of structure should be done and same rules/restrictions as of UAPI property should be applied.
New vs old userspace problem exists even today as you rightly highlighted in mail below, however we are planning to propose that we clean the hardware state once the userspace client switches or same client switches the pipeline.
Quoted 5 lines
Also since it's records inside a single blob, it's like a new file
format: every record needs a standard header that allows skipping it appropriately if userspace does not understand it, or you need a standard index telling where everything is. Making all records the same size would waste space, and extendability requires variable size.
The design currently implements 1 hardware block by a struct drm_color_op data structure. Multiple such blocks make the pipeline. So userspace just needs to get the pipeline and then parse blocks 1 by 1. For blocks which it doesn't understand it can just skip and move to the next one. Each block is differentiated by a unique "name" standardized by an enum which will be part of the UAPI. Thus we will have scope for variable size blob to represent the particular hardware pipeline, userspace can parse and implement whichever blocks it understands. Only rule defined by UAPI is the way the respective block is to be parsed and programmed.
Quoted 6 lines
I also would not assume that we can declare a standard set of blocks and that nothing else will be needed. The existing hardware is too diverse for that from what I have understood. I assume that some hardware have blocks unique to them, and they want to at least expose that functionality through a UAPI that allows at least generic enumeration of functionality, even if it needs specialized userspace code to actually make use of.
Yeah, this is right and for that reason we came up with an idea of DRM_CB_PRIVATE name for the hardware block. This will tell userspace that this is private hardware block for a particular hardware vendor. Generic userspace will ignore this block. Vendor specific HAL or compositor implementation can parse and use this block. To interpret the blob_id and assign to a respective data structure, private_flags will be used. These private_flags will be agreed upon by HAL and vendor implementation, generic userspace will ignore them. For all other cases, private_flags is a don't care field.
Quoted 14 lines
If we go with
Documentation/gpu/rfc/plane_color_pipeline.rst new file +6
-0+struct drm_color_op { + enum color_op_block name; + enum color_op_type type; + u32 blob_id; + u32 private_flags; +};as in your proposal, I believe it can work (sorry, looking further down, I have assumed too much of 'type'), but the enumerations will become long, and the details
blob_idis still specific to 'type'. This is unavoidable, but we can still choose the form between blobs and properties, integers and strings.
Quoted 4 lines
I have a feeling that introspection will be valuable here, to help people understand what their hardware could do if they had the code to use it. 'name' and 'type' being integers require a translation table to strings before they are readable, and it would be best if the kernel itself provided that translation.
Name and type can be standardized in enum and well documented in the UAPI. For all the standard hardware blocks common for all vendors and serving most of the common usecases, we can define standard names in enum. These can be easily interpreted by a table as done in many cases already in driver and userspace.
Quoted 5 lines
I don't understand how '
private_flags' could be useful. There must not be any "hidden" features. Everything a block can be programmed to do via this UAPI must be clearly documented, there cannot be anything private. If two hardware versions of a block differ in a meaningful or significant way, they need to be exposed as different types of blocks.
The idea of private_flags partially explained above is to have an option for vendors to expose hardware blocks unique or custom to their hardware. private_flags can help differentiate incase multiple such hardware blocks exists on the platform.
For how we intend to use it, please refer below patch in the series:
https://patchwork.freedesktop.org/patch/554928/?series=123024&rev=1
Also, IGT showing the usage:
https://patchwork.freedesktop.org/patch/554831/?series=123018&rev=1
Quoted 5 lines
OTOH, if one goes with a (new) DRM object with string named properties model, all that struct versioning and file format hassle has mostly a clear and well- understood solution. We only need to define the rules of how userspace needs to deal with properties or values it does not understand, so that the kernel can keep adding more.
With the current blob approach with fixed data structure for parsing, we can generalize all the hardware blocks and custom implementations. It is future proof with scope to add any new hardware block in future as well. Rules to interpret, parse, program the hardware can be defined in the blob approach as well and ideally should be defined and fixed in the UAPI documentation.
Quoted 1 line
Therefore, I'm not yet convinced with the "all blobs" design.
Looking forward to collaboratively solve the problem for the community. Will improve the design based on all the feedback.
Quoted 59 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+Exposing a color pipeline to user space +======================================= + +To advertise the available color pipelines, an immutable ENUM +property "GET_COLOR_PIPELINE" is introduced. +This enum property has blob id's as values. With each blob id +representing a distinct color pipeline based on underlying HW +capabilities and their respective combinations. + +The following output of drm_info [1], shows how color pipelinesQuoted 8 lines
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +8
-0+are visible to userspace. + +├───Plane 0 + │ ├───Object ID: 31 + │ ├───CRTCs: {0} + │ ├───Legacy info + ... + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no colorpipeline,
Quoted 1 line
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ color pipeline 1, color pipeline2}=
Quoted 16 lines
Quoted 16 lines
no color pipeline
Quoted 15 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +15
-0+ +To understand the capabilities of individual pipelines, first the +userspace need to retrieve the pipeline blob with the blob ids +retrieved by reading the enum property. + +Once the color pipeline is retrieved, user can then parse through +individual drm_color_op blocks to understand the capabilities of +each hardware block. + +Check IGT series to see how user space can parse through color pipelines. +Refer the IGT series here: +https://patchwork.freedesktop.org/series/123018/ + +Setting up a color pipeline +===========================Quoted 8 lines
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +8
-0+ +Once the user space decides on a color pipeline, it can set the +pipeline and the corresponding data for the hardware blocks +within the pipeline with the BLOB property "SET_COLOR_PIPELINE". + +To achieve this two structures are introduced + +1. struct drm_color_op_data: It represents data to be passed ontoindividual
Quoted 1 line
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ color hardwareblocks. It
Quoted 7 lines
Quoted 7 lines
contains three members
Documentation/gpu/rfc/plane_color_pipeline.rst line 312
Quoted 6 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +6
-0+ a) name: to identify the color operation block + b) blob_id: pointing to the blob with data for the + corresponding hardware block + + struct drm_color_op_data { + enum color_op_block name;Why is this a global fixed enum rather than a pipeline specific ordinal or a unique- per-device ID?
Quoted 6 lines
There is no reason to believe that a 'name' always matches a hardware block 1:1. When drivers accumulate multiple different alternative pipelines due to backwards-compatibility reasons, the same 'name' could be implemented by different hardware blocks, or the same hardware block could implement different 'name's from different pipelines.
We have some fixed hardware operations in the color pipeline, its good to define and identify them with name as identifier. For any other custom block we can use the PRIVATE type. Driver should expose the name of the hardware block for a particular pipeline based on the actual usage, and as long as that is standardized it should be ok if same hardware block is named differently across pipeline and vice-versa.
Quoted 3 lines
The names have also a problem. If you name something "pre-csc", then how do you name the thing that the next hardware version adds between "pre-csc" and "csc"?
Most of the common hardware blocks can be named based on the operation they are intended to be used for. For some custom blocks it can be defined as PRIVATE. But this area can surely be improved based on the suggestion to make it more generic.
Quoted 26 lines
Quoted 4 lines
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +4
-0+ u32 blob_id; + }; + +2. struct drm_color_pipeline: This structure represents theaggregate
Quoted 2 lines
Quoted 2 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 2 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +2
-0+ pipeline to be set. it contains the following members + a) num: pipeline number to beselected
Quoted 1 line
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ b) size: size of the data to be passedonto
Quoted 15 lines
Quoted 15 lines
the driver
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ c) data: array of struct
drm_color_op_datawith data forDocumentation/gpu/rfc/plane_color_pipeline.rst
Quoted 12 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +12
-0+ the hardware block/s that userspace wants to + set values for. + + struct drm_color_pipeline { + int num; + int size; + struct drm_color_op_data *data; + }; + + User can either: + 1. send data for all the color operations in a pipeline as shown in [2]. + The color operation data need not be in order that the pipelineadvertises
Quoted 14 lines
Quoted 9 lines
Quoted 9 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 341
Quoted 9 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +9
-0+ however, it should not contain data for any + color operation that is not present in the pipeline. + + Note: This check for pipeline is not yet implemented but if the + wider proposal is accepted we have few solutions in mind. + + 2. send data for a subset of color operations as shown in [3].For the + color operation that userspace does not send data will retain their + older state.Retaining existing state, especially for operations that userspace does not understand, can lead to incorrect and unexpected results. That's why I say that userspace must understand all operations in a pipeline, and all parameters of all used operations before it can actually use that pipeline.
By retaining state here, we mean the state set by the same client while using the same pipeline. If client wants to just alter one or subset of the hardware blocks in the pipeline, it can just send that to driver. Rest of the pipeline which was previously programmed by the same client will remain intact.
However once the client switches pipeline, driver will disable the pipeline and client will have to program all the blocks again with the new pipeline.
Quoted 10 lines
Otherwise we have the same problem as KMS properties have in general
today: when new things are added that userspace does not understand, how will the userspace be able to maintain its old behaviour?That question has two answers today:
- userspace learns to program everything, and accidentally (mal)functions until then
- you do not switch between KMS clients that might leave incorrect state in not-understood properties
Quoted 5 lines
Neither is a good answer, and the problem does not seem to have any practical traction either.
For color pipelines I hope we can, and believe that we must, do better to maintain correct behaviour while extending functionality.
Yes agree, we are thinking to reset and disable the pipeline once client switches. One of the ideas could be to use file_priv to achieve that.
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 345
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +4
-0+ + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob + property to NULL as shown in both [2] and [3] +Is it a reset and disable, or only disable?
It is reset and disable.
Quoted 2 lines
How is the reset state defined, if that state becomes active when the pipeline is next enabled and data not set for the operations?
Idea is to set NULL to all blobs in the pipeline and disable the hardware blocks. New pipeline should allocate new blobs and enable them back.
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 3 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +3
-0+ 4. change the color pipeline as demonstrated in [3]. + On the new pipeline, the user is expected to setup all the +color hardwareblock
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 2 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +2
-0+ Once the user requests a pipeline change, the driver will +provide it a cleanslate
Quoted 2 lines
+ which means that all the data previously set by the
+ user will be discarded
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
even if
Quoted 2 lines
+ there are common hardware blocks between the
+ two(previous and current)pipelines.
Quoted 1 line
+
Yes, alternative pipelines need to be completely independent.
Agree
Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 9 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +9
-0+IGT implementation can be found here [4] + +Representing Fixed function hardware +==================================== + +To provide support for fixed function hardware, the driver could +expose vendor specific struct drm_color_op with parameters that +both the userspace and driver agrees on. To demonstrate, let's +consider a hyphothetical fixed function hardware block that converts BT601to BT2020.
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 365
Quoted 5 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +5
-0+The driver can choose to advertise the block as such. + +struct drm_color_op color_pipeline_X[] = { + { + .name = DRM_CB_PRIVATE,What if the hardware has 5 different custom blocks like this, multiple in the same pipeline. How do you 'name' them?
Partially explained above, but private_flags can help in differentiating this. A link for implementation shared above for reference.
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst line 366
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ .type = FIXED_FUNCTION,I have been assuming that 'type' uniquely defines both the operation and the contents of the parameter blob, but this does not look like it. What defines the operation and the parameters?
Statement is true for all other generic blocks, only for private blocks this is a bit different. Here interpretation depends on the private_flags which can be documented by the respective vendor for the custom HAL implementation.
Quoted 10 lines
Quoted 8 lines
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +7
-0+ .blob_id = 45; + }, +} + +Where the blob represents some vendor specific enums, strings or +any other appropriate data types which both the user-space and +drivers are alignedon.
We have a word for that "data ... aligned on": UAPI.
Oh ok.
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 375
Quoted 3 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +3
-0+ +blob:45 { + VENDORXXX_BT602_TO_BT2020,Repeating the question from above, how will userspace know that the contents of this blob need to be
VENDORXXX_BT602_TO_BT2020and not something else?
Explained above.
Quoted 32 lines
Quoted 4 lines
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +4
-0+}; + +For enabling or passing parameters to such blocks, the user can +send data to the driver wrapped within a blob like any other coloroperation block.
Quoted 7 lines
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +7
-0+ + struct drm_color_op_data { + .name = DRM_CB_PRIVATE; + .blob_id = 46; + } ; + +where blob with id 46 contains data to enable the fixed functionhardware(s).
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
+
One major thing missing from the RFC is an explanation on why we want to go with a prescriptive model, rather than a descriptive model. This question came up in Dave's response to Simon's RFC:
https://lore.kernel.org/dri-devel/[email protected] l
Quoted 6 lines
Quoted 6 lines
.co
m/This is a rough attempt at such an explanation:
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/diffs?commit_id=bc99737623796b39925767d6f8cbc097ad
Quoted 2 lines
Quoted 2 lines
0b4d
8d
Quoted 7 lines
Hey Harry, that's a good piece!
Quoted 3 lines
Sure Harry, totally agree to this and will include in documentation to highlight the rationale of going with prescriptive model.
Uma, the cover letter had descriptive and prescriptive mixed up.
Agree, thanks for pointing out. Will fix the documentation.
Thanks Pekka for all the pointers.
Regards,
Uma Shankar
Quoted 16 lines
Thanks,
pqQuoted 12 lines
Quoted 12 lines
Harry
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+References +========== + +[1] https://gitlab.freedesktop.org/emersion/drm_info +[2] +https://patchwork.freedesktop.org/patch/554827/?series=123018&rev +=1 +[3] +https://patchwork.freedesktop.org/patch/554826/?series=123018&rev +=1 [4] https://patchwork.freedesktop.org/series/123018/
Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
Quoted 49 lines · Uma
Quoted 49 lines
-----Original Message-----
From: dri-devel <[email protected]> On Behalf Of Pekka Paalanen
Sent: Wednesday, August 30, 2023 5:59 PM
To: Shankar, Uma <[email protected]>
Cc: [email protected]; Borah, Chaitanya Kumar <[email protected]>; [email protected]; wayland- [email protected]
Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color PipelineQuoted 32 lines · Uma
Quoted 32 lines
-----Original Message-----
From: Harry Wentland <[email protected]>
Sent: Wednesday, August 30, 2023 1:10 AM
To: Shankar, Uma <[email protected]>; [email protected]; dri- [email protected]
Cc: Borah, Chaitanya Kumar <[email protected]>; wayland- [email protected]
Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color PipelineQuoted 19 lines · Uma Shankar
Add the documentation for the new proposed Plane Color Pipeline.
Co-developed-by: Chaitanya Kumar Borah
<[email protected]>
Signed-off-by: Chaitanya Kumar Borah
<[email protected]>Signed-off-by Uma Shankar <[email protected]>--- .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++ 1 file changed, 394 insertions(+)create mode 100644
Documentation/gpu/rfc/plane_color_pipeline.rstDocumentation/gpu/rfc/plane_color_pipeline.rst new file +0
-0...
Quoted 3 lines
Quoted 1 line
Hi Uma!
Thanks Pekka for the feedback and useful inputs.
Hi Uma,
sorry to say, but the overall feeling I get from this proposal is that it is just the current color related KMS properties wrapped in a pipeline blob. That is not the re-design I believe we are looking for, and the feeling is based on several details that are just copied from the current property design. Also the "private" stuff has to go.
All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying LUT tap distribution, and parametrized curves are good development, but right now we are looking at things one step higher level: the overall color pipeline design and how to represent any operation. Most of this series is considering details below the current attention level, hence I'm paying attention only to the first few patches.
More below.
Quoted 16 lines
Quoted 9 lines
Quoted 9 lines
Quoted 5 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 3 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +3
-0+This color pipeline is then packaged within a blob for the user +space to retrieve it. Details can be found in the next section +Not sure I like blobs that contain other blob ids.
It provides flexibility and helps with just one interface to userspace. Its easy to handle and manage once we get the hang of it 😊.
Quoted 6 lines
Quoted 2 lines
We can clearly define the steps of parsing and data structures to be used while interpreting and parsing the blobs.
Don't forget extendability. Possibly every single struct will need some kind of versioning, and then it's not simple to parse anymore. Add to that new/old kernel vs. old/new userspace, and it seems a bit nightmarish to design.
Quoted 5 lines
Structure to be used to interpret the blob should be defined as UAPI only and is not expected to change once agreed upon. It should be interpreted like a standard property. So structure to be used, say for 3dLut or 1dlut or CTM operations should be standardized and fixed. No versioning of structure should be done and same rules/restrictions as of UAPI property should be applied.
That sounds like a UAPI that cannot be extended, either. So in a few years we'd be looking at replacing it. Or maybe you are just re-inventing the KMS object property system as blobs?
Replacing a single KMS property is much easier than replacing the foundations of the whole color pipeline design.
Quoted 11 lines
...
Quoted 4 lines
I have a feeling that introspection will be valuable here, to help people understand what their hardware could do if they had the code to use it. 'name' and 'type' being integers require a translation table to strings before they are readable, and it would be best if the kernel itself provided that translation.
Name and type can be standardized in enum and well documented in the UAPI. For all the standard hardware blocks common for all vendors and serving most of the common usecases, we can define standard names in enum. These can be easily interpreted by a table as done in many cases already in driver and userspace.
Yeah, but it won't help with the type-specific parameter blobs that are totally custom per each operation type. With the KMS property system we could have more generic introspection into those as well, at least naming all the parameters they have.
Plus, you don't have to patch drm_info every time something new gets added in the kernel in order to see it named.
Quoted 6 lines
...
Quoted 1 line
Therefore, I'm not yet convinced with the "all blobs" design.
Looking forward to collaboratively solve the problem for the community. Will improve the design based on all the feedback.
I have the feeling that we should also see Harry's draft.
Intel's special LUT type is a whole another story, and we can always have that fitted with any UAPI paradigm.
Quoted 61 lines
...
Quoted 26 lines
Quoted 4 lines
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +4
-0+ u32 blob_id; + }; + +2. struct drm_color_pipeline: This structure represents theaggregate
Quoted 2 lines
Quoted 2 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 2 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +2
-0+ pipeline to be set. it contains the following members + a) num: pipeline number to beselected
Quoted 1 line
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ b) size: size of the data to be passedonto
Quoted 15 lines
Quoted 15 lines
the driver
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ c) data: array of struct
drm_color_op_datawith data forDocumentation/gpu/rfc/plane_color_pipeline.rst
Quoted 12 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +12
-0+ the hardware block/s that userspace wants to + set values for. + + struct drm_color_pipeline { + int num; + int size; + struct drm_color_op_data *data; + }; + + User can either: + 1. send data for all the color operations in a pipeline as shown in [2]. + The color operation data need not be in order that the pipelineadvertises
Quoted 14 lines
Quoted 9 lines
Quoted 9 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 341
Quoted 9 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +9
-0+ however, it should not contain data for any + color operation that is not present in the pipeline. + + Note: This check for pipeline is not yet implemented but if the + wider proposal is accepted we have few solutions in mind. + + 2. send data for a subset of color operations as shown in [3].For the + color operation that userspace does not send data will retain their + older state.Retaining existing state, especially for operations that userspace does not understand, can lead to incorrect and unexpected results. That's why I say that userspace must understand all operations in a pipeline, and all parameters of all used operations before it can actually use that pipeline.
By retaining state here, we mean the state set by the same client while using the same pipeline. If client wants to just alter one or subset of the hardware blocks in the pipeline, it can just send that to driver. Rest of the pipeline which was previously programmed by the same client will remain intact.
However once the client switches pipeline, driver will disable the pipeline and client will have to program all the blocks again with the new pipeline.
Quoted 10 lines
Otherwise we have the same problem as KMS properties have in general
today: when new things are added that userspace does not understand, how will the userspace be able to maintain its old behaviour?That question has two answers today:
- userspace learns to program everything, and accidentally (mal)functions until then
- you do not switch between KMS clients that might leave incorrect state in not-understood properties
Quoted 8 lines
Quoted 5 lines
Neither is a good answer, and the problem does not seem to have any practical traction either.
For color pipelines I hope we can, and believe that we must, do better to maintain correct behaviour while extending functionality.
Yes agree, we are thinking to reset and disable the pipeline once client switches. One of the ideas could be to use
file_privto achieve that.
I would assume that each color operation in each pipeline is already independent. Userspace cannot know which color operation is mapped to which hardware block, and should not need to care. That means the kernel maintains software state with the UAPI objects, not with the hardware blocks. Therefore, there is no inherent need to reset any pipeline on switch. Hardware programming is a different matter.
In the hackfest or after it, people raised the possibility of having a standard property in each KMS colorop object if possible: boolean "pass-through", or any equivalent. If an object is set to pass-through, it is as if it did not exist - it does not alter the values going through it in any way. Not even clamping if that could make a difference.
This allows adding new arbitrary blocks in old pipelines without forcing userspace to abandon the pipeline, as long as userspace learns to use the "pass-through" from the start. This reduces the number of alternative pipelines that need to be advertised when adding new features.
If an object is not pass-through, then userspace absolutely must understand the values programmed into every property of that object, or there is no knowing what the object actually does. If userspace does not know what an object does, the object can be assumed to produce an unexpected result, i.e. wrong result. No userspace is going to willingly malfunction.
Quoted 19 lines
...
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +7
-0+Representing Fixed function hardware +==================================== + +To provide support for fixed function hardware, the driver could +expose vendor specific struct drm_color_op with parameters that +both the userspace and driver agrees on. To demonstrate, let's +consider a hyphothetical fixed function hardware block that converts BT601to BT2020.
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 365
Quoted 5 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +5
-0+The driver can choose to advertise the block as such. + +struct drm_color_op color_pipeline_X[] = { + { + .name = DRM_CB_PRIVATE,What if the hardware has 5 different custom blocks like this, multiple in the same pipeline. How do you 'name' them?
Quoted 2 lines
Partially explained above, but
private_flagscan help in differentiating this. A link for implementation shared above for reference.
But they are all CB_PRIVATE, so what positions do they take in the pipeline?
The names CB_PRE_CSC and CB_POST_CSC imply that the name defines the position in a pipeline. That is wrong with the names since the order is defined by the record order in a pipeline blob, right?
Quoted 9 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst line 366
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ .type = FIXED_FUNCTION,I have been assuming that 'type' uniquely defines both the operation and the contents of the parameter blob, but this does not look like it. What defines the operation and the parameters?
Statement is true for all other generic blocks, only for private blocks this is a bit different. Here interpretation depends on the
private_flagswhich can be documented by the respective vendor for the custom HAL implementation.
Exceptions to a rule are bad API design. In this case, it can easily be avoided.
Now I'm actually confused about how 'name' and 'type' depend on 'private_flags'. You have to decipher 'private_flags' to understand what 'name' means too?
Hmm, but isn't 'name' used for identifying the block/operation in the blob that sets up the parameters for a whole pipeline? But then 'name' does not uniquely identify a block?
Thanks,
pq
Quoted 13 lines
...
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+References +========== + +[1] https://gitlab.freedesktop.org/emersion/drm_info +[2] +https://patchwork.freedesktop.org/patch/554827/?series=123018&rev +=1 +[3] +https://patchwork.freedesktop.org/patch/554826/?series=123018&rev +=1 [4] https://patchwork.freedesktop.org/series/123018/
RE: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
Quoted 71 lines
-----Original Message-----
From: Pekka Paalanen <[email protected]>
Sent: Tuesday, September 5, 2023 5:03 PM
To: Shankar, Uma <[email protected]>
Cc: [email protected]; Borah, Chaitanya Kumar <[email protected]>; [email protected]; wayland- [email protected]; Harry Wentland <[email protected]>; Sebastian Wick <[email protected]>; [email protected]; Jonas Adahl <[email protected]>
Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color PipelineQuoted 50 lines · Uma
Quoted 50 lines
-----Original Message-----
From: dri-devel <[email protected]> On Behalf Of Pekka Paalanen
Sent: Wednesday, August 30, 2023 5:59 PM
To: Shankar, Uma <[email protected]>
Cc: [email protected]; Borah, Chaitanya Kumar <[email protected]>; [email protected]; wayland- [email protected]
Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color PipelineQuoted 33 lines · Uma
Quoted 33 lines
-----Original Message-----
From: Harry Wentland <[email protected]>
Sent: Wednesday, August 30, 2023 1:10 AM
To: Shankar, Uma <[email protected]>; [email protected]; dri- [email protected]
Cc: Borah, Chaitanya Kumar <[email protected]>; wayland- [email protected]
Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color PipelineQuoted 19 lines · Uma Shankar
Add the documentation for the new proposed Plane Color Pipeline.
Co-developed-by: Chaitanya Kumar Borah
<[email protected]>
Signed-off-by: Chaitanya Kumar Borah
<[email protected]>Signed-off-by Uma Shankar <[email protected]>--- .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++ 1 file changed, 394 insertions(+)create mode 100644
Documentation/gpu/rfc/plane_color_pipeline.rstDocumentation/gpu/rfc/plane_color_pipeline.rst new file +0
-0...
Quoted 3 lines
Quoted 1 line
Hi Uma!
Thanks Pekka for the feedback and useful inputs.
Quoted 7 lines
Hi Uma,
sorry to say, but the overall feeling I get from this proposal is that it is just the current color related KMS properties wrapped in a pipeline blob. That is not the re-design I believe we are looking for, and the feeling is based on several details that are just copied from the current property design. Also the "private" stuff has to go.
Hi Pekka,
Ok, got the concerns in general. We will try to evaluate in deeper detail the property based design and come back if there are some issues or inputs.
At Intel we don't need private as of now, but we thought of having an option to enable any custom hardware or vendor. But we can drop the same for now if community doesn't feel the need for it.
Quoted 6 lines
All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying LUT tap distribution, and parametrized curves are good development, but right now we are looking at things one step higher level: the overall color pipeline design and how to represent any operation. Most of this series is considering details below the current attention level, hence I'm paying attention only to the first few patches.
We will need to precisely describe the hardware in userspace. Number of luts, precision, segments etc.., we can't just pass EOTF's as enum from userspace and let driver put hardcoded values to LUT. This will be nothing but an extension of descriptive behaviour. This will be needed as there are multiple colorspaces possible and even LUTS can be used to perform tone mapping. So, we need userspace to be able to program luts directly.
This is something we must expose to userspace. We will check if this can be fitted in property based approach.
Thanks for all the feedback and comments on the design. We will work on it and get back.
Regards,
Uma Shankar
Quoted 244 lines
More below.
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 6 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +4
-0+This color pipeline is then packaged within a blob for the +user space to retrieve it. Details can be found in the next +section +Not sure I like blobs that contain other blob ids.
It provides flexibility and helps with just one interface to userspace. Its easy to handle and manage once we get the hang of it 😊.
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
We can clearly define the steps of parsing and data structures to be used while interpreting and parsing the blobs.
Don't forget extendability. Possibly every single struct will need some kind of versioning, and then it's not simple to parse anymore. Add to that new/old kernel vs. old/new userspace, and it seems a bit
nightmarish to design.
Quoted 3 lines
Structure to be used to interpret the blob should be defined as UAPI only and is not expected to change once agreed upon. It should be interpreted
like a standard property.
Quoted 3 lines
So structure to be used, say for 3dLut or 1dlut or CTM operations should be standardized and fixed. No versioning of structure should be done and same rules/restrictions as of UAPI property should be applied.
That sounds like a UAPI that cannot be extended, either. So in a few years we'd be looking at replacing it. Or maybe you are just re-inventing the KMS object property system as blobs?
Replacing a single KMS property is much easier than replacing the foundations of the whole color pipeline design.
Quoted 4 lines
...
Quoted 2 lines
I have a feeling that introspection will be valuable here, to help people understand what their hardware could do if they had the code to use
it. 'name'
Quoted 2 lines
Quoted 2 lines
and 'type' being integers require a translation table to strings before they are readable, and it would be best if the kernel itself provided that
translation.
Quoted 5 lines
Name and type can be standardized in enum and well documented in the UAPI. For all the standard hardware blocks common for all vendors and serving most of the common usecases, we can define standard names in enum. These can be easily interpreted by a table as done in many cases already
in driver and userspace.
Yeah, but it won't help with the type-specific parameter blobs that are totally custom per each operation type. With the KMS property system we could have more generic introspection into those as well, at least naming all the parameters they have.
Plus, you don't have to patch
drm_infoevery time something new gets added in the kernel in order to see it named.Quoted 6 lines
...
Quoted 1 line
Therefore, I'm not yet convinced with the "all blobs" design.
Looking forward to collaboratively solve the problem for the community. Will improve the design based on all the feedback.
I have the feeling that we should also see Harry's draft.
Intel's special LUT type is a whole another story, and we can always have that fitted with any UAPI paradigm.
Quoted 8 lines
...
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +4
-0+ u32 blob_id; + }; + +2. struct drm_color_pipeline: This structure represents theaggregate
Quoted 1 line
Quoted 1 line
Quoted 1 line
+ pipeline to be set. it contains the following
members
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ a) num: pipeline number to beselected
Quoted 1 line
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ b) size: size of the data to bepassed
Quoted 5 lines
Quoted 5 lines
onto
Quoted 4 lines
Quoted 4 lines
the driver
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ c) data: array of struct
drm_color_op_datawith data forQuoted 1 line
+ the hardware block/s that userspace
wants to
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ set values for. + + struct drm_color_pipeline { + int num; + int size; + struct drm_color_op_data *data; + }; + + User can either: + 1. send data for all the color operations in a pipeline as shown in[2].
Quoted 10 lines
Quoted 10 lines
Quoted 2 lines
Quoted 2 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 2 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +2
-0+ The color operation data need not be in order that the +pipelineadvertises
Quoted 7 lines
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +7
-0+ however, it should not contain data for any + color operation that is not present in the pipeline. + + Note: This check for pipeline is not yet implemented but if the + wider proposal is accepted we have few solutions in mind. + + 2. send data for a subset of color operations as shown in [3].Forthe
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ color operation that userspace does not send data will retaintheir
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst line 341
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ older state.Retaining existing state, especially for operations that userspace does not understand, can lead to incorrect and unexpected results. That's why I say that userspace must understand all operations in a pipeline, and all parameters of all used operations before it can actually use
that pipeline.
Quoted 5 lines
By retaining state here, we mean the state set by the same client while using the same pipeline. If client wants to just alter one or subset of the hardware blocks in the pipeline, it can just send that to driver. Rest of the pipeline which was previously programmed by the same
client will remain intact.
Quoted 3 lines
However once the client switches pipeline, driver will disable the pipeline and client will have to program all the blocks again with the new
pipeline.
Quoted 11 lines
Quoted 11 lines
Otherwise we have the same problem as KMS properties have in general
today: when new things are added that userspace does not understand, how will the userspace be able to maintain its old behaviour?That question has two answers today:
- userspace learns to program everything, and accidentally (mal)functions until then
- you do not switch between KMS clients that might leave incorrect state in not-understood propertiesQuoted 7 lines
Quoted 5 lines
Neither is a good answer, and the problem does not seem to have any practical traction either.
For color pipelines I hope we can, and believe that we must, do better to maintain correct behaviour while extending functionality.
Yes agree, we are thinking to reset and disable the pipeline once client
switches.
Quoted 1 line
One of the ideas could be to use
file_privto achieve that.I would assume that each color operation in each pipeline is already independent. Userspace cannot know which color operation is mapped to which hardware block, and should not need to care. That means the kernel maintains software state with the UAPI objects, not with the hardware blocks. Therefore, there is no inherent need to reset any pipeline on switch. Hardware programming is a different matter.
In the hackfest or after it, people raised the possibility of having a standard property in each KMS colorop object if possible: boolean "pass-through", or any equivalent. If an object is set to pass-through, it is as if it did not exist - it does not alter the values going through it in any way. Not even clamping if that could make a difference.
This allows adding new arbitrary blocks in old pipelines without forcing userspace to abandon the pipeline, as long as userspace learns to use the "pass-through" from the start. This reduces the number of alternative pipelines that need to be advertised when adding new features.
If an object is not pass-through, then userspace absolutely must understand the values programmed into every property of that object, or there is no knowing what the object actually does. If userspace does not know what an object does, the object can be assumed to produce an unexpected result, i.e. wrong result. No userspace is going to willingly malfunction.
Quoted 20 lines
...
Quoted 9 lines
Quoted 8 lines
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +8
-0+Representing Fixed function hardware +==================================== + +To provide support for fixed function hardware, the driver +could expose vendor specific struct drm_color_op with +parameters that both the userspace and driver agrees on. To +demonstrate, let's consider a hyphothetical fixed function +hardware block that converts BT601to BT2020.
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 365
Quoted 5 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +5
-0+The driver can choose to advertise the block as such. + +struct drm_color_op color_pipeline_X[] = { + { + .name = DRM_CB_PRIVATE,What if the hardware has 5 different custom blocks like this, multiple in the same pipeline. How do you 'name' them?
Quoted 2 lines
Partially explained above, but
private_flagscan help in differentiating this. A link for implementation shared above for reference.But they are all
CB_PRIVATE, so what positions do they take in the pipeline?The names
CB_PRE_CSCandCB_POST_CSCimply that the name defines the position in a pipeline. That is wrong with the names since the order is defined by the record order in a pipeline blob, right?Quoted 9 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst line 366
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ .type = FIXED_FUNCTION,I have been assuming that 'type' uniquely defines both the operation and the contents of the parameter blob, but this does not look like it. What defines the operation and the parameters?
Statement is true for all other generic blocks, only for private blocks this is a bit different. Here interpretation depends on the
private_flagswhich can be documented by the respective vendor for thecustom HAL implementation.
Exceptions to a rule are bad API design. In this case, it can easily be avoided.
Now I'm actually confused about how 'name' and 'type' depend on '
private_flags'. You have to decipher 'private_flags' to understand what 'name' means too?Hmm, but isn't 'name' used for identifying the block/operation in the blob that sets up the parameters for a whole pipeline? But then 'name'
does not uniquely identify a block?Thanks,
pqQuoted 15 lines
...
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +12
-0+References +========== + +[1] https://gitlab.freedesktop.org/emersion/drm_info +[2] +https://patchwork.freedesktop.org/patch/554827/?series=123018 +&rev +=1 +[3] +https://patchwork.freedesktop.org/patch/554826/?series=123018 +&rev +=1 [4] https://patchwork.freedesktop.org/series/123018/
Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
Quoted 92 lines · Uma
Quoted 69 lines
-----Original Message-----
From: Pekka Paalanen <[email protected]>
Sent: Tuesday, September 5, 2023 5:03 PM
To: Shankar, Uma <[email protected]>
Cc: [email protected]; Borah, Chaitanya Kumar <[email protected]>; [email protected]; wayland- [email protected]; Harry Wentland <[email protected]>; Sebastian Wick <[email protected]>; [email protected]; Jonas Adahl <[email protected]>
Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color PipelineQuoted 50 lines · Uma
Quoted 50 lines
-----Original Message-----
From: dri-devel <[email protected]> On Behalf Of Pekka Paalanen
Sent: Wednesday, August 30, 2023 5:59 PM
To: Shankar, Uma <[email protected]>
Cc: [email protected]; Borah, Chaitanya Kumar <[email protected]>; [email protected]; wayland- [email protected]
Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color PipelineQuoted 33 lines · Uma
Quoted 33 lines
-----Original Message-----
From: Harry Wentland <[email protected]>
Sent: Wednesday, August 30, 2023 1:10 AM
To: Shankar, Uma <[email protected]>; [email protected]; dri- [email protected]
Cc: Borah, Chaitanya Kumar <[email protected]>; wayland- [email protected]
Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color PipelineQuoted 19 lines · Uma Shankar
Add the documentation for the new proposed Plane Color Pipeline.
Co-developed-by: Chaitanya Kumar Borah
<[email protected]>
Signed-off-by: Chaitanya Kumar Borah
<[email protected]>Signed-off-by Uma Shankar <[email protected]>--- .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++ 1 file changed, 394 insertions(+)create mode 100644
Documentation/gpu/rfc/plane_color_pipeline.rstDocumentation/gpu/rfc/plane_color_pipeline.rst new file +0
-0...
Quoted 3 lines
Quoted 1 line
Hi Uma!
Thanks Pekka for the feedback and useful inputs.
Quoted 7 lines
Hi Uma,
sorry to say, but the overall feeling I get from this proposal is that it is just the current color related KMS properties wrapped in a pipeline blob. That is not the re-design I believe we are looking for, and the feeling is based on several details that are just copied from the current property design. Also the "private" stuff has to go.
Hi Pekka,
Ok, got the concerns in general. We will try to evaluate in deeper detail the property based design and come back if there are some issues or inputs.At Intel we don't need private as of now, but we thought of having an option to enable any custom hardware or vendor. But we can drop the same for now if community doesn't feel the need for it.
Quoted 6 lines
All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying LUT tap distribution, and parametrized curves are good development, but right now we are looking at things one step higher level: the overall color pipeline design and how to represent any operation. Most of this series is considering details below the current attention level, hence I'm paying attention only to the first few patches.
Quoted 5 lines
We will need to precisely describe the hardware in userspace. Number of luts, precision, segments etc.., we can't just pass EOTF's as enum from userspace and let driver put hardcoded values to LUT. This will be nothing but an extension of descriptive behaviour. This will be needed as there are multiple colorspaces possible and even LUTS can be used to perform tone mapping. So, we need userspace to be able to program luts directly.
Hi Uma,
yes, we do need to expose freely programmable LUTs when hardware has them. That's why I say it is good development.
However, this is not an either-or situation.
We must also be able to expose fixed-function curve blocks when hardware has them. Please, do not confuse this with a descriptive design. This is not about saying "this FB is using PQ encoding, convert it to NNN for me".
This is about defining an operation, that is mathematically defined as "the PQ EOTF with normalized domain and range", for example. This is prescriptive, because the exact mathematical formula of the operation is defined, and it does not depend on any properties of the block's input or output. It contains no opinions on how to convert something to another, like a conversion from a color system and viewing environment to another does (e.g. sRGB<->BT.2100/PQ). Every driver exposing this operation must implement it exactly the same, with a small allowed error tolerance.
There are no limitations on how it can be used. Userspace can choose to apply that formula on anything it likes, and use the result in any way it likes, even if it is an utterly non-standard pipeline not making any sense *to us*.
You could argue that an operation to "convert PQ to HLG" is also prescriptive for example, and yes, it is if implementations have to adhere to a single specific formula for it. But if implementations are allowed choose any formula they believe is the best to implement that operation, then it is descriptive: "I have PQ content, I want HLG out, do whatever, I don't care".
If you really have a fixed-function hardware block that literally uses one specific formula to convert PQ to HLG, you can and probably should expose that as a colorop. It is prescriptive, because the formula is fully known to userspace, and userspace will choose to use it for its formula, and not because it converts PQ to HLG.
However, if you had to combine multiple hardware blocks to achieve the PQ-to-HLG formula, then that does not make sense to expose as a colorop (other than for backward compatibility if your previous hardware generation had it as a single fixed-function block). In practise, it would be too rigid to be useful in more than few specific situations. It would be much more flexible to expose the actual hardware blocks you have, and let userspace use them any way it likes. This is about finding the right balance in abstraction for UAPI.
Quoted 2 lines
This is something we must expose to userspace. We will check if this can be fitted in property based approach.
I'm sure it can be exposed. The trivial option is to define a colorop with the specific operation type that defines a property that will hold the LUT configuration blob you have designed.
You do not need to force literally everything into a million properties, you can still use blobs where they make sense, like needing a variable or great number of elements of some type.
As such, the overall color pipeline UAPI design has little effect on how you would design your LUT UAPI structures.
Thanks,
pq
Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
In reply to Uma ShankarQuoted 16 lines · Uma Shankar
Add the documentation for the new proposed Plane Color Pipeline.
Co-developed-by Chaitanya Kumar Borah <[email protected]>Signed-off-by Chaitanya Kumar Borah <[email protected]>Signed-off-by Uma Shankar <[email protected]>--- .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++ 1 file changed, 394 insertions(+)create mode 100644
Documentation/gpu/rfc/plane_color_pipeline.rstDocumentation/gpu/rfc/plane_color_pipeline.rst new file +0
-0
Documentation/gpu/rfc/plane_color_pipeline.rst line 7
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +7
-0@@ -0,0 +1,394 @@ +======================================= + Plane Color Pipeline: A UAPI proposal +======================================= + +To build the proposal on, lets take the premise of a color pipeline as shown +below. +
Hi Uma,
Thanks for posting this. A few comments, with some echoing the sentiments of other commenters on the patch set.
Quoted 79 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +79
-0+ +-------------------------------------------+ + | RAM | + | +------+ +---------+ +---------+ | + | | FB 1 | | FB 2 | | FB N | | + | +------+ +---------+ +---------+ | + +-------------------------------------------+ + | Plane Color Hardware Block | + +--------------------------------------------+ + | +---v-----+ +---v-------+ +---v------+ | + | | Plane A | | Plane B | | Plane N | | + | | Pre-CSC | | Pre-CSC | | Pre-CSC | | + | +---+-----+ +---+-------+ +---+------+ | + | | | | | + | +---v-----+ +---v-------+ +---v------+ | + | |Plane A | | Plane B | | Plane N | | + | |CSC/CTM | | CSC/CTM | | CSC/CTM | | + | +---+-----+ +----+------+ +----+-----+ | + | | | | | + | +---v-----+ +----v------+ +----v-----+ | + | | Plane A | | Plane B | | Plane N | | + | |Post-CSC | | Post-CSC | | Post-CSC | | + | +---+-----+ +----+------+ +----+-----+ | + | | | | | + +--------------------------------------------+ ++------v--------------v---------------v-------| +|| || +|| Pipe Blender || ++--------------------+------------------------+ +| | | +| +-----------v----------+ | +| | Pipe Pre-CSC | | +| | | | +| +-----------+----------+ | +| | Pipe Color | +| +-----------v----------+ Hardware | +| | Pipe CSC/CTM | | +| | | | +| +-----------+----------+ | +| | | +| +-----------v----------+ | +| | Pipe Post-CSC | | +| | | | +| +-----------+----------+ | +| | | ++---------------------------------------------+ + | + v + Pipe Output + +Each plane consists of the following color blocks + * Pre-CSC : This block can used to linearize the input frame buffer data. + The linear data then can be further acted on by the following + color hardware blocks in the display hardware pipeline + + * CSC/CTM: Used to program color transformation matrix, this block is used + to perform color space conversions like BT2020 to BT709 or BT601 + etc. This block acts on the linearized data coming from the + Pre-CSC HW block. + + * Post-CSC: This HW block can be used to non-linearize frame buffer data to + match the sink. Another use case of it could be to perform Tone + mapping for HDR use-cases. + +Data from multiple planes will then be fed to pipe/crtc where it will get blended. +There is a similar set of HW blocks available at pipe/crtc level which acts on +this blended data. + +Below is a sample usecase fo video playback with sub-titles and playback +controls + +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ +│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ +│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │ +│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐ +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ +(subtitles) │ + │ +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │
Documentation/gpu/rfc/plane_color_pipeline.rst line 96
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├───┐ │ +│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │ +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ +(Playback controls UI) │ │ + │ │ +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │ +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐ │ │ +│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │ +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │
BT601 to pass-through seems odd here. Shouldn't this be BT2020 pass-through?
Quoted 29 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +29
-0+(video frame) │ │ │ + │ │ │ +┌────────────────────────────────────────────────────────────────────────┴─┴─┘ +│ +│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐ +│ │ CRTC │ │ CRTC │ │ CRTC │ +└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────► TO Port + │ │ │ │ │ │ + └─────────────┘ └─────────────┘ └───────────────┘ + +This RFC is intended to propose an uAPI for the pre-blending color pipeline +(however, can be also extended to post blending pipeline). + +Below are the design considerations while designing the uAPI. + +1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used) +2. Position of the HW block can be programmable +3. LUTs can be one dimentional or three dimentional +4. Number of LUT entries can vary across platforms +5. Precision of LUT entries can vary across platforms +6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic, + Piece-Wise Linear(PWL) etc +7. There can be parameterized/non-parameterized fixed function HW blocks. + e.g. Just a hardware bit, to convert from one color space to another. + +Plane Color Pipeline: Design Proposal +===================================== +Each Color Hardware block will be represented by the structure drm_color_op. +
Documentation/gpu/rfc/plane_color_pipeline.rst line 136
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+struct drm_color_op { + enum color_op_block name; + enum color_op_type type; > + u32 blob_id; + u32 private_flags; +}; + +The members of which will constitute: +1. name: A standardised enum for the color hardware block +2. type: The type of mathematical operation done by the hardware block. + e.g. 1D Curve, 3D Curve, Matrix etc.
Using both 'type' and 'name' to identify the block sounds like an over complication to me. From a UAPI perspective we mainly care about the functionality the block provides, and type should be able to cover this on its own. If name is needed to declare a divergence from the standard definition for the type, why wouldn't a new type just be declared?
I'm currently not seeing any benefit that name provides.
Documentation/gpu/rfc/plane_color_pipeline.rst line 142
Quoted 6 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +6
-0+3. blob id: Id pointing to a blob containing information about the hardware + block advertising the respective capabilities to the userspace. + It can be an optional field depending on the members "name" and "type". +4. private_flags: This can be used to provide vendor specific hints + to user space +
If OEMs are declaring essentially hints in this property, how is user space going to handle this when they don't know how to interpret or handle them? To give any compositor a chance to use the color block these hints would have to result in no modification of the behavior agreed upon by the 'type', at which point I don't see what purpose they serve. The blob UAPI being fixed for a given color block type would also pose an issue here.
On the other hand, if these flags are only used for vendor specific hardware blocks it sounds like we are clubbing a proprietary block + definition into the color pipeline. I don't think this co-exists well with the intent of the original proposal.
Quoted 52 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +52
-0+ + For example to represent LUTs, we introduce the drm_color_lut_range + structure. It can represent LUTs with varied number of entries and + distributions (Multi segmented, Logarithmic etc). + + struct drm_color_lut_range { + /* DRM_MODE_LUT_* */ + __u32 flags; + /* number of points on the curve */ + __u16 count; + /* input/output bits per component */ + __u8 input_bpc, output_bpc; + /* input start/end values */ + __s32 start, end; + /* output min/max values */ + __s32 min, max; + }; + +Note: More details on exact usage and implementation of this structure can be + found in the RFC. This structure is taken as is from the series. + https://patchwork.freedesktop.org/series/90825/ + However, we can add more members to it to encompass all use-cases. + For example. we can add a precision field to advertise the + bitdepth of the LUTs. Similarly, we can reserve some bits in the flag + field for vendor specific use cases. + + At the same time, we don't need to pass any additional information for the + CSC block as userspace and driver already agrees struct drm_color_ctm as + a uAPI to pass on data. + +These color operations will form the building blocks of a color pipeline which +best represents the underlying Hardware. Color operations can be re-arranged, +substracted or added to create distinct color pipelines to accurately describe +the Hardware blocks present in the display engine. + +In this proposal, a color pipeline is represented as an array of +struct drm_color_op. + +struct drm_color_op color_pipeline_1[] + +For example to represent the pre-blending color pipeline as described above + +We can create a color pipeline like this. + +struct drm_color_op color_pipeline_1[] = { + { + .name = DRM_CB_PRE_CSC, + .type = CURVE_1D, + .blob_id = 0; /* actual value to be populated during plane + initialization*/ + }, + {
Documentation/gpu/rfc/plane_color_pipeline.rst line 204
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ .name = DRM_CB_CSC, + .type = MATRIX, + .blob_id = 0; + }, + { + .name = DRM_CB_POST_CSC, + .type = CURVE_1D, + .blob_id = 0; + }, +};
I'm noticing that in the shared IGT patch set these names are being used to determine position instead of, well, the actual position in the color pipeline. We definitely need to be careful to avoid coupling functionality with usage, and I think having a name field is going to cause developers to make that mistake / shortcut.
Quoted 49 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +49
-0+ +Then, for individual color operation, we add blobs to advertise the capability +of the particular Hardware block. In case of the example pipeline, we add +blobs of type drm_color_lut_range for the "pre-csc" and "post-csc". +For the "csc" block we pass no blob id to user space as it is known to both +user space and driver that drm_color_ctm structure is to be used for such +operation. + +To represent, this in a diagram. + + struct drm_color_op color_pipeline_1[] + +---------------------------+ + | | drm_color_op + | +---------------------+--+-----------+---------------------+ + | | | | | | + | | | | | +-----------------+ | + | | | | | | name | | + | | | | | +-----------------+ | + | | | | | | type | | + | | color_op_1 | | | +-----------------+ | + | | | | | | blob id | +--------+ + | | | | | +-----------------+ | | + | | | | | | private | | | + | | | | | | flags | | | + | | | | | +-----------------+ | | + | | | | | | | + | +---------------------+--+-----------+---------------------+ | + | | | + | | | + | +---------------------+ | | + | | | | drm_color_lut_range | + | | color_op_2 | | +-------------------------+ | + | | | | | | | + | | | | | +---------------------+ | | + | +---------------------+ | | | segment 1 { | |<---+ + | | | | ... | | + | +---------------------+ | | | .input_bpc = 16, | | + | | | | | | .output_bpc = 16, | | + | | color_op_3 | | | | ... | | + | | | | | | } | | + | | | | | +---------------------+ | + | +---------------------+ | | | + | . | | +---------------------+ | + | . | | | segment 2 { | | + | . | | | ... | | + +---------------------------+ | | } | | + | | | | + | | | | + | | | |
Documentation/gpu/rfc/plane_color_pipeline.rst line 263
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ | +---------------------+ | + | . | + | . | + | . | + +-------------------------+ + + + +This color pipeline is then packaged within a blob for the user space to +retrieve it. Details can be found in the next section
While I get the intent of the blob nesting, as others have mentioned this does feel like a recreation of what we already have with DRM objects. DRM objects would also have the advantage of allowing for easier OEM extension. Instead of private flags, an OEM could introduce behavior modifiers or additional features on a color block object via additional properties (these properties would obviously have to default to a disabled state to ensure default behavior is what is defined by the base color block type).
Quoted 103 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +103
-0+ +Exposing a color pipeline to user space +======================================= + +To advertise the available color pipelines, an immutable ENUM property +"GET_COLOR_PIPELINE" is introduced. +This enum property has blob id's as values. With each blob id representing +a distinct color pipeline based on underlying HW capabilities and their +respective combinations. + +The following output of drm_info [1], shows how color pipelines are visible +to userspace. + +├───Plane 0 + │ ├───Object ID: 31 + │ ├───CRTCs: {0} + │ ├───Legacy info + ... + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline, + color pipeline 1, color pipeline 2}= no color pipeline + +To understand the capabilities of individual pipelines, first the userspace +need to retrieve the pipeline blob with the blob ids retrieved by reading the +enum property. + +Once the color pipeline is retrieved, user can then parse through +individual drm_color_op blocks to understand the capabilities of each +hardware block. + +Check IGT series to see how user space can parse through color pipelines. +Refer the IGT series here: https://patchwork.freedesktop.org/series/123018/ + +Setting up a color pipeline +=========================== + +Once the user space decides on a color pipeline, it can set the pipeline and +the corresponding data for the hardware blocks within the pipeline with +the BLOB property "SET_COLOR_PIPELINE". + +To achieve this two structures are introduced + +1. struct drm_color_op_data: It represents data to be passed onto individual + color hardware blocks. It contains three members + a) name: to identify the color operation block + b) blob_id: pointing to the blob with data for the + corresponding hardware block + + struct drm_color_op_data { + enum color_op_block name; + u32 blob_id; + }; + +2. struct drm_color_pipeline: This structure represents the aggregate + pipeline to be set. it contains the following members + a) num: pipeline number to be selected + b) size: size of the data to be passed onto the driver + c) data: array of struct drm_color_op_data with data for + the hardware block/s that userspace wants to + set values for. + + struct drm_color_pipeline { + int num; + int size; + struct drm_color_op_data *data; + }; + + User can either: + 1. send data for all the color operations in a pipeline as shown in [2]. + The color operation data need not be in order that the pipeline advertises + however, it should not contain data for any + color operation that is not present in the pipeline. + + Note: This check for pipeline is not yet implemented but if the + wider proposal is accepted we have few solutions in mind. + + 2. send data for a subset of color operations as shown in [3].For the + color operation that userspace does not send data will retain their + older state. + + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob + property to NULL as shown in both [2] and [3] + + 4. change the color pipeline as demonstrated in [3]. + On the new pipeline, the user is expected to setup all the color hardware block + Once the user requests a pipeline change, the driver will provide it a clean slate + which means that all the data previously set by the user will be discarded even if + there are common hardware blocks between the two(previous and current) pipelines. + +IGT implementation can be found here [4] + +Representing Fixed function hardware +==================================== + +To provide support for fixed function hardware, the driver could expose vendor +specific struct drm_color_op with parameters that both the userspace and +driver agrees on. To demonstrate, let's consider a hyphothetical fixed +function hardware block that converts BT601 to BT2020. +The driver can choose to advertise the block as such. + +struct drm_color_op color_pipeline_X[] = { + { + .name = DRM_CB_PRIVATE, + .type = FIXED_FUNCTION,
Documentation/gpu/rfc/plane_color_pipeline.rst line 376
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ .blob_id = 45; + }, +} + +Where the blob represents some vendor specific enums, strings or any other +appropriate data types which both the user-space and drivers are aligned on. + +blob:45 { + VENDORXXX_BT602_TO_BT2020, +};
As far as I see it, one of the primary advantages of moving to this new color pipeline is the standardization of the color block definition. User space looks at the 'type', and if it supports it then it can look at other expected information such as 'lut_size', 'bit_depth', etc to determine the programming considerations.
The second we start having color blocks fully defined by private_flags and vendor specific blobs we end up with the same scenario we have today with vendor specific properties on our DRM CRTC and DRM PLANE objects. While we aren't going to make all the different OEM hardware match, we still want to encourage common approaches to block declaration so user space can easily identify functionality it supports across a wide range of hardware.
Documentation/gpu/rfc/plane_color_pipeline.rst line 387
Quoted 11 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +11
-0+ +For enabling or passing parameters to such blocks, the user can send data +to the driver wrapped within a blob like any other color operation block. + + struct drm_color_op_data { + .name = DRM_CB_PRIVATE; + .blob_id = 46; + } ; + +where blob with id 46 contains data to enable the fixed function hardware(s). +
All that said, thanks again for working on this. I'm definitely curious to see how this will all come together.
- Christopher
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +7
-0+References +========== + +[1] https://gitlab.freedesktop.org/emersion/drm_info +[2] https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1 +[3] https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1 +[4] https://patchwork.freedesktop.org/series/123018/
RE: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
Quoted 35 lines
-----Original Message-----
From: Christopher Braga <[email protected]>
Sent: Friday, September 8, 2023 1:39 AM
To: Shankar, Uma <[email protected]>; [email protected]; dri- [email protected]
Cc: Borah, Chaitanya Kumar <[email protected]>; wayland- [email protected]
Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color PipelineQuoted 20 lines · Uma Shankar
Add the documentation for the new proposed Plane Color Pipeline.
Co-developed-by: Chaitanya Kumar Borah
<[email protected]>Signed-off-by Chaitanya Kumar Borah <[email protected]>Signed-off-by Uma Shankar <[email protected]>--- .../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++++++++++ 1 file changed, 394 insertions(+)create mode 100644
Documentation/gpu/rfc/plane_color_pipeline.rstDocumentation/gpu/rfc/plane_color_pipeline.rst new file +2
-0@@ -0,0 +1,394 @@ +======================================= + Plane Color Pipeline: A UAPI proposal
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 7
Quoted 5 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +5
-0+======================================= + +To build the proposal on, lets take the premise of a color pipeline +as shown below. +Hi Uma,
Thanks for posting this. A few comments, with some echoing the sentiments of other commenters on the patch set.
Hi Christopher,
Thanks for your inputs on the series and my apologies for a late reply.
Quoted 149 lines
Quoted 54 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +54
-0+ +-------------------------------------------+ + | RAM | + | +------+ +---------+ +---------+ | + | | FB 1 | | FB 2 | | FB N | | + | +------+ +---------+ +---------+ | + +-------------------------------------------+ + | Plane Color Hardware Block | + +--------------------------------------------+ + | +---v-----+ +---v-------+ +---v------+ | + | | Plane A | | Plane B | | Plane N | | + | | Pre-CSC | | Pre-CSC | | Pre-CSC | | + | +---+-----+ +---+-------+ +---+------+ | + | | | | | + | +---v-----+ +---v-------+ +---v------+ | + | |Plane A | | Plane B | | Plane N | | + | |CSC/CTM | | CSC/CTM | | CSC/CTM | | + | +---+-----+ +----+------+ +----+-----+ | + | | | | | + | +---v-----+ +----v------+ +----v-----+ | + | | Plane A | | Plane B | | Plane N | | + | |Post-CSC | | Post-CSC | | Post-CSC | | + | +---+-----+ +----+------+ +----+-----+ | + | | | | | + +--------------------------------------------+ ++------v--------------v---------------v-------| +|| || +|| Pipe Blender || ++--------------------+------------------------+ +| | | +| +-----------v----------+ | +| | Pipe Pre-CSC | | +| | | | +| +-----------+----------+ | +| | Pipe Color | +| +-----------v----------+ Hardware | +| | Pipe CSC/CTM | | +| | | | +| +-----------+----------+ | +| | | +| +-----------v----------+ | +| | Pipe Post-CSC | | +| | | | +| +-----------+----------+ | +| | | ++---------------------------------------------+ + | + v + Pipe Output + +Each plane consists of the following color blocks + * Pre-CSC : This block can used to linearize the input frame buffer data. + The linear data then can be further acted on by the following + color hardware blocks in the display hardware pipeline +Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ * CSC/CTM: Used to program color transformation matrix, this block is used + to perform color space conversions like BT2020 to BT709 or BT601 + etc. This block acts on the linearized data coming from the + Pre-CSC HW block. + + * Post-CSC: This HW block can be used to non-linearize frame buffer data to + match the sink. Another use case of it could be to perform Tone + mapping for HDR use-cases. + +Data from multiple planes will then be fed to pipe/crtc where it will getblended.
Documentation/gpu/rfc/plane_color_pipeline.rst line 78
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +7
-0+There is a similar set of HW blocks available at pipe/crtc level +which acts on this blended data. + +Below is a sample usecase fo video playback with sub-titles and +playback controls + +┌────────────┐ ┌─────────────┐ ┌─────────────┐┌─────────────┐ > +│FB1 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ > +│ ├───►│Linearize ├────►│ BT709 to ├───►│ SDR to HDR │ > +│BT709 SDR │ │ │ │ BT2020 │ │ Tone Mapping├─────┐ > +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ > +(subtitles) │ > + │ > +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ > +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ > +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├───┐ │ > +│BT601 SDR │ │ │ │ BT2020 │ │ Tone Mapping│ │ │ > +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ > +(Playback controls UI) │ │ > + │ │ > +┌────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ > +│FB2 │ │PRE-CSC │ │ CTM Matrix │ │ POST-CSC │ │ │ > +│ ├───►│Linearize ├────►│ BT601 to ├───►│ SDR to HDR ├─┐ │ │ > +│BT2020 HDR │ │ │ │ pass-through│ │ pass-through│ │ │ │ > +└────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │BT601 to pass-through seems odd here. Shouldn't this be BT2020 pass-through?
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 4 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +3
-0+(video frame) │ │ │ + +│ │ │+┌─────────────────────────────────────────────────────────────── ───── > +────┴─┴─┘ > +│ > +│ ┌─────────────┐ ┌─────────────┐ ┌───────────────┐ > +│ │ CRTC │ │ CRTC │ │ CRTC │ > +└─┤ PRE-CSC ├─────►│ CSC/CTM ├─────►│ POST-CSC ├─────►TO Port
Quoted 22 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +22
-0+ │ │ │ │ │ │ + └─────────────┘ └─────────────┘ └───────────────┘ + +This RFC is intended to propose an uAPI for the pre-blending color +pipeline (however, can be also extended to post blending pipeline). + +Below are the design considerations while designing the uAPI. + +1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only +one can be used) 2. Position of the HW block can be programmable 3. +LUTs can be one dimentional or three dimentional 4. Number of LUT +entries can vary across platforms 5. Precision of LUT entries can +vary across platforms 6. Distribution of LUT entries may vary. e.g +Mutli-segmented, Logarithmic, + Piece-Wise Linear(PWL) etc +7. There can be parameterized/non-parameterized fixed function HW blocks. + e.g. Just a hardware bit, to convert from one color space to another. + +Plane Color Pipeline: Design Proposal +===================================== +Each Color Hardware block will be represented by the structure drm_color_op. +Documentation/gpu/rfc/plane_color_pipeline.rst line 136
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+struct drm_color_op { + enum color_op_block name; + enum color_op_type type; > + u32 blob_id; + u32 private_flags; +}; + +The members of which will constitute: +1. name: A standardised enum for the color hardware block 2. type: +The type of mathematical operation done by the hardware block. + e.g. 1D Curve, 3D Curve, Matrix etc.Using both 'type' and 'name' to identify the block sounds like an over complication to me. From a UAPI perspective we mainly care about the functionality the block provides, and type should be able to cover this on its own. If name is needed to declare a divergence from the standard definition for the type, why wouldn't a new type just be declared?
Quoted 1 line
I'm currently not seeing any benefit that name provides.
We kept name just to give some logical identification to the hardware block with a human readable name. But I agree given the combinations which could arise, its hard to come up with a fixed name for a block. We can drop it.
Quoted 13 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 142
Quoted 6 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +6
-0+3. blob id: Id pointing to a blob containing information about the hardware + block advertising the respective capabilities to the userspace. + It can be an optional field depending on the members "name" and "type". +4. private_flags: This can be used to provide vendor specific hints + to user space +If OEMs are declaring essentially hints in this property, how is user space going to handle this when they don't know how to interpret or handle them? To give any compositor a chance to use the color block these hints would have to result in no modification of the behavior agreed upon by the 'type', at which point I don't see what purpose they serve. The blob UAPI being fixed for a given color block type would also pose an issue here.
Quoted 3 lines
On the other hand, if these flags are only used for vendor specific hardware blocks it sounds like we are clubbing a proprietary block + definition into the color pipeline. I don't think this co-exists well with the intent of the original proposal.
Our idea was to have an option to allow some custom hardware implementation which can't be represented with generic design. However with the feedback received, we can surely drop this from the generic structure. For future a custom PRIVATE_TYPE can be made for which respective driver can implement the relevant data structures and blob contained within the driver and supported by its custom userspace. This way generic userspace can totally ignore it and implementation will be standardized.
Quoted 61 lines
Quoted 24 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +24
-0+ + For example to represent LUTs, we introduce the drm_color_lut_range + structure. It can represent LUTs with varied number of entries and + distributions (Multi segmented, Logarithmic etc). + + struct drm_color_lut_range { + /* DRM_MODE_LUT_* */ + __u32 flags; + /* number of points on the curve */ + __u16 count; + /* input/output bits per component */ + __u8 input_bpc, output_bpc; + /* input start/end values */ + __s32 start, end; + /* output min/max values */ + __s32 min, max; + }; + +Note: More details on exact usage and implementation of this structure can be + found in the RFC. This structure is taken as is from the series. + https://patchwork.freedesktop.org/series/90825/ + However, we can add more members to it to encompass all use-cases. + For example. we can add a precision field to advertise the + bitdepth of the LUTs. Similarly, we can reserve some bits in the flagDocumentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ field for vendor specific use cases. + + At the same time, we don't need to pass any additional information for the + CSC block as userspace and driver already agrees struct drm_color_ctm as + a uAPI to pass on data. + +These color operations will form the building blocks of a color +pipeline which best represents the underlying Hardware. Color +operations can be re-arranged, substracted or added to create +distinct color pipelines to accurately describe the Hardware blocks present inthe display engine.
Quoted 26 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +26
-0+ +In this proposal, a color pipeline is represented as an array of +struct drm_color_op. + +struct drm_color_op color_pipeline_1[] + +For example to represent the pre-blending color pipeline as described +above + +We can create a color pipeline like this. + +struct drm_color_op color_pipeline_1[] = { + { + .name = DRM_CB_PRE_CSC, + .type = CURVE_1D, + .blob_id = 0; /* actual value to be populated during plane + initialization*/ + }, + { + .name = DRM_CB_CSC, + .type = MATRIX, + .blob_id = 0; + }, + { + .name = DRM_CB_POST_CSC, + .type = CURVE_1D,
Quoted 8 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 204
Quoted 3 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +3
-0+ .blob_id = 0; + }, +};I'm noticing that in the shared IGT patch set these names are being used to determine position instead of, well, the actual position in the color pipeline. We definitely need to be careful to avoid coupling functionality with usage, and I think having a name field is going to cause developers to make that mistake / shortcut.
Yeah we can drop the idea of name and go with type only instead to avoid any confusion
Quoted 59 lines
Quoted 49 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +49
-0+ +Then, for individual color operation, we add blobs to advertise the capability +of the particular Hardware block. In case of the example pipeline, we add +blobs of type drm_color_lut_range for the "pre-csc" and "post-csc". +For the "csc" block we pass no blob id to user space as it is known to both +user space and driver that drm_color_ctm structure is to be used for such +operation. + +To represent, this in a diagram. + + struct drm_color_op color_pipeline_1[] + +---------------------------+ + | | drm_color_op + | +---------------------+--+-----------+---------------------+ + | | | | | | + | | | | | +-----------------+ | + | | | | | | name | | + | | | | | +-----------------+ | + | | | | | | type | | + | | color_op_1 | | | +-----------------+ | + | | | | | | blob id | +--------+ + | | | | | +-----------------+ | | + | | | | | | private | | | + | | | | | | flags | | | + | | | | | +-----------------+ | | + | | | | | | | + | +---------------------+--+-----------+---------------------+ | + | | | + | | | + | +---------------------+ | | + | | | | drm_color_lut_range | + | | color_op_2 | | +-------------------------+ | + | | | | | | | + | | | | | +---------------------+ | | + | +---------------------+ | | | segment 1 { | |<---+ + | | | | ... | | + | +---------------------+ | | | .input_bpc = 16, | | + | | | | | | .output_bpc = 16, | | + | | color_op_3 | | | | ... | | + | | | | | | } | | + | | | | | +---------------------+ | + | +---------------------+ | | | + | . | | +---------------------+ | + | . | | | segment 2 { | | + | . | | | ... | | + +---------------------------+ | | } | | + | | | | + | | | | + | | | |Documentation/gpu/rfc/plane_color_pipeline.rst line 263
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ | +---------------------+ | + | . | + | . | + | . | + +-------------------------+ + + + +This color pipeline is then packaged within a blob for the user space to +retrieve it. Details can be found in the next section
Quoted 8 lines
While I get the intent of the blob nesting, as others have mentioned this does feel like a recreation of what we already have with DRM objects. DRM objects would also have the advantage of allowing for easier OEM extension. Instead of private flags, an OEM could introduce behavior modifiers or additional features on a color block object via additional properties (these properties would obviously have to default to a disabled state to ensure default behavior is what is defined by the base color block type).
There are pros and cons to both the approaches, however we respect the feedback of community and will try to adhere to a universally acceptable and scalable design. We will work with Harry Wentland and other community developers to come up with good color implementation in upstream, which meets the needs of all stakeholders and is generic and scalable.
Quoted 131 lines
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ +Exposing a color pipeline to user space +======================================= + +To advertise the available color pipelines, an immutable ENUM property +"GET_COLOR_PIPELINE" is introduced. +This enum property has blob id's as values. With each blob id representing +a distinct color pipeline based on underlying HW capabilities and their +respective combinations. +Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+The following output of drm_info [1], shows how color pipelines are visible +to userspace. + +├───Plane 0 + │ ├───Object ID: 31 + │ ├───CRTCs: {0} + │ ├───Legacy info + ... + │ ├───"GET_COLOR_PIPELINE" (immutable): enum {no color pipeline, + color pipeline 1, color pipeline2}= no color pipeline
Quoted 12 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +12
-0+ +To understand the capabilities of individual pipelines, first the userspace +need to retrieve the pipeline blob with the blob ids retrieved by reading the +enum property. + +Once the color pipeline is retrieved, user can then parse through +individual drm_color_op blocks to understand the capabilities of each +hardware block. + +Check IGT series to see how user space can parse through color pipelines. +Refer the IGT series here: https://patchwork.freedesktop.org/series/123018/ +Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+Setting up a color pipeline +=========================== + +Once the user space decides on a color pipeline, it can set the pipeline and +the corresponding data for the hardware blocks within the pipeline with +the BLOB property "SET_COLOR_PIPELINE". + +To achieve this two structures are introduced + +1. struct drm_color_op_data: It represents data to be passed ontoindividual
Documentation/gpu/rfc/plane_color_pipeline.rst line 284
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ color hardwareblocks. It contains three members
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 12 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +12
-0+ a) name: to identify the color operation block + b) blob_id: pointing to the blob with data for the + corresponding hardware block + + struct drm_color_op_data { + enum color_op_block name; + u32 blob_id; + }; + +2. struct drm_color_pipeline: This structure represents the aggregate + pipeline to be set. it contains the following members + a) num: pipeline number to beselected
Documentation/gpu/rfc/plane_color_pipeline.rst line 297
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ b) size: size of the data to be passedonto the driver
Documentation/gpu/rfc/plane_color_pipeline.rst line 298
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ c) data: array of struct
drm_color_op_datawith data forDocumentation/gpu/rfc/plane_color_pipeline.rst
Quoted 12 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +12
-0+ the hardware block/s that userspace wants to + set values for. + + struct drm_color_pipeline { + int num; + int size; + struct drm_color_op_data *data; + }; + + User can either: + 1. send data for all the color operations in a pipeline as shown in [2]. + The color operation data need not be in order that the pipelineadvertises
Quoted 5 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +5
-0+ however, it should not contain data for any + color operation that is not present in the pipeline. + + Note: This check for pipeline is not yet implemented but if the + wider proposal is accepted we have few solutions in mind.Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ + 2. send data for a subset of color operations as shown in [3].For the + color operation that userspace does not send data will retain their + older state. + + 3. reset/disable the pipeline by setting the "SET_COLOR_PIPELINE" blob + property to NULL as shown in both [2] and [3] + + 4. change the color pipeline as demonstrated in [3]. + On the new pipeline, the user is expected to setup all the colorhardware block
Documentation/gpu/rfc/plane_color_pipeline.rst line 326
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ Once the user requests a pipeline change, the driver will provide it aclean slate
Documentation/gpu/rfc/plane_color_pipeline.rst line 327
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ which means that all the data previously set by the user will bediscarded even if
Documentation/gpu/rfc/plane_color_pipeline.rst line 328
Quoted 1 line
Documentation/gpu/rfc/plane_color_pipeline.rst new file +1
-0+ there are common hardware blocks between the two(previous andcurrent) pipelines.
Quoted 16 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +16
-0+ +IGT implementation can be found here [4] + +Representing Fixed function hardware +==================================== + +To provide support for fixed function hardware, the driver could expose vendor +specific struct drm_color_op with parameters that both the userspace and +driver agrees on. To demonstrate, let's consider a hyphothetical fixed +function hardware block that converts BT601 to BT2020. +The driver can choose to advertise the block as such. + +struct drm_color_op color_pipeline_X[] = { + { + .name = DRM_CB_PRIVATE, + .type = FIXED_FUNCTION,Documentation/gpu/rfc/plane_color_pipeline.rst line 376
Quoted 10 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +10
-0+ .blob_id = 45; + }, +} + +Where the blob represents some vendor specific enums, strings or any other +appropriate data types which both the user-space and drivers are aligned on. + +blob:45 { + VENDORXXX_BT602_TO_BT2020, +};As far as I see it, one of the primary advantages of moving to this new color pipeline is the standardization of the color block definition. User space looks at the 'type', and if it supports it then it can look at other expected information such as '
lut_size', 'bit_depth', etc to determine the programming considerations.
Quoted 7 lines
The second we start having color blocks fully defined by
private_flagsand vendor specific blobs we end up with the same scenario we have today with vendor specific properties on our DRM CRTC and DRM PLANE objects. While we aren't going to make all the different OEM hardware match, we still want to encourage common approaches to block declaration so user space can easily identify functionality it supports across a wide range of hardware.
Yes we can skip this PRIVATE flag to avoid such situations. For Intel, we don’t need it and generic types should suffice.
Quoted 15 lines
Documentation/gpu/rfc/plane_color_pipeline.rst line 387
Quoted 12 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +11
-0+ +For enabling or passing parameters to such blocks, the user can send data +to the driver wrapped within a blob like any other color operation block. + + struct drm_color_op_data { + .name = DRM_CB_PRIVATE; + .blob_id = 46; + } ; + +where blob with id 46 contains data to enable the fixed function hardware(s). +All that said, thanks again for working on this. I'm definitely curious to see how this will all come together.
Thanks Christopher for your review and constructive feedback. We will work with rest of the community to make this happen.
Apologies again for being late on my reply.
Regards,
Uma Shankar
Quoted 9 lines
- Christopher
Quoted 7 lines
Documentation/gpu/rfc/plane_color_pipeline.rst new file +7
-0+References +========== + +[1] https://gitlab.freedesktop.org/emersion/drm_info +[2] https://patchwork.freedesktop.org/patch/554827/?series=123018&rev=1 +[3] https://patchwork.freedesktop.org/patch/554826/?series=123018&rev=1 +[4] https://patchwork.freedesktop.org/series/123018/
RFC 2/33 drm: Add color operation structure
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
Each Color Hardware block will be represented uniquely in the color pipeline. Define the structure to represent the same.
These color operations will form the building blocks of a color pipeline which best represents the underlying Hardware. Color operations can be re-arranged, substracted or added to create distinct color pipelines to accurately describe the Hardware blocks present in the display engine.
--- include/uapi/drm/drm_mode.h | 72 +++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+)
include/uapi/drm/drm_mode.h +72 -0
@@ -943,6 +943,78 @@ struct hdr_output_metadata { }; }; +/** +* enum color_op_block +* +* Enums to identify hardware color blocks. +* +* @DRM_CB_PRE_CSC: LUT before the CTM unit +* @DRM_CB_CSC: CTM hardware supporting 3x3 matrix +* @DRM_CB_POST_CSC: LUT after the CTM unit +* @DRM_CB_3D_LUT: LUT hardware with coefficients for all +* color components +* @DRM_CB_PRIVATE: Vendor specific hardware unit. Vendor +* can expose a custom hardware by defining a +* color operation block with this name as +* identifier +*/ +enum color_op_block { +DRM_CB_INVAL =-1 , + +DRM_CB_PRE_CSC =0 , +DRM_CB_CSC , +DRM_CB_POST_CSC , +DRM_CB_3D_LUT , + +/* Any new generic hardware block can be updated here */ + +/* +* PRIVATE is kept at 255 to make it future proof and leave +* scope for any new addition +*/ +DRM_CB_PRIVATE =255 , +DRM_CB_MAX =DRM_CB_PRIVATE , +}; + +/** +* enum color_op_type +* +* These enums are to identify the mathematical operation that +* a hardware block is capable of. +* @CURVE_1D: It represents a one dimensional lookup table +* @CURVE_3D: Represents lut value for each color component for 3d lut capable hardware +* @MATRIX: It represents co-efficients for a CSC/CTM matrix hardware +* @FIXED_FUNCTION: To enable and program any custom fixed function hardware unit +*/ +enum color_op_type { +CURVE_1D , +CURVE_3D , +MATRIX , +FIXED_FUNCTION , +}; + +/** +* @struct drm_color_op +* +* This structure is used to represent the capability of +* individual color hardware blocks. +* +* @name: a standardized enum to identify the color hardware block +* @type: The type of mathematical operation it can perform +* @blob_id: Id pointing to a blob containing information about +* the hardware block which advertizes its capabilities +* to the userspace. It can be an optional field depending +* on the members "name" and "type". +* @private_flags: This can be used to provide vendor specific hints +* to user space +*/ +struct drm_color_op { +enum color_op_block name; +enum color_op_type type; +__u32 blob_id; +__u32 private_flags; +}; +/** * DRM_MODE_PAGE_FLIP_EVENT *
Signature · 1 line
2.38.1
Re: [RFC 02/33] drm: Add color operation structure
Quoted 31 lines · On Tue, 29 Aug 2023 21:33:51 +0530 Uma Shankar <[email protected]> wrote:
From: Chaitanya Kumar Borah <[email protected]>
Each Color Hardware block will be represented uniquely in the color pipeline. Define the structure to represent the same.
These color operations will form the building blocks of a color pipeline which best represents the underlying Hardware. Color operations can be re-arranged, substracted or added to create distinct color pipelines to accurately describe the Hardware blocks present in the display engine.
Co-developed-by Uma Shankar <[email protected]>Signed-off-by Uma Shankar <[email protected]>Signed-off-by Chaitanya Kumar Borah <[email protected]>--- include/uapi/drm/drm_mode.h | 72 +++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+)include/uapi/drm/drm_mode.h +4
-0@@ -943,6 +943,78 @@ struct hdr_output_metadata { }; }; +/** +* enum color_op_block +* +* Enums to identify hardware color blocks.
include/uapi/drm/drm_mode.h struct hdr_output_metadata { · line 959
Quoted 10 lines
include/uapi/drm/drm_mode.h +10
-0+* +* @DRM_CB_PRE_CSC: LUT before the CTM unit +* @DRM_CB_CSC: CTM hardware supporting 3x3 matrix +* @DRM_CB_POST_CSC: LUT after the CTM unit +* @DRM_CB_3D_LUT: LUT hardware with coefficients for all +* color components +* @DRM_CB_PRIVATE: Vendor specific hardware unit. Vendor +* can expose a custom hardware by defining a +* color operation block with this name as +* identifier
This naming scheme does not seem to work. It assumes a far too rigid pipeline, just like the old KMS property design. What if you have two other operations between PRE_CSC and CSC?
What sense do PRE_CSC and POST_CSC make if you don't happen to have a CSC operation?
What if a driver put POST_CSC before PRE_CSC in its pipeline?
What if your CSC is actually a series of three independent operations, and in addition you have PRE_CSC and POST_CSC?
3D_LUT is an operation category, not a name. The same could be said about private.
Given that all these are also UAPI, do we also need protect old userspace from seeing values it does not understand?
Quoted 24 lines
include/uapi/drm/drm_mode.h +24
-0+*/ +enum color_op_block { +DRM_CB_INVAL =-1 , + +DRM_CB_PRE_CSC =0 , +DRM_CB_CSC , +DRM_CB_POST_CSC , +DRM_CB_3D_LUT , + +/* Any new generic hardware block can be updated here */ + +/* +* PRIVATE is kept at 255 to make it future proof and leave +* scope for any new addition +*/ +DRM_CB_PRIVATE =255 , +DRM_CB_MAX =DRM_CB_PRIVATE , +}; + +/** +* enum color_op_type +* +* These enums are to identify the mathematical operation that +* a hardware block is capable of.
include/uapi/drm/drm_mode.h struct hdr_output_metadata { · line 993
Quoted 10 lines
include/uapi/drm/drm_mode.h +10
-0+* @CURVE_1D: It represents a one dimensional lookup table +* @CURVE_3D: Represents lut value for each color component for 3d lut capable hardware +* @MATRIX: It represents co-efficients for a CSC/CTM matrix hardware +* @FIXED_FUNCTION: To enable and program any custom fixed function hardware unit +*/ +enum color_op_type { +CURVE_1D , +CURVE_3D , +MATRIX , +FIXED_FUNCTION ,
My assumption was that a color_op_type would clearly and uniquely define the mathematical model of the operation and the UABI structure of the parameter blob. That means we need different values for uniform vs. exponentially vs. programmable distributed 1D LUT, etc.
If there is a 1D curve with pre-programmed (fixed and named) curves, we need to enumerate all the curve types somehow. Probably each fixed curve type should not be a different operation type, because that would explode the number of alternative pipelines.
A 3D curve in my mind is a function {x,y,z} = f(t), while I suspect you meant a 3D LUT which is a {x,y,z} = f(t,u,v) - a 3-vector field in three dimensional space.
A matrix element could be with or without an offset vector I guess.
FIXED_FUNCTION would need to be replaced with e.g. your example VENDORXXX_BT602_TO_BT2020 to work.
Have I missed something, how did you intend this to work?
Thanks,
pq
Quoted 27 lines
include/uapi/drm/drm_mode.h +24
-0+}; + +/** +* @struct drm_color_op +* +* This structure is used to represent the capability of +* individual color hardware blocks. +* +* @name: a standardized enum to identify the color hardware block +* @type: The type of mathematical operation it can perform +* @blob_id: Id pointing to a blob containing information about +* the hardware block which advertizes its capabilities +* to the userspace. It can be an optional field depending +* on the members "name" and "type". +* @private_flags: This can be used to provide vendor specific hints +* to user space +*/ +struct drm_color_op { +enum color_op_block name; +enum color_op_type type; +__u32 blob_id; +__u32 private_flags; +}; +/** * DRM_MODE_PAGE_FLIP_EVENT *
RE: [RFC 02/33] drm: Add color operation structure
Quoted 61 lines
-----Original Message-----
From: dri-devel <[email protected]> On Behalf Of Pekka Paalanen
Sent: Wednesday, August 30, 2023 6:30 PM
To: Shankar, Uma <[email protected]>
Cc: [email protected]; Borah, Chaitanya Kumar <[email protected]>; [email protected]; wayland- [email protected]
Subject: Re: [RFC 02/33] drm: Add color operation structureQuoted 31 lines · On Tue, 29 Aug 2023 21:33:51 +0530 Uma Shankar <[email protected]> wrote:
From: Chaitanya Kumar Borah <[email protected]>
Each Color Hardware block will be represented uniquely in the color pipeline. Define the structure to represent the same.
These color operations will form the building blocks of a color pipeline which best represents the underlying Hardware. Color operations can be re-arranged, substracted or added to create distinct color pipelines to accurately describe the Hardware blocks present in the display engine.
Co-developed-by Uma Shankar <[email protected]>Signed-off-by Uma Shankar <[email protected]>Signed-off-by Chaitanya Kumar Borah <[email protected]>--- include/uapi/drm/drm_mode.h | 72 +++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+)include/uapi/drm/drm_mode.h +4
-0@@ -943,6 +943,78 @@ struct hdr_output_metadata { }; }; +/** +* enum color_op_block +* +* Enums to identify hardware color blocks. include/uapi/drm/drm_mode.h struct hdr_output_metadata { · line 959
Quoted 10 lines
include/uapi/drm/drm_mode.h +10
-0+* +* @DRM_CB_PRE_CSC: LUT before the CTM unit +* @DRM_CB_CSC: CTM hardware supporting 3x3 matrix +* @DRM_CB_POST_CSC: LUT after the CTM unit +* @DRM_CB_3D_LUT: LUT hardware with coefficients for all +* color components +* @DRM_CB_PRIVATE: Vendor specific hardware unit. Vendor +* can expose a custom hardware by defining a +* color operation block with this name as +* identifier This naming scheme does not seem to work. It assumes a far too rigid pipeline, just like the old KMS property design. What if you have two other operations between
PRE_CSCand CSC?
Quoted 2 lines
What sense do
PRE_CSCandPOST_CSCmake if you don't happen to have a CSC operation?
Sure, we can re-look at the naming. However, it will be good to define some standard operations common to all vendors and keep the rest as vendor private.
Quoted 4 lines
What if a driver put
POST_CSCbeforePRE_CSCin its pipeline?What if your CSC is actually a series of three independent operations, and in addition you have
PRE_CSCandPOST_CSC?
We should try to standardized the operations as much as possible and leave rest as vendor private. Current proposal allows us to do that.
Quoted 2 lines
3D_LUT is an operation category, not a name. The same could be said about private.
Sure, will fix this.
Quoted 2 lines
Given that all these are also UAPI, do we also need protect old userspace from seeing values it does not understand?
For the values userspace doesn't understand, it can ignore the blocks. We should ensure that userspace always gets a clean state wrt color hardware state and no baggage from another client should be there. With that there is no burden of disabling that particular block will be there on an older userspace.
Quoted 31 lines
Quoted 31 lines
include/uapi/drm/drm_mode.h +31
-0+*/ +enum color_op_block { +DRM_CB_INVAL =-1 , + +DRM_CB_PRE_CSC =0 , +DRM_CB_CSC , +DRM_CB_POST_CSC , +DRM_CB_3D_LUT , + +/* Any new generic hardware block can be updated here */ + +/* +* PRIVATE is kept at 255 to make it future proof and leave +* scope for any new addition +*/ +DRM_CB_PRIVATE =255 , +DRM_CB_MAX =DRM_CB_PRIVATE , +}; + +/** +* enum color_op_type +* +* These enums are to identify the mathematical operation that +* a hardware block is capable of. +* @CURVE_1D: It represents a one dimensional lookup table +* @CURVE_3D: Represents lut value for each color component for 3d +lut capable hardware +* @MATRIX: It represents co-efficients for a CSC/CTM matrix hardware +* @FIXED_FUNCTION: To enable and program any custom fixed function +hardware unit */ enum color_op_type { +CURVE_1D ,
Quoted 8 lines
include/uapi/drm/drm_mode.h struct hdr_output_metadata { · line 993
Quoted 3 lines
include/uapi/drm/drm_mode.h +3
-0+CURVE_3D , +MATRIX , + FIXED_FUNCTION,My assumption was that a
color_op_typewould clearly and uniquely define the mathematical model of the operation and the UABI structure of the parameter blob. That means we need different values for uniform vs. exponentially vs. programmable distributed 1D LUT, etc.
In the hardware the LUTS are programmed as they are received from userspace. So once the userspace gets to know the distribution of LUTS, segments, precision, Number of lut samples, it can create the lut values to be programmed.
This information on the distribution of luts in the hardware can be extracted by the drm_color_lut_range structure which is exposed as blob in the hardware block with TYPE set as CURVE_1D.
Quoted 4 lines
If there is a 1D curve with pre-programmed (fixed and named) curves, we need to enumerate all the curve types somehow. Probably each fixed curve type should not be a different operation type, because that would explode the number of alternative pipelines.
Exposing the lut distribution with drm_color_lut_range would do this job.
Quoted 2 lines
A 3D curve in my mind is a function {x,y,z} = f(t), while I suspect you meant a 3D LUT which is a {x,y,z} =
f(t,u,v)- a 3-vector field in three dimensional space.
Yes right, we can optimize and fine tune this.
Quoted 6 lines
A matrix element could be with or without an offset vector I guess.
FIXED_FUNCTIONwould need to be replaced with e.g. your exampleVENDORXXX_BT602_TO_BT2020to work.Have I missed something, how did you intend this to work?
FIXED_FUNCTION is for some descriptive implementation. Some cases where Hardware just have a bit to enable and rest of the implementation is within hardware. Since type is used for other general hardware blocks, its not intended to cover vendor specific implementation and a private_flags are used instead.
Thanks Pekka for the feedback.
Regards,
Uma Shankar
Quoted 32 lines
Thanks,
pqQuoted 27 lines
include/uapi/drm/drm_mode.h +24
-0+}; + +/** +* @struct drm_color_op +* +* This structure is used to represent the capability of +* individual color hardware blocks. +* +* @name: a standardized enum to identify the color hardware block +* @type: The type of mathematical operation it can perform +* @blob_id: Id pointing to a blob containing information about +* the hardware block which advertizes its capabilities +* to the userspace. It can be an optional field depending +* on the members "name" and "type". +* @private_flags: This can be used to provide vendor specific hints +* to user space +*/ +struct drm_color_op { +enum color_op_block name; +enum color_op_type type; +__u32 blob_id; +__u32 private_flags; +}; +/** * DRM_MODE_PAGE_FLIP_EVENT *
Re: [RFC 02/33] drm: Add color operation structure
Quoted 59 lines · Uma
Quoted 59 lines
-----Original Message-----
From: dri-devel <[email protected]> On Behalf Of Pekka Paalanen
Sent: Wednesday, August 30, 2023 6:30 PM
To: Shankar, Uma <[email protected]>
Cc: [email protected]; Borah, Chaitanya Kumar <[email protected]>; [email protected]; wayland- [email protected]
Subject: Re: [RFC 02/33] drm: Add color operation structureQuoted 31 lines · On Tue, 29 Aug 2023 21:33:51 +0530 Uma Shankar <[email protected]> wrote:
From: Chaitanya Kumar Borah <[email protected]>
Each Color Hardware block will be represented uniquely in the color pipeline. Define the structure to represent the same.
These color operations will form the building blocks of a color pipeline which best represents the underlying Hardware. Color operations can be re-arranged, substracted or added to create distinct color pipelines to accurately describe the Hardware blocks present in the display engine.
Co-developed-by Uma Shankar <[email protected]>Signed-off-by Uma Shankar <[email protected]>Signed-off-by Chaitanya Kumar Borah <[email protected]>--- include/uapi/drm/drm_mode.h | 72 +++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+)include/uapi/drm/drm_mode.h +4
-0@@ -943,6 +943,78 @@ struct hdr_output_metadata { }; }; +/** +* enum color_op_block +* +* Enums to identify hardware color blocks. include/uapi/drm/drm_mode.h struct hdr_output_metadata { · line 959
Quoted 10 lines
include/uapi/drm/drm_mode.h +10
-0+* +* @DRM_CB_PRE_CSC: LUT before the CTM unit +* @DRM_CB_CSC: CTM hardware supporting 3x3 matrix +* @DRM_CB_POST_CSC: LUT after the CTM unit +* @DRM_CB_3D_LUT: LUT hardware with coefficients for all +* color components +* @DRM_CB_PRIVATE: Vendor specific hardware unit. Vendor +* can expose a custom hardware by defining a +* color operation block with this name as +* identifier This naming scheme does not seem to work. It assumes a far too rigid pipeline, just like the old KMS property design. What if you have two other operations between
PRE_CSCand CSC?
Quoted 5 lines
Quoted 2 lines
What sense do
PRE_CSCandPOST_CSCmake if you don't happen to have a CSC operation?Sure, we can re-look at the naming. However, it will be good to define some standard operations common to all vendors and keep the rest as vendor private.
My opinion is that these "names" should not even be an enum at all.
You're talking about operations, but operation is 'type', not 'name'.
There needs to be no pre-defined, hardcoded pipeline structure in the UAPI design. Your naming scheme, and the old KMS color property design, assume that there is such a rigid pre-defined structure that all hardware and drivers must adhere to. That does not work.
Drivers need to be able to expose any arbitrary operations in any arbitrary order in each pipeline, with no pre-determined or hinted "intended use". We want to define the operations, and not say what they should be used for. Usage examples are for accompanying documentation, not API tokens.
This idea is at the core of the new color pipeline design we have been discussing.
Quoted 61 lines
Quoted 4 lines
What if a driver put
POST_CSCbeforePRE_CSCin its pipeline?What if your CSC is actually a series of three independent operations, and in addition you have
PRE_CSCandPOST_CSC?We should try to standardized the operations as much as possible and leave rest as vendor private. Current proposal allows us to do that.
Quoted 2 lines
3D_LUT is an operation category, not a name. The same could be said about private.
Sure, will fix this.
Quoted 2 lines
Given that all these are also UAPI, do we also need protect old userspace from seeing values it does not understand?
For the values userspace doesn't understand, it can ignore the blocks. We should ensure that userspace always gets a clean state wrt color hardware state and no baggage from another client should be there. With that there is no burden of disabling that particular block will be there on an older userspace.
Quoted 31 lines
Quoted 31 lines
include/uapi/drm/drm_mode.h +31
-0+*/ +enum color_op_block { +DRM_CB_INVAL =-1 , + +DRM_CB_PRE_CSC =0 , +DRM_CB_CSC , +DRM_CB_POST_CSC , +DRM_CB_3D_LUT , + +/* Any new generic hardware block can be updated here */ + +/* +* PRIVATE is kept at 255 to make it future proof and leave +* scope for any new addition +*/ +DRM_CB_PRIVATE =255 , +DRM_CB_MAX =DRM_CB_PRIVATE , +}; + +/** +* enum color_op_type +* +* These enums are to identify the mathematical operation that +* a hardware block is capable of. +* @CURVE_1D: It represents a one dimensional lookup table +* @CURVE_3D: Represents lut value for each color component for 3d +lut capable hardware +* @MATRIX: It represents co-efficients for a CSC/CTM matrix hardware +* @FIXED_FUNCTION: To enable and program any custom fixed function +hardware unit */ enum color_op_type { +CURVE_1D ,Quoted 8 lines
include/uapi/drm/drm_mode.h struct hdr_output_metadata { · line 993
Quoted 3 lines
include/uapi/drm/drm_mode.h +3
-0+CURVE_3D , +MATRIX , + FIXED_FUNCTION,My assumption was that a
color_op_typewould clearly and uniquely define the mathematical model of the operation and the UABI structure of the parameter blob. That means we need different values for uniform vs. exponentially vs. programmable distributed 1D LUT, etc.
Quoted 7 lines
In the hardware the LUTS are programmed as they are received from userspace. So once the userspace gets to know the distribution of LUTS, segments, precision, Number of lut samples, it can create the lut values to be programmed.
This information on the distribution of luts in the hardware can be extracted by the
drm_color_lut_rangestructure which is exposed as blob in the hardware block with TYPE set asCURVE_1D.
I was referring to the special cases like FIXED_FUNCTION and PRIVATE. If you have an operation of type PRIVATE, you have no idea what it does until you decipher 'private_flags', but you cannot decipher 'private_flags' until you know the driver name because it is vendor-specific.
Userspace hates keying things off driver name. It means no other driver can expose the same and expect userspace to just start using it. It prevents an originally single-vendor feature to become a common multi-vendor feature in future hardware generations without patching every KMS client. Think about OpenGL or Vulkan: they have vendor-specific extensions, but if another vendor implements the same, they don't make up another extension but claim to support the original one.
Quoted 6 lines
Quoted 4 lines
If there is a 1D curve with pre-programmed (fixed and named) curves, we need to enumerate all the curve types somehow. Probably each fixed curve type should not be a different operation type, because that would explode the number of alternative pipelines.
Exposing the lut distribution with
drm_color_lut_rangewould do this job.
I'm talking about named fixed curves, PQ EOTF for example. You'd probably put it in FIXED_FUNCTION.
Or are you suggesting that userspace should take the LUT parameterisation from the driver and check if it happens to match PQ EOTF close enough?
We really need (string) names for fixed curves, so that they can be readily identified. Maybe Intel does not have them then, but AMD has.
Quoted 4 lines
Quoted 2 lines
A 3D curve in my mind is a function {x,y,z} = f(t), while I suspect you meant a 3D LUT which is a {x,y,z} =
f(t,u,v)- a 3-vector field in three dimensional space.Yes right, we can optimize and fine tune this.
I mean you used a wrong term. It's not a 3D curve when you have a 3D LUT.
Quoted 11 lines
Quoted 6 lines
A matrix element could be with or without an offset vector I guess.
FIXED_FUNCTIONwould need to be replaced with e.g. your exampleVENDORXXX_BT602_TO_BT2020to work.Have I missed something, how did you intend this to work?
FIXED_FUNCTIONis for some descriptive implementation. Some cases where Hardware just have a bit to enable and rest of the implementation is within hardware. Since type is used for other general hardware blocks, its not intended to cover vendor specific implementation and aprivate_flagsare used instead.
As I've said, 'private_flags' cannot exist.
Thanks,
pq
Quoted 39 lines
Thanks Pekka for the feedback.
Regards,
Uma ShankarQuoted 32 lines
Thanks,
pqQuoted 27 lines
include/uapi/drm/drm_mode.h +24
-0+}; + +/** +* @struct drm_color_op +* +* This structure is used to represent the capability of +* individual color hardware blocks. +* +* @name: a standardized enum to identify the color hardware block +* @type: The type of mathematical operation it can perform +* @blob_id: Id pointing to a blob containing information about +* the hardware block which advertizes its capabilities +* to the userspace. It can be an optional field depending +* on the members "name" and "type". +* @private_flags: This can be used to provide vendor specific hints +* to user space +*/ +struct drm_color_op { +enum color_op_block name; +enum color_op_type type; +__u32 blob_id; +__u32 private_flags; +}; +/** * DRM_MODE_PAGE_FLIP_EVENT *
RFC 3/33 drm: Add plane get color pipeline property
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
Each hardware plane can consist of multiple color hardware blocks. These hardware blocks are defined by a color pipeline. In case, hardware blocks can be re-arranged/muxed a distinct pipeline can be defined to represent the same.
Introduce a new enum plane property "GET_COLOR_PIPELINE" to expose the color pipelines that a particular plane supports. This enum property has blob id's as values. With each blob id representing a distinct color pipeline based on underlying HW capabilities and their respective combinations. Add helpers to create and attach the property to a plane.
--- drivers/gpu/drm/drm_color_mgmt.c | 46 ++++++++++++++++++++++++++++++++ include/drm/drm_plane.h | 10 +++++++ 2 files changed, 56 insertions(+)
drivers/gpu/drm/drm_color_mgmt.c +46 -0
@@ -588,6 +588,52 @@ int drm_plane_create_color_properties(struct drm_plane *plane, }EXPORT_SYMBOL (drm_plane_create_color_properties); +/** +* drm_plane_create_get_color_pipeline_property - create property to expose color pipelines +* @dev: DRM device +* @plane: plane object +* @num_val: number of color pipelines supported +* +* create enum property to expose color pipelines to userspace. This enum +* property has blob id's as values. With each blob id representing +* a distinct color pipeline based on underlying HW capabilities and +* their respective combinations. +*/ +int drm_plane_create_get_color_pipeline_property (struct drm_device *dev, +struct drm_plane *plane, +int num_val) +{ +struct drm_property *prop; + + prop =drm_property_create (dev,DRM_MODE_PROP_ENUM | +DRM_MODE_PROP_IMMUTABLE , +"GET_COLOR_PIPELINE" , num_val); +if (!prop) +return -ENOMEM ; + + plane->get_color_pipeline_prop = prop; + +return 0 ; +} +EXPORT_SYMBOL (drm_plane_create_get_color_pipeline_property); + +/** +* drm_plane_attach_get_color_pipeline_property - attach get color pipeline property to a plane +* @plane: plane object +* +* Attach "GET_COLOR_PIPELINE" property to a plane. The property will be visible to +* the userspace once we attach the property. +*/ +void drm_plane_attach_get_color_pipeline_property (struct drm_plane *plane) +{ +if (!plane->get_color_pipeline_prop) +return ; + +drm_object_attach_property (&plane->base, + plane->get_color_pipeline_prop,0 ); +} +EXPORT_SYMBOL (drm_plane_attach_get_color_pipeline_property); +/** * drm_color_lut_check - check validity of lookup table * @lut: property blob containing LUT to check
include/drm/drm_plane.h +10 -0
@@ -748,6 +748,12 @@ struct drm_plane {* scaling. */ struct drm_property *scaling_filter_property; + +/** +* @get_color_pipeline_prop: Optional Plane property to get the color pipelines +* that the plane supports +*/ +struct drm_property *get_color_pipeline_prop; };#define obj_to_plane (x) container_of(x, struct drm_plane, base)
@@ -945,5 +951,9 @@ drm_plane_get_damage_clips(const struct drm_plane_state *state);int drm_plane_create_scaling_filter_property (struct drm_plane *plane,unsigned int supported_filters); +int drm_plane_create_get_color_pipeline_property (struct drm_device *dev, +struct drm_plane *plane, +int num_val); +void drm_plane_attach_get_color_pipeline_property (struct drm_plane *plane);#endif
Signature · 1 line
2.38.1
RFC 4/33 drm: Add helper to add color pipeline
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
Create a helper function to add a color pipeline for a plane. Color pipeline is an array of struct drm_color_op which represent a possible logical combination of color operations. Color operations can be re-arranged, substracted or added to create distinct color pipelines to accurately describe the Hardware blocks present in the display engine.
--- drivers/gpu/drm/drm_color_mgmt.c | 42 ++++++++++++++++++++++++++++++++ include/drm/drm_plane.h | 3 +++ 2 files changed, 45 insertions(+)
drivers/gpu/drm/drm_color_mgmt.c +42 -0
@@ -634,6 +634,48 @@ void drm_plane_attach_get_color_pipeline_property(struct drm_plane *plane) }EXPORT_SYMBOL (drm_plane_attach_get_color_pipeline_property); +/** +* drm_plane_add_color_pipeline - helper to add a color pipeline +* @plane: plane object +* @name: name of the color pipeline +* @color_pipeline: an array of 'struct drm_color_op' to represent a color pipeline +* @len: size of the color pipeline +* +* This helper can be used to add a distinct color pipeline to a plane. A driver +* can provide a meaningful name to the pipeline as it desires. +*/ +int drm_plane_add_color_pipeline (struct drm_plane *plane,char *name, +struct drm_color_op *color_pipeline, +size_t len) +{ +int ret; +struct drm_property *prop; +struct drm_property_blob *blob; +u32 id =0 ; + + prop = plane->get_color_pipeline_prop; + +if (color_pipeline && !len) +return -EINVAL ; + +if (color_pipeline) { + blob =drm_property_create_blob (plane->dev, len, color_pipeline); +if (IS_ERR (blob)) +return PTR_ERR (blob); + + id = blob->base.id; + }; + + ret =drm_property_add_enum (prop, id, name); +if (ret) { +if (blob) +drm_property_blob_put (blob); +return ret; + } +return 0 ; +} +EXPORT_SYMBOL (drm_plane_add_color_pipeline); +/** * drm_color_lut_check - check validity of lookup table * @lut: property blob containing LUT to check
include/drm/drm_plane.h +3 -0
@@ -955,5 +955,8 @@ int drm_plane_create_get_color_pipeline_property(struct drm_device *dev,struct drm_plane *plane, int num_val);void drm_plane_attach_get_color_pipeline_property (struct drm_plane *plane); +int drm_plane_add_color_pipeline (struct drm_plane *plane,char *name, +struct drm_color_op *color_pipeline, +size_t len);#endif
Signature · 1 line
2.38.1
RFC 5/33 drm: Add structures for setting color pipeline
In reply to Uma ShankarAdd structures using which user space can set a color pipeline it desires. The patch introduces two structures
struct drm_color_op_data represents data to be passed onto individual color hardware blocks.
struct drm_color_pipeline represents the aggregate of drm_color_op_data structures to program the respective color hardware blocks of the pipeline. It also contains the pipeline number to be set.
--- include/uapi/drm/drm_mode.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
include/uapi/drm/drm_mode.h +30 -0
@@ -1015,6 +1015,36 @@ struct drm_color_op {__u32 private_flags; }; +/** +* struct drm_color_op_data +* +* Structure for userspace to send data to a particular color operation +* block. +* +* @name: Name of the color block for which the data is being sent +* @blob_id: Id pointing to blob with data for the color operation block +*/ +struct drm_color_op_data { +enum color_op_block name; +__u32 blob_id; +}; + +/** +* struct drm_color_pipeline +* +* This structure represents the aggregate pipeline to be set +* +* @num: pipeline number to be selected +* @size: size of the data to be passed onto the driver +* @data: array of struct drm_color_op_data with data for the +* hardware block/s that user space wants to set values for +*/ +struct drm_color_pipeline { +int num; +int size; +struct drm_color_op_data *data; +}; +/** * DRM_MODE_PAGE_FLIP_EVENT *
Signature · 1 line
2.38.1
RFC 6/33 drm: Add set colorpipeline property
In reply to Uma ShankarAdd a new plane blob property "SET_COLOR_PIPELINE" using which the user can select a color pipeline and send data for corresponding hardware blocks.
Once the user space decides on a color pipeline, it can set the pipeline and corresponding data for the hardware blocks within the pipeline.
--- drivers/gpu/drm/drm_atomic_uapi.c | 12 +++++++++ drivers/gpu/drm/drm_color_mgmt.c | 42 +++++++++++++++++++++++++++++++ include/drm/drm_plane.h | 22 ++++++++++++++++ 3 files changed, 76 insertions(+)
drivers/gpu/drm/drm_atomic_uapi.c +12 -0
@@ -590,6 +590,15 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane,return ret; }else if (property == plane->scaling_filter_property) { state->scaling_filter = val; + }else if (property == plane->set_color_pipeline_prop) { + ret =drm_atomic_replace_property_blob_from_id (dev, + &state->set_color_pipeline_data, + val, +-1 , +sizeof (struct drm_color_pipeline ), + &replaced); + state->color_mgmt_changed |= replaced; +return ret; }else if (plane->funcs->atomic_set_property) {return plane->funcs->atomic_set_property (plane, state, property, val);
@@ -651,6 +660,9 @@ drm_atomic_plane_get_property(struct drm_plane *plane, state->fb_damage_clips->base.id : 0; }else if (property == plane->scaling_filter_property) { *val = state->scaling_filter; + }else if (property == plane->set_color_pipeline_prop) { + *val = (state->set_color_pipeline_data) ? + state->set_color_pipeline_data->base.id :0 ; }else if (plane->funcs->atomic_get_property) {return plane->funcs->atomic_get_property (plane, state, property, val); }else {
drivers/gpu/drm/drm_color_mgmt.c +42 -0
48 lines not on this page · raw message
include/drm/drm_plane.h +22 -0
@@ -237,6 +237,20 @@ struct drm_plane_state {/** @state: backpointer to global drm_atomic_state */ struct drm_atomic_state *state; + +/** +* @set_color_pipeline_data: +* +* Stores information about the current selected color pipeline +*/ +struct drm_property_blob *set_color_pipeline_data; + +/** +* @color_mgmt_changed: Plane color pipeline state has changed +* Used by the atomic helpers and +* drivers to steer the atomic commit control flow. +*/ + u8 color_mgmt_changed : 1; }; static inline struct drm_rect
@@ -754,6 +768,11 @@ struct drm_plane {* that the plane supports */ struct drm_property *get_color_pipeline_prop; + +/** +* @set_color_pipeline_prop: Optional Plane property to set the color pipeline +*/ +struct drm_property *set_color_pipeline_prop; };#define obj_to_plane (x) container_of(x, struct drm_plane, base)
@@ -955,6 +974,9 @@ int drm_plane_create_get_color_pipeline_property(struct drm_device *dev,struct drm_plane *plane, int num_val);void drm_plane_attach_get_color_pipeline_property (struct drm_plane *plane); +int drm_plane_create_set_color_pipeline_property (struct drm_device *dev, +struct drm_plane *plane); +void drm_plane_attach_set_color_pipeline_property (struct drm_plane *plane);int drm_plane_add_color_pipeline (struct drm_plane *plane,char *name,struct drm_color_op *color_pipeline,size_t len);
Signature · 1 line
2.38.1
RFC 7/33 drm: Add Enhanced Gamma LUT precision structure
In reply to Uma ShankarExisting LUT precision structure is having only 16 bit precision. This is not enough for upcoming enhanced hardwares and advance usecases like HDR processing. Hence added a new structure with 32 bit precision values.
--- include/uapi/drm/drm_mode.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
include/uapi/drm/drm_mode.h +17 -0
@@ -943,6 +943,23 @@ struct hdr_output_metadata { }; }; +/** +* struct drm_color_lut_ext - Represents high precision lut values +* +* Creating 64 bit palette entries for better data +* precision. This will be required for HDR and +* similar color processing usecases. +*/ +struct drm_color_lut_ext { +/* +* Data is U32.32 fixed point format. +*/ +__u64 red; +__u64 green; +__u64 blue; +__u64 reserved; +}; +/** * enum color_op_block *
Signature · 1 line
2.38.1
RFC 8/33 drm: Add color lut range structure
In reply to Uma ShankarAdd color lut range structure which is to be used to advertize the capabilities of pre-csc/post-csc color operation blocks.
--- include/uapi/drm/drm_mode.h | 77 +++++++++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+)
include/uapi/drm/drm_mode.h +77 -0
@@ -1091,6 +1091,83 @@ struct drm_color_pipeline { DRM_MODE_PAGE_FLIP_ASYNC | \ DRM_MODE_PAGE_FLIP_TARGET) +/** +* DRM_MODE_LUT_POST_CSC +* +* LUT is for post csc (after CTM) +*/ +#define DRM_MODE_LUT_POST_CSC BIT(0) + +/** +* DRM_MODE_LUT_PRE_CSC +* +* LUT is for pre csc (before CTM) +*/ +#define DRM_MODE_LUT_PRE_CSC BIT(1) + +/** +* DRM_MODE_LUT_INTERPOLATE +* +* linearly interpolate between the points +*/ +#define DRM_MODE_LUT_INTERPOLATE BIT(2) + +/** +* DRM_MODE_LUT_REUSE_LAST +* +* the last value of the previous range is the +* first value of the current range. +*/ +#define DRM_MODE_LUT_REUSE_LAST BIT(3) + +/** +* DRM_MODE_LUT_NON_DECREASING +* +* the curve must be non-decreasing +*/ +#define DRM_MODE_LUT_NON_DECREASING BIT(4) + +/** +* DRM_MODE_LUT_REFLECT_NEGATIVE +* +* the curve is reflected across origin for negative inputs +*/ +#define DRM_MODE_LUT_REFLECT_NEGATIVE BIT(5) + +/** +* DRM_MODE_LUT_SINGLE_CHANNEL +* +* the same curve (red) is used for blue and green channels as well +*/ +#define DRM_MODE_LUT_SINGLE_CHANNEL BIT(6) + +/** +* struct drm_color_lut_range +* +* structure to advertise capability of a color hardware +* block that accepts LUT values. It can represent LUTs with +* varied number of entries and distributions +* (Multi segmented, Logarithmic etc). +*/ + +struct drm_color_lut_range { +/* DRM_MODE_LUT_* */ +__u32 flags; +/* number of points on the curve */ +__u16 count; +/* input/output bits per component */ +__u8 input_bpc, output_bpc; +/* input start/end values */ +__s32 start, end; +/* output min/max values */ +__s32 min, max; +}; + +enum lut_type { +LUT_TYPE_PRE_CSC =0 , +LUT_TYPE_POST_CSC =1 , +}; +/* * Request a page flip on the specified crtc. *
Signature · 1 line
2.38.1
RFC 9/33 drm: Add color information to plane state
In reply to Uma ShankarAdd a new structure drm_plane_color to plane state. It consists of blobs with data needed for respective color HW blocks.
Currently defining below blobs
pre-csc: can be used to linearize the input frame buffer data.
csc: used for color space conversion.
post-csc: can be used non-linearize frame buffer data or
to perform Tone mapping for HDR use-cases
private: can be used for vendor specific fixed function operations
This can be extended to include other color operations as well.
--- include/drm/drm_plane.h | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+)
include/drm/drm_plane.h +41 -0
@@ -245,6 +245,47 @@ struct drm_plane_state {*/ struct drm_property_blob *set_color_pipeline_data; +/** +* @drm_plane_color: +* +* Encapsulates all color states. +*/ +struct drm_plane_color { +/** +* @pre_csc_lut: +* +* Lookup table for converting framebuffer pixel data before apply the +* color conversion matrix @ctm. See drm_plane_enable_color_mgmt(). The +* blob (if not NULL) is an array of &struct drm_color_lut_ext. +*/ +struct drm_property_blob *pre_csc_lut; + +/** +* @ctm: +* +* Color transformation matrix. See drm_plane_enable_color_mgmt(). The +* blob (if not NULL) is a &struct drm_color_ctm. +*/ +struct drm_property_blob *ctm; + +/** +* @post_csc_lut: +* +* Lookup table for converting framebuffer pixel data after applying the +* color conversion matrix @ctm. See drm_plane_enable_color_mgmt(). The +* blob (if not NULL) is an array of &struct drm_color_lut_ext. +*/ +struct drm_property_blob *post_csc_lut; + +/** +* @private_color_op_data: +* +* This blob is intended for drivers to implement driver private color operations. +* For example: Parameterized/non-parameterized fixed function operations +*/ +struct drm_property_blob *private_color_op_data; + } color; +/** * @color_mgmt_changed: Plane color pipeline state has changed * Used by the atomic helpers and
Signature · 1 line
2.38.1
RFC 10/33 drm: Manage color blob states
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
This patch manages the references for color blobs.
--- drivers/gpu/drm/drm_atomic_state_helper.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
drivers/gpu/drm/drm_atomic_state_helper.c +18 -0
@@ -338,6 +338,19 @@ void __drm_atomic_helper_plane_duplicate_state(struct drm_plane *plane, state->fence =NULL ; state->commit =NULL ; state->fb_damage_clips =NULL ; + +if (state->set_color_pipeline_data) +drm_property_blob_get (state->set_color_pipeline_data); +if (state->color.pre_csc_lut) +drm_property_blob_get (state->color.pre_csc_lut); +if (state->color.ctm) +drm_property_blob_get (state->color.ctm); +if (state->color.post_csc_lut) +drm_property_blob_get (state->color.post_csc_lut); +if (state->color.private_color_op_data) +drm_property_blob_get (state->color.private_color_op_data); + + state->color_mgmt_changed = false; }EXPORT_SYMBOL (__drm_atomic_helper_plane_duplicate_state);
@@ -384,6 +397,11 @@ void __drm_atomic_helper_plane_destroy_state(struct drm_plane_state *state)drm_crtc_commit_put (state->commit);drm_property_blob_put (state->fb_damage_clips); +drm_property_blob_put (state->set_color_pipeline_data); +drm_property_blob_put (state->color.pre_csc_lut); +drm_property_blob_put (state->color.ctm); +drm_property_blob_put (state->color.post_csc_lut); +drm_property_blob_put (state->color.private_color_op_data); }EXPORT_SYMBOL (__drm_atomic_helper_plane_destroy_state);
Signature · 1 line
2.38.1
RFC 11/33 drm: Replace individual color blobs
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
Replace the color operation blobs depending on the values sent by userspace.
--- drivers/gpu/drm/drm_atomic_uapi.c | 97 +++++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+)
drivers/gpu/drm/drm_atomic_uapi.c +97 -0
@@ -404,6 +404,92 @@ drm_atomic_replace_property_blob_from_id(struct drm_device *dev,return 0 ; } +/* +* Helper to replace individual color blobs for a plane. The function +* changes all the color blobs sent by userspace agnostic of the color +* pipeline chosen. Since, the information about color pipeline is +* available at driver level, the driver should check for +* the sanity of the userspace data. +*/ +static +int drm_plane_replace_color_op_blobs (struct drm_plane *plane, +struct drm_plane_state *state, +uint64_t color_pipeline_blob_id, +bool *replaced) +{ +struct drm_device *dev = plane->dev; +struct drm_property_blob *new_blob; +struct drm_color_pipeline *color_pipeline; +struct drm_color_op_data *color_op; +int ret =0 , i; +bool blob_replaced = false; +bool temp_replaced = false; + + new_blob =drm_property_lookup_blob (dev, color_pipeline_blob_id); + +if (!new_blob) { + ret = -EINVAL ; + goto out; + } + + color_pipeline = new_blob->data; + color_op =kzalloc (color_pipeline->size,GFP_KERNEL ); +if (!color_op) { + ret = -ENOMEM ; + goto mem_fail; + } + +if (copy_from_user (color_op, color_pipeline->data, color_pipeline->size)) { + ret = -EFAULT ; + goto copy_fail; + } + +for (i =0 ; i < color_pipeline->size /sizeof (struct drm_color_op_data ); i++) { +if (color_op[i].name ==DRM_CB_CSC ) { + ret =drm_atomic_replace_property_blob_from_id (dev, + &state->color.ctm, + color_op[i].blob_id, +-1 ,sizeof (struct drm_color_ctm ), + &blob_replaced); + }else if (color_op[i].name ==DRM_CB_PRE_CSC ) { + ret =drm_atomic_replace_property_blob_from_id (dev, + &state->color.pre_csc_lut, + color_op[i].blob_id, +-1 ,sizeof (struct drm_color_lut_ext ), + &blob_replaced); + }else if (color_op[i].name ==DRM_CB_POST_CSC ) { + ret =drm_atomic_replace_property_blob_from_id (dev, + &state->color.post_csc_lut, + color_op[i].blob_id, +-1 ,sizeof (struct drm_color_lut_ext ), + &blob_replaced); + }else if (color_op[i].name ==DRM_CB_PRIVATE ) { + ret =drm_atomic_replace_property_blob_from_id (dev, + &state->color.private_color_op_data, + color_op[i].blob_id, +-1 ,-1 , + &blob_replaced); + }else { + ret = -EINVAL ; + goto copy_fail; + } + +if (ret) + goto copy_fail; + + temp_replaced |= blob_replaced; + } + +copy_fail: +kfree (color_op); +mem_fail: +drm_property_blob_put (new_blob); +out: +if (!ret) + *replaced |= temp_replaced; +return ret; +} +static int drm_atomic_crtc_set_property (struct drm_crtc *crtc,struct drm_crtc_state *state,struct drm_property *property,uint64_t val)
@@ -597,6 +683,17 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane,-1 ,sizeof (struct drm_color_pipeline ), &replaced); +if (replaced) { +/* Consider actual color parameter change only when +* individual color blobs are replaced. Hence, reset +* the replaced boolean. +*/ + replaced = false; + ret =drm_plane_replace_color_op_blobs (plane, state, + val, + &replaced); + } + state->color_mgmt_changed |= replaced;return ret; }else if (plane->funcs->atomic_set_property) {
Signature · 1 line
2.38.1
RFC 12/33 drm: Reset pipeline when user sends NULL blob
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
User can disable the color pipeline entirely, thereby disabling all the color hardware blocks in the pipeline.
User should set NULL as the blob id and invoke SET_COLOR_PIPELINE property. Driver will disable all the color hardware blocks by updating respective blob id's as NULL.
--- drivers/gpu/drm/drm_atomic_uapi.c | 50 +++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+)
drivers/gpu/drm/drm_atomic_uapi.c +50 -0
@@ -411,6 +411,53 @@ drm_atomic_replace_property_blob_from_id(struct drm_device *dev,* available at driver level, the driver should check for * the sanity of the userspace data. */ +static +int drm_plane_reset_color_op_blobs (struct drm_plane *plane, +struct drm_plane_state *state, +bool *replaced) +{ +struct drm_device *dev = plane->dev; +int ret; +bool blob_replaced = false; +bool temp_replaced = false; + + ret =drm_atomic_replace_property_blob_from_id (dev, + &state->color.ctm, +0 ,-1 ,-1 , + &blob_replaced); + temp_replaced |= blob_replaced; +if (ret) + goto out; + + ret =drm_atomic_replace_property_blob_from_id (dev, + &state->color.pre_csc_lut, +0 ,-1 ,-1 , + &blob_replaced); + temp_replaced |= blob_replaced; + +if (ret) + goto out; + + ret =drm_atomic_replace_property_blob_from_id (dev, + &state->color.post_csc_lut, +0 ,-1 ,-1 , + &blob_replaced); + temp_replaced |= blob_replaced; + + ret =drm_atomic_replace_property_blob_from_id (dev, + &state->color.private_color_op_data, +0 ,-1 ,-1 , + &blob_replaced); + temp_replaced |= blob_replaced; + +if (ret) + goto out; +out: +if (!ret) + *replaced |= temp_replaced; +return ret; +} + static int drm_plane_replace_color_op_blobs(struct drm_plane *plane, struct drm_plane_state *state,
@@ -425,6 +472,9 @@ int drm_plane_replace_color_op_blobs(struct drm_plane *plane,bool blob_replaced = false;bool temp_replaced = false; +if (!color_pipeline_blob_id) +return drm_plane_reset_color_op_blobs (plane, state, replaced); + new_blob =drm_property_lookup_blob (dev, color_pipeline_blob_id);if (!new_blob) {
Signature · 1 line
2.38.1
RFC 13/33 drm: Reset plane color state on pipeline switch request
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
When a pipeline switch is requested by user, driver resets blobs for all the hardware blocks to get to clean state. These are then populated with the new blob id's as programmed by user. For the already enabled hardware blocks, if the user does not add entry in the new switch request, the blob id's will remain NULL eventually resulting in disabling of that hardware block.
--- drivers/gpu/drm/drm_atomic_uapi.c | 52 ++++++++++++++++++++++++++++--- 1 file changed, 48 insertions(+), 4 deletions(-)
drivers/gpu/drm/drm_atomic_uapi.c +48 -4
@@ -362,6 +362,38 @@ static s32 __user *get_out_fence_for_connector(struct drm_atomic_state *state,return fence_ptr; } +static +bool color_pipeline_change_requested (struct drm_device *dev, +struct drm_property_blob *plane_cp_set_blob, +uint64_t blob_id) +{ +bool is_change_requested = false; +struct drm_property_blob *new_blob =NULL ; +struct drm_color_pipeline *old_cp, *new_cp; + +/* +* User is setting the pipeline for the first time +*/ +if (!plane_cp_set_blob) + goto out; + + old_cp = plane_cp_set_blob->data; + +if (blob_id !=0 ) { + new_blob =drm_property_lookup_blob (dev, blob_id); +if (!new_blob) + goto out; + + new_cp = new_blob->data; + +if (old_cp->num != new_cp->num) + is_change_requested = true; + } +drm_property_blob_put (new_blob); +out: +return is_change_requested; +} + static int drm_atomic_replace_property_blob_from_id(struct drm_device *dev, struct drm_property_blob **blob,
@@ -727,6 +759,12 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane, }else if (property == plane->scaling_filter_property) { state->scaling_filter = val; }else if (property == plane->set_color_pipeline_prop) { +bool cp_change_requested; + + cp_change_requested =color_pipeline_change_requested (dev, + state->set_color_pipeline_data, + val); + ret = drm_atomic_replace_property_blob_from_id(dev, &state->set_color_pipeline_data, val,
@@ -736,12 +774,18 @@ static int drm_atomic_plane_set_property(struct drm_plane *plane,if (replaced) {/* Consider actual color parameter change only when * individual color blobs are replaced. Hence, reset -* the replaced boolean. +* the replaced boolean but first reset all color +* blobs if color pipeline change is requested. */ +if (val && cp_change_requested) + ret =drm_plane_reset_color_op_blobs (plane, + state, &replaced); replaced = false; - ret =drm_plane_replace_color_op_blobs (plane, state, - val, - &replaced); +if (!ret) { + ret =drm_plane_replace_color_op_blobs (plane, state, + val, + &replaced); + } } state->color_mgmt_changed |= replaced;
Signature · 1 line
2.38.1
RFC 14/33 drm/i915/color: Add lut range for SDR planes
In reply to Uma ShankarAdd lut range information for SDR planes. This is used to hint the userspace what kind of LUT values are needed by the hardware block. Pre-CSC and Post-CSC blocks have similar lut range for HDR planes.
--- drivers/gpu/drm/i915/display/intel_color.c | 55 ++++++++++++++++++++++ 1 file changed, 55 insertions(+)
drivers/gpu/drm/i915/display/intel_color.c +55 -0
@@ -3778,6 +3778,61 @@ static const struct intel_color_funcs ilk_color_funcs = { .get_config = ilk_get_config, }; +/* FIXME input bpc? */ +static const struct drm_color_lut_range xelpd_pre_post_csc_sdr[] = { +/* segment 1 */ + { + .flags = (DRM_MODE_LUT_POST_CSC | +DRM_MODE_LUT_PRE_CSC | +DRM_MODE_LUT_REFLECT_NEGATIVE | +DRM_MODE_LUT_INTERPOLATE | +DRM_MODE_LUT_NON_DECREASING ), + .count =32 , + .input_bpc =16 , .output_bpc =16 , + .start =0 , .end = (1 <<16 ) - (1 <<16 ) /33 , + .min =0 , .max = (1 <<16 ) -1 , + }, +/* segment 2 */ + { + .flags = (DRM_MODE_LUT_POST_CSC | +DRM_MODE_LUT_PRE_CSC | +DRM_MODE_LUT_REFLECT_NEGATIVE | +DRM_MODE_LUT_INTERPOLATE | +DRM_MODE_LUT_REUSE_LAST | +DRM_MODE_LUT_NON_DECREASING ), + .count =1 , + .input_bpc =16 , .output_bpc =16 , + .start = (1 <<16 ) - (1 <<16 ) /33 , .end =1 <<16 , + .min =0 , .max =1 <<16 , + }, +/* Segment 3 */ + { + .flags = (DRM_MODE_LUT_POST_CSC | +DRM_MODE_LUT_PRE_CSC | +DRM_MODE_LUT_REFLECT_NEGATIVE | +DRM_MODE_LUT_INTERPOLATE | +DRM_MODE_LUT_REUSE_LAST | +DRM_MODE_LUT_NON_DECREASING ), + .count =1 , + .input_bpc =16 , .output_bpc =16 , + .start =1 <<16 , .end =3 <<16 , + .min =0 , .max = (8 <<16 ) -1 , + }, +/* Segment 4 */ + { + .flags = (DRM_MODE_LUT_POST_CSC | +DRM_MODE_LUT_PRE_CSC | +DRM_MODE_LUT_REFLECT_NEGATIVE | +DRM_MODE_LUT_INTERPOLATE | +DRM_MODE_LUT_REUSE_LAST | +DRM_MODE_LUT_NON_DECREASING ), + .count =1 , + .input_bpc =16 , .output_bpc =16 , + .start =3 <<16 , .end =7 <<16 , + .min =0 , .max = (8 <<16 ) -1 , + }, +}; +void intel_color_crtc_init (struct intel_crtc *crtc) {struct drm_i915_private *i915 =to_i915 (crtc->base.dev);
Signature · 1 line
2.38.1
RFC 15/33 drm/i915/color: Add lut range for HDR planes
In reply to Uma ShankarAdd lut range information for HDR planes. This is used to hint the userspace what kind of LUT values are needed by the hardware block. Pre-CSC and Post-CSC blocks have different lut ranges for HDR planes.
--- drivers/gpu/drm/i915/display/intel_color.c | 108 +++++++++++++++++++++ 1 file changed, 108 insertions(+)
drivers/gpu/drm/i915/display/intel_color.c +108 -0
@@ -3833,6 +3833,114 @@ static const struct drm_color_lut_range xelpd_pre_post_csc_sdr[] = { }, }; +/* FIXME input bpc? */ +static const struct drm_color_lut_range xelpd_pre_csc_hdr[] = { +/* segment 1 */ + { + .flags = (DRM_MODE_LUT_PRE_CSC | +DRM_MODE_LUT_REFLECT_NEGATIVE | +DRM_MODE_LUT_INTERPOLATE | +DRM_MODE_LUT_NON_DECREASING ), + .count =128 , + .input_bpc =24 , .output_bpc =16 , + .start =0 , .end = (1 <<24 ) -1 , + .min =0 , .max = (1 <<24 ) -1 , + }, +/* segment 2 */ + { + .flags = (DRM_MODE_LUT_PRE_CSC | +DRM_MODE_LUT_REFLECT_NEGATIVE | +DRM_MODE_LUT_INTERPOLATE | +DRM_MODE_LUT_REUSE_LAST | +DRM_MODE_LUT_NON_DECREASING ), + .count =1 , + .input_bpc =24 , .output_bpc =16 , + .start = (1 <<24 ) -1 , .end =1 <<24 , + .min =0 , .max = (1 <<27 ) -1 , + }, +/* Segment 3 */ + { + .flags = (DRM_MODE_LUT_PRE_CSC | +DRM_MODE_LUT_REFLECT_NEGATIVE | +DRM_MODE_LUT_INTERPOLATE | +DRM_MODE_LUT_REUSE_LAST | +DRM_MODE_LUT_NON_DECREASING ), + .count =1 , + .input_bpc =24 , .output_bpc =16 , + .start =1 <<24 , .end =3 <<24 , + .min =0 , .max = (1 <<27 ) -1 , + }, +/* Segment 4 */ + { + .flags = (DRM_MODE_LUT_PRE_CSC | +DRM_MODE_LUT_REFLECT_NEGATIVE | +DRM_MODE_LUT_INTERPOLATE | +DRM_MODE_LUT_REUSE_LAST | +DRM_MODE_LUT_NON_DECREASING ), + .count =1 , + .input_bpc =24 , .output_bpc =16 , + .start =3 <<24 , .end =7 <<24 , + .min =0 , .max = (1 <<27 ) -1 , + }, +}; + +/* FIXME input bpc? */ +static const struct drm_color_lut_range xelpd_post_csc_hdr[] = { +/* +* ToDo: Add Segment 1 +* There is an optional fine segment added with 9 lut values +* Will be added later +*/ + +/* segment 2 */ + { + .flags = (DRM_MODE_LUT_POST_CSC | +DRM_MODE_LUT_REFLECT_NEGATIVE | +DRM_MODE_LUT_INTERPOLATE | +DRM_MODE_LUT_NON_DECREASING ), + .count =32 , + .input_bpc =24 , .output_bpc =16 , + .start =0 , .end = (1 <<24 ) -1 , + .min =0 , .max = (1 <<24 ) -1 , + }, +/* segment 3 */ + { + .flags = (DRM_MODE_LUT_POST_CSC | +DRM_MODE_LUT_REFLECT_NEGATIVE | +DRM_MODE_LUT_INTERPOLATE | +DRM_MODE_LUT_REUSE_LAST | +DRM_MODE_LUT_NON_DECREASING ), + .count =1 , + .input_bpc =24 , .output_bpc =16 , + .start = (1 <<24 ) -1 , .end =1 <<24 , + .min =0 , .max =1 <<24 , + }, +/* Segment 4 */ + { + .flags = (DRM_MODE_LUT_POST_CSC | +DRM_MODE_LUT_REFLECT_NEGATIVE | +DRM_MODE_LUT_INTERPOLATE | +DRM_MODE_LUT_REUSE_LAST | +DRM_MODE_LUT_NON_DECREASING ), + .count =1 , + .input_bpc =24 , .output_bpc =16 , + .start =1 <<24 , .end =3 <<24 , + .min =0 , .max = (3 <<24 ), + }, +/* Segment 5 */ + { + .flags = (DRM_MODE_LUT_POST_CSC | +DRM_MODE_LUT_REFLECT_NEGATIVE | +DRM_MODE_LUT_INTERPOLATE | +DRM_MODE_LUT_REUSE_LAST | +DRM_MODE_LUT_NON_DECREASING ), + .count =1 , + .input_bpc =24 , .output_bpc =16 , + .start =3 <<24 , .end =7 <<24 , + .min =0 , .max = (7 <<24 ), + }, +}; +void intel_color_crtc_init (struct intel_crtc *crtc) {struct drm_i915_private *i915 =to_i915 (crtc->base.dev);
Signature · 1 line
2.38.1
RFC 16/33 drm/i915/color: Add color pipeline for HDR planes
In reply to Uma ShankarAdd color pipeline for HDR planes. It consists of the following hardware blocks.
* Pre-CSC : This block can used to linearize the input frame buffer data. The linear data then can be further acted on by the following color hardware blocks in the display hardware pipeline
* CSC/CTM: Used to program color transformation matrix, this block is used to perform color space conversions like BT2020 to BT709 or BT601 etc. This block acts on the linearized data coming from the Pre-CSC HW block.
* Post-CSC: This HW block can be used to non-linearize frame buffer data to match the sink. Another use case of it could be to perform Tone mapping for HDR use-cases.
--- drivers/gpu/drm/i915/display/intel_color.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
drivers/gpu/drm/i915/display/intel_color.c +18 -0
@@ -3941,6 +3941,24 @@ static const struct drm_color_lut_range xelpd_post_csc_hdr[] = { }, }; +struct drm_color_op color_pipeline_hdr[] = { + { + .name =DRM_CB_PRE_CSC , + .type =CURVE_1D , + .blob_id =0 ,/* To be updated during plane initialization */ + }, + { + .name =DRM_CB_CSC , + .type =MATRIX , + .blob_id =0 , + }, + { + .name =DRM_CB_POST_CSC , + .type =CURVE_1D , + .blob_id =0 , + }, +}; +void intel_color_crtc_init (struct intel_crtc *crtc) {struct drm_i915_private *i915 =to_i915 (crtc->base.dev);
Signature · 1 line
2.38.1
RFC 17/33 drm/i915/color: Add color pipeline for SDR planes
In reply to Uma ShankarSDR planes provides programmable color hardware blocks for Pre-CSC and Post-CSC operations. Add a color pipeline to expose these capabilities.
--- drivers/gpu/drm/i915/display/intel_color.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
drivers/gpu/drm/i915/display/intel_color.c +17 -0
@@ -3941,6 +3941,23 @@ static const struct drm_color_lut_range xelpd_post_csc_hdr[] = { }, }; +struct drm_color_op color_pipeline_sdr[] = { + { + .name =DRM_CB_PRE_CSC , + .type =CURVE_1D , + .blob_id =0 ,/* To be updated during plane initialization */ + }, +/* +* SDR planes have fixed function CSC capabilities. +* TODO: Add support for it +*/ + { + .name =DRM_CB_POST_CSC , + .type =CURVE_1D , + .blob_id =0 , + }, +}; +struct drm_color_op color_pipeline_hdr[] = { { .name =DRM_CB_PRE_CSC ,
Signature · 1 line
2.38.1
RFC 18/33 drm/i915/color: Add HDR plane LUT range data to color pipeline
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
Create a helper function to add details about LUT ranges that HDR planes can support. Userspace can parse through this information to generate proper LUT data for respective hardware blocks. It will be exposed to the user space by the color pipeline.
--- drivers/gpu/drm/i915/display/intel_color.c | 47 ++++++++++++++++++++++ 1 file changed, 47 insertions(+)
drivers/gpu/drm/i915/display/intel_color.c +47 -0
@@ -28,6 +28,7 @@#include "intel_de.h" #include "intel_display_types.h" #include "intel_dsb.h" +#include "skl_universal_plane.h" struct intel_color_funcs {int (*color_check)(struct intel_crtc_state *crtc_state);
@@ -3976,6 +3977,52 @@ struct drm_color_op color_pipeline_hdr[] = { }, }; +__maybe_unused +static int intel_prepare_plane_color_pipeline (struct drm_plane *plane) +{ +struct drm_i915_private *i915 =to_i915 (plane->dev); +struct drm_property_blob *blob[2 ] = {NULL }; +int ret =0 , i =0 ; + +if (icl_is_hdr_plane (i915,to_intel_plane (plane)->id)) { + blob[i] =drm_property_create_blob (plane->dev, +sizeof (xelpd_pre_csc_hdr), + xelpd_pre_csc_hdr); +if (IS_ERR (blob[i])) { + ret =PTR_ERR (blob[i]); + goto out; + } + +/* +* In HDR color pipeline PRE-CSC and POST-CSC are positioned +* at 0th and 2nd index/position +*/ + color_pipeline_hdr[0 ].blob_id = + blob[i++]->base.id; + + blob[i] =drm_property_create_blob (plane->dev, +sizeof (xelpd_post_csc_hdr), + xelpd_post_csc_hdr); +if (IS_ERR (blob[i])) { + ret =PTR_ERR (blob[i]); + goto out; + } + + color_pipeline_hdr[2 ].blob_id = + blob[i++]->base.id; + } + +out: +if (ret) { +for (int j =0 ; j < i; j++) { +if (blob[j]) +drm_property_blob_put (blob[j]); + } + } + +return ret; +}; +void intel_color_crtc_init (struct intel_crtc *crtc) {struct drm_i915_private *i915 =to_i915 (crtc->base.dev);
Signature · 1 line
2.38.1
RFC 19/33 drm/i915/color: Add SDR plane LUT range data to color pipeline
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
Add LUT ranges for color blocks in SDR planes. Userspace can parse through this information to generate proper LUT data for respective hardware blocks. It will be exposed to the user space by the color pipeline.
--- drivers/gpu/drm/i915/display/intel_color.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
drivers/gpu/drm/i915/display/intel_color.c +16 -0
@@ -4010,6 +4010,22 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane) color_pipeline_hdr[2 ].blob_id = blob[i++]->base.id; + }else { + blob[i] =drm_property_create_blob (plane->dev, +sizeof (xelpd_pre_post_csc_sdr), + xelpd_pre_post_csc_sdr); +if (IS_ERR (blob[i])) { + ret =PTR_ERR (blob[i]); + goto out; + } + +/* +* In SDR color pipeline PRE-CSC and POST-CSC blocks are positioned +* at 0th and 1st index/postion. +* LUT ranges for SDR planes are similar for pre and post-csc blocks +*/ + color_pipeline_sdr[0 ].blob_id = + color_pipeline_sdr[1 ].blob_id = blob[i++]->base.id; } out:
Signature · 1 line
2.38.1
RFC 20/33 drm/i915/color: Add color pipelines to plane
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
Add supported plane color pipelines. To represent all hardware blocks in their inactive state, we introduce a pipeline called "no color pipeline" which is the default pipeline. Add respective color pipelines for SDR and HDR planes. Create and attach plane enum property "GET_COLOR_PIPELINE" to expose these pipelines to userspace.
--- drivers/gpu/drm/i915/display/intel_color.c | 31 +++++++++++++++++++++- drivers/gpu/drm/i915/display/intel_color.h | 3 ++- 2 files changed, 32 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/display/intel_color.c +30 -1
@@ -3977,7 +3977,6 @@ struct drm_color_op color_pipeline_hdr[] = { }, }; -__maybe_unused static int intel_prepare_plane_color_pipeline (struct drm_plane *plane) {struct drm_i915_private *i915 =to_i915 (plane->dev);
@@ -4039,6 +4038,36 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)return ret; }; +__maybe_unused +void intel_color_plane_init (struct drm_plane *plane) +{ +struct drm_i915_private *i915 =to_i915 (plane->dev); + +if (DISPLAY_VER (i915) <13 ) +return ; + +drm_plane_create_get_color_pipeline_property (plane->dev, plane,2 ); + +intel_prepare_plane_color_pipeline (plane); + +/* +* default pipeline is set as 0 or "no color pipeline". All color h/w +* blocks are disabled at this stage. +*/ +drm_plane_add_color_pipeline (plane,"no color pipeline" ,NULL ,0 ); + +if (icl_is_hdr_plane (i915,to_intel_plane (plane)->id)) +drm_plane_add_color_pipeline (plane,"color pipeline hdr" , + color_pipeline_hdr, +sizeof (color_pipeline_hdr)); +else +drm_plane_add_color_pipeline (plane,"color pipeline sdr" , + color_pipeline_sdr, +sizeof (color_pipeline_sdr)); + +drm_plane_attach_get_color_pipeline_property (plane); +} +void intel_color_crtc_init (struct intel_crtc *crtc) {struct drm_i915_private *i915 =to_i915 (crtc->base.dev);
drivers/gpu/drm/i915/display/intel_color.h +2 -1
@@ -10,6 +10,7 @@struct intel_crtc_state ;struct intel_crtc ; +struct drm_plane ;struct drm_i915_private ;struct drm_property_blob ;
@@ -29,5 +30,5 @@ bool intel_color_lut_equal(const struct intel_crtc_state *crtc_state,const struct drm_property_blob *blob2, bool is_pre_csc_lut);void intel_color_assert_luts (const struct intel_crtc_state *crtc_state); - +void intel_color_plane_init (struct drm_plane *plane);#endif /* __INTEL_COLOR_H__ */
Signature · 1 line
2.38.1
RFC 21/33 drm/i915/color: Create and attach set color pipeline property
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
Create and attach "SET_COLOR_PIPELINE" property to planes.
--- drivers/gpu/drm/i915/display/intel_color.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/gpu/drm/i915/display/intel_color.c +3 -0
@@ -4066,6 +4066,9 @@ void intel_color_plane_init(struct drm_plane *plane) sizeof(color_pipeline_sdr));drm_plane_attach_get_color_pipeline_property (plane); + +drm_plane_create_set_color_pipeline_property (plane->dev, plane); +drm_plane_attach_set_color_pipeline_property (plane); }void intel_color_crtc_init (struct intel_crtc *crtc)
Signature · 1 line
2.38.1
RFC 22/33 drm/i915/color: Add plane color callbacks
In reply to Uma ShankarAdd callbacks for color plane operations.
load_plane_luts: used to load pre/post csc lutsload_plane_csc_matrix: used to load csc matrix
--- drivers/gpu/drm/i915/display/intel_color.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/gpu/drm/i915/display/intel_color.c +6 -0
@@ -81,6 +81,12 @@ struct intel_color_funcs {* Read config other than LUTs and CSCs, before them. Optional. */ void (*get_config)(struct intel_crtc_state *crtc_state); + +/* +* Plane color callbacks +*/ +void (*load_plane_csc_matrix)(const struct drm_plane_state *plane_state); +void (*load_plane_luts)(const struct drm_plane_state *plane_state); };#define CTM_COEFF_SIGN (1ULL << 63)
Signature · 1 line
2.38.1
RFC 23/33 drm/i915/color: Load plane color luts from atomic flip
In reply to Uma ShankarLoad plane color luts as part of atomic plane updates. This will be done only if the plane color luts are changed.
--- drivers/gpu/drm/i915/display/intel_color.c | 8 ++++++++ drivers/gpu/drm/i915/display/intel_color.h | 2 ++ drivers/gpu/drm/i915/display/skl_universal_plane.c | 4 ++++ 3 files changed, 14 insertions(+)
drivers/gpu/drm/i915/display/intel_color.c +8 -0
@@ -1863,6 +1863,14 @@ void intel_color_load_luts(const struct intel_crtc_state *crtc_state) i915->display.funcs.color->load_luts (crtc_state); } +void intel_color_load_plane_luts (const struct drm_plane_state *plane_state) +{ +struct drm_i915_private *i915 =to_i915 (plane_state->plane->dev); + +if (i915->display.funcs.color->load_plane_luts) + i915->display.funcs.color->load_plane_luts (plane_state); +} +void intel_color_commit_noarm (const struct intel_crtc_state *crtc_state) {struct drm_i915_private *i915 =to_i915 (crtc_state->uapi.crtc->dev);
drivers/gpu/drm/i915/display/intel_color.h +2 -0
@@ -13,6 +13,7 @@ struct intel_crtc;struct drm_plane ;struct drm_i915_private ;struct drm_property_blob ; +struct drm_plane_state ;void intel_color_init_hooks (struct drm_i915_private *i915);int intel_color_init (struct drm_i915_private *i915);
@@ -31,4 +32,5 @@ bool intel_color_lut_equal(const struct intel_crtc_state *crtc_state, bool is_pre_csc_lut);void intel_color_assert_luts (const struct intel_crtc_state *crtc_state);void intel_color_plane_init (struct drm_plane *plane); +void intel_color_load_plane_luts (const struct drm_plane_state *plane_state);#endif /* __INTEL_COLOR_H__ */
drivers/gpu/drm/i915/display/skl_universal_plane.c +4 -0
@@ -11,6 +11,7 @@#include "i915_drv.h" #include "i915_reg.h" #include "intel_atomic_plane.h" +#include "intel_color.h" #include "intel_de.h" #include "intel_display_irq.h" #include "intel_display_types.h"
@@ -1268,6 +1269,9 @@ icl_plane_update_noarm(struct intel_plane *plane,if (plane_state->force_black)icl_plane_csc_load_black (plane); +if (plane_state->uapi.color_mgmt_changed) +intel_color_load_plane_luts (&plane_state->uapi); +intel_psr2_program_plane_sel_fetch_noarm (plane, crtc_state, plane_state, color_plane); }
Signature · 1 line
2.38.1
RFC 24/33 drm/i915/xelpd: Add plane color check to glk_plane_color_ctl
In reply to Uma ShankarExtended glk_plane_color_ctl to have plane color checks. This helps enabling the csc, degamma or gamma block based on user inputs.
--- drivers/gpu/drm/i915/display/skl_universal_plane.c | 13 ++++++++++++- drivers/gpu/drm/i915/i915_reg.h | 1 + 2 files changed, 13 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/display/skl_universal_plane.c +12 -1
@@ -965,7 +965,18 @@ static u32 glk_plane_color_ctl(const struct intel_crtc_state *crtc_state,struct intel_plane *plane =to_intel_plane (plane_state->uapi.plane);u32 plane_color_ctl =0 ; - plane_color_ctl |=PLANE_COLOR_PLANE_GAMMA_DISABLE ; +/* FIXME needs hw.gamma_lut */ +if (!plane_state->uapi.color.pre_csc_lut) + plane_color_ctl |=PLANE_COLOR_PLANE_GAMMA_DISABLE ; + +/* FIXME needs hw.degamma_lut */ +if (plane_state->uapi.color.post_csc_lut) + plane_color_ctl |=PLANE_COLOR_PRE_CSC_GAMMA_ENABLE ; + +/* FIXME needs hw.ctm */ +if (plane_state->uapi.color.ctm) + plane_color_ctl |=PLANE_COLOR_PLANE_CSC_ENABLE ; + plane_color_ctl |=glk_plane_color_ctl_alpha (plane_state);if (fb->format->is_yuv && !icl_is_hdr_plane (dev_priv, plane->id)) {
drivers/gpu/drm/i915/i915_reg.h +1 -0
@@ -3730,6 +3730,7 @@#define PLANE_COLOR_CSC_MODE_YUV709_TO_RGB709 REG_FIELD_PREP(PLANE_COLOR_CSC_MODE_MASK, 2)#define PLANE_COLOR_CSC_MODE_YUV2020_TO_RGB2020 REG_FIELD_PREP(PLANE_COLOR_CSC_MODE_MASK, 3)#define PLANE_COLOR_CSC_MODE_RGB709_TO_RGB2020 REG_FIELD_PREP(PLANE_COLOR_CSC_MODE_MASK, 4) +#define PLANE_COLOR_PRE_CSC_GAMMA_ENABLE REG_BIT(14)#define PLANE_COLOR_PLANE_GAMMA_DISABLE REG_BIT(13)#define PLANE_COLOR_ALPHA_MASK REG_GENMASK(5, 4)#define PLANE_COLOR_ALPHA_DISABLE REG_FIELD_PREP(PLANE_COLOR_ALPHA_MASK, 0)
Signature · 1 line
2.38.1
RFC 25/33 drm/i915/xelpd: Add register definitions for Plane Degamma
In reply to Uma ShankarAdd macros to define Plane Degamma registers
--- drivers/gpu/drm/i915/i915_reg.h | 49 +++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)
drivers/gpu/drm/i915/i915_reg.h +49 -0
@@ -6226,6 +6226,55 @@ enum skl_power_gate {#define _MMIO_PLANE_GAMC (plane, i, a, b) _MMIO(_PIPE(plane, a, b) + (i) * 4) +/* Display13 Plane Degmma Reg */ +#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_A 0x701d0 +#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_B 0x711d0 +#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_2_A 0x702d0 +#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_2_B 0x712d0 +#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1 (pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_A, \ + _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_B) +#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_2 (pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_INDEX_ENH_2_A, \ + _PLANE_PRE_CSC_GAMC_INDEX_ENH_2_B) +#define PLANE_PRE_CSC_GAMC_INDEX_ENH (pipe, plane, i) \ + _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_INDEX_ENH_1(pipe), \ + _PLANE_PRE_CSC_GAMC_INDEX_ENH_2(pipe)) + +#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1_A 0x701d4 +#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1_B 0x711d4 +#define _PLANE_PRE_CSC_GAMC_DATA_ENH_2_A 0x702d4 +#define _PLANE_PRE_CSC_GAMC_DATA_ENH_2_B 0x712d4 +#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1 (pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_DATA_ENH_1_A, \ + _PLANE_PRE_CSC_GAMC_DATA_ENH_1_B) +#define _PLANE_PRE_CSC_GAMC_DATA_ENH_2 (pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_DATA_ENH_2_A, \ + _PLANE_PRE_CSC_GAMC_DATA_ENH_2_B) +#define PLANE_PRE_CSC_GAMC_DATA_ENH (pipe, plane, i) \ + _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_DATA_ENH_1(pipe), \ + _PLANE_PRE_CSC_GAMC_DATA_ENH_2(pipe)) + +#define _PLANE_PRE_CSC_GAMC_INDEX_1_A 0x704d0 +#define _PLANE_PRE_CSC_GAMC_INDEX_1_B 0x714d0 +#define _PLANE_PRE_CSC_GAMC_INDEX_2_A 0x705d0 +#define _PLANE_PRE_CSC_GAMC_INDEX_2_B 0x715d0 +#define _PLANE_PRE_CSC_GAMC_INDEX_1 (pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_INDEX_1_A, \ + _PLANE_PRE_CSC_GAMC_INDEX_1_B) +#define _PLANE_PRE_CSC_GAMC_INDEX_2 (pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_INDEX_2_A, \ + _PLANE_PRE_CSC_GAMC_INDEX_2_B) +#define PLANE_PRE_CSC_GAMC_INDEX (pipe, plane, i) \ + _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_INDEX_1(pipe), \ + _PLANE_PRE_CSC_GAMC_INDEX_2(pipe)) + +#define _PLANE_PRE_CSC_GAMC_DATA_1_A 0x704d4 +#define _PLANE_PRE_CSC_GAMC_DATA_1_B 0x714d4 +#define _PLANE_PRE_CSC_GAMC_DATA_2_A 0x705d4 +#define _PLANE_PRE_CSC_GAMC_DATA_2_B 0x715d4 +#define _PLANE_PRE_CSC_GAMC_DATA_1 (pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_DATA_1_A, \ + _PLANE_PRE_CSC_GAMC_DATA_1_B) +#define _PLANE_PRE_CSC_GAMC_DATA_2 (pipe) _PIPE(pipe, _PLANE_PRE_CSC_GAMC_DATA_2_A, \ + _PLANE_PRE_CSC_GAMC_DATA_2_B) +#define PLANE_PRE_CSC_GAMC_DATA (pipe, plane, i) \ + _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_DATA_1(pipe), \ + _PLANE_PRE_CSC_GAMC_DATA_2(pipe)) +/* Plane CSC Registers */ #define _PLANE_CSC_RY_GY_1_A 0x70210#define _PLANE_CSC_RY_GY_2_A 0x70310
Signature · 1 line
2.38.1
RFC 26/33 drm/i915/color: Add color functions for ADL
In reply to Uma ShankarRegister color callbacks for ADL and beyond. While we have to register new callbacks for pre-blending color operations, re-use callbacks for post-blend operations.
--- drivers/gpu/drm/i915/display/intel_color.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/display/intel_color.c +13 -1
@@ -3704,6 +3704,16 @@ static const struct intel_color_funcs i9xx_color_funcs = { .get_config = i9xx_get_config, }; +static const struct intel_color_funcs xelpd_color_funcs = { + .color_check = icl_color_check, + .color_commit_noarm = icl_color_commit_noarm, + .color_commit_arm = icl_color_commit_arm, + .load_luts = icl_load_luts, + .read_luts = icl_read_luts, + .lut_equal = icl_lut_equal, + .read_csc = icl_read_csc, +}; +static const struct intel_color_funcs tgl_color_funcs = { .color_check = icl_color_check, .color_commit_noarm = icl_color_commit_noarm,
@@ -4141,7 +4151,9 @@ void intel_color_init_hooks(struct drm_i915_private *i915)else i915->display.funcs.color = &i9xx_color_funcs; }else { -if (DISPLAY_VER (i915) >=12 ) +if (DISPLAY_VER (i915) >=13 ) + i915->display.funcs.color = &xelpd_color_funcs; +else if (DISPLAY_VER (i915) ==12 ) i915->display.funcs.color = &tgl_color_funcs;else if (DISPLAY_VER (i915) ==11 ) i915->display.funcs.color = &icl_color_funcs;
Signature · 1 line
2.38.1
RFC 27/33 drm/i915/color: Program Plane Pre-CSC Registers
In reply to Uma ShankarExtract the LUT and program plane pre-csc registers.
--- drivers/gpu/drm/i915/display/intel_color.c | 120 +++++++++++++++++++++ drivers/gpu/drm/i915/i915_reg.h | 1 + 2 files changed, 121 insertions(+)
drivers/gpu/drm/i915/display/intel_color.c +120 -0
@@ -185,6 +185,29 @@ static bool lut_is_legacy(const struct drm_property_blob *lut)return lut &&drm_color_lut_size (lut) ==LEGACY_LUT_LENGTH ; } +/* +* Added to accommodate enhanced LUT precision. +* Max LUT precision is 32 bits. +*/ +static u64 drm_color_lut_extract_ext (u64 user_input,u32 bit_precision) +{ +u64 val = user_input &0xffffffff ; +u32 max; + +if (bit_precision >32 ) +return 0 ; + + max =0xffffffff >> (32 - bit_precision); +/* Round only if we're not using full precision. */ +if (bit_precision <32 ) { + val +=1UL << (32 - bit_precision -1 ); + val >>=32 - bit_precision; + } + +return ((user_input &0xffffffff00000000 ) | +clamp_val (val,0 , max)); +} +/* * When using limited range, multiply the matrix given by userspace by * the matrix that we would use for the limited range.
@@ -1856,6 +1879,102 @@ static void chv_load_luts(const struct intel_crtc_state *crtc_state) crtc_state->cgm_mode); } +static void xelpd_program_plane_pre_csc_lut (const struct drm_plane_state *state, +struct drm_color_lut_ext *pre_csc_lut, +u32 offset) +{ +struct drm_i915_private *dev_priv =to_i915 (state->plane->dev); +enum pipe pipe =to_intel_plane (state->plane)->pipe; +enum plane_id plane =to_intel_plane (state->plane)->id; +u32 i, lut_size; + +if (icl_is_hdr_plane (dev_priv, plane)) { + lut_size =128 ; + +intel_de_write_fw (dev_priv,PLANE_PRE_CSC_GAMC_INDEX_ENH (pipe, plane,0 ), +PLANE_PAL_PREC_AUTO_INCREMENT ); + +if (pre_csc_lut) { +for (i =0 ; i < lut_size; i++) { +u64 word =drm_color_lut_extract_ext (pre_csc_lut[i].green,24 ); +u32 lut_val = (word &0xffffff ); + +intel_de_write_fw (dev_priv, +PLANE_PRE_CSC_GAMC_DATA_ENH (pipe, plane,0 ), + lut_val); + } + +/* Program the max register to clamp values > 1.0. */ +while (i <131 ) +intel_de_write_fw (dev_priv, +PLANE_PRE_CSC_GAMC_DATA_ENH (pipe, plane,0 ), + pre_csc_lut[i++].green); + }else { +for (i =0 ; i < lut_size; i++) { +u32 v = (i * ((1 <<24 ) -1 )) / (lut_size -1 ); + +intel_de_write_fw (dev_priv, +PLANE_PRE_CSC_GAMC_DATA_ENH (pipe, plane,0 ), v); + } + +do { +intel_de_write_fw (dev_priv, +PLANE_PRE_CSC_GAMC_DATA_ENH (pipe, plane,0 ), +1 <<24 ); + }while (i++ <130 ); + } + +intel_de_write_fw (dev_priv,PLANE_PRE_CSC_GAMC_INDEX_ENH (pipe, plane,0 ),0 ); + }else { + lut_size =32 ; + +/* +* First 3 planes are HDR, so reduce by 3 to get to the right +* SDR plane offset +*/ + plane = plane -3 ; + +intel_de_write_fw (dev_priv,PLANE_PRE_CSC_GAMC_INDEX (pipe, plane,0 ), +PLANE_PAL_PREC_AUTO_INCREMENT ); + +if (pre_csc_lut) { +for (i =0 ; i < lut_size; i++) +intel_de_write_fw (dev_priv,PLANE_PRE_CSC_GAMC_DATA (pipe, plane,0 ), + pre_csc_lut[i].green); +/* Program the max register to clamp values > 1.0. */ +while (i <35 ) +intel_de_write_fw (dev_priv,PLANE_PRE_CSC_GAMC_DATA (pipe, plane,0 ), + pre_csc_lut[i++].green); + }else { +for (i =0 ; i < lut_size; i++) { +u32 v = (i * ((1 <<16 ) -1 )) / (lut_size -1 ); + +intel_de_write_fw (dev_priv, +PLANE_PRE_CSC_GAMC_DATA (pipe, plane,0 ), v); + } + +do { +intel_de_write_fw (dev_priv,PLANE_PRE_CSC_GAMC_DATA (pipe, plane,0 ), +1 <<16 ); + }while (i++ <34 ); + } + +intel_de_write_fw (dev_priv,PLANE_PRE_CSC_GAMC_INDEX (pipe, plane,0 ),0 ); + } +} + +static void xelpd_plane_load_luts (const struct drm_plane_state *plane_state) +{ +const struct drm_property_blob *pre_csc_lut_blob = + plane_state->color.pre_csc_lut; +struct drm_color_lut_ext *pre_csc_lut =NULL ; + +if (pre_csc_lut_blob) { + pre_csc_lut = pre_csc_lut_blob->data; +xelpd_program_plane_pre_csc_lut (plane_state, pre_csc_lut,0 ); + } +} +void intel_color_load_luts (const struct intel_crtc_state *crtc_state) {struct drm_i915_private *i915 =to_i915 (crtc_state->uapi.crtc->dev);
@@ -3712,6 +3831,7 @@ static const struct intel_color_funcs xelpd_color_funcs = { .read_luts = icl_read_luts, .lut_equal = icl_lut_equal, .read_csc = icl_read_csc, + .load_plane_luts = xelpd_plane_load_luts, };static const struct intel_color_funcs tgl_color_funcs = {
drivers/gpu/drm/i915/i915_reg.h +1 -0
@@ -6238,6 +6238,7 @@ enum skl_power_gate {#define PLANE_PRE_CSC_GAMC_INDEX_ENH (pipe, plane, i) \ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_INDEX_ENH_1(pipe), \ _PLANE_PRE_CSC_GAMC_INDEX_ENH_2(pipe)) +#define PLANE_PAL_PREC_AUTO_INCREMENT REG_BIT(10)#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1_A 0x701d4#define _PLANE_PRE_CSC_GAMC_DATA_ENH_1_B 0x711d4
Signature · 1 line
2.38.1
RFC 28/33 drm/i915/xelpd: Add register definitions for Plane Post CSC
In reply to Uma ShankarAdd macros to define Plane Post CSC registers
--- drivers/gpu/drm/i915/i915_reg.h | 73 +++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+)
drivers/gpu/drm/i915/i915_reg.h +73 -0
@@ -6226,6 +6226,79 @@ enum skl_power_gate {#define _MMIO_PLANE_GAMC (plane, i, a, b) _MMIO(_PIPE(plane, a, b) + (i) * 4) +/* Display13 Plane Gamma Reg */ +#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_A 0x70160 +#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_B 0x71160 +#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_A 0x70260 +#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_B 0x71260 +#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1 (pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_A, \ + _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_B) +#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2 (pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_A, \ + _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_B) +#define PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH (pipe, plane, i) \ + _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1(pipe), \ + _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2(pipe)) + +#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_A 0x70164 +#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_B 0x71164 +#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_A 0x70264 +#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_B 0x71264 +#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1 (pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_A, \ + _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_B) +#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2 (pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_A, \ + _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_B) +#define PLANE_POST_CSC_GAMC_SEG0_DATA_ENH (pipe, plane, i) \ + _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1(pipe), \ + _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2(pipe)) + +#define _PLANE_POST_CSC_GAMC_INDEX_ENH_1_A 0x701d8 +#define _PLANE_POST_CSC_GAMC_INDEX_ENH_1_B 0x711d8 +#define _PLANE_POST_CSC_GAMC_INDEX_ENH_2_A 0x702d8 +#define _PLANE_POST_CSC_GAMC_INDEX_ENH_2_B 0x712d8 +#define _PLANE_POST_CSC_GAMC_INDEX_ENH_1 (pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_INDEX_ENH_1_A, \ + _PLANE_POST_CSC_GAMC_INDEX_ENH_1_B) +#define _PLANE_POST_CSC_GAMC_INDEX_ENH_2 (pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_INDEX_ENH_2_A, \ + _PLANE_POST_CSC_GAMC_INDEX_ENH_2_B) +#define PLANE_POST_CSC_GAMC_INDEX_ENH (pipe, plane, i) \ + _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_INDEX_ENH_1(pipe), \ + _PLANE_POST_CSC_GAMC_INDEX_ENH_2(pipe)) + +#define _PLANE_POST_CSC_GAMC_DATA_ENH_1_A 0x701dc +#define _PLANE_POST_CSC_GAMC_DATA_ENH_1_B 0x711dc +#define _PLANE_POST_CSC_GAMC_DATA_ENH_2_A 0x702dc +#define _PLANE_POST_CSC_GAMC_DATA_ENH_2_B 0x712dc +#define _PLANE_POST_CSC_GAMC_DATA_ENH_1 (pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_DATA_ENH_1_A, \ + _PLANE_POST_CSC_GAMC_DATA_ENH_1_B) +#define _PLANE_POST_CSC_GAMC_DATA_ENH_2 (pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_DATA_ENH_2_A, \ + _PLANE_POST_CSC_GAMC_DATA_ENH_2_B) +#define PLANE_POST_CSC_GAMC_DATA_ENH (pipe, plane, i) \ + _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_DATA_ENH_1(pipe), \ + _PLANE_POST_CSC_GAMC_DATA_ENH_2(pipe)) + +#define _PLANE_POST_CSC_GAMC_INDEX_1_A 0x704d8 +#define _PLANE_POST_CSC_GAMC_INDEX_1_B 0x714d8 +#define _PLANE_POST_CSC_GAMC_INDEX_2_A 0x705d8 +#define _PLANE_POST_CSC_GAMC_INDEX_2_B 0x715d8 +#define _PLANE_POST_CSC_GAMC_INDEX_1 (pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_INDEX_1_A, \ + _PLANE_POST_CSC_GAMC_INDEX_1_B) +#define _PLANE_POST_CSC_GAMC_INDEX_2 (pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_INDEX_2_A, \ + _PLANE_POST_CSC_GAMC_INDEX_2_B) +#define PLANE_POST_CSC_GAMC_INDEX (pipe, plane, i) \ + _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_INDEX_1(pipe), \ + _PLANE_POST_CSC_GAMC_INDEX_2(pipe)) + +#define _PLANE_POST_CSC_GAMC_DATA_1_A 0x704dc +#define _PLANE_POST_CSC_GAMC_DATA_1_B 0x714dc +#define _PLANE_POST_CSC_GAMC_DATA_2_A 0x705dc +#define _PLANE_POST_CSC_GAMC_DATA_2_B 0x715dc +#define _PLANE_POST_CSC_GAMC_DATA_1 (pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_DATA_1_A, \ + _PLANE_POST_CSC_GAMC_DATA_1_B) +#define _PLANE_POST_CSC_GAMC_DATA_2 (pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_DATA_2_A, \ + _PLANE_POST_CSC_GAMC_DATA_2_B) +#define PLANE_POST_CSC_GAMC_DATA (pipe, plane, i) \ + _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_DATA_1(pipe), \ + _PLANE_POST_CSC_GAMC_DATA_2(pipe)) +/* Display13 Plane Degmma Reg */ #define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_A 0x701d0#define _PLANE_PRE_CSC_GAMC_INDEX_ENH_1_B 0x711d0
Signature · 1 line
2.38.1
RFC 29/33 drm/i915/xelpd: Program Plane Post CSC Registers
In reply to Uma ShankarExtract the LUT and program plane post csc registers.
--- drivers/gpu/drm/i915/display/intel_color.c | 95 +++++++++++++++++++++- 1 file changed, 94 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/display/intel_color.c +94 -1
@@ -1963,16 +1963,109 @@ static void xelpd_program_plane_pre_csc_lut(const struct drm_plane_state *state, } } +static void xelpd_program_plane_post_csc_lut (const struct drm_plane_state *state, +struct drm_color_lut_ext *post_csc_lut, +u32 offset) +{ +struct drm_i915_private *dev_priv =to_i915 (state->plane->dev); +enum pipe pipe =to_intel_plane (state->plane)->pipe; +enum plane_id plane =to_intel_plane (state->plane)->id; +u32 i, lut_size; + +if (icl_is_hdr_plane (dev_priv, plane)) { +intel_de_write_fw (dev_priv,PLANE_POST_CSC_GAMC_INDEX_ENH (pipe, plane,0 ), + offset |PLANE_PAL_PREC_AUTO_INCREMENT ); +if (post_csc_lut) { + lut_size =32 ; +for (i =0 ; i < lut_size; i++) { +u64 word =drm_color_lut_extract_ext (post_csc_lut[i].green,24 ); +u32 lut_val = (word &0xffffff ); + +intel_de_write_fw (dev_priv, +PLANE_POST_CSC_GAMC_DATA_ENH (pipe, plane,0 ), + lut_val); + } + +do { +/* Program the max register to clamp values > 1.0. */ +intel_de_write_fw (dev_priv, +PLANE_POST_CSC_GAMC_DATA_ENH (pipe, plane,0 ), + post_csc_lut[i].green); + }while (i++ <34 ); + }else { + lut_size =32 ; +for (i =0 ; i < lut_size; i++) { +u32 v = (i * ((1 <<24 ) -1 )) / (lut_size -1 ); + +intel_de_write_fw (dev_priv, +PLANE_POST_CSC_GAMC_DATA_ENH (pipe, plane,0 ), v); + } + +do { +intel_de_write_fw (dev_priv, +PLANE_POST_CSC_GAMC_DATA_ENH (pipe, plane,0 ), +1 <<24 ); + }while (i++ <34 ); + } + +intel_de_write_fw (dev_priv,PLANE_POST_CSC_GAMC_INDEX_ENH (pipe, plane,0 ),0 ); + }else { + lut_size =32 ; +/* +* First 3 planes are HDR, so reduce by 3 to get to the right +* SDR plane offset +*/ + plane = plane -3 ; + +intel_de_write_fw (dev_priv,PLANE_POST_CSC_GAMC_INDEX (pipe, plane,0 ), + offset |PLANE_PAL_PREC_AUTO_INCREMENT ); + +if (post_csc_lut) { +for (i =0 ; i < lut_size; i++) +intel_de_write_fw (dev_priv, +PLANE_POST_CSC_GAMC_DATA (pipe, plane,0 ), + post_csc_lut[i].green &0xffff ); +/* Program the max register to clamp values > 1.0. */ +while (i <35 ) +intel_de_write_fw (dev_priv, +PLANE_POST_CSC_GAMC_DATA (pipe, plane,0 ), + post_csc_lut[i++].green &0x3ffff ); + }else { +for (i =0 ; i < lut_size; i++) { +u32 v = (i * ((1 <<16 ) -1 )) / (lut_size -1 ); + +intel_de_write_fw (dev_priv, +PLANE_POST_CSC_GAMC_DATA (pipe, plane,0 ), v); + } + +do { +intel_de_write_fw (dev_priv, +PLANE_POST_CSC_GAMC_DATA (pipe, plane,0 ), + (1 <<16 )); + }while (i++ <34 ); + } + +intel_de_write_fw (dev_priv,PLANE_POST_CSC_GAMC_INDEX (pipe, plane,0 ),0 ); + } +} +static void xelpd_plane_load_luts (const struct drm_plane_state *plane_state) {const struct drm_property_blob *pre_csc_lut_blob = plane_state->color.pre_csc_lut; -struct drm_color_lut_ext *pre_csc_lut =NULL ; +const struct drm_property_blob *post_csc_lut_blob = + plane_state->color.post_csc_lut; +struct drm_color_lut_ext *pre_csc_lut, *post_csc_lut;if (pre_csc_lut_blob) { pre_csc_lut = pre_csc_lut_blob->data;xelpd_program_plane_pre_csc_lut (plane_state, pre_csc_lut,0 ); } + +if (post_csc_lut_blob) { + post_csc_lut = post_csc_lut_blob->data; +xelpd_program_plane_post_csc_lut (plane_state, post_csc_lut,0 ); + } }void intel_color_load_luts (const struct intel_crtc_state *crtc_state)
Signature · 1 line
2.38.1
RFC 30/33 drm/i915/color: Enable Plane CSC
In reply to Uma ShankarImplement plane CSC for Xe_LPD.
--- drivers/gpu/drm/i915/display/intel_color.c | 86 +++++++++++++++++++ drivers/gpu/drm/i915/display/intel_color.h | 1 + .../drm/i915/display/skl_universal_plane.c | 4 +- 3 files changed, 90 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/display/intel_color.c +86 -0
@@ -2068,6 +2068,83 @@ static void xelpd_plane_load_luts(const struct drm_plane_state *plane_state) } } +static void xelpd_load_plane_csc_matrix (const struct drm_plane_state *state) +{ +struct drm_i915_private *dev_priv =to_i915 (state->plane->dev); +enum pipe pipe =to_intel_plane (state->plane)->pipe; +enum plane_id plane =to_intel_plane (state->plane)->id; +struct drm_color_ctm *ctm; +const u64 *input; +u16 coeffs[9 ] = {}; +u16 postoff =0 ; +int i; + +if (!icl_is_hdr_plane (dev_priv, plane) || !state->color.ctm) +return ; + + ctm = state->color.ctm->data; + input = ctm->matrix; + +/* +* Convert fixed point S31.32 input to format supported by the +* hardware. +*/ +for (i =0 ; i <ARRAY_SIZE (coeffs); i++) { +u64 abs_coeff = ((1ULL <<63 ) -1 ) & input[i]; + +/* +* Clamp input value to min/max supported by +* hardware. +*/ + abs_coeff =clamp_val (abs_coeff,0 ,CTM_COEFF_4_0 -1 ); + +/* sign bit */ +if (CTM_COEFF_NEGATIVE (input[i])) + coeffs[i] |=1 <<15 ; + +if (abs_coeff <CTM_COEFF_0_125 ) + coeffs[i] |= (3 <<12 ) | +ILK_CSC_COEFF_FP (abs_coeff,12 ); +else if (abs_coeff <CTM_COEFF_0_25 ) + coeffs[i] |= (2 <<12 ) | +ILK_CSC_COEFF_FP (abs_coeff,11 ); +else if (abs_coeff <CTM_COEFF_0_5 ) + coeffs[i] |= (1 <<12 ) | +ILK_CSC_COEFF_FP (abs_coeff,10 ); +else if (abs_coeff <CTM_COEFF_1_0 ) + coeffs[i] |=ILK_CSC_COEFF_FP (abs_coeff,9 ); +else if (abs_coeff <CTM_COEFF_2_0 ) + coeffs[i] |= (7 <<12 ) | +ILK_CSC_COEFF_FP (abs_coeff,8 ); +else + coeffs[i] |= (6 <<12 ) | +ILK_CSC_COEFF_FP (abs_coeff,7 ); + } + +intel_de_write_fw (dev_priv,PLANE_CSC_COEFF (pipe, plane,0 ), + coeffs[0 ] <<16 | coeffs[1 ]); +intel_de_write_fw (dev_priv,PLANE_CSC_COEFF (pipe, plane,1 ), + coeffs[2 ] <<16 ); + +intel_de_write_fw (dev_priv,PLANE_CSC_COEFF (pipe, plane,2 ), + coeffs[3 ] <<16 | coeffs[4 ]); +intel_de_write_fw (dev_priv,PLANE_CSC_COEFF (pipe, plane,3 ), + coeffs[5 ] <<16 ); + +intel_de_write_fw (dev_priv,PLANE_CSC_COEFF (pipe, plane,4 ), + coeffs[6 ] <<16 | coeffs[7 ]); +intel_de_write_fw (dev_priv,PLANE_CSC_COEFF (pipe, plane,5 ), + coeffs[8 ] <<16 ); + +intel_de_write_fw (dev_priv,PLANE_CSC_PREOFF (pipe, plane,0 ),0 ); +intel_de_write_fw (dev_priv,PLANE_CSC_PREOFF (pipe, plane,1 ),0 ); +intel_de_write_fw (dev_priv,PLANE_CSC_PREOFF (pipe, plane,2 ),0 ); + +intel_de_write_fw (dev_priv,PLANE_CSC_POSTOFF (pipe, plane,0 ), postoff); +intel_de_write_fw (dev_priv,PLANE_CSC_POSTOFF (pipe, plane,1 ), postoff); +intel_de_write_fw (dev_priv,PLANE_CSC_POSTOFF (pipe, plane,2 ), postoff); +} +void intel_color_load_luts (const struct intel_crtc_state *crtc_state) {struct drm_i915_private *i915 =to_i915 (crtc_state->uapi.crtc->dev);
@@ -2083,6 +2160,14 @@ void intel_color_load_plane_luts(const struct drm_plane_state *plane_state) i915->display.funcs.color->load_plane_luts (plane_state); } +void intel_color_load_plane_csc_matrix (const struct drm_plane_state *plane_state) +{ +struct drm_i915_private *i915 =to_i915 (plane_state->plane->dev); + +if (i915->display.funcs.color->load_plane_csc_matrix) + i915->display.funcs.color->load_plane_csc_matrix (plane_state); +} +void intel_color_commit_noarm (const struct intel_crtc_state *crtc_state) {struct drm_i915_private *i915 =to_i915 (crtc_state->uapi.crtc->dev);
@@ -3925,6 +4010,7 @@ static const struct intel_color_funcs xelpd_color_funcs = { .lut_equal = icl_lut_equal, .read_csc = icl_read_csc, .load_plane_luts = xelpd_plane_load_luts, + .load_plane_csc_matrix = xelpd_load_plane_csc_matrix, };static const struct intel_color_funcs tgl_color_funcs = {
drivers/gpu/drm/i915/display/intel_color.h +1 -0
@@ -33,4 +33,5 @@ bool intel_color_lut_equal(const struct intel_crtc_state *crtc_state,void intel_color_assert_luts (const struct intel_crtc_state *crtc_state);void intel_color_plane_init (struct drm_plane *plane);void intel_color_load_plane_luts (const struct drm_plane_state *plane_state); +void intel_color_load_plane_csc_matrix (const struct drm_plane_state *plane_state);#endif /* __INTEL_COLOR_H__ */
drivers/gpu/drm/i915/display/skl_universal_plane.c +3 -1
@@ -1280,8 +1280,10 @@ icl_plane_update_noarm(struct intel_plane *plane,if (plane_state->force_black)icl_plane_csc_load_black (plane); -if (plane_state->uapi.color_mgmt_changed) +if (plane_state->uapi.color_mgmt_changed) {intel_color_load_plane_luts (&plane_state->uapi); +intel_color_load_plane_csc_matrix (&plane_state->uapi); + }intel_psr2_program_plane_sel_fetch_noarm (plane, crtc_state, plane_state, color_plane); }
Signature · 1 line
2.38.1
RFC 31/33 drm/i915/color: Enable plane color features
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
Initialize and expose all plane color features.
--- drivers/gpu/drm/i915/display/intel_color.c | 1 - drivers/gpu/drm/i915/display/skl_universal_plane.c | 1 + 2 files changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/i915/display/intel_color.c +0 -1
@@ -4361,7 +4361,6 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)return ret; }; -__maybe_unused void intel_color_plane_init (struct drm_plane *plane) {struct drm_i915_private *i915 =to_i915 (plane->dev);
drivers/gpu/drm/i915/display/skl_universal_plane.c +1 -0
@@ -2389,6 +2389,7 @@ skl_universal_plane_create(struct drm_i915_private *dev_priv, BIT(DRM_SCALING_FILTER_NEAREST_NEIGHBOR));intel_plane_helper_add (plane); +intel_color_plane_init (&plane->base);return plane;
Signature · 1 line
2.38.1
RFC 32/33 drm/i915/color: Add a dummy pipeline with 3D LUT
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
This patch is to demonstrate how a pipeline can be added.
--- drivers/gpu/drm/drm_atomic_state_helper.c | 3 ++ drivers/gpu/drm/drm_atomic_uapi.c | 15 +++++++++ drivers/gpu/drm/i915/display/intel_color.c | 37 ++++++++++++++++++++-- include/drm/drm_plane.h | 6 ++++ 4 files changed, 59 insertions(+), 2 deletions(-)
drivers/gpu/drm/drm_atomic_state_helper.c +3 -0
@@ -349,6 +349,8 @@ void __drm_atomic_helper_plane_duplicate_state(struct drm_plane *plane,drm_property_blob_get (state->color.post_csc_lut);if (state->color.private_color_op_data)drm_property_blob_get (state->color.private_color_op_data); +if (state->color.lut_3d) +drm_property_blob_get (state->color.lut_3d); state->color_mgmt_changed = false; }
@@ -402,6 +404,7 @@ void __drm_atomic_helper_plane_destroy_state(struct drm_plane_state *state)drm_property_blob_put (state->color.ctm);drm_property_blob_put (state->color.post_csc_lut);drm_property_blob_put (state->color.private_color_op_data); +drm_property_blob_put (state->color.lut_3d); }EXPORT_SYMBOL (__drm_atomic_helper_plane_destroy_state);
drivers/gpu/drm/drm_atomic_uapi.c +15 -0
@@ -482,6 +482,15 @@ int drm_plane_reset_color_op_blobs(struct drm_plane *plane, &blob_replaced); temp_replaced |= blob_replaced; +if (ret) + goto out; + + ret =drm_atomic_replace_property_blob_from_id (dev, + &state->color.lut_3d, +0 ,-1 ,-1 , + &blob_replaced); + temp_replaced |= blob_replaced; +if (ret) goto out; out:
@@ -551,6 +560,12 @@ int drm_plane_replace_color_op_blobs(struct drm_plane *plane, color_op[i].blob_id,-1 ,-1 , &blob_replaced); + }else if (color_op[i].name ==DRM_CB_3D_LUT ) { + ret =drm_atomic_replace_property_blob_from_id (dev, + &state->color.lut_3d, + color_op[i].blob_id, +-1 ,sizeof (struct drm_color_lut_ext ), + &blob_replaced); }else { ret = -EINVAL ; goto copy_fail;
drivers/gpu/drm/i915/display/intel_color.c +35 -2
@@ -4265,6 +4265,19 @@ static const struct drm_color_lut_range xelpd_post_csc_hdr[] = { }, }; +static const struct drm_color_lut_range dummy_3d_lut_range[] = { + { + .flags = (DRM_MODE_LUT_POST_CSC | +DRM_MODE_LUT_REFLECT_NEGATIVE | +DRM_MODE_LUT_INTERPOLATE | +DRM_MODE_LUT_NON_DECREASING ), + .count =32 , + .input_bpc =24 , .output_bpc =16 , + .start =0 , .end = (1 <<24 ) -1 , + .min =0 , .max = (1 <<24 ) -1 , + }, +}; +struct drm_color_op color_pipeline_sdr[] = { { .name =DRM_CB_PRE_CSC ,
@@ -4300,10 +4313,17 @@ struct drm_color_op color_pipeline_hdr[] = { }, }; +struct drm_color_op color_pipeline_3dlut[] = { + { + .name =DRM_CB_3D_LUT , + .type =CURVE_3D , + }, +}; +static int intel_prepare_plane_color_pipeline (struct drm_plane *plane) {struct drm_i915_private *i915 =to_i915 (plane->dev); -struct drm_property_blob *blob[2 ] = {NULL }; +struct drm_property_blob *blob[3 ] = {NULL };int ret =0 , i =0 ;if (icl_is_hdr_plane (i915,to_intel_plane (plane)->id)) {
@@ -4350,6 +4370,17 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane) color_pipeline_sdr[1 ].blob_id = blob[i++]->base.id; } + blob[i] =drm_property_create_blob (plane->dev, +sizeof (dummy_3d_lut_range), + dummy_3d_lut_range); + +if (IS_ERR (blob[i])) { + ret =PTR_ERR (blob[i]); + goto out; + } + + color_pipeline_3dlut[0 ].blob_id = blob[i++]->base.id; + out:if (ret) {for (int j =0 ; j < i; j++) {
@@ -4368,7 +4399,7 @@ void intel_color_plane_init(struct drm_plane *plane)if (DISPLAY_VER (i915) <13 )return ; -drm_plane_create_get_color_pipeline_property (plane->dev, plane,2 ); +drm_plane_create_get_color_pipeline_property (plane->dev, plane,3 );intel_prepare_plane_color_pipeline (plane);
@@ -4387,6 +4418,8 @@ void intel_color_plane_init(struct drm_plane *plane) color_pipeline_sdr, sizeof(color_pipeline_sdr)); +drm_plane_add_color_pipeline (plane,"color pipeline 3dlut" , color_pipeline_3dlut, +sizeof (color_pipeline_3dlut));drm_plane_attach_get_color_pipeline_property (plane);drm_plane_create_set_color_pipeline_property (plane->dev, plane);
include/drm/drm_plane.h +6 -0
@@ -284,6 +284,12 @@ struct drm_plane_state {* For example: Parameterized/non-parameterized fixed function operations */ struct drm_property_blob *private_color_op_data; + +/* @lut_3d: +* +* Three dimensional luts +*/ +struct drm_property_blob *lut_3d; } color;/**
Signature · 1 line
2.38.1
RFC 33/33 drm/i915/color: Add example implementation for vendor specific color operation
In reply to Uma ShankarFrom: Chaitanya Kumar Borah <[email protected]>
This is an example of how vendor specific color operation could be supported by the uapi
--- drivers/gpu/drm/i915/display/intel_color.c | 42 ++++++++++++++++--- drivers/gpu/drm/i915/display/intel_color.h | 1 + .../drm/i915/display/skl_universal_plane.c | 1 + include/uapi/drm/i915_drm.h | 25 +++++++++++ 4 files changed, 64 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/display/intel_color.c +37 -5
@@ -21,6 +21,7 @@* DEALINGS IN THE SOFTWARE. * */ +#include <uapi/drm/i915_drm.h> #include "i915_reg.h" #include "intel_color.h"
@@ -87,6 +88,7 @@ struct intel_color_funcs {*/ void (*load_plane_csc_matrix)(const struct drm_plane_state *plane_state);void (*load_plane_luts)(const struct drm_plane_state *plane_state); +void (*load_private)(const struct drm_plane_state *plane_state); };#define CTM_COEFF_SIGN (1ULL << 63)
@@ -2145,6 +2147,25 @@ static void xelpd_load_plane_csc_matrix(const struct drm_plane_state *state)intel_de_write_fw (dev_priv,PLANE_CSC_POSTOFF (pipe, plane,2 ), postoff); } +static void xelpd_load_private (const struct drm_plane_state *state) +{ +struct drm_i915_private *i915 =to_i915 (state->plane->dev); +struct i915_color_op_data *op_data; +enum plane_id plane =to_intel_plane (state->plane)->id; +int i, num; + +if (icl_is_hdr_plane (i915, plane) || !state->color.private_color_op_data) +return ; + + op_data = state->color.private_color_op_data->data; + num = state->color.private_color_op_data->length /sizeof (struct i915_color_op_data ); + +for (i =0 ; i < num; i++) { +if (op_data[i].flag ==I915_COLOR_OP_FIXED_FUNC_CSC ) +DRM_DEBUG_KMS ("CSC OP [%d]" , op_data[i].csc_type); + } +} +void intel_color_load_luts (const struct intel_crtc_state *crtc_state) {struct drm_i915_private *i915 =to_i915 (crtc_state->uapi.crtc->dev);
@@ -2168,6 +2189,14 @@ void intel_color_load_plane_csc_matrix(const struct drm_plane_state *plane_state i915->display.funcs.color->load_plane_csc_matrix (plane_state); } +void intel_color_load_private (const struct drm_plane_state *plane_state) +{ +struct drm_i915_private *i915 =to_i915 (plane_state->plane->dev); + +if (i915->display.funcs.color->load_private) + i915->display.funcs.color->load_private (plane_state); +} +void intel_color_commit_noarm (const struct intel_crtc_state *crtc_state) {struct drm_i915_private *i915 =to_i915 (crtc_state->uapi.crtc->dev);
@@ -4011,6 +4040,7 @@ static const struct intel_color_funcs xelpd_color_funcs = { .read_csc = icl_read_csc, .load_plane_luts = xelpd_plane_load_luts, .load_plane_csc_matrix = xelpd_load_plane_csc_matrix, + .load_private = xelpd_load_private, };static const struct intel_color_funcs tgl_color_funcs = {
@@ -4284,10 +4314,12 @@ struct drm_color_op color_pipeline_sdr[] = { .type = CURVE_1D, .blob_id = 0, /* To be updated during plane initialization */ }, -/* -* SDR planes have fixed function CSC capabilities. -* TODO: Add support for it -*/ + { + .name = DRM_CB_PRIVATE, + .type = FIXED_FUNCTION, + .blob_id = 0, + .private_flags = I915_COLOR_OP_FIXED_FUNC_CSC, + }, { .name = DRM_CB_POST_CSC, .type = CURVE_1D,
@@ -4367,7 +4399,7 @@ static int intel_prepare_plane_color_pipeline(struct drm_plane *plane)* LUT ranges for SDR planes are similar for pre and post-csc blocks */ color_pipeline_sdr[0 ].blob_id = - color_pipeline_sdr[1 ].blob_id = blob[i++]->base.id; + color_pipeline_sdr[2 ].blob_id = blob[i++]->base.id; } blob[i] = drm_property_create_blob(plane->dev,
drivers/gpu/drm/i915/display/intel_color.h +1 -0
@@ -34,4 +34,5 @@ void intel_color_assert_luts(const struct intel_crtc_state *crtc_state);void intel_color_plane_init (struct drm_plane *plane);void intel_color_load_plane_luts (const struct drm_plane_state *plane_state);void intel_color_load_plane_csc_matrix (const struct drm_plane_state *plane_state); +void intel_color_load_private (const struct drm_plane_state *plane_state);#endif /* __INTEL_COLOR_H__ */
drivers/gpu/drm/i915/display/skl_universal_plane.c +1 -0
@@ -1283,6 +1283,7 @@ icl_plane_update_noarm(struct intel_plane *plane,if (plane_state->uapi.color_mgmt_changed) {intel_color_load_plane_luts (&plane_state->uapi);intel_color_load_plane_csc_matrix (&plane_state->uapi); +intel_color_load_private (&plane_state->uapi); }intel_psr2_program_plane_sel_fetch_noarm (plane, crtc_state, plane_state, color_plane);
include/uapi/drm/i915_drm.h +25 -0
@@ -3841,6 +3841,31 @@ struct drm_i915_gem_create_ext_set_pat {/* ID of the protected content session managed by i915 when PXP is active */ #define I915_PROTECTED_CONTENT_DEFAULT_SESSION 0xf +/* I915 specific color operation */ +#define I915_COLOR_OP_FIXED_FUNC_CSC (1 << 0) + +/** +* enum i915_csc_operation +* +* Color conversion operations which can be performed by a fixed function h/w +* of type I915_COLOR_OP_FIXED_FUNC_CSC +*/ +enum i915_csc_operation { +I915_CSC_YUV601_TO_RGB601 , +I915_CSC_YUV709_TO_RGB709 , +I915_CSC_YUV2020_TO_RGB2020 , +I915_CSC_RGB709_TO_RGB2020 , +I915_CSC_MAX , +}; + +struct i915_color_op_data { +__u32 flag;/* to identify i915 specific color operation */ +union { +enum i915_csc_operation csc_type; +/* Add more structures here */ + }; +}; +#if defined(__cplusplus) }#endif
Signature · 1 line
2.38.1
+CC Naseer and Chris, FYI
See https://patchwork.freedesktop.org/series/123024/ for whole series.
Quoted 17 lines · Uma Shankar
Introduction
============Modern hardwares have various color processing capabilities both at pre-blending and post-blending phases in the color pipeline. The current drm implementation exposes only the post-blending color hardware blocks. Support for pre-blending hardware is missing. There are multiple use cases where pre-blending color hardware will be useful:
a) Linearization of input buffers encoded in various transfer functions.
b) Color Space conversion
c) Tone mapping
d) Frame buffer format conversion
e) Non-linearization of buffer(apply transfer function)
f) 3D Luts
Quoted 7 lines
and other miscellaneous color operations.
Hence, there is a need to expose the color capabilities of the hardware to user-space. This will help userspace/middleware to use display hardware for color processing and blending instead of doing it through GPU shaders.
Thanks, Uma, for sending this. I've been working on something similar but you beat me to it. :)
Quoted 60 lines
Work done so far and relevant references
========================================Some implementation is done by Intel and AMD/Igalia to address the same. Broad consensus is there that we need a generic API at drm core to suffice the use case of various HW vendors. Below are the links capturing the discussion so far.
Intel's Plane Color Implementation: https://patchwork.freedesktop.org/series/90825/
AMD's Plane Color Implementation: https://patchwork.freedesktop.org/series/116862/Hackfest conclusions
====================HDR/Color Hackfest was organised by Redhat to bring all the industry stakeholders together and converge on a common uapi expectations. Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia and other prominent user-space developers and maintainers.
Discussions happened on the uapi expectations, opens, nature of hardware of multiple hardware vendors, challenges in generalizing the same and the path forward. Consensus was made that drm core should implement descriptive APIs and not go with prescriptive APIs. DRM core should just expose the hardware capabilities; enabling, customizing and programming the same should be done by the user-space. Driver should just honor the user space request without doing any operations internally.
Thanks to Simon Ser, for nicely documenting the design consensus and an UAPI RFC which can be referred to here:
Design considerations
=====================Following are the important aspects taken into account while designing the current RFC proposal:
1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one can be used)
2. Position of the HW block in the pipeline can be programmable
3. LUTs can be one dimentional or three dimentional
4. Number of LUT entries can vary across platforms
5. Precision of LUT entries can vary across platforms
6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic, Piece-Wise Linear(PWL) etc
7. There can be parameterized/non-parameterized fixed function HW blocks. e.g. Just a hardware bit, to convert from one color space to another.
8. Custom non-standard HW implementation.
9. Leaving scope for some vendor defined pescriptive implementation if required. 10.Scope to handle any modification in hardware as technology evolvesThe current proposal takes into account the above considerations while keeping the implementation as generic as possible leaving scope for future additions or modifications.
This proposal is also in line to the details mentioned by Simon's RFC covering all the aspects discussed in hackfest.
Quoted 8 lines
Outline of the implementation
============================Each Color Hardware block will be represented by a data structure
drm_color_op. These color operations will form the building blocks of a color pipeline which best represents the underlying Hardware. Color operations can be re-arranged, substracted or added to create distinct color pipelines to accurately describe the Hardware blocks present in the display engine.
Who is doing the arranging of color operations? IMO a driver should define one or more respective pipelines that can be selected by userspace. This seems to be what you're talking about after (I haven't reviewed the whole thing yet). Might be best to drop this sentence or to add clarifications in order to avoid confusion.
Quoted 8 lines
In this proposal, a color pipeline is represented as an array of struct
drm_color_op. For individual color operation, we add blobs to advertise the capability of the particular Hardware block.This color pipeline is then packaged within a blob for the user space to retrieve it.
Would it be better to expose the drm_color_ops directly, instead of packing a array of drm_color_ops into a blob and then giving that to userspace.
Quoted 9 lines
To advertise the available color pipelines, an immutable ENUM property "
GET_COLOR_PIPELINE" is introduced. This enum property has blob id's as values. With each blob id representing a distinct color pipeline based on underlying HW capabilities and their respective combinations.Once the user space decides on a color pipeline, it can set the pipeline and the corresponding data for the hardware blocks within the pipeline with the BLOB property "
SET_COLOR_PIPELINE".
When I discussed this at the hackfest with Simon he proposed a new DRM object, (I've called it a drm_colorop) to represent a color operation. Each drm_colorop has a "NEXT" pointer to another drm_colorop, or NULL if its the last in the pipeline. You can then have a mutable enum property on the plane to discover and select a color pipeline.
This seems a bit more transparent than a blob. You can see my changes (unfortunately very WIP, don't look too closely at individual patches) at
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/diffs
libdrm changes:
https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diffs
IGT changes:
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1/diffs
I'll take time to review your whole series and will see whether we can somehow keep the best parts of each.
Curious to hear other opinions on the blob vs new DRM object question as well.
Quoted 11 lines
Refer to
Documentation/gpu/rfc/plane_color_pipeline.rstadded in the patchIGT and test details
====================A set of IGT tests is written to demonstrate the usage of the proposed UAPIs along with some sanity validation.
Details of the IGT test can be found here:
https://patchwork.freedesktop.org/series/123018/
Quoted 7 lines
Opens
=====a. To come up with a list of common HW blocks which can be defined generically by the DRM
core in agreement with all the stakeholders
b. To enhance/finalize the data structure to define segmented LUTs generically.
It would be good to add some basic support in VKMS. My work has been based on VKMS. Once we kinda settle on an approach I'll look at exposing the AMD private properties from Melissa through the API.
Quoted 22 lines
Out of Scope
============a. The coefficients for CTM and LUT value calculations are out of scope of the proposal. b. The onus of programming the HW blocks and their values is on user-space. Driver will
just provide the interface for the same.
c. In order to compute LUT values and coefficients, a helper library can be created in
user-space. However, it is out of scope for the current proposal.Acknowledgements and credits
============================There are multiple contributors who have helped us to reach to this proposal. Special mention to Ville Syrjala<[email protected]>, Pekka Paalanen<[email protected]>, Simon Ser<[email protected]>, Harry Wentland<[email protected]>,
Melissa Wen<[email protected]>, Jonas<[email protected]>, Sebastian Wick<[email protected]>, Bhanu<[email protected]> and Shashank<[email protected]>.Also, thanks to Carlos <[email protected]> and the Redhat team for organizing the HDR hackfest.
Quoted 7 lines
UAPI dependency and Usermode development
========================================The current KMS implementation requires a user space consumer for it to be accepted upstream. Work is ongoing in weston and mutter community to get color management and HDR support implemented in the respective stacks.
If we can get AMD properties encoded using a Color Pipeline API we can probably use gamescope as the userspace vehicle.
I'm reviewing this in sequence, so there's a chance I'm missing context. Please bear with me if some of my comments are answered later in the series.
Again, thanks for sending this.
Harry
Quoted 73 lines
=================================================================================
We have tried to take care of all the scenarios and use-cases which possibly could exists in the current proposal. Thanks to everyone who has contributed in all color management discussions so far. Let's work together to improve the current proposal and get this thing implemented in upstream linux. All the feedback and suggestions to enhance the design are welcome.
Regards,
Uma Shankar
Chaitanya Kumar BorahCc Ville Syrjala <[email protected]>Cc Pekka Paalanen <[email protected]>Cc Simon Ser <[email protected]>Cc Harry Wentland <[email protected]>Cc Melissa Wen <[email protected]>Cc Jonas Ådahl <[email protected]>Cc Sebastian Wick <[email protected]>Cc Shashank Sharma <[email protected]>Cc Alexander Goins <[email protected]>Chaitanya Kumar Borah (14):
drm: Add color operation structure
drm: Add plane get color pipeline property
drm: Add helper to add color pipeline
drm: Manage color blob states
drm: Replace individual color blobs
drm: Reset pipeline when user sends NULL blob
drm: Reset plane color state on pipeline switch request drm/i915/color: Add HDR plane LUT range data to color pipeline drm/i915/color: Add SDR plane LUT range data to color pipeline drm/i915/color: Add color pipelines to plane drm/i915/color: Create and attach set color pipeline property drm/i915/color: Enable plane color features drm/i915/color: Add a dummy pipeline with 3D LUT drm/i915/color: Add example implementation for vendor specific color
operationUma Shankar (19):
drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
drm: Add structures for setting color pipeline
drm: Add set colorpipeline property
drm: Add Enhanced Gamma LUT precision structure
drm: Add color lut range structure
drm: Add color information to plane state drm/i915/color: Add lut range for SDR planes drm/i915/color: Add lut range for HDR planes drm/i915/color: Add color pipeline for HDR planes drm/i915/color: Add color pipeline for SDR planes drm/i915/color: Add plane color callbacks drm/i915/color: Load plane color luts from atomic flip drm/i915/xelpd: Add plane color check toglk_plane_color_ctldrm/i915/xelpd: Add register definitions for Plane Degamma drm/i915/color: Add color functions for ADL drm/i915/color: Program Plane Pre-CSC Registers drm/i915/xelpd: Add register definitions for Plane Post CSC drm/i915/xelpd: Program Plane Post CSC Registers drm/i915/color: Enable Plane CSC.../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++ drivers/gpu/drm/drm_atomic_state_helper.c | 21 + drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++ drivers/gpu/drm/drm_color_mgmt.c | 130 ++++ drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++- drivers/gpu/drm/i915/display/intel_color.h | 7 +- .../drm/i915/display/skl_universal_plane.c | 21 +- drivers/gpu/drm/i915/i915_reg.h | 124 ++++ include/drm/drm_plane.h | 82 +++ include/uapi/drm/drm_mode.h | 196 +++++ include/uapi/drm/i915_drm.h | 25 + 11 files changed, 1899 insertions(+), 3 deletions(-)create mode 100644
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 40 lines
-----Original Message-----
From: Harry Wentland <[email protected]>
Sent: Wednesday, August 30, 2023 12:56 AM
To: Shankar, Uma <[email protected]>; [email protected]; dri- [email protected]
Cc: [email protected]; Ville Syrjala <[email protected]>; Pekka Paalanen <[email protected]>; Simon Ser <[email protected]>; Melissa Wen <[email protected]>; Jonas Ådahl <[email protected]>; Sebastian Wick <[email protected]>; Shashank Sharma <[email protected]>; Alexander Goins <[email protected]>; Naseer Ahmed <[email protected]>; Christopher Braga <[email protected]>
Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline+CC Naseer and Chris, FYI
See https://patchwork.freedesktop.org/series/123024/ for whole series.
Quoted 19 lines · Uma Shankar
Introduction
============Modern hardwares have various color processing capabilities both at pre-blending and post-blending phases in the color pipeline. The current drm implementation exposes only the post-blending color hardware blocks. Support for pre-blending hardware is missing. There are multiple use cases where pre-blending color hardware will be useful:
a) Linearization of input buffers encoded in various transfer functions.
b) Color Space conversion
c) Tone mapping
d) Frame buffer format conversion
e) Non-linearization of buffer(apply transfer function)
f) 3D Lutsand other miscellaneous color operations.
Quoted 8 lines
Quoted 5 lines
Hence, there is a need to expose the color capabilities of the hardware to user-space. This will help userspace/middleware to use display hardware for color processing and blending instead of doing it through GPU shaders.
Thanks, Uma, for sending this. I've been working on something similar but you beat me to it. :)
Thanks Harry for the useful feedback and overall collaboration on this so far.
Quoted 80 lines
Quoted 23 lines
Work done so far and relevant references
========================================Some implementation is done by Intel and AMD/Igalia to address the same. Broad consensus is there that we need a generic API at drm core to suffice the use case of various HW vendors. Below are the links capturing the discussion so far.
Intel's Plane Color Implementation:
https://patchwork.freedesktop.org/series/90825/
AMD's Plane Color Implementation:
https://patchwork.freedesktop.org/series/116862/Hackfest conclusions
====================HDR/Color Hackfest was organised by Redhat to bring all the industry stakeholders together and converge on a common uapi expectations. Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia and other prominent user-space developers and maintainers.
Quoted 6 lines
Discussions happened on the uapi expectations, opens, nature of hardware of multiple hardware vendors, challenges in generalizing the same and the path forward. Consensus was made that drm core should implement descriptive APIs and not go with prescriptive APIs. DRM core should just expose the hardware capabilities; enabling, customizing and programming the same should be done by the user-space. Driver should just
honor the user space request without doing any operations internally.
Quoted 6 lines
Thanks to Simon Ser, for nicely documenting the design consensus and an UAPI RFC which can be referred to here:
https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5
nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn48Quoted 11 lines
Design considerations
=====================Following are the important aspects taken into account while designing the current RFC
proposal:1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one
can be used)
Quoted 7 lines
2. Position of the HW block in the pipeline can be programmable
3. LUTs can be one dimentional or three dimentional
4. Number of LUT entries can vary across platforms
5. Precision of LUT entries can vary across platforms
6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic, Piece-Wise Linear(PWL) etc
7. There can be parameterized/non-parameterized fixed function HWblocks.
Quoted 3 lines
e.g. Just a hardware bit, to convert from one color space to another.
8. Custom non-standard HW implementation.
9. Leaving scope for some vendor defined pescriptive implementation ifrequired.
Quoted 4 lines
10.Scope to handle any modification in hardware as technology evolves
The current proposal takes into account the above considerations while keeping the implementation as generic as possible leaving scope for future
additions or modifications.
Quoted 5 lines
This proposal is also in line to the details mentioned by Simon's RFC covering all the aspects discussed in hackfest.
Quoted 8 lines
Outline of the implementation
============================Each Color Hardware block will be represented by a data structure
drm_color_op. These color operations will form the building blocks of a color pipeline which best represents the underlying Hardware. Color operations can be re-arranged, substracted or added to create distinct color pipelines to accurately describe the Hardware blocks present in the display
Quoted 6 lines
engine.
Who is doing the arranging of color operations? IMO a driver should define one or more respective pipelines that can be selected by userspace. This seems to be what you're talking about after (I haven't reviewed the whole thing yet). Might be best to drop this sentence or to add clarifications in order to avoid confusion.
Yes it's the driver who will set the pipeline based on the underlying hardware arrangement and possible combinations. There can be multiple pipelines possible if hardware can be muxed or order can be re-arranged (all viable combinations should be defined as a pipeline in driver). Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
Quoted 11 lines
Quoted 8 lines
In this proposal, a color pipeline is represented as an array of struct
drm_color_op. For individual color operation, we add blobs to advertise the capability of the particular Hardware block.This color pipeline is then packaged within a blob for the user space to retrieve it.
Would it be better to expose the
drm_color_opsdirectly, instead of packing a array ofdrm_color_opsinto a blob and then giving that to userspace.
Advantage we see in packing in blobs is that interface will be cleaner. There will be just one GET_COLOR_PIPELINE property to invoke by user and then its just parsing the data. This way the entire underlying hardware blocks with pipeline will be available to user. For a particular hardware block in pipeline, user can get the relevant details from blob representing that particular block. We have created IGT tests (links mentioned in cover-letter) to demonstrate how it can be done. This is just to clarify the idea.
Also since info is passed via blobs we have the flexibility to even define segmented LUTs and PWL hardware blocks. Also we have left scope for custom private hardware blocks as well which driver can work with respective HAL and get that implemented.
We can even define prescriptive operations as a private entry and enable it if a certain driver and HAL agree.
Quoted 10 lines
Quoted 9 lines
To advertise the available color pipelines, an immutable ENUM property "
GET_COLOR_PIPELINE" is introduced. This enum property has blob id's as values. With each blob id representing a distinct color pipeline based on underlying HW capabilities and their respective combinations.Once the user space decides on a color pipeline, it can set the pipeline and the corresponding data for the hardware blocks within the pipeline with the BLOB property "
SET_COLOR_PIPELINE".
Quoted 5 lines
When I discussed this at the hackfest with Simon he proposed a new DRM object, (I've called it a
drm_colorop) to represent a color operation.
Eachdrm_colorophas a "NEXT" pointer to anotherdrm_colorop, or NULL if its the last in the pipeline. You can then have a mutable enum property on the plane to discover and select a color pipeline.
Yes, the proposal is inspired by this idea. Sure, we can work together to enhance the design. Personally I feel the one proposed in the current RFC will do the same thing envisioned by Simon and you Harry. Management of the pipeline, addition, deletion and flexibility to represent hardware is more with blobs with the relevant structures agreed in UAPI.
Quoted 6 lines
This seems a bit more transparent than a blob. You can see my changes (unfortunately very WIP, don't look too closely at individual patches) at
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/diffslibdrm changes:
https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diffs
Sure, will check the same.
Quoted 5 lines
IGT changes:
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1/diffsI'll take time to review your whole series and will see whether we can somehow keep the best parts of each.
Thanks and agree. Let's work together and get this implemented in DRM.
Quoted 1 line
Curious to hear other opinions on the blob vs new DRM object question as well.
Yeah, request community and stakeholders to share feedback and suggestions. We will work on the feedback to improve the design.
Quoted 16 lines
Quoted 9 lines
Refer to
Documentation/gpu/rfc/plane_color_pipeline.rstadded in the patchIGT and test details
====================A set of IGT tests is written to demonstrate the usage of the proposed UAPIs along with some sanity validation.
Quoted 7 lines
Details of the IGT test can be found here:
https://patchwork.freedesktop.org/series/123018/Opens
=====a. To come up with a list of common HW blocks which can be defined generically
Quoted 8 lines
by the DRM
Quoted 3 lines
core in agreement with all the stakeholders b. To enhance/finalize
the data structure to define segmented LUTs generically.It would be good to add some basic support in VKMS. My work has been based on VKMS. Once we kinda settle on an approach I'll look at exposing the AMD private properties from Melissa through the API.
Yeah sure Harry.
Quoted 33 lines
Quoted 5 lines
Out of Scope
============a. The coefficients for CTM and LUT value calculations are out of scope of the
proposal.
Quoted 1 line
b. The onus of programming the HW blocks and their values is on user-space.
Driver will
Quoted 2 lines
just provide the interface for the same.
c. In order to compute LUT values and coefficients, a helper library can be createdin
Quoted 5 lines
user-space. However, it is out of scope for the current proposal.
Acknowledgements and credits
============================Quoted 7 lines
There are multiple contributors who have helped us to reach to this proposal. Special mention to Ville Syrjala<[email protected]>, Pekka Paalanen<[email protected]>,
Simon Ser<[email protected]>, Harry Wentland<[email protected]>, Melissa Wen<[email protected]>, Jonas<[email protected]>, Sebastian Wick<[email protected]>,Bhanu<[email protected]> and
Shashank<[email protected]>.Quoted 2 lines
Also, thanks to Carlos <[email protected]> and the Redhat team for
organizing the HDR hackfest.
Quoted 5 lines
UAPI dependency and Usermode development
========================================
Quoted 8 lines
Quoted 1 line
The current KMS implementation requires a user space consumer for it to be
accepted upstream.
Quoted 3 lines
Work is ongoing in weston and mutter community to get color management and HDR support implemented in the respective stacks.
If we can get AMD properties encoded using a Color Pipeline API we can probably use gamescope as the userspace vehicle.
Yeah, nice.
Quoted 2 lines
I'm reviewing this in sequence, so there's a chance I'm missing context. Please bear with me if some of my comments are answered later in the series.
No worries, really appreciate the feedback and support.
Quoted 1 line
Again, thanks for sending this.
Always welcome.
Regards,
Uma Shankar
Quoted 80 lines
Harry
Quoted 1 line
====================================================================
==Quoted 6 lines
===========
We have tried to take care of all the scenarios and use-cases which possibly could exists in the current proposal. Thanks to everyone who has contributed in all color management discussions so far. Let's work together to improve the current proposal and get this thing implemented in
upstream linux. All the feedback and suggestions to enhance the design are welcome.
Quoted 67 lines
Regards,
Uma Shankar
Chaitanya Kumar BorahCc Ville Syrjala <[email protected]>Cc Pekka Paalanen <[email protected]>Cc Simon Ser <[email protected]>Cc Harry Wentland <[email protected]>Cc Melissa Wen <[email protected]>Cc Jonas Ådahl <[email protected]>Cc Sebastian Wick <[email protected]>Cc Shashank Sharma <[email protected]>Cc Alexander Goins <[email protected]>Chaitanya Kumar Borah (14):
drm: Add color operation structure
drm: Add plane get color pipeline property
drm: Add helper to add color pipeline
drm: Manage color blob states
drm: Replace individual color blobs
drm: Reset pipeline when user sends NULL blob
drm: Reset plane color state on pipeline switch request drm/i915/color: Add HDR plane LUT range data to color pipeline drm/i915/color: Add SDR plane LUT range data to color pipeline drm/i915/color: Add color pipelines to plane drm/i915/color: Create and attach set color pipeline property drm/i915/color: Enable plane color features drm/i915/color: Add a dummy pipeline with 3D LUT drm/i915/color: Add example implementation for vendor specific color
operationUma Shankar (19):
drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
drm: Add structures for setting color pipeline
drm: Add set colorpipeline property
drm: Add Enhanced Gamma LUT precision structure
drm: Add color lut range structure
drm: Add color information to plane state drm/i915/color: Add lut range for SDR planes drm/i915/color: Add lut range for HDR planes drm/i915/color: Add color pipeline for HDR planes drm/i915/color: Add color pipeline for SDR planes drm/i915/color: Add plane color callbacks drm/i915/color: Load plane color luts from atomic flip drm/i915/xelpd: Add plane color check toglk_plane_color_ctldrm/i915/xelpd: Add register definitions for Plane Degamma drm/i915/color: Add color functions for ADL drm/i915/color: Program Plane Pre-CSC Registers drm/i915/xelpd: Add register definitions for Plane Post CSC drm/i915/xelpd: Program Plane Post CSC Registers drm/i915/color: Enable Plane CSC.../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++ drivers/gpu/drm/drm_atomic_state_helper.c | 21 + drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++ drivers/gpu/drm/drm_color_mgmt.c | 130 ++++ drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++- drivers/gpu/drm/i915/display/intel_color.h | 7 +- .../drm/i915/display/skl_universal_plane.c | 21 +- drivers/gpu/drm/i915/i915_reg.h | 124 ++++ include/drm/drm_plane.h | 82 +++ include/uapi/drm/drm_mode.h | 196 +++++ include/uapi/drm/i915_drm.h | 25 + 11 files changed, 1899 insertions(+), 3 deletions(-)create mode 100644
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 152 lines · Uma
Quoted 40 lines
-----Original Message-----
From: Harry Wentland <[email protected]>
Sent: Wednesday, August 30, 2023 12:56 AM
To: Shankar, Uma <[email protected]>; [email protected]; dri- [email protected]
Cc: [email protected]; Ville Syrjala <[email protected]>; Pekka Paalanen <[email protected]>; Simon Ser <[email protected]>; Melissa Wen <[email protected]>; Jonas Ådahl <[email protected]>; Sebastian Wick <[email protected]>; Shashank Sharma <[email protected]>; Alexander Goins <[email protected]>; Naseer Ahmed <[email protected]>; Christopher Braga <[email protected]>
Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline+CC Naseer and Chris, FYI
See https://patchwork.freedesktop.org/series/123024/ for whole series.
Quoted 19 lines · Uma Shankar
Introduction
============Modern hardwares have various color processing capabilities both at pre-blending and post-blending phases in the color pipeline. The current drm implementation exposes only the post-blending color hardware blocks. Support for pre-blending hardware is missing. There are multiple use cases where pre-blending color hardware will be useful:
a) Linearization of input buffers encoded in various transfer functions.
b) Color Space conversion
c) Tone mapping
d) Frame buffer format conversion
e) Non-linearization of buffer(apply transfer function)
f) 3D Lutsand other miscellaneous color operations.
Quoted 8 lines
Quoted 5 lines
Hence, there is a need to expose the color capabilities of the hardware to user-space. This will help userspace/middleware to use display hardware for color processing and blending instead of doing it through GPU shaders.
Thanks, Uma, for sending this. I've been working on something similar but you beat me to it. :)
Thanks Harry for the useful feedback and overall collaboration on this so far.
Quoted 80 lines
Quoted 23 lines
Work done so far and relevant references
========================================Some implementation is done by Intel and AMD/Igalia to address the same. Broad consensus is there that we need a generic API at drm core to suffice the use case of various HW vendors. Below are the links capturing the discussion so far.
Intel's Plane Color Implementation:
https://patchwork.freedesktop.org/series/90825/
AMD's Plane Color Implementation:
https://patchwork.freedesktop.org/series/116862/Hackfest conclusions
====================HDR/Color Hackfest was organised by Redhat to bring all the industry stakeholders together and converge on a common uapi expectations. Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia and other prominent user-space developers and maintainers.
Quoted 6 lines
Discussions happened on the uapi expectations, opens, nature of hardware of multiple hardware vendors, challenges in generalizing the same and the path forward. Consensus was made that drm core should implement descriptive APIs and not go with prescriptive APIs. DRM core should just expose the hardware capabilities; enabling, customizing and programming the same should be done by the user-space. Driver should just
honor the user space request without doing any operations internally.
Quoted 6 lines
Thanks to Simon Ser, for nicely documenting the design consensus and an UAPI RFC which can be referred to here:
https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze_hD5
nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QWn48Quoted 11 lines
Design considerations
=====================Following are the important aspects taken into account while designing the current RFC
proposal:1. Individual HW blocks can be muxed. (e.g. out of two HW blocks only one
can be used)
Quoted 7 lines
2. Position of the HW block in the pipeline can be programmable
3. LUTs can be one dimentional or three dimentional
4. Number of LUT entries can vary across platforms
5. Precision of LUT entries can vary across platforms
6. Distribution of LUT entries may vary. e.g Mutli-segmented, Logarithmic, Piece-Wise Linear(PWL) etc
7. There can be parameterized/non-parameterized fixed function HWblocks.
Quoted 3 lines
e.g. Just a hardware bit, to convert from one color space to another.
8. Custom non-standard HW implementation.
9. Leaving scope for some vendor defined pescriptive implementation ifrequired.
Quoted 4 lines
10.Scope to handle any modification in hardware as technology evolves
The current proposal takes into account the above considerations while keeping the implementation as generic as possible leaving scope for future
additions or modifications.
Quoted 5 lines
This proposal is also in line to the details mentioned by Simon's RFC covering all the aspects discussed in hackfest.
Quoted 8 lines
Outline of the implementation
============================Each Color Hardware block will be represented by a data structure
drm_color_op. These color operations will form the building blocks of a color pipeline which best represents the underlying Hardware. Color operations can be re-arranged, substracted or added to create distinct color pipelines to accurately describe the Hardware blocks present in the displayQuoted 6 lines
engine.
Who is doing the arranging of color operations? IMO a driver should define one or more respective pipelines that can be selected by userspace. This seems to be what you're talking about after (I haven't reviewed the whole thing yet). Might be best to drop this sentence or to add clarifications in order to avoid confusion.
Yes it's the driver who will set the pipeline based on the underlying hardware arrangement and possible combinations. There can be multiple pipelines possible if hardware can be muxed or order can be re-arranged (all viable combinations should be defined as a pipeline in driver). Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
Quoted 9 lines
Quoted 8 lines
In this proposal, a color pipeline is represented as an array of struct
drm_color_op. For individual color operation, we add blobs to advertise the capability of the particular Hardware block.This color pipeline is then packaged within a blob for the user space to retrieve it.
Quoted 6 lines
Quoted 2 lines
Would it be better to expose the
drm_color_opsdirectly, instead of packing a array ofdrm_color_opsinto a blob and then giving that to userspace.Advantage we see in packing in blobs is that interface will be cleaner. There will be just one
GET_COLOR_PIPELINEproperty to invoke by user and then its just parsing the data. This way the entire underlying hardware blocks with pipeline will be available to user.
I don't see how parsing a blob is easier than requesting the color ops from the kernel. User space is already equiped with getting KMS objects and the igt test code from Harry shows that this is all pretty trivial plumbing.
Quoted 3 lines
For a particular hardware block in pipeline, user can get the relevant details from blob representing that particular block. We have created IGT tests (links mentioned in cover-letter) to demonstrate how it can be done. This is just to clarify the idea.
The blob is also not introspectable with the usual tools whereas exposing them as properties would be.
It also would, like Pekka correctly noted, bring a whole bunch of issues about compatibility and versioning that are well understood with objects
+ properties.
Quoted 3 lines
Also since info is passed via blobs we have the flexibility to even define segmented LUTs and PWL hardware blocks. Also we have left scope for custom private hardware blocks as well which driver can work with respective HAL and get that implemented.
When color ops are real KMS objects they still can have properties which are blobs that can store LUTs and other data.
And we should avoid private blocks at all cost. In fact, I don't think the KMS rules have changed in that regard and it simply is not allowed.
Quoted 33 lines
We can even define prescriptive operations as a private entry and enable it if a certain driver and HAL agree.
Quoted 10 lines
Quoted 9 lines
To advertise the available color pipelines, an immutable ENUM property "
GET_COLOR_PIPELINE" is introduced. This enum property has blob id's as values. With each blob id representing a distinct color pipeline based on underlying HW capabilities and their respective combinations.Once the user space decides on a color pipeline, it can set the pipeline and the corresponding data for the hardware blocks within the pipeline with the BLOB property "
SET_COLOR_PIPELINE".Quoted 5 lines
When I discussed this at the hackfest with Simon he proposed a new DRM object, (I've called it a
drm_colorop) to represent a color operation.
Eachdrm_colorophas a "NEXT" pointer to anotherdrm_colorop, or NULL if its the last in the pipeline. You can then have a mutable enum property on the plane to discover and select a color pipeline.Yes, the proposal is inspired by this idea. Sure, we can work together to enhance the design. Personally I feel the one proposed in the current RFC will do the same thing envisioned by Simon and you Harry. Management of the pipeline, addition, deletion and flexibility to represent hardware is more with blobs with the relevant structures agreed in UAPI.
Quoted 6 lines
This seems a bit more transparent than a blob. You can see my changes (unfortunately very WIP, don't look too closely at individual patches) at
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/diffslibdrm changes:
https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diffsSure, will check the same.
Quoted 9 lines
Quoted 5 lines
IGT changes:
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_requests/1/diffsI'll take time to review your whole series and will see whether we can somehow keep the best parts of each.
Thanks and agree. Let's work together and get this implemented in DRM.
Quoted 1 line
Curious to hear other opinions on the blob vs new DRM object question as well.
I can see a few advantages with the blob approach.
User space can store the entire state in one blob and just assign a new blob to change to another pipeline configuration.
However, I would argue that changing a lot of properties is already common practice and works well. User space can deal with it and has abstractions to make this managable.
There also is no need for a new KMS object kindbu from Harrys work so far the new KMS object kind is surprisingly simple and works just as expected, which is a good sign.
With the blobs one can store general information about the pipeline and not only about the color ops themselves. So far it's not clear if we actually need this, and if we do, we could probably use color ops at the start of the pipeline which apply to all ops.
There are also a few drawbacks to the blob design. It's not very KMS-y which makes working with it different than with the rest of KMS and this mismatch just adds more mental burden. It probably also introduces a new set of problems that we might not even be aware of. It also makes it less introspectable as Pekka noted.
When I came up with the KMS Color Pipeline API, I also started with the blob approach because of the flexibility in it but it does add a fair amount of hidden complexity. At the hackfest Simon managed to convince me of the idea of using a new KMS object kind for it and it seemed to simplify a bunch of things. With Harrys work I would argue that it is ineed that case. It's pretty simple and still seems to do everything it needs to.
So, yeah, I'm favoring the color op approach and not the blob approach.
Quoted 25 lines
Yeah, request community and stakeholders to share feedback and suggestions. We will work on the feedback to improve the design.
Quoted 21 lines
Quoted 9 lines
Refer to
Documentation/gpu/rfc/plane_color_pipeline.rstadded in the patchIGT and test details
====================A set of IGT tests is written to demonstrate the usage of the proposed UAPIs along with some sanity validation.
Quoted 7 lines
Details of the IGT test can be found here:
https://patchwork.freedesktop.org/series/123018/Opens
=====a. To come up with a list of common HW blocks which can be defined generically
by the DRM
Quoted 3 lines
core in agreement with all the stakeholders b. To enhance/finalize
the data structure to define segmented LUTs generically.
Quoted 5 lines
Quoted 3 lines
It would be good to add some basic support in VKMS. My work has been based on VKMS. Once we kinda settle on an approach I'll look at exposing the AMD private properties from Melissa through the API.
Yeah sure Harry.
I think this is a great idea! Let's see if we can learn some lessons from both patch series and then continue with the VKMS implementation and igt tests for it before every vendor implements their view of how this should all work.
Quoted 137 lines
Quoted 34 lines
Quoted 6 lines
Out of Scope
============a. The coefficients for CTM and LUT value calculations are out of scope of the
proposal.
Quoted 1 line
b. The onus of programming the HW blocks and their values is on user-space.
Driver will
Quoted 2 lines
just provide the interface for the same.
c. In order to compute LUT values and coefficients, a helper library can be createdin
Quoted 5 lines
user-space. However, it is out of scope for the current proposal.
Acknowledgements and credits
============================Quoted 7 lines
There are multiple contributors who have helped us to reach to this proposal. Special mention to Ville Syrjala<[email protected]>, Pekka Paalanen<[email protected]>,
Simon Ser<[email protected]>, Harry Wentland<[email protected]>, Melissa Wen<[email protected]>, Jonas<[email protected]>, Sebastian Wick<[email protected]>,Bhanu<[email protected]> and
Shashank<[email protected]>.Quoted 2 lines
Also, thanks to Carlos <[email protected]> and the Redhat team for
organizing the HDR hackfest.
Quoted 5 lines
UAPI dependency and Usermode development
========================================Quoted 8 lines
Quoted 1 line
The current KMS implementation requires a user space consumer for it to be
accepted upstream.
Quoted 3 lines
Work is ongoing in weston and mutter community to get color management and HDR support implemented in the respective stacks.
If we can get AMD properties encoded using a Color Pipeline API we can probably use gamescope as the userspace vehicle.
Yeah, nice.
Quoted 2 lines
I'm reviewing this in sequence, so there's a chance I'm missing context. Please bear with me if some of my comments are answered later in the series.
No worries, really appreciate the feedback and support.
Quoted 1 line
Again, thanks for sending this.
Always welcome.
Regards,
Uma ShankarQuoted 80 lines
Harry
Quoted 1 line
====================================================================
==Quoted 6 lines
===========
We have tried to take care of all the scenarios and use-cases which possibly could exists in the current proposal. Thanks to everyone who has contributed in all color management discussions so far. Let's work together to improve the current proposal and get this thing implemented in
upstream linux. All the feedback and suggestions to enhance the design are welcome.
Quoted 67 lines
Regards,
Uma Shankar
Chaitanya Kumar BorahCc Ville Syrjala <[email protected]>Cc Pekka Paalanen <[email protected]>Cc Simon Ser <[email protected]>Cc Harry Wentland <[email protected]>Cc Melissa Wen <[email protected]>Cc Jonas Ådahl <[email protected]>Cc Sebastian Wick <[email protected]>Cc Shashank Sharma <[email protected]>Cc Alexander Goins <[email protected]>Chaitanya Kumar Borah (14):
drm: Add color operation structure
drm: Add plane get color pipeline property
drm: Add helper to add color pipeline
drm: Manage color blob states
drm: Replace individual color blobs
drm: Reset pipeline when user sends NULL blob
drm: Reset plane color state on pipeline switch request drm/i915/color: Add HDR plane LUT range data to color pipeline drm/i915/color: Add SDR plane LUT range data to color pipeline drm/i915/color: Add color pipelines to plane drm/i915/color: Create and attach set color pipeline property drm/i915/color: Enable plane color features drm/i915/color: Add a dummy pipeline with 3D LUT drm/i915/color: Add example implementation for vendor specific color
operationUma Shankar (19):
drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
drm: Add structures for setting color pipeline
drm: Add set colorpipeline property
drm: Add Enhanced Gamma LUT precision structure
drm: Add color lut range structure
drm: Add color information to plane state drm/i915/color: Add lut range for SDR planes drm/i915/color: Add lut range for HDR planes drm/i915/color: Add color pipeline for HDR planes drm/i915/color: Add color pipeline for SDR planes drm/i915/color: Add plane color callbacks drm/i915/color: Load plane color luts from atomic flip drm/i915/xelpd: Add plane color check toglk_plane_color_ctldrm/i915/xelpd: Add register definitions for Plane Degamma drm/i915/color: Add color functions for ADL drm/i915/color: Program Plane Pre-CSC Registers drm/i915/xelpd: Add register definitions for Plane Post CSC drm/i915/xelpd: Program Plane Post CSC Registers drm/i915/color: Enable Plane CSC.../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++ drivers/gpu/drm/drm_atomic_state_helper.c | 21 + drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++ drivers/gpu/drm/drm_color_mgmt.c | 130 ++++ drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++- drivers/gpu/drm/i915/display/intel_color.h | 7 +- .../drm/i915/display/skl_universal_plane.c | 21 +- drivers/gpu/drm/i915/i915_reg.h | 124 ++++ include/drm/drm_plane.h | 82 +++ include/uapi/drm/drm_mode.h | 196 +++++ include/uapi/drm/i915_drm.h | 25 + 11 files changed, 1899 insertions(+), 3 deletions(-)create mode 100644
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 191 lines
-----Original Message-----
From: Sebastian Wick <[email protected]>
Sent: Thursday, August 31, 2023 2:46 AM
To: Shankar, Uma <[email protected]>
Cc: Harry Wentland <[email protected]>; intel- [email protected]; [email protected]; wayland- [email protected]; Ville Syrjala <[email protected]>; Pekka Paalanen <[email protected]>; Simon Ser <[email protected]>; Melissa Wen <[email protected]>; Jonas Ådahl <[email protected]>; Shashank Sharma <[email protected]>; Alexander Goins <[email protected]>; Naseer Ahmed <[email protected]>; Christopher Braga <[email protected]>
Subject: Re: [RFC 00/33] Add Support for Plane Color PipelineQuoted 48 lines · Uma
Quoted 48 lines
-----Original Message-----
From: Harry Wentland <[email protected]>
Sent: Wednesday, August 30, 2023 12:56 AM
To: Shankar, Uma <[email protected]>; [email protected]; dri- [email protected]
Cc: [email protected]; Ville Syrjala <[email protected]>; Pekka Paalanen <[email protected]>; Simon Ser <[email protected]>; Melissa Wen <[email protected]>; Jonas Ådahl <[email protected]>; Sebastian Wick <[email protected]>; Shashank Sharma <[email protected]>; Alexander Goins <[email protected]>; Naseer Ahmed <[email protected]>; Christopher Braga <[email protected]>
Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline+CC Naseer and Chris, FYI
See https://patchwork.freedesktop.org/series/123024/ for whole series.
Quoted 25 lines · Uma Shankar
Introduction
============Modern hardwares have various color processing capabilities both at pre-blending and post-blending phases in the color pipeline. The current drm implementation exposes only the post-blending color hardware blocks. Support for pre-blending hardware is missing. There are multiple use cases where pre-blending color hardware will be
useful:
a) Linearization of input buffers encoded in various transfer functions.
b) Color Space conversion
c) Tone mapping
d) Frame buffer format conversion
e) Non-linearization of buffer(apply transfer function)
f) 3D Lutsand other miscellaneous color operations.
Hence, there is a need to expose the color capabilities of the hardware to user-space. This will help userspace/middleware to use display hardware for color processing and blending instead of doing it through GPU shaders.
Quoted 25 lines
Quoted 2 lines
Thanks, Uma, for sending this. I've been working on something similar but you beat me to it. :)
Thanks Harry for the useful feedback and overall collaboration on this so far.
Quoted 20 lines
Quoted 20 lines
Work done so far and relevant references
========================================Some implementation is done by Intel and AMD/Igalia to address the same. Broad consensus is there that we need a generic API at drm core to suffice the use case of various HW vendors. Below are the links capturing the discussion so far.
Intel's Plane Color Implementation:
https://patchwork.freedesktop.org/series/90825/
AMD's Plane Color Implementation:
https://patchwork.freedesktop.org/series/116862/Hackfest conclusions
====================HDR/Color Hackfest was organised by Redhat to bring all the industry stakeholders together and converge on a common uapi
expectations.
Quoted 12 lines
Quoted 12 lines
Quoted 10 lines
Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia and other prominent user-space developers and maintainers.
Discussions happened on the uapi expectations, opens, nature of hardware of multiple hardware vendors, challenges in generalizing the same and the path forward. Consensus was made that drm core should implement descriptive APIs and not go with prescriptive APIs. DRM core should just expose the hardware capabilities; enabling, customizing and programming the same should be done by the user-space. Driver should just
honor the user space request without doing any operations internally.
Quoted 1 line
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Thanks to Simon Ser, for nicely documenting the design consensus and an UAPI RFC which can be referred to here:
https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze
_hD5
nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QQuoted 11 lines
Quoted 11 lines
Wn48
Quoted 10 lines
Design considerations
=====================Following are the important aspects taken into account while designing the current RFC
proposal:Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
1. Individual HW blocks can be muxed. (e.g. out of two HW blocks
only onecan be used)
Quoted 5 lines
2. Position of the HW block in the pipeline can be programmable
3. LUTs can be one dimentional or three dimentional
4. Number of LUT entries can vary across platforms
5. Precision of LUT entries can vary across platforms
6. Distribution of LUT entries may vary. e.g Mutli-segmented,Logarithmic,
Quoted 16 lines
Quoted 16 lines
Quoted 2 lines
Piece-Wise Linear(PWL) etc
7. There can be parameterized/non-parameterized fixed function HWblocks.
Quoted 4 lines
e.g. Just a hardware bit, to convert from one color space to another.
8. Custom non-standard HW implementation.
9. Leaving scope for some vendor defined pescriptive
implementation ifrequired.
Quoted 6 lines
10.Scope to handle any modification in hardware as technology
evolvesThe current proposal takes into account the above considerations while keeping the implementation as generic as possible leaving scope for future
additions or modifications.
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
This proposal is also in line to the details mentioned by Simon's RFC covering all the aspects discussed in hackfest.
Outline of the implementation
============================Each Color Hardware block will be represented by a data structure
drm_color_op.Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
These color operations will form the building blocks of a color pipeline which best represents the underlying Hardware. Color operations can be re-arranged, substracted or added to create distinct color pipelines to accurately describe the Hardware blocks present in the display
engine.
Who is doing the arranging of color operations? IMO a driver should define one or more respective pipelines that can be selected by userspace. This seems to be what you're talking about after (I haven't reviewed the whole thing yet). Might be best to drop this sentence or
to add clarifications in order to avoid confusion.
Quoted 4 lines
Yes it's the driver who will set the pipeline based on the underlying hardware arrangement and possible combinations. There can be multiple pipelines possible if hardware can be muxed or order can be re-arranged (all
viable combinations should be defined as a pipeline in driver).
Quoted 13 lines
Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
Quoted 11 lines
Quoted 8 lines
In this proposal, a color pipeline is represented as an array of struct
drm_color_op. For individual color operation, we add blobs to advertise the capability of the particular Hardware block.This color pipeline is then packaged within a blob for the user space to retrieve it.
Would it be better to expose the
drm_color_opsdirectly, instead of packing a array ofdrm_color_opsinto a blob and then giving that touserspace.
Quoted 3 lines
Advantage we see in packing in blobs is that interface will be cleaner. There will be just one
GET_COLOR_PIPELINEproperty to invoke by user
Quoted 7 lines
and then its just parsing the data.
Quoted 1 line
This way the entire underlying hardware blocks with pipeline will be available to
user.
I don't see how parsing a blob is easier than requesting the color ops from the kernel. User space is already equiped with getting KMS objects and the igt test code from Harry shows that this is all pretty trivial plumbing.
There are multiple color operations possible with unique lut distribution and capabilities. Also the order of hardware blocks and possibility of multiple pipelines. Having all the information with one query and property and also be able to set the same with just one property call using blobs is better than multiple calls to driver. This can be useful in high refresh rate cases where not much time is there to program the hardware state. Latency of multiple calls to driver can be avoided.
Quoted 11 lines
Quoted 3 lines
For a particular hardware block in pipeline, user can get the relevant details from blob representing that particular block. We have created IGT tests (links mentioned in cover-letter) to demonstrate how it can be done.
This is just to clarify the idea.
The blob is also not introspectable with the usual tools whereas exposing them as properties would be.
It also would, like Pekka correctly noted, bring a whole bunch of issues about compatibility and versioning that are well understood with objects
+ properties.
The blob should be standardized in the UAPI and structures to parse them should be fixed. With this compatibility issues can be prevented.
Quoted 10 lines
Quoted 3 lines
Also since info is passed via blobs we have the flexibility to even define segmented LUTs and PWL hardware blocks. Also we have left scope for custom private hardware blocks as well which driver can work with
respective HAL and get that implemented.
When color ops are real KMS objects they still can have properties which are blobs that can store LUTs and other data.
And we should avoid private blocks at all cost. In fact, I don't think the KMS rules have changed in that regard and it simply is not allowed.
Private blocks are not standardized but are vendor specific. So generic userspace will ignore these. However vendor and its respective HAL can make use of this field and leaves a scope to cater to such hardware vendors need. This doesn't affect the expectation of the standardized color operations which will be defined as enum in UAPI.
Quoted 48 lines
Quoted 5 lines
We can even define prescriptive operations as a private entry and enable it if a certain driver and HAL agree.
Quoted 2 lines
Quoted 2 lines
To advertise the available color pipelines, an immutable ENUM property "
GET_COLOR_PIPELINE" is introduced. This enum property hasblob id's as values.
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
With each blob id representing a distinct color pipeline based on underlying HW capabilities and their respective combinations.
Once the user space decides on a color pipeline, it can set the pipeline and the corresponding data for the hardware blocks within the pipeline with the BLOB property "
SET_COLOR_PIPELINE".Quoted 7 lines
Quoted 5 lines
When I discussed this at the hackfest with Simon he proposed a new DRM object, (I've called it a
drm_colorop) to represent a color operation. Eachdrm_colorophas a "NEXT" pointer to anotherdrm_colorop, or NULL if its the last in the pipeline. You can then have a mutable enum property on the plane to discover and select a color pipeline.Yes, the proposal is inspired by this idea. Sure, we can work together to enhance
the design.
Quoted 3 lines
Personally I feel the one proposed in the current RFC will do the same thing envisioned by Simon and you Harry. Management of the pipeline, addition, deletion and flexibility to represent hardware is more with blobs with
the relevant structures agreed in UAPI.
Quoted 22 lines
Quoted 10 lines
This seems a bit more transparent than a blob. You can see my changes (unfortunately very WIP, don't look too closely at individual patches) at
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/di
ffslibdrm changes:
https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diff
sSure, will check the same.
Quoted 6 lines
IGT changes:
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_reque
sts/1/diffsI'll take time to review your whole series and will see whether we can somehow keep the best parts of each.
Thanks and agree. Let's work together and get this implemented in DRM.
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Curious to hear other opinions on the blob vs new DRM object question as
well.
I can see a few advantages with the blob approach.
User space can store the entire state in one blob and just assign a new blob to change to another pipeline configuration.
Agree
Quoted 3 lines
However, I would argue that changing a lot of properties is already common practice and works well. User space can deal with it and has abstractions to make this managable.
Blob gives a lot of flexibility and ability to define the hardware capabilities generically. Lut distribution, number of segments, samples in each segment, precision of luts etc. can be precisely defined and userspace will get a complete picture of the underlying hardware and its capabilities. Also this is being done with just 2 properties. Leaving scope for future addition of any standard color operation as well.
I feel blob approach once properly documented is a bit more flexible and scalable. Maybe I am bit biased here 😊 but all ideas are welcome.
We have implemented some IGT's as well to explain the design better. Link below:
https://patchwork.freedesktop.org/series/123018/
Thanks Sebastian for the feedback.
Regards,
Uma Shankar
Quoted 207 lines
There also is no need for a new KMS object kindbu from Harrys work so far the new KMS object kind is surprisingly simple and works just as expected, which is a good sign.
With the blobs one can store general information about the pipeline and not only about the color ops themselves. So far it's not clear if we actually need this, and if we do, we could probably use color ops at the start of the pipeline which apply to all ops.
There are also a few drawbacks to the blob design. It's not very KMS-y which makes working with it different than with the rest of KMS and this mismatch just adds more mental burden. It probably also introduces a new set of problems that we might not even be aware of. It also makes it less introspectable as Pekka noted.
When I came up with the KMS Color Pipeline API, I also started with the blob approach because of the flexibility in it but it does add a fair amount of hidden complexity. At the hackfest Simon managed to convince me of the idea of using a new KMS object kind for it and it seemed to simplify a bunch of things. With Harrys work I would argue that it is ineed that case. It's pretty simple and still seems to do everything it needs to.
So, yeah, I'm favoring the color op approach and not the blob approach.
Quoted 26 lines
Yeah, request community and stakeholders to share feedback and suggestions. We will work on the feedback to improve the design.
Quoted 22 lines
Quoted 9 lines
Refer to
Documentation/gpu/rfc/plane_color_pipeline.rstadded in the patchIGT and test details
====================A set of IGT tests is written to demonstrate the usage of the proposed UAPIs along with some sanity validation.
Quoted 8 lines
Details of the IGT test can be found here:
https://patchwork.freedesktop.org/series/123018/Opens
=====a. To come up with a list of common HW blocks which can be defined generically
by the DRM
Quoted 3 lines
core in agreement with all the stakeholders b. To
enhance/finalize the data structure to define segmented LUTs generically.Quoted 5 lines
Quoted 3 lines
It would be good to add some basic support in VKMS. My work has been based on VKMS. Once we kinda settle on an approach I'll look at exposing the AMD private properties from Melissa through the API.
Yeah sure Harry.
I think this is a great idea! Let's see if we can learn some lessons from both patch series and then continue with the VKMS implementation and igt tests for it before every vendor implements their view of how this should all work.
Quoted 9 lines
Quoted 9 lines
Quoted 7 lines
Out of Scope
============a. The coefficients for CTM and LUT value calculations are out of scope of the
proposal.
Quoted 1 line
b. The onus of programming the HW blocks and their values is on user-
space.
Quoted 44 lines
Quoted 31 lines
Driver will
Quoted 3 lines
just provide the interface for the same.
c. In order to compute LUT values and coefficients, a helper library can be createdin
Quoted 5 lines
user-space. However, it is out of scope for the current proposal.
Acknowledgements and credits
============================Quoted 8 lines
There are multiple contributors who have helped us to reach to this proposal. Special mention to Ville Syrjala<[email protected]>, Pekka Paalanen<[email protected]>,
Simon Ser<[email protected]>, Harry Wentland<[email protected]>, Melissa Wen<[email protected]>, Jonas<[email protected]>, Sebastian Wick<[email protected]>,Bhanu<[email protected]> and
Shashank<[email protected]>.Quoted 3 lines
Also, thanks to Carlos <[email protected]> and the Redhat team for
organizing the HDR hackfest.
Quoted 7 lines
UAPI dependency and Usermode development
========================================The current KMS implementation requires a user space consumer for it to be
Quoted 7 lines
accepted upstream.
Quoted 3 lines
Work is ongoing in weston and mutter community to get color management and HDR support implemented in the respective stacks.
If we can get AMD properties encoded using a Color Pipeline API we can probably use gamescope as the userspace vehicle.
Yeah, nice.
Quoted 2 lines
I'm reviewing this in sequence, so there's a chance I'm missing context. Please bear with me if some of my comments are answered later in the series.
Quoted 13 lines
No worries, really appreciate the feedback and support.
Quoted 1 line
Again, thanks for sending this.
Always welcome.
Regards,
Uma ShankarQuoted 4 lines
Harry
Quoted 1 line
=================================================================
===Quoted 78 lines
Quoted 78 lines
==
Quoted 7 lines
===========
We have tried to take care of all the scenarios and use-cases which possibly could exists in the current proposal. Thanks to everyone who has contributed in all color management discussions so far. Let's work together to improve the current proposal and get this thing implemented in
upstream linux. All the feedback and suggestions to enhance the design are welcome.
Quoted 68 lines
Regards,
Uma Shankar
Chaitanya Kumar BorahCc Ville Syrjala <[email protected]>Cc Pekka Paalanen <[email protected]>Cc Simon Ser <[email protected]>Cc Harry Wentland <[email protected]>Cc Melissa Wen <[email protected]>Cc Jonas Ådahl <[email protected]>Cc Sebastian Wick <[email protected]>Cc Shashank Sharma <[email protected]>Cc Alexander Goins <[email protected]>Chaitanya Kumar Borah (14):
drm: Add color operation structure
drm: Add plane get color pipeline property
drm: Add helper to add color pipeline
drm: Manage color blob states
drm: Replace individual color blobs
drm: Reset pipeline when user sends NULL blob
drm: Reset plane color state on pipeline switch request drm/i915/color: Add HDR plane LUT range data to color pipeline drm/i915/color: Add SDR plane LUT range data to color pipeline drm/i915/color: Add color pipelines to plane drm/i915/color: Create and attach set color pipeline property drm/i915/color: Enable plane color features drm/i915/color: Add a dummy pipeline with 3D LUT drm/i915/color: Add example implementation for vendor specific color
operationUma Shankar (19):
drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
drm: Add structures for setting color pipeline
drm: Add set colorpipeline property
drm: Add Enhanced Gamma LUT precision structure
drm: Add color lut range structure
drm: Add color information to plane state drm/i915/color: Add lut range for SDR planes drm/i915/color: Add lut range for HDR planes drm/i915/color: Add color pipeline for HDR planes drm/i915/color: Add color pipeline for SDR planes drm/i915/color: Add plane color callbacks drm/i915/color: Load plane color luts from atomic flip drm/i915/xelpd: Add plane color check toglk_plane_color_ctldrm/i915/xelpd: Add register definitions for Plane Degamma drm/i915/color: Add color functions for ADL drm/i915/color: Program Plane Pre-CSC Registers drm/i915/xelpd: Add register definitions for Plane Post CSC drm/i915/xelpd: Program Plane Post CSC Registers drm/i915/color: Enable Plane CSC.../gpu/rfc/plane_color_pipeline.rst | 394 ++++++++++ drivers/gpu/drm/drm_atomic_state_helper.c | 21 + drivers/gpu/drm/drm_atomic_uapi.c | 218 ++++++ drivers/gpu/drm/drm_color_mgmt.c | 130 ++++ drivers/gpu/drm/i915/display/intel_color.c | 684 +++++++++++++++++- drivers/gpu/drm/i915/display/intel_color.h | 7 +- .../drm/i915/display/skl_universal_plane.c | 21 +- drivers/gpu/drm/i915/i915_reg.h | 124 ++++ include/drm/drm_plane.h | 82 +++ include/uapi/drm/drm_mode.h | 196 +++++ include/uapi/drm/i915_drm.h | 25 + 11 files changed, 1899 insertions(+), 3 deletions(-)create mode 100644
Documentation/gpu/rfc/plane_color_pipeline.rst
Quoted 197 lines · Uma
Quoted 189 lines
-----Original Message-----
From: Sebastian Wick <[email protected]>
Sent: Thursday, August 31, 2023 2:46 AM
To: Shankar, Uma <[email protected]>
Cc: Harry Wentland <[email protected]>; intel- [email protected]; [email protected]; wayland- [email protected]; Ville Syrjala <[email protected]>; Pekka Paalanen <[email protected]>; Simon Ser <[email protected]>; Melissa Wen <[email protected]>; Jonas Ådahl <[email protected]>; Shashank Sharma <[email protected]>; Alexander Goins <[email protected]>; Naseer Ahmed <[email protected]>; Christopher Braga <[email protected]>
Subject: Re: [RFC 00/33] Add Support for Plane Color PipelineQuoted 48 lines · Uma
Quoted 48 lines
-----Original Message-----
From: Harry Wentland <[email protected]>
Sent: Wednesday, August 30, 2023 12:56 AM
To: Shankar, Uma <[email protected]>; [email protected]; dri- [email protected]
Cc: [email protected]; Ville Syrjala <[email protected]>; Pekka Paalanen <[email protected]>; Simon Ser <[email protected]>; Melissa Wen <[email protected]>; Jonas Ådahl <[email protected]>; Sebastian Wick <[email protected]>; Shashank Sharma <[email protected]>; Alexander Goins <[email protected]>; Naseer Ahmed <[email protected]>; Christopher Braga <[email protected]>
Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline+CC Naseer and Chris, FYI
See https://patchwork.freedesktop.org/series/123024/ for whole series.
Quoted 25 lines · Uma Shankar
Introduction
============Modern hardwares have various color processing capabilities both at pre-blending and post-blending phases in the color pipeline. The current drm implementation exposes only the post-blending color hardware blocks. Support for pre-blending hardware is missing. There are multiple use cases where pre-blending color hardware will be
useful:
a) Linearization of input buffers encoded in various transfer functions.
b) Color Space conversion
c) Tone mapping
d) Frame buffer format conversion
e) Non-linearization of buffer(apply transfer function)
f) 3D Lutsand other miscellaneous color operations.
Hence, there is a need to expose the color capabilities of the hardware to user-space. This will help userspace/middleware to use display hardware for color processing and blending instead of doing it through GPU shaders.
Quoted 25 lines
Quoted 2 lines
Thanks, Uma, for sending this. I've been working on something similar but you beat me to it. :)
Thanks Harry for the useful feedback and overall collaboration on this so far.
Quoted 20 lines
Quoted 20 lines
Work done so far and relevant references
========================================Some implementation is done by Intel and AMD/Igalia to address the same. Broad consensus is there that we need a generic API at drm core to suffice the use case of various HW vendors. Below are the links capturing the discussion so far.
Intel's Plane Color Implementation:
https://patchwork.freedesktop.org/series/90825/
AMD's Plane Color Implementation:
https://patchwork.freedesktop.org/series/116862/Hackfest conclusions
====================HDR/Color Hackfest was organised by Redhat to bring all the industry stakeholders together and converge on a common uapi
expectations.
Quoted 12 lines
Quoted 12 lines
Quoted 10 lines
Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia and other prominent user-space developers and maintainers.
Discussions happened on the uapi expectations, opens, nature of hardware of multiple hardware vendors, challenges in generalizing the same and the path forward. Consensus was made that drm core should implement descriptive APIs and not go with prescriptive APIs. DRM core should just expose the hardware capabilities; enabling, customizing and programming the same should be done by the user-space. Driver should just
honor the user space request without doing any operations internally.
Quoted 1 line
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Thanks to Simon Ser, for nicely documenting the design consensus and an UAPI RFC which can be referred to here:
https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze
_hD5
nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QQuoted 11 lines
Quoted 11 lines
Wn48
Quoted 10 lines
Design considerations
=====================Following are the important aspects taken into account while designing the current RFC
proposal:Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
1. Individual HW blocks can be muxed. (e.g. out of two HW blocks
only onecan be used)
Quoted 5 lines
2. Position of the HW block in the pipeline can be programmable
3. LUTs can be one dimentional or three dimentional
4. Number of LUT entries can vary across platforms
5. Precision of LUT entries can vary across platforms
6. Distribution of LUT entries may vary. e.g Mutli-segmented,Logarithmic,
Quoted 16 lines
Quoted 16 lines
Quoted 2 lines
Piece-Wise Linear(PWL) etc
7. There can be parameterized/non-parameterized fixed function HWblocks.
Quoted 4 lines
e.g. Just a hardware bit, to convert from one color space to another.
8. Custom non-standard HW implementation.
9. Leaving scope for some vendor defined pescriptive
implementation ifrequired.
Quoted 6 lines
10.Scope to handle any modification in hardware as technology
evolvesThe current proposal takes into account the above considerations while keeping the implementation as generic as possible leaving scope for future
additions or modifications.
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
This proposal is also in line to the details mentioned by Simon's RFC covering all the aspects discussed in hackfest.
Outline of the implementation
============================Each Color Hardware block will be represented by a data structure
drm_color_op.Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
These color operations will form the building blocks of a color pipeline which best represents the underlying Hardware. Color operations can be re-arranged, substracted or added to create distinct color pipelines to accurately describe the Hardware blocks present in the display
engine.
Who is doing the arranging of color operations? IMO a driver should define one or more respective pipelines that can be selected by userspace. This seems to be what you're talking about after (I haven't reviewed the whole thing yet). Might be best to drop this sentence or
to add clarifications in order to avoid confusion.
Quoted 4 lines
Yes it's the driver who will set the pipeline based on the underlying hardware arrangement and possible combinations. There can be multiple pipelines possible if hardware can be muxed or order can be re-arranged (all
viable combinations should be defined as a pipeline in driver).
Quoted 13 lines
Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
Quoted 11 lines
Quoted 8 lines
In this proposal, a color pipeline is represented as an array of struct
drm_color_op. For individual color operation, we add blobs to advertise the capability of the particular Hardware block.This color pipeline is then packaged within a blob for the user space to retrieve it.
Would it be better to expose the
drm_color_opsdirectly, instead of packing a array ofdrm_color_opsinto a blob and then giving that touserspace.
Quoted 3 lines
Advantage we see in packing in blobs is that interface will be cleaner. There will be just one
GET_COLOR_PIPELINEproperty to invoke by userQuoted 7 lines
and then its just parsing the data.
Quoted 1 line
This way the entire underlying hardware blocks with pipeline will be available to
user.
I don't see how parsing a blob is easier than requesting the color ops from the kernel. User space is already equiped with getting KMS objects and the igt test code from Harry shows that this is all pretty trivial plumbing.
Quoted 6 lines
There are multiple color operations possible with unique lut distribution and capabilities. Also the order of hardware blocks and possibility of multiple pipelines. Having all the information with one query and property and also be able to set the same with just one property call using blobs is better than multiple calls to driver. This can be useful in high refresh rate cases where not much time is there to program the hardware state. Latency of multiple calls to driver can be avoided.
Hi,
querying all that information only happens once, at KMS client start-up.
Setting up a color pipeline is already a single call: the atomic commit ioctl.
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines
For a particular hardware block in pipeline, user can get the relevant details from blob representing that particular block. We have created IGT tests (links mentioned in cover-letter) to demonstrate how it can be done.
This is just to clarify the idea.
The blob is also not introspectable with the usual tools whereas exposing them as properties would be.
Quoted 6 lines
Quoted 3 lines
It also would, like Pekka correctly noted, bring a whole bunch of issues about compatibility and versioning that are well understood with objects
+ properties.The blob should be standardized in the UAPI and structures to parse them should be fixed. With this compatibility issues can be prevented.
I think that is short-sighted.
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Also since info is passed via blobs we have the flexibility to even define segmented LUTs and PWL hardware blocks. Also we have left scope for custom private hardware blocks as well which driver can work with
respective HAL and get that implemented.
When color ops are real KMS objects they still can have properties which are blobs that can store LUTs and other data.
Quoted 7 lines
Quoted 2 lines
And we should avoid private blocks at all cost. In fact, I don't think the KMS rules have changed in that regard and it simply is not allowed.
Private blocks are not standardized but are vendor specific. So generic userspace will ignore these. However vendor and its respective HAL can make use of this field and leaves a scope to cater to such hardware vendors need. This doesn't affect the expectation of the standardized color operations which will be defined as enum in UAPI.
Vendors can have and expose their own unique snowflake operations without any "private" as well: pick an unused operation type code, and document what it does. Advertise it in some pipelines.
Vendor HALs can make use of it, but it also allows generic userspace to make use of it at will, and it allows other vendors to implement the same and benefit from it without needing to patch every userspace.
Or does Intel not want other vendors to potentially make use of their UAPIs?
Quoted 56 lines
Quoted 48 lines
Quoted 5 lines
We can even define prescriptive operations as a private entry and enable it if a certain driver and HAL agree.
Quoted 2 lines
Quoted 2 lines
To advertise the available color pipelines, an immutable ENUM property "
GET_COLOR_PIPELINE" is introduced. This enum property hasblob id's as values.
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
With each blob id representing a distinct color pipeline based on underlying HW capabilities and their respective combinations.
Once the user space decides on a color pipeline, it can set the pipeline and the corresponding data for the hardware blocks within the pipeline with the BLOB property "
SET_COLOR_PIPELINE".Quoted 7 lines
Quoted 5 lines
When I discussed this at the hackfest with Simon he proposed a new DRM object, (I've called it a
drm_colorop) to represent a color operation. Eachdrm_colorophas a "NEXT" pointer to anotherdrm_colorop, or NULL if its the last in the pipeline. You can then have a mutable enum property on the plane to discover and select a color pipeline.Yes, the proposal is inspired by this idea. Sure, we can work together to enhance
the design.
Quoted 3 lines
Personally I feel the one proposed in the current RFC will do the same thing envisioned by Simon and you Harry. Management of the pipeline, addition, deletion and flexibility to represent hardware is more with blobs with
the relevant structures agreed in UAPI.
Quoted 22 lines
Quoted 10 lines
This seems a bit more transparent than a blob. You can see my changes (unfortunately very WIP, don't look too closely at individual patches) at
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/di
ffslibdrm changes:
https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diff
sSure, will check the same.
Quoted 6 lines
IGT changes:
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_reque
sts/1/diffsI'll take time to review your whole series and will see whether we can somehow keep the best parts of each.
Thanks and agree. Let's work together and get this implemented in DRM.
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Curious to hear other opinions on the blob vs new DRM object question as
well.
I can see a few advantages with the blob approach.
User space can store the entire state in one blob and just assign a new blob to change to another pipeline configuration.
Quoted 7 lines
Agree
Quoted 3 lines
However, I would argue that changing a lot of properties is already common practice and works well. User space can deal with it and has abstractions to make this managable.
Blob gives a lot of flexibility and ability to define the hardware capabilities generically.
The structure of the atomic commit ioctl and the KMS property system is even more flexible.
Quoted 4 lines
Lut distribution, number of segments, samples in each segment, precision of luts etc. can be precisely defined and userspace will get a complete picture of the underlying hardware and its capabilities. Also this is being done with just 2 properties. Leaving scope for future addition of any standard color operation as well.
The number of properties does not seem too useful to strictly minimise over other aspects.
Thanks,
pq
Quoted 11 lines
I feel blob approach once properly documented is a bit more flexible and scalable. Maybe I am bit biased here 😊 but all ideas are welcome.
We have implemented some IGT's as well to explain the design better. Link below:
https://patchwork.freedesktop.org/series/123018/Thanks Sebastian for the feedback.
Regards,
Uma Shankar
Quoted 207 lines · Pekka Paalanen
Quoted 197 lines · Uma
Quoted 189 lines
-----Original Message-----
From: Sebastian Wick <[email protected]>
Sent: Thursday, August 31, 2023 2:46 AM
To: Shankar, Uma <[email protected]>
Cc: Harry Wentland <[email protected]>; intel- [email protected]; [email protected]; wayland- [email protected]; Ville Syrjala <[email protected]>; Pekka Paalanen <[email protected]>; Simon Ser <[email protected]>; Melissa Wen <[email protected]>; Jonas Ådahl <[email protected]>; Shashank Sharma <[email protected]>; Alexander Goins <[email protected]>; Naseer Ahmed <[email protected]>; Christopher Braga <[email protected]>
Subject: Re: [RFC 00/33] Add Support for Plane Color PipelineQuoted 48 lines · Uma
Quoted 48 lines
-----Original Message-----
From: Harry Wentland <[email protected]>
Sent: Wednesday, August 30, 2023 12:56 AM
To: Shankar, Uma <[email protected]>; [email protected]; dri- [email protected]
Cc: [email protected]; Ville Syrjala <[email protected]>; Pekka Paalanen <[email protected]>; Simon Ser <[email protected]>; Melissa Wen <[email protected]>; Jonas Ådahl <[email protected]>; Sebastian Wick <[email protected]>; Shashank Sharma <[email protected]>; Alexander Goins <[email protected]>; Naseer Ahmed <[email protected]>; Christopher Braga <[email protected]>
Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline+CC Naseer and Chris, FYI
See https://patchwork.freedesktop.org/series/123024/ for whole series.
Quoted 25 lines · Uma Shankar
Introduction
============Modern hardwares have various color processing capabilities both at pre-blending and post-blending phases in the color pipeline. The current drm implementation exposes only the post-blending color hardware blocks. Support for pre-blending hardware is missing. There are multiple use cases where pre-blending color hardware will be
useful:
a) Linearization of input buffers encoded in various transfer functions.
b) Color Space conversion
c) Tone mapping
d) Frame buffer format conversion
e) Non-linearization of buffer(apply transfer function)
f) 3D Lutsand other miscellaneous color operations.
Hence, there is a need to expose the color capabilities of the hardware to user-space. This will help userspace/middleware to use display hardware for color processing and blending instead of doing it through GPU shaders.
Quoted 25 lines
Quoted 2 lines
Thanks, Uma, for sending this. I've been working on something similar but you beat me to it. :)
Thanks Harry for the useful feedback and overall collaboration on this so far.
Quoted 20 lines
Quoted 20 lines
Work done so far and relevant references
========================================Some implementation is done by Intel and AMD/Igalia to address the same. Broad consensus is there that we need a generic API at drm core to suffice the use case of various HW vendors. Below are the links capturing the discussion so far.
Intel's Plane Color Implementation:
https://patchwork.freedesktop.org/series/90825/
AMD's Plane Color Implementation:
https://patchwork.freedesktop.org/series/116862/Hackfest conclusions
====================HDR/Color Hackfest was organised by Redhat to bring all the industry stakeholders together and converge on a common uapi
expectations.
Quoted 12 lines
Quoted 12 lines
Quoted 10 lines
Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia and other prominent user-space developers and maintainers.
Discussions happened on the uapi expectations, opens, nature of hardware of multiple hardware vendors, challenges in generalizing the same and the path forward. Consensus was made that drm core should implement descriptive APIs and not go with prescriptive APIs. DRM core should just expose the hardware capabilities; enabling, customizing and programming the same should be done by the user-space. Driver should just
honor the user space request without doing any operations internally.
Quoted 1 line
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Thanks to Simon Ser, for nicely documenting the design consensus and an UAPI RFC which can be referred to here:
https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze
_hD5
nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QQuoted 11 lines
Quoted 11 lines
Wn48
Quoted 10 lines
Design considerations
=====================Following are the important aspects taken into account while designing the current RFC
proposal:Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
1. Individual HW blocks can be muxed. (e.g. out of two HW blocks
only onecan be used)
Quoted 5 lines
2. Position of the HW block in the pipeline can be programmable
3. LUTs can be one dimentional or three dimentional
4. Number of LUT entries can vary across platforms
5. Precision of LUT entries can vary across platforms
6. Distribution of LUT entries may vary. e.g Mutli-segmented,Logarithmic,
Quoted 16 lines
Quoted 16 lines
Quoted 2 lines
Piece-Wise Linear(PWL) etc
7. There can be parameterized/non-parameterized fixed function HWblocks.
Quoted 4 lines
e.g. Just a hardware bit, to convert from one color space to another.
8. Custom non-standard HW implementation.
9. Leaving scope for some vendor defined pescriptive
implementation ifrequired.
Quoted 6 lines
10.Scope to handle any modification in hardware as technology
evolvesThe current proposal takes into account the above considerations while keeping the implementation as generic as possible leaving scope for future
additions or modifications.
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
This proposal is also in line to the details mentioned by Simon's RFC covering all the aspects discussed in hackfest.
Outline of the implementation
============================Each Color Hardware block will be represented by a data structure
drm_color_op.Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
These color operations will form the building blocks of a color pipeline which best represents the underlying Hardware. Color operations can be re-arranged, substracted or added to create distinct color pipelines to accurately describe the Hardware blocks present in the display
engine.
Who is doing the arranging of color operations? IMO a driver should define one or more respective pipelines that can be selected by userspace. This seems to be what you're talking about after (I haven't reviewed the whole thing yet). Might be best to drop this sentence or
to add clarifications in order to avoid confusion.
Quoted 4 lines
Yes it's the driver who will set the pipeline based on the underlying hardware arrangement and possible combinations. There can be multiple pipelines possible if hardware can be muxed or order can be re-arranged (all
viable combinations should be defined as a pipeline in driver).
Quoted 13 lines
Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
Quoted 11 lines
Quoted 8 lines
In this proposal, a color pipeline is represented as an array of struct
drm_color_op. For individual color operation, we add blobs to advertise the capability of the particular Hardware block.This color pipeline is then packaged within a blob for the user space to retrieve it.
Would it be better to expose the
drm_color_opsdirectly, instead of packing a array ofdrm_color_opsinto a blob and then giving that touserspace.
Quoted 3 lines
Advantage we see in packing in blobs is that interface will be cleaner. There will be just one
GET_COLOR_PIPELINEproperty to invoke by userQuoted 7 lines
and then its just parsing the data.
Quoted 1 line
This way the entire underlying hardware blocks with pipeline will be available to
user.
I don't see how parsing a blob is easier than requesting the color ops from the kernel. User space is already equiped with getting KMS objects and the igt test code from Harry shows that this is all pretty trivial plumbing.
Quoted 6 lines
There are multiple color operations possible with unique lut distribution and capabilities. Also the order of hardware blocks and possibility of multiple pipelines. Having all the information with one query and property and also be able to set the same with just one property call using blobs is better than multiple calls to driver. This can be useful in high refresh rate cases where not much time is there to program the hardware state. Latency of multiple calls to driver can be avoided.
Quoted 5 lines
Hi,
querying all that information only happens once, at KMS client start-up.
Setting up a color pipeline is already a single call: the atomic commit ioctl.
Well, clients also issue a bunch of ioctl to set some properties to the desired state and then you might run through a whole bunch of configurations to find one that actually works, so there is a case to be made that there are a lot of ioctls involved.
I just don't think this is an issue right now. Nobody has been complaining about the ioctls being a limiting factor so why should we optimize for this? Especially because it brings with it a bunch of disadvantages.
Anyway, I agree with the sentiment here: this is not something we should optimize for.
Quoted 136 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
For a particular hardware block in pipeline, user can get the relevant details from blob representing that particular block. We have created IGT tests (links mentioned in cover-letter) to demonstrate how it can be done.
This is just to clarify the idea.
The blob is also not introspectable with the usual tools whereas exposing them as properties would be.
Quoted 6 lines
Quoted 3 lines
It also would, like Pekka correctly noted, bring a whole bunch of issues about compatibility and versioning that are well understood with objects
+ properties.The blob should be standardized in the UAPI and structures to parse them should be fixed. With this compatibility issues can be prevented.
I think that is short-sighted.
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Also since info is passed via blobs we have the flexibility to even define segmented LUTs and PWL hardware blocks. Also we have left scope for custom private hardware blocks as well which driver can work with
respective HAL and get that implemented.
When color ops are real KMS objects they still can have properties which are blobs that can store LUTs and other data.
Quoted 7 lines
Quoted 2 lines
And we should avoid private blocks at all cost. In fact, I don't think the KMS rules have changed in that regard and it simply is not allowed.
Private blocks are not standardized but are vendor specific. So generic userspace will ignore these. However vendor and its respective HAL can make use of this field and leaves a scope to cater to such hardware vendors need. This doesn't affect the expectation of the standardized color operations which will be defined as enum in UAPI.
Vendors can have and expose their own unique snowflake operations without any "private" as well: pick an unused operation type code, and document what it does. Advertise it in some pipelines.
Vendor HALs can make use of it, but it also allows generic userspace to make use of it at will, and it allows other vendors to implement the same and benefit from it without needing to patch every userspace.
Or does Intel not want other vendors to potentially make use of their UAPIs?
Quoted 56 lines
Quoted 48 lines
Quoted 5 lines
We can even define prescriptive operations as a private entry and enable it if a certain driver and HAL agree.
Quoted 2 lines
Quoted 2 lines
To advertise the available color pipelines, an immutable ENUM property "
GET_COLOR_PIPELINE" is introduced. This enum property hasblob id's as values.
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
With each blob id representing a distinct color pipeline based on underlying HW capabilities and their respective combinations.
Once the user space decides on a color pipeline, it can set the pipeline and the corresponding data for the hardware blocks within the pipeline with the BLOB property "
SET_COLOR_PIPELINE".Quoted 7 lines
Quoted 5 lines
When I discussed this at the hackfest with Simon he proposed a new DRM object, (I've called it a
drm_colorop) to represent a color operation. Eachdrm_colorophas a "NEXT" pointer to anotherdrm_colorop, or NULL if its the last in the pipeline. You can then have a mutable enum property on the plane to discover and select a color pipeline.Yes, the proposal is inspired by this idea. Sure, we can work together to enhance
the design.
Quoted 3 lines
Personally I feel the one proposed in the current RFC will do the same thing envisioned by Simon and you Harry. Management of the pipeline, addition, deletion and flexibility to represent hardware is more with blobs with
the relevant structures agreed in UAPI.
Quoted 22 lines
Quoted 10 lines
This seems a bit more transparent than a blob. You can see my changes (unfortunately very WIP, don't look too closely at individual patches) at
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/di
ffslibdrm changes:
https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diff
sSure, will check the same.
Quoted 6 lines
IGT changes:
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_reque
sts/1/diffsI'll take time to review your whole series and will see whether we can somehow keep the best parts of each.
Thanks and agree. Let's work together and get this implemented in DRM.
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Curious to hear other opinions on the blob vs new DRM object question as
well.
I can see a few advantages with the blob approach.
User space can store the entire state in one blob and just assign a new blob to change to another pipeline configuration.
Quoted 7 lines
Agree
Quoted 3 lines
However, I would argue that changing a lot of properties is already common practice and works well. User space can deal with it and has abstractions to make this managable.
Blob gives a lot of flexibility and ability to define the hardware capabilities generically.
The structure of the atomic commit ioctl and the KMS property system is even more flexible.
Quoted 4 lines
Lut distribution, number of segments, samples in each segment, precision of luts etc. can be precisely defined and userspace will get a complete picture of the underlying hardware and its capabilities. Also this is being done with just 2 properties. Leaving scope for future addition of any standard color operation as well.
The number of properties does not seem too useful to strictly minimise over other aspects.
Thanks,
pqQuoted 11 lines
I feel blob approach once properly documented is a bit more flexible and scalable. Maybe I am bit biased here 😊 but all ideas are welcome.
We have implemented some IGT's as well to explain the design better. Link below:
https://patchwork.freedesktop.org/series/123018/Thanks Sebastian for the feedback.
Regards,
Uma Shankar
Quoted 210 lines · Sebastian Wick
Quoted 209 lines · Pekka Paalanen
Quoted 197 lines · Uma
Quoted 189 lines
-----Original Message-----
From: Sebastian Wick <[email protected]>
Sent: Thursday, August 31, 2023 2:46 AM
To: Shankar, Uma <[email protected]>
Cc: Harry Wentland <[email protected]>; intel- [email protected]; [email protected]; wayland- [email protected]; Ville Syrjala <[email protected]>; Pekka Paalanen <[email protected]>; Simon Ser <[email protected]>; Melissa Wen <[email protected]>; Jonas Ådahl <[email protected]>; Shashank Sharma <[email protected]>; Alexander Goins <[email protected]>; Naseer Ahmed <[email protected]>; Christopher Braga <[email protected]>
Subject: Re: [RFC 00/33] Add Support for Plane Color PipelineQuoted 48 lines · Uma
Quoted 48 lines
-----Original Message-----
From: Harry Wentland <[email protected]>
Sent: Wednesday, August 30, 2023 12:56 AM
To: Shankar, Uma <[email protected]>; [email protected]; dri- [email protected]
Cc: [email protected]; Ville Syrjala <[email protected]>; Pekka Paalanen <[email protected]>; Simon Ser <[email protected]>; Melissa Wen <[email protected]>; Jonas Ådahl <[email protected]>; Sebastian Wick <[email protected]>; Shashank Sharma <[email protected]>; Alexander Goins <[email protected]>; Naseer Ahmed <[email protected]>; Christopher Braga <[email protected]>
Subject: Re: [RFC 00/33] Add Support for Plane Color Pipeline+CC Naseer and Chris, FYI
See https://patchwork.freedesktop.org/series/123024/ for whole series.
Quoted 25 lines · Uma Shankar
Introduction
============Modern hardwares have various color processing capabilities both at pre-blending and post-blending phases in the color pipeline. The current drm implementation exposes only the post-blending color hardware blocks. Support for pre-blending hardware is missing. There are multiple use cases where pre-blending color hardware will be
useful:
a) Linearization of input buffers encoded in various transfer functions.
b) Color Space conversion
c) Tone mapping
d) Frame buffer format conversion
e) Non-linearization of buffer(apply transfer function)
f) 3D Lutsand other miscellaneous color operations.
Hence, there is a need to expose the color capabilities of the hardware to user-space. This will help userspace/middleware to use display hardware for color processing and blending instead of doing it through GPU shaders.
Quoted 25 lines
Quoted 2 lines
Thanks, Uma, for sending this. I've been working on something similar but you beat me to it. :)
Thanks Harry for the useful feedback and overall collaboration on this so far.
Quoted 20 lines
Quoted 20 lines
Work done so far and relevant references
========================================Some implementation is done by Intel and AMD/Igalia to address the same. Broad consensus is there that we need a generic API at drm core to suffice the use case of various HW vendors. Below are the links capturing the discussion so far.
Intel's Plane Color Implementation:
https://patchwork.freedesktop.org/series/90825/
AMD's Plane Color Implementation:
https://patchwork.freedesktop.org/series/116862/Hackfest conclusions
====================HDR/Color Hackfest was organised by Redhat to bring all the industry stakeholders together and converge on a common uapi
expectations.
Quoted 12 lines
Quoted 12 lines
Quoted 10 lines
Participants from Intel, AMD, Nvidia, Collabora, Redhat, Igalia and other prominent user-space developers and maintainers.
Discussions happened on the uapi expectations, opens, nature of hardware of multiple hardware vendors, challenges in generalizing the same and the path forward. Consensus was made that drm core should implement descriptive APIs and not go with prescriptive APIs. DRM core should just expose the hardware capabilities; enabling, customizing and programming the same should be done by the user-space. Driver should just
honor the user space request without doing any operations internally.
Quoted 1 line
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Thanks to Simon Ser, for nicely documenting the design consensus and an UAPI RFC which can be referred to here:
https://lore.kernel.org/dri-devel/QMers3awXvNCQlyhWdTtsPwkp5ie9bze
_hD5
nAccFW7a_RXlWjYB7MoUW_8CKLT2bSQwIXVi5H6VULYIxCdgvryZoAoJnC5lZgyK1QQuoted 11 lines
Quoted 11 lines
Wn48
Quoted 10 lines
Design considerations
=====================Following are the important aspects taken into account while designing the current RFC
proposal:Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
1. Individual HW blocks can be muxed. (e.g. out of two HW blocks
only onecan be used)
Quoted 5 lines
2. Position of the HW block in the pipeline can be programmable
3. LUTs can be one dimentional or three dimentional
4. Number of LUT entries can vary across platforms
5. Precision of LUT entries can vary across platforms
6. Distribution of LUT entries may vary. e.g Mutli-segmented,Logarithmic,
Quoted 16 lines
Quoted 16 lines
Quoted 2 lines
Piece-Wise Linear(PWL) etc
7. There can be parameterized/non-parameterized fixed function HWblocks.
Quoted 4 lines
e.g. Just a hardware bit, to convert from one color space to another.
8. Custom non-standard HW implementation.
9. Leaving scope for some vendor defined pescriptive
implementation ifrequired.
Quoted 6 lines
10.Scope to handle any modification in hardware as technology
evolvesThe current proposal takes into account the above considerations while keeping the implementation as generic as possible leaving scope for future
additions or modifications.
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
This proposal is also in line to the details mentioned by Simon's RFC covering all the aspects discussed in hackfest.
Outline of the implementation
============================Each Color Hardware block will be represented by a data structure
drm_color_op.Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
These color operations will form the building blocks of a color pipeline which best represents the underlying Hardware. Color operations can be re-arranged, substracted or added to create distinct color pipelines to accurately describe the Hardware blocks present in the display
engine.
Who is doing the arranging of color operations? IMO a driver should define one or more respective pipelines that can be selected by userspace. This seems to be what you're talking about after (I haven't reviewed the whole thing yet). Might be best to drop this sentence or
to add clarifications in order to avoid confusion.
Quoted 4 lines
Yes it's the driver who will set the pipeline based on the underlying hardware arrangement and possible combinations. There can be multiple pipelines possible if hardware can be muxed or order can be re-arranged (all
viable combinations should be defined as a pipeline in driver).
Quoted 13 lines
Yeah, I will re-phrase this to help clarify it and avoid any ambiguity.
Quoted 11 lines
Quoted 8 lines
In this proposal, a color pipeline is represented as an array of struct
drm_color_op. For individual color operation, we add blobs to advertise the capability of the particular Hardware block.This color pipeline is then packaged within a blob for the user space to retrieve it.
Would it be better to expose the
drm_color_opsdirectly, instead of packing a array ofdrm_color_opsinto a blob and then giving that touserspace.
Quoted 3 lines
Advantage we see in packing in blobs is that interface will be cleaner. There will be just one
GET_COLOR_PIPELINEproperty to invoke by userQuoted 7 lines
and then its just parsing the data.
Quoted 1 line
This way the entire underlying hardware blocks with pipeline will be available to
user.
I don't see how parsing a blob is easier than requesting the color ops from the kernel. User space is already equiped with getting KMS objects and the igt test code from Harry shows that this is all pretty trivial plumbing.
Quoted 6 lines
There are multiple color operations possible with unique lut distribution and capabilities. Also the order of hardware blocks and possibility of multiple pipelines. Having all the information with one query and property and also be able to set the same with just one property call using blobs is better than multiple calls to driver. This can be useful in high refresh rate cases where not much time is there to program the hardware state. Latency of multiple calls to driver can be avoided.
Hi,
Quoted 8 lines
Quoted 3 lines
querying all that information only happens once, at KMS client start-up.
Setting up a color pipeline is already a single call: the atomic commit ioctl.
Well, clients also issue a bunch of ioctl to set some properties to the desired state and then you might run through a whole bunch of configurations to find one that actually works, so there is a case to be made that there are a lot of ioctls involved.
pq pointed at that for atomic this is actually really just a single ioctl so really there is no advantage here at all.
Quoted 146 lines
I just don't think this is an issue right now. Nobody has been complaining about the ioctls being a limiting factor so why should we optimize for this? Especially because it brings with it a bunch of disadvantages.
Anyway, I agree with the sentiment here: this is not something we should optimize for.
Quoted 136 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
For a particular hardware block in pipeline, user can get the relevant details from blob representing that particular block. We have created IGT tests (links mentioned in cover-letter) to demonstrate how it can be done.
This is just to clarify the idea.
The blob is also not introspectable with the usual tools whereas exposing them as properties would be.
Quoted 6 lines
Quoted 3 lines
It also would, like Pekka correctly noted, bring a whole bunch of issues about compatibility and versioning that are well understood with objects
+ properties.The blob should be standardized in the UAPI and structures to parse them should be fixed. With this compatibility issues can be prevented.
I think that is short-sighted.
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Also since info is passed via blobs we have the flexibility to even define segmented LUTs and PWL hardware blocks. Also we have left scope for custom private hardware blocks as well which driver can work with
respective HAL and get that implemented.
When color ops are real KMS objects they still can have properties which are blobs that can store LUTs and other data.
Quoted 7 lines
Quoted 2 lines
And we should avoid private blocks at all cost. In fact, I don't think the KMS rules have changed in that regard and it simply is not allowed.
Private blocks are not standardized but are vendor specific. So generic userspace will ignore these. However vendor and its respective HAL can make use of this field and leaves a scope to cater to such hardware vendors need. This doesn't affect the expectation of the standardized color operations which will be defined as enum in UAPI.
Vendors can have and expose their own unique snowflake operations without any "private" as well: pick an unused operation type code, and document what it does. Advertise it in some pipelines.
Vendor HALs can make use of it, but it also allows generic userspace to make use of it at will, and it allows other vendors to implement the same and benefit from it without needing to patch every userspace.
Or does Intel not want other vendors to potentially make use of their UAPIs?
Quoted 56 lines
Quoted 48 lines
Quoted 5 lines
We can even define prescriptive operations as a private entry and enable it if a certain driver and HAL agree.
Quoted 2 lines
Quoted 2 lines
To advertise the available color pipelines, an immutable ENUM property "
GET_COLOR_PIPELINE" is introduced. This enum property hasblob id's as values.
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
With each blob id representing a distinct color pipeline based on underlying HW capabilities and their respective combinations.
Once the user space decides on a color pipeline, it can set the pipeline and the corresponding data for the hardware blocks within the pipeline with the BLOB property "
SET_COLOR_PIPELINE".Quoted 7 lines
Quoted 5 lines
When I discussed this at the hackfest with Simon he proposed a new DRM object, (I've called it a
drm_colorop) to represent a color operation. Eachdrm_colorophas a "NEXT" pointer to anotherdrm_colorop, or NULL if its the last in the pipeline. You can then have a mutable enum property on the plane to discover and select a color pipeline.Yes, the proposal is inspired by this idea. Sure, we can work together to enhance
the design.
Quoted 3 lines
Personally I feel the one proposed in the current RFC will do the same thing envisioned by Simon and you Harry. Management of the pipeline, addition, deletion and flexibility to represent hardware is more with blobs with
the relevant structures agreed in UAPI.
Quoted 22 lines
Quoted 10 lines
This seems a bit more transparent than a blob. You can see my changes (unfortunately very WIP, don't look too closely at individual patches) at
https://gitlab.freedesktop.org/hwentland/linux/-/merge_requests/5/di
ffslibdrm changes:
https://gitlab.freedesktop.org/hwentland/drm/-/merge_requests/1/diff
sSure, will check the same.
Quoted 6 lines
IGT changes:
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/merge_reque
sts/1/diffsI'll take time to review your whole series and will see whether we can somehow keep the best parts of each.
Thanks and agree. Let's work together and get this implemented in DRM.
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Curious to hear other opinions on the blob vs new DRM object question as
well.
I can see a few advantages with the blob approach.
User space can store the entire state in one blob and just assign a new blob to change to another pipeline configuration.
Quoted 7 lines
Agree
Quoted 3 lines
However, I would argue that changing a lot of properties is already common practice and works well. User space can deal with it and has abstractions to make this managable.
Blob gives a lot of flexibility and ability to define the hardware capabilities generically.
The structure of the atomic commit ioctl and the KMS property system is even more flexible.
Quoted 4 lines
Lut distribution, number of segments, samples in each segment, precision of luts etc. can be precisely defined and userspace will get a complete picture of the underlying hardware and its capabilities. Also this is being done with just 2 properties. Leaving scope for future addition of any standard color operation as well.
The number of properties does not seem too useful to strictly minimise over other aspects.
Thanks,
pqQuoted 11 lines
I feel blob approach once properly documented is a bit more flexible and scalable. Maybe I am bit biased here 😊 but all ideas are welcome.
We have implemented some IGT's as well to explain the design better. Link below:
https://patchwork.freedesktop.org/series/123018/Thanks Sebastian for the feedback.
Regards,
Uma Shankar
Patchwork
CI[Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for Add Support for Plane Color Pipeline
In reply to Uma Shankar== Series Details ==
== Series Details ==
Series: Add Support for Plane Color Pipeline
URL : https://patchwork.freedesktop.org/series/123023/
State : warning
== Summary ==
Error: dim checkpatch failed5b6e9790b630 drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline -:14: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating? #14:
new file mode 100644
-:19: WARNING:SPDX_LICENSE_TAG: Missing or malformed SPDX-License-Identifier tag in line 1 #19: FILE: Documentation/gpu/rfc/plane_color_pipeline.rst:1:
+=======================================
total: 0 errors, 2 warnings, 0 checks, 394 lines checked 4aeebe895139 drm: Add color operation structure970dacdffa01 drm: Add plane get color pipeline property 7b46fd8d80a1 drm: Add helper to add color pipeline 97ea5d644a12 drm: Add structures for setting color pipeline 1e6046947815 drm: Add set colorpipeline property
-:28: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #28: FILE: drivers/gpu/drm/drm_atomic_uapi.c:595:
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->set_color_pipeline_data,
total: 0 errors, 0 warnings, 1 checks, 112 lines checked 63ce75dc0166 drm: Add Enhanced Gamma LUT precision structure bef188076fbe drm: Add color lut range structureae5fe78f0d56 drm: Add color information to plane state 19343ed5ea63 drm: Manage color blob states862b423779e4 drm: Replace individual color blobs
-:31: CHECK:PREFER_KERNEL_TYPES: Prefer kernel type 'u64' over 'uint64_t' #31: FILE: drivers/gpu/drm/drm_atomic_uapi.c:417:
+ uint64_t color_pipeline_blob_id,
-:64: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #64: FILE: drivers/gpu/drm/drm_atomic_uapi.c:450:
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.ctm,
-:70: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #70: FILE: drivers/gpu/drm/drm_atomic_uapi.c:456:
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.pre_csc_lut,
-:76: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #76: FILE: drivers/gpu/drm/drm_atomic_uapi.c:462:
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.post_csc_lut,
-:82: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #82: FILE: drivers/gpu/drm/drm_atomic_uapi.c:468:
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.private_color_op_data,
total: 0 errors, 0 warnings, 5 checks, 109 lines checked ea3a48a26b45 drm: Reset pipeline when user sends NULL blob b75c6a76b293 drm: Reset plane color state on pipeline switch request -:67: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #67: FILE: drivers/gpu/drm/drm_atomic_uapi.c:765:
+ cp_change_requested = color_pipeline_change_requested(dev,
+ state->set_color_pipeline_data,
-:83: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #83: FILE: drivers/gpu/drm/drm_atomic_uapi.c:782:
+ ret = drm_plane_reset_color_op_blobs(plane,
+ state, &replaced);
total: 0 errors, 0 warnings, 2 checks, 72 lines checkeda018bab5b7c5 drm/i915/color: Add lut range for SDR planes 11ce867cfea0 drm/i915/color: Add lut range for HDR planes a4e18914ef30 drm/i915/color: Add color pipeline for HDR planes cfc3198574b2 drm/i915/color: Add color pipeline for SDR planes 35769022d6c9 drm/i915/color: Add HDR plane LUT range data to color pipeline 4e6de9ba7fbd drm/i915/color: Add SDR plane LUT range data to color pipeline 1ca1fc5c040f drm/i915/color: Add color pipelines to plane 9b1b69f14b09 drm/i915/color: Create and attach set color pipeline property d9b196b1c211 drm/i915/color: Add plane color callbacks5eba0edf5e53 drm/i915/color: Load plane color luts from atomic flip 30d22691d27a drm/i915/xelpd: Add plane color check to glk_plane_color_ctl 7b6f6ba37ab3 drm/i915/xelpd: Add register definitions for Plane Degamma -:29: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects? #29: FILE: drivers/gpu/drm/i915/i915_reg.h:6238:
+#define PLANE_PRE_CSC_GAMC_INDEX_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_INDEX_ENH_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_INDEX_ENH_2(pipe))
-:41: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects? #41: FILE: drivers/gpu/drm/i915/i915_reg.h:6250:
+#define PLANE_PRE_CSC_GAMC_DATA_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_DATA_ENH_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_DATA_ENH_2(pipe))
-:53: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects? #53: FILE: drivers/gpu/drm/i915/i915_reg.h:6262:
+#define PLANE_PRE_CSC_GAMC_INDEX(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_INDEX_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_INDEX_2(pipe))
-:65: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects? #65: FILE: drivers/gpu/drm/i915/i915_reg.h:6274:
+#define PLANE_PRE_CSC_GAMC_DATA(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_PRE_CSC_GAMC_DATA_1(pipe), \
+ _PLANE_PRE_CSC_GAMC_DATA_2(pipe))
total: 0 errors, 0 warnings, 4 checks, 55 lines checked e4902513d633 drm/i915/color: Add color functions for ADL b7087ab3594d drm/i915/color: Program Plane Pre-CSC Registers af571e45b3e7 drm/i915/xelpd: Add register definitions for Plane Post CSC -:25: WARNING:LONG_LINE: line length of 110 exceeds 100 columns #25: FILE: drivers/gpu/drm/i915/i915_reg.h:6234:
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1_A, \
-:27: WARNING:LONG_LINE: line length of 110 exceeds 100 columns #27: FILE: drivers/gpu/drm/i915/i915_reg.h:6236:
+#define _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2_A, \
-:29: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects? #29: FILE: drivers/gpu/drm/i915/i915_reg.h:6238:
+#define PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_SEG0_INDEX_ENH_2(pipe))
-:37: WARNING:LONG_LINE: line length of 109 exceeds 100 columns #37: FILE: drivers/gpu/drm/i915/i915_reg.h:6246:
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1_A, \
-:39: WARNING:LONG_LINE: line length of 109 exceeds 100 columns #39: FILE: drivers/gpu/drm/i915/i915_reg.h:6248:
+#define _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2(pipe) _PIPE(pipe, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2_A, \
-:41: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects? #41: FILE: drivers/gpu/drm/i915/i915_reg.h:6250:
+#define PLANE_POST_CSC_GAMC_SEG0_DATA_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_SEG0_DATA_ENH_2(pipe))
-:53: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects? #53: FILE: drivers/gpu/drm/i915/i915_reg.h:6262:
+#define PLANE_POST_CSC_GAMC_INDEX_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_INDEX_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_INDEX_ENH_2(pipe))
-:65: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects? #65: FILE: drivers/gpu/drm/i915/i915_reg.h:6274:
+#define PLANE_POST_CSC_GAMC_DATA_ENH(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_DATA_ENH_1(pipe), \
+ _PLANE_POST_CSC_GAMC_DATA_ENH_2(pipe))
-:77: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects? #77: FILE: drivers/gpu/drm/i915/i915_reg.h:6286:
+#define PLANE_POST_CSC_GAMC_INDEX(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_INDEX_1(pipe), \
+ _PLANE_POST_CSC_GAMC_INDEX_2(pipe))
-:89: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'pipe' - possible side-effects? #89: FILE: drivers/gpu/drm/i915/i915_reg.h:6298:
+#define PLANE_POST_CSC_GAMC_DATA(pipe, plane, i) \
+ _MMIO_PLANE_GAMC(plane, i, _PLANE_POST_CSC_GAMC_DATA_1(pipe), \
+ _PLANE_POST_CSC_GAMC_DATA_2(pipe))
total: 0 errors, 4 warnings, 6 checks, 79 lines checked 7746bc9e6186 drm/i915/xelpd: Program Plane Post CSC Registers 47f636cfee5f drm/i915/color: Enable Plane CSCef86823dca18 drm/i915/color: Enable plane color features 138b81f07c65 drm/i915/color: Add a dummy pipeline with 3D LUT -:59: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #59: FILE: drivers/gpu/drm/drm_atomic_uapi.c:565:
+ ret = drm_atomic_replace_property_blob_from_id(dev,
+ &state->color.lut_3d,
total: 0 errors, 0 warnings, 1 checks, 124 lines checkedd1d9a14565f2 drm/i915/color: Add example implementation for vendor specific color operation
Patchwork
CI[Intel-gfx] ✗ Fi.CI.SPARSE: warning for Add Support for Plane Color Pipeline
In reply to Uma Shankar== Series Details ==
== Series Details ==
Series: Add Support for Plane Color Pipeline
URL : https://patchwork.freedesktop.org/series/123023/
State : warning
== Summary ==
Error: dim sparse failed
Sparse version: v0.6.2
Fast mode used, each commit won't be checked separately.
Patchwork
CI[Intel-gfx] ✗ Fi.CI.BAT: failure for Add Support for Plane Color Pipeline
In reply to Uma Shankar== Series Details ==
== Series Details ==
Series: Add Support for Plane Color Pipeline
URL : https://patchwork.freedesktop.org/series/123023/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_13574 -> Patchwork_123023v1
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with Patchwork_123023v1 absolutely need to be verified manually.
If you think the reported changes have nothing to do with the changes introduced in Patchwork_123023v1, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/index.html
Participating hosts (40 -> 38)
------------------------------
Missing (2): fi-kbl-soraka fi-snb-2520m
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in Patchwork_123023v1:
### IGT changes ###
#### Possible regressions ####
* igt@debugfs_test@read_all_entries:
- bat-adlp-11: [PASS][1] -> [ABORT][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-adlp-11/igt@debugfs_test@read_all_entries.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-adlp-11/igt@debugfs_test@read_all_entries.html
* igt@i915_module_load@load:
- bat-dg2-11: [PASS][3] -> [ABORT][4]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-dg2-11/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-dg2-11/igt@[email protected]
- bat-dg2-9: [PASS][5] -> [ABORT][6]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-dg2-9/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-dg2-9/igt@[email protected]
#### Suppressed ####
The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.
* igt@i915_module_load@load:
- {bat-dg2-13}: [DMESG-WARN][7] ([i915#8879]) -> [ABORT][8]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-dg2-13/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-dg2-13/igt@[email protected]
- {bat-dg2-14}: [DMESG-WARN][9] ([i915#8879]) -> [ABORT][10]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-dg2-14/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-dg2-14/igt@[email protected]
Known issues
------------
Here are the changes found in Patchwork_123023v1 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@i915_suspend@basic-s3-without-i915:
- bat-rpls-2: [PASS][11] -> [ABORT][12] ([i915#7978] / [i915#8668])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-rpls-2/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-rpls-2/igt@[email protected]
* igt@kms_psr@sprite_plane_onoff:
- bat-rplp-1: NOTRUN -> [ABORT][13] ([i915#8442] / [i915#8668] / [i915#8712])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-rplp-1/igt@kms_psr@sprite_plane_onoff.html
#### Warnings ####
* igt@kms_psr@cursor_plane_move:
- bat-rplp-1: [ABORT][14] ([i915#8469] / [i915#8668]) -> [SKIP][15] ([i915#1072])
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13574/bat-rplp-1/igt@kms_psr@cursor_plane_move.html
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/bat-rplp-1/igt@kms_psr@cursor_plane_move.html
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
[i915#7978]: https://gitlab.freedesktop.org/drm/intel/issues/7978
[i915#8442]: https://gitlab.freedesktop.org/drm/intel/issues/8442
[i915#8469]: https://gitlab.freedesktop.org/drm/intel/issues/8469
[i915#8668]: https://gitlab.freedesktop.org/drm/intel/issues/8668
[i915#8712]: https://gitlab.freedesktop.org/drm/intel/issues/8712
[i915#8879]: https://gitlab.freedesktop.org/drm/intel/issues/8879
Build changes
-------------
* Linux: CI_DRM_13574 -> Patchwork_123023v1
CI-20190529: 20190529
CI_DRM_13574: 2a7ec376910bf7ab4ae8a2594e73d9b5c1570046 @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_7459: 7459
Patchwork_123023v1: 2a7ec376910bf7ab4ae8a2594e73d9b5c1570046 @ git://anongit.freedesktop.org/gfx-ci/linux
### Linux commits
ecc6154eb822 drm/i915/color: Add example implementation for vendor specific color operation c10cba0b8b80 drm/i915/color: Add a dummy pipeline with 3D LUT6e9e83d26c3d drm/i915/color: Enable plane color featurescd9576deea7b drm/i915/color: Enable Plane CSC5074441371bb drm/i915/xelpd: Program Plane Post CSC Registers89decd8c4135 drm/i915/xelpd: Add register definitions for Plane Post CSC a8a81ddbe737 drm/i915/color: Program Plane Pre-CSC Registers57032feab0b9 drm/i915/color: Add color functions for ADLb3d1021ff46e drm/i915/xelpd: Add register definitions for Plane Degamma 98d568bd1d0a drm/i915/xelpd: Add plane color check to glk_plane_color_ctl 34b5a55fbbab drm/i915/color: Load plane color luts from atomic flip d8d431692734 drm/i915/color: Add plane color callbacks22ea46c5f4db drm/i915/color: Create and attach set color pipeline property 13852a46688b drm/i915/color: Add color pipelines to plane5666e64f3220 drm/i915/color: Add SDR plane LUT range data to color pipeline 95df87f4d41d drm/i915/color: Add HDR plane LUT range data to color pipeline d4bc2af110fe drm/i915/color: Add color pipeline for SDR planes5ba690dfd71c drm/i915/color: Add color pipeline for HDR planes6e555e736ce4 drm/i915/color: Add lut range for HDR planesf08d49d53577 drm/i915/color: Add lut range for SDR planesbdd8237bbb19 drm: Reset plane color state on pipeline switch request 835faea2c02e drm: Reset pipeline when user sends NULL blobf207dc394c37 drm: Replace individual color blobs9191449a2b6d drm: Manage color blob states3c033443a4d7 drm: Add color information to plane statec4886eb7d358 drm: Add color lut range structuref86cbf894428 drm: Add Enhanced Gamma LUT precision structurefcca79be7769 drm: Add set colorpipeline property66caa50d297f drm: Add structures for setting color pipeline49719816cfa5 drm: Add helper to add color pipelinea0c8dce28b55 drm: Add plane get color pipeline property9d16aa3dfca1 drm: Add color operation structure761ad4cedd79 drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_123023v1/index.html