Skip to contentSkip to conversation

drm/amd/display: fix BT.2020 YCbCr output CSC matrices

PATCH Applied Nathan Lucas 3 participants 6 messages Aug 2 – Aug 10

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover drm/amd/display: fix BT.2020 YCbCr output CSC matrices

This patch series fixes COLOR_SPACE_YCBCR2020_TYPE in dc_hw_sequencer.c which is used for COLOR_SPACE_2020_YCBCR_LIMITED output but does not have correct luma and chroma scaling for that output range, and fixes its copies in dce_transform.c and dce110_opp_csc_v.c.

It looks like this matrix was originally added in commit 40df2f809e8f ("drm/amd/display: color space ycbcr709 support") as a raw BT.2020-NCL RGB to YCbCr conversion matrix that had not yet been prepared for unsigned limited or full-range quantization. Commit 973a9c810c78 ("drm/amd/display: Fix COLOR_SPACE_YCBCR2020_TYPE matrix") applied offsets for unsigned limited-range quantization but did not fix the luma and chroma scales which are still full range. Because of this, output on a calibrated display is too bright with incorrect color, although it still looks roughly correct at low luminance in my testing.

Later, in commit 51e6668ab4ba
("drm/amd/display: add missing CSC entries for BT.2020 for DCE IPs"), the incorrect COLOR_SPACE_YCBCR2020_TYPE matrix coefficients were duplicated to DCE COLOR_SPACE_2020_YCBCR_LIMITED and COLOR_SPACE_2020_YCBCR_FULL, so those matrices are also currently incorrect.

The first patch adds separate matrices for full and limited-range. The derivation is in the commit message. The second patch replaces the copied matrix coefficients in dce_transform.c and dce110_opp_csc_v.c with the new ones.

For testing I have only been able to check the limited-range DCN path on a 9070 XT with a patched kernel. amdgpu_dm_get_output_color_space does not currently have a path to select COLOR_SPACE_2020_YCBCR_FULL although the enum value exists. Also, I cannot test the DCE paths as I do not have the hardware to do so.

I ran into this issue measuring PQ test patterns through mpv with the following command line:

mpv \
--no-config \
--vo=gpu-next \
--gpu-context=displayvk \
--gpu-api=vulkan \
--vulkan-display-plane=3 \
--vulkan-display-mode=22 \
--target-trc=pq \
--target-prim=bt.2020 \
--target-colorspace-hint-mode=source \
--target-colorspace-hint=yes \
--tone-mapping=clip \
--hdr-compute-peak=no \
[pattern file]

Free test patterns:
https://diversifiedvideosolutions.com/hdr-10.html

Below is a table of values I measured on a Samsung S95H (US model) with a Calibrite Display Pro HL meter.

Three sets of meter readings below:
Unpatched - Unpatched Fedora 44 kernel 7.1.4-200.fc44.x86_64
Patched - The patches in these e-mails applied to Fedora 44 kernel

            7.1.4-200.fc44.x86_64

NVS/Kodi - NVIDIA Shield Pro with Kodi

5% HDR10/PQ
Test Pattern Meter Readings (nits)
============== ==============================

PQ%       Nits   Unpatched   Patched   NVS/Kodi
15        1.01        1.63      1.01     1.02
25        5.24        9.68      5.46     5.12
35       18.78       40        20.5     21.2
45       56.55      128        62.0     62.1
55      148.03      369       155      155
65      384.71     1106       403      401
75      973.13     2681*     1030     1025

* This is at the peak capability for this TV.

PQ brightness overtracks a bit in Patched and NVS/Kodi, but that is likely due to the TV's default calibration or my meter since the readings are so similar. Unpatched stands out as being way too bright.

Nathan Lucas (2):
drm/amd/display: fix BT.2020 YCbCr limited output CSC matrix drm/amd/display: fix BT.2020 YCbCr output CSC matrices for DCE

 .../drm/amd/display/dc/core/dc_hw_sequencer.c | 31 ++++++++++++-------
 .../drm/amd/display/dc/dce/dce_transform.c    |  7 +++--
 .../amd/display/dc/dce110/dce110_opp_csc_v.c  |  7 +++--
 3 files changed, 28 insertions(+), 17 deletions(-)
Signature · 1 line
2.55.0
#1 lore raw

PATCH 1/2 drm/amd/display: fix BT.2020 YCbCr limited output CSC matrix

COLOR_SPACE_YCBCR2020_TYPE, which is selected for COLOR_SPACE_2020_YCBCR_LIMITED color_space, has coefficients that are incorrect for limited-range output. Its luma and chroma scaling is full-range so output is too bright and colors are incorrect.

COLOR_SPACE_YCBCR2020_TYPE is closer to a full-range conversion matrix with incorrect luma offset, so correct the luma offset for full-range and rename it to COLOR_SPACE_YCBCR2020_FULL_TYPE.

Add COLOR_SPACE_YCBCR2020_LIMITED_TYPE with correct scaling and range for limited-range output.

Fix related functions so COLOR_SPACE_YCBCR2020_LIMITED_TYPE and COLOR_SPACE_YCBCR2020_FULL_TYPE are correctly selected based on dc_color_space.

Derivation of both matrices follows ITU-T H.273:

Table 4, MatrixCoefficients 9, BT.2020-NCL weights:
KR = 0.2627, KB = 0.0593, KG = 1 - KR - KB = 0.6780.

Equations 45-47 in matrix form:

            [  KR             KG             KB            0 ]
M2020_NCL = [ -KR/(2(1-KB))  -KG/(2(1-KB))   1/2           0 ]
            [  1/2           -KG/(2(1-KR))  -KB/(2(1-KR))  0 ]
            [  0              0              0             1 ]

Limited and Full transforms based on equations 30-32 and 36-38 with bit depth 10, normalized by 1023:

            [ 876/1023   0         0         64/1023  ]
MLimited  = [ 0          896/1023  0         512/1023 ]
            [ 0          0         896/1023  512/1023 ]
            [ 0          0         0         1        ]
            [ 1023/1023  0         0         0        ]
    MFull = [ 0          1023/1023 0         512/1023 ]
            [ 0          0         1023/1023 512/1023 ]
            [ 0          0         0         1        ]

M2020_NCL_Limited = MLimited x M2020_NCL
M2020_NCL_Full = MFull x M2020_NCL

The upper three rows of M2020_NCL_* are stored in CR, Y, CB order. Each M2020_NCL_* value is stored as Round(value * 8192) in its 16-bit two's-complement representation.

Fixes 973a9c810c78 ("drm/amd/display: Fix COLOR_SPACE_YCBCR2020_TYPE matrix")
Assisted-by OpenAI-Codex:GPT-5.6-Sol
---
 .../drm/amd/display/dc/core/dc_hw_sequencer.c | 31 ++++++++++++-------
 1 file changed, 20 insertions(+), 11 deletions(-)
drivers/gpu/drm/amd/display/dc/core/dc_hw_sequencer.c +20 -11
index 11411fa94665..e686815f9a0e 100644
@@ -59,7 +59,8 @@ enum dc_color_space_type {
 	COLOR_SPACE_RGB_LIMITED_TYPE,
 	COLOR_SPACE_YCBCR601_TYPE,
 	COLOR_SPACE_YCBCR709_TYPE,
-	COLOR_SPACE_YCBCR2020_TYPE,
+	COLOR_SPACE_YCBCR2020_LIMITED_TYPE,
+	COLOR_SPACE_YCBCR2020_FULL_TYPE,
 	COLOR_SPACE_YCBCR601_LIMITED_TYPE,
 	COLOR_SPACE_YCBCR709_LIMITED_TYPE,
 	COLOR_SPACE_YCBCR709_BLACK_TYPE,
@@ -111,9 +112,15 @@ static const struct out_csc_color_matrix_type output_csc_matrix[] = {
 		{ 0xE00, 0xF349, 0xFEB7, 0x1000,
 		  0x6CE, 0x16E3, 0x24F,  0x200,
 		  0xFCCB, 0xF535, 0xE00, 0x1000} },
-	{ COLOR_SPACE_YCBCR2020_TYPE,
+	/* Corrected. Not included in the TODO above. */
+	{ COLOR_SPACE_YCBCR2020_LIMITED_TYPE,
+		{ 0x0E04, 0xF31D, 0xFEDF, 0x1004,
+		  0x0733, 0x1294, 0x01A0, 0x0201,
+		  0xFC16, 0xF5E6, 0x0E04, 0x1004} },
+	/* Corrected. Not included in the TODO above. */
+	{ COLOR_SPACE_YCBCR2020_FULL_TYPE,
 		{ 0x1000, 0xF149, 0xFEB7, 0x1004,
-		  0x0868, 0x15B2, 0x01E6, 0x201,
+		  0x0868, 0x15B2, 0x01E6, 0,
 		  0xFB88, 0xF478, 0x1000, 0x1004} },
 	{ COLOR_SPACE_YCBCR709_BLACK_TYPE,
 		{ 0x0000, 0x0000, 0x0000, 0x1000,
@@ -180,14 +187,14 @@ static bool is_ycbcr709_type(
 	return ret;
 }
 
-static bool is_ycbcr2020_type(
-	enum dc_color_space color_space)
+static bool is_ycbcr2020_limited_type(enum dc_color_space color_space)
 {
-	bool ret = false;
+	return color_space == COLOR_SPACE_2020_YCBCR_LIMITED;
+}
 
-	if (color_space == COLOR_SPACE_2020_YCBCR_LIMITED || color_space == COLOR_SPACE_2020_YCBCR_FULL)
-		ret = true;
-	return ret;
+static bool is_ycbcr2020_full_type(enum dc_color_space color_space)
+{
+	return color_space == COLOR_SPACE_2020_YCBCR_FULL;
 }
 
 static bool is_ycbcr709_limited_type(
@@ -216,8 +223,10 @@ static enum dc_color_space_type get_color_space_type(enum dc_color_space color_s
 		type = COLOR_SPACE_YCBCR601_LIMITED_TYPE;
 	else if (is_ycbcr709_limited_type(color_space))
 		type = COLOR_SPACE_YCBCR709_LIMITED_TYPE;
-	else if (is_ycbcr2020_type(color_space))
-		type = COLOR_SPACE_YCBCR2020_TYPE;
+	else if (is_ycbcr2020_limited_type(color_space))
+		type = COLOR_SPACE_YCBCR2020_LIMITED_TYPE;
+	else if (is_ycbcr2020_full_type(color_space))
+		type = COLOR_SPACE_YCBCR2020_FULL_TYPE;
 	else if (color_space == COLOR_SPACE_YCBCR709)
 		type = COLOR_SPACE_YCBCR709_BLACK_TYPE;
 	else if (color_space == COLOR_SPACE_YCBCR709_BLACK)
Signature · 1 line
2.55.0
#2 lore raw

PATCH 2/2 drm/amd/display: fix BT.2020 YCbCr output CSC matrices for DCE

In reply to Nathan Lucas

The commit cited by the Fixes tag added separate limited and full-range BT.2020 YCbCr entries to the DCE output CSC tables, but populated both entries with the same matrix copied from the common DC table. That matrix combined full-range scaling with limited-range luma offset and was incorrect for both limited and full-range output.

Replace the coefficients in both entries in the DCE paths with those from the new COLOR_SPACE_YCBCR2020_LIMITED_TYPE
and COLOR_SPACE_YCBCR2020_FULL_TYPE entries in the preceding commit ("drm/amd/display: fix BT.2020 YCbCr limited output CSC matrix").

Fixes 51e6668ab4ba ("drm/amd/display: add missing CSC entries for BT.2020 for DCE IPs")
Assisted-by OpenAI-Codex:GPT-5.6-Sol
---
 drivers/gpu/drm/amd/display/dc/dce/dce_transform.c       | 7 ++++---
 drivers/gpu/drm/amd/display/dc/dce110/dce110_opp_csc_v.c | 7 ++++---
 2 files changed, 8 insertions(+), 6 deletions(-)
drivers/gpu/drm/amd/display/dc/dce/dce_transform.c +4 -3
index 864491bfd7f7..83f819890020 100644
@@ -115,10 +115,11 @@ static const struct out_csc_color_matrix global_color_matrix[] = {
 	{ 0x2000, 0, 0, 0, 0, 0x2000, 0, 0, 0, 0, 0x2000, 0} },
 { COLOR_SPACE_2020_RGB_LIMITEDRANGE,
 	{ 0x1B67, 0, 0, 0x201, 0, 0x1B67, 0, 0x201, 0, 0, 0x1B67, 0x201} },
-{ COLOR_SPACE_2020_YCBCR_LIMITED, { 0x1000, 0xF149, 0xFEB7, 0x1004, 0x0868,
-	0x15B2, 0x01E6, 0x201, 0xFB88, 0xF478, 0x1000, 0x1004} },
+/* COLOR_SPACE_2020_YCBCR_* values corrected. Not included in the TODO above. */
+{ COLOR_SPACE_2020_YCBCR_LIMITED, { 0x0E04, 0xF31D, 0xFEDF, 0x1004, 0x0733,
+	0x1294, 0x01A0, 0x201, 0xFC16, 0xF5E6, 0x0E04, 0x1004} },
 { COLOR_SPACE_2020_YCBCR_FULL, { 0x1000, 0xF149, 0xFEB7, 0x1004, 0x0868, 0x15B2,
-	0x01E6, 0x201, 0xFB88, 0xF478, 0x1000, 0x1004} }
+	0x01E6, 0, 0xFB88, 0xF478, 0x1000, 0x1004} }
 };
 
 static bool setup_scaling_configuration(
drivers/gpu/drm/amd/display/dc/dce110/dce110_opp_csc_v.c +4 -3
index 1ed018aaa4bb..f5f8cd2d47a5 100644
@@ -93,10 +93,11 @@ static const struct out_csc_color_matrix global_color_matrix[] = {
 	{ 0x2000, 0, 0, 0, 0, 0x2000, 0, 0, 0, 0, 0x2000, 0} },
 { COLOR_SPACE_2020_RGB_LIMITEDRANGE,
 	{ 0x1B67, 0, 0, 0x201, 0, 0x1B67, 0, 0x201, 0, 0, 0x1B67, 0x201} },
-{ COLOR_SPACE_2020_YCBCR_LIMITED, { 0x1000, 0xF149, 0xFEB7, 0x1004, 0x0868,
-	0x15B2, 0x01E6, 0x201, 0xFB88, 0xF478, 0x1000, 0x1004} },
+/* COLOR_SPACE_2020_YCBCR_* values corrected. Not included in the TODO above. */
+{ COLOR_SPACE_2020_YCBCR_LIMITED, { 0x0E04, 0xF31D, 0xFEDF, 0x1004, 0x0733,
+	0x1294, 0x01A0, 0x201, 0xFC16, 0xF5E6, 0x0E04, 0x1004} },
 { COLOR_SPACE_2020_YCBCR_FULL, { 0x1000, 0xF149, 0xFEB7, 0x1004, 0x0868, 0x15B2,
-	0x01E6, 0x201, 0xFB88, 0xF478, 0x1000, 0x1004} }
+	0x01E6, 0, 0xFB88, 0xF478, 0x1000, 0x1004} }
 };
 
 enum csc_color_mode {
Signature · 1 line
2.55.0
Tested-by on v1 2/2
#3 lore raw

Re: [PATCH 2/2] drm/amd/display: fix BT.2020 YCbCr output CSC matrices for DCE

Hi Nathan,

I hit exactly this on a DCE 11.2 part, so here is a data point for the path you could not test.

Setup: XFX RX 580 8GB (Polaris10, DCE 11.2) as an external GPU on an Orange Pi 5 Plus (RK3588, arm64) over PCIe Gen3 x4, driving a Sony 65" HDR10 TV at 3840x2160@60 with HDR enabled under KWin/Wayland. The sink lists 3840x2160@60 only in its YCbCr 4:2:0 video data block and caps TMDS at 300 MHz, so the stream comes out as PIXEL_ENCODING_YCBCR420 (confirmed in hardware: FMT_CONTROL = 0x002a0000, FMT_PIXEL_ENCODING = 2), and with the compositor setting Colorspace=BT2020_RGB the output color space ends up COLOR_SPACE_2020_YCBCR_LIMITED.

My tree predates 51e6668ab4ba, so global_color_matrix[] had no BT.2020 row at all. That is worth recording as its own failure mode: with no matching entry, dce110_opp_set_csc_default() programs no matrix and configure_graphics_mode() leaves OUTPUT_CSC_GRPH_MODE at 0, so the output CSC stays bypassed and plain RGB is fed to the 4:2:0 formatter while the AVI infoframe declares YCbCr 4:2:0 with BT.2020 colorimetry. The sink then applies a BT.2020 YCbCr to RGB matrix to RGB data: luma lands on green, Cb/Cr sit near mid scale, and the picture is washed out with a heavy green cast. SDR on the same link was fine, since it stays on the BT.709 entry.

Applying 51e6668ab4ba plus this series on top of 7.0 fixes it: colors are correct on that TV in HDR, across a reboot, with no amdgpu errors in the log. As a cross-check, the same desktop on a second monitor driven by a completely different pipeline (the SoC's own display controller) now matches it.

I also booted 51e6668ab4ba without this series on the same setup and could not tell the two apart by eye. That is not a counter-argument to the fix, just a note on sensitivity: the compositor maps SDR content to around 350 nits, so nothing gets near the top of the PQ range where the full-range luma scaling would clip, and the residual difference stays below what this TV's processing reveals.

Thanks,
Igor

#4 lore raw

Re: [PATCH 2/2] drm/amd/display: fix BT.2020 YCbCr output CSC matrices for DCE

Hi Igor,

Thank you for taking the time to test the patches. Your RX 580 should hit the new limited-range matrix in dce_transform.c which I was not able to directly test.

Thanks again,
Nathan

Quoted 43 lines · Igor Paunovic

Hi Nathan,

I hit exactly this on a DCE 11.2 part, so here is a data point for the path you could not test.

Setup: XFX RX 580 8GB (Polaris10, DCE 11.2) as an external GPU on an Orange Pi 5 Plus (RK3588, arm64) over PCIe Gen3 x4, driving a Sony 65" HDR10 TV at 3840x2160@60 with HDR enabled under KWin/Wayland. The sink lists 3840x2160@60 only in its YCbCr 4:2:0 video data block and caps TMDS at 300 MHz, so the stream comes out as PIXEL_ENCODING_YCBCR420 (confirmed in hardware: FMT_CONTROL = 0x002a0000, FMT_PIXEL_ENCODING = 2), and with the compositor setting Colorspace=BT2020_RGB the output color space ends up COLOR_SPACE_2020_YCBCR_LIMITED.

My tree predates 51e6668ab4ba, so global_color_matrix[] had no BT.2020 row at all. That is worth recording as its own failure mode: with no matching entry, dce110_opp_set_csc_default() programs no matrix and configure_graphics_mode() leaves OUTPUT_CSC_GRPH_MODE at 0, so the output CSC stays bypassed and plain RGB is fed to the 4:2:0 formatter while the AVI infoframe declares YCbCr 4:2:0 with BT.2020 colorimetry. The sink then applies a BT.2020 YCbCr to RGB matrix to RGB data: luma lands on green, Cb/Cr sit near mid scale, and the picture is washed out with a heavy green cast. SDR on the same link was fine, since it stays on the BT.709 entry.

Applying 51e6668ab4ba plus this series on top of 7.0 fixes it: colors are correct on that TV in HDR, across a reboot, with no amdgpu errors in the log. As a cross-check, the same desktop on a second monitor driven by a completely different pipeline (the SoC's own display controller) now matches it.

I also booted 51e6668ab4ba without this series on the same setup and could not tell the two apart by eye. That is not a counter-argument to the fix, just a note on sensitivity: the compositor maps SDR content to around 350 nits, so nothing gets near the top of the PQ range where the full-range luma scaling would clip, and the residual difference stays below what this TV's processing reveals.

Thanks,
Igor

Applied
#5 lore raw
In reply to Nathan Lucas

Applied the series. Thanks!

Alex

Quoted 93 lines · Nathan Lucas

This patch series fixes COLOR_SPACE_YCBCR2020_TYPE in dc_hw_sequencer.c which is used for COLOR_SPACE_2020_YCBCR_LIMITED output but does not have correct luma and chroma scaling for that output range, and fixes its copies in dce_transform.c and dce110_opp_csc_v.c.

It looks like this matrix was originally added in commit 40df2f809e8f ("drm/amd/display: color space ycbcr709 support") as a raw BT.2020-NCL RGB to YCbCr conversion matrix that had not yet been prepared for unsigned limited or full-range quantization. Commit 973a9c810c78 ("drm/amd/display: Fix COLOR_SPACE_YCBCR2020_TYPE matrix") applied offsets for unsigned limited-range quantization but did not fix the luma and chroma scales which are still full range. Because of this, output on a calibrated display is too bright with incorrect color, although it still looks roughly correct at low luminance in my testing.

Later, in commit 51e6668ab4ba
("drm/amd/display: add missing CSC entries for BT.2020 for DCE IPs"), the incorrect COLOR_SPACE_YCBCR2020_TYPE matrix coefficients were duplicated to DCE COLOR_SPACE_2020_YCBCR_LIMITED and COLOR_SPACE_2020_YCBCR_FULL, so those matrices are also currently incorrect.

The first patch adds separate matrices for full and limited-range. The derivation is in the commit message. The second patch replaces the copied matrix coefficients in dce_transform.c and dce110_opp_csc_v.c with the new ones.

For testing I have only been able to check the limited-range DCN path on a 9070 XT with a patched kernel. amdgpu_dm_get_output_color_space does not currently have a path to select COLOR_SPACE_2020_YCBCR_FULL although the enum value exists. Also, I cannot test the DCE paths as I do not have the hardware to do so.

I ran into this issue measuring PQ test patterns through mpv with the following command line:

mpv \
--no-config \
--vo=gpu-next \
--gpu-context=displayvk \
--gpu-api=vulkan \
--vulkan-display-plane=3 \
--vulkan-display-mode=22 \
--target-trc=pq \
--target-prim=bt.2020 \
--target-colorspace-hint-mode=source \
--target-colorspace-hint=yes \
--tone-mapping=clip \
--hdr-compute-peak=no \
[pattern file]

Free test patterns:
https://diversifiedvideosolutions.com/hdr-10.html

Below is a table of values I measured on a Samsung S95H (US model) with a Calibrite Display Pro HL meter.

Three sets of meter readings below:
Unpatched - Unpatched Fedora 44 kernel 7.1.4-200.fc44.x86_64
Patched - The patches in these e-mails applied to Fedora 44 kernel

            7.1.4-200.fc44.x86_64

NVS/Kodi - NVIDIA Shield Pro with Kodi

5% HDR10/PQ
Test Pattern Meter Readings (nits)
============== ==============================

PQ%       Nits   Unpatched   Patched   NVS/Kodi
15        1.01        1.63      1.01     1.02
25        5.24        9.68      5.46     5.12
35       18.78       40        20.5     21.2
45       56.55      128        62.0     62.1
55      148.03      369       155      155
65      384.71     1106       403      401
75      973.13     2681*     1030     1025

* This is at the peak capability for this TV.

PQ brightness overtracks a bit in Patched and NVS/Kodi, but that is likely due to the TV's default calibration or my meter since the readings are so similar. Unpatched stands out as being way too bright.

Nathan Lucas (2):
drm/amd/display: fix BT.2020 YCbCr limited output CSC matrix drm/amd/display: fix BT.2020 YCbCr output CSC matrices for DCE

 .../drm/amd/display/dc/core/dc_hw_sequencer.c | 31 ++++++++++++-------
 .../drm/amd/display/dc/dce/dce_transform.c    |  7 +++--
 .../amd/display/dc/dce110/dce110_opp_csc_v.c  |  7 +++--
 3 files changed, 28 insertions(+), 17 deletions(-)

--
2.55.0