Skip to contentSkip to conversation

drm/amdgpu/vcn: fix integer overflow in dec_msg buffer count check

PATCH In review David (Ming Qiang) Wu 2 participants 2 messages Aug 11 – Aug 13

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 drm/amdgpu/vcn: fix integer overflow in dec_msg buffer count check

if supplied msg[2] in the header is too large, 4 times
of this unsigned 32 bit value will overflow and the test could pass.

v3: Using two-helper form is heavier than needed.
Since len_dw is a plain u32, a division-based test is overflow-free by construction.

Fixes b193019860d6 ("drm/amdgpu/vcn3: Prevent OOB reads when parsing dec msg")
Fixes 0a78f2bac142 ("drm/amdgpu/vcn4: Prevent OOB reads when parsing dec msg")
---
 drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c | 8 ++++++--
 drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c | 8 ++++++--
 2 files changed, 12 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c +6 -2
index 81bba3ec2a93..686f5f758cb1 100644
@@ -1964,8 +1964,12 @@ static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
 	len_dw = msg[1] / 4;
 	num_buffers = msg[2];
 
-	/* Verify that all indices fit within the claimed length. Each index is 4 DWORDs */
-	if (num_buffers > len_dw || 6 + num_buffers * 4 > len_dw) {
+	/* Verify that all indices fit within the claimed length.
+	 * There are 6 dwords in the header before the first buffer.
+	 * Each buffer has 4 dwords. Extra dwords will be ignored
+	 * at the end of the last buffer.
+	 */
+	if (len_dw < 6 || num_buffers > (len_dw - 6) / 4) {
 		DRM_ERROR("VCN message has too many buffers!\n");
 		r = -EINVAL;
 		goto out;
drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c +6 -2
index 0cce78b205a8..6f0a51c436db 100644
@@ -1880,8 +1880,12 @@ static int vcn_v4_0_dec_msg(struct amdgpu_cs_parser *p, struct amdgpu_job *job,
 	len_dw = msg[1] / 4;
 	num_buffers = msg[2];
 
-	/* Verify that all indices fit within the claimed length. Each index is 4 DWORDs */
-	if (num_buffers > len_dw || 6 + num_buffers * 4 > len_dw) {
+	/* Verify that all indices fit within the claimed length.
+	 * There are 6 dwords in the header before the first buffer.
+	 * Each buffer has 4 dwords. Extra dwords will be ignored
+	 * at the end of the last buffer.
+	 */
+	if (len_dw < 6 || num_buffers > (len_dw - 6) / 4) {
 		DRM_ERROR("VCN message has too many buffers!\n");
 		r = -EINVAL;
 		goto out;
Signature · 1 line
2.43.0
Reviewed-by on v3 1/1
#1 lore raw

AMD General

Quoted 74 lines

-----Original Message-----
From: Wu, David <[email protected]>
Sent: Tuesday, August 11, 2026 1:07 PM
To: [email protected]; Deucher, Alexander <[email protected]>
Cc: Liu, Leo <[email protected]>; [email protected]
Subject: [PATCH v3] drm/amdgpu/vcn: fix integer overflow in dec_msg buffer count check

if supplied msg[2] in the header is too large, 4 times of this unsigned 32 bit value will overflow and the test could pass.

v3: Using two-helper form is heavier than needed.
Since len_dw is a plain u32, a division-based test is overflow-free by construction.

Fixes b193019860d6 ("drm/amdgpu/vcn3: Prevent OOB reads when parsing dec

msg")

Fixes 0a78f2bac142 ("drm/amdgpu/vcn4: Prevent OOB reads when parsing dec

msg")

---
 drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c | 8 ++++++--
drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c | 8 ++++++--
 2 files changed, 12 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c +0 -1
index 81bba3ec2a93..686f5f758cb1 100644
@@ -1964,8 +1964,12 @@ static int vcn_v3_0_dec_msg(struct
 amdgpu_cs_parser *p, struct amdgpu_job *job,
      len_dw = msg[1] / 4;
      num_buffers = msg[2];
 
-     /* Verify that all indices fit within the claimed length. Each index is 4

DWORDs */

drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c +6 -1
-     if (num_buffers > len_dw || 6 + num_buffers * 4 > len_dw) {
+     /* Verify that all indices fit within the claimed length.
+      * There are 6 dwords in the header before the first buffer.
+      * Each buffer has 4 dwords. Extra dwords will be ignored
+      * at the end of the last buffer.
+      */
+     if (len_dw < 6 || num_buffers > (len_dw - 6) / 4) {
              DRM_ERROR("VCN message has too many buffers!\n");
              r = -EINVAL;
              goto out;
drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c +0 -1
index 0cce78b205a8..6f0a51c436db 100644
@@ -1880,8 +1880,12 @@ static int vcn_v4_0_dec_msg(struct
 amdgpu_cs_parser *p, struct amdgpu_job *job,
      len_dw = msg[1] / 4;
      num_buffers = msg[2];
 
-     /* Verify that all indices fit within the claimed length. Each index is 4

DWORDs */

drivers/gpu/drm/amd/amdgpu/vcn_v4_0.c +6 -2
-     if (num_buffers > len_dw || 6 + num_buffers * 4 > len_dw) {
+     /* Verify that all indices fit within the claimed length.
+      * There are 6 dwords in the header before the first buffer.
+      * Each buffer has 4 dwords. Extra dwords will be ignored
+      * at the end of the last buffer.
+      */
+     if (len_dw < 6 || num_buffers > (len_dw - 6) / 4) {
              DRM_ERROR("VCN message has too many buffers!\n");
              r = -EINVAL;
              goto out;
--

2.43.0