Skip to contentSkip to conversation

drm/amdgpu,radeon: fix integer overflow in pitch alignment

PATCH Superseded Werner Kasselman 3 participants 13 messages Apr 6 – Apr 17

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover drm/amdgpu,radeon: fix integer overflow in pitch alignment

Both amdgpu_gem_align_pitch() and radeon_align_pitch() use signed int for the pitch calculation. When alignment rounding pushes the width to a boundary value, 'aligned * cpp' overflows signed 32-bit int to 0.

This defeats the overflow guards in drm_mode_create_dumb() because these drivers bypass drm_mode_size_dumb() and perform their own alignment rounding, which can push the pitch past the pre-validated range.

A zero pitch propagates to a zero-size GEM object allocation reachable from unprivileged userspace via DRM_IOCTL_MODE_CREATE_DUMB on the render node.

Both drivers need the same fix: add an overflow check in the alignment function and reject zero pitch/size in the dumb_create callback. The proper long-term fix is to convert both drivers to use drm_mode_size_dumb() as Thomas Zimmermann's series is doing for other drivers.

Werner Kasselman (2):
drm/amdgpu: fix integer overflow in amdgpu_gem_align_pitch() drm/radeon: fix integer overflow in radeon_align_pitch()

 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 13 +++++++++++++
 drivers/gpu/drm/radeon/radeon_gem.c     |  9 +++++++++
 2 files changed, 22 insertions(+)
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/2 drm/amdgpu: fix integer overflow in amdgpu_gem_align_pitch()

amdgpu_gem_align_pitch() uses signed int for the pitch calculation. When alignment rounding pushes the width to a boundary value (e.g., 2^30 for cpp=4), the multiplication 'aligned * cpp' overflows signed 32-bit int, producing 0 or a negative value.

The overflow guard in drm_mode_create_dumb() validates width * cpp BEFORE the driver callback, but amdgpu_mode_dumb_create() bypasses the generic drm_mode_size_dumb() helper and performs its own alignment rounding, which can push the pitch past the pre-validated range.

A zero pitch propagates to a zero-size GEM object allocation via amdgpu_gem_object_create(). The 0-byte BO passes amdgpu_bo_validate_size() (since 0 < man->size) and is returned to userspace with a valid handle. This object can then be mmap'd or referenced in GPU command submissions, potentially causing out-of-bounds access to adjacent slab memory.

DRM_IOCTL_MODE_CREATE_DUMB requires no DRM authentication, so any local user with access to /dev/dri/renderD* can trigger this with e.g. width=1073741760, bpp=32, height=1.

Add an overflow check in amdgpu_gem_align_pitch() to detect when 'aligned * cpp' would exceed INT_MAX, returning 0 in that case. Add corresponding checks in amdgpu_mode_dumb_create() to reject pitch=0 and size=0 with -EINVAL.

The proper long-term fix is to convert amdgpu to use drm_mode_size_dumb() which centralizes pitch/size calculation with proper overflow guards, as is being done for other drivers in Thomas Zimmermann's dumb-buffer series.

Found via AST-based call-graph analysis using sqry.

Fixes 087451f372bf ("drm/amdgpu: use generic fb helpers instead of setting up AMD own's.")
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +13 -0
index a6107109a2b8..b4341abba20c 100644
@@ -1246,6 +1246,15 @@ static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
 
 	aligned += pitch_mask;
 	aligned &= ~pitch_mask;
+
+	/* Sanity check to avoid integer overflow in aligned * cpp.
+	 * The caller (drm_mode_create_dumb) validates width * cpp fits
+	 * in u32 before alignment, but rounding up can push aligned
+	 * past INT_MAX / cpp, causing signed overflow to 0 or negative.
+	 */
+	if (aligned > INT_MAX / (cpp ? cpp : 1) || aligned <= 0)
+		return 0;
+
 	return aligned * cpp;
 }
 
@@ -1273,8 +1282,12 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv,
 
 	args->pitch = amdgpu_gem_align_pitch(adev, args->width,
 					     DIV_ROUND_UP(args->bpp, 8), 0);
+	if (!args->pitch)
+		return -EINVAL;
 	args->size = (u64)args->pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
+	if (!args->size)
+		return -EINVAL;
 	domain = amdgpu_bo_get_preferred_domain(adev,
 				amdgpu_display_supported_domains(adev, flags));
 	r = amdgpu_gem_object_create(adev, args->size, 0, domain, flags,
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH 1/2] drm/amdgpu: fix integer overflow in amdgpu_gem_align_pitch()

Quoted 2 lines · Werner Kasselman

amdgpu_gem_align_pitch() uses signed int for the pitch calculation.

Can you convert amdgpu_gem_align_pitch() to use unsigned ints? The width passed to it is always unsigned. That would simplify things.

Alex

Quoted 75 lines

When alignment rounding pushes the width to a boundary value (e.g., 2^30 for cpp=4), the multiplication 'aligned * cpp' overflows signed 32-bit int, producing 0 or a negative value.

The overflow guard in drm_mode_create_dumb() validates width * cpp BEFORE the driver callback, but amdgpu_mode_dumb_create() bypasses the generic drm_mode_size_dumb() helper and performs its own alignment rounding, which can push the pitch past the pre-validated range.

A zero pitch propagates to a zero-size GEM object allocation via amdgpu_gem_object_create(). The 0-byte BO passes amdgpu_bo_validate_size() (since 0 < man->size) and is returned to userspace with a valid handle. This object can then be mmap'd or referenced in GPU command submissions, potentially causing out-of-bounds access to adjacent slab memory.

DRM_IOCTL_MODE_CREATE_DUMB requires no DRM authentication, so any local user with access to /dev/dri/renderD* can trigger this with e.g. width=1073741760, bpp=32, height=1.

Add an overflow check in amdgpu_gem_align_pitch() to detect when 'aligned * cpp' would exceed INT_MAX, returning 0 in that case. Add corresponding checks in amdgpu_mode_dumb_create() to reject pitch=0 and size=0 with -EINVAL.

The proper long-term fix is to convert amdgpu to use drm_mode_size_dumb() which centralizes pitch/size calculation with proper overflow guards, as is being done for other drivers in Thomas Zimmermann's dumb-buffer series.

Found via AST-based call-graph analysis using sqry.

Fixes 087451f372bf ("drm/amdgpu: use generic fb helpers instead of setting up AMD own's.")
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +13 -1
index a6107109a2b8..b4341abba20c 100644
@@ -1246,6 +1246,15 @@ static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
 
        aligned += pitch_mask;
        aligned &= ~pitch_mask;
+
+       /* Sanity check to avoid integer overflow in aligned * cpp.
+        * The caller (drm_mode_create_dumb) validates width * cpp fits
+        * in u32 before alignment, but rounding up can push aligned
+        * past INT_MAX / cpp, causing signed overflow to 0 or negative.
+        */
+       if (aligned > INT_MAX / (cpp ? cpp : 1) || aligned <= 0)
+               return 0;
+
        return aligned * cpp;
 }
 
@@ -1273,8 +1282,12 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv,
 
        args->pitch = amdgpu_gem_align_pitch(adev, args->width,
                                             DIV_ROUND_UP(args->bpp, 8), 0);
+       if (!args->pitch)
+               return -EINVAL;
        args->size = (u64)args->pitch * args->height;
        args->size = ALIGN(args->size, PAGE_SIZE);
+       if (!args->size)
+               return -EINVAL;
        domain = amdgpu_bo_get_preferred_domain(adev,
                                amdgpu_display_supported_domains(adev, flags));
        r = amdgpu_gem_object_create(adev, args->size, 0, domain, flags,
--

2.43.0

#3 lore raw

PATCH v2 1/1 drm/amdgpu: fix integer overflow in amdgpu_gem_align_pitch()

In reply to Werner Kasselman

amdgpu_gem_align_pitch() is passed u32 width and cpp from dumb buffer creation but uses signed int internally. The round-up add and the aligned * cpp multiplication can overflow, returning zero or a negative pitch. A zero pitch propagates to a zero-sized GEM object allocation that reaches userspace via DRM_IOCTL_MODE_CREATE_DUMB.

Switch the helper to unsigned int and use check_add_overflow() / check_mul_overflow() so wraparound returns zero. Reject a zero pitch or size in amdgpu_mode_dumb_create() rather than allocating a zero- byte BO.

Fixes 8e911ab770f7 ("drm: amdgpu: Replace drm_fb_get_bpp_depth() with drm_format_plane_cpp()")
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +17 -8
index a6107109a2b8..0d9309f792a4 100644
@@ -27,6 +27,7 @@
  */
 #include <linux/ktime.h>
 #include <linux/module.h>
+#include <linux/overflow.h>
 #include <linux/pagemap.h>
 #include <linux/pci.h>
 #include <linux/dma-buf.h>
@@ -1223,13 +1224,14 @@ int amdgpu_gem_list_handles_ioctl(struct drm_device *dev, void *data,
 	return ret;
 }
 
-static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
-				  int width,
-				  int cpp,
-				  bool tiled)
+static unsigned int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
+					   unsigned int width,
+					   unsigned int cpp,
+					   bool tiled)
 {
-	int aligned = width;
-	int pitch_mask = 0;
+	unsigned int aligned = width;
+	unsigned int pitch_mask = 0;
+	unsigned int pitch;
 
 	switch (cpp) {
 	case 1:
@@ -1244,9 +1246,12 @@ static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
 		break;
 	}
 
-	aligned += pitch_mask;
+	if (check_add_overflow(aligned, pitch_mask, &aligned))
+		return 0;
 	aligned &= ~pitch_mask;
-	return aligned * cpp;
+	if (check_mul_overflow(aligned, cpp, &pitch))
+		return 0;
+	return pitch;
 }
 
 int amdgpu_mode_dumb_create(struct drm_file *file_priv,
@@ -1273,8 +1278,12 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv,
 
 	args->pitch = amdgpu_gem_align_pitch(adev, args->width,
 					     DIV_ROUND_UP(args->bpp, 8), 0);
+	if (!args->pitch)
+		return -EINVAL;
 	args->size = (u64)args->pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
+	if (!args->size)
+		return -EINVAL;
 	domain = amdgpu_bo_get_preferred_domain(adev,
 				amdgpu_display_supported_domains(adev, flags));
 	r = amdgpu_gem_object_create(adev, args->size, 0, domain, flags,
Signature · 1 line
2.43.0
maintainer
#4 lore raw

Re: [PATCH v2] drm/amdgpu: fix integer overflow in amdgpu_gem_align_pitch()

Quoted 6 lines · Werner Kasselman

amdgpu_gem_align_pitch() is passed u32 width and cpp from dumb buffer creation but uses signed int internally. The round-up add and the aligned * cpp multiplication can overflow, returning zero or a negative pitch. A zero pitch propagates to a zero-sized GEM object allocation that reaches userspace via DRM_IOCTL_MODE_CREATE_DUMB.

Quoted 7 lines

Switch the helper to unsigned int and use check_add_overflow() / check_mul_overflow() so wraparound returns zero. Reject a zero pitch or size in amdgpu_mode_dumb_create() rather than allocating a zero- byte BO.

Fixes 8e911ab770f7 ("drm: amdgpu: Replace drm_fb_get_bpp_depth() with drm_format_plane_cpp()")

The patch looks valid to me, but I think we can drop this CC: stable.

Since dump buffers are a kernel only interface the fix has no practical relevance and is only of cosmetic nature.

Regards,
Christian.

Quoted 66 lines
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +17 -8
index a6107109a2b8..0d9309f792a4 100644
@@ -27,6 +27,7 @@
  */
 #include <linux/ktime.h>
 #include <linux/module.h>
+#include <linux/overflow.h>
 #include <linux/pagemap.h>
 #include <linux/pci.h>
 #include <linux/dma-buf.h>
@@ -1223,13 +1224,14 @@ int amdgpu_gem_list_handles_ioctl(struct drm_device *dev, void *data,
 	return ret;
 }
 
-static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
-				  int width,
-				  int cpp,
-				  bool tiled)
+static unsigned int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
+					   unsigned int width,
+					   unsigned int cpp,
+					   bool tiled)
 {
-	int aligned = width;
-	int pitch_mask = 0;
+	unsigned int aligned = width;
+	unsigned int pitch_mask = 0;
+	unsigned int pitch;
 
 	switch (cpp) {
 	case 1:
@@ -1244,9 +1246,12 @@ static int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
 		break;
 	}
 
-	aligned += pitch_mask;
+	if (check_add_overflow(aligned, pitch_mask, &aligned))
+		return 0;
 	aligned &= ~pitch_mask;
-	return aligned * cpp;
+	if (check_mul_overflow(aligned, cpp, &pitch))
+		return 0;
+	return pitch;
 }
 
 int amdgpu_mode_dumb_create(struct drm_file *file_priv,
@@ -1273,8 +1278,12 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv,
 
 	args->pitch = amdgpu_gem_align_pitch(adev, args->width,
 					     DIV_ROUND_UP(args->bpp, 8), 0);
+	if (!args->pitch)
+		return -EINVAL;
 	args->size = (u64)args->pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
+	if (!args->size)
+		return -EINVAL;
 	domain = amdgpu_bo_get_preferred_domain(adev,
 				amdgpu_display_supported_domains(adev, flags));
 	r = amdgpu_gem_object_create(adev, args->size, 0, domain, flags,
#5 lore raw

Re: [PATCH v2] drm/amdgpu: fix integer overflow in amdgpu_gem_align_pitch()

Quoted 4 lines · Christian König

The patch looks valid to me, but I think we can drop this CC: stable.

Since dump buffers are a kernel only interface the fix has no practical relevance and is only of cosmetic nature.

Ack, please drop it at apply time.

Werner

#6 lore raw

Re: [PATCH v2] drm/amdgpu: fix integer overflow in amdgpu_gem_align_pitch()

Applied thanks!

Alex

Quoted 10 lines · Werner Kasselman
Quoted 4 lines · Christian König

The patch looks valid to me, but I think we can drop this CC: stable.

Since dump buffers are a kernel only interface the fix has no practical relevance and is only of cosmetic nature.

Ack, please drop it at apply time.

Werner

#7 lore raw

PATCH 2/2 drm/radeon: fix integer overflow in radeon_align_pitch()

In reply to Werner Kasselman

radeon_align_pitch() has the same integer overflow as amdgpu's variant:
'aligned * cpp' can overflow signed int to 0 when alignment rounding pushes the width past INT_MAX/cpp. This produces a 0-byte GEM buffer via radeon_mode_dumb_create(), reachable from unprivileged userspace via DRM_IOCTL_MODE_CREATE_DUMB on the render node.

Add an overflow check in radeon_align_pitch() and reject zero pitch/size in radeon_mode_dumb_create().

Found via AST-based call-graph analysis using sqry.

Fixes ff72145badb8 ("drm: dumb scanout create/mmap for intel/radeon (v3)")
---
 drivers/gpu/drm/radeon/radeon_gem.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/radeon/radeon_gem.c +9 -0
index 20fc87409f2e..2cd179fef347 100644
@@ -828,6 +828,11 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
 
 	aligned += pitch_mask;
 	aligned &= ~pitch_mask;
+
+	/* Guard against integer overflow in aligned * cpp. */
+	if (aligned > INT_MAX / (cpp ? cpp : 1) || aligned <= 0)
+		return 0;
+
 	return aligned * cpp;
 }
 
@@ -842,8 +847,12 @@ int radeon_mode_dumb_create(struct drm_file *file_priv,
 
 	args->pitch = radeon_align_pitch(rdev, args->width,
 					 DIV_ROUND_UP(args->bpp, 8), 0);
+	if (!args->pitch)
+		return -EINVAL;
 	args->size = (u64)args->pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
+	if (!args->size)
+		return -EINVAL;
 
 	r = radeon_gem_object_create(rdev, args->size, 0,
 				     RADEON_GEM_DOMAIN_VRAM, 0,
Signature · 1 line
2.43.0
#8 lore raw

Re: [PATCH 2/2] drm/radeon: fix integer overflow in radeon_align_pitch()

Quoted 7 lines · Werner Kasselman

radeon_align_pitch() has the same integer overflow as amdgpu's variant:
'aligned * cpp' can overflow signed int to 0 when alignment rounding pushes the width past INT_MAX/cpp. This produces a 0-byte GEM buffer via radeon_mode_dumb_create(), reachable from unprivileged userspace via DRM_IOCTL_MODE_CREATE_DUMB on the render node.

Quoted 8 lines

Add an overflow check in radeon_align_pitch() and reject zero pitch/size in radeon_mode_dumb_create().

Found via AST-based call-graph analysis using sqry.

Fixes ff72145badb8 ("drm: dumb scanout create/mmap for intel/radeon (v3)")

Can you fix this up similar to the amdgpu patch?

Thanks,

Alex

Quoted 36 lines
---
 drivers/gpu/drm/radeon/radeon_gem.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/radeon/radeon_gem.c +9 -1
index 20fc87409f2e..2cd179fef347 100644
@@ -828,6 +828,11 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
 
        aligned += pitch_mask;
        aligned &= ~pitch_mask;
+
+       /* Guard against integer overflow in aligned * cpp. */
+       if (aligned > INT_MAX / (cpp ? cpp : 1) || aligned <= 0)
+               return 0;
+
        return aligned * cpp;
 }
 
@@ -842,8 +847,12 @@ int radeon_mode_dumb_create(struct drm_file *file_priv,
 
        args->pitch = radeon_align_pitch(rdev, args->width,
                                         DIV_ROUND_UP(args->bpp, 8), 0);
+       if (!args->pitch)
+               return -EINVAL;
        args->size = (u64)args->pitch * args->height;
        args->size = ALIGN(args->size, PAGE_SIZE);
+       if (!args->size)
+               return -EINVAL;
 
        r = radeon_gem_object_create(rdev, args->size, 0,
                                     RADEON_GEM_DOMAIN_VRAM, 0,
--

2.43.0

#9 lore raw

PATCH v2 1/1 drm/radeon: fix integer overflow in radeon_align_pitch()

radeon_align_pitch() has the same kind of overflow issue as the old amdgpu helper: the alignment round-up add and the final 'aligned * cpp' calculation can overflow signed int.

If that wraps to 0, radeon_mode_dumb_create() can end up with an invalid pitch value from DRM_IOCTL_MODE_CREATE_DUMB.

Fix this by using check_add_overflow() for the alignment round-up and check_mul_overflow() for the final pitch calculation, returning 0 on overflow.

Found via AST-based call-graph analysis using sqry.

Fixes ff72145badb8 ("drm: dumb scanout create/mmap for intel/radeon (v3)")

---
v2:
- Use overflow helpers like amdgpu.
- Drop the stale zero pitch/size change from the original submission.
- Fix the changelog wording around reachability.

 drivers/gpu/drm/radeon/radeon_gem.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +6 -6
index 2cd179fef347..8ce180e22d1d 100644
@@ -28,6 +28,7 @@
 
 #include <linux/debugfs.h>
 #include <linux/iosys-map.h>
+#include <linux/overflow.h>
 #include <linux/pci.h>
 
 #include <drm/drm_device.h>
@@ -812,6 +813,7 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
 	int aligned = width;
 	int align_large = (ASIC_IS_AVIVO(rdev)) || tiled;
 	int pitch_mask = 0;
+	int pitch;
 
 	switch (cpp) {
 	case 1:
@@ -826,14 +828,12 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
 		break;
 	}
 
-	aligned += pitch_mask;
+	if (check_add_overflow(aligned, pitch_mask, &aligned))
+		return 0;
 	aligned &= ~pitch_mask;
-
-	/* Guard against integer overflow in aligned * cpp. */
-	if (aligned > INT_MAX / (cpp ? cpp : 1) || aligned <= 0)
+	if (check_mul_overflow(aligned, cpp, &pitch))
 		return 0;
-
-	return aligned * cpp;
+	return pitch;
 }
 
 int radeon_mode_dumb_create(struct drm_file *file_priv,
Signature · 1 line
2.43.0
#10 lore raw

Re: [PATCH v2] drm/radeon: fix integer overflow in radeon_align_pitch()

Can you squash this with the previous radeon patch? I only applied the amdgpu patch at this point.

Alex

Quoted 67 lines · Werner Kasselman

radeon_align_pitch() has the same kind of overflow issue as the old amdgpu helper: the alignment round-up add and the final 'aligned * cpp' calculation can overflow signed int.

If that wraps to 0, radeon_mode_dumb_create() can end up with an invalid pitch value from DRM_IOCTL_MODE_CREATE_DUMB.

Fix this by using check_add_overflow() for the alignment round-up and check_mul_overflow() for the final pitch calculation, returning 0 on overflow.

Found via AST-based call-graph analysis using sqry.

Fixes ff72145badb8 ("drm: dumb scanout create/mmap for intel/radeon (v3)")

---
v2:
- Use overflow helpers like amdgpu.
- Drop the stale zero pitch/size change from the original submission.
- Fix the changelog wording around reachability.

 drivers/gpu/drm/radeon/radeon_gem.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +6 -7
index 2cd179fef347..8ce180e22d1d 100644
@@ -28,6 +28,7 @@
 
 #include <linux/debugfs.h>
 #include <linux/iosys-map.h>
+#include <linux/overflow.h>
 #include <linux/pci.h>
 
 #include <drm/drm_device.h>
@@ -812,6 +813,7 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
        int aligned = width;
        int align_large = (ASIC_IS_AVIVO(rdev)) || tiled;
        int pitch_mask = 0;
+       int pitch;
 
        switch (cpp) {
        case 1:
@@ -826,14 +828,12 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
                break;
        }
 
-       aligned += pitch_mask;
+       if (check_add_overflow(aligned, pitch_mask, &aligned))
+               return 0;
        aligned &= ~pitch_mask;
-
-       /* Guard against integer overflow in aligned * cpp. */
-       if (aligned > INT_MAX / (cpp ? cpp : 1) || aligned <= 0)
+       if (check_mul_overflow(aligned, cpp, &pitch))
                return 0;
-
-       return aligned * cpp;
+       return pitch;
 }
 
 int radeon_mode_dumb_create(struct drm_file *file_priv,
--

2.43.0

#11 lore raw

PATCH v3 1/1 drm/radeon: fix integer overflow in radeon_align_pitch()

radeon_align_pitch() has the same kind of overflow issue as the old amdgpu helper: both the alignment round-up add and the final 'aligned * cpp' calculation can overflow signed int.

If that wraps, radeon_mode_dumb_create() can end up returning an invalid pitch or creating a zero-sized dumb buffer.

Fix this by using check_add_overflow() for the alignment round-up and check_mul_overflow() for the final pitch calculation, returning 0 on overflow. Also reject zero pitch and size in radeon_mode_dumb_create().

Found via AST-based call-graph analysis using sqry.

Fixes ff72145badb8 ("drm: dumb scanout create/mmap for intel/radeon (v3)")

---
v3:
- Squash this fix with the earlier zero pitch/size validation change.
- Use overflow helpers for both the alignment round-up and final pitch calculation.

 drivers/gpu/drm/radeon/radeon_gem.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +11 -2
index 20fc87409f2e..8ce180e22d1d 100644
@@ -28,6 +28,7 @@
 
 #include <linux/debugfs.h>
 #include <linux/iosys-map.h>
+#include <linux/overflow.h>
 #include <linux/pci.h>
 
 #include <drm/drm_device.h>
@@ -812,6 +813,7 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
 	int aligned = width;
 	int align_large = (ASIC_IS_AVIVO(rdev)) || tiled;
 	int pitch_mask = 0;
+	int pitch;
 
 	switch (cpp) {
 	case 1:
@@ -826,9 +828,12 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
 		break;
 	}
 
-	aligned += pitch_mask;
+	if (check_add_overflow(aligned, pitch_mask, &aligned))
+		return 0;
 	aligned &= ~pitch_mask;
-	return aligned * cpp;
+	if (check_mul_overflow(aligned, cpp, &pitch))
+		return 0;
+	return pitch;
 }
 
 int radeon_mode_dumb_create(struct drm_file *file_priv,
@@ -842,8 +847,12 @@ int radeon_mode_dumb_create(struct drm_file *file_priv,
 
 	args->pitch = radeon_align_pitch(rdev, args->width,
 					 DIV_ROUND_UP(args->bpp, 8), 0);
+	if (!args->pitch)
+		return -EINVAL;
 	args->size = (u64)args->pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
+	if (!args->size)
+		return -EINVAL;
 
 	r = radeon_gem_object_create(rdev, args->size, 0,
 				     RADEON_GEM_DOMAIN_VRAM, 0,
Signature · 1 line
2.43.0
Applied
#12 lore raw

Re: [PATCH v3] drm/radeon: fix integer overflow in radeon_align_pitch()

Applied. Thanks!

Quoted 77 lines · Werner Kasselman

radeon_align_pitch() has the same kind of overflow issue as the old amdgpu helper: both the alignment round-up add and the final 'aligned * cpp' calculation can overflow signed int.

If that wraps, radeon_mode_dumb_create() can end up returning an invalid pitch or creating a zero-sized dumb buffer.

Fix this by using check_add_overflow() for the alignment round-up and check_mul_overflow() for the final pitch calculation, returning 0 on overflow. Also reject zero pitch and size in radeon_mode_dumb_create().

Found via AST-based call-graph analysis using sqry.

Fixes ff72145badb8 ("drm: dumb scanout create/mmap for intel/radeon (v3)")

---
v3:
- Squash this fix with the earlier zero pitch/size validation change.
- Use overflow helpers for both the alignment round-up and final pitch calculation.

 drivers/gpu/drm/radeon/radeon_gem.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +11 -3
index 20fc87409f2e..8ce180e22d1d 100644
@@ -28,6 +28,7 @@
 
 #include <linux/debugfs.h>
 #include <linux/iosys-map.h>
+#include <linux/overflow.h>
 #include <linux/pci.h>
 
 #include <drm/drm_device.h>
@@ -812,6 +813,7 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
        int aligned = width;
        int align_large = (ASIC_IS_AVIVO(rdev)) || tiled;
        int pitch_mask = 0;
+       int pitch;
 
        switch (cpp) {
        case 1:
@@ -826,9 +828,12 @@ int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tile
                break;
        }
 
-       aligned += pitch_mask;
+       if (check_add_overflow(aligned, pitch_mask, &aligned))
+               return 0;
        aligned &= ~pitch_mask;
-       return aligned * cpp;
+       if (check_mul_overflow(aligned, cpp, &pitch))
+               return 0;
+       return pitch;
 }
 
 int radeon_mode_dumb_create(struct drm_file *file_priv,
@@ -842,8 +847,12 @@ int radeon_mode_dumb_create(struct drm_file *file_priv,
 
        args->pitch = radeon_align_pitch(rdev, args->width,
                                         DIV_ROUND_UP(args->bpp, 8), 0);
+       if (!args->pitch)
+               return -EINVAL;
        args->size = (u64)args->pitch * args->height;
        args->size = ALIGN(args->size, PAGE_SIZE);
+       if (!args->size)
+               return -EINVAL;
 
        r = radeon_gem_object_create(rdev, args->size, 0,
                                     RADEON_GEM_DOMAIN_VRAM, 0,
--

2.43.0