Skip to contentSkip to conversation

staging: atomisp: fix heap buffer overflow in framebuffer conversion

PATCH Superseded Soufiane 6 participants 14 messages Feb 10 – Feb 27 2 messages not in the archive

Subsystems (4)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Soufiane

author
#0 lore raw

PATCH 1/1 staging: atomisp: fix heap buffer overflow in framebuffer conversion

Validate sizeimage against the allocated frame buffer size before hmm_store() to prevent out-of-bounds write.

---
 drivers/staging/media/atomisp/pci/atomisp_cmd.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/staging/media/atomisp/pci/atomisp_cmd.c +5 -0
index 3a4eb4f6d3be..ca7ffc7855ac 100644
@@ -3326,6 +3326,11 @@ atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg,
 		goto err;
 	}
 
+	if (arg->fmt.sizeimage > res->data_bytes) {
+		ret = -EINVAL;
+		goto err;
+	}
+
 	tmp_buf = vmalloc(arg->fmt.sizeimage);
 	if (!tmp_buf) {
 		ret = -ENOMEM;

---
base-commit: 72c395024dac5e215136cbff793455f065603b06
change-id: 20260210-atomisp-fix-8e083f753688

Best regards,

Signature · 1 line
Soufiane <[email protected]>
maintainer
#1 lore raw
Quoted 31 lines · On Tue, Feb 10, 2026 at 04:26:31PM +0100, Soufiane via B4 Relay wrote:

From: Soufiane <[email protected]>

Validate sizeimage against the allocated frame buffer size before hmm_store() to prevent out-of-bounds write.

---
 drivers/staging/media/atomisp/pci/atomisp_cmd.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/staging/media/atomisp/pci/atomisp_cmd.c +5 -1
index 3a4eb4f6d3be..ca7ffc7855ac 100644
@@ -3326,6 +3326,11 @@ atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg,
 		goto err;
 	}
 
+	if (arg->fmt.sizeimage > res->data_bytes) {
+		ret = -EINVAL;
+		goto err;
+	}
+
 	tmp_buf = vmalloc(arg->fmt.sizeimage);
 	if (!tmp_buf) {
 		ret = -ENOMEM;
 
---

base-commit: 72c395024dac5e215136cbff793455f065603b06
change-id: 20260210-atomisp-fix-8e083f753688

Quoted 6 lines

Best regards,

Signature · 1 line
Soufiane <[email protected]>

Hi,

This is the friendly patch-bot of Greg Kroah-Hartman. You have sent him a patch that has triggered this response. He used to manually respond to these common problems, but in order to save his sanity (he kept writing the same thing over and over, yet to different people), I was created. Hopefully you will not take offence and will fix the problem in your patch and resubmit it so that it can be accepted into the Linux kernel tree.

You are receiving this message because of the following common error(s) as indicated below:

- It looks like you did not use your "real" name for the patch on either the Signed-off-by: line, or the From: line (both of which have to match). Please read the kernel file, Documentation/process/submitting-patches.rst for how to do this correctly.

If you wish to discuss this problem further, or you have questions about how to resolve this issue, please feel free to respond to this email and Greg will reply once he has dug out from the pending patches received from other developers.

thanks,

greg k-h's patch email bot

#2 lore raw
In reply to Soufiane
Quoted 6 lines · On Tue, Feb 10, 2026 at 04:26:31PM +0100, Soufiane via B4 Relay wrote:

From: Soufiane <[email protected]>

Validate sizeimage against the allocated frame buffer size before hmm_store() to prevent out-of-bounds write.

We need a Fixes tag if the bug is real.

drivers/staging/media/atomisp/pci/atomisp_cmd.c atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg, · line 3328

Quoted 12 lines
---
 drivers/staging/media/atomisp/pci/atomisp_cmd.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/staging/media/atomisp/pci/atomisp_cmd.c +0 -0
index 3a4eb4f6d3be..ca7ffc7855ac 100644
@@ -3326,6 +3326,11 @@ atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg,
 		goto err;
 	}
 

There is some sketchy stuff happening in this code but I'm not sure I understand the issue. The code looks like this:

3317 /* Note: the padded width on an ia_css_frame is in elements, not in
3318 bytes. The RAW frame we use here should always be a 16bit RAW
3319 frame. This is why we bytesperline/2 is equal to the padded with */
3320 if (ia_css_frame_allocate(&res, arg->fmt.width, arg->fmt.height,
3321 sh_format, padded_width, 0)) {

This allocates res. Why would it allocate something smaller than arg->fmt.sizeimage? How did you find this bug? By testing or reading the code? Do you have a reproducer?

  3322                  ret = -ENOMEM;
  3323                  goto err;
  3324          }

drivers/staging/media/atomisp/pci/atomisp_cmd.c atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg, · line 3333

Quoted 5 lines
drivers/staging/media/atomisp/pci/atomisp_cmd.c +5 -0
+	if (arg->fmt.sizeimage > res->data_bytes) {
+		ret = -EINVAL;
+		goto err;
+	}
+

3325
3326 tmp_buf = vmalloc(arg->fmt.sizeimage);
3327 if (!tmp_buf) {
3328 ret = -ENOMEM;
3329 goto err;
3330 }
3331 if (copy_from_user(tmp_buf, (void __user __force *)arg->base,
3332 arg->fmt.sizeimage)) {
3333 ret = -EFAULT;
3334 goto err;
3335 }
3336

  3337          if (hmm_store(res->data, tmp_buf, arg->fmt.sizeimage)) {
                              ^^^^^^^^^

The worry is that the buffer this references is too small. I would prefer instead if there were some bounds checking before the memcpy() calls in hmm_store(). They would use a different, smaller limit if only part of the buffer could be used. I don't know if that bounds checking is really required though...

  3338                  ret = -EINVAL;
  3339                  goto err;
  3340          }
  3341  

regards,
dan carpenter

reviewer
#3 lore raw

Hi Dan, Soufiane,

Quoted 58 lines · Dan Carpenter
Quoted 6 lines · On Tue, Feb 10, 2026 at 04:26:31PM +0100, Soufiane via B4 Relay wrote:

From: Soufiane <[email protected]>

Validate sizeimage against the allocated frame buffer size before hmm_store() to prevent out-of-bounds write.

We need a Fixes tag if the bug is real.

drivers/staging/media/atomisp/pci/atomisp_cmd.c atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg, · line 3328

Quoted 12 lines
---
 drivers/staging/media/atomisp/pci/atomisp_cmd.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/staging/media/atomisp/pci/atomisp_cmd.c +0 -0
index 3a4eb4f6d3be..ca7ffc7855ac 100644
@@ -3326,6 +3326,11 @@ atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg,
 		goto err;
 	}
 

There is some sketchy stuff happening in this code but I'm not sure I understand the issue. The code looks like this:

3317 /* Note: the padded width on an ia_css_frame is in elements, not in
3318 bytes. The RAW frame we use here should always be a 16bit RAW
3319 frame. This is why we bytesperline/2 is equal to the padded with */
3320 if (ia_css_frame_allocate(&res, arg->fmt.width, arg->fmt.height,
3321 sh_format, padded_width, 0)) {

This allocates res. Why would it allocate something smaller than arg->fmt.sizeimage? How did you find this bug? By testing or reading the code? Do you have a reproducer?

  3322                  ret = -ENOMEM;
  3323                  goto err;
  3324          }

drivers/staging/media/atomisp/pci/atomisp_cmd.c atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg, · line 3333

Quoted 5 lines
drivers/staging/media/atomisp/pci/atomisp_cmd.c +5 -0
+	if (arg->fmt.sizeimage > res->data_bytes) {
+		ret = -EINVAL;
+		goto err;
+	}
+

3325
3326 tmp_buf = vmalloc(arg->fmt.sizeimage);
3327 if (!tmp_buf) {
3328 ret = -ENOMEM;
3329 goto err;
3330 }
3331 if (copy_from_user(tmp_buf, (void __user __force *)arg->base,
3332 arg->fmt.sizeimage)) {
3333 ret = -EFAULT;
3334 goto err;
3335 }
3336

Quoted 7 lines
  3337          if (hmm_store(res->data, tmp_buf, arg->fmt.sizeimage)) {
                              ^^^^^^^^^

The worry is that the buffer this references is too small. I would prefer instead if there were some bounds checking before the memcpy() calls in hmm_store(). They would use a different, smaller limit if only part of the buffer could be used. I don't know if that bounds checking is really required though...

Indeed. Beyond that, even I have to admit I have little idea what this IOCTL is supposed to be doing. Possibly feed in a raw frame for processing? But that's not supposed to be implemented like this... The TODO file contains an entry that says "Remove/disable private IOCTLs" -- we should move to use parameter buffers instead.

I'm not sure anyone depends on these IOCTLs at the moment, but definitely some obviously are associated with some risk.

The world looked different when this code was written.

I'd disable all private IOCTLs in the driver, with the possible exception of ATOMISP_IOC_S_ISP_PARM, which is close to the parameter buffer approach already.

Also cc LMML, Greg and Andy.

Signature · 3 lines
Kind regards,

Sakari Ailus
#4 lore raw
Quoted 1 line · Sakari Ailus

On Tue, Feb 10, 2026 at 09:53:50PM +0300, Dan Carpenter wrote:

...

Quoted 6 lines

Beyond that, even I have to admit I have little idea what this IOCTL is supposed to be doing. Possibly feed in a raw frame for processing? But that's not supposed to be implemented like this... The TODO file contains an entry that says "Remove/disable private IOCTLs" -- we should move to use parameter buffers instead.

Quoted 8 lines

I'm not sure anyone depends on these IOCTLs at the moment, but definitely some obviously are associated with some risk.

The world looked different when this code was written.

I'd disable all private IOCTLs in the driver, with the possible exception of ATOMISP_IOC_S_ISP_PARM, which is close to the parameter buffer approach already.

I agree with a caveat (see below).

Quoted 1 line

Also cc LMML, Greg and Andy.

Perhaps also ask Hans or other people from libcamera? What does that use when it comes to AtomISP?

Signature · 2 lines
With Best Regards,
Andy Shevchenko
#5 lore raw
In reply to Sakari Ailus

Hi,

Quoted 75 lines · Sakari Ailus

Hi Dan, Soufiane,

Quoted 58 lines · Dan Carpenter
Quoted 6 lines · On Tue, Feb 10, 2026 at 04:26:31PM +0100, Soufiane via B4 Relay wrote:

From: Soufiane <[email protected]>

Validate sizeimage against the allocated frame buffer size before hmm_store() to prevent out-of-bounds write.

We need a Fixes tag if the bug is real.

drivers/staging/media/atomisp/pci/atomisp_cmd.c atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg, · line 3328

Quoted 12 lines
---
 drivers/staging/media/atomisp/pci/atomisp_cmd.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/staging/media/atomisp/pci/atomisp_cmd.c +0 -0
index 3a4eb4f6d3be..ca7ffc7855ac 100644
@@ -3326,6 +3326,11 @@ atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg,
 		goto err;
 	}
 

There is some sketchy stuff happening in this code but I'm not sure I understand the issue. The code looks like this:

3317 /* Note: the padded width on an ia_css_frame is in elements, not in
3318 bytes. The RAW frame we use here should always be a 16bit RAW
3319 frame. This is why we bytesperline/2 is equal to the padded with */
3320 if (ia_css_frame_allocate(&res, arg->fmt.width, arg->fmt.height,
3321 sh_format, padded_width, 0)) {

This allocates res. Why would it allocate something smaller than arg->fmt.sizeimage? How did you find this bug? By testing or reading the code? Do you have a reproducer?

  3322                  ret = -ENOMEM;
  3323                  goto err;
  3324          }

drivers/staging/media/atomisp/pci/atomisp_cmd.c atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg, · line 3333

Quoted 5 lines
drivers/staging/media/atomisp/pci/atomisp_cmd.c +5 -0
+	if (arg->fmt.sizeimage > res->data_bytes) {
+		ret = -EINVAL;
+		goto err;
+	}
+

3325
3326 tmp_buf = vmalloc(arg->fmt.sizeimage);
3327 if (!tmp_buf) {
3328 ret = -ENOMEM;
3329 goto err;
3330 }
3331 if (copy_from_user(tmp_buf, (void __user __force *)arg->base,
3332 arg->fmt.sizeimage)) {
3333 ret = -EFAULT;
3334 goto err;
3335 }
3336

Quoted 7 lines
  3337          if (hmm_store(res->data, tmp_buf, arg->fmt.sizeimage)) {
                              ^^^^^^^^^

The worry is that the buffer this references is too small. I would prefer instead if there were some bounds checking before the memcpy() calls in hmm_store(). They would use a different, smaller limit if only part of the buffer could be used. I don't know if that bounds checking is really required though...

Indeed. Beyond that, even I have to admit I have little idea what this IOCTL is supposed to be doing. Possibly feed in a raw frame for processing? But that's not supposed to be implemented like this... The TODO file contains an entry that says "Remove/disable private IOCTLs" -- we should move to use parameter buffers instead.

Quoted 8 lines

I'm not sure anyone depends on these IOCTLs at the moment, but definitely some obviously are associated with some risk.

The world looked different when this code was written.

I'd disable all private IOCTLs in the driver, with the possible exception of ATOMISP_IOC_S_ISP_PARM, which is close to the parameter buffer approach already.

Ack, my suggestion would be to completely remove the entire atomisp_vidioc_default() function from:

drivers/staging/media/atomisp/pci/atomisp_ioctl.c

(and stop setting .vidioc_default)

as mentioned this is something on the TODO list anyways.

This will cause a bunch of code to turn into dead code, but I would like to keep that code around since when we add support for a parameter buffer queue that code can serve as an example how to send parameters to the ISP.

Regards,

Hans

#6 lore raw
Quoted 1 line · On Wed, Feb 11, 2026 at 12:28:35PM +0100, [email protected] wrote:

On 11-Feb-26 09:11, Sakari Ailus wrote:

...

Quoted 4 lines

This will cause a bunch of code to turn into dead code, but I would like to keep that code around since when we add support for a parameter buffer queue that code can serve as an example how to send parameters to the ISP.

But it's forever in the Git index, we can remove it, so it's just matter of convenience to keep it in a working copy (tree). That being said, I would rather drop the dead code to avoid a stream of not-so-useful white space, style, and similar cleanups.

Signature · 2 lines
With Best Regards,
Andy Shevchenko
#7 lore raw

Hi,

Quoted 10 lines · Andy Shevchenko
Quoted 1 line · On Wed, Feb 11, 2026 at 12:28:35PM +0100, [email protected] wrote:

On 11-Feb-26 09:11, Sakari Ailus wrote:

...

Quoted 4 lines

This will cause a bunch of code to turn into dead code, but I would like to keep that code around since when we add support for a parameter buffer queue that code can serve as an example how to send parameters to the ISP.

Quoted 4 lines

But it's forever in the Git index, we can remove it, so it's just matter of convenience to keep it in a working copy (tree). That being said, I would rather drop the dead code to avoid a stream of not-so-useful white space, style, and similar cleanups.

That is a good point, dropping some of the dead-code stemming from this is fine with me.

We should probably stop pruning dead code when we get deep into the helpers to pack things into fw specific formats.

Regards,

Hans

reviewer
#8 lore raw

Hi Hans, Andy,

Quoted 18 lines · On Wed, Feb 11, 2026 at 12:50:18PM +0100, [email protected] wrote:

Hi,

Quoted 10 lines · Andy Shevchenko
Quoted 1 line · On Wed, Feb 11, 2026 at 12:28:35PM +0100, [email protected] wrote:

On 11-Feb-26 09:11, Sakari Ailus wrote:

...

Quoted 4 lines

This will cause a bunch of code to turn into dead code, but I would like to keep that code around since when we add support for a parameter buffer queue that code can serve as an example how to send parameters to the ISP.

Quoted 4 lines

But it's forever in the Git index, we can remove it, so it's just matter of convenience to keep it in a working copy (tree). That being said, I would rather drop the dead code to avoid a stream of not-so-useful white space, style, and similar cleanups.

Quoted 6 lines

That is a good point, dropping some of the dead-code stemming from this is fine with me.

We should probably stop pruning dead code when we get deep into the helpers to pack things into fw specific formats.

Either works for me, however the actual IOCTL handling related code contains less redundancy than the rest of the driver. When it comes to this patch, I'd keep the changes small allow easy backporting.

Signature · 1 line
Sakari Ailus
#9 lore raw

Hi,

Quoted 24 lines · Sakari Ailus

Hi Hans, Andy,

Quoted 21 lines · On Wed, Feb 11, 2026 at 12:50:18PM +0100, [email protected] wrote:

Hi,

Quoted 10 lines · Andy Shevchenko
Quoted 1 line · On Wed, Feb 11, 2026 at 12:28:35PM +0100, [email protected] wrote:

On 11-Feb-26 09:11, Sakari Ailus wrote:

...

Quoted 4 lines

This will cause a bunch of code to turn into dead code, but I would like to keep that code around since when we add support for a parameter buffer queue that code can serve as an example how to send parameters to the ISP.

Quoted 4 lines

But it's forever in the Git index, we can remove it, so it's just matter of convenience to keep it in a working copy (tree). That being said, I would rather drop the dead code to avoid a stream of not-so-useful white space, style, and similar cleanups.

That is a good point, dropping some of the dead-code stemming from this is fine with me.

Quoted 7 lines
Quoted 3 lines

We should probably stop pruning dead code when we get deep into the helpers to pack things into fw specific formats.

Either works for me, however the actual IOCTL handling related code contains less redundancy than the rest of the driver. When it comes to this patch, I'd keep the changes small allow easy backporting.

Ack, as said we can start with a patch just dropping the default ioctl handler. That should be easy to backport.

Removing some of the then unused functions can be done as followup patches.

Regards,

Hans

#10 lore raw
Quoted 5 lines · On Wed, Feb 11, 2026 at 01:31:19PM +0100, [email protected] wrote:
Quoted 4 lines · Sakari Ailus
Quoted 3 lines · On Wed, Feb 11, 2026 at 12:50:18PM +0100, [email protected] wrote:
Quoted 2 lines · Andy Shevchenko
Quoted 1 line · On Wed, Feb 11, 2026 at 12:28:35PM +0100, [email protected] wrote:

On 11-Feb-26 09:11, Sakari Ailus wrote:

...

Quoted 21 lines
Quoted 13 lines
Quoted 13 lines
Quoted 9 lines
Quoted 4 lines

This will cause a bunch of code to turn into dead code, but I would like to keep that code around since when we add support for a parameter buffer queue that code can serve as an example how to send parameters to the ISP.

But it's forever in the Git index, we can remove it, so it's just matter of convenience to keep it in a working copy (tree). That being said, I would rather drop the dead code to avoid a stream of not-so-useful white space, style, and similar cleanups.

That is a good point, dropping some of the dead-code stemming from this is fine with me.

Quoted 7 lines
Quoted 3 lines

We should probably stop pruning dead code when we get deep into the helpers to pack things into fw specific formats.

Either works for me, however the actual IOCTL handling related code contains less redundancy than the rest of the driver. When it comes to this patch, I'd keep the changes small allow easy backporting.

Quoted 5 lines

Ack, as said we can start with a patch just dropping the default ioctl handler. That should be easy to backport.

Removing some of the then unused functions can be done as followup patches.

I am on the same page.

Signature · 2 lines
With Best Regards,
Andy Shevchenko
#11 lore raw
In reply to Sakari Ailus

Hi Sakari,

I agree that removing the private IOCTL handler is the better approach. While fuzzing the driver I found the same class of unchecked user-controlled size fields in several other handlers (ATOMISP_IOC_S_DIS_VECTOR, morph table, shading table), so removing atomisp_vidioc_default() eliminates all of them at once.

I'm cool with sending a patch removing atomisp_vidioc_default() as Hans suggested, if that would be helpful.

Regards,
Soufiane Dani

11 Feb 2026 at 09:11 by [email protected]:

Quoted 93 lines

Hi Dan, Soufiane,

Quoted 57 lines · Dan Carpenter
Quoted 6 lines · On Tue, Feb 10, 2026 at 04:26:31PM +0100, Soufiane via B4 Relay wrote:

From: Soufiane <[email protected]>

Validate sizeimage against the allocated frame buffer size before hmm_store() to prevent out-of-bounds write.

We need a Fixes tag if the bug is real.

drivers/staging/media/atomisp/pci/atomisp_cmd.c atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg, · line 3328

Quoted 12 lines
---
 drivers/staging/media/atomisp/pci/atomisp_cmd.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/staging/media/atomisp/pci/atomisp_cmd.c +0 -0
index 3a4eb4f6d3be..ca7ffc7855ac 100644
@@ -3326,6 +3326,11 @@ atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg,
 		goto err;
 	}
 

There is some sketchy stuff happening in this code but I'm not sure I understand the issue. The code looks like this:

3317 /* Note: the padded width on an ia_css_frame is in elements, not in
3318 bytes. The RAW frame we use here should always be a 16bit RAW
3319 frame. This is why we bytesperline/2 is equal to the padded with */
3320 if (ia_css_frame_allocate(&res, arg->fmt.width, arg->fmt.height,
3321 sh_format, padded_width, 0)) {

This allocates res. Why would it allocate something smaller than arg->fmt.sizeimage? How did you find this bug? By testing or reading the code? Do you have a reproducer?

3322 ret = -ENOMEM;
3323 goto err;
3324 }

drivers/staging/media/atomisp/pci/atomisp_cmd.c atomisp_v4l2_framebuffer_to_css_frame(const struct v4l2_framebuffer *arg, · line 3333

Quoted 5 lines
drivers/staging/media/atomisp/pci/atomisp_cmd.c +5 -0
+	if (arg->fmt.sizeimage > res->data_bytes) {
+		ret = -EINVAL;
+		goto err;
+	}
+

3325
3326 tmp_buf = vmalloc(arg->fmt.sizeimage);
3327 if (!tmp_buf) {
3328 ret = -ENOMEM;
3329 goto err;
3330 }
3331 if (copy_from_user(tmp_buf, (void __user __force *)arg->base,
3332 arg->fmt.sizeimage)) {
3333 ret = -EFAULT;
3334 goto err;
3335 }

Quoted 9 lines

3336
3337 if (hmm_store(res->data, tmp_buf, arg->fmt.sizeimage)) {
^^^^^^^^^
The worry is that the buffer this references is too small. I would prefer instead if there were some bounds checking before the memcpy() calls in hmm_store(). They would use a different, smaller limit if only part of the buffer could be used. I don't know if that bounds checking is really required though...

Indeed. Beyond that, even I have to admit I have little idea what this IOCTL is supposed to be doing. Possibly feed in a raw frame for processing? But that's not supposed to be implemented like this... The TODO file contains an entry that says "Remove/disable private IOCTLs" -- we should move to use parameter buffers instead.

I'm not sure anyone depends on these IOCTLs at the moment, but definitely some obviously are associated with some risk.

The world looked different when this code was written.

I'd disable all private IOCTLs in the driver, with the possible exception of ATOMISP_IOC_S_ISP_PARM, which is close to the parameter buffer approach already.

Also cc LMML, Greg and Andy.

Signature · 3 lines
Kind regards,

Sakari Ailus
reviewer
#12 lore raw

Hi Soufiane,

Quoted 11 lines · On Wed, Feb 11, 2026 at 02:43:17PM +0100, [email protected] wrote:

Hi Sakari,

I agree that removing the private IOCTL handler is the better approach. While fuzzing the driver I found the same class of unchecked user-controlled size fields in several other handlers (ATOMISP_IOC_S_DIS_VECTOR, morph table, shading table), so removing atomisp_vidioc_default() eliminates all of them at once.

I'm cool with sending a patch removing atomisp_vidioc_default() as Hans suggested, if that would be helpful.

Oops. I read your message after posting the patch...

Indeed it sounds like we should disable all private IOCTLs, also the only one that should have been there to begin with. (I can update my patch as well.)

Signature · 3 lines
Kind regards,

Sakari Ailus

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#15 lore raw

Hi Dan,

The issue is that res->data_bytes and arg->fmt.sizeimage are computed from independent sources with no validation linking them.

ia_css_frame_allocate() computes data_bytes internally based on width, height, format, and padded_width through frame_init_planes():

  frame_allocate_with_data()
    -> frame_create(width, height, ...)
    -> ia_css_frame_init_planes()
       -> frame_init_single_plane() / frame_init_nv_planes() / ... -> frame->data_bytes = stride * height  (varies by format) -> frame_allocate_buffer_data()
       -> hmm_alloc(frame->data_bytes)

But arg->fmt.sizeimage is a separate user-controlled field in struct v4l2_framebuffer. Nothing enforces that sizeimage matches the data_bytes computed from width/height/format. A user can pass:

  width=100, height=100  -> small data_bytes allocation sizeimage=1048576      -> 1MB copy via hmm_store()

The hmm_store() then does memcpy() with the sizeimage length into the data_bytes-sized buffer.

I found this by code review, then confirmed with a userspace harness compiled with AFL++/ASAN that simulates the allocation and copy. The ASAN output shows heap-buffer-overflow immediately with mismatched values.

The ioctl path is:

  ioctl(fd, ATOMISP_IOC_S_ISP_FPN_TABLE, &fb)
    -> atomisp_fixed_pattern_table()
    -> atomisp_v4l2_framebuffer_to_css_frame()

Regarding your suggestion about bounds checking in hmm_store() - that would also work, but hmm_store() is a generic function used elsewhere. Validating at the call site before we even vmalloc the oversized tmp_buf seems cleaner and catches it earlier.

Regards,
Soufiane Dani

11 Feb 2026 at 06:35 by [email protected]:

Quoted 9 lines

Please send this email to the list.

This information is not secret and should be included in the commit message.

regards,
dan carpenter