Skip to contentSkip to conversation

media: meson: Add NULL check after the call to kmalloc()

PATCH In review Austin Kim 3 participants 5 messages Sep 4, 2019

Subsystems (4)
lore
Share & tools
lore

Austin Kim

author
#0 lore raw

PATCH 1/1 media: meson: Add NULL check after the call to kmalloc()

If the kmalloc() return NULL, the NULL pointer dereference will occur.

	new_ts->ts = ts;

Add exception check after the call to kmalloc() is made.

---
 drivers/staging/media/meson/vdec/vdec_helpers.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/staging/media/meson/vdec/vdec_helpers.c +4 -0
index f16948b..e7e56d5 100644
@@ -206,6 +206,10 @@ void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset)
 	unsigned long flags;
 
 	new_ts = kmalloc(sizeof(*new_ts), GFP_KERNEL);
+	if (!new_ts) {
+		dev_err(sess->core->dev, "Failed to kmalloc()\n");
+		return;
+	}
 	new_ts->ts = ts;
 	new_ts->offset = offset;
 
Signature · 1 line
2.6.2
#1 lore raw
Quoted 15 lines · Austin Kim

If the kmalloc() return NULL, the NULL pointer dereference will occur.

	new_ts->ts = ts;

Add exception check after the call to kmalloc() is made.

---
 drivers/staging/media/meson/vdec/vdec_helpers.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/staging/media/meson/vdec/vdec_helpers.c +0 -0
index f16948b..e7e56d5 100644

drivers/staging/media/meson/vdec/vdec_helpers.c void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset) · line 214

Quoted 9 lines
drivers/staging/media/meson/vdec/vdec_helpers.c +4 -0
@@ -206,6 +206,10 @@ void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset)
 	unsigned long flags;
 
 	new_ts = kmalloc(sizeof(*new_ts), GFP_KERNEL);
+	if (!new_ts) {
+		dev_err(sess->core->dev, "Failed to kmalloc()\n");
+		return;
+	}
 	new_ts->ts = ts;
 	new_ts->offset = offset;

No need to printk() on error, AFAIK the mm subsystem should already make some noise if an allocation fails.
This is not a proper fix - you need to make the function return an error (-ENOMEM) to let the caller know allocation failed, and allow that to propagate the error. There's only one caller, which already seems capable of returning errors (there's an -EAGAIN), so it shouldn't be that hard - though of course one needs to undo what has been done so far.

Also, unrelated to the kmalloc() handling: amvdec_add_ts_reorder() could be moved to esparser.c and made static, or at the very least the EXPORT_SYMBOL can be removed since vdec_helpers.o is linked in to the same module as the sole user. That probably goes for all the EXPORT_SYMBOL(amvdec_*).

Rasmus

author
#2 lore raw

2019년 9월 4일 (수) 오후 5:43, Rasmus Villemoes <[email protected]>님이 작성:

Quoted 19 lines · Austin Kim
Quoted 17 lines

If the kmalloc() return NULL, the NULL pointer dereference will occur.

      new_ts->ts = ts;

Add exception check after the call to kmalloc() is made.

---
 drivers/staging/media/meson/vdec/vdec_helpers.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/staging/media/meson/vdec/vdec_helpers.c +0 -0
index f16948b..e7e56d5 100644
@@ -206,6 +206,10 @@ void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset)
      unsigned long flags;
 
Quoted 10 lines

drivers/staging/media/meson/vdec/vdec_helpers.c void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset) · line 214

Quoted 7 lines
drivers/staging/media/meson/vdec/vdec_helpers.c +4 -0
      new_ts = kmalloc(sizeof(*new_ts), GFP_KERNEL);
+     if (!new_ts) {
+             dev_err(sess->core->dev, "Failed to kmalloc()\n");
+             return;
+     }
      new_ts->ts = ts;
      new_ts->offset = offset;

No need to printk() on error, AFAIK the mm subsystem should already make some noise if an allocation fails.

Thanks for valuable feedback.
BTW, if the kmalloc return NULL, mm subsystem throws error log with 100%? If no, please share error signature of kernel.

Quoted 11 lines

This is not a proper fix - you need to make the function return an error (-ENOMEM) to let the caller know allocation failed, and allow that to propagate the error. There's only one caller, which already seems capable of returning errors (there's an -EAGAIN), so it shouldn't be that hard - though of course one needs to undo what has been done so far.

Also, unrelated to the kmalloc() handling: amvdec_add_ts_reorder() could be moved to esparser.c and made static, or at the very least the EXPORT_SYMBOL can be removed since vdec_helpers.o is linked in to the same module as the sole user. That probably goes for all the EXPORT_SYMBOL(amvdec_*).

I agreed with your feedback.
- On memory allocation failure, it should have returned (-ENOMEM) rather than 'return' statement.
- The call to amvdec_add_ts_reorder() is only made by esparser_queue().

Let me resend the patch if necessary.

Thanks,
Austin Kim

Quoted 2 lines

Rasmus

maintainer
#3 lore raw
In reply to Austin Kim
Quoted 15 lines · Austin Kim

If the kmalloc() return NULL, the NULL pointer dereference will occur.

	new_ts->ts = ts;

Add exception check after the call to kmalloc() is made.

---
 drivers/staging/media/meson/vdec/vdec_helpers.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/staging/media/meson/vdec/vdec_helpers.c +0 -0
index f16948b..e7e56d5 100644

drivers/staging/media/meson/vdec/vdec_helpers.c void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset) · line 210

Quoted 5 lines
drivers/staging/media/meson/vdec/vdec_helpers.c +2 -0
@@ -206,6 +206,10 @@ void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset)
 	unsigned long flags;
 
 	new_ts = kmalloc(sizeof(*new_ts), GFP_KERNEL);
+	if (!new_ts) {
+		dev_err(sess->core->dev, "Failed to kmalloc()\n");

Did you run this through checkpatch? I think it will say that this line is not ok.

drivers/staging/media/meson/vdec/vdec_helpers.c void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset) · line 211

Quoted 1 line
drivers/staging/media/meson/vdec/vdec_helpers.c +1 -0
+		return;

Shouldn't you return an -ENOMEM error somehow?

thanks,

greg k-h

author
#4 lore raw

2019년 9월 4일 (수) 오후 5:45, Greg KH <[email protected]>님이 작성:

Quoted 26 lines · Austin Kim
Quoted 15 lines

If the kmalloc() return NULL, the NULL pointer dereference will occur.

      new_ts->ts = ts;

Add exception check after the call to kmalloc() is made.

---
 drivers/staging/media/meson/vdec/vdec_helpers.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/staging/media/meson/vdec/vdec_helpers.c +0 -0
index f16948b..e7e56d5 100644

drivers/staging/media/meson/vdec/vdec_helpers.c void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset) · line 210

Quoted 5 lines
drivers/staging/media/meson/vdec/vdec_helpers.c +2 -0
@@ -206,6 +206,10 @@ void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset)
      unsigned long flags;
 
      new_ts = kmalloc(sizeof(*new_ts), GFP_KERNEL);
+     if (!new_ts) {
+             dev_err(sess->core->dev, "Failed to kmalloc()\n");

Did you run this through checkpatch? I think it will say that this line is not ok.

Quoted 3 lines

drivers/staging/media/meson/vdec/vdec_helpers.c void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset) · line 211

Quoted 1 line
drivers/staging/media/meson/vdec/vdec_helpers.c +1 -0
+             return;

Shouldn't you return an -ENOMEM error somehow?

I agreed with your feedback.
Let me take a look at the code more and then resend the patch if necessary.

Thanks,
Austin Kim

Quoted 4 lines

thanks,

greg k-h