Skip to contentSkip to conversation

consolemap: Fix a memory leaking bug in drivers/tty/vt/consolemap.c

PATCH Superseded Gen Zhang 4 participants 12 messages May 21, 2019 – May 25, 2019 TTY layer and serial drivers

lore
Share & tools
lore

Gen Zhang

author
#0 lore raw

PATCH 1/1 consolemap: Fix a memory leaking bug in drivers/tty/vt/consolemap.c

In function con_insert_unipair(), when allocation for p2 and p1[n] fails, ENOMEM is returned, but previously allocated p1 is not freed, remains as leaking memory. Thus we should free p1 as well when this allocation fails.

---

drivers/tty/vt/consolemap.c +4 -1
index b28aa0d..47fbd73 100644
@@ -489,7 +489,10 @@ con_insert_unipair(struct uni_pagedir *p, u_short unicode, u_short fontpos)
 	p2 = p1[n = (unicode >> 6) & 0x1f];
 	if (!p2) {
 		p2 = p1[n] = kmalloc_array(64, sizeof(u16), GFP_KERNEL);
-		if (!p2) return -ENOMEM;
+		if (!p2) {
+			kfree(p1);
+			return -ENOMEM;
+		}
 		memset(p2, 0xff, 64*sizeof(u16)); /* No glyphs for the characters (yet) */
 	}
 
#1 lore raw
Quoted 13 lines · Gen Zhang

In function con_insert_unipair(), when allocation for p2 and p1[n] fails, ENOMEM is returned, but previously allocated p1 is not freed, remains as leaking memory. Thus we should free p1 as well when this allocation fails.

---

drivers/tty/vt/consolemap.c +0 -0
index b28aa0d..47fbd73 100644

drivers/tty/vt/consolemap.c con_insert_unipair(struct uni_pagedir *p, u_short unicode, u_short fontpos) · line 495

Quoted 8 lines
drivers/tty/vt/consolemap.c +4 -1
@@ -489,7 +489,10 @@ con_insert_unipair(struct uni_pagedir *p, u_short unicode, u_short fontpos)
 	p2 = p1[n = (unicode >> 6) & 0x1f];
 	if (!p2) {
 		p2 = p1[n] = kmalloc_array(64, sizeof(u16), GFP_KERNEL);
-		if (!p2) return -ENOMEM;
+		if (!p2) {
+			kfree(p1);
+			return -ENOMEM;
+		}

This doesn't look safe to me: p->uni_pgdir[n] will still have a handle to the freed memory, won't it?

(And please direct these patches to Greg, as he's the current maintainer; I'm happy to stay CCed, of course.)

-Kees

Quoted 3 lines
drivers/tty/vt/consolemap.c +0 -0
 		memset(p2, 0xff, 64*sizeof(u16)); /* No glyphs for the characters (yet) */
 	}
 
Signature · 1 line
Kees Cook
author
#2 lore raw
Quoted 3 lines · Kees Cook

This doesn't look safe to me: p->uni_pgdir[n] will still have a handle to the freed memory, won't it?

Thanks for your reply, Kees!
I think you are right. Maybe we should do this:

	kfree(p1);
	p->uni_pgdir[n] = NULL;

Is this correct?

Quoted 3 lines

(And please direct these patches to Greg, as he's the current maintainer; I'm happy to stay CCed, of course.)

I will follow your suggestions, thanks!
Gen

Quoted 8 lines

-Kees

Quoted 3 lines
drivers/tty/vt/consolemap.c +0 -0
 		memset(p2, 0xff, 64*sizeof(u16)); /* No glyphs for the characters (yet) */
 	}
Signature · 1 line
Kees Cook
#3 lore raw
Quoted 9 lines · Gen Zhang
Quoted 3 lines · Kees Cook

This doesn't look safe to me: p->uni_pgdir[n] will still have a handle to the freed memory, won't it?

Thanks for your reply, Kees!
I think you are right. Maybe we should do this:

	kfree(p1);
	p->uni_pgdir[n] = NULL;

Is this correct?

That's what I'm not sure about. I *think* so, from reading the code, but I'd love to have Greg (or someone more familiar with the code) to double-check this.

Otherwise, yeah, this looks right. Please send a v2 and we can debate the correctness there, if it turns out to be wrong. :)

Signature · 1 line
Kees Cook
author
#4 lore raw
Quoted 11 lines · Kees Cook
Quoted 9 lines · Gen Zhang
Quoted 3 lines · Kees Cook

This doesn't look safe to me: p->uni_pgdir[n] will still have a handle to the freed memory, won't it?

Thanks for your reply, Kees!
I think you are right. Maybe we should do this:

	kfree(p1);
	p->uni_pgdir[n] = NULL;

Is this correct?

Quoted 9 lines

That's what I'm not sure about. I *think* so, from reading the code, but I'd love to have Greg (or someone more familiar with the code) to double-check this.

Otherwise, yeah, this looks right. Please send a v2 and we can debate the correctness there, if it turns out to be wrong. :)

Signature · 1 line
Kees Cook

Thanks for your suggestions, Kees.
I follow your guidance and work on resubmitting a patch. Thanks
Gen

author
#5 lore raw

PATCH v2 1/1 consolemap: Fix a memory leaking bug in drivers/tty/vt/consolemap.c

In reply to Kees Cook

In function con_insert_unipair(), when allocation for p2 and p1[n] fails, ENOMEM is returned, but previously allocated p1 is not freed, remains as leaking memory. Thus we should free p1 as well when this allocation fails.

---

drivers/tty/vt/consolemap.c +5 -1
index b28aa0d..79fcc96 100644
@@ -489,7 +489,11 @@ con_insert_unipair(struct uni_pagedir *p, u_short unicode, u_short fontpos)
 	p2 = p1[n = (unicode >> 6) & 0x1f];
 	if (!p2) {
 		p2 = p1[n] = kmalloc_array(64, sizeof(u16), GFP_KERNEL);
-		if (!p2) return -ENOMEM;
+		if (!p2) {
+			kfree(p1);
+			p->uni_pgdir[n] = NULL;
+			return -ENOMEM;
+		}
 		memset(p2, 0xff, 64*sizeof(u16)); /* No glyphs for the characters (yet) */
 	}
 

---

Reviewed-by on v2 1/1
#6 lore raw
Quoted 6 lines · Gen Zhang

In function con_insert_unipair(), when allocation for p2 and p1[n] fails, ENOMEM is returned, but previously allocated p1 is not freed, remains as leaking memory. Thus we should free p1 as well when this allocation fails.

As far as I can see this is correct, as it's just restoring the prior state before the p1 allocation.

Quoted 19 lines

---

drivers/tty/vt/consolemap.c +5 -2
index b28aa0d..79fcc96 100644
@@ -489,7 +489,11 @@ con_insert_unipair(struct uni_pagedir *p, u_short unicode, u_short fontpos)
 	p2 = p1[n = (unicode >> 6) & 0x1f];
 	if (!p2) {
 		p2 = p1[n] = kmalloc_array(64, sizeof(u16), GFP_KERNEL);
-		if (!p2) return -ENOMEM;
+		if (!p2) {
+			kfree(p1);
+			p->uni_pgdir[n] = NULL;
+			return -ENOMEM;
+		}
 		memset(p2, 0xff, 64*sizeof(u16)); /* No glyphs for the characters (yet) */
 	}
 
---
Signature · 1 line
Kees Cook
author
#7 lore raw
Quoted 6 lines · Kees Cook
Quoted 5 lines · Gen Zhang

In function con_insert_unipair(), when allocation for p2 and p1[n] fails, ENOMEM is returned, but previously allocated p1 is not freed, remains as leaking memory. Thus we should free p1 as well when this allocation fails.

Quoted 7 lines
Quoted 1 line

As far as I can see this is correct, as it's just restoring the prior state before the p1 allocation.

Thanks for your review, Kees!

Thanks
Gen

Quoted 22 lines
Quoted 19 lines

---

drivers/tty/vt/consolemap.c +5 -2
index b28aa0d..79fcc96 100644
@@ -489,7 +489,11 @@ con_insert_unipair(struct uni_pagedir *p, u_short unicode, u_short fontpos)
 	p2 = p1[n = (unicode >> 6) & 0x1f];
 	if (!p2) {
 		p2 = p1[n] = kmalloc_array(64, sizeof(u16), GFP_KERNEL);
-		if (!p2) return -ENOMEM;
+		if (!p2) {
+			kfree(p1);
+			p->uni_pgdir[n] = NULL;
+			return -ENOMEM;
+		}
 		memset(p2, 0xff, 64*sizeof(u16)); /* No glyphs for the characters (yet) */
 	}
 
---
Signature · 1 line
Kees Cook
author
#8 lore raw

PATCH v2 1/1 consolemap: Fix a memory leaking bug in con_insert_unipair()

In reply to Kees Cook

In function con_insert_unipair(), when allocation for p2 and p1[n] fails, ENOMEM is returned, but previously allocated p1 is not freed, remains as leaking memory. Thus we should free p1 as well when this allocation fails.

---

drivers/tty/vt/consolemap.c +5 -1
index b28aa0d..79fcc96 100644
@@ -489,7 +489,11 @@ con_insert_unipair(struct uni_pagedir *p, u_short unicode, u_short fontpos)
 	p2 = p1[n = (unicode >> 6) & 0x1f];
 	if (!p2) {
 		p2 = p1[n] = kmalloc_array(64, sizeof(u16), GFP_KERNEL);
-		if (!p2) return -ENOMEM;
+		if (!p2) {
+			kfree(p1);
+			p->uni_pgdir[n] = NULL;
+			return -ENOMEM;
+		}
 		memset(p2, 0xff, 64*sizeof(u16)); /* No glyphs for the characters (yet) */
 	}
 

---

maintainer
#9 lore raw

Re: [PATCH v2] consolemap: Fix a memory leaking bug in con_insert_unipair()

Quoted 7 lines · Gen Zhang

In function con_insert_unipair(), when allocation for p2 and p1[n] fails, ENOMEM is returned, but previously allocated p1 is not freed, remains as leaking memory. Thus we should free p1 as well when this allocation fails.

Any reason you keep dropping me from this thread?

It's as if you don't want me to apply the patch :(

greg k-h

#10 lore raw
In reply to Kees Cook

Kees,

Quoted 10 lines · Kees Cook
Quoted 6 lines · Gen Zhang

In function con_insert_unipair(), when allocation for p2 and p1[n] fails, ENOMEM is returned, but previously allocated p1 is not freed, remains as leaking memory. Thus we should free p1 as well when this allocation fails.

As far as I can see this is correct, as it's just restoring the prior state before the p1 allocation.

Are you sure this is correct? It looks like p1 is only allocated if p->uni_pgdir[n = unicode >> 11] == NULL.

I only mention this because I have seen a few patches from Gen today regarding memory leaks and devm_kzalloc() that are not correct.

Quoted 20 lines
Quoted 18 lines

---

drivers/tty/vt/consolemap.c +5 -1
index b28aa0d..79fcc96 100644
@@ -489,7 +489,11 @@ con_insert_unipair(struct uni_pagedir *p, u_short unicode, u_short fontpos)
 	p2 = p1[n = (unicode >> 6) & 0x1f];
 	if (!p2) {
 		p2 = p1[n] = kmalloc_array(64, sizeof(u16), GFP_KERNEL);
-		if (!p2) return -ENOMEM;
+		if (!p2) {
+			kfree(p1);
+			p->uni_pgdir[n] = NULL;
+			return -ENOMEM;
+		}
 		memset(p2, 0xff, 64*sizeof(u16)); /* No glyphs for the characters (yet) */
 	}
 
Quoted 1 line
Quoted 1 line

---

Cheers
Jon

Signature · 1 line
nvpublic
#11 lore raw
Quoted 9 lines · Jon Hunter

Kees,

Quoted 6 lines · Kees Cook
Quoted 5 lines · Gen Zhang

In function con_insert_unipair(), when allocation for p2 and p1[n] fails, ENOMEM is returned, but previously allocated p1 is not freed, remains as leaking memory. Thus we should free p1 as well when this allocation fails.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

As far as I can see this is correct, as it's just restoring the prior state before the p1 allocation.

Are you sure this is correct? It looks like p1 is only allocated if p->uni_pgdir[n = unicode >> 11] == NULL.

Thanks, yes, I looked more closely, and this is wrong. It's probably fine to leave it as it was (since it appears to just be allocating on demand). If we really want to restore the state on failure, we also can't re-use "n", which is no longer valid. Here, I think, would be a complete patch to check for allocation and use a separate index for the other array:

drivers/tty/vt/consolemap.c +13 -4
index b28aa0d289f8..5f77cffc53b8 100644
@@ -474,7 +474,8 @@ static int con_unify_unimap(struct vc_data *conp, struct uni_pagedir *p)
 static int
 con_insert_unipair(struct uni_pagedir *p, u_short unicode, u_short fontpos)
 {
-	int i, n;
+	int i, n, high;
+	bool p1_alloced = false;
 	u16 **p1, *p2;
 
 	p1 = p->uni_pgdir[n = unicode >> 11];
@@ -482,14 +483,22 @@ con_insert_unipair(struct uni_pagedir *p, u_short unicode, u_short fontpos)
 		p1 = p->uni_pgdir[n] = kmalloc_array(32, sizeof(u16 *),
 						     GFP_KERNEL);
 		if (!p1) return -ENOMEM;
+		p1_alloced = true;
 		for (i = 0; i < 32; i++)
 			p1[i] = NULL;
 	}
 
-	p2 = p1[n = (unicode >> 6) & 0x1f];
+	p2 = p1[high = (unicode >> 6) & 0x1f];
 	if (!p2) {
-		p2 = p1[n] = kmalloc_array(64, sizeof(u16), GFP_KERNEL);
-		if (!p2) return -ENOMEM;
+		p2 = p1[high] = kmalloc_array(64, sizeof(u16), GFP_KERNEL);
+		if (!p2) {
+			if (p1_alloced) {
+				kfree(p1);
+				p->uni_pgdir[n] = NULL;
+			}
+			return -ENOMEM;
+		}
+
 		memset(p2, 0xff, 64*sizeof(u16)); /* No glyphs for the characters (yet) */
 	}
 

But, frankly, probably the patch should just be removed...

-Kees

Quoted 31 lines

I only mention this because I have seen a few patches from Gen today regarding memory leaks and devm_kzalloc() that are not correct.

Quoted 20 lines
Quoted 18 lines

---

drivers/tty/vt/consolemap.c +5 -1
index b28aa0d..79fcc96 100644
@@ -489,7 +489,11 @@ con_insert_unipair(struct uni_pagedir *p, u_short unicode, u_short fontpos)
 	p2 = p1[n = (unicode >> 6) & 0x1f];
 	if (!p2) {
 		p2 = p1[n] = kmalloc_array(64, sizeof(u16), GFP_KERNEL);
-		if (!p2) return -ENOMEM;
+		if (!p2) {
+			kfree(p1);
+			p->uni_pgdir[n] = NULL;
+			return -ENOMEM;
+		}
 		memset(p2, 0xff, 64*sizeof(u16)); /* No glyphs for the characters (yet) */
 	}
 
Quoted 1 line
Quoted 1 line

---

Cheers
Jon

Signature · 1 line
nvpublic
Signature · 1 line
Kees Cook