Skip to contentSkip to conversation

a.out issue

DISCUSSION Florian Heinz 4 participants 10 messages Nov 11, 2004 – Nov 12, 2004

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi ppl,

there seems to be a bug related to a.out-binfmt.

try executing this binary:
perl -e'print"\x07\x01".("\x00"x13)."\xc0".("\x00"x16)'>eout (it may be neccessary to turn memory overcommit on before)

This should result in a kernel-oops.
Doing this in a loop will eat fd's and memory.

seems like find_vma_prepare does not what insert_vm_struct expects when the whole addresspace is occupied.

#1 lore raw

Hello Florian,

Quoted 6 lines · Florian Heinz

try executing this binary:
perl -e'print"\x07\x01".("\x00"x13)."\xc0".("\x00"x16)'>eout (it may be neccessary to turn memory overcommit on before)

This should result in a kernel-oops.
Doing this in a loop will eat fd's and memory.

No oops over here:

Linux penguin 2.6.9 #1 SMP Wed Oct 20 16:11:52 CEST 2004 i686 AMD Athlon(tm) MP 2200+ AuthenticAMD GNU/Linux

Yours sincerely,

Signature · 3 lines
 Ed Schouten <[email protected]>
 Website: http://g-rave.nl/
 GPG key: finger [email protected]
#2 lore raw
In reply to Florian Heinz
Quoted 7 lines · * Florian Heinz ([email protected]) wrote:

there seems to be a bug related to a.out-binfmt.

try executing this binary:
perl -e'print"\x07\x01".("\x00"x13)."\xc0".("\x00"x16)'>eout (it may be neccessary to turn memory overcommit on before)

This should result in a kernel-oops.

No oops here. What kernel version? Can you post your oops?

thanks,
-chris

Signature · 1 line
Linux Security Modules     http://lsm.immunix.org     http://lsm.bkbits.net
#4 lore raw
Quoted 3 lines · * Ed Schouten ([email protected]) wrote:

Have you set:

sysctl -w vm.overcommit_memory=1

I actually set it to 2, now with 1 it's Oopsing. Thanks. -chris

Signature · 1 line
Linux Security Modules     http://lsm.immunix.org     http://lsm.bkbits.net
#6 lore raw
In reply to Florian Heinz

On Thu, Nov 11, 2004 at 11:09:07PM +0100, Florian Heinz took 20 lines to write:

Quoted 8 lines

Hi ppl,

there seems to be a bug related to a.out-binfmt.

try executing this binary:
perl -e'print"\x07\x01".("\x00"x13)."\xc0".("\x00"x16)'>eout (it may be neccessary to turn memory overcommit on before)

Quoted 5 lines

This should result in a kernel-oops.
Doing this in a loop will eat fd's and memory.

seems like find_vma_prepare does not what insert_vm_struct expects when the whole addresspace is occupied.

No oops over here, with overcommit set to 0, 1, or 2.

$ uname -a
Linux luther 2.6.9 #12 Sun Oct 31 07:43:57 EST 2004 i686 unknown unknown
GNU/Linux

Kurt

Signature · 2 lines
Keep Cool, but Don't Freeze
	- Hellman's Mayonnaise
#7 lore raw
In reply to Florian Heinz
Quoted 2 lines · * Florian Heinz ([email protected]) wrote:

seems like find_vma_prepare does not what insert_vm_struct expects when the whole addresspace is occupied.

The setup_arg_pages() is inserting an overlapping region. If nothing
else, this will fix that problem. Perhaps there's a better solution.

thanks,
-chris

===== fs/exec.c 1.143 vs edited =====

fs/exec.c +7 -0
@@ -413,6 +413,7 @@
 
 	down_write(&mm->mmap_sem);
 	{
+		struct vm_area_struct *vma;
 		mpnt->vm_mm = mm;
 #ifdef CONFIG_STACK_GROWSUP
 		mpnt->vm_start = stack_base;
@@ -433,6 +434,12 @@
 			mpnt->vm_flags = VM_STACK_FLAGS;
 		mpnt->vm_flags |= mm->def_flags;
 		mpnt->vm_page_prot = protection_map[mpnt->vm_flags & 0x7];
+		vma = find_vma(mm, mpnt->vm_start);
+		if (vma) {
+			up_write(&mm->mmap_sem);
+			kmem_cache_free(vm_area_cachep, mpnt);
+			return -ENOMEM;
+		}
 		insert_vm_struct(mm, mpnt);
 		mm->stack_vm = mm->total_vm = vma_pages(mpnt);
 	}
#8 lore raw

On Thu, Nov 11, 2004 at 07:27:27PM -0800, Chris Wright took 39 lines to write:

Quoted 6 lines · * Florian Heinz ([email protected]) wrote:
Quoted 2 lines

seems like find_vma_prepare does not what insert_vm_struct expects when the whole addresspace is occupied.

The setup_arg_pages() is inserting an overlapping region. If nothing
else, this will fix that problem. Perhaps there's a better solution.

It solves the oops here (I didn't get the oops at first because I didn't have CONFIG_BINFMT_AOUT set). Sort of. Now I just get "Killed" with vm.overcommit_memory set to 1; with it set to 0 I get a seg fault.

Kurt

Signature · 1 line
Let He who taketh the Plunge Remember to return it by Tuesday.
#9 lore raw
Quoted 10 lines · * Kurt Wall ([email protected]) wrote:

On Thu, Nov 11, 2004 at 07:27:27PM -0800, Chris Wright took 39 lines to write:

Quoted 6 lines · * Florian Heinz ([email protected]) wrote:
Quoted 2 lines

seems like find_vma_prepare does not what insert_vm_struct expects when the whole addresspace is occupied.

The setup_arg_pages() is inserting an overlapping region. If nothing
else, this will fix that problem. Perhaps there's a better solution.

It solves the oops here (I didn't get the oops at first because I didn't have CONFIG_BINFMT_AOUT set).

Heh, you're better off with it config'd off ;-)

Quoted 2 lines

Sort of. Now I just get "Killed" with vm.overcommit_memory set to 1; with it set to 0 I get a seg fault.

Yeah, it should generate a SIGKILL and terminate the program. Thanks for testing. The patch below should fixup that segfault as well.

-chris

Signature · 42 lines
Linux Security Modules     http://lsm.immunix.org     http://lsm.bkbits.net


===== fs/binfmt_aout.c 1.25 vs edited =====
--- 1.25/fs/binfmt_aout.c	2004-10-18 22:26:36 -07:00
+++ edited/fs/binfmt_aout.c	2004-11-11 22:28:58 -08:00
@@ -43,13 +43,18 @@
 	.min_coredump	= PAGE_SIZE
 };
 
-static void set_brk(unsigned long start, unsigned long end)
+#define BAD_ADDR(x)	((unsigned long)(x) >= TASK_SIZE)
+
+static int set_brk(unsigned long start, unsigned long end)
 {
 	start = PAGE_ALIGN(start);
 	end = PAGE_ALIGN(end);
-	if (end <= start)
-		return;
-	do_brk(start, end - start);
+	if (end > start) {
+		unsigned long addr = do_brk(start, end - start);
+		if (BAD_ADDR(addr))
+			return addr;
+	}
+	return 0;
 }
 
 /*
@@ -413,7 +418,11 @@
 beyond_if:
 	set_binfmt(&aout_format);
 
-	set_brk(current->mm->start_brk, current->mm->brk);
+	retval = set_brk(current->mm->start_brk, current->mm->brk);
+	if (retval < 0) {
+		send_sig(SIGKILL, current, 0);
+		return retval;
+	}
 
 	retval = setup_arg_pages(bprm, EXSTACK_DEFAULT);
 	if (retval < 0) {