Skip to contentSkip to conversation

Overcommit (OOM) problem on embedded device (PPChameleon)

DISCUSSION Per Hallsmark 3 participants 6 messages Mar 30, 2005 – Apr 1, 2005

lore
Share & tools
lore

Per Hallsmark

author
#0 lore raw

I think your main problem is that the stock 2.4.x kernel doesn't support ovecommit_memory type 2 (i believe)...
Look in mm/mmap.c and search for overcommit_memory, then do same in the sources for your redhat kernels.

Perhaps there's a patch for it floating around somewhere? ;-)

Quoted 1 line

===== Original Message From Martin Egholm Nielsen <[email protected]>

=====

Quoted 78 lines

Hi there,

I hope this is the place to go...

I have a some problems figuring out the OOM-killer and configuring the overcommit_memory parameter. Hope someone here can guide me in the right directions...

Specs:
I'm having an embedded Linux system running on a PPC405EP (PPChameleon) with 64 megs of RAM, some flash, but (ofcourse) no swap space. It runs a 2.4.20 kernel patched with drivers for my device.

Problem:
I have an application that is killed by the OOM (I guess) when it tries to "use" more memory than present on the system.
Bolied down, memory is allocated with "sbrk" and then touch'ed (see test-application below).

With "/proc/sys/vm/overcommit_memory" set to 2, I expected that "sbrk" would return "-1L" (0xFFFFFFFF), but it doesn't, hence is terminated/killed by the kernel.

However, both my desktop Linux (RH 7.3)/2.4.18-10/i386 and Linux (FC2)/2.6.5/i386 did what I expected:

# ./exhaust_mem
...
ffffffff

Out of memory
# #Yeaaaah!

Having searched the web, I see that this may be related with the fact that there is no swap enabled on the embedded device. However, I tried disabling the swap (commented in fstab), but the desktop linux still behaves "correct".

Can I do anything in order to get it the way I expected?

Best regards,
Martin Egholm

=== exhaust_mem.c ===

#include <unistd.h>
#include <stdio.h>
#define SIZE 1000000

int main( int i )
{

  while ( 1 ) {
    char *v = sbrk( SIZE );
    char *p;

    printf( "%x\n\n", v );

    if ((long)v < 0) {
      fprintf(stderr, "Out of memory\n");
      exit(1);
    } // if

    for (p = v; p < v + SIZE; ++p) {
      *p = 42;
    } // for

} // while
} // main

_______________________________________________
Linuxppc-embedded mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc-embedded

#1 lore raw

Hi Per,

Quoted 2 lines

I think your main problem is that the stock 2.4.x kernel doesn't support ovecommit_memory type 2 (i believe)...

Doooh! You're so right!
I read the docs on overcommit_memory from 2.6 - stupid me!

Quoted 2 lines

Look in mm/mmap.c and search for overcommit_memory, then do same in the sources for your redhat kernels.

I'll look into RH kernel to see if it isn't like you say :-)

Quoted 1 line

Perhaps there's a patch for it floating around somewhere? ;-)

Now, that would be really nice.
Though I have no idea of where to look!?

Thanks for clearing my mind!

BR,
Martin

Quoted 89 lines
Quoted 3 lines

===== Original Message From Martin Egholm Nielsen <[email protected]>

=====

Quoted 78 lines

Hi there,

I hope this is the place to go...

I have a some problems figuring out the OOM-killer and configuring the overcommit_memory parameter. Hope someone here can guide me in the right directions...

Specs:
I'm having an embedded Linux system running on a PPC405EP (PPChameleon) with 64 megs of RAM, some flash, but (ofcourse) no swap space. It runs a 2.4.20 kernel patched with drivers for my device.

Problem:
I have an application that is killed by the OOM (I guess) when it tries to "use" more memory than present on the system.
Bolied down, memory is allocated with "sbrk" and then touch'ed (see test-application below).

With "/proc/sys/vm/overcommit_memory" set to 2, I expected that "sbrk" would return "-1L" (0xFFFFFFFF), but it doesn't, hence is terminated/killed by the kernel.

However, both my desktop Linux (RH 7.3)/2.4.18-10/i386 and Linux (FC2)/2.6.5/i386 did what I expected:

# ./exhaust_mem
...
ffffffff

Out of memory
# #Yeaaaah!

Having searched the web, I see that this may be related with the fact that there is no swap enabled on the embedded device. However, I tried disabling the swap (commented in fstab), but the desktop linux still behaves "correct".

Can I do anything in order to get it the way I expected?

Best regards,
Martin Egholm

=== exhaust_mem.c ===

#include <unistd.h>
#include <stdio.h>
#define SIZE 1000000

int main( int i )
{

 while ( 1 ) {
   char *v = sbrk( SIZE );
   char *p;

   printf( "%x\n\n", v );

   if ((long)v < 0) {
     fprintf(stderr, "Out of memory\n");
     exit(1);
   } // if

   for (p = v; p < v + SIZE; ++p) {
     *p = 42;
   } // for

} // while
} // main

_______________________________________________
Linuxppc-embedded mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc-embedded

#2 lore raw
In reply to Per Hallsmark

Hi Per,

Quoted 2 lines

I think your main problem is that the stock 2.4.x kernel doesn't support ovecommit_memory type 2 (i believe)...

Doooh! You're so right!
I read the docs on overcommit_memory from 2.6 - stupid me!

Quoted 2 lines

Look in mm/mmap.c and search for overcommit_memory, then do same in the sources for your redhat kernels.

I'll look into RH kernel to see if it isn't like you say :-)

Quoted 1 line

Perhaps there's a patch for it floating around somewhere? ;-)

Now, that would be really nice.
Though I have no idea of where to look!?

Thanks for clearing my mind!

BR,
Martin

Quoted 89 lines
Quoted 3 lines

===== Original Message From Martin Egholm Nielsen <[email protected]>

=====

Quoted 78 lines

Hi there,

I hope this is the place to go...

I have a some problems figuring out the OOM-killer and configuring the overcommit_memory parameter. Hope someone here can guide me in the right directions...

Specs:
I'm having an embedded Linux system running on a PPC405EP (PPChameleon) with 64 megs of RAM, some flash, but (ofcourse) no swap space. It runs a 2.4.20 kernel patched with drivers for my device.

Problem:
I have an application that is killed by the OOM (I guess) when it tries to "use" more memory than present on the system.
Bolied down, memory is allocated with "sbrk" and then touch'ed (see test-application below).

With "/proc/sys/vm/overcommit_memory" set to 2, I expected that "sbrk" would return "-1L" (0xFFFFFFFF), but it doesn't, hence is terminated/killed by the kernel.

However, both my desktop Linux (RH 7.3)/2.4.18-10/i386 and Linux (FC2)/2.6.5/i386 did what I expected:

# ./exhaust_mem
...
ffffffff

Out of memory
# #Yeaaaah!

Having searched the web, I see that this may be related with the fact that there is no swap enabled on the embedded device. However, I tried disabling the swap (commented in fstab), but the desktop linux still behaves "correct".

Can I do anything in order to get it the way I expected?

Best regards,
Martin Egholm

=== exhaust_mem.c ===

#include <unistd.h>
#include <stdio.h>
#define SIZE 1000000

int main( int i )
{

 while ( 1 ) {
   char *v = sbrk( SIZE );
   char *p;

   printf( "%x\n\n", v );

   if ((long)v < 0) {
     fprintf(stderr, "Out of memory\n");
     exit(1);
   } // if

   for (p = v; p < v + SIZE; ++p) {
     *p = 42;
   } // for

} // while
} // main

_______________________________________________
Linuxppc-embedded mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc-embedded

#3 lore raw
Quoted 10 lines
Quoted 4 lines

Look in mm/mmap.c and search for overcommit_memory, then do same in the sources for your redhat kernels.

I'll look into RH kernel to see if it isn't like you say :-)

Quoted 1 line

Perhaps there's a patch for it floating around somewhere? ;-)

Now, that would be really nice.
Though I have no idea of where to look!?

Here is what I do. Patch is against BK linuxppc-2.4 version 1.1285 (2.4.28-pre3) but might work.

Burns 5% of the RAM but alternatives all seem worse and I never have much luck trying to convince my colleagues that dynamic allocation has no place in an embedded system.

If you find out why RH works it would be interesting ... perhaps there is a better way.

David

#

# mm/mmap.c
#   Add pessimistic overcommit mode similar to 2.6 mode 2.
#   This allows malloc aka sbrk() to actually fail before
#   process is killed.
#   Overloaded sysctl_overcommit_memory to be both enable
#   and ratio to avoid making a new sysctl.
# 
diff -Nru a/mm/mmap.c b/mm/mmap.c
--- a/mm/mmap.c	2005-03-30 07:15:13 -08:00
+++ b/mm/mmap.c	2005-03-30 07:15:13 -08:00
@@ -45,9 +45,13 @@
 	__S000, __S001, __S010, __S011, __S100, __S101, __S110, __S111
 };

-int sysctl_overcommit_memory;
+int sysctl_overcommit_memory = 98;
+
int max_map_count = DEFAULT_MAX_MAP_COUNT;

+extern unsigned long totalram_pages;
+extern unsigned long totalhigh_pages;
+
/* Check that a process has enough memory to allocate a
* new virtual mapping.
*/
@@ -66,7 +70,7 @@
unsigned long free;

         /* Sometimes we want to use more memory than we have. */

- if (sysctl_overcommit_memory)
+ if (sysctl_overcommit_memory == 1)
return 1;

/* The page cache contains buffer pages these days.. */
@@ -91,7 +95,20 @@
free += (dentry_stat.nr_unused * sizeof(struct dentry)) >>
PAGE_SHIFT;
free += (inodes_stat.nr_unused * sizeof(struct inode)) >>
PAGE_SHIFT;

+ /*
+ * Leave the last 3% for root
+ */
+ if (current->euid)
+ free -= free / 32;
+
+ /* Strict mode do not allocate last bit of memory */
+ if (sysctl_overcommit_memory) {
+ pages += (totalram_pages - totalhigh_pages)
+ * (100 - sysctl_overcommit_memory) / 100;
+ }
+
return free > pages;
+
}

/* Remove one vm structure from the inode's i_mapping address space. */

#4 lore raw

Hi David,

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Look in mm/mmap.c and search for overcommit_memory, then do same in the sources for your redhat kernels.

I'll look into RH kernel to see if it isn't like you say :-)

Quoted 1 line

Perhaps there's a patch for it floating around somewhere? ;-)

Now, that would be really nice.
Though I have no idea of where to look!?

Here is what I do. Patch is against BK linuxppc-2.4 version 1.1285 (2.4.28-pre3) but might work.

Well, the patch goes straight into my 2.4.20 ppc-kernel - but we have to wait until tomorrow to see if it does the magic.

Quoted 1 line

Burns 5% of the RAM but alternatives all seem worse and I never

I can live with that (3%)...

Quoted 2 lines

If you find out why RH works it would be interesting ... perhaps there is a better way.

I downloaded the 2.4.20-10 kernel SRPM from RH8 - it contains all the patches they apply to the serene 2.4.20 kernel. There is some stuff regarding introducing the "overcommit_memory_accounting" variable from the 2.5 kernel, but I haven't looked into it in depth.

I'll try with your patch first :-)

BR,
Martin Egholm

Quoted 60 lines

#

# mm/mmap.c
#   Add pessimistic overcommit mode similar to 2.6 mode 2.
#   This allows malloc aka sbrk() to actually fail before
#   process is killed.
#   Overloaded sysctl_overcommit_memory to be both enable
#   and ratio to avoid making a new sysctl.
# diff -Nru a/mm/mmap.c b/mm/mmap.c
mm/mmap.c +19 -2
@@ -45,9 +45,13 @@
 	__S000, __S001, __S010, __S011, __S100, __S101, __S110, __S111
 };
 
-int sysctl_overcommit_memory;
+int sysctl_overcommit_memory = 98;
+
 int max_map_count = DEFAULT_MAX_MAP_COUNT;
 
+extern unsigned long totalram_pages;
+extern unsigned long totalhigh_pages;
+
 /* Check that a process has enough memory to allocate a
  * new virtual mapping.
  */

@@ -66,7 +70,7 @@
 	unsigned long free;
 
         /* Sometimes we want to use more memory than we have. */
-	if (sysctl_overcommit_memory)
+	if (sysctl_overcommit_memory == 1)
 	    return 1;
 
 	/* The page cache contains buffer pages these days.. */

@@ -91,7 +95,20 @@
 	free += (dentry_stat.nr_unused * sizeof(struct dentry)) >>
 PAGE_SHIFT;
 	free += (inodes_stat.nr_unused * sizeof(struct inode)) >>
 PAGE_SHIFT;
 
+	/*
+	 * Leave the last 3% for root
+	 */
+	if (current->euid)
+		free -= free / 32;
+
+	/* Strict mode do not allocate last bit of memory */
+	if (sysctl_overcommit_memory) {
+		pages += (totalram_pages - totalhigh_pages)
+			* (100 - sysctl_overcommit_memory) / 100;
+	}
+
 	return free > pages;
+
 }

/* Remove one vm structure from the inode's i_mapping address space. */

#5 lore raw
In reply to David Adair

Helloooo David!

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Look in mm/mmap.c and search for overcommit_memory, then do same in the sources for your redhat kernels.

I'll look into RH kernel to see if it isn't like you say :-)

Quoted 1 line

Perhaps there's a patch for it floating around somewhere? ;-)

Now, that would be really nice.
Though I have no idea of where to look!?

Here is what I do. Patch is against BK linuxppc-2.4 version 1.1285 (2.4.28-pre3) but might work.

Uhhuuuuuuu!! It works!!!

With my simple exhaust_mem example it works with a value of 98 (that is 5% of waste).
However, I need setting it to 95 (so ~8% waste) in order to make it work entirely with my other application... But that's better than letting the OOM take over control.

Quoted 6 lines

Burns 5% of the RAM but alternatives all seem worse and I never have much luck trying to convince my colleagues that dynamic allocation has no place in an embedded system.

If you find out why RH works it would be interesting ... perhaps there is a better way.

I'll let you know if I find the triggering patch in RH's kernels :-)

THANKS,
Martin Egholm

Quoted 64 lines

David

#

# mm/mmap.c
#   Add pessimistic overcommit mode similar to 2.6 mode 2.
#   This allows malloc aka sbrk() to actually fail before
#   process is killed.
#   Overloaded sysctl_overcommit_memory to be both enable
#   and ratio to avoid making a new sysctl.
# diff -Nru a/mm/mmap.c b/mm/mmap.c
mm/mmap.c +19 -2
@@ -45,9 +45,13 @@
 	__S000, __S001, __S010, __S011, __S100, __S101, __S110, __S111
 };
 
-int sysctl_overcommit_memory;
+int sysctl_overcommit_memory = 98;
+
 int max_map_count = DEFAULT_MAX_MAP_COUNT;
 
+extern unsigned long totalram_pages;
+extern unsigned long totalhigh_pages;
+
 /* Check that a process has enough memory to allocate a
  * new virtual mapping.
  */

@@ -66,7 +70,7 @@
 	unsigned long free;
 
         /* Sometimes we want to use more memory than we have. */
-	if (sysctl_overcommit_memory)
+	if (sysctl_overcommit_memory == 1)
 	    return 1;
 
 	/* The page cache contains buffer pages these days.. */

@@ -91,7 +95,20 @@
 	free += (dentry_stat.nr_unused * sizeof(struct dentry)) >>
 PAGE_SHIFT;
 	free += (inodes_stat.nr_unused * sizeof(struct inode)) >>
 PAGE_SHIFT;
 
+	/*
+	 * Leave the last 3% for root
+	 */
+	if (current->euid)
+		free -= free / 32;
+
+	/* Strict mode do not allocate last bit of memory */
+	if (sysctl_overcommit_memory) {
+		pages += (totalram_pages - totalhigh_pages)
+			* (100 - sysctl_overcommit_memory) / 100;
+	}
+
 	return free > pages;
+
 }

/* Remove one vm structure from the inode's i_mapping address space. */