Skip to contentSkip to conversation

mtest01: free memory allocated

LTP Francesco RUNDO 6 participants 9 messages Jan 4, 2010 – Jan 11, 2010

lore
Share & tools
lore

Francesco RUNDO

author
#0 lore raw

Hi All,

I'm running LTP (I'm using ltp-full-20090731....but asap I will upgrade to latest) on SH based platforms.
Now, during a test-session. I've noted that the test "mtest01" reduced drastically the system memory and after its execution this memory wasn't de-allocated.

I've analysed the mtest01.c code and I've noted that no "free()" istruction was associated to the related malloc:

......
if((mem = (char*)malloc(chunksize)) == NULL) {
......

I've simply added a "free(mem)" of the allocated memory and the issue was addressed successfully.

I've attached the trivial patch I've developed.

Best Regards,
--
Francesco

Added missed "free" istruction to release memory previosuly allocated.

Signed-off-by Francesco Rundo <[email protected]>
testcases/kernel/mem/mtest01/mtest01.c renamed from testcases/kernel/mem/mtest01/mtest01.c.origin +3 -0
@@ -258,5 +258,8 @@
     else
       tst_resm(TPASS, "%llu kbytes allocated only.", original_maxbytes/1024);
   }
+
+  free(mem);
+  
   exit(0);
 }

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev

_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#1 lore raw
Quoted 38 lines · Francesco RUNDO

Hi All,

I'm running LTP (I'm using ltp-full-20090731....but asap I will upgrade to latest) on SH based platforms.
Now, during a test-session. I've noted that the test "mtest01" reduced drastically the system memory and after its execution this memory wasn't de-allocated.

I've analysed the mtest01.c code and I've noted that no "free()" istruction was associated to the related malloc:

......
if((mem = (char*)malloc(chunksize)) == NULL) {
......

I've simply added a "free(mem)" of the allocated memory and the issue was addressed successfully.

I've attached the trivial patch I've developed.

Best Regards,
--
Francesco
Added missed "free" istruction to release memory previosuly allocated.

Signed-off-by Francesco Rundo <[email protected]>

--- ltp-full-20090731/testcases/kernel/mem/mtest01/
mtest01.c.origin 2009-02-26 13:02:27.000000000 +0100
+++ ltp-full-20090731/testcases/kernel/mem/mtest01/mtest01.c

2009-12-15 14:03:28.779240000 +0100
testcases/kernel/mem/mtest01/mtest01.c renamed from testcases/kernel/mem/mtest01/mtest01.c.origin +3 -0
@@ -258,5 +258,8 @@
    else
      tst_resm(TPASS, "%llu kbytes allocated only.",  
 original_maxbytes/1024);
  }
+
+  free(mem);
+
  exit(0);
Quoted 1 line

}

1. Does the version off cvs still have this issue?
2. Unless the test exits the code block immediately, I'd definitely do:

if (mem)

     free (mem);

to avoid making a bad free call.

Thanks,
-Garrett

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#2 lore raw
Quoted 7 lines

2. Unless the test exits the code block immediately, I'd definitely do:

if (mem)

     free (mem);

to avoid making a bad free call.

free(NULL) is valid.

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#6 lore raw
In reply to Francesco RUNDO

Francesco RUNDO napsal(a):

Quoted 10 lines

Hi All,

I'm running LTP (I'm using ltp-full-20090731....but asap I will upgrade to latest) on SH based platforms.
Now, during a test-session. I've noted that the test "mtest01" reduced drastically the system memory and after its execution this memory wasn't de-allocated.

I've analysed the mtest01.c code and I've noted that no "free()" istruction was associated to the related malloc:

Does this mean the kernel doesn't free processes' allocated memory on exit? Is this intentional (and documented somewhere)?

Quoted 6 lines

......
if((mem = (char*)malloc(chunksize)) == NULL) {
......

I've simply added a "free(mem)" of the allocated memory and the issue was addressed successfully.

This isn't complete by far. You don't free all the allocations, and there are code paths which don't pass your line before exit.

Regards
Jiri Palecek

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#7 lore raw
Quoted 33 lines

-----Original Message-----
From: Jiri Palecek [mailto:[email protected]]
Sent: Tuesday, January 05, 2010 5:20 PM
To: Francesco RUNDO
Cc: LTP
Subject: Re: [LTP] mtest01: free memory allocated

Francesco RUNDO napsal(a):

Quoted 3 lines

Hi All,

I'm running LTP (I'm using ltp-full-20090731....but asap I will

upgrade

Quoted 2 lines

to latest) on SH based platforms.
Now, during a test-session. I've noted that the test "mtest01"

reduced

Quoted 1 line

drastically the system memory and after its execution this memory

wasn't

Quoted 4 lines

de-allocated.

I've analysed the mtest01.c code and I've noted that no "free()" istruction was associated to the related malloc:

Does this mean the kernel doesn't free processes' allocated memory on exit? Is
this intentional (and documented somewhere)?

Quoted 6 lines

......
if((mem = (char*)malloc(chunksize)) == NULL) {
......

I've simply added a "free(mem)" of the allocated memory and the issue was addressed successfully.

Quoted 7 lines

This isn't complete by far. You don't free all the allocations, and there are
code paths which don't pass your line before exit.

Regards
Jiri Palecek

We've documented this behavior internally as a low priority issue for a while now.
This is the root cause one of our developers came back with, which might clarify what Francesco is seeing:

Issue:

Across architectures, mtest tests sometimes report a fail because they won't run unless less than 80% of memory is used. This is because the test attempts to fill up the memory to the 80% limit specified by the -p option.

Some previous test in the LTP suite is taking up memory it should not have, as this happens even on machines with 1GB of memory.

mtest01 runs "mtest01 -p80" which mallocs chunks of memory until 80% of memory is used.
mtest01w runs "mtest01 -p80 -w" which mallocs and writes chunks of memory until 80% of memory is used.

Response:

mtest01 -p80 -w; mtest01 -p80 -w; free; free; free... ; mtest01 -p80 -w

on the same command line. the second mtest01 always returns with a failure. while the mtest01 after enough free(usually 4 or 5) always passes.

Looks like it is because of the aging of the caches, which are not freed yet when the second mtest01 is executed and are dropped due to aging by the time last mtest01 is executed.
The information (sysinfo()) mtest is using to determine the amount of memory allocated does not include the memory in caches(free). mtest01 sees that 80% of the memory is already in use and thus do not go for malloc-ing and returns with a failure, however if it had tried, the malloc would have succeeded because of the reuse of the free slab caches.

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#8 lore raw

Henry Yei napsal(a):

Quoted 59 lines
Quoted 33 lines

-----Original Message-----
From: Jiri Palecek [mailto:[email protected]]
Sent: Tuesday, January 05, 2010 5:20 PM
To: Francesco RUNDO
Cc: LTP
Subject: Re: [LTP] mtest01: free memory allocated

Francesco RUNDO napsal(a):

Quoted 3 lines

Hi All,

I'm running LTP (I'm using ltp-full-20090731....but asap I will

upgrade

Quoted 2 lines

to latest) on SH based platforms.
Now, during a test-session. I've noted that the test "mtest01"

reduced

Quoted 1 line

drastically the system memory and after its execution this memory

wasn't

Quoted 4 lines

de-allocated.

I've analysed the mtest01.c code and I've noted that no "free()" istruction was associated to the related malloc:

Does this mean the kernel doesn't free processes' allocated memory on exit? Is
this intentional (and documented somewhere)?

Quoted 6 lines

......
if((mem = (char*)malloc(chunksize)) == NULL) {
......

I've simply added a "free(mem)" of the allocated memory and the issue was addressed successfully.

Quoted 7 lines

This isn't complete by far. You don't free all the allocations, and there are
code paths which don't pass your line before exit.

Regards
Jiri Palecek

We've documented this behavior internally as a low priority issue for a while now.
This is the root cause one of our developers came back with, which might clarify what Francesco is seeing:

Issue:

Across architectures, mtest tests sometimes report a fail because they won't run unless less than 80% of memory is used. This is because the test attempts to fill up the memory to the 80% limit specified by the -p option.

Some previous test in the LTP suite is taking up memory it should not have, as this happens even on machines with 1GB of memory.

mtest01 runs "mtest01 -p80" which mallocs chunks of memory until 80% of memory is used.
mtest01w runs "mtest01 -p80 -w" which mallocs and writes chunks of memory until 80% of memory is used.

Quoted 6 lines

Response:

mtest01 -p80 -w; mtest01 -p80 -w; free; free; free... ; mtest01 -p80 -w

on the same command line. the second mtest01 always returns with a failure. while the mtest01 after enough free(usually 4 or 5) always passes.

Indeed, I reproduced it on a machine without swap (is this reproducible with swap?), although the second mtest didn't fail, but just allocated ridiculously small amount of memory. However, I'm curious if this is what Francesco is seeing (eg. if the memory is eventually freed on his machine).

Quoted 8 lines

Looks like it is because of the aging of the caches, which are not freed yet when the second mtest01 is executed and are dropped due to aging by the time last mtest01 is executed.
The information (sysinfo()) mtest is using to determine the amount of memory allocated does not include the memory in caches(free). mtest01 sees that 80% of the memory is already in use and thus do not go for malloc-ing and returns with a failure, however if it had tried, the malloc would have succeeded because of the reuse of the free slab caches.

I'm afraid this isn't related to slab caches, as slabinfo between the two tests doesn't show anything suspicious (it's logical, too - if the test just before consumed 80% of memory in user pages, it's unlikely the memory would be allocated to slab after that).

Regards
Jiri Palecek

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#3 lore raw
In reply to Garrett Cooper

Garrett,

Quoted 46 lines · Garrett Cooper
Quoted 38 lines · Francesco RUNDO

Hi All,

I'm running LTP (I'm using ltp-full-20090731....but asap I will upgrade to latest) on SH based platforms.
Now, during a test-session. I've noted that the test "mtest01" reduced drastically the system memory and after its execution this memory wasn't de-allocated.

I've analysed the mtest01.c code and I've noted that no "free()" istruction was associated to the related malloc:

......
if((mem = (char*)malloc(chunksize)) == NULL) {
......

I've simply added a "free(mem)" of the allocated memory and the issue was addressed successfully.

I've attached the trivial patch I've developed.

Best Regards,
--
Francesco
Added missed "free" istruction to release memory previosuly allocated.

Signed-off-by Francesco Rundo <[email protected]>

--- ltp-full-20090731/testcases/kernel/mem/mtest01/
mtest01.c.origin 2009-02-26 13:02:27.000000000 +0100
+++ ltp-full-20090731/testcases/kernel/mem/mtest01/mtest01.c

2009-12-15 14:03:28.779240000 +0100
testcases/kernel/mem/mtest01/mtest01.c renamed from testcases/kernel/mem/mtest01/mtest01.c.origin +3 -0
@@ -258,5 +258,8 @@
    else
      tst_resm(TPASS, "%llu kbytes allocated only.",  
 original_maxbytes/1024);
  }
+
+  free(mem);
+
  exit(0);
Quoted 1 line

}

1. Does the version off cvs still have this issue?
2. Unless the test exits the code block immediately, I'd definitely do:

Quoted 4 lines

if (mem)

     free (mem);

to avoid making a bad free call.

Have you made any changes to this ? I do not see any commits.

Regards--
Subrata

Quoted 4 lines

Thanks,
-Garrett

Quoted 10 lines

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#4 lore raw
Quoted 52 lines · Subrata Modak

Garrett,

Quoted 49 lines · Garrett Cooper
Quoted 38 lines · Francesco RUNDO

Hi All,

I'm running LTP (I'm using ltp-full-20090731....but asap I will upgrade to latest) on SH based platforms.
Now, during a test-session. I've noted that the test "mtest01" reduced drastically the system memory and after its execution this memory wasn't de-allocated.

I've analysed the mtest01.c code and I've noted that no "free()" istruction was associated to the related malloc:

......
if((mem = (char*)malloc(chunksize)) == NULL) {
......

I've simply added a "free(mem)" of the allocated memory and the issue was addressed successfully.

I've attached the trivial patch I've developed.

Best Regards,
--
Francesco
Added missed "free" istruction to release memory previosuly allocated.

Signed-off-by Francesco Rundo <[email protected]>

--- ltp-full-20090731/testcases/kernel/mem/mtest01/
mtest01.c.origin 2009-02-26 13:02:27.000000000 +0100
+++ ltp-full-20090731/testcases/kernel/mem/mtest01/mtest01.c

2009-12-15 14:03:28.779240000 +0100
testcases/kernel/mem/mtest01/mtest01.c renamed from testcases/kernel/mem/mtest01/mtest01.c.origin +3 -0
@@ -258,5 +258,8 @@
   else
     tst_resm(TPASS, "%llu kbytes allocated only.",  
 original_maxbytes/1024);
 }
+
+  free(mem);
+
 exit(0);
Quoted 1 line

}

1. Does the version off cvs still have this issue?
2. Unless the test exits the code block immediately, I'd definitely do:

if (mem)

    free (mem);
Quoted 6 lines
Quoted 1 line

to avoid making a bad free call.

Have you made any changes to this ? I do not see any commits.

Regards--
Subrata

Nope. I've left it untouched because of the discussion that was being made in the thread. Thanks,
-Garrett

Quoted 1 line

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

author
#5 lore raw
In reply to Subrata Modak

Dear All,

Ok, likely a sort of flag has to be added in the "mtest01.c" code.....something like that:

mem_flag=0;
....................

if((mem = (char*)malloc(chunksize)) == NULL) {

.......

exit(1)
}

mem_flag=1:

.......

in order to understand if the malloc() has been executed or not.....and then the patch will be:

if ((mem_flag==1)&&(mem != NULL))

{
        free(mem);
        mem_flag=0;
}

Without a patch as above (or like that), in the code of "mtest01" we got scenarios (mainly in the cross architectures like embedded systems) in which the LTP run a "malloc()" without "free()". This is always wrong even though the system resource is enough to cover this memory consumption.

Regards
--
FR

Quoted 51 lines · Subrata Modak

Garrett,

Quoted 46 lines · Garrett Cooper
Quoted 33 lines · Francesco RUNDO

Hi All,

I'm running LTP (I'm using ltp-full-20090731....but asap I will upgrade to latest) on SH based platforms.
Now, during a test-session. I've noted that the test "mtest01" reduced drastically the system memory and after its execution this memory wasn't de-allocated.

I've analysed the mtest01.c code and I've noted that no "free()" istruction was associated to the related malloc:

......
if((mem = (char*)malloc(chunksize)) == NULL) {
......

I've simply added a "free(mem)" of the allocated memory and the issue was addressed successfully.

I've attached the trivial patch I've developed.

Best Regards,
--
Francesco
Added missed "free" istruction to release memory previosuly allocated.

Signed-off-by Francesco Rundo <[email protected]>

--- ltp-full-20090731/testcases/kernel/mem/mtest01/
mtest01.c.origin 2009-02-26 13:02:27.000000000 +0100
+++ ltp-full-20090731/testcases/kernel/mem/mtest01/mtest01.c

2009-12-15 14:03:28.779240000 +0100
testcases/kernel/mem/mtest01/mtest01.c renamed from testcases/kernel/mem/mtest01/mtest01.c.origin +3 -0
@@ -258,5 +258,8 @@
   else
     tst_resm(TPASS, "%llu kbytes allocated only.",  
 original_maxbytes/1024);
 }
+
+  free(mem);
+
 exit(0);
Quoted 3 lines

}

1. Does the version off cvs still have this issue?
2. Unless the test exits the code block immediately, I'd definitely do:

if (mem)

    free (mem);
Quoted 29 lines
Quoted 3 lines

to avoid making a bad free call.

Have you made any changes to this ? I do not see any commits.

Regards--
Subrata

Quoted 14 lines

Thanks,
-Garrett

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev _______________________________________________ Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev

_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list