madvise09: Add MADV_FREE test
PATCH 1/1 madvise09: Add MADV_FREE test
We test the madvise(MADV_FREE) by running a process in a memory cgroup with fairly small memory limits. The test process forks a child, moves it to the newly created cgroup. The child allocates memory, marks it MADV_FREE then forks a memory hungry child that allocates and faults memory in a loop. Due to undeterministic nature of the OOM (that kills the memory hungry child sooner or later) we have to retry at two levels. First problem is that rarely it gets too rogue and both processes in the cgroup gets killed. The second is that sometimes the memory hungry child is killed too fast (before the kernel has chance to free the pages), so we rerun it a (for a few times) if that happens.
The test expects memory cgroup mounted in the standard /sys/fs/cgroup/ path, which is OK since the functionality tested was added to kernel 4.5 and the test would be skipped on older distros anyway.
Also the test expects that the MADV_FREE pages will not be freed immediatelly hence the test will fail if the whole system is under memory pressure.
The memory limits were choosen to be 2MB and 4MB for memsw limit. I've tried to use to base these on usage_in_bytes but that turned out to be source of random errors. It worked fine and produces stable numbers 99% of the time but then the numbers have changed and caused the test to fail in fork() with ENOMEM.
--- runtest/syscalls | 1 + testcases/kernel/syscalls/.gitignore | 1 + testcases/kernel/syscalls/madvise/madvise09.c | 229 ++++++++++++++++++++++++++ 3 files changed, 231 insertions(+) create mode 100644 testcases/kernel/syscalls/madvise/madvise09.c
runtest/syscalls +1 -0
@@ -753,6 +753,7 @@ madvise05 madvise05 madvise06 madvise06 madvise07 madvise07 madvise08 madvise08 +madvise09 madvise09 newuname01 newuname01
testcases/kernel/syscalls/.gitignore +1 -0
@@ -514,6 +514,7 @@ /madvise/madvise06 /madvise/madvise07 /madvise/madvise08 +/madvise/madvise09 /mallopt/mallopt01 /mbind/mbind01 /memcmp/memcmp01
testcases/kernel/syscalls/madvise/madvise09.c new file +229 -0
229 lines not on this page · raw message
Signature · 1 line
2.10.2
PATCH 1/1 madvise09: Add MADV_FREE test
Hi!
Ping.
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 1/1 madvise09: Add MADV_FREE test
In reply to Cyril Hrubis----- Original Message -----
Quoted 8 lines
We test the
madvise(MADV_FREE)by running a process in a memory cgroup with fairly small memory limits. The test process forks a child, moves it to the newly created cgroup. The child allocates memory, marks itMADV_FREEthen forks a memory hungry child that allocates and faults memory in a loop. Due to undeterministic nature of the OOM (that kills the memory hungry child sooner or later) we have to retry at two levels. First problem is that rarely it gets too rogue and both processes in the cgroup gets killed.
Hi,
Could we protect "the good child" via oom_adj or similar so OOM doesn't touch it?
Quoted 3 lines
The second is that sometimes the memory hungry child
is killed too fast (before the kernel has chance to free the pages), so we rerun it a (for a few times) if that happens.
How about we progressively make each retry slower? [1]
Quoted 10 lines
The test expects memory cgroup mounted in the standard
/sys/fs/cgroup/path, which is OK since the functionality tested was added to kernel 4.5 and the test would be skipped on older distros anyway.Also the test expects that the
MADV_FREEpages will not be freed immediatelly hence the test will fail if the whole system is under memory pressure.The memory limits were choosen to be 2MB and 4MB for memsw limit.
This looks a bit small to me. Have you checked how much is used just after fork on a system with 64k pages? My concern is that we hit this limit before we do anything.
<snip>
testcases/kernel/syscalls/madvise/madvise09.c line 78
Quoted 12 lines
testcases/kernel/syscalls/madvise/madvise09.c new file +12
-0+ +static void memory_pressure_child (void ) +{ +size_t i, page_size =getpagesize (); +char *ptr; + +for (;;) { + ptr =mmap (NULL ,1000 * page_size,PROT_READ |PROT_WRITE , +MAP_PRIVATE |MAP_ANONYMOUS ,-1 ,0 ); + +for (i =0 ; i <1000 * page_size; i++) + ptr[i] = i %100 ;
[1] some kind of sleep here, so that retries will get gradually slower, giving kernel more time to free marked pages.
Quoted 31 lines
testcases/kernel/syscalls/madvise/madvise09.c new file +31
-0+ } + +abort (); +} + +static void setup_cgroup_paths (int pid) +{ +snprintf (cgroup_path,sizeof (cgroup_path), +MEMCG_PATH "ltp_madvise09_%i/" , pid); +snprintf (tasks_path,sizeof (tasks_path),"%s/tasks" , cgroup_path); +snprintf (limit_in_bytes_path,sizeof (limit_in_bytes_path), +"%s/memory.limit_in_bytes" , cgroup_path); +snprintf (memsw_limit_in_bytes_path,sizeof (memsw_limit_in_bytes_path), +"%s/memory.memsw.limit_in_bytes" , cgroup_path); +} + +static void child (void ) +{ +size_t i, page_size =getpagesize (); +char *ptr; +unsigned int usage, old_limit, old_memsw_limit; +int status, pid, retries =10 ; + +SAFE_MKDIR (cgroup_path,0777 ); +SAFE_FILE_PRINTF (tasks_path,"%i" ,getpid ()); + + ptr =SAFE_MMAP (NULL ,PAGES * page_size,PROT_READ |PROT_WRITE , +MAP_PRIVATE |MAP_ANONYMOUS ,-1 ,0 ); + +for (i =0 ; i <PAGES * page_size; i++) + ptr[i] ='a' ;
testcases/kernel/syscalls/madvise/madvise09.c line 119
Quoted 10 lines
testcases/kernel/syscalls/madvise/madvise09.c new file +10
-0+ +if (madvise (ptr,PAGES * page_size,MADV_FREE )) { +if (errno ==EINVAL ) +tst_brk (TCONF |TERRNO ,"MADV_FREE is not supported" ); + +tst_brk (TBROK |TERRNO ,"MADV_FREE failed" ); + } + +if (ptr[page_size] !='a' ) +tst_res (TFAIL ,"MADV_FREE pages were freed immediatelly" );
This could be problem on swapless system, from man 2 madvise:
"On a swapless system, freeing pages in a given range happens instantly, regardless of memory pressure."
Quoted 13 lines
testcases/kernel/syscalls/madvise/madvise09.c new file +13
-0+else +tst_res (TPASS ,"MADV_FREE pages were not freed immediatelly" ); + + ptr[0 ] ='b' ; + ptr[10 * page_size] ='b' ; + + usage = (1024 *1024 ); +tst_res (TINFO ,"Setting memory limits to %u %u" ,2 * usage,4 * usage); + +SAFE_FILE_SCANF (limit_in_bytes_path,"%u" , &old_limit); +SAFE_FILE_SCANF (memsw_limit_in_bytes_path,"%u" , &old_memsw_limit); +SAFE_FILE_PRINTF (limit_in_bytes_path,"%u" ,2 * usage); +SAFE_FILE_PRINTF (memsw_limit_in_bytes_path,"%u" ,4 * usage);
testcases/kernel/syscalls/madvise/madvise09.c line 142
Quoted 10 lines
testcases/kernel/syscalls/madvise/madvise09.c new file +10
-0+ +do { + pid =SAFE_FORK (); +if (!pid) +memory_pressure_child (); + +tst_res (TINFO ,"Memory hungry child %i started." , pid); + +SAFE_WAIT (&status); + }while (--retries >0 && ptr[page_size]);
Shouldn't this while break if _any_ of the pages is freed. This seems to check only 2nd one.
Regards,
Jan
PATCH 1/1 madvise09: Add MADV_FREE test
Hi!
Quoted 2 lines
Could we protect "the good child" via
oom_adjor similar so OOM doesn't touch it?
I was thinking about it, but haven't tried that since the testcase is complex enoug even without that and we would need to keep the retry in the test code anyway. And this kind of failure is actually pretty rare, I only caught it a few times when I was testing the test stability and did several thousands of reruns.
Quoted 5 lines
Quoted 3 lines
The second is that sometimes the memory hungry child
is killed too fast (before the kernel has chance to free the pages), so we rerun it a (for a few times) if that happens.How about we progressively make each retry slower? [1]
Sounds reasonable.
Quoted 8 lines
Quoted 8 lines
The test expects memory cgroup mounted in the standard
/sys/fs/cgroup/path, which is OK since the functionality tested was added to kernel 4.5 and the test would be skipped on older distros anyway.Also the test expects that the
MADV_FREEpages will not be freed immediatelly hence the test will fail if the whole system is under memory pressure.
Quoted 5 lines
Quoted 1 line
The memory limits were choosen to be 2MB and 4MB for memsw limit.
This looks a bit small to me. Have you checked how much is used just after fork on a system with 64k pages? My concern is that we hit this limit before we do anything.
Hmm, good point, I only tried it on x86_64 so far where a forked child takes 64 pages (4k). I guess that we can safely bump the limits to 8MB and 16MB which should be more than enough.
Quoted 6 lines
Quoted 6 lines
testcases/kernel/syscalls/madvise/madvise09.c new file +6
-0+static void memory_pressure_child (void ) +{ +size_t i, page_size =getpagesize (); +char *ptr; + +for (;;) {
Quoted 8 lines
testcases/kernel/syscalls/madvise/madvise09.c line 78
Quoted 5 lines
testcases/kernel/syscalls/madvise/madvise09.c new file +5
-0+ ptr =mmap (NULL ,1000 * page_size,PROT_READ |PROT_WRITE , +MAP_PRIVATE |MAP_ANONYMOUS ,-1 ,0 ); + +for (i =0 ; i <1000 * page_size; i++) + ptr[i] = i %100 ;[1] some kind of sleep here, so that retries will get gradually slower, giving kernel more time to free marked pages.
Sure.
Quoted 37 lines
Quoted 37 lines
testcases/kernel/syscalls/madvise/madvise09.c new file +37
-0+ } + +abort (); +} + +static void setup_cgroup_paths (int pid) +{ +snprintf (cgroup_path,sizeof (cgroup_path), +MEMCG_PATH "ltp_madvise09_%i/" , pid); +snprintf (tasks_path,sizeof (tasks_path),"%s/tasks" , cgroup_path); +snprintf (limit_in_bytes_path,sizeof (limit_in_bytes_path), +"%s/memory.limit_in_bytes" , cgroup_path); +snprintf (memsw_limit_in_bytes_path,sizeof (memsw_limit_in_bytes_path), +"%s/memory.memsw.limit_in_bytes" , cgroup_path); +} + +static void child (void ) +{ +size_t i, page_size =getpagesize (); +char *ptr; +unsigned int usage, old_limit, old_memsw_limit; +int status, pid, retries =10 ; + +SAFE_MKDIR (cgroup_path,0777 ); +SAFE_FILE_PRINTF (tasks_path,"%i" ,getpid ()); + + ptr =SAFE_MMAP (NULL ,PAGES * page_size,PROT_READ |PROT_WRITE , +MAP_PRIVATE |MAP_ANONYMOUS ,-1 ,0 ); + +for (i =0 ; i <PAGES * page_size; i++) + ptr[i] ='a' ; + +if (madvise (ptr,PAGES * page_size,MADV_FREE )) { +if (errno ==EINVAL ) +tst_brk (TCONF |TERRNO ,"MADV_FREE is not supported" ); + +tst_brk (TBROK |TERRNO ,"MADV_FREE failed" );
Quoted 8 lines
testcases/kernel/syscalls/madvise/madvise09.c line 119
Quoted 4 lines
testcases/kernel/syscalls/madvise/madvise09.c new file +4
-0+ } + +if (ptr[page_size] !='a' ) +tst_res (TFAIL ,"MADV_FREE pages were freed immediatelly" );This could be problem on swapless system, from man 2 madvise:
"On a swapless system, freeing pages in a given range happens instantly, regardless of memory pressure."
Right, we need to detect swapless system and adjust the testcase accordingly.
Quoted 18 lines
Quoted 18 lines
testcases/kernel/syscalls/madvise/madvise09.c new file +18
-0+else +tst_res (TPASS ,"MADV_FREE pages were not freed immediatelly" ); + + ptr[0 ] ='b' ; + ptr[10 * page_size] ='b' ; + + usage = (1024 *1024 ); +tst_res (TINFO ,"Setting memory limits to %u %u" ,2 * usage,4 * usage); + +SAFE_FILE_SCANF (limit_in_bytes_path,"%u" , &old_limit); +SAFE_FILE_SCANF (memsw_limit_in_bytes_path,"%u" , &old_memsw_limit); +SAFE_FILE_PRINTF (limit_in_bytes_path,"%u" ,2 * usage); +SAFE_FILE_PRINTF (memsw_limit_in_bytes_path,"%u" ,4 * usage); + + do { + pid = SAFE_FORK(); + if (!pid) + memory_pressure_child();
Quoted 8 lines
testcases/kernel/syscalls/madvise/madvise09.c line 142
Quoted 5 lines
testcases/kernel/syscalls/madvise/madvise09.c new file +5
-0+ +tst_res (TINFO ,"Memory hungry child %i started." , pid); + +SAFE_WAIT (&status); + }while (--retries >0 && ptr[page_size]);Shouldn't this while break if _any_ of the pages is freed. This seems to check only 2nd one.
In practice either all or none pages are freed 99% of the time (if the memory pressure child is killed too fast and freeing of these pages haven't had time to kick in). So I haven't bothered with adding a function that loops over the array and checks pages. In highly unlikely scenario we just fail to detect that only part of the pages was freed and do one more loop. I guess that we can make it more robust by checking if any of the pages was freed here.
Signature · 2 lines
Cyril Hrubis [email protected]