syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Kernel version is 5.4.81+,the available RAM is less than free,as follow:
[root@liuxp mywork]# head /proc/meminfo
MemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kB
So use available RAM to avoid OOM killer.
--- lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-)
lib/tst_memutils.c +26 -3
@@ -10,14 +10,33 @@#define TST_NO_DEFAULT_MAIN #include "tst_test.h" +#include "tst_safe_stdio.h" #define BLOCKSIZE (16 * 1024 * 1024) +static unsigned long get_available_ram (void ) +{ +char buf[60 ];/* actual lines we expect are ~30 chars or less */ +unsigned long available_kb =0 ; +FILE *fp; + + fp =SAFE_FOPEN ("/proc/meminfo" ,"r" ); +while (fgets (buf,sizeof (buf), fp)) { +if (sscanf (buf,"MemAvailable: %lu %*s\n" , &available_kb) ==1 ){ +break ; + } + } +SAFE_FCLOSE (fp); + +return 1024 * available_kb; +} +void tst_pollute_memory (size_t maxsize,int fillchar) {size_t i, map_count =0 , safety =0 , blocksize =BLOCKSIZE ;void **map_blocks;struct sysinfo info; +unsigned long available_ram =get_available_ram ();SAFE_SYSINFO (&info); safety =MAX (4096 *SAFE_SYSCONF (_SC_PAGESIZE),128 *1024 *1024 );
@@ -26,15 +45,19 @@ void tst_pollute_memory(size_t maxsize, int fillchar)if (info.freeswap > safety) safety =0 ; +/*"MemAvailable" field maybe not exist, or freeram less than available_ram*/ +if (available_ram ==0 || info.freeram < available_ram) + available_ram = info.freeram; +/* Not enough free memory to avoid invoking OOM killer */ -if (info.freeram <= safety) +if (available_ram <= safety)return ;if (!maxsize) maxsize =SIZE_MAX ; -if (info.freeram - safety < maxsize / info.mem_unit) - maxsize = (info.freeram - safety) * info.mem_unit; +if (available_ram - safety < maxsize / info.mem_unit) + maxsize = (available_ram - safety) * info.mem_unit; blocksize =MIN (maxsize, blocksize); map_count = maxsize / blocksize;
Signature · 1 line
1.8.3.1
PATCH 2/2 syscalls/ioctl: ioctl_sg01.c:TBROK: Test killed!
[root@test ioctl]# ./ioctl_sg01
tst_test.c:1263: TINFO: Timeout per run is 0h 05m 00s ioctl_sg01.c:81: TINFO: Found SCSI device /dev/sg2
Test timeouted, sending SIGKILL!
Test timeouted, sending SIGKILL!
Test timeouted, sending SIGKILL!
Test timeouted, sending SIGKILL!
tst_test.c:1308: TINFO: If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1
tst_test.c:1309: TBROK: Test killed! (timeout?)
[root@test ioctl]# lscpu |grep Model
Model: 85 Model name: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
[root@test ioctl]# free -h
total used free shared buff/cache available
Mem: 502G 2.7G 499G 82M 820M 497G Swap: 0B 0B 0B
512G RAM is too common,so add timeout 600 seconds,result is TPASS.
--- testcases/kernel/syscalls/ioctl/ioctl_sg01.c | 1 + 1 file changed, 1 insertion(+)
testcases/kernel/syscalls/ioctl/ioctl_sg01.c +1 -0
@@ -123,6 +123,7 @@ static void run(void) }static struct tst_test test = { + .timeout =600 , .test_all = run, .setup = setup, .cleanup = cleanup,
Signature · 1 line
1.8.3.1
PATCH 2/2 syscalls/ioctl: ioctl_sg01.c:TBROK: Test killed!
Quoted 33 lines · Xinpeng Liu
[root@test ioctl]# ./ioctl_sg01
tst_test.c:1263: TINFO: Timeout per run is 0h 05m 00s ioctl_sg01.c:81: TINFO: Found SCSI device/dev/sg2
Test timeouted, sending SIGKILL!
Test timeouted, sending SIGKILL!
Test timeouted, sending SIGKILL!
Test timeouted, sending SIGKILL!
tst_test.c:1308: TINFO: If you are running on slow machine, try exportingLTP_TIMEOUT_MUL> 1
tst_test.c:1309: TBROK: Test killed! (timeout?)[root@test ioctl]# lscpu |grep Model
Model: 85 Model name: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz[root@test ioctl]# free -h
total used free shared buff/cache availableMem: 502G 2.7G 499G 82M 820M 497G Swap: 0B 0B 0B512G RAM is too common,so add timeout 600 seconds,result is TPASS.
Signed-off-by Xinpeng Liu <[email protected]>--- testcases/kernel/syscalls/ioctl/ioctl_sg01.c | 1 + 1 file changed, 1 insertion(+)testcases/kernel/syscalls/ioctl/ioctl_sg01.c +0
-0
testcases/kernel/syscalls/ioctl/ioctl_sg01.c static void run(void) · line 127
Quoted 5 lines
testcases/kernel/syscalls/ioctl/ioctl_sg01.c +1
-0@@ -123,6 +123,7 @@ static void run(void) }static struct tst_test test = { + .timeout =600 ,
It's hard to say 600s is proper for other huge systems.
What about setting .timeout = -1 to disable it like what we did for the oom0* series test:
https://github.com/linux-test-project/ltp/blob/master/lib/tst_test.c#L1251
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/f4afeaf0/attachment.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
In reply to Xinpeng LiuHi Xinpeng,
Quoted 40 lines · Xinpeng Liu
Kernel version is 5.4.81+,the available RAM is less than free,as follow:
[root@liuxp mywork]# head/proc/meminfoMemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kBSo use available RAM to avoid OOM killer.
Signed-off-by Xinpeng Liu <[email protected]>--- lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-)lib/tst_memutils.c +15
-0@@ -10,14 +10,33 @@#define TST_NO_DEFAULT_MAIN #include "tst_test.h" +#include "tst_safe_stdio.h" #define BLOCKSIZE (16 * 1024 * 1024) +static unsigned long get_available_ram (void ) +{ +char buf[60 ];/* actual lines we expect are ~30 chars or less */ +unsigned long available_kb =0 ; +FILE *fp; + + fp =SAFE_FOPEN ("/proc/meminfo" ,"r" ); +while (fgets (buf,sizeof (buf), fp)) { +if (sscanf (buf,"MemAvailable: %lu %*s\n" , &available_kb) ==1 ){ +break ; + } + } +SAFE_FCLOSE (fp); +
lib/tst_memutils.c line 41
Quoted 10 lines
lib/tst_memutils.c +4
-0+return 1024 * available_kb; +} +void tst_pollute_memory (size_t maxsize,int fillchar) {size_t i, map_count =0 , safety =0 , blocksize =BLOCKSIZE ;void **map_blocks;struct sysinfo info; +unsigned long available_ram =get_available_ram ();
LTP provides SAFE_READ_MEMINFO() macro to be used in /proc/meminfo reading. See:
https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85
Quoted 35 lines
lib/tst_memutils.c +0
-0SAFE_SYSINFO (&info);safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024);lib/tst_memutils.c +1
-0@@ -26,15 +45,19 @@ void tst_pollute_memory(size_t maxsize, int fillchar)if (info.freeswap > safety) safety =0 ; +/*"MemAvailable" field maybe not exist, or freeram less than available_ram*/
lib/tst_memutils.c +6
-4+if (available_ram ==0 || info.freeram < available_ram) + available_ram = info.freeram; +/* Not enough free memory to avoid invoking OOM killer */ -if (info.freeram <= safety) +if (available_ram <= safety)return ;if (!maxsize) maxsize =SIZE_MAX ; -if (info.freeram - safety < maxsize / info.mem_unit) - maxsize = (info.freeram - safety) * info.mem_unit; +if (available_ram - safety < maxsize / info.mem_unit) + maxsize = (available_ram - safety) * info.mem_unit; blocksize =MIN (maxsize, blocksize); map_count = maxsize / blocksize; --1.8.3.1
--
Mailing list info: https://lists.linux.it/listinfo/ltp
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/e340dd17/attachment.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
ok?thanks for your direction!
From: Li Wang
Date: 2021-01-27 12:27
To: Xinpeng Liu
CC: LTP List
Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi Xinpeng,
On Wed, Jan 27, 2021 at 11:28 AM Xinpeng Liu <[email protected]> wrote:
Kernel version is 5.4.81+,the available RAM is less than free,as follow:
[root@liuxp mywork]# head /proc/meminfo
MemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kB
So use available RAM to avoid OOM killer.
--- lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-)
lib/tst_memutils.c +19 -0
@@ -10,14 +10,33 @@#define TST_NO_DEFAULT_MAIN #include "tst_test.h" +#include "tst_safe_stdio.h" #define BLOCKSIZE (16 * 1024 * 1024) +static unsigned long get_available_ram (void ) +{ +char buf[60 ];/* actual lines we expect are ~30 chars or less */ +unsigned long available_kb =0 ; +FILE *fp; + + fp =SAFE_FOPEN ("/proc/meminfo" ,"r" ); +while (fgets (buf,sizeof (buf), fp)) { +if (sscanf (buf,"MemAvailable: %lu %*s\n" , &available_kb) ==1 ){ +break ; + } + } +SAFE_FCLOSE (fp); + +return 1024 * available_kb; +} +void tst_pollute_memory (size_t maxsize,int fillchar) {size_t i, map_count =0 , safety =0 , blocksize =BLOCKSIZE ;void **map_blocks;struct sysinfo info; +unsigned long available_ram =get_available_ram ();
LTP provides SAFE_READ_MEMINFO() macro to be used in /proc/meminfo reading.
See: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85
SAFE_SYSINFO(&info);
safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024);@@ -26,15 +45,19 @@ void tst_pollute_memory(size_t maxsize, int fillchar)
if (info.freeswap > safety)
safety = 0;
+ /*"MemAvailable" field maybe not exist, or freeram less than available_ram*/
+ if(available_ram == 0 || info.freeram < available_ram)
+ available_ram = info.freeram;
+
/* Not enough free memory to avoid invoking OOM killer */
- if (info.freeram <= safety)
+ if (available_ram <= safety)
return;
if (!maxsize)
maxsize = SIZE_MAX;- if (info.freeram - safety < maxsize / info.mem_unit)
- maxsize = (info.freeram - safety) * info.mem_unit;
+ if (available_ram - safety < maxsize / info.mem_unit)
+ maxsize = (available_ram - safety) * info.mem_unit;
blocksize = MIN(maxsize, blocksize);
map_count = maxsize / blocksize;Signature · 14 lines
1.8.3.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/1e9d01f7/attachment-0001.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
In reply to Li WangHi Li,
Have a question about using macro SAFE_READ_MEMINFO get MemAvailable value, Some old kernels maybe not privode "MemAvailable" field, which will broken.
From: Li Wang
Date: 2021-01-27 12:27
To: Xinpeng Liu
CC: LTP List
Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi Xinpeng,
On Wed, Jan 27, 2021 at 11:28 AM Xinpeng Liu <[email protected]> wrote:
Kernel version is 5.4.81+,the available RAM is less than free,as follow:
[root@liuxp mywork]# head /proc/meminfo
MemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kB
So use available RAM to avoid OOM killer.
--- lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-)
lib/tst_memutils.c +19 -0
@@ -10,14 +10,33 @@#define TST_NO_DEFAULT_MAIN #include "tst_test.h" +#include "tst_safe_stdio.h" #define BLOCKSIZE (16 * 1024 * 1024) +static unsigned long get_available_ram (void ) +{ +char buf[60 ];/* actual lines we expect are ~30 chars or less */ +unsigned long available_kb =0 ; +FILE *fp; + + fp =SAFE_FOPEN ("/proc/meminfo" ,"r" ); +while (fgets (buf,sizeof (buf), fp)) { +if (sscanf (buf,"MemAvailable: %lu %*s\n" , &available_kb) ==1 ){ +break ; + } + } +SAFE_FCLOSE (fp); + +return 1024 * available_kb; +} +void tst_pollute_memory (size_t maxsize,int fillchar) {size_t i, map_count =0 , safety =0 , blocksize =BLOCKSIZE ;void **map_blocks;struct sysinfo info; +unsigned long available_ram =get_available_ram ();
LTP provides SAFE_READ_MEMINFO() macro to be used in /proc/meminfo reading.
See: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85
SAFE_SYSINFO(&info);
safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024);@@ -26,15 +45,19 @@ void tst_pollute_memory(size_t maxsize, int fillchar)
if (info.freeswap > safety)
safety = 0;
+ /*"MemAvailable" field maybe not exist, or freeram less than available_ram*/
+ if(available_ram == 0 || info.freeram < available_ram)
+ available_ram = info.freeram;
+
/* Not enough free memory to avoid invoking OOM killer */
- if (info.freeram <= safety)
+ if (available_ram <= safety)
return;
if (!maxsize)
maxsize = SIZE_MAX;- if (info.freeram - safety < maxsize / info.mem_unit)
- maxsize = (info.freeram - safety) * info.mem_unit;
+ if (available_ram - safety < maxsize / info.mem_unit)
+ maxsize = (available_ram - safety) * info.mem_unit;
blocksize = MIN(maxsize, blocksize);
map_count = maxsize / blocksize;Signature · 14 lines
1.8.3.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/bed16118/attachment-0001.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Hi Xinpeng,
I sent to you the case swapping01 solving this(via FILE_LINES_SCANF) already, feel free to take an reference:
https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85
Quoted 117 lines · On Wed, Jan 27, 2021 at 2:54 PM [email protected] <…
Hi Li,
Have a question about using macroSAFE_READ_MEMINFOget MemAvailable value,
Some old kernels maybe not privode "MemAvailable" field, which will broken.*From:* Li Wang <[email protected]>
*Date:* 2021-01-27 12:27
*To:* Xinpeng Liu <[email protected]>
*CC:* LTP List <[email protected]>
*Subject:* Re: [LTP] [PATCH 1/2] syscalls/ioctl:ioctl_sg01.c:ioctl_sg01invoked oom-killer
Hi Xinpeng,Quoted 40 lines · Xinpeng Liu
Kernel version is 5.4.81+,the available RAM is less than free,as follow:
[root@liuxp mywork]# head/proc/meminfoMemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kBSo use available RAM to avoid OOM killer.
Signed-off-by Xinpeng Liu <[email protected]>--- lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-)lib/tst_memutils.c +15
-0@@ -10,14 +10,33 @@#define TST_NO_DEFAULT_MAIN #include "tst_test.h" +#include "tst_safe_stdio.h" #define BLOCKSIZE (16 * 1024 * 1024) +static unsigned long get_available_ram (void ) +{ +char buf[60 ];/* actual lines we expect are ~30 chars or less */ +unsigned long available_kb =0 ; +FILE *fp; + + fp =SAFE_FOPEN ("/proc/meminfo" ,"r" ); +while (fgets (buf,sizeof (buf), fp)) { +if (sscanf (buf,"MemAvailable: %lu %*s\n" , &available_kb) ==1 ){ +break ; + } + } +SAFE_FCLOSE (fp); +lib/tst_memutils.c line 41
Quoted 10 lines
lib/tst_memutils.c +4
-0+return 1024 * available_kb; +} +void tst_pollute_memory (size_t maxsize,int fillchar) {size_t i, map_count =0 , safety =0 , blocksize =BLOCKSIZE ;void **map_blocks;struct sysinfo info; +unsigned long available_ram =get_available_ram ();LTP provides
SAFE_READ_MEMINFO()macro to be used in/proc/meminforeading. See:
https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85Quoted 32 lines
lib/tst_memutils.c +0
-0SAFE_SYSINFO (&info);safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 *1024);
lib/tst_memutils.c +1
-0@@ -26,15 +45,19 @@ void tst_pollute_memory(size_t maxsize, int fillchar)if (info.freeswap > safety) safety =0 ; +/*"MemAvailable" field maybe not exist, or freeram less than available_ram*/
lib/tst_memutils.c +6
-4+if (available_ram ==0 || info.freeram < available_ram) + available_ram = info.freeram; +/* Not enough free memory to avoid invoking OOM killer */ -if (info.freeram <= safety) +if (available_ram <= safety)return ;if (!maxsize) maxsize =SIZE_MAX ; -if (info.freeram - safety < maxsize / info.mem_unit) - maxsize = (info.freeram - safety) * info.mem_unit; +if (available_ram - safety < maxsize / info.mem_unit) + maxsize = (available_ram - safety) * info.mem_unit; blocksize =MIN (maxsize, blocksize); map_count = maxsize / blocksize; --1.8.3.1
Quoted 4 lines
--
Mailing list info: https://lists.linux.it/listinfo/ltp--
Regards,
Li Wang
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/d99fe5a0/attachment.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Li Wang <[email protected]> wrote:
Hi Xinpeng,
Quoted 6 lines
I sent to you the case swapping01 solving
this(via FILE_LINES_SCANF)already, feel free to take an reference:https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85
Or, maybe we can extract this process into a function and put it in tst_memutils.h, to convinently reuse by other testcases too?
void tst_get_MemAvailable(void);
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/3b37c2f4/attachment.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Hi!
Quoted 10 lines
Quoted 5 lines
I sent to you the case swapping01 solving
this(via FILE_LINES_SCANF)already, feel free to take an reference:https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85
Or, maybe we can extract this process into a function and put it in
tst_memutils.h, to convinently reuse by other testcases too?void
tst_get_MemAvailable(void);
Please do not use CamelCase.
It should be tst_get_mem_available(void) and it should also return unsigned long.
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Quoted 8 lines · Cyril Hrubis
Hi!
Quoted 4 lines
Quoted 4 lines
I sent to you the case swapping01 solving
this(via FILE_LINES_SCANF)already, feel free to take an reference:https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85
Quoted 2 lines
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Or, maybe we can extract this process into a function and put it in
tst_memutils.h, to convinently reuse by other testcases too?void
tst_get_MemAvailable(void);Please do not use CamelCase.
+1
Sorry, I just pasted the name by the editor, and YES, we should avoid that.
Quoted 3 lines
It should be
tst_get_mem_available(void)and it should also return unsigned long.
Absolutely right.
[email protected] <[email protected]> wrote:
In this testcase,we first check MemAvailable. If MemAvailable doesn't
Quoted 3 lines
exist,then use info.freeram.
Maybe not other cases need do these.
Yes, but we also could make use of tst_get_mem_available() here because, in the patch, you're trying to avoid that (MemFree > MemAvailable) situation.
If the fix is correct, we just need to get both and choose the smaller one to use, isn't it?
Quoted 6 lines
Have a question about using macro
SAFE_READ_MEMINFOget MemAvailableQuoted 4 lines
value,
Some old kernels maybe not privode "MemAvailable" field, which will broken.
The most different of SAFE_* macro is that will exit with TBROK if not get it expected. As Cyril proposes another one FILE_LINES_SCANF has no such concern.
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/22cc467d/attachment-0001.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
In reply to Li WangIn this testcase,we first check MemAvailable. If MemAvailable doesn't exist,then use info.freeram. Maybe not other cases need do these.
From: Li Wang
Date: 2021-01-27 15:58
To: [email protected]
CC: ltp
Subject: Re: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi Xinpeng,
I sent to you the case swapping01 solving this(via FILE_LINES_SCANF) already, feel free to take an reference:
https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85
On Wed, Jan 27, 2021 at 2:54 PM [email protected] <[email protected]> wrote:
Hi Li,
Have a question about using macro SAFE_READ_MEMINFO get MemAvailable value, Some old kernels maybe not privode "MemAvailable" field, which will broken.
From: Li Wang
Date: 2021-01-27 12:27
To: Xinpeng Liu
CC: LTP List
Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi Xinpeng,
On Wed, Jan 27, 2021 at 11:28 AM Xinpeng Liu <[email protected]> wrote:
Kernel version is 5.4.81+,the available RAM is less than free,as follow:
[root@liuxp mywork]# head /proc/meminfo
MemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kB
So use available RAM to avoid OOM killer.
--- lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-)
lib/tst_memutils.c +19 -0
@@ -10,14 +10,33 @@#define TST_NO_DEFAULT_MAIN #include "tst_test.h" +#include "tst_safe_stdio.h" #define BLOCKSIZE (16 * 1024 * 1024) +static unsigned long get_available_ram (void ) +{ +char buf[60 ];/* actual lines we expect are ~30 chars or less */ +unsigned long available_kb =0 ; +FILE *fp; + + fp =SAFE_FOPEN ("/proc/meminfo" ,"r" ); +while (fgets (buf,sizeof (buf), fp)) { +if (sscanf (buf,"MemAvailable: %lu %*s\n" , &available_kb) ==1 ){ +break ; + } + } +SAFE_FCLOSE (fp); + +return 1024 * available_kb; +} +void tst_pollute_memory (size_t maxsize,int fillchar) {size_t i, map_count =0 , safety =0 , blocksize =BLOCKSIZE ;void **map_blocks;struct sysinfo info; +unsigned long available_ram =get_available_ram ();
LTP provides SAFE_READ_MEMINFO() macro to be used in /proc/meminfo reading.
See: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/mem/swapping/swapping01.c#L85
SAFE_SYSINFO(&info);
safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024);@@ -26,15 +45,19 @@ void tst_pollute_memory(size_t maxsize, int fillchar)
if (info.freeswap > safety)
safety = 0;
+ /*"MemAvailable" field maybe not exist, or freeram less than available_ram*/
+ if(available_ram == 0 || info.freeram < available_ram)
+ available_ram = info.freeram;
+
/* Not enough free memory to avoid invoking OOM killer */
- if (info.freeram <= safety)
+ if (available_ram <= safety)
return;
if (!maxsize)
maxsize = SIZE_MAX;- if (info.freeram - safety < maxsize / info.mem_unit)
- maxsize = (info.freeram - safety) * info.mem_unit;
+ if (available_ram - safety < maxsize / info.mem_unit)
+ maxsize = (available_ram - safety) * info.mem_unit;
blocksize = MIN(maxsize, blocksize);
map_count = maxsize / blocksize;Signature · 19 lines
1.8.3.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp -- Regards, Li Wang -- Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/f3f4d4d7/attachment.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Quoted 10 lines
Signed-off-by Xinpeng Liu <[email protected]>--- lib/tst_memutils.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-)lib/tst_memutils.c +0
-0
lib/tst_memutils.c line 18
Quoted 9 lines
lib/tst_memutils.c +3
-0@@ -10,14 +10,33 @@#define TST_NO_DEFAULT_MAIN #include "tst_test.h" +#include "tst_safe_stdio.h" #define BLOCKSIZE (16 * 1024 * 1024) +static unsigned long get_available_ram (void ) +{
Can we prefix this function with tst_ and make it non-static?
I guess that there may be other tests that may use it later on.
lib/tst_memutils.c line 29
Quoted 11 lines
lib/tst_memutils.c +11
-0+char buf[60 ];/* actual lines we expect are ~30 chars or less */ +unsigned long available_kb =0 ; +FILE *fp; + + fp =SAFE_FOPEN ("/proc/meminfo" ,"r" ); +while (fgets (buf,sizeof (buf), fp)) { +if (sscanf (buf,"MemAvailable: %lu %*s\n" , &available_kb) ==1 ){ +break ; + } + } +SAFE_FCLOSE (fp);
Just use FILE_LINES_SCANF() instead.
Also we should fall back to something as 90% of (MemFree + Cached) here if MemAvailable is not present so that the function returns sensible number on older kernels as well.
lib/tst_memutils.c line 31
Quoted 1 line
lib/tst_memutils.c +1
-0+return 1024 * available_kb;
Can we just return kilobytes instead?
It will be less likely to overflow if we do all the calculations in kilobytes instead.
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
In reply to Xinpeng LiuHi!
Quoted 3 lines
MemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kB
This sounds really strange, usually MemFree is smaller than MemAvailable since MemFree does not include page cache.
Is this freshly booted system?
Can you send the whole meminfo here please?
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Quoted 11 lines · Cyril Hrubis
Hi!
Quoted 3 lines
MemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kBThis sounds really strange, usually MemFree is smaller than MemAvailable since MemFree does not include page cache.
Is this freshly booted system?
Can you send the whole meminfo here please?
Please also send the contents of /proc/sys/vm/min_free_kbytes. I suspect that that's where the OOM issues are actually coming from.
Signature · 7 lines
Martin Doucha [email protected] QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
[root@test-env-nm05-compute-14e5e72e38 ~]# w
18:05:00 up 15 days, 1:06, 3 users, load average: 0.39, 0.42, 0.47
From: Martin Doucha
Date: 2021-01-27 17:53
To: Cyril Hrubis; Xinpeng Liu
CC: ltp
Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Quoted 11 lines · Cyril Hrubis
Hi!
Quoted 3 lines
MemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kBThis sounds really strange, usually MemFree is smaller than MemAvailable since MemFree does not include page cache.
Is this freshly booted system?
Can you send the whole meminfo here please?
Please also send the contents of /proc/sys/vm/min_free_kbytes. I suspect that that's where the OOM issues are actually coming from.
Signature · 10 lines
Martin Doucha [email protected] QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/437569b4/attachment.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
In reply to Cyril Hrubis[root@test-env-nm05-compute-14e5e72e38 ~]# cat /proc/meminfo
MemTotal: 526997420 kB MemFree: 520224908 kB MemAvailable: 519936744 kB Buffers: 0 kB Cached: 2509036 kB SwapCached: 0 kB Active: 906868 kB Inactive: 2398084 kB Active(anon): 816396 kB Inactive(anon): 77236 kB Active(file): 90472 kB Inactive(file): 2320848 kB Unevictable: 610056 kB Mlocked: 610056 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 0 kB Writeback: 0 kB AnonPages: 1406336 kB Mapped: 118628 kB Shmem: 84400 kB KReclaimable: 193752 kB Slab: 703668 kB SReclaimable: 193752 kB SUnreclaim: 509916 kB KernelStack: 12672 kB PageTables: 14056 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 263498708 kB Committed_AS: 10263760 kB VmallocTotal: 34359738367 kB VmallocUsed: 651756 kB VmallocChunk: 0 kB Percpu: 62464 kB HardwareCorrupted: 88 kB AnonHugePages: 0 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB FileHugePages: 0 kB FilePmdMapped: 0 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 0 kB DirectMap4k: 1371808 kB DirectMap2M: 49661952 kB DirectMap1G: 486539264 kB
[root@test-env-nm05-compute-14e5e72e38 ~]# cat /proc/sys/vm/min_free_kbytes 90112
From: Cyril Hrubis
Date: 2021-01-27 17:39
To: Xinpeng Liu
CC: ltp
Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi!
Quoted 3 lines
MemTotal: 198101744 kB MemFree: 189303148 kB MemAvailable: 188566732 kB
This sounds really strange, usually MemFree is smaller than MemAvailable since MemFree does not include page cache.
Is this freshly booted system?
Can you send the whole meminfo here please?
Signature · 5 lines
Cyril Hrubis [email protected] -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/d67a18b3/attachment.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Quoted 2 lines · On 27. 01. 21 11:04, [email protected] wrote:
[root@test-env-nm05-compute-14e5e72e38 ~]# cat ?/proc/sys/vm/min_free_kbytes 90112
Yep, there it is. min_free_kbytes is 90MB but we only leave 64MB safety margin. I'll prepare a patch that'll increase safety margin to 2*min_free_kbytes if needed.
Signature · 7 lines
Martin Doucha [email protected] QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); now safety margin is 128MB,not 64MB. Right?
From: Martin Doucha
Date: 2021-01-27 19:28
To: [email protected]; Cyril Hrubis
CC: ltp
Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Quoted 2 lines · On 27. 01. 21 11:04, [email protected] wrote:
[root@test-env-nm05-compute-14e5e72e38 ~]# cat
/proc/sys/vm/min_free_kbytes90112
Yep, there it is. min_free_kbytes is 90MB but we only leave 64MB safety margin. I'll prepare a patch that'll increase safety margin to 2*min_free_kbytes if needed.
Signature · 10 lines
Martin Doucha [email protected] QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/43f122fb/attachment.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Quoted 2 lines · On 27. 01. 21 12:41, [email protected] wrote:
safety =
MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); now safety margin is 128MB,not 64MB. Right?
Yes, right, sorry.
Signature · 7 lines
Martin Doucha [email protected] QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
available memory can avoid to oom-killer.
#man free
available
Estimation of how much memory is available for starting new applications, without swapping. Unlike the data provided by the cache or free fields, this field takes into account page cache and also that not all reclaimable memory slabs will be reclaimed due to items being in use (MemAvailable in /proc/meminfo, available on kernels 3.14, emulated on kernels 2.6.27+, otherwise the same as free)
[root@bogon ltp]# cat my.diff
commit eb28176a3351c6854620aaa8248bf17edea210ae
Author: Xinpeng Liu <[email protected]> Date: Mon Jan 25 20:58:20 2021 +0800
syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Kernel version is 5.4.81+,the available RAM is less than free,as follow:
[root@liuxp mywork]# head /proc/meminfo
MemTotal: 198101744 kB
MemFree: 189303148 kB
MemAvailable: 188566732 kB
So use available RAM to avoid OOM killer.
include/tst_memutils.h +2 -0
@@ -6,6 +6,8 @@#ifndef TST_MEMUTILS_H__ #define TST_MEMUTILS_H__ +unsigned long tst_get_mem_available (void ); +/* * Fill up to maxsize physical memory with fillchar, then free it for reuse. * If maxsize is zero, fill as much memory as possible. This function is
lib/tst_memutils.c +20 -3
@@ -13,11 +13,21 @@#define BLOCKSIZE (16 * 1024 * 1024) +unsigned long tst_get_mem_available (void ) +{ +unsigned long available_kb =0 ; + +FILE_LINES_SCANF ("/proc/meminfo" ,"MemAvailable: %lu" , &available_kb); + +return available_kb; +} +void tst_pollute_memory (size_t maxsize,int fillchar) {size_t i, map_count =0 , safety =0 , blocksize =BLOCKSIZE ;void **map_blocks;struct sysinfo info; +unsigned long available_ram;SAFE_SYSINFO (&info); safety =MAX (4096 *SAFE_SYSCONF (_SC_PAGESIZE),128 *1024 *1024 );
@@ -26,15 +36,22 @@ void tst_pollute_memory(size_t maxsize, int fillchar)if (info.freeswap > safety) safety =0 ; + available_ram =1024 *tst_get_mem_available (); + available_ram /= info.mem_unit; + +/*"MemAvailable" field maybe not exist, or freeram less than available_ram*/ +if (available_ram ==0 || info.freeram < available_ram) + available_ram = info.freeram; +/* Not enough free memory to avoid invoking OOM killer */ -if (info.freeram <= safety) +if (available_ram <= safety)return ;if (!maxsize) maxsize =SIZE_MAX ; -if (info.freeram - safety < maxsize / info.mem_unit) - maxsize = (info.freeram - safety) * info.mem_unit; +if (available_ram - safety < maxsize / info.mem_unit) + maxsize = (available_ram - safety) * info.mem_unit; blocksize =MIN (maxsize, blocksize); map_count = maxsize / blocksize;
From: Martin Doucha
Date: 2021-01-27 19:46
To: [email protected]; Cyril Hrubis
CC: ltp
Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Quoted 2 lines · On 27. 01. 21 12:41, [email protected] wrote:
safety =
MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); now safety margin is 128MB,not 64MB. Right?
Yes, right, sorry.
Signature · 10 lines
Martin Doucha [email protected] QA Engineer for Software Maintenance SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210127/182cafa3/attachment.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
In reply to [email protected]Hi Xinpeng,
[root@test-env-nm05-compute-14e5e72e38 ~]# cat /proc/meminfo
Quoted 13 lines
MemTotal: 526997420 kB MemFree: 520224908 kB MemAvailable: 519936744 kB Buffers: 0 kB Cached: 2509036 kB SwapCached: 0 kB ... SwapTotal: 0 kB SwapFree: 0 kB ... CommitLimit: 263498708 kB Committed_AS: 10263760 kB
Quoted 4 lines
[root@test-env-nm05-compute-14e5e72e38 ~]# cat
/proc/sys/vm/min_free_kbytes
90112
After looking back on this problem, I prefer to think the reasons were caused by lower CommitLimit.
CommitLimit: 263498708 kB < MemAvailable: 519936744 kB
If you try to enable all swap-disk or reset to a high ratio in overcommit_ratio
to make it larger than MemAvailable, probably no OOM occurs anymore.
Btw, I also observed that ioctl_sg01 almost being killed by OOM every time on an aarch64 with little swap space, but if I add more swap or set a high value of overcommit_ratio, the problem is gone. (I manually tried with another x86_64 to confirm this too)
total used free shared buff/cache available
Mem: 259828 5365 247383 68 7079 231296 Swap: 4095 55 4040
---
MemTotal: 266063872 kB MemFree: 253320768 kB MemAvailable: 236848064 kB Buffers: 1472 kB Cached: 6755456 kB SwapCached: 12160 kB ... CommitLimit: 137226176 kB Committed_AS: 1206912 kB
---
The previous method in the patch[1] seems not good enough, but that can help to verify if OOM disappears when resetting the overcommit_ratio.
[1] http://lists.linux.it/pipermail/ltp/2021-February/020907.html
Hence, another improvement way based on the above is to allocate proper memory-size according to CommitLimit value when detecting the value of CommitLimit is less than MemAvailable. That will make the test happy with a little swap-space size system.
Any thoughts, or comments?
lib/tst_memutils.c +5 -0
@@ -36,6 +36,13 @@ void tst_pollute_memory(size_t maxsize, int fillchar)if (info.freeram - safety < maxsize / info.mem_unit) maxsize = (info.freeram - safety) * info.mem_unit; +/* +* To respect CommitLimit to prevent test invoking OOM killer, +* this may appear on system with a smaller swap-disk (or disabled). +*/ + if (SAFE_READ_MEMINFO("CommitLimit:") <
SAFE_READ_MEMINFO("MemAvailable:"))
+ maxsize = SAFE_READ_MEMINFO("CommitLimit:") * 1024 -
(safety * info.mem_unit);
+
blocksize = MIN(maxsize, blocksize);
map_count = maxsize / blocksize;
map_blocks = SAFE_MALLOC(map_count * sizeof(void *));========================
About the MemAvailable < MemFree, I think that is correct behavior on your system and not the OOM root-cause.
Generally, we assumed the MemAvailable higher than MemFree, but we sometimes also allow situations to break that. We'd better count all of the different free watermarks from /proc/zoneinfo, then add the sum of the low watermarks to MemAvailable, if get a value larger than MemFree, that should be OK from my perspective.
-----
# echo 675840 > /proc/sys/vm/min_free_kbytes
# cat /proc/meminfo |grep -i mem MemTotal: 5888584 kB MemFree: 4518064 kB MemAvailable: 3692008 kB Shmem: 21128 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB
# cat /proc/zoneinfo |grep low -B 3
...
pages free 3840
min 440
low 550
--
Node 0, zone DMA32
pages free 355602
min 79706
low 99632
--
Node 0, zone Normal
pages free 0
min 0
low 0
--
Node 0, zone Movable
pages free 0
min 0
low 0
--
Node 0, zone Device
pages free 0
min 0
low 0
--
Node 1, zone DMA
pages free 0
min 0
low 0
--
Node 1, zone DMA32
pages free 0
min 0
low 0
--
nr_kernel_misc_reclaimable 0
pages free 769192
min 88812
low 111015(111015+99632+550)*4 + 3692008(MemAvailable) > 5888584(MemFree)
Btw the formula to count MemAvailable is:
available = MemFree - totalreserve_pages + pages[LRU_ACTIVE_FILE] + pages[LRU_INACTIVE_FILE] - min(pagecache / 2, wmark_low)
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210304/07089c65/attachment-0001.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
lib/tst_memutils.c +7 -0
@@ -36,6 +36,13 @@ void tst_pollute_memory(size_t maxsize, int fillchar)if (info.freeram - safety < maxsize / info.mem_unit) maxsize = (info.freeram - safety) * info.mem_unit; ==>Thanks,but the maxsize original code need to be deleted,Right? +/* +* To respect CommitLimit to prevent test invoking OOM killer, +* this may appear on system with a smaller swap-disk (or disabled). +*/ +if (SAFE_READ_MEMINFO ("CommitLimit:" ) <SAFE_READ_MEMINFO ("MemAvailable:" )) + maxsize =SAFE_READ_MEMINFO ("CommitLimit:" ) *1024 - (safety * info.mem_unit); + blocksize =MIN (maxsize, blocksize);
map_count = maxsize / blocksize;
map_blocks = SAFE_MALLOC(map_count * sizeof(void *));Thanks?
From: Li Wang
Date: 2021-03-04 15:52
To: [email protected]
CC: Cyril Hrubis; ltp; Martin Doucha
Subject: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi Xinpeng,
[root@test-env-nm05-compute-14e5e72e38 ~]# cat /proc/meminfo
MemTotal: 526997420 kB MemFree: 520224908 kB MemAvailable: 519936744 kB Buffers: 0 kB Cached: 2509036 kB SwapCached: 0 kB ... SwapTotal: 0 kB SwapFree: 0 kB ... CommitLimit: 263498708 kB Committed_AS: 10263760 kB
[root@test-env-nm05-compute-14e5e72e38 ~]# cat /proc/sys/vm/min_free_kbytes 90112
After looking back on this problem, I prefer to think the reasons were caused by lower CommitLimit.
CommitLimit: 263498708 kB < MemAvailable: 519936744 kB
If you try to enable all swap-disk or reset to a high ratio in overcommit_ratio to make it larger than MemAvailable, probably no OOM occurs anymore.
Btw, I also observed that ioctl_sg01 almost being killed by OOM every time on an aarch64 with little swap space, but if I add more swap or set a high value of overcommit_ratio, the problem is gone. (I manually tried with another x86_64 to confirm this too)
total used free shared buff/cache available
Mem: 259828 5365 247383 68 7079 231296 Swap: 4095 55 4040--- MemTotal: 266063872 kB MemFree: 253320768 kB MemAvailable: 236848064 kB Buffers: 1472 kB Cached: 6755456 kB SwapCached: 12160 kB ... CommitLimit: 137226176 kB Committed_AS: 1206912 kB
---
The previous method in the patch[1] seems not good enough, but that can help to verify if OOM disappears when resetting the overcommit_ratio.
[1] http://lists.linux.it/pipermail/ltp/2021-February/020907.html
Hence, another improvement way based on the above is to allocate proper memory-size according to CommitLimit value when detecting the value of CommitLimit is less than MemAvailable. That will make the test happy with a little swap-space size system.
Any thoughts, or comments?
lib/tst_memutils.c +7 -0
@@ -36,6 +36,13 @@ void tst_pollute_memory(size_t maxsize, int fillchar)if (info.freeram - safety < maxsize / info.mem_unit) maxsize = (info.freeram - safety) * info.mem_unit; +/* +* To respect CommitLimit to prevent test invoking OOM killer, +* this may appear on system with a smaller swap-disk (or disabled). +*/ +if (SAFE_READ_MEMINFO ("CommitLimit:" ) <SAFE_READ_MEMINFO ("MemAvailable:" )) + maxsize =SAFE_READ_MEMINFO ("CommitLimit:" ) *1024 - (safety * info.mem_unit); + blocksize =MIN (maxsize, blocksize); map_count = maxsize / blocksize; map_blocks =SAFE_MALLOC (map_count *sizeof (void *));
========================
About the MemAvailable < MemFree, I think that is correct behavior on your system and not the OOM root-cause.
Generally, we assumed the MemAvailable higher than MemFree, but we sometimes also allow situations to break that. We'd better count all of the different free watermarks from /proc/zoneinfo, then add the sum of the low watermarks to MemAvailable, if get a value larger than MemFree, that should be OK from my perspective.
-----
# echo 675840 > /proc/sys/vm/min_free_kbytes
# cat /proc/meminfo |grep -i mem MemTotal: 5888584 kB MemFree: 4518064 kB MemAvailable: 3692008 kB Shmem: 21128 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB
# cat /proc/zoneinfo |grep low -B 3
...
pages free 3840
min 440
low 550
--
Node 0, zone DMA32
pages free 355602
min 79706
low 99632
--
Node 0, zone Normal
pages free 0
min 0
low 0
--
Node 0, zone Movable
pages free 0
min 0
low 0
--
Node 0, zone Device
pages free 0
min 0
low 0
--
Node 1, zone DMA
pages free 0
min 0
low 0
--
Node 1, zone DMA32
pages free 0
min 0
low 0
--
nr_kernel_misc_reclaimable 0
pages free 769192
min 88812
low 111015(111015+99632+550)*4 + 3692008(MemAvailable) > 5888584(MemFree)
Btw the formula to count MemAvailable is:
available = MemFree - totalreserve_pages + pages[LRU_ACTIVE_FILE] + pages[LRU_INACTIVE_FILE] - min(pagecache / 2, wmark_low)
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210305/35aaf9fc/attachment-0001.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Hi Xinpeng,
lib/tst_memutils.c void tst_pollute_memory(size_t maxsize, int fillchar) · line 40
Quoted 8 lines · On Fri, Mar 5, 2021 at 1:52 PM [email protected] <…
lib/tst_memutils.c +0
-0@@ -36,6 +36,13 @@ void tst_pollute_memory(size_t maxsize, int fillchar)if (info.freeram - safety < maxsize / info.mem_unit) maxsize = (info.freeram - safety) * info.mem_unit; ==>Thanks,but the maxsize original code need to be deleted,Right?
No, the maxsize code is also useful, it needs to define the value in most common situations(i.e. CommitLimit > MemAvailable).
But I'm still hesitating to use ComitLimit as the threshold for 'maxsize'. Because according to the Linux document, it says that only take effort when overcommit_memory is setting to 2. But our test system all set 0 by default.
"This limit is only adhered to if strict overcommit accounting is enabled
(mode 2 in 'vm.overcommit_memory')."
see:
https://github.com/torvalds/linux/blob/master/Documentation/filesystems/proc.rst
Seems to use CommitLimit looks a bit strict and strange to test.
And I even think the way to use MemAvailable is acceptable if MemFree > MemAvailable, just like what you did in your last patch.
I'm still not very sure so far~
(But one thing I can confirm that MemAvailable < MemFree is correct behavior sometimes)
Quoted 15 lines
lib/tst_memutils.c +3
-0+/* +* To respect CommitLimit to prevent test invoking OOM killer, +* this may appear on system with a smaller swap-disk (or disabled).
lib/tst_memutils.c +2
-0+*/ + if (SAFE_READ_MEMINFO("CommitLimit:") <
SAFE_READ_MEMINFO("MemAvailable:"))
+ maxsize =SAFE_READ_MEMINFO("CommitLimit:")* 1024 -
(safety * info.mem_unit);
+blocksize = MIN(maxsize, blocksize); map_count = maxsize / blocksize; map_blocks = SAFE_MALLOC(map_count * sizeof(void *));
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210305/d6f661f7/attachment.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Hi Li Wang,
I think your patch is good.
1.CommitLimit is the memory that can be allocated by application. 2.ioctl_sg01 in serveral machines with your patch,the result is passed.
Thanks!
From: Li Wang
Date: 2021-03-05 17:02
To: [email protected]
CC: Cyril Hrubis; ltp; mdoucha
Subject: Re: Re: [LTP] [PATCH 1/2] syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer Hi Xinpeng,
On Fri, Mar 5, 2021 at 1:52 PM [email protected] <[email protected]> wrote:
lib/tst_memutils.c +0 -0
@@ -36,6 +36,13 @@ void tst_pollute_memory(size_t maxsize, int fillchar)if (info.freeram - safety < maxsize / info.mem_unit) maxsize = (info.freeram - safety) * info.mem_unit; ==>Thanks,but the maxsize original code need to be deleted,Right?
No, the maxsize code is also useful, it needs to define the value in most common situations(i.e. CommitLimit > MemAvailable).
But I'm still hesitating to use ComitLimit as the threshold for 'maxsize'. Because according to the Linux document, it says that only take effort when overcommit_memory is setting to 2. But our test system all set 0 by default.
"This limit is only adhered to if strict overcommit accounting is enabled
(mode 2 in 'vm.overcommit_memory')."
see: https://github.com/torvalds/linux/blob/master/Documentation/filesystems/proc.rst
Seems to use CommitLimit looks a bit strict and strange to test.
And I even think the way to use MemAvailable is acceptable if MemFree > MemAvailable, just like what you did in your last patch.
I'm still not very sure so far~
(But one thing I can confirm that MemAvailable < MemFree is correct behavior sometimes)
+ /*
+ * To respect CommitLimit to prevent test invoking OOM killer,
+ * this may appear on system with a smaller swap-disk (or disabled).
+ */
+ if (SAFE_READ_MEMINFO("CommitLimit:") < SAFE_READ_MEMINFO("MemAvailable:"))
+ maxsize = SAFE_READ_MEMINFO("CommitLimit:") * 1024 - (safety * info.mem_unit);
+
blocksize = MIN(maxsize, blocksize);
map_count = maxsize / blocksize;
map_blocks = SAFE_MALLOC(map_count * sizeof(void *));Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210324/39164c88/attachment.htm>
PATCH 1/2 syscalls/ioctl: ioctl_sg01.c: ioctl_sg01 invoked oom-killer
Hi all,
Seems this ioctl_sg01 problem is caused by the kernel commit:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=8c7829b04c523cdc732cb77f59f03320e09f3386
I start a new thread for tracking it in Linux-MM just now:
https://lists.linux.it/pipermail/ltp/2021-April/021903.html
[email protected] <[email protected]> wrote:
Hi Li Wang,
Quoted 5 lines
I think your patch is good.
1.CommitLimit is the memory that can be allocated by application. 2.ioctl_sg01 in serveral machines with your patch,the result is passed.
Thank you Xinpeng,
That is just an eclectic workaround but not the key point of the OOM occurs issue.
I'd not suggest merging my patch to LTP:).
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210411/44f80803/attachment.htm>