Skip to contentSkip to conversation

madvise09:Change PAGES size to the value more than 32

PATCH New [email protected] 4 participants 8 messages Sep 4, 2019 – Sep 12, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 madvise09:Change PAGES size to the value more than 32

From: Shuang Qiu <[email protected]>

In upstream patch 1a61ab (mm: memcontrol: replace zone summing with lruvec_page_state()), it modify the lruvec state in batch,equal and less than 32 MADV_FREE pages will not trigger the account of lruvec_stat,and will not be free in memory pressure either. So the testcase may fail with:
...
madvise09.c:219: INFO: Memory hungry child 6178 started, try 10 madvise09.c:254: INFO: Memory map: pppppppppppppppppppppppppppppppp madvise09.c:259: FAIL: No MADV_FREE page was freed on low memory
...
Change the PAGES to the value more than 32 can fix such issue.

---
 testcases/kernel/syscalls/madvise/madvise09.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/syscalls/madvise/madvise09.c +1 -1
index 01075f6..3759053 100644
@@ -57,7 +57,7 @@ static int sleep_between_faults;
 
 static int swap_accounting_enabled;
 
-#define PAGES 32
+#define PAGES 64
 #define TOUCHED_PAGE1 0
 #define TOUCHED_PAGE2 10
 
Signature · 1 line
1.9.1
#1 lore raw

PATCH 1/1 madvise09:Change PAGES size to the value more than 32

Quoted 13 lines · On Wed, Sep 04, 2019 at 02:00:23PM +0800, [email protected] wrote:

From: Shuang Qiu <[email protected]>

In upstream patch 1a61ab (mm: memcontrol: replace zone summing with lruvec_page_state()), it modify the lruvec state in batch,equal and less than 32 MADV_FREE pages will not trigger the account of lruvec_stat,and will not be free in memory pressure either. So the testcase may fail with:
...
madvise09.c:219: INFO: Memory hungry child 6178 started, try 10 madvise09.c:254: INFO: Memory map: pppppppppppppppppppppppppppppppp madvise09.c:259: FAIL: No MADV_FREE page was freed on low memory
...
Change the PAGES to the value more than 32 can fix such issue.

Quoted 1 line

On ppc64le, I was getting retries, as OOM was getting hit. The test would ultimately time out. This fixes it for me.

Thanks.
Cascardo.

Quoted 23 lines
---
 testcases/kernel/syscalls/madvise/madvise09.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/syscalls/madvise/madvise09.c +1 -1
index 01075f6..3759053 100644
@@ -57,7 +57,7 @@ static int sleep_between_faults;
 
 static int swap_accounting_enabled;
 
-#define PAGES 32
+#define PAGES 64
 #define TOUCHED_PAGE1 0
 #define TOUCHED_PAGE2 10
 
Signature · 5 lines
1.9.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw

PATCH 1/1 madvise09:Change PAGES size to the value more than 32

In reply to [email protected]
Quoted 24 lines · On Wed, Sep 4, 2019 at 2:01 PM <[email protected]> wrote:

From: Shuang Qiu <[email protected]>

In upstream patch 1a61ab (mm: memcontrol: replace zone summing with lruvec_page_state()), it modify the lruvec state in batch,equal and less than 32 MADV_FREE pages will not trigger the account of lruvec_stat,and will not be free in memory pressure either. So the testcase may fail with:
...
madvise09.c:219: INFO: Memory hungry child 6178 started, try 10 madvise09.c:254: INFO: Memory map: pppppppppppppppppppppppppppppppp madvise09.c:259: FAIL: No MADV_FREE page was freed on low memory
...
Change the PAGES to the value more than 32 can fix such issue.

---
 testcases/kernel/syscalls/madvise/madvise09.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/syscalls/madvise/madvise09.c +0 -0
index 01075f6..3759053 100644

testcases/kernel/syscalls/madvise/madvise09.c static int sleep_between_faults; · line 60

Quoted 5 lines
testcases/kernel/syscalls/madvise/madvise09.c +1 -1
@@ -57,7 +57,7 @@ static int sleep_between_faults;
 
 static int swap_accounting_enabled;
 
-#define PAGES 32
+#define PAGES 64

I'm not sure why 64 pages is a proper value? Can you explain more?

Signature · 2 lines
Regards,
Li Wang
author
#3 lore raw

PATCH 1/1 madvise09:Change PAGES size to the value more than 32

Quoted 25 lines · Li Wang
Quoted 24 lines · On Wed, Sep 4, 2019 at 2:01 PM <[email protected]> wrote:

From: Shuang Qiu <[email protected]>

In upstream patch 1a61ab (mm: memcontrol: replace zone summing with lruvec_page_state()), it modify the lruvec state in batch,equal and less than 32 MADV_FREE pages will not trigger the account of lruvec_stat,and will not be free in memory pressure either. So the testcase may fail with:
...
madvise09.c:219: INFO: Memory hungry child 6178 started, try 10 madvise09.c:254: INFO: Memory map: pppppppppppppppppppppppppppppppp madvise09.c:259: FAIL: No MADV_FREE page was freed on low memory
...
Change the PAGES to the value more than 32 can fix such issue.

---
  testcases/kernel/syscalls/madvise/madvise09.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/syscalls/madvise/madvise09.c +0 -0
index 01075f6..3759053 100644
@@ -57,7 +57,7 @@ static int sleep_between_faults;
 
Quoted 5 lines

testcases/kernel/syscalls/madvise/madvise09.c static int sleep_between_faults; · line 60

Quoted 4 lines
testcases/kernel/syscalls/madvise/madvise09.c +1 -1
  static int swap_accounting_enabled;
 
-#define PAGES 32
+#define PAGES 64

I'm not sure why 64 pages is a proper value? Can you explain more?

I think any value which larger than 32 is ok.
So I tested 64 and works fine.

Thanks
Shuang

Quoted 1 line
#4 lore raw

PATCH 1/1 madvise09:Change PAGES size to the value more than 32

Quoted 27 lines · Shuang Qiu
Quoted 25 lines · Li Wang
Quoted 24 lines · On Wed, Sep 4, 2019 at 2:01 PM <[email protected]> wrote:

From: Shuang Qiu <[email protected]>

In upstream patch 1a61ab (mm: memcontrol: replace zone summing with lruvec_page_state()), it modify the lruvec state in batch,equal and less than 32 MADV_FREE pages will not trigger the account of lruvec_stat,and will not be free in memory pressure either. So the testcase may fail with:
...
madvise09.c:219: INFO: Memory hungry child 6178 started, try 10 madvise09.c:254: INFO: Memory map: pppppppppppppppppppppppppppppppp madvise09.c:259: FAIL: No MADV_FREE page was freed on low memory
...
Change the PAGES to the value more than 32 can fix such issue.

---
  testcases/kernel/syscalls/madvise/madvise09.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
testcases/kernel/syscalls/madvise/madvise09.c +0 -0
index 01075f6..3759053 100644
@@ -57,7 +57,7 @@ static int sleep_between_faults;
 
Quoted 7 lines
Quoted 5 lines

testcases/kernel/syscalls/madvise/madvise09.c static int sleep_between_faults; · line 60

Quoted 4 lines
testcases/kernel/syscalls/madvise/madvise09.c +1 -1
  static int swap_accounting_enabled;
 
-#define PAGES 32
+#define PAGES 64

I'm not sure why 64 pages is a proper value? Can you explain more?

I think any value which larger than 32 is ok.
So I tested 64 and works fine.

I just tried with '42' randomly, but it doesn't work. I guess that might have a waterline to trigger this memory page reclaiming.

I looked at the code calling path, it seems the 'sc->nr_reclaimed' is the key value to control that, but still not 100% sure, I will keep debugging it for a while.

try_to_free_mem_cgroup_pages
do_try_to_free_pages
shrink_zone
shrink_node
shrink_node_memcg
get_scan_count
lruvec_lru_size
lruvec_page_state

Signature · 2 lines
Regards,
Li Wang
#5 lore raw

PATCH 1/1 madvise09:Change PAGES size to the value more than 32

In reply to Li Wang

Hi!

Quoted 1 line

I'm not sure why 64 pages is a proper value? Can you explain more?

I've just talked to Michal Hocko and the conclusion is that we should allocate several megabytes to avoid depending kernel implementation details, which would mean allocating thousands of pages.

Signature · 2 lines
Cyril Hrubis
[email protected]
#6 lore raw

PATCH 1/1 madvise09:Change PAGES size to the value more than 32

Quoted 7 lines · Cyril Hrubis

Hi!

Quoted 1 line

I'm not sure why 64 pages is a proper value? Can you explain more?

I've just talked to Michal Hocko and the conclusion is that we should allocate several megabytes to avoid depending kernel implementation details, which would mean allocating thousands of pages.

Sounds reasonable. Allocating thousands of pages must trigger the MADV_FREE page reclaiming.

@shuang.qiu, Hi Shuang, would you mind to rewrite the patch, or should I do that?

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20190912/2100c785/attachment.htm>
author
#7 lore raw

PATCH 1/1 madvise09:Change PAGES size to the value more than 32

Quoted 13 lines · Li Wang

On Wed, Sep 11, 2019 at 7:19 PM Cyril Hrubis <[email protected] <mailto:[email protected]>> wrote:

Hi!
> I'm not sure why 64 pages is a proper value? Can you explain more?

I've just talked to Michal Hocko and the conclusion is that we should allocate several megabytes to avoid depending kernel implementation details, which would mean allocating thousands of pages.

Quoted 5 lines

Sounds reasonable. Allocating thousands of pages must trigger the?MADV_FREE page reclaiming.

@shuang.qiu, Hi Shuang, would you mind to rewrite the patch, or should I do that?

Thanks for helping to investigate this issue.
I will update the patch.
32 pages is really small in current system which may be held on reclaiming small piece of memory,thousands of pages is reasonable.

Thanks
Shuang

Quoted 4 lines
Signature · 3 lines
-- 
Regards,
Li Wang

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20190912/4465138e/attachment.htm>