Skip to contentSkip to conversation

memcontrol03: Using clean page cache to avoid dependency on IO rate

PATCH Superseded Wei Gao via ltp 3 participants 6 messages Mar 8, 2024 – Mar 15, 2024

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 memcontrol03: Using clean page cache to avoid dependency on IO rate

Bad IO situation(storage bandwidth ~10MB/sec) will lead background writeback has uncertain progress for dirty page. So system can not reclaim enough memory for new process and finally lead a unexpected OOM.

memcontrol03.c:218: TPASS: Expect: (A/B/E memory.current=0) ~= 0 memcontrol03.c:116: TPASS: Child 1918 killed by OOM
memcontrol03.c:224: TPASS: Expect: (A/B memory.current=52588544) ~= 52428800 memcontrol03.c:129: TFAIL: Expected child 1944 to exit(0), but instead killed by SIGKILL

---
 testcases/kernel/controllers/memcg/memcontrol03.c | 2 ++
 1 file changed, 2 insertions(+)
testcases/kernel/controllers/memcg/memcontrol03.c +2 -0
index 9c6c808e0..9903ba82b 100644
@@ -201,6 +201,7 @@ static void test_memcg_min(void)
 		sleep(1);
 	}
 
+	sync();
 	alloc_anon_in_child(trunk_cg[G], MB(148), 0);
 
 	SAFE_CG_SCANF(trunk_cg[B], "memory.current", "%ld", c);
@@ -217,6 +218,7 @@ static void test_memcg_min(void)
 	TST_EXP_EXPR(values_close(c[2], 0, 1),
 		     "(A/B/E memory.current=%ld) ~= 0", c[2]);
 
+	sync();
 	alloc_anon_in_child(trunk_cg[G], MB(170), 1);
 
 	SAFE_CG_SCANF(trunk_cg[B], "memory.current", "%ld", c);
Signature · 5 lines
2.35.3


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

Hi!

Quoted 9 lines

Bad IO situation(storage bandwidth ~10MB/sec) will lead background writeback has uncertain progress for dirty page. So system can not reclaim enough memory for new process and finally lead a unexpected OOM.

memcontrol03.c:218: TPASS: Expect: (A/B/E memory.current=0) ~= 0 memcontrol03.c:116: TPASS: Child 1918 killed by OOM
memcontrol03.c:224: TPASS: Expect: (A/B memory.current=52588544) ~= 52428800 memcontrol03.c:129: TFAIL: Expected child 1944 to exit(0), but instead killed by SIGKILL

This is strange the order of messages here does not make any sense. the last line in the log shouldn't be there at all, the test should have ended with the "Expect: (A/B memory.current=52588544) ~= 52428800"

Can you please send the whole log?

Quoted 25 lines
---
 testcases/kernel/controllers/memcg/memcontrol03.c | 2 ++
 1 file changed, 2 insertions(+)
testcases/kernel/controllers/memcg/memcontrol03.c +2 -0
index 9c6c808e0..9903ba82b 100644
@@ -201,6 +201,7 @@ static void test_memcg_min(void)
 		sleep(1);
 	}
 
+	sync();
 	alloc_anon_in_child(trunk_cg[G], MB(148), 0);
 
 	SAFE_CG_SCANF(trunk_cg[B], "memory.current", "%ld", c);
@@ -217,6 +218,7 @@ static void test_memcg_min(void)
 	TST_EXP_EXPR(values_close(c[2], 0, 1),
 		     "(A/B/E memory.current=%ld) ~= 0", c[2]);
 
+	sync();
 	alloc_anon_in_child(trunk_cg[G], MB(170), 1);
 
 	SAFE_CG_SCANF(trunk_cg[B], "memory.current", "%ld", c);
Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw
In reply to Wei Gao via ltp

Hi!

Quoted 9 lines

Bad IO situation(storage bandwidth ~10MB/sec) will lead background writeback has uncertain progress for dirty page. So system can not reclaim enough memory for new process and finally lead a unexpected OOM.

memcontrol03.c:218: TPASS: Expect: (A/B/E memory.current=0) ~= 0 memcontrol03.c:116: TPASS: Child 1918 killed by OOM
memcontrol03.c:224: TPASS: Expect: (A/B memory.current=52588544) ~= 52428800 memcontrol03.c:129: TFAIL: Expected child 1944 to exit(0), but instead killed by SIGKILL

So I've read the corresponding bug:

https://bugzilla.suse.com/show_bug.cgi?id=1218178

And now I understand what this is supposed to fix. Hover doing sync() is a too big hammer, there is no need to sync the whole system, we just need to make sure that the pages from the file we wrote into are written to a pernament storage, so we likely just need:

testcases/kernel/controllers/memcg/memcontrol03.c +1 -0
index e927dfd19..6d9c490b5 100644
@@ -144,6 +144,7 @@ static void alloc_pagecache_in_child(const struct tst_cg_group *const cg,
        tst_res(TINFO, "Child %d in %s: Allocating pagecache: %"PRIdPTR,
                getpid(), tst_cg_group_name(cg), size);
        alloc_pagecache(fd, size);
+       SAFE_FSYNC(fd);
 
        TST_CHECKPOINT_WAKE(CHILD_IDLE);
        TST_CHECKPOINT_WAIT(TEST_DONE);
Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#3 lore raw

PATCH v2 1/1 memcontrol03: Using clean page cache to avoid dependency on IO rate

In reply to Wei Gao via ltp

Bad IO situation(storage bandwidth ~10MB/sec) will lead background writeback has uncertain progress for dirty page. So system can not reclaim enough memory for new process and finally lead a unexpected OOM.

memcontrol03.c:218: TPASS: Expect: (A/B/E memory.current=0) ~= 0 memcontrol03.c:116: TPASS: Child 1918 killed by OOM
memcontrol03.c:224: TPASS: Expect: (A/B memory.current=52588544) ~= 52428800 memcontrol03.c:129: TFAIL: Expected child 1944 to exit(0), but instead killed by SIGKILL

---
 testcases/kernel/controllers/memcg/memcontrol03.c | 1 +
 1 file changed, 1 insertion(+)
testcases/kernel/controllers/memcg/memcontrol03.c +1 -0
index 9c6c808e0..e2fc85cc8 100644
@@ -144,6 +144,7 @@ static void alloc_pagecache_in_child(const struct tst_cg_group *const cg,
 	tst_res(TINFO, "Child %d in %s: Allocating pagecache: %"PRIdPTR,
 		getpid(), tst_cg_group_name(cg), size);
 	alloc_pagecache(fd, size);
+	SAFE_FSYNC(fd);
 
 	TST_CHECKPOINT_WAKE(CHILD_IDLE);
 	TST_CHECKPOINT_WAIT(TEST_DONE);
Signature · 5 lines
2.35.3


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
Reviewed-by on v2 1/1
#4 lore raw

Hi Wei,

Quoted 4 lines

Bad IO situation(storage bandwidth ~10MB/sec) will lead background writeback has uncertain progress for dirty page. So system can not reclaim enough memory for new process and finally lead a unexpected OOM.

Use fsync() to make sure that the pages from the file we wrote into are written to a permanent storage.

Quoted 4 lines

memcontrol03.c:218: TPASS: Expect: (A/B/E memory.current=0) ~= 0 memcontrol03.c:116: TPASS: Child 1918 killed by OOM
memcontrol03.c:224: TPASS: Expect: (A/B memory.current=52588544) ~= 52428800 memcontrol03.c:129: TFAIL: Expected child 1944 to exit(0), but instead killed by SIGKILL

(Please do this yourself next time)

Kind regards,
Petr

Quoted 17 lines
---
 testcases/kernel/controllers/memcg/memcontrol03.c | 1 +
 1 file changed, 1 insertion(+)
testcases/kernel/controllers/memcg/memcontrol03.c +1 -0
index 9c6c808e0..e2fc85cc8 100644
@@ -144,6 +144,7 @@ static void alloc_pagecache_in_child(const struct tst_cg_group *const cg,
 	tst_res(TINFO, "Child %d in %s: Allocating pagecache: %"PRIdPTR,
 		getpid(), tst_cg_group_name(cg), size);
 	alloc_pagecache(fd, size);
+	SAFE_FSYNC(fd);
 
 	TST_CHECKPOINT_WAKE(CHILD_IDLE);
 	TST_CHECKPOINT_WAIT(TEST_DONE);
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#5 lore raw

Hi Wei,

thanks, merged!

Kind regards,
Petr

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp