memcontrol04
v3 cover memcontrol04
Hello,
This adds another test from the kselftests.
V3:
* The cgroup -> cg API shortening and other changes were already merged
* Change the expected events in F depending on memory_recursiveprot. This should fix the issue reported by Li Wang
V2:
* Add more debugging info to the test output
Richard Palethorpe (2):
API/cgroup: Expose memory_recursiveprot V2 mount opt
memcontrol04: Copy from kselftest
include/tst_cgroup.h | 3 + lib/tst_cgroup.c | 13 +- runtest/controllers | 1 + testcases/kernel/controllers/memcg/.gitignore | 1 + .../kernel/controllers/memcg/memcontrol04.c | 246 ++++++++++++++++++ 5 files changed, 263 insertions(+), 1 deletion(-) create mode 100644 testcases/kernel/controllers/memcg/memcontrol04.c
Signature · 5 lines
2.34.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 1/2 API/cgroup: Expose memory_recursiveprot V2 mount opt
This changes the effect of trunk nodes' memory.low and memory.min on their leaf nodes. So we need to change the expectations of some tests.
--- include/tst_cgroup.h | 3 +++ lib/tst_cgroup.c | 13 ++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)
include/tst_cgroup.h +3 -0
@@ -215,4 +215,7 @@ int safe_cg_occursin(const char *file, const int lineno,const char *const file_name, const char *const needle); +int tst_cg_memory_recursiveprot (void ) + __attribute__ ((warn_unused_result)); +#endif /* TST_CGROUP_H */
lib/tst_cgroup.c +12 -1
@@ -76,6 +76,8 @@ struct cgroup_root { int we_mounted_it:1;/* cpuset is in compatability mode */ int no_cpuset_prefix:1; +/* V2 is mounted with memory_recursive_prot */ + int memory_recursiveprot:1; };/* Controller sub-systems */
@@ -368,7 +370,7 @@ static void cgroup_root_scan(const char *const mnt_type,const struct cgroup_ctrl *const_ctrl;struct cgroup_ctrl *ctrl;uint32_t ctrl_field =0 ; -int no_prefix =0 ; +int no_prefix =0 , memory_recursiveprot =0 ;char buf[BUFSIZ ];char *tok;const int mnt_dfd =SAFE_OPEN (mnt_dir,O_PATH |O_DIRECTORY );
@@ -376,6 +378,9 @@ static void cgroup_root_scan(const char *const mnt_type,if (!strcmp (mnt_type,"cgroup" )) goto v1; +for (tok =strtok (mnt_opts,"," ); tok; tok =strtok (NULL ,"," )) + memory_recursiveprot |= !strcmp ("memory_recursiveprot" , tok); +SAFE_FILE_READAT (mnt_dfd,"cgroup.controllers" , buf,sizeof (buf));for (tok =strtok (buf," " ); tok; tok =strtok (NULL ," " )) {
@@ -433,6 +438,7 @@ backref: root->mnt_dir.dir_fd = mnt_dfd; root->ctrl_field = ctrl_field; root->no_cpuset_prefix = no_prefix; + root->memory_recursiveprot = memory_recursiveprot;for_each_ctrl (ctrl) {if (has_ctrl (root->ctrl_field, ctrl))
@@ -1212,3 +1218,8 @@ int safe_cg_occursin(const char *const file, const int lineno,return !!strstr (buf, needle); } + +int tst_cg_memory_recursiveprot (void ) +{ +return roots[0 ].memory_recursiveprot; +}
Signature · 5 lines
2.34.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 1/2] API/cgroup: Expose memory_recursiveprot V2 mount opt
Quoted 2 lines · Richard Palethorpe
This changes the effect of trunk nodes' memory.low and memory.min on their leaf nodes. So we need to change the expectations of some tests.
How much are LTP runs striving to not affect environment? IIRC, the memory_recursiveprot is "remountable"; in the long-term it's likely worth watching the memory_recursiveprot behavior only.
I.e. instead of carrying two sets of expectations you can warp the environment for the set that's more likely.
Michal
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 1/2] API/cgroup: Expose memory_recursiveprot V2 mount opt
Hello Michal,
Quoted 5 lines · Michal Koutný
Quoted 2 lines · Richard Palethorpe
This changes the effect of trunk nodes' memory.low and memory.min on their leaf nodes. So we need to change the expectations of some tests.
How much are LTP runs striving to not affect environment?
As a general rule we try to leave the environment as we found it so that testing is more deterministic. For the CGroup testing in particular I decided not to change anything so that we do not have to worry about how the init system will react.
Quoted 7 lines
IIRC, the
memory_recursiveprotis "remountable"; in the long-term it's likely worth watching thememory_recursiveprotbehavior only.I.e. instead of carrying two sets of expectations you can warp the environment for the set that's more likely.
Michal
Thinking about it, the reason why I was testing without memory_recursiveprot is because I'm just direct booting the kernel with bash as init and running the test. So the LTP is mounting the CGroups and it should mount with memory_recursiveprot, but it is not.
Probably we have to support older products as well which don't have memory_recursiveprot enabled and are using V2 (unlikely I guess, but it is safest to assume this is the case). So we can still run the test, but report CONF instead of PASS/FAIL. This way we will at least still catch kernel warnings and panics.
Signature · 5 lines
Thank you, Richard. -- Mailing list info: https://lists.linux.it/listinfo/ltp
Li Wang
Reviewed-byRe: [LTP] [PATCH v3 1/2] API/cgroup: Expose memory_recursiveprot V2 mount opt
Quoted 9 lines · Richard Palethorpe
Hello Michal,
Quoted 1 line · Michal Koutný
On Tue, Feb 22, 2022 at 12:45:46PM +0000, Richard Palethorpe <
Quoted 3 lines · [email protected]> wrote:
Quoted 2 lines
This changes the effect of trunk nodes' memory.low and memory.min on their leaf nodes. So we need to change the expectations of some tests.
Quoted 7 lines
Quoted 1 line
How much are LTP runs striving to not affect environment?
As a general rule we try to leave the environment as we found it so that testing is more deterministic. For the CGroup testing in particular I decided not to change anything so that we do not have to worry about how the init system will react.
+1
Quoted 14 lines
Quoted 8 lines
IIRC, the
memory_recursiveprotis "remountable"; in the long-term it's likely worth watching thememory_recursiveprotbehavior only.I.e. instead of carrying two sets of expectations you can warp the environment for the set that's more likely.
Michal
Thinking about it, the reason why I was testing without
memory_recursiveprotis because I'm just direct booting the kernel with bash as init and running the test. So the LTP is mounting the CGroups and it should mount withmemory_recursiveprot, but it is not.
Quoted 6 lines
Probably we have to support older products as well which don't have
memory_recursiveprotenabled and are using V2 (unlikely I guess, but it is safest to assume this is the case). So we can still run the test, but report CONF instead of PASS/FAIL. This way we will at least still catch kernel warnings and panics.
This sounds reasonable at least to me.
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 2/2 memcontrol04: Copy from kselftest
In reply to Richard Palethorpe via ltpAt first glance this test looks the same as memcontrol03. However there are some significant changes which complicate combining the two.
--- runtest/controllers | 1 + testcases/kernel/controllers/memcg/.gitignore | 1 + .../kernel/controllers/memcg/memcontrol04.c | 246 ++++++++++++++++++ 3 files changed, 248 insertions(+) create mode 100644 testcases/kernel/controllers/memcg/memcontrol04.c
runtest/controllers +1 -0
@@ -20,6 +20,7 @@ memcg_control memcg_control_test.sh memcontrol01 memcontrol01 memcontrol02 memcontrol02 memcontrol03 memcontrol03 +memcontrol04 memcontrol04 cgroup_fj_function_debug cgroup_fj_function.sh debug cgroup_fj_function_cpuset cgroup_fj_function.sh cpuset
testcases/kernel/controllers/memcg/.gitignore +1 -0
@@ -8,3 +8,4 @@ memcontrol01 memcontrol02 memcontrol03 +memcontrol04
testcases/kernel/controllers/memcg/memcontrol04.c new file +246 -0
246 lines not on this page · raw message
Signature · 5 lines
2.34.1 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 2/2] memcontrol04: Copy from kselftest
Hello.
Quoted 12 lines · Richard Palethorpe
+ * "First, this test creates the following hierarchy:
+ * A memory.low = 50M, memory.max = 200M + * A/B memory.low = 50M, memory.current = 50M + * A/B/C memory.low = 75M, memory.current = 50M + * A/B/D memory.low = 25M, memory.current = 50M + * A/B/E memory.low = 500M, memory.current = 0 + * A/B/F memory.low = 0, memory.current = 50M+ *
+ * Usages are pagecache
+ * Then it creates A/G and creates a significant
+ * memory pressure in it.
+ *
Quoted 4 lines
+ * A/B memory.current ~= 50M + * A/B/C memory.current ~= 33M + * A/B/D memory.current ~= 17M + * A/B/E memory.current ~= 0
This nicely misses the expected consumption of the F cgroup (I see it's missing in the original too). But one can expect from complementarity it's zero (if one accepts these values, which unfortunately is not true with hierarchical & scaled reclaim protection).
testcases/kernel/controllers/memcg/memcontrol04.c line 216
Quoted 7 lines
testcases/kernel/controllers/memcg/memcontrol04.c new file +7
-0+if (i <E || (i ==F &&tst_cg_memory_recursiveprot ())) { +TST_EXP_EXPR (low >0 , +"(%c low events=%ld) > 0" , id, low); + }else { +TST_EXP_EXPR (low ==0 , +"(%c low events=%ld) == 0" , id, low); + }
Despite this makes the test behavior consistent, I think this is unexpected behavior with recursive_memoryprot. With the given configuration, there should never be residual protection that F assumes.
Unless there is a good explanation [1], I'd consider non-zero memory.events:low in F the test failure here.
Michal
[1] I will need to look into some more detailed tracing data.
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 2/2] memcontrol04: Copy from kselftest
Quoted 21 lines · Michal Koutný
Hello.
Quoted 12 lines · Richard Palethorpe
+ * "First, this test creates the following hierarchy:
+ * A memory.low = 50M, memory.max = 200M + * A/B memory.low = 50M, memory.current = 50M + * A/B/C memory.low = 75M, memory.current = 50M + * A/B/D memory.low = 25M, memory.current = 50M + * A/B/E memory.low = 500M, memory.current = 0 + * A/B/F memory.low = 0, memory.current = 50M+ *
+ * Usages are pagecache
+ * Then it creates A/G and creates a significant
+ * memory pressure in it.
+ *Quoted 4 lines
+ * A/B memory.current ~= 50M + * A/B/C memory.current ~= 33M + * A/B/D memory.current ~= 17M + * A/B/E memory.current ~= 0
Quoted 2 lines
This nicely misses the expected consumption of the F cgroup (I see it's
+1
Quoted 12 lines
missing in the original too). But one can expect from complementarity it's zero (if one accepts these values, which unfortunately is not true with hierarchical & scaled reclaim protection).
testcases/kernel/controllers/memcg/memcontrol04.c line 216
Quoted 7 lines
testcases/kernel/controllers/memcg/memcontrol04.c new file +7
-0+if (i <E || (i ==F &&tst_cg_memory_recursiveprot ())) { +TST_EXP_EXPR (low >0 , +"(%c low events=%ld) > 0" , id, low); + }else { +TST_EXP_EXPR (low ==0 , +"(%c low events=%ld) == 0" , id, low); + }
Quoted 7 lines
Despite this makes the test behavior consistent, I think this is unexpected behavior with
recursive_memoryprot. With the given configuration, there should never be residual protection that F assumes.Unless there is a good explanation [1], I'd consider non-zero memory.events:low in F the test failure here.
Hmm, the documentation does not give an explicit description for recursive_memoryprot. If things like you said, it is more like a CGroup bug in the kernel.
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 2/2] memcontrol04: Copy from kselftest
In reply to Richard Palethorpe via ltpRichard Palethorpe <[email protected]> wrote:
+// SPDX-License-Identifier: GPL-2.0
Quoted 19 lines
+/*\
+ *
+ * [Description]
+ *
+ * Conversion of the forth kself test incgroup/test_memcontrol.c.
+ *
+ * Original description:
+ * "First, this test creates the following hierarchy:+ * A memory.low = 50M, memory.max = 200M + * A/B memory.low = 50M, memory.current = 50M + * A/B/C memory.low = 75M, memory.current = 50M + * A/B/D memory.low = 25M, memory.current = 50M + * A/B/E memory.low = 500M, memory.current = 0 + * A/B/F memory.low = 0, memory.current = 50M+ *
+ * Usages are pagecache
+ * Then it creates A/G and creates a significant
+ * memory pressure in it.
+ *
Quoted 9 lines
+ * A/B memory.current ~= 50M + * A/B/C memory.current ~= 33M + * A/B/D memory.current ~= 17M + * A/B/E memory.current ~= 0+ *
+ * After that it tries to allocate more than there is unprotected
+ * memory in A available, and checks that memory.low protects
+ * pagecache even in this case."
This is not exactly the original description of memory.low test, it looks like modified from memory.min part.
Maybe we'd better copy from line#398~#418.
https://github.com/torvalds/linux/blob/master/tools/testing/selftests/cgroup/test_memcontrol.c#L398
Signature · 2 lines
Regards, Li Wang
Signature · 2 lines
-- Mailing list info: https://lists.linux.it/listinfo/ltp