CGroups
cover CGroups
This is a request for comment on a new CGroups API. I have tried to keep the existing API functions, but there are many changes in the implementation. Please see the commit message to "CGroup API rewrite" in this series.
A previous failed attempt to correct some problems and discussion is
here: http://lists.linux.it/pipermail/ltp/2020-November/019586.html
This is a much bigger change than I would like, but CGroups make it very difficult to do anything simply.
I have done a direct conversion of the test cases to the new API, but I am not sure that it makes sense to call tst_cgroup_move_current within the run method of a test because after the first iteration it will be a NOP. There is also the issue that on the unified hierarchy when calling
tst_cgroup_move_current(TST_CGROUP_MEMORY);
... testing ...tst_cgroup_move_current(TST_CGROUP_CPUSET);
the second call is a NOP as there is only one CGroup, but when the hierarchies are mounted seperately on V1 the current process will not be added to cpuset CGroup until the second call. We probably do not want different behaviour between commonly used hierarchies.
In a lot of cases, the test probably only requires the process to be moved into a CGroup during setup, but this requires an investigation of each test. Some tests scan for NUMA information and use this in the CGroup config which complicates matters. However it seems unlikely to me that the available NUMA nodes will change between test iterations unless we are testing hotplugging or failover.
For tests which are actually testing CGroups themselves, most of the API is too high-level. Please see the kernel self tests for what may be required for those kinds of tests.
This is not meant for full review and has not been tested on older setups yet.
Richard Palethorpe (5):
safe_file_ops: Introduce openat and printfat API CGroup API rewrite
CGroup API tests
CGroup test guidelines
cgroups: convert tests to use API rewrite
doc/test-writing-guidelines.txt | 18 +- include/tst_cgroup.h | 103 ++- include/tst_safe_file_ops.h | 38 + lib/newlib_tests/.gitignore | 1 + lib/newlib_tests/test21.c | 30 +- lib/newlib_tests/tst_cgroup.c | 61 ++ lib/tst_cgroup.c | 960 ++++++++++++++++--------- lib/tst_safe_file_ops.c | 126 ++++ testcases/kernel/mem/cpuset/cpuset01.c | 18 +- testcases/kernel/mem/include/mem.h | 2 +- testcases/kernel/mem/ksm/ksm02.c | 8 +- testcases/kernel/mem/ksm/ksm03.c | 8 +- testcases/kernel/mem/ksm/ksm04.c | 15 +- testcases/kernel/mem/lib/mem.c | 8 +- testcases/kernel/mem/oom/oom03.c | 12 +- testcases/kernel/mem/oom/oom04.c | 10 +- testcases/kernel/mem/oom/oom05.c | 23 +- 17 files changed, 996 insertions(+), 445 deletions(-) create mode 100644 lib/newlib_tests/tst_cgroup.c create mode 100644 lib/tst_safe_file_ops.c
Signature · 1 line
2.29.2
RFC PATCH 1/5 safe_file_ops: Introduce openat and printfat API
Create safe wrappers for the *at commands. This allows us to print to files in a directory without constructing the full file path using sprintf.
--- include/tst_safe_file_ops.h | 38 +++++++++++ lib/tst_safe_file_ops.c | 126 ++++++++++++++++++++++++++++++++++++ 2 files changed, 164 insertions(+) create mode 100644 lib/tst_safe_file_ops.c
include/tst_safe_file_ops.h +38 -0
@@ -46,4 +46,42 @@#define TST_MOUNT_OVERLAY () \ (mount_overlay(__FILE__, __LINE__, 0) == 0) +#define SAFE_OPENAT (dirfd, path, oflags, ...) \ + safe_openat(__FILE__, __LINE__, \ + (dirfd), (path), (oflags), ## __VA_ARGS__) + +#define SAFE_FILE_READAT (dirfd, path, buf, nbyte) \ + safe_file_readat(__FILE__, __LINE__, \ + (dirfd), (path), (buf), (nbyte)) + + +#define SAFE_FILE_PRINTFAT (dirfd, path, fmt, ...) \ + safe_file_printfat(__FILE__, __LINE__, \ + (dirfd), (path), (fmt), __VA_ARGS__) + +#define SAFE_UNLINKAT (dirfd, path, flags) \ + safe_unlinkat(__FILE__, __LINE__, (dirfd), (path), (flags)) + + +int safe_openat (const char *file,const int lineno, +int dirfd,const char *path,int oflags, ...); + +ssize_t safe_file_readat (const char *file,const int lineno, +int dirfd,const char *path,char *buf,size_t nbyte); + +int tst_file_vprintfat (int dirfd,const char *path,const char *fmt,va_list va); +int tst_file_printfat (int dirfd,const char *path,char *fmt, ...) + __attribute__ ((format (printf,3 ,4 ))); + +int safe_file_vprintfat (const char *file,const int lineno, +int dirfd,const char *path, +const char *fmt,va_list va); + +int safe_file_printfat (const char *file,const int lineno, +int dirfd,const char *path,char *fmt, ...) + __attribute__ ((format (printf,5 ,6 ))); + +int safe_unlinkat (const char *file,const int lineno, +int dirfd,const char *path,int flags); +#endif /* TST_SAFE_FILE_OPS */
lib/tst_safe_file_ops.c new file +126 -0
126 lines not on this page · raw message
Signature · 1 line
2.29.2
RFC PATCH 2/5 CGroup API rewrite
In reply to Richard PalethorpeThis builds on the work done by Li Wang, but with many additions and changes. Instead of always creating a new CGroup mount, it scans the current CGroup configuration and tries to use what is presently available.
This is because remounting a CGroup rebinds an existing CGroup root to a new mount point. If any of the mount options are different from the current mount then the operation will fail. Most setups already have CGroups mounted, but the CGroup configurations vary a lot.
So we have a choice of:
A) Expect a particular configuration and refuse to run otherwise
B) Expect no CGroups and mount them
C) Try to use what is available
D) (C) and try to mount them if nothing is present
E) (C), (D) and try to cleanup what we created
(A) and (B) are more simple for the LTP library, but require our users to do more work, perhaps a lot more. Depending on the test environment; disabling or reconfiguring CGroups may be difficult. Because many tests require CGroups to expose a bug in other subsystems (e.g. the memory subsystem), I do not think (A) and (B) are viable options. They may be acceptable for testing core CGroup features where a pristine system is required (e.g. first mount), but that is not the current focus of these changes.
(C) is probably the simplest viable option, but this tries to implement (E). Mounting the CGroups if none are available simplifies testing within environments like Rapido where testing is done in the recovery console and no CGroups are mounted by the system manager. On the other hand it is quite simple for the user to mount some CGroups.
Mounting is relatively simple so does not add much complexity, however cleaning up the mounts is broken so possibly we should only attempt (C) or (D). It appears that the test process has some references to the mount struct and so it can not be removed by the test, but I am partly guessing. Using MNT_DETACH at least allows the directory to be removed, but it's not clear if the root is eventually destroyed. It appears that a separate process (i.e. the umount command) can properly remove the mount and the CGroup is destroyed, but this is still not fully clear without more tracing.
To allow tests to be run in parallel this also creates a new leaf CGroup for each test. These are collected under the LTP trunk CGroup which also contains a drain leaf CGroup. The library will create the LTP CGroup if it is not present, but also the user can pre-create it and grant write access to non-root users. This way tests which require CGroups do not automatically require admin privileges. Please see the comments within the code for more info.
This does not contain a declarative interface for tests to specify CGroup options, like for e.g.
struct tst_test_ test = {
.cgroups = {
{ SET, "memory.max", "10000" }
{ SET, "cpuset.foo", "bar" }
{ MOVE_CURRENT, NULL, NULL }
},
...
};However something like this should be easy to add once enough tests have been converted to the new API.
--- include/tst_cgroup.h | 103 ++++- lib/tst_cgroup.c | 960 ++++++++++++++++++++++++++++--------------- 2 files changed, 701 insertions(+), 362 deletions(-)
include/tst_cgroup.h +81 -22
@@ -2,46 +2,105 @@/* * Copyright (c) 2020 Red Hat, Inc. * Copyright (c) 2020 Li Wang <[email protected]> +* Copyright (c) 2020 Richard Palethorpe <[email protected]> */ #ifndef TST_CGROUP_H #define TST_CGROUP_H -#define PATH_TMP_CG_MEM "/tmp/cgroup_mem" -#define PATH_TMP_CG_CST "/tmp/cgroup_cst" -enum tst_cgroup_ver {TST_CGROUP_V1 =1 ,TST_CGROUP_V2 =2 , };enum tst_cgroup_ctrl { -TST_CGROUP_MEMCG =1 , +TST_CGROUP_MEMORY =1 ,TST_CGROUP_CPUSET =2 , -/* add cgroup controller */ +}; +#define TST_CGROUP_MAX TST_CGROUP_CPUSET + +/* The level of CGroup cleanup which will be attempted after the +* test. When running tests in parallel you should select +* TST_CGORUP_CLEANUP_{TEST,NONE}. Note that we only try to cleanup +* CGroups we mounted/created in the current test. +*/ +enum tst_cgroup_cleanup { +/* Unmount the CGroup roots */ +TST_CGROUP_CLEANUP_ROOT , +/* Cleanup up test CGroups and the overall LTP CGroup */ +TST_CGROUP_CLEANUP_LTP , +/* Cleanup up individual test CGroups */ +TST_CGROUP_CLEANUP_TEST , +/* Don't drain or delete any groups, including the individual +* test CGroups +*/ +TST_CGROUP_CLEANUP_NONE , }; -enum tst_cgroup_ver tst_cgroup_version (void ); +/* Used to specify CGroup hierarchy configuration options, allowing a +* test to request a particular CGroup structure. +* +* It is expected this will expand to include options for changing +* things such as the cgroup names, threading, inheritance etc. +*/ +struct tst_cgroup_opts { +/* Only try to mount V1 CGroup controllers. This will not +* prevent V2 from being used if it is already mounted, it +* only indicates that we should mount V1 controllers if +* nothing is present. By default we try to mount V2 first. */ +int only_mount_v1:1 ; +enum tst_cgroup_cleanup cleanup; +}; -/* To mount/umount specified cgroup controller on 'cgroup_dir' path */ -void tst_cgroup_mount (enum tst_cgroup_ctrl ctrl,const char *cgroup_dir); -void tst_cgroup_umount (const char *cgroup_dir); +/* Search the system for mounted cgroups and available +* controllers. Called automatically by tst_cgroup_require. +*/ +void tst_cgroup_scan (void ); +/* Print the config detected by tst_cgroup_scan */ +void tst_cgroup_print_config (void ); -/* To move current process PID to the mounted cgroup tasks */ -void tst_cgroup_move_current (const char *cgroup_dir); +/* Read from a controller file in the test's default CGroup. Use +* tst_cgroup_require to ensure the specified controller is active on +* the test's default CGroup +*/ +#define SAFE_CGROUP_READ (type, path, buf, nbyte) \ + safe_cgroup_printf(__FILE__, __LINE__, (type), (path), (buf), (nbyte)) -/* To set cgroup controller knob with new value */ -void tst_cgroup_set_knob (const char *cgroup_dir,const char *knob,long value); +ssize_t safe_cgroup_read (const char *file,const int lineno, +enum tst_cgroup_ctrl type, +const char *path,char *buf,size_t nbyte); -/* Set of functions to set knobs under the memory controller */ -void tst_cgroup_mem_set_maxbytes (const char *cgroup_dir,long memsz); -int tst_cgroup_mem_swapacct_enabled (const char *cgroup_dir); -void tst_cgroup_mem_set_maxswap (const char *cgroup_dir,long memsz); +/* Print to a control file in the test's default CGroup */ +#define SAFE_CGROUP_PRINTF (type, path, fmt, ...) \ + safe_cgroup_printf(__FILE__, __LINE__, (type), (path), (fmt), __VA_ARGS__) -/* Set of functions to read/write cpuset controller files content */ -void tst_cgroup_cpuset_read_files (const char *cgroup_dir,const char *filename, -char *retbuf,size_t retbuf_sz); -void tst_cgroup_cpuset_write_files (const char *cgroup_dir,const char *filename, -const char *buf); +void safe_cgroup_printf (const char *file,const int lineno, +enum tst_cgroup_ctrl type, +const char *path,const char *fmt, ...) + __attribute__ ((format (printf,5 ,6 ))); + +/* Ensure the specified controller is available in the test's default +* CGroup, mounting/enabling it if necessary */ +void tst_cgroup_require (enum tst_cgroup_ctrl type, +const struct tst_cgroup_opts *options); + +/* Tear down any CGroups created by calls to tst_cgroup_require */ +void tst_cgroup_cleanup (const struct tst_cgroup_opts *options); + +/* Move the current process to the test's default CGroup */ +void tst_cgroup_move_current (enum tst_cgroup_ctrl type); + +/* Set of functions to set knobs under the test's default memory +* controller. +*/ +void tst_cgroup_mem_set_maxbytes (size_t memsz); +int tst_cgroup_mem_swapacct_enabled (void ); +void tst_cgroup_mem_set_maxswap (size_t memsz); + +/* Set of functions to read/write the test's default cpuset controller +* files content. +*/ +void tst_cgroup_cpuset_read_files (const char *name,char *buf,size_t nbyte); +void tst_cgroup_cpuset_write_files (const char *name,const char *buf);#endif /* TST_CGROUP_H */
lib/tst_cgroup.c +620 -340
1,073 lines not on this page · raw message
Signature · 1 line
2.29.2
RFC PATCH 2/5 CGroup API rewrite
Quoted 24 lines · Richard Palethorpe via ltp
So we have a choice of:
A) Expect a particular configuration and refuse to run otherwise
B) Expect no CGroups and mount them
C) Try to use what is available
D) (C) and try to mount them if nothing is present
E) (C), (D) and try to cleanup what we created(A) and (B) are more simple for the LTP library, but require our users to do more work, perhaps a lot more. Depending on the test environment; disabling or reconfiguring CGroups may be difficult. Because many tests require CGroups to expose a bug in other subsystems (e.g. the memory subsystem), I do not think (A) and (B) are viable options. They may be acceptable for testing core CGroup features where a pristine system is required (e.g. first mount), but that is not the current focus of these changes.
(C) is probably the simplest viable option, but this tries to implement (E). Mounting the CGroups if none are available simplifies testing within environments like Rapido where testing is done in the recovery console and no CGroups are mounted by the system manager. On the other hand it is quite simple for the user to mount some CGroups.
Quoted 5 lines
Mounting is relatively simple so does not add much complexity, however cleaning up the mounts is broken so possibly we should only attempt (C) or (D). It appears that the test process has some references to the mount struct and so it can not be removed by the test, but I am
Yes, and that's also the reason I like the drain_dir in this patch.
Quoted 30 lines
partly guessing. Using
MNT_DETACHat least allows the directory to be removed, but it's not clear if the root is eventually destroyed. It appears that a separate process (i.e. the umount command) can properly remove the mount and the CGroup is destroyed, but this is still not fully clear without more tracing.[...]
+/* Determine if a mounted cgroup hierarchy (tree) is unique and record it if so.
+ *
+ * For CGroups V2 this is very simple as there is only one
+ * hierarchy. We just record which controllers are available and check
+ * if this matches what we read from any previous mounts to verify our
+ * own logic (and possibly detect races).
+ *
+ * For V1 the set of controllers S is partitioned into sets {P_1,P_2,
+ * ...,P_n} with one or more controllers in each partion. Each
+ * partitionP_ncan be mounted multiple times, but the same
+ * controller can not appear in more than one partition. Usually each
+ * partition contains a single controller, but, for example, cpu and
+ * cpuacct are often mounted together in the same partiion.
+ *
+ * Each controller partition has its own hierarchy/root/tree which we
+ * must track and update independently.
+ */
+static voidtst_cgroup_get_tree(const char *type, const char *path, char *opts)
lib/tst_cgroup.c line 230
Quoted 3 lines
lib/tst_cgroup.c +3
-0+{ +struct tst_cgroup_root *t = trees; +struct tst_cgroup_ss *c;
why not naming *c to *ss? to make it tidier for readability just like what you did in other functions.
static void tst_cgroup_get_tree(const char *type, const char *path, char *opts)
{
struct tst_cgroup_root *t = trees;
- struct tst_cgroup_ss *c;
+ struct tst_cgroup_ss *ss;
uint32_t ss_field = 0;
int no_prefix = 0;
char buf[BUFSIZ];
@@ -290,9 +290,9 @@ backref:
t->ss_field = ss_field;
t->no_prefix = no_prefix;- tst_for_each_css(c) {
- if (t->ss_field & (1 << c->indx))
- c->tree = t;
+ tst_for_each_css(ss) {
+ if (t->ss_field & (1 << ss->indx))
+ ss->tree = t;
}Quoted 119 lines
[...]
lib/tst_cgroup.c +7
-6+void tst_cgroup_require (enum tst_cgroup_ctrl type, +const struct tst_cgroup_opts *options) { -char *cgroup_new_dir; -char knob_path[PATH_MAX ]; +struct tst_cgroup_ss *ss =tst_cgroup_get_ss (type); +struct tst_cgroup_root *t; +char path[PATH_MAX ]; - cgroup_new_dir =tst_cgroup_get_path (cgroup_dir); +if (!options) + options = &default_opts; -if (tst_cg_ver &TST_CGROUP_V1 ) { - sprintf(knob_path, "%s/%s", - cgroup_new_dir,"/memory.memsw.limit_in_bytes");
lib/tst_cgroup.c +2
-3+if (ss->tree) + goto ltpdir; -if ((access (knob_path,F_OK ) ==-1 )) { -if (errno ==ENOENT ) - tst_res(TCONF, "memcg swap accounting isdisabled");
(quoted diff) +0
-2- else - tst_brk(TBROK | TERRNO, "failed to access%s",
knob_path);lib/tst_cgroup.c +16
-3- }else { -return 1 ; - } +tst_cgroup_scan (); + +if (ss->tree) + goto ltpdir; + +if (!tst_cgroup_v2_mounted () && !options->only_mount_v1) +tst_cgroup_mount_v2 (); + +if (ss->tree) + goto ltpdir; + +tst_cgroup_mount_v1 (type); + +if (!ss->tree) { + tst_brk(TCONF, + "%s controller required, but not available",ss->name); }lib/tst_cgroup.c +2
-6-if (tst_cg_ver &TST_CGROUP_V2 ) { -sprintf (knob_path,"%s/%s" , - cgroup_new_dir,"/memory.swap.max" ); +ltpdir: + t = ss->tree; -if ((access (knob_path,F_OK ) ==-1 )) { -if (errno ==ENOENT ) - tst_res(TCONF, "memcg swap accounting isdisabled");
(quoted diff) +0
-2- else - tst_brk(TBROK | TERRNO, "failed to access%s",
knob_path);lib/tst_cgroup.c +16
-4- }else { -return 1 ; - } +if (tst_cgroup_ss_on_v2 (ss)) { +tst_file_printfat (t->mnt_dir,"cgroup.subtree_control" , +"+%s" , ss->name); } -return 0 ; +sprintf (path,"%s/%s" , t->path, ltp_cgroup_dir); + +if (!mkdir (path,0777 )) { + t->we_created_ltp_dir =1 ; + goto draindir; + } + +if (errno ==EEXIST ) + goto draindir; + +if (errno ==EACCES ) { + tst_brk(TCONF | TERRNO, + "Lack permission to make %s; premake it or run asroot",
lib/tst_cgroup.c +29
-0+ path); + } + +tst_brk (TBROK |TERRNO ,"mkdir(%s, 0777)" , path); + +draindir: +if (!t->ltp_dir) + t->ltp_dir =SAFE_OPEN (path,O_PATH |O_DIRECTORY ); + +if (tst_cgroup_ss_on_v2 (ss)) { +SAFE_FILE_PRINTFAT (t->ltp_dir,"cgroup.subtree_control" , +"+%s" , ss->name); + }else { +SAFE_FILE_PRINTFAT (t->ltp_dir,"cgroup.clone_children" , +"%d" ,1 ); + +if (type ==TST_CGROUP_CPUSET ) +tst_cgroup_copy_cpuset (t); + } + +sprintf (path,"%s/%s/%s" , + t->path, ltp_cgroup_dir, ltp_cgroup_drain_dir); + +if (!mkdir (path,0777 ) || errno ==EEXIST ) + goto testdir; + +if (errno ==EACCES ) { + tst_brk(TCONF | TERRNO, + "Lack permission to make %s; grant access or runas root",
lib/tst_cgroup.c +1
-0+ path);
lib/tst_cgroup.c line 579
Quoted 10 lines
lib/tst_cgroup.c +10
-0+ } + +tst_brk (TBROK |TERRNO ,"mkdir(%s, 0777)" , path); + +testdir: +if (!t->drain_dir) + t->drain_dir =SAFE_OPEN (path,O_PATH |O_DIRECTORY ); + +if (!test_cgroup_dir[0 ]) +sprintf (test_cgroup_dir,"test-%d" ,getpid ());
As I was mentioned in 0/5 that maybe we should create test_cgroup_dir for different progress so that the test could use the same controller with various configurations in parallel.
e.g. child_1 set SIZE to memory.limit_in_bytes
child_2 set SIZE/2 to memory.limit_in_bytes
Any possibility to move this to tst_cgroup_move_current?
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210102/bc700a01/attachment.htm>
RFC PATCH 2/5 CGroup API rewrite
Hello Li,
Quoted 10 lines · Li Wang
Quoted 1 line
As I was mentioned in 0/5 that maybe we should create
test_cgroup_dirfor different progress so that the test could use the same controller with various configurations in parallel.e.g.
child_1set SIZE to memory.limit_in_bytes
child_2set SIZE/2 to memory.limit_in_bytesAny possibility to move this to
tst_cgroup_move_current?
Yes I suppose we can try this. Is there a test which already requires it?
Signature · 2 lines
Thank you, Richard.
RFC PATCH 2/5 CGroup API rewrite
Quoted 9 lines · Richard Palethorpe
Hello Li,
Quoted 4 lines · Li Wang
Quoted 1 line
As I was mentioned in 0/5 that maybe we should create
test_cgroup_dirfor different progress so that the test could use the same controllerwith
Quoted 9 lines
Quoted 6 lines
various configurations in parallel.
e.g.
child_1set SIZE to memory.limit_in_bytes
child_2set SIZE/2 to memory.limit_in_bytesAny possibility to move this to
tst_cgroup_move_current?Yes I suppose we can try this. Is there a test which already requires it?
So far we don't have such a test, I remember that in the previous Lib is also to keep expandability.
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210104/38a04d2e/attachment.htm>
RFC PATCH 2/5 CGroup API rewrite
Hello Li,
Quoted 17 lines · Li Wang
Quoted 14 lines · Richard Palethorpe
Hello Li,
Quoted 4 lines · Li Wang
Quoted 1 line
As I was mentioned in 0/5 that maybe we should create
test_cgroup_dirfor different progress so that the test could use the same controllerwith
Quoted 5 lines
various configurations in parallel.
e.g.
child_1set SIZE to memory.limit_in_bytes
child_2set SIZE/2 to memory.limit_in_bytes
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Any possibility to move this to
tst_cgroup_move_current?Yes I suppose we can try this. Is there a test which already requires it?
So far we don't have such a test, I remember that in the previous Lib is also to keep expandability.
I think we have at least two problems:
1) This allows many CGroups to be created for each test and we must clean them up, adding some complication.
2) It's not clear if a future test will require the CGroup hierarchy to be the same as the process hierarchy or different. Depending what behaviour for tst_cgroup_move_current we choose it will make some configurations impossible.
So if we add this then it will add complexity, but I am not sure it will help in the future. If we make it flexible enough to support any hierarchy this will add a lot of complication.
Also if we did need this feature in the future, then we can add some new functions which take a sub-group (or hierarchy) parameter. e.g.
void tst_cgroup_move(enum tst_cgroup_ctrl type, pid_t pid,
struct tst_cgroup_tree *path);
Alternate versions of other functions would also need to be added. Also some changes to the internal data structures may be needed. However it would keep the current API functions simple.
So until we have a test which requires this, I think the best option is to do nothing :-)
Signature · 2 lines
Thank you, Richard.
RFC PATCH 2/5 CGroup API rewrite
Hi Richard,
Quoted 23 lines · Richard Palethorpe
...
Quoted 11 lines
Quoted 11 lines
Quoted 2 lines
As I was mentioned in 0/5 that maybe we should create
test_cgroup_dirfor different progress so that the test could use the same controllerwith
Quoted 6 lines
various configurations in parallel.
e.g.
child_1set SIZE to memory.limit_in_bytes
child_2set SIZE/2 to memory.limit_in_bytesAny possibility to move this to
tst_cgroup_move_current?Yes I suppose we can try this. Is there a test which already requires
it?
Quoted 4 lines
Quoted 1 line
So far we don't have such a test, I remember that in the previous Lib is also to keep expandability.
I think we have at least two problems:
1) This allows many CGroups to be created for each test and we must clean them up, adding some complication.
Quoted 9 lines
2) It's not clear if a future test will require the CGroup hierarchy to be the same as the process hierarchy or different. Depending what behaviour for
tst_cgroup_move_currentwe choose it will make some configurations impossible.So if we add this then it will add complexity, but I am not sure it will help in the future. If we make it flexible enough to support any hierarchy this will add a lot of complication.
Okay. I do NOT insist to implement a future feature at this early moment, but do u think we'd better mention this in documents? To let people(avoid abusing it) know that the current CGroup lib hasn't supported children using the same controller in parallel?
And another new query is, do we really need to export many cleanup-levels to users? I guess we can handle it in the library with intelligently choose levels no matter in parallel or sequential.
i.e. As now only one directory(test-pid/) created for one test in a hierarchy, how about going with TST_CGROUP_CLEANUP_ROOT level by default, unless it detects more or equal to two directories (that probably means parallel) goes TST_CGROUP_CLEANUP_TEST?
Quoted 7 lines
Also if we did need this feature in the future, then we can add some new functions which take a sub-group (or hierarchy) parameter. e.g.
void tst_cgroup_move(enum
tst_cgroup_ctrltype,pid_tpid,struct tst_cgroup_tree *path);
Sounds good to me.
Quoted 13 lines
Alternate versions of other functions would also need to be added. Also some changes to the internal data structures may be needed. However it would keep the current API functions simple.
So until we have a test which requires this, I think the best option is to do nothing :-)
--
Thank you,
Richard.
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210120/e56cad4d/attachment-0001.htm>
RFC PATCH 2/5 CGroup API rewrite
Hello Li,
Quoted 37 lines · Li Wang
Hi Richard,
Quoted 23 lines · Richard Palethorpe
...
Quoted 11 lines
Quoted 11 lines
Quoted 2 lines
As I was mentioned in 0/5 that maybe we should create
test_cgroup_dirfor different progress so that the test could use the same controllerwith
Quoted 6 lines
various configurations in parallel.
e.g.
child_1set SIZE to memory.limit_in_bytes
child_2set SIZE/2 to memory.limit_in_bytesAny possibility to move this to
tst_cgroup_move_current?Yes I suppose we can try this. Is there a test which already requires
it?
Quoted 4 lines
Quoted 1 line
So far we don't have such a test, I remember that in the previous Lib is also to keep expandability.
I think we have at least two problems:
1) This allows many CGroups to be created for each test and we must clean them up, adding some complication.
Quoted 9 lines
2) It's not clear if a future test will require the CGroup hierarchy to be the same as the process hierarchy or different. Depending what behaviour for
tst_cgroup_move_currentwe choose it will make some configurations impossible.So if we add this then it will add complexity, but I am not sure it will help in the future. If we make it flexible enough to support any hierarchy this will add a lot of complication.
Quoted 4 lines
Okay. I do NOT insist to implement a future feature at this early moment, but do u think we'd better mention this in documents? To let people(avoid abusing it) know that the current CGroup lib hasn't supported children using the same controller in parallel?
Yes, this is a good point.
Quoted 4 lines
And another new query is, do we really need to export many cleanup-levels to users? I guess we can handle it in the library with intelligently choose levels no matter in parallel or sequential.
No, I should remove it. I was using it for debugging, but there is probably a simpler way.
Quoted 5 lines
i.e. As now only one directory(test-pid/) created for one test in a hierarchy, how about going with
TST_CGROUP_CLEANUP_ROOTlevel by default, unless it detects more or equal to two directories (that probably means parallel) goesTST_CGROUP_CLEANUP_TEST?
That is likely to result in race conditions. However it won't cleanup any CGroups which already exist (except the test CGroup). So the user can pre-create the LTP CGroup before running tests in parallel.
Quoted 26 lines
Quoted 9 lines
Also if we did need this feature in the future, then we can add some new functions which take a sub-group (or hierarchy) parameter. e.g.
void tst_cgroup_move(enum
tst_cgroup_ctrltype,pid_tpid,struct tst_cgroup_tree *path);Sounds good to me.
Quoted 13 lines
Alternate versions of other functions would also need to be added. Also some changes to the internal data structures may be needed. However it would keep the current API functions simple.
So until we have a test which requires this, I think the best option is to do nothing :-)
--
Thank you,
Richard.
Signature · 2 lines
Thank you, Richard.
RFC PATCH 3/5 CGroup API tests
In reply to Richard PalethorpeRewrite of existing test and new test for the CGroups API.
--- lib/newlib_tests/.gitignore | 1 + lib/newlib_tests/test21.c | 30 +++++------------ lib/newlib_tests/tst_cgroup.c | 61 +++++++++++++++++++++++++++++++++++ 3 files changed, 70 insertions(+), 22 deletions(-) create mode 100644 lib/newlib_tests/tst_cgroup.c
lib/newlib_tests/.gitignore +1 -0
@@ -16,6 +16,7 @@ test15 test16 tst_capability01 tst_capability02 +tst_cgroup tst_device tst_safe_fileops tst_res_hexd
lib/newlib_tests/test21.c +8 -22
@@ -11,8 +11,6 @@#include "tst_test.h" #include "tst_cgroup.h" -#define PATH_CGROUP1 "/mnt/liwang1" -#define PATH_CGROUP2 "/mnt/liwang2"#define MEMSIZE 1024 * 1024static void do_test (void )
@@ -21,19 +19,16 @@ static void do_test(void)switch (pid) {case 0 : -tst_cgroup_move_current (PATH_CGROUP1 ); -tst_cgroup_mem_set_maxbytes (PATH_CGROUP1 ,MEMSIZE ); -tst_cgroup_mem_set_maxswap (PATH_CGROUP1 ,MEMSIZE ); - -tst_cgroup_move_current (PATH_CGROUP2 ); - +tst_cgroup_move_current (TST_CGROUP_MEMORY ); +tst_cgroup_mem_set_maxbytes (MEMSIZE ); +tst_cgroup_mem_set_maxswap (MEMSIZE );break ;default : -tst_cgroup_move_current (PATH_TMP_CG_CST ); +tst_cgroup_move_current (TST_CGROUP_CPUSET ); -tst_cgroup_move_current (PATH_TMP_CG_MEM ); -tst_cgroup_mem_set_maxbytes (PATH_TMP_CG_MEM ,MEMSIZE ); -tst_cgroup_mem_set_maxswap (PATH_TMP_CG_MEM ,MEMSIZE ); +tst_cgroup_move_current (TST_CGROUP_MEMORY ); +tst_cgroup_mem_set_maxbytes (MEMSIZE ); +tst_cgroup_mem_set_maxswap (MEMSIZE );break ; }
@@ -42,20 +37,11 @@ static void do_test(void)static void setup (void ) { -tst_cgroup_mount (TST_CGROUP_MEMCG ,PATH_TMP_CG_MEM ); -tst_cgroup_mount (TST_CGROUP_MEMCG ,PATH_CGROUP1 ); - -tst_cgroup_mount (TST_CGROUP_CPUSET ,PATH_TMP_CG_CST ); -tst_cgroup_mount (TST_CGROUP_CPUSET ,PATH_CGROUP2 ); +tst_cgroup_require (TST_CGROUP_MEMORY ,NULL ); }static void cleanup (void ) { -tst_cgroup_umount (PATH_TMP_CG_MEM ); -tst_cgroup_umount (PATH_CGROUP1 ); - -tst_cgroup_umount (PATH_TMP_CG_CST ); -tst_cgroup_umount (PATH_CGROUP2 ); }static struct tst_test test = {
lib/newlib_tests/tst_cgroup.c new file +61 -0
61 lines not on this page · raw message
Signature · 1 line
2.29.2
RFC PATCH 3/5 CGroup API tests
Richard Palethorpe via ltp <[email protected]> wrote:
Rewrite of existing test and new test for the CGroups API.
Quoted 16 lines
...
lib/newlib_tests/test21.c +0
-2@@ -11,8 +11,6 @@#include "tst_test.h" #include "tst_cgroup.h" -#define PATH_CGROUP1 "/mnt/liwang1" -#define PATH_CGROUP2 "/mnt/liwang2"#define MEMSIZE 1024 * 1024static void do_test (void )@@ -21,19 +19,16 @@ static void do_test(void)switch (pid) {case 0 :
lib/newlib_tests/test21.c static void do_test(void) · line 25
Quoted 10 lines
lib/newlib_tests/test21.c +3
-6-tst_cgroup_move_current (PATH_CGROUP1 ); -tst_cgroup_mem_set_maxbytes (PATH_CGROUP1 ,MEMSIZE ); -tst_cgroup_mem_set_maxswap (PATH_CGROUP1 ,MEMSIZE ); - -tst_cgroup_move_current (PATH_CGROUP2 ); - +tst_cgroup_move_current (TST_CGROUP_MEMORY ); +tst_cgroup_mem_set_maxbytes (MEMSIZE ); +tst_cgroup_mem_set_maxswap (MEMSIZE );
We'd better set a different value(maybe MEMSIZE/2) for the child's CGROUP_MEMORY because test21 is designed to verify there is no race condition when multithread using the same controller. In other words, we need to avoid that iteration become a NOP.
Quoted 15 lines
lib/newlib_tests/test21.c +4
-4break ;default : -tst_cgroup_move_current (PATH_TMP_CG_CST ); +tst_cgroup_move_current (TST_CGROUP_CPUSET ); -tst_cgroup_move_current (PATH_TMP_CG_MEM ); -tst_cgroup_mem_set_maxbytes (PATH_TMP_CG_MEM ,MEMSIZE ); -tst_cgroup_mem_set_maxswap (PATH_TMP_CG_MEM ,MEMSIZE ); +tst_cgroup_move_current (TST_CGROUP_MEMORY ); +tst_cgroup_mem_set_maxbytes (MEMSIZE ); +tst_cgroup_mem_set_maxswap (MEMSIZE );break ; }
lib/newlib_tests/test21.c static void do_test(void) · line 41
Quoted 10 lines
lib/newlib_tests/test21.c +1
-5@@ -42,20 +37,11 @@ static void do_test(void)static void setup (void ) { -tst_cgroup_mount (TST_CGROUP_MEMCG ,PATH_TMP_CG_MEM ); -tst_cgroup_mount (TST_CGROUP_MEMCG ,PATH_CGROUP1 ); - -tst_cgroup_mount (TST_CGROUP_CPUSET ,PATH_TMP_CG_CST ); -tst_cgroup_mount (TST_CGROUP_CPUSET ,PATH_CGROUP2 ); +tst_cgroup_require (TST_CGROUP_MEMORY ,NULL );
The test requires _CPUSET too:
tst_cgroup_require(TST_CGROUP_CPUSET, NULL);
}
Quoted 10 lines
lib/newlib_tests/test21.c +0
-5static void cleanup (void ) { -tst_cgroup_umount (PATH_TMP_CG_MEM ); -tst_cgroup_umount (PATH_CGROUP1 ); - -tst_cgroup_umount (PATH_TMP_CG_CST ); -tst_cgroup_umount (PATH_CGROUP2 ); }
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210102/9e77d7a7/attachment-0001.htm>
RFC PATCH 4/5 CGroup test guidelines
In reply to Richard Palethorpe--- doc/test-writing-guidelines.txt | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
doc/test-writing-guidelines.txt +10 -8
@@ -2148,16 +2148,18 @@ the field value of file. 2.2.36 Using Control Group ^^^^^^^^^^^^^^^^^^^^^^^^^^ -Some of LTP tests need Control Group in their configuration, tst_cgroup.h provides -APIs to make cgroup unified mounting at setup phase to be possible. The method?is + +Some of LTP tests need Control Groups in their configurations, tst_cgroup.h provides +APIs to discover and use CGroups. The method?is extracted from mem.h with the purpose of?extendible for further developing, and trying to compatible the current two versions of cgroup. -Considering there are many differences between cgroup v1 and v2, here we capsulate +Considering there are many differences between cgroup v1 and v2, here we encapsulate the detail of cgroup mounting in high-level functions, which will be easier to use -cgroup without caring about too much technical thing.? ? +cgroup without caring about too much technical thing. Also, we do cgroup mount/umount work for the different hierarchy automatically. +If that is required. [source,c] -------------------------------------------------------------------------------
@@ -2167,8 +2169,8 @@ static void run(void) { ... - tst_cgroup_move_current(PATH_TMP_CG_MEM); - tst_cgroup_mem_set_maxbytes(PATH_TMP_CG_MEM, MEMSIZE); + tst_cgroup_move_current(TST_CGROUP_MEMORY); + tst_cgroup_mem_set_maxbytes(MEMSIZE); // do test under cgroup ...
@@ -2176,12 +2178,12 @@ static void run(void) static void setup(void) { - tst_cgroup_mount(TST_CGROUP_MEMCG, PATH_TMP_CG_MEM); + tst_cgroup_require(TST_CGROUP_MEMORY, NULL); } static void cleanup(void) { - tst_cgroup_umount(PATH_TMP_CG_MEM); + tst_cgroup_cleanup(NULL); } struct tst_test test = {
Signature · 1 line
2.29.2
RFC PATCH 5/5 cgroups: convert tests to use API rewrite
In reply to Richard Palethorpe--- testcases/kernel/mem/cpuset/cpuset01.c | 18 +++++++++--------- testcases/kernel/mem/include/mem.h | 2 +- testcases/kernel/mem/ksm/ksm02.c | 8 ++++---- testcases/kernel/mem/ksm/ksm03.c | 8 ++++---- testcases/kernel/mem/ksm/ksm04.c | 15 +++++++-------- testcases/kernel/mem/lib/mem.c | 8 ++++---- testcases/kernel/mem/oom/oom03.c | 12 ++++++------ testcases/kernel/mem/oom/oom04.c | 10 +++++----- testcases/kernel/mem/oom/oom05.c | 23 +++++++++++------------ 9 files changed, 51 insertions(+), 53 deletions(-)
testcases/kernel/mem/cpuset/cpuset01.c +9 -9
@@ -51,11 +51,11 @@ static void test_cpuset(void)unsigned long nmask[MAXNODES /BITS_PER_LONG ] = {0 };char mems[BUFSIZ ], buf[BUFSIZ ]; -tst_cgroup_cpuset_read_files (PATH_TMP_CG_CST ,"cpus" , buf,BUFSIZ ); -tst_cgroup_cpuset_write_files (PATH_TMP_CG_CST ,"cpus" , buf); -tst_cgroup_cpuset_read_files (PATH_TMP_CG_CST ,"mems" , mems,BUFSIZ ); -tst_cgroup_cpuset_write_files (PATH_TMP_CG_CST ,"mems" , mems); -tst_cgroup_move_current (PATH_TMP_CG_CST ); +tst_cgroup_cpuset_read_files ("cpus" , buf,BUFSIZ ); +tst_cgroup_cpuset_write_files ("cpus" , buf); +tst_cgroup_cpuset_read_files ("mems" , mems,BUFSIZ ); +tst_cgroup_cpuset_write_files ("mems" , mems); +tst_cgroup_move_current (TST_CGROUP_CPUSET ); child =SAFE_FORK ();if (child ==0 ) {
@@ -70,9 +70,9 @@ static void test_cpuset(void) }snprintf (buf,BUFSIZ ,"%d" , nodes[0 ]); -tst_cgroup_cpuset_write_files (PATH_TMP_CG_CST ,"mems" , buf); +tst_cgroup_cpuset_write_files ("mems" , buf);snprintf (buf,BUFSIZ ,"%d" , nodes[1 ]); -tst_cgroup_cpuset_write_files (PATH_TMP_CG_CST ,"mems" , buf); +tst_cgroup_cpuset_write_files ("mems" , buf);SAFE_WAITPID (child, &status,WUNTRACED |WCONTINUED );if (WEXITSTATUS (status) !=0 ) {
@@ -85,7 +85,7 @@ static void test_cpuset(void)static void setup (void ) { -tst_cgroup_mount (TST_CGROUP_CPUSET ,PATH_TMP_CG_CST ); +tst_cgroup_require (TST_CGROUP_CPUSET ,NULL ); ncpus =count_cpu ();if (get_allowed_nodes_arr (NH_MEMS |NH_CPUS , &nnodes, &nodes) <0 )tst_brk (TBROK |TERRNO ,"get_allowed_nodes_arr" );
@@ -95,7 +95,7 @@ static void setup(void)static void cleanup (void ) { -tst_cgroup_umount (PATH_TMP_CG_CST ); +tst_cgroup_cleanup (NULL ); } static void sighandler(int signo LTP_ATTRIBUTE_UNUSED)
testcases/kernel/mem/include/mem.h +1 -1
@@ -61,7 +61,7 @@ void check_hugepage(void);void write_memcg (void );/* cpuset/memcg - include/tst_cgroup.h */ -void write_cpusets (const char *cgroup_dir,long nd); +void write_cpusets (long nd);/* shared */ unsigned int get_a_numa_node (void );
testcases/kernel/mem/ksm/ksm02.c +4 -4
@@ -76,8 +76,8 @@ static void verify_ksm(void) }create_same_memory (size, num, unit); -write_cpusets (PATH_TMP_CG_CST , node); -tst_cgroup_move_current (PATH_TMP_CG_CST ); +write_cpusets (node); +tst_cgroup_move_current (TST_CGROUP_CPUSET );create_same_memory (size, num, unit); }
@@ -87,7 +87,7 @@ static void cleanup(void)FILE_PRINTF (PATH_KSM "merge_across_nodes" ,"%d" , merge_across_nodes); -tst_cgroup_umount (PATH_TMP_CG_CST ); +tst_cgroup_cleanup (NULL ); }static void setup (void )
@@ -103,7 +103,7 @@ static void setup(void)SAFE_FILE_PRINTF (PATH_KSM "merge_across_nodes" ,"1" ); } -tst_cgroup_mount (TST_CGROUP_CPUSET ,PATH_TMP_CG_CST ); +tst_cgroup_require (TST_CGROUP_CPUSET ,NULL ); }static struct tst_test test = {
testcases/kernel/mem/ksm/ksm03.c +4 -4
@@ -61,8 +61,8 @@static void verify_ksm (void ) { -tst_cgroup_move_current (PATH_TMP_CG_MEM ); -tst_cgroup_mem_set_maxbytes (PATH_TMP_CG_MEM ,TESTMEM ); +tst_cgroup_move_current (TST_CGROUP_MEMORY ); +tst_cgroup_mem_set_maxbytes (TESTMEM );create_same_memory (size, num, unit); }
@@ -78,7 +78,7 @@ static void setup(void) }parse_ksm_options (opt_sizestr, &size, opt_numstr, &num, opt_unitstr, &unit); -tst_cgroup_mount (TST_CGROUP_MEMCG ,PATH_TMP_CG_MEM ); +tst_cgroup_require (TST_CGROUP_MEMORY ,NULL ); }static void cleanup (void )
@@ -86,7 +86,7 @@ static void cleanup(void)if (access (PATH_KSM "merge_across_nodes" ,F_OK ) ==0 )FILE_PRINTF (PATH_KSM "merge_across_nodes" ,"%d" , merge_across_nodes); -tst_cgroup_umount (PATH_TMP_CG_MEM ); +tst_cgroup_cleanup (NULL ); }static struct tst_test test = {
testcases/kernel/mem/ksm/ksm04.c +7 -8
@@ -67,8 +67,8 @@ static void verify_ksm(void) node =get_a_numa_node ();set_node (nmask, node); -tst_cgroup_move_current (PATH_TMP_CG_MEM ); -tst_cgroup_mem_set_maxbytes (PATH_TMP_CG_MEM ,TESTMEM ); +tst_cgroup_move_current (TST_CGROUP_MEMORY ); +tst_cgroup_mem_set_maxbytes (TESTMEM );if (set_mempolicy (MPOL_BIND , nmask,MAXNODES ) ==-1 ) {if (errno !=ENOSYS )
@@ -79,8 +79,8 @@ static void verify_ksm(void) }create_same_memory (size, num, unit); -write_cpusets (PATH_TMP_CG_CST , node); -tst_cgroup_move_current (PATH_TMP_CG_CST ); +write_cpusets (node); +tst_cgroup_move_current (TST_CGROUP_CPUSET );create_same_memory (size, num, unit); }
@@ -90,8 +90,7 @@ static void cleanup(void)FILE_PRINTF (PATH_KSM "merge_across_nodes" ,"%d" , merge_across_nodes); -tst_cgroup_umount (PATH_TMP_CG_MEM ); -tst_cgroup_umount (PATH_TMP_CG_CST ); +tst_cgroup_cleanup (NULL ); }static void setup (void )
@@ -107,8 +106,8 @@ static void setup(void)parse_ksm_options (opt_sizestr, &size, opt_numstr, &num, opt_unitstr, &unit); -tst_cgroup_mount (TST_CGROUP_MEMCG ,PATH_TMP_CG_MEM ); -tst_cgroup_mount (TST_CGROUP_CPUSET ,PATH_TMP_CG_CST ); +tst_cgroup_require (TST_CGROUP_MEMORY ,NULL ); +tst_cgroup_require (TST_CGROUP_CPUSET ,NULL ); }static struct tst_test test = {
testcases/kernel/mem/lib/mem.c +4 -4
@@ -629,13 +629,13 @@ static void gather_node_cpus(char *cpus, long nd) cpus[strlen (cpus) -1 ] ='\0' ; } -void write_cpusets (const char *cgroup_dir,long nd) +void write_cpusets (long nd) {char buf[BUFSIZ ];char cpus[BUFSIZ ] ="" ;snprintf (buf,BUFSIZ ,"%ld" , nd); -tst_cgroup_cpuset_write_files (cgroup_dir,"mems" , buf); +tst_cgroup_cpuset_write_files ("mems" , buf);gather_node_cpus (cpus, nd);/*
@@ -644,11 +644,11 @@ void write_cpusets(const char *cgroup_dir, long nd)* the value of cpuset.cpus. */ if (strlen (cpus) !=0 ) { -tst_cgroup_cpuset_write_files (cgroup_dir,"cpus" , cpus); +tst_cgroup_cpuset_write_files ("cpus" , cpus); }else {tst_res (TINFO ,"No CPUs in the node%ld; " "using only CPU0" , nd); -tst_cgroup_cpuset_write_files (cgroup_dir,"cpus" ,"0" ); +tst_cgroup_cpuset_write_files ("cpus" ,"0" ); } }
testcases/kernel/mem/oom/oom03.c +6 -6
@@ -42,13 +42,13 @@ static void verify_oom(void)tst_brk (TCONF ,"test is not designed for 32-bit system." );#endif -tst_cgroup_move_current (PATH_TMP_CG_MEM ); -tst_cgroup_mem_set_maxbytes (PATH_TMP_CG_MEM ,TESTMEM ); +tst_cgroup_move_current (TST_CGROUP_MEMORY ); +tst_cgroup_mem_set_maxbytes (TESTMEM );testoom (0 ,0 ,ENOMEM ,1 ); -if (tst_cgroup_mem_swapacct_enabled (PATH_TMP_CG_MEM )) { -tst_cgroup_mem_set_maxswap (PATH_TMP_CG_MEM ,TESTMEM ); +if (tst_cgroup_mem_swapacct_enabled ()) { +tst_cgroup_mem_set_maxswap (TESTMEM );testoom (0 ,1 ,ENOMEM ,1 ); }
@@ -65,14 +65,14 @@ static void setup(void) { overcommit =get_sys_tune ("overcommit_memory" );set_sys_tune ("overcommit_memory" ,1 ,1 ); -tst_cgroup_mount (TST_CGROUP_MEMCG ,PATH_TMP_CG_MEM ); +tst_cgroup_require (TST_CGROUP_MEMORY ,NULL ); }static void cleanup (void ) {if (overcommit !=-1 )set_sys_tune ("overcommit_memory" , overcommit,0 ); -tst_cgroup_umount (PATH_TMP_CG_MEM ); +tst_cgroup_cleanup (NULL ); }static struct tst_test test = {
testcases/kernel/mem/oom/oom04.c +5 -5
@@ -42,7 +42,7 @@ static void verify_oom(void)tst_brk (TCONF ,"test is not designed for 32-bit system." );#endif -tst_cgroup_move_current (PATH_TMP_CG_CST ); +tst_cgroup_move_current (TST_CGROUP_CPUSET );tst_res (TINFO ,"OOM on CPUSET..." );testoom (0 ,0 ,ENOMEM ,1 );
@@ -53,7 +53,7 @@ static void verify_oom(void)* is in charge of cpuset.memory_migrate, we can write * 1 to cpuset.memory_migrate to enable the migration. */ -tst_cgroup_cpuset_write_files (PATH_TMP_CG_CST ,"memory_migrate" ,"1" ); +tst_cgroup_cpuset_write_files ("memory_migrate" ,"1" );tst_res (TINFO ,"OOM on CPUSET with mem migrate:" );testoom (0 ,0 ,ENOMEM ,1 ); }
@@ -69,7 +69,7 @@ static void setup(void) overcommit =get_sys_tune ("overcommit_memory" );set_sys_tune ("overcommit_memory" ,1 ,1 ); -tst_cgroup_mount (TST_CGROUP_CPUSET ,PATH_TMP_CG_CST ); +tst_cgroup_require (TST_CGROUP_CPUSET ,NULL );/* * Some nodes do not contain memory, so use
@@ -81,14 +81,14 @@ static void setup(void)if (ret <0 )tst_brk (TBROK ,"Failed to get a memory node " "using get_allowed_nodes()" ); -write_cpusets (PATH_TMP_CG_CST , memnode); +write_cpusets (memnode); }static void cleanup (void ) {if (overcommit !=-1 )set_sys_tune ("overcommit_memory" , overcommit,0 ); -tst_cgroup_umount (PATH_TMP_CG_CST ); +tst_cgroup_cleanup (NULL ); }static struct tst_test test = {
testcases/kernel/mem/oom/oom05.c +11 -12
@@ -43,9 +43,9 @@ static void verify_oom(void)#endif tst_res (TINFO ,"OOM on CPUSET & MEMCG..." ); -tst_cgroup_move_current (PATH_TMP_CG_MEM ); -tst_cgroup_move_current (PATH_TMP_CG_CST ); -tst_cgroup_mem_set_maxbytes (PATH_TMP_CG_MEM ,TESTMEM ); +tst_cgroup_move_current (TST_CGROUP_MEMORY ); +tst_cgroup_move_current (TST_CGROUP_CPUSET ); +tst_cgroup_mem_set_maxbytes (TESTMEM );testoom (0 ,0 ,ENOMEM ,1 );/*
@@ -54,21 +54,21 @@ static void verify_oom(void)* 1 to cpuset.memory_migrate to enable the migration. */ if (is_numa (NULL ,NH_MEMS ,2 )) { -tst_cgroup_cpuset_write_files (PATH_TMP_CG_CST ,"memory_migrate" ,"1" ); +tst_cgroup_cpuset_write_files ("memory_migrate" ,"1" );tst_res (TINFO ,"OOM on CPUSET & MEMCG with " "cpuset.memory_migrate=1" );testoom (0 ,0 ,ENOMEM ,1 ); } -if (tst_cgroup_mem_swapacct_enabled (PATH_TMP_CG_MEM )) { +if (tst_cgroup_mem_swapacct_enabled ()) {tst_res (TINFO ,"OOM on CPUSET & MEMCG with " "special memswap limitation:" ); -tst_cgroup_mem_set_maxswap (PATH_TMP_CG_MEM ,TESTMEM ); +tst_cgroup_mem_set_maxswap (TESTMEM );testoom (0 ,0 ,ENOMEM ,1 );tst_res (TINFO ,"OOM on CPUSET & MEMCG with " "disabled memswap limitation:" ); -tst_cgroup_mem_set_maxswap (PATH_TMP_CG_MEM ,-1 ); +tst_cgroup_mem_set_maxswap (-1 );testoom (0 ,0 ,ENOMEM ,1 ); } }
@@ -83,8 +83,8 @@ void setup(void) overcommit =get_sys_tune ("overcommit_memory" );set_sys_tune ("overcommit_memory" ,1 ,1 ); -tst_cgroup_mount (TST_CGROUP_MEMCG ,PATH_TMP_CG_MEM ); -tst_cgroup_mount (TST_CGROUP_CPUSET ,PATH_TMP_CG_CST ); +tst_cgroup_require (TST_CGROUP_MEMORY ,NULL ); +tst_cgroup_require (TST_CGROUP_CPUSET ,NULL );/* * Some nodes do not contain memory, so use
@@ -96,15 +96,14 @@ void setup(void)if (ret <0 )tst_brk (TBROK ,"Failed to get a memory node " "using get_allowed_nodes()" ); -write_cpusets (PATH_TMP_CG_CST , memnode); +write_cpusets (memnode); }void cleanup (void ) {if (overcommit !=-1 )set_sys_tune ("overcommit_memory" , overcommit,0 ); -tst_cgroup_umount (PATH_TMP_CG_MEM ); -tst_cgroup_umount (PATH_TMP_CG_CST ); +tst_cgroup_cleanup (NULL ); }static struct tst_test test = {
Signature · 1 line
2.29.2
cover CGroups
In reply to Richard PalethorpeHi Richard,
Thanks for your work, comments see below.
Quoted 5 lines · Richard Palethorpe via ltp
This is a request for comment on a new CGroups API. I have tried to keep the existing API functions, but there are many changes in the implementation. Please see the commit message to "CGroup API rewrite" in this series.
Quoted 9 lines
A previous failed attempt to correct some problems and discussion is
here: http://lists.linux.it/pipermail/ltp/2020-November/019586.html
This is a much bigger change than I would like, but CGroups make it very difficult to do anything simply.I have done a direct conversion of the test cases to the new API, but I am not sure that it makes sense to call
tst_cgroup_move_currentwithin the run method of a test because after the first iteration it
Hmm, I feel that is a rare scenario in our real test. Mostly we just need to set it once in a process.
Also, another race condition we are facing is to set the same hierarchy in a different process parallel. i.e.
// Child_1: set MEMSIZE maxbytes
if (fork() == 0) {
tst_cgroup_move_current(TST_CGROUP_MEMORY);
tst_cgroup_mem_set_maxbytes(MEMSIZE);
}
// Child_2: set MEMSIZE/2 maxbytes
if (fork() == 0) {
tst_cgroup_move_current(TST_CGROUP_MEMORY);
tst_cgroup_mem_set_maxbytes(MEMSIZE/2);
}For the previous CGroup test-lib, we solved that via mount the same controller at different places. In this new CGroup lib, I guess creating dynamic directories in tst_cgroup_move_current might work for us, and I'll comment more about it in patch2/5.
Quoted 12 lines
will be a NOP. There is also the issue that on the unified hierarchy when calling
tst_cgroup_move_current(TST_CGROUP_MEMORY);
... testing ...tst_cgroup_move_current(TST_CGROUP_CPUSET);the second call is a NOP as there is only one CGroup, but when the hierarchies are mounted seperately on V1 the current process will not be added to cpuset CGroup until the second call. We probably do not want different behaviour between commonly used hierarchies.
That's true, but it is mainly caused by different versions of CGroup. We could NOT unify the unsupported behavior, so maybe the wiser choice is to let _CPUSET test skipping(TCONF) directly on CGroup_V2?
Quoted 14 lines
In a lot of cases, the test probably only requires the process to be moved into a CGroup during setup, but this requires an investigation of each test. Some tests scan for NUMA information and use this in the CGroup config which complicates matters. However it seems unlikely to me that the available NUMA nodes will change between test iterations unless we are testing hotplugging or failover.
For tests which are actually testing CGroups themselves, most of the API is too high-level. Please see the kernel self tests for what may be required for those kinds of tests.
This is not meant for full review and has not been tested on older setups yet.
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20201231/7a901c48/attachment.htm>
cover CGroups
Hello Li,
Quoted 3 lines · Li Wang
Hi Richard,
Thanks for your work, comments see below.
Thanks for the review!
Quoted 14 lines
Quoted 10 lines · Richard Palethorpe via ltp
This is a request for comment on a new CGroups API. I have tried to keep the existing API functions, but there are many changes in the implementation. Please see the commit message to "CGroup API rewrite" in this series.
A previous failed attempt to correct some problems and discussion is
here: http://lists.linux.it/pipermail/ltp/2020-November/019586.html
This is a much bigger change than I would like, but CGroups make it very difficult to do anything simply.
Quoted 7 lines
Quoted 4 lines
I have done a direct conversion of the test cases to the new API, but I am not sure that it makes sense to call
tst_cgroup_move_currentwithin the run method of a test because after the first iteration itHmm, I feel that is a rare scenario in our real test. Mostly we just need to set it once in a process.
I suppose we can just move it into setup then?
Quoted 34 lines
Also, another race condition we are facing is to set the same hierarchy in a different process parallel. i.e.
//
Child_1: set MEMSIZE maxbytesif (fork() == 0) { tst_cgroup_move_current(TST_CGROUP_MEMORY); tst_cgroup_mem_set_maxbytes(MEMSIZE); } // Child_2: set MEMSIZE/2 maxbytes if (fork() == 0) { tst_cgroup_move_current(TST_CGROUP_MEMORY); tst_cgroup_mem_set_maxbytes(MEMSIZE/2); }For the previous CGroup test-lib, we solved that via mount the same controller at different places. In this new CGroup lib, I guess creating dynamic directories in
tst_cgroup_move_currentmight work for us, and I'll comment more about it in patch2/5.Quoted 12 lines
will be a NOP. There is also the issue that on the unified hierarchy when calling
tst_cgroup_move_current(TST_CGROUP_MEMORY);
... testing ...tst_cgroup_move_current(TST_CGROUP_CPUSET);the second call is a NOP as there is only one CGroup, but when the hierarchies are mounted seperately on V1 the current process will not be added to cpuset CGroup until the second call. We probably do not want different behaviour between commonly used hierarchies.
Quoted 4 lines
That's true, but it is mainly caused by different versions of CGroup. We could NOT unify the unsupported behavior, so maybe the wiser choice is to let
_CPUSETtest skipping(TCONF) directly onCGroup_V2?
Yes. I wonder if tst_cgroup_move_current should return a value to indicate it was a NOP? Or maybe it should throw an error when called on CGroup_V2 and it would have been a NOP?
Signature · 2 lines
Thank you, Richard.
cover CGroups
Quoted 10 lines · Richard Palethorpe
Quoted 7 lines
Quoted 4 lines
I have done a direct conversion of the test cases to the new API, but I am not sure that it makes sense to call
tst_cgroup_move_currentwithin the run method of a test because after the first iteration itHmm, I feel that is a rare scenario in our real test. Mostly we just need to set it once in a process.
I suppose we can just move it into setup then?
For tests that only need a one-time CGroup setting we can, but for multiple requirements of the same controller in many processes it seems not working.
Quoted 9 lines
Quoted 4 lines
That's true, but it is mainly caused by different versions of CGroup. We could NOT unify the unsupported behavior, so maybe the wiser choice is to let
_CPUSETtest skipping(TCONF) directly onCGroup_V2?Yes. I wonder if
tst_cgroup_move_currentshould return a value to indicate it was a NOP? Or maybe it should throw an error when called onCGroup_V2and it would have been a NOP?
That sound like we have to handle the returned value/error additionally? If so, I'd prefer to encapsulate that in CGroup-lib.
Signature · 5 lines
Regards, Li Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linux.it/pipermail/ltp/attachments/20210104/624a097e/attachment.htm>