syscalls: Add set_mempolicy numa tests.
PATCH v3 1/1 syscalls: Add set_mempolicy numa tests.
This is initial attempt to replace numa.sh tests that despite having been fixed several times have still many shortcommings that wouldn't easy to fix. It's not finished nor 100% replacement at this point but it should be pretty good start.
The main selling points of these testcases are:
The memory allocated for the testing is tracked exactly. We are using get_mempolicy() with MPOL_F_NODE | MPOL_F_ADDR that returns the node ID on which specified address is allocated on to count pages allocated per node after we set desired memory policy.
We also check for free memory on each numa memory mode and skip nodes that don't have sufficient amount of memory for a particular test. The tests checks usuall for twice as much memory per each node in order to allow for allocations to be "misplaced".
The tests for file based shared interleaved mappings are no longer mapping a single small file but rather than that we accumulate statistic for larger amount of files over longer period of time and we also allow for small offset (currently 10%). We should probably also increase the number of samples we take as currently it's about 5MB in total on x86 although I haven't managed to make this test fail so far. This also fixes the test on Btrfs where the synthetic test that expects the pages to be distributed exactly equally fails.
--- include/tst_numa.h | 79 ++++++++ lib/tst_numa.c | 189 ++++++++++++++++++ runtest/numa | 5 + .../kernel/syscalls/set_mempolicy/.gitignore | 4 + .../kernel/syscalls/set_mempolicy/Makefile | 7 + .../syscalls/set_mempolicy/set_mempolicy.h | 26 +++ .../syscalls/set_mempolicy/set_mempolicy01.c | 117 +++++++++++ .../syscalls/set_mempolicy/set_mempolicy02.c | 115 +++++++++++ .../syscalls/set_mempolicy/set_mempolicy03.c | 110 ++++++++++ .../syscalls/set_mempolicy/set_mempolicy04.c | 140 +++++++++++++ 10 files changed, 792 insertions(+) create mode 100644 include/tst_numa.h create mode 100644 lib/tst_numa.c create mode 100644 testcases/kernel/syscalls/set_mempolicy/.gitignore create mode 100644 testcases/kernel/syscalls/set_mempolicy/Makefile create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy.h create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy01.c create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy02.c create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy03.c create mode 100644 testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c
include/tst_numa.h new file +79 -0
@@ -0,0 +1,79 @@ +/* +* SPDX-License-Identifier: GPL-2.0-or-later +* +* Copyright (c) 2018 Cyril Hrubis <[email protected]> +*/ + +#ifndef TST_NUMA_H__ +#define TST_NUMA_H__ + +/** +* Numa nodemap. +*/ +struct tst_nodemap { +/** Number of nodes in map */ +unsigned int cnt; +/** Page allocation counters */ +unsigned int *counters; +/** Array of numa ids */ +unsigned int map[]; +}; + +/** +* Clears numa counters. The counters are lazy-allocated on first call of this function. +* +* @nodes Numa nodemap. +*/ +void tst_nodemap_reset_counters (struct tst_nodemap *nodes); + +/** +* Prints pages allocated per each node. +* +* @nodes Numa nodemap. +*/ +void tst_nodemap_print_counters (struct tst_nodemap *nodes); + +/** +* Allocates the requested number of pages, using mmap(), faults them and adds +* the amount of pages allocated per node to the nodemap counters. We also make +* sure that only the newly allocated pages are accounted for correctly. The +* nodemap has to have counters initialized which happens on first counter +* reset. +* +* @nodes Nodemap with initialized counters. +* @path If non-NULL the mapping is backed up by a file. +* @pages Number of pages to be allocated. +*/ +void tst_numa_alloc_parse (struct tst_nodemap *nodes,const char *path, +unsigned int pages); + +/** +* Frees nodemap. +* +* @nodes Numa nodemap to be freed. +*/ +void tst_nodemap_free (struct tst_nodemap *nodes); + +/** +* Bitflags for tst_get_nodemap() function. +*/ +enum tst_numa_types { +TST_NUMA_ANY =0x00 , +TST_NUMA_MEM =0x01 , +}; + +/** +* Allocates and returns numa node map, which is an array of numa nodes which +* contain desired resources e.g. memory. +* +* @type Bitflags of enum tst_numa_types specifying desired resources. +* @min_mem_kb Minimal free RAM on memory nodes, if given node has less than +* requested amount of free+buffers memory it's not included in +* the resulting list of nodes. +* +* @return On success returns allocated and initialized struct tst_nodemap which contains +* array of numa node ids that contains desired resources. +*/ +struct tst_nodemap *tst_get_nodemap (int type,size_t min_mem_kb); + +#endif /* TST_NUMA_H__ */
lib/tst_numa.c new file +189 -0
189 lines not on this page · raw message
runtest/numa +5 -0
@@ -11,3 +11,8 @@ move_pages09 move_pages09 move_pages10 move_pages10 move_pages11 move_pages11 move_pages12 move_pages12 + +set_mempolicy01 set_mempolicy01 +set_mempolicy02 set_mempolicy02 +set_mempolicy03 set_mempolicy03 +set_mempolicy04 set_mempolicy04
testcases/kernel/syscalls/set_mempolicy/.gitignore new file +4 -0
@@ -0,0 +1,4 @@ +/set_mempolicy01 +/set_mempolicy02 +set_mempolicy03 +set_mempolicy04
testcases/kernel/syscalls/set_mempolicy/Makefile new file +7 -0
@@ -0,0 +1,7 @@ +top_srcdir ?= ../../../.. + +include $(top_srcdir)/include/mk/testcases.mk + +LDLIBS += $(NUMA_LIBS) + +include $(top_srcdir)/include/mk/generic_leaf_target.mk
testcases/kernel/syscalls/set_mempolicy/set_mempolicy.h new file +26 -0
@@ -0,0 +1,26 @@ +/* +* SPDX-License-Identifier: GPL-2.0-or-later +* +* Copyright (c) 2018 Cyril Hrubis <[email protected]> +*/ + +#ifndef SET_MEMPOLICY_H__ +#define SET_MEMPOLICY_H__ + +static const char *mode_name (int mode) +{ +switch (mode) { +case MPOL_DEFAULT : +return "MPOL_DEFAULT" ; +case MPOL_BIND : +return "MPOL_BIND" ; +case MPOL_PREFERRED : +return "MPOL_PREFERRED" ; +//case MPOL_LOCAL: +// return "MPOL_LOCAL"; +default : +return "???" ; + } +} + +#endif /* SET_MEMPOLICY_H__ */
testcases/kernel/syscalls/set_mempolicy/set_mempolicy01.c new file +117 -0
117 lines not on this page · raw message
testcases/kernel/syscalls/set_mempolicy/set_mempolicy02.c new file +115 -0
115 lines not on this page · raw message
testcases/kernel/syscalls/set_mempolicy/set_mempolicy03.c new file +110 -0
110 lines not on this page · raw message
testcases/kernel/syscalls/set_mempolicy/set_mempolicy04.c new file +140 -0
140 lines not on this page · raw message
Signature · 1 line
2.18.1
PATCH v3 1/1 syscalls: Add set_mempolicy numa tests.
----- Original Message -----
Quoted 27 lines
This is initial attempt to replace
numa.shtests that despite having been fixed several times have still many shortcommings that wouldn't easy to fix. It's not finished nor 100% replacement at this point but it should be pretty good start.The main selling points of these testcases are:
The memory allocated for the testing is tracked exactly. We are using
get_mempolicy()withMPOL_F_NODE|MPOL_F_ADDRthat returns the node ID on which specified address is allocated on to count pages allocated per node after
we set desired memory policy.We also check for free memory on each numa memory mode and skip nodes that don't have sufficient amount of memory for a particular test. The tests checks usuall for twice as much memory per each node in order to allow for allocations to be "misplaced".
The tests for file based shared interleaved mappings are no longer mapping a single small file but rather than that we accumulate statistic for larger amount of files over longer period of time and we also allow for small offset (currently 10%). We should probably also increase the number of samples we take as currently it's about 5MB in total on x86 although I haven't managed to make this test fail so far. This also fixes the test on Btrfs where the synthetic test that expects the pages to be distributed exactly equally fails.
Quoted 4 lines
Signed-off-by Cyril Hrubis <[email protected]>CC Michal Hocko <[email protected]>CC Vlastimil Babka <[email protected]>CC Jan Stancek <[email protected]>
Hi,
lib/tst_numa.c line 62
Quoted 4 lines
lib/tst_numa.c new file +4
-0+ + +void tst_numa_alloc_parse (struct tst_nodemap *nodes,const char *path, +unsigned int pages)
I'd change this to a function that only counts allocated pages, and increases counters.
Quoted 13 lines
lib/tst_numa.c new file +13
-0+{ +size_t page_size =getpagesize (); +char *ptr; +int fd =-1 ; +int flags =MAP_PRIVATE |MAP_ANONYMOUS ; +int node; +unsigned int i; + +if (path) { + fd =SAFE_OPEN (path,O_CREAT |O_EXCL |O_RDWR ,0666 ); +SAFE_FTRUNCATE (fd, pages * page_size); + flags =MAP_SHARED ; + }
lib/tst_numa.c line 85
Quoted 10 lines
lib/tst_numa.c new file +10
-0+ + ptr =SAFE_MMAP (NULL , pages * page_size, +PROT_READ |PROT_WRITE , flags, fd,0 ); + +if (path) { +SAFE_CLOSE (fd); +SAFE_UNLINK (path); + } + +memset (ptr,'a' , pages * page_size);
I think all the alloc stuff shouldn't be in library. It's quite possible tests will need different ways to allocate memory.
Maybe the test will want to mmap, but not touch any pages, or pass different flags to mmap. Allocate with other functions than mmap, etc.
The rest of API looks good to me.
Regards,
Jan
Quoted 16 lines
lib/tst_numa.c new file +3
-0+ +for (i =0 ; i < pages; i++) { + get_mempolicy(&node, NULL, 0, ptr + i * page_size, MPOL_F_NODE |
MPOL_F_ADDR);lib/tst_numa.c new file +12
-0+ +if (node <0 || (unsigned int )node >= nodes->cnt) { +tst_res (TWARN ,"get_mempolicy(...) returned invalid node %i\n" , node); +continue ; + } + +inc_counter (node, nodes); + } + +SAFE_MUNMAP (ptr, pages * page_size); +} +
PATCH v3 1/1 syscalls: Add set_mempolicy numa tests.
Hi!
Quoted 7 lines
lib/tst_numa.c line 62
Quoted 4 lines
lib/tst_numa.c new file +4
-0+ + +void tst_numa_alloc_parse (struct tst_nodemap *nodes,const char *path, +unsigned int pages)I'd change this to a function that only counts allocated pages, and increases counters.
Fair enough.
Quoted 27 lines
Quoted 13 lines
lib/tst_numa.c new file +13
-0+{ +size_t page_size =getpagesize (); +char *ptr; +int fd =-1 ; +int flags =MAP_PRIVATE |MAP_ANONYMOUS ; +int node; +unsigned int i; + +if (path) { + fd =SAFE_OPEN (path,O_CREAT |O_EXCL |O_RDWR ,0666 ); +SAFE_FTRUNCATE (fd, pages * page_size); + flags =MAP_SHARED ; + }lib/tst_numa.c line 85
Quoted 10 lines
lib/tst_numa.c new file +10
-0+ + ptr =SAFE_MMAP (NULL , pages * page_size, +PROT_READ |PROT_WRITE , flags, fd,0 ); + +if (path) { +SAFE_CLOSE (fd); +SAFE_UNLINK (path); + } + +memset (ptr,'a' , pages * page_size);I think all the alloc stuff shouldn't be in library. It's quite possible tests will need different ways to allocate memory.
Quoted 3 lines
Maybe the test will want to mmap, but not touch any pages, or pass different flags to mmap. Allocate with other functions than mmap, etc.
It should be put into some kind of common place though since it's used from several tests even at this point.
What about splitting the tst_numa_alloc_parse() to tst_numa_alloc() and tst_numa_parse() and keeping the functionality in the library?
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH v3 1/1 syscalls: Add set_mempolicy numa tests.
----- Original Message -----
Quoted 38 lines
Hi!
Quoted 7 lines
lib/tst_numa.c line 62
Quoted 4 lines
lib/tst_numa.c new file +4
-0+ + +void tst_numa_alloc_parse (struct tst_nodemap *nodes,const char *path, +unsigned int pages)I'd change this to a function that only counts allocated pages, and increases counters.
Fair enough.
Quoted 27 lines
Quoted 13 lines
lib/tst_numa.c new file +13
-0+{ +size_t page_size =getpagesize (); +char *ptr; +int fd =-1 ; +int flags =MAP_PRIVATE |MAP_ANONYMOUS ; +int node; +unsigned int i; + +if (path) { + fd =SAFE_OPEN (path,O_CREAT |O_EXCL |O_RDWR ,0666 ); +SAFE_FTRUNCATE (fd, pages * page_size); + flags =MAP_SHARED ; + }lib/tst_numa.c line 85
Quoted 10 lines
lib/tst_numa.c new file +10
-0+ + ptr =SAFE_MMAP (NULL , pages * page_size, +PROT_READ |PROT_WRITE , flags, fd,0 ); + +if (path) { +SAFE_CLOSE (fd); +SAFE_UNLINK (path); + } + +memset (ptr,'a' , pages * page_size);I think all the alloc stuff shouldn't be in library. It's quite possible tests will need different ways to allocate memory.
Quoted 6 lines
Quoted 3 lines
Maybe the test will want to mmap, but not touch any pages, or pass different flags to mmap. Allocate with other functions than mmap, etc.
It should be put into some kind of common place though since it's used from several tests even at this point.
set_mempolicy sub-dir?
Quoted 3 lines
What about splitting the
tst_numa_alloc_parse()totst_numa_alloc()andtst_numa_parse()and keeping the functionality in the library?
That looks more flexible.
Though I'm not 100% sold on single alloc func in library. My concern is it will be difficult for it to cover all possible scenarios and API will keep changing as we find more.
Regards,
Jan
Quoted 5 lines
--
Cyril Hrubis
[email protected]
PATCH v3 1/1 syscalls: Add set_mempolicy numa tests.
Hi!
Quoted 11 lines
Quoted 9 lines
Quoted 6 lines
I think all the alloc stuff shouldn't be in library. It's quite possible tests will need different ways to allocate memory.
Maybe the test will want to mmap, but not touch any pages, or pass different flags to mmap. Allocate with other functions than mmap, etc.
It should be put into some kind of common place though since it's used from several tests even at this point.
set_mempolicysub-dir?
That wouldn't not work once I will add mbind tests.
Quoted 8 lines
Quoted 2 lines
What about splitting the
tst_numa_alloc_parse()totst_numa_alloc()andtst_numa_parse()and keeping the functionality in the library?That looks more flexible.
Though I'm not 100% sold on single alloc func in library. My concern is it will be difficult for it to cover all possible scenarios and API will keep changing as we find more.
The worst that can happen is that we will end up with a few alloc functions each for different purpose, but I do not think that this is necessarily bad.
Signature · 2 lines
Cyril Hrubis [email protected]