Memory Binding API v0.2
Subsystems (6)
PATCH 1/1 Memory Binding API v0.2
Here is the latest version of the Mem Binding API. It's a follow-up to the patch posted a week or so ago. It incorporates some changes, and should be a bit more efficient, readable, and functional. Bigger, better, faster, eh? It needs to be patched on top of the Simple binding API that I posted a minute ago..
This API as well as the Simple Binding were both originally part of a NUMA Binding API discussed at length on LSE-Tech, and met with approval there. I hope that these can be included in the mainline.
Cheers!
-Matt
arch/i386/config.in +2 -0
@@ -168,6 +168,8 @@ bool 'Multiquad NUMA system' CONFIG_X86_NUMAQ if [ "$CONFIG_X86_NUMAQ" = y ]; then define_bool CONFIG_MULTIQUAD y + bool 'Memory Binding API Support' CONFIG_MEMBIND + bool 'NUMA Memory Allocation Support' CONFIG_NUMA fi fi
arch/i386/kernel/entry.S +2 -0
@@ -754,6 +754,8 @@ .long sys_sched_setaffinity .long sys_sched_getaffinity .long sys_check_topology + .long sys_mem_setbinding + .long sys_mem_getbinding .rept NR_syscalls-(.-sys_call_table)/4 .long sys_ni_syscall
include/asm-i386/unistd.h +2 -0
@@ -248,6 +248,8 @@#define __NR_sched_setaffinity 241#define __NR_sched_getaffinity 242#define __NR_check_topology 243 +#define __NR_mem_setbinding 244 +#define __NR_mem_getbinding 245/* user-visible error numbers are in the range -1 - -124: see <asm-i386/errno.h> */
include/linux/init_task.h +5 -0
@@ -59,6 +59,11 @@ children: LIST_HEAD_INIT(tsk.children), \ sibling: LIST_HEAD_INIT(tsk.sibling), \ thread_group: LIST_HEAD_INIT(tsk.thread_group), \ + memblk_binding: { \ + bitmask: MEMBLK_NO_BINDING, \ + behavior: MPOL_STRICT, \ + lock: SPIN_LOCK_UNLOCKED \ + }, \ wait_chldexit: __WAIT_QUEUE_HEAD_INITIALIZER(tsk.wait_chldexit),\ real_timer: { \ function: it_real_fn \
include/linux/membind.h +51 -0
51 lines not on this page · raw message
include/linux/mmzone.h +1 -0
@@ -138,6 +138,7 @@unsigned long node_start_mapnr;unsigned long node_size;int node_id; +int memblk_id;/* A unique ID for each memory block */ struct pglist_data *node_next; } pg_data_t;
include/linux/sched.h +4 -0
@@ -27,6 +27,7 @@#include <linux/securebits.h> #include <linux/fs_struct.h> #include <linux/compiler.h> +#include <linux/membind.h> struct exec_domain ;
@@ -302,6 +303,9 @@struct task_struct *pidhash_next;struct task_struct **pidhash_pprev; +/* additional Memory Binding stuff */ +memblk_list_t memblk_binding; +wait_queue_head_t wait_chldexit;/* for wait4() */ struct completion *vfork_done;/* for vfork() */
kernel/sys.c +51 -0
57 lines not on this page · raw message
mm/numa.c +67 -10
118 lines not on this page · raw message
mm/page_alloc.c +5 -0
@@ -42,6 +42,8 @@static int zone_balance_min[MAX_NR_ZONES ]__initdata = {20 ,20 ,20 , };static int zone_balance_max[MAX_NR_ZONES ]__initdata = {255 ,255 ,255 , }; +extern unsigned long memblk_online_map; +/* * Temporary debugging check for pages not lying within a given zone. */
@@ -927,6 +929,9 @@void __init free_area_init (unsigned long *zones_size) {free_area_init_core (0 , &contig_page_data, &mem_map, zones_size,0 ,0 ,0 ); + contig_page_data.node_id =0 ; + contig_page_data.memblk_id =0 ; + memblk_online_map =1UL ; }static int __init setup_mem_frac (char *str)
Hi Matt,
Quoted 5 lines · Matthew Dobson
Here is the latest version of the Mem Binding API. It's a follow-up to the patch posted a week or so ago. It incorporates some changes, and should be a bit more efficient, readable, and functional. Bigger, better, faster, eh? It needs to be patched on top of the Simple binding API that I posted a minute ago..
the patch is a good start for introducing the NUMA API which we definitely need for some coherence among NUMA developments. Also I think stuff like memory hot-add/remove will absolutely need a concept like memblk.
I'm just having some small technical comments:
You're using spinlocks for protecting the memblk structure. Mostly these structures would be just read during the lifetime of a task. Besides: the current syscalls allow changing them only from the current context, there is no danger of getting the wrong values here, because current isn't allocating pages while it changes its memblk_list. Even if you change the memblk variables from another task, it isn't that bad if they're unprotected. After all just before changing the values you might have allocated memory by using the old values... I think we can live without the spinlocks.
The implemented syscalls only change the memblk_list of the current task. Would you please consider extending them to arbitrary PIDs? With reasonable protection, of course.
The include file include/linux/membind.h is kind of small and isn't supposed to grow too much. How about putting this into include/linux/numa.h where we could gather some more NUMA stuff and more things to come with the NUMA API?
In _alloc_pages() you might want to check for online memblks when initialising the mask:
memblk_mask = curr->memblk_binding.bitmask & memblk_online_map;
In the search_twice branch I'd reset the mask to memblk_online_map. There are chances that the desired memblk has been freed in the mean time.
Do you have plans for adding the setlaunch() part of the NUMA API? Guess that should take care of both memblk_list and cpus_allowed...
Thanks,
best regards,
Erich