latest linus-2.5 BK broken
Subsystems (7)
Quoted 8 lines · Linus Torvalds
On Wed, 19 Jun 2002, Rusty Russell wrote:
NO. They want to be node-affine. They don't want to specify what CPUs they attach to.
So you're going to have separate interfaces for that? Gag me with a volvo, but that's idiotic.
Quoted 6 lines
Besides, even that would be broken. You want bitmaps, because bitmaps is really what it is all about. It's NOT about "I must run on this CPU", it can equally well be "I mustn't run on those two CPU's that are hosting the RT part of this thing" or something like that.
Linus
A bit mask is a very good choice for the sched_setaffinity() interface. I would suggest an additional argument be added which would indicate the resource that the process is to be affined to. That way this interface could be used for binding processes to cpus, memory nodes, perhaps NUMA nodes, and, as discussed recently in another thread, other processes. Personally, I see NUMA nodes as an overkill, if a process can be bound to cpus and memory nodes.
There has been an effort made to address the needs for binding processes to processors, memory nodes, etc. for NUMA machines. A proposed API has been developed and implemented. See
http://lse.sourceforge.net/numa/numa_api.html for a spec on the API. Matt Dobson has posted the implementation to lkml as a patch against 2.5 several times, but has not seen much discussion. I could see much of the capabilities provided in the NUMA API being provided through the sched_setaffinity() as described above.
Michael Hohnbaum
[email protected]
Quoted 2 lines · Michael Hohnbaum
A bit mask is a very good choice for the
sched_setaffinity()interface. [...]
thanks :)
Quoted 7 lines
[...] I would suggest an additional argument be added which would indicate the resource that the process is to be affined to. That way this interface could be used for binding processes to cpus, memory nodes, perhaps NUMA nodes, and, as discussed recently in another thread, other processes. Personally, I see NUMA nodes as an overkill, if a process can be bound to cpus and memory nodes.
are you sure we want one generic, process-based affinity interface?
i think the affinity to certain memory regions might need to be more finegrained than this. Eg. it could be useful to define a per-file (per-inode) 'backing store memory node' that the file is affine to. This will eg. cause the pagecache to be allocated in the memory node. Process-based affinity does not describe this in a natural way. Another example, memory maps: we might want to have a certain memory map (vma) allocated in a given memory node, independently of where the process that is faulting a given pages resides.
and it might certainly make sense to have some sort of 'default memory affinity' for a process as well, but this should be a different syscall - it really does a much different thing than CPU affinity. The CPU resource is 'used' only temporarily with little footprint, while memory usage is often for a very long timespan, and the affinity strategies differ greatly. Also, memory as a resource is much more complex than CPU, eg. it must handle things like over-allocation, fallback to 'nearby' nodes if a node is full, etc.
so i'd suggest to actually create a good memory-affinity syscall interface instead of trying to generalize it into the simple, robust, finite CPU-affinity syscalls.
Ingo
another thought would be that the 'default' memory affinity can be derived from the CPU affinity. A default process, one which is affine to all CPUs, can have memory allocated from all memory nodes. A process which is bound to a given set of CPUs, should get its memory allocated from the nodes that 'belong' to those CPUs.
the topology might not be as simple as this, but generally it's the CPU that drives the topology, so a given CPU affinity mask leads to a specific 'preferred memory nodes' bitmask - there isnt much choice needed on the user's part, in fact it might be contraproductive to bind a process to some CPU and bind its memory allocations to a very distant memory node. While mathematically there is not necesserily any 1:1 relationship between CPU affinity and 'best memory affinity', technologically there is.
per-object affinity might still be possible under these scheme, it would override whatever 'default' memory affinity is derived from the CPU affinity mask. [that would enable for example for an important database file to be locked to a given memory node, and helper processes executing on distant CPUs will not cause a distant pagecache page to be allocated.]
another advantage is that this removes the burden from the application writer, of having to figure out the actual memory topology and fitting the CPU affinity to the memory affinity (and vice versa). The kernel can figure out a good default memory affinity based on the CPU affinity mask.
(so everything so far points in the direction of having a simple CPU affinity syscall, which we have now.)
Ingo
Quoted 19 lines · Ingo Molnar
Quoted 6 lines · Michael Hohnbaum
[...] I would suggest an additional argument be added
which would indicate the resource that the process is to be
affined to. That way this interface could be used for binding
processes to cpus, memory nodes, perhaps NUMA nodes, and, as discussed recently in another thread, other processes. Personally, I see NUMA nodes as an overkill, if a process
can be bound to cpus and memory nodes.are you sure we want one generic, process-based affinity interface?
i think the affinity to certain memory regions might need to be more
finegrained than this. Eg. it could be useful to define a per-file
(per-inode) 'backing store memory node' that the file is affine to. This
will eg. cause the pagecache to be allocated in the memory node.
Process-based affinity does not describe this in a natural way. Another
example, memory maps: we might want to have a certain memory map (vma) allocated in a given memory node, independently of where the process that is faulting a given pages resides.
Quoted 8 lines
and it might certainly make sense to have some sort of 'default memory affinity' for a process as well, but this should be a different syscall - it really does a much different thing than CPU affinity. The CPU resource is 'used' only temporarily with little footprint, while memory usage is often for a very long timespan, and the affinity strategies differ greatly. Also, memory as a resource is much more complex than CPU, eg. it must handle things like over-allocation, fallback to 'nearby' nodes if a node is full, etc.
I've attatched copies of the patch that Michael referred to in his email so you
can see where we're going with this. I think that we have (at least the
beginnings) of what you've described. The patch allows processes to bind to specific CPU's (via bitmask) and/or specific memory blocks. You can set these up to complement each other, or to something completely arbitrary (for debugging purposes, etc). It also includes the beginnings of very simple topology info with some simple arch-independent calls (cpu_to_node, node_to_cpu, node_to_memblk, etc.) Of course these do require some lower level hooks for each architecture that wants to use them, but they should be simple calls. I've been sidetracked on other things for about a month, but I plan on getting back to this patch ASAP (this week), and porting it forward to the latest version. It is currently only up to 2.5.14.
If anyone has any suggestions for other features, changes, comments, flames, ANYTHING, please let me know.
Quoted 3 lines
so i'd suggest to actually create a good memory-affinity syscall interface instead of trying to generalize it into the simple, robust, finite CPU-affinity syscalls.
See above ;)
-Matt
Quoted 9 lines
Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
include/asm-i386/core_ibmnumaq.h +61 -0
@@ -0,0 +1,61 @@ +/* +* linux/include/asm-i386/mmzone.h +* +* Written by: Matthew Dobson, IBM Corporation +* +* Copyright (C) 2002, IBM Corp. +* +* All rights reserved. +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, but +* WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or +* NON INFRINGEMENT. See the GNU General Public License for more +* details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. +* +* Send feedback to <[email protected]> +*/ +#ifndef _ASM_CORE_IBMNUMAQ_H_ +#define _ASM_CORE_IBMNUMAQ_H_ + +/* +* These functions need to be defined for every architecture. +* The first five are necessary for the NUMA API to function. +* The last is needed by several pieces of NUMA code. +*/ + +/* Returns the number of the node containing CPU 'cpu' */ +#define _cpu_to_node (cpu) (cpu_to_logical_apicid(cpu) >> 4) + +/* Returns the number of the node containing MemBlk 'memblk' */ +#define _memblk_to_node (memblk) (memblk) + +/* Returns the number of the node containing Node 'nid'. This architecture is flat, +so it is a pretty simple function. */ +#define _node_to_node (nid) (nid) + +/* Returns the number of the first CPU on Node 'node' */ +static inline int _node_to_cpu (int node) +{ +int i, cpu, logical_apicid = node <<4 ; + +for (i =1 ; i <16 ; i <<=1 ) +if ((cpu =logical_apicid_to_cpu (logical_apicid | i)) >=0 ) +return cpu; + +return 0 ; +} + +/* Returns the number of the first MemBlk on Node 'node' */ +#define _node_to_memblk (node) (node) + +#endif /* _ASM_CORE_IBMNUMAQ_H_ */
include/asm-i386/mmzone.h +45 -0
45 lines not on this page · raw message
kernel/Makefile +1 -1
@@ -15,7 +15,7 @@ obj-y = sched.o dma.o fork.o exec_domain.o panic.o printk.o \ module.o exit.o itimer.o info.o time.o softirq.o resource.o \ sysctl.o capability.o ptrace.o timer.o user.o \ - signal.o sys.o kmod.o context.o futex.o platform.o + signal.o sys.o kmod.o context.o futex.o platform.o numa.o obj-$(CONFIG_UID16) += uid16.o obj-$(CONFIG_MODULES) += ksyms.o
kernel/fork.c +14 -0
@@ -707,6 +707,20 @@spin_lock_init (&p->sigmask_lock); }#endif +if (!null_restrict (&p->numa_launch_policy)){ + p->numa_binding = p->numa_launch_policy; + p->cpus_allowed = p->numa_binding.cpus.list & p->numa_restrict.cpus.list; +if (!(p->cpus_allowed & cpu_online_map)) +BUG (); +if (p->cpus_allowed & (1UL <<smp_processor_id ())) + p->thread_info->cpu =smp_processor_id (); +else + p->thread_info->cpu =__ffs (p->cpus_allowed & cpu_online_map); + }else + p->thread_info->cpu =smp_processor_id (); +numa_set_init (&p->numa_launch_policy); +rwlock_init (&p->numa_api_lock); + p->array =NULL ; p->lock_depth =-1 ;/* -1 = no lock */ p->start_time = jiffies;
kernel/numa.c +378 -0
378 lines not on this page · raw message
mm/numa.c +76 -13
137 lines not on this page · raw message
mm/page_alloc.c +7 -0
@@ -41,6 +41,9 @@static int zone_balance_min[MAX_NR_ZONES ]__initdata = {20 ,20 ,20 , };static int zone_balance_max[MAX_NR_ZONES ]__initdata = {255 ,255 ,255 , }; +extern int nummemblks; +extern unsigned long memblk_online_map; +/* * Free_page() adds the page to the free lists. This is optimized for * fast normal cases (no error jumps taken normally).
@@ -955,6 +958,10 @@void __init free_area_init (unsigned long *zones_size) {free_area_init_core (0 , &contig_page_data, &mem_map, zones_size,0 ,0 ,0 ); + contig_page_data.node_id =0 ; + contig_page_data.memblk_id =0 ; + nummemblks =1 ; + memblk_online_map =1UL ; }static int __init setup_mem_frac (char *str)
include/linux/init_task.h +4 -0
@@ -59,6 +59,10 @@ children: LIST_HEAD_INIT(tsk.children), \ sibling: LIST_HEAD_INIT(tsk.sibling), \ thread_group: LIST_HEAD_INIT(tsk.thread_group), \ + numa_restrict: NEW_NUMA_SET, \ + numa_binding: NEW_NUMA_SET, \ + numa_launch_policy: NEW_NUMA_SET, \ + numa_api_lock: RW_LOCK_UNLOCKED, \ wait_chldexit: __WAIT_QUEUE_HEAD_INITIALIZER(tsk.wait_chldexit),\ real_timer: { \ function: it_real_fn \
include/linux/mmzone.h +5 -3
@@ -136,6 +136,7 @@unsigned long node_start_mapnr;unsigned long node_size;int node_id; +int memblk_id;/* A unique ID for each memory block (physical contiguous chunk of memory) */ struct pglist_data *node_next; } pg_data_t;
@@ -163,14 +164,15 @@#define NODE_MEM_MAP (nid) mem_map#define MAX_NR_NODES 1 -#else /* !CONFIG_DISCONTIGMEM */ +#endif /* !CONFIG_DISCONTIGMEM */ -#include <asm/mmzone.h> +#if defined (CONFIG_DISCONTIGMEM ) || defined (CONFIG_NUMA ) +#include <asm/mmzone.h> /* page->zone is currently 8 bits ... */ #define MAX_NR_NODES (255 / MAX_NR_ZONES) -#endif /* !CONFIG_DISCONTIGMEM */ +#endif /* CONFIG_DISCONTIGMEM || CONFIG_NUMA */ #define MAP_ALIGN (x) ((((x) % sizeof(mem_map_t)) == 0) ? (x) : ((x) + \ sizeof(mem_map_t) - ((x) % sizeof(mem_map_t))))
include/linux/numa.h +76 -0
76 lines not on this page · raw message
include/linux/sched.h +7 -0
@@ -28,6 +28,7 @@#include <linux/securebits.h> #include <linux/fs_struct.h> #include <linux/compiler.h> +#include <linux/numa.h> struct exec_domain ;
@@ -286,6 +287,12 @@struct task_struct *pidhash_next;struct task_struct **pidhash_pprev; +/* additional NUMA stuff */ +numa_set_t numa_restrict; +numa_set_t numa_binding; +numa_set_t numa_launch_policy; +rwlock_t numa_api_lock;/* protects the preceding 3 structs */ +wait_queue_head_t wait_chldexit;/* for wait4() */ struct completion *vfork_done;/* for vfork() */
include/linux/smp.h +1 -0
@@ -90,6 +90,7 @@#define cpu_number_map (cpu) 0#define smp_call_function (func,info,retry,wait) ({ 0; })#define cpu_online_map 1 +#define memblk_online_map 1static inline void smp_send_reschedule (int cpu) { }static inline void smp_send_reschedule_all (void ) { }#define __per_cpu_data
kernel/sched.c +2 -4
@@ -357,7 +357,7 @@runqueue_t *rq;preempt_disable (); - rq =this_rq (); + rq =task_rq (p);spin_lock_irq (&rq->lock); p->state =TASK_RUNNING ;
@@ -371,7 +371,6 @@ p->sleep_avg = p->sleep_avg *CHILD_PENALTY /100 ; p->prio =effective_prio (p); } - p->thread_info->cpu =smp_processor_id ();activate_task (p, rq);spin_unlock_irq (&rq->lock);
@@ -1662,8 +1661,7 @@migration_req_t req;runqueue_t *rq; - new_mask &= cpu_online_map; -if (!new_mask) +if (!(new_mask & cpu_online_map))BUG ();preempt_disable ();
include/linux/prctl.h +27 -0
@@ -26,4 +26,31 @@# define PR_FPEMU_NOPRINT 1/* silently emulate fp operations accesses */ # define PR_FPEMU_SIGFPE 2/* don't emulate fp operations, send SIGFPE instead */ +/* Get/Set Restricted CPUs and MemBlks */ +#define PR_SET_RESTRICTED_CPUS 11 +#define PR_SET_RESTRICTED_MEMBLKS 12 +#define PR_GET_RESTRICTED_CPUS 13 +#define PR_GET_RESTRICTED_MEMBLKS 14 + +/* Get CPU/Node */ +#define PR_GET_CPU 15 +#define PR_GET_NODE 16 + +/* X to Node conversion functions */ +#define PR_CPU_TO_NODE 17 +#define PR_MEMBLK_TO_NODE 18 +#define PR_NODE_TO_NODE 19 + +/* Node to X conversion functions */ +#define PR_NODE_TO_CPU 20 +#define PR_NODE_TO_MEMBLK 21 + +/* Set CPU/MemBlk/Memory Bindings */ +#define PR_BIND_TO_CPUS 22 +#define PR_BIND_TO_MEMBLKS 23 +#define PR_BIND_MEMORY 24 + +/* Set Launch Policy */ +#define PR_SET_LAUNCH_POLICY 25 +#endif /* _LINUX_PRCTL_H */
kernel/sys.c +46 -0
58 lines not on this page · raw message
Quoted 11 lines · Ingo Molnar
Quoted 7 lines · Michael Hohnbaum
[...] I would suggest an additional argument be added which would indicate the resource that the process is to be affined to. That way this interface could be used for binding processes to cpus, memory nodes, perhaps NUMA nodes, and, as discussed recently in another thread, other processes. Personally, I see NUMA nodes as an overkill, if a process can be bound to cpus and memory nodes.
Quoted 1 line
are you sure we want one generic, process-based affinity interface?
No, I'm not sure that is what we want. I see that as a compromise solution. Something that would allow some of the simple binding capabilities, but not necessarily a full blown solution.
I agree with your comments below that memory binding/allocation is much more complex than CPU binding, so additional flexibility in specifying memory binding is needed. However, wanting to start simple, the first step is to affine a process to memory on one or more nodes.
Quoted 11 lines
i think the affinity to certain memory regions might need to be more finegrained than this. Eg. it could be useful to define a per-file (per-inode) 'backing store memory node' that the file is affine to. This will eg. cause the pagecache to be allocated in the memory node. Process-based affinity does not describe this in a natural way. Another example, memory maps: we might want to have a certain memory map (vma) allocated in a given memory node, independently of where the process that is faulting a given pages resides.
and it might certainly make sense to have some sort of 'default memory affinity' for a process as well, but this should be a different syscall -
This is close to what is currently implemented - memory is allocated, by default on the node that the process is executing on when the request for memory is made. Even if a process is affined to multiple CPUs that span node boundaries, it is performant to dispatch the process on only one node (providing the cpu cycles are available). The NUMA extensions to the scheduler try to do this. Similarly, all memory for a process should be allocated from that one node. If memory is exhausted on that node, any other nodes that the process has affinity to cpus on should then be used. In other words, each process should have a home node that is preferred for dispatch and memory allocation. The process may have affinity to other nodes, which would be used only if the home quad had a significant resource shortage.
Quoted 10 lines
it really does a much different thing than CPU affinity. The CPU resource is 'used' only temporarily with little footprint, while memory usage is often for a very long timespan, and the affinity strategies differ greatly. Also, memory as a resource is much more complex than CPU, eg. it must handle things like over-allocation, fallback to 'nearby' nodes if a node is full, etc.
so i'd suggest to actually create a good memory-affinity syscall interface instead of trying to generalize it into the simple, robust, finite CPU-affinity syscalls.
We have attempted to do that. Please look at the API definition
http://lse.sourceforge.net/numa/numa_api.html If it would help, we could break out just the memory portion of this API (both in the specification and the implementation) and submit those for comment. What do you think?
Quoted 3 lines
Ingo
Michael Hohnbaum
[email protected]