Skip to contentSkip to conversation

NUMA API for Linux

DISCUSSION Matthew Dobson 8 participants 37 messages Apr 7, 2004 – Apr 15, 2004

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

Andi,
I must be missing something here, but did you not include mempolicy.h
and policy.c in these patches? I can't seem to find them anywhere?!? It's really hard to evaluate your patches if the core of them is missing!

Andrew already mentioned your mistake on the i386 syscalls which needs to be fixed.

Also, this snippet of code is in 2 of your patches (#1 and #6) causing rejects:

@@ -435,6 +445,8 @@

struct page *shmem_nopage(struct vm_area_struct * vma,

                         unsigned long address, int *type);

+int shmem_set_policy(struct vm_area_struct *vma, struct mempolicy *new);
+struct mempolicy *shmem_get_policy(struct vm_area_struct *vma, unsigned long addr);
struct file *shmem_file_setup(char * name, loff_t size, unsigned long
flags);
void shmem_lock(struct file * file, int lock);
int shmem_zero_setup(struct vm_area_struct *);

Just from the patches you posted, I would really disagree that these are ready for merging into -mm.

-Matt

#1 lore raw
Quoted 4 lines · On Wed, 07 Apr 2004 14:24:19 -0700 Matthew Dobson <[email protected]> wrote:

I must be missing something here, but did you not include mempolicy.h
and policy.c in these patches? I can't seem to find them anywhere?!? It's really hard to evaluate your patches if the core of them is missing!

It was in the core patch and also in the last patch I sent Andrew. See ftp://ftp.suse.com/pub/people/ak/numa/* for the full patches

Quoted 3 lines

Andrew already mentioned your mistake on the i386 syscalls which needs to be fixed.

That's already fixed

Quoted 12 lines

Also, this snippet of code is in 2 of your patches (#1 and #6) causing rejects:

(quoted diff) +2 -0
@@ -435,6 +445,8 @@
 
 struct page *shmem_nopage(struct vm_area_struct * vma,
                         unsigned long address, int *type);
+int shmem_set_policy(struct vm_area_struct *vma, struct mempolicy
 *new);
+struct mempolicy *shmem_get_policy(struct vm_area_struct *vma, unsigned
 long addr);
 struct file *shmem_file_setup(char * name, loff_t size, unsigned long
Quoted 3 lines

flags);
void shmem_lock(struct file * file, int lock);
int shmem_zero_setup(struct vm_area_struct *);

It didn't reject for me.

Quoted 2 lines

Just from the patches you posted, I would really disagree that these are ready for merging into -mm.

Why so?

-Andi

#2 lore raw
Quoted 10 lines · Andi Kleen
Quoted 4 lines · On Wed, 07 Apr 2004 14:24:19 -0700 Matthew Dobson <[email protected]> wrote:

I must be missing something here, but did you not include mempolicy.h
and policy.c in these patches? I can't seem to find them anywhere?!? It's really hard to evaluate your patches if the core of them is missing!

It was in the core patch and also in the last patch I sent Andrew. See ftp://ftp.suse.com/pub/people/ak/numa/* for the full patches

Ok.. I'll check that link, but what you posted didn't have the files (mempolicy.h & policy.c) in the patch:

[mcd@arrakis numa_api]$ diffstat numa_api-01-core.patch

 include/linux/gfp.h   |   25 +++++++++++++++++++++++--
 include/linux/mm.h    |   17 +++++++++++++++++
 include/linux/sched.h |    4 ++++
 kernel/sys.c          |    3 +++
 mm/Makefile           |    1 +
 5 files changed, 48 insertions(+), 2 deletions(-)

Maybe it got lost somewhere between your mailer and mine? The patch you posted to LKML yesterday was clearly done without the -N option to diff:

diff -u linux-2.6.5-numa/kernel/sys.c-o linux-2.6.5-numa/kernel/sys.c

Quoted 5 lines
Quoted 3 lines

Andrew already mentioned your mistake on the i386 syscalls which needs to be fixed.

That's already fixed

Good.

Quoted 5 lines
Quoted 5 lines

Also, this snippet of code is in 2 of your patches (#1 and #6) causing rejects:

(quoted diff) +0 -0
@@ -435,6 +445,8 @@
 
Quoted 13 lines
Quoted 10 lines
(quoted diff) +2 -0
 struct page *shmem_nopage(struct vm_area_struct * vma,
                         unsigned long address, int *type);
+int shmem_set_policy(struct vm_area_struct *vma, struct mempolicy
 *new);
+struct mempolicy *shmem_get_policy(struct vm_area_struct *vma, unsigned
 long addr);
 struct file *shmem_file_setup(char * name, loff_t size, unsigned long

flags);
void shmem_lock(struct file * file, int lock);
int shmem_zero_setup(struct vm_area_struct *);

It didn't reject for me.

I don't know why. The same code addition is in both the 'core' patch and the 'shm' patch. Adding it twice causes patch throw a reject.

Quoted 6 lines
Quoted 2 lines

Just from the patches you posted, I would really disagree that these are ready for merging into -mm.

Why so?

-Andi

Well, if for no other reason than all the code isn't posted!

-Matt

#3 lore raw
Quoted 9 lines · On Wed, 07 Apr 2004 14:41:02 -0700 Matthew Dobson <[email protected]> wrote:
Quoted 8 lines · Andi Kleen
Quoted 4 lines · On Wed, 07 Apr 2004 14:24:19 -0700 Matthew Dobson <[email protected]> wrote:

I must be missing something here, but did you not include mempolicy.h
and policy.c in these patches? I can't seem to find them anywhere?!? It's really hard to evaluate your patches if the core of them is missing!

Quoted 5 lines
Quoted 2 lines

It was in the core patch and also in the last patch I sent Andrew. See ftp://ftp.suse.com/pub/people/ak/numa/* for the full patches

Ok.. I'll check that link, but what you posted didn't have the files (mempolicy.h & policy.c) in the patch:

Indeed. Must have gone missing. Here are the files for reference.

The full current broken out patchkit is in
ftp.suse.com:/pub/people/ak/numa/2.6.5mc2/

Core NUMA API code

This is the core NUMA API code. This includes NUMA policy aware wrappers for get_free_pages and alloc_page_vma(). On non NUMA kernels these are defined away.

The system calls mbind (see http://www.firstfloor.org/~andi/mbind.html),
get_mempolicy (http://www.firstfloor.org/~andi/get_mempolicy.html) and set_mempolicy (http://www.firstfloor.org/~andi/set_mempolicy.html) are implemented here.

Adds a vm_policy field to the VMA and to the process. The process also has field for interleaving. VMA interleaving uses the offset into the VMA, but that's not possible for process allocations.

include/linux/gfp.h renamed from include/linux/gfp.h-o +23 -2
diff -u linux-2.6.5-mc2-numa/include/linux/gfp.h-o linux-2.6.5-mc2-numa/include/linux/gfp.h
@@ -4,6 +4,8 @@
 #include <linux/mmzone.h>
 #include <linux/stddef.h>
 #include <linux/linkage.h>
+#include <linux/config.h>
+
 /*
  * GFP bitmasks..
  */
@@ -73,10 +75,29 @@
 	return __alloc_pages(gfp_mask, order, NODE_DATA(nid)->node_zonelists + (gfp_mask & GFP_ZONEMASK));
 }
 
+extern struct page *alloc_pages_current(unsigned gfp_mask, unsigned order);
+struct vm_area_struct;
+
+#ifdef CONFIG_NUMA
+static inline struct page * alloc_pages(unsigned int gfp_mask, unsigned int order)
+{
+	if (unlikely(order >= MAX_ORDER))
+		return NULL;
+
+	return alloc_pages_current(gfp_mask, order);
+}
+extern struct page *__alloc_page_vma(unsigned gfp_mask, struct vm_area_struct *vma, 
+				   unsigned long off);
+
+extern struct page *alloc_page_vma(unsigned gfp_mask, struct vm_area_struct *vma, 
+				   unsigned long addr);
+#else
 #define alloc_pages(gfp_mask, order) \
 		alloc_pages_node(numa_node_id(), gfp_mask, order)
-#define alloc_page(gfp_mask) \
-		alloc_pages_node(numa_node_id(), gfp_mask, 0)
+#define alloc_page_vma(gfp_mask, vma, addr) alloc_pages(gfp_mask, 0)
+#define __alloc_page_vma(gfp_mask, vma, addr) alloc_pages(gfp_mask, 0)
+#endif
+#define alloc_page(gfp_mask) alloc_pages(gfp_mask, 0)
 
 extern unsigned long FASTCALL(__get_free_pages(unsigned int gfp_mask, unsigned int order));
 extern unsigned long FASTCALL(get_zeroed_page(unsigned int gfp_mask));
include/linux/mempolicy.h renamed from include/linux/mempolicy.h-o +219 -0

219 lines not on this page · raw message

include/linux/mm.h renamed from include/linux/mm.h-o +17 -0
diff -u linux-2.6.5-mc2-numa/include/linux/mm.h-o linux-2.6.5-mc2-numa/include/linux/mm.h
@@ -12,6 +12,7 @@
 #include <linux/mmzone.h>
 #include <linux/rbtree.h>
 #include <linux/fs.h>
+#include <linux/mempolicy.h>
 
 #ifndef CONFIG_DISCONTIGMEM          /* Don't use mapnrs, do it properly */
 extern unsigned long max_mapnr;
@@ -47,6 +48,9 @@
  *
  * This structure is exactly 64 bytes on ia32.  Please think very, very hard
  * before adding anything to it.
+ * [Now 4 bytes more on 32bit NUMA machines. Sorry. -AK.
+ * But if you want to recover the 4 bytes justr remove vm_next. It is redundant 
+ * with vm_rb. Will be a lot of editing work though. vm_rb.color is redundant too.] 
  */
 struct vm_area_struct {
 	struct mm_struct * vm_mm;	/* The address space we belong to. */
@@ -77,6 +81,10 @@
 					   units, *not* PAGE_CACHE_SIZE */
 	struct file * vm_file;		/* File we map to (can be NULL). */
 	void * vm_private_data;		/* was vm_pte (shared mem) */
+
+#ifdef CONFIG_NUMA
+	struct mempolicy *vm_policy;	/* NUMA policy for the VMA */
+#endif
 };
 
 /*
@@ -148,6 +156,8 @@
 	void (*close)(struct vm_area_struct * area);
 	struct page * (*nopage)(struct vm_area_struct * area, unsigned long address, int *type);
 	int (*populate)(struct vm_area_struct * area, unsigned long address, unsigned long len, pgprot_t prot, unsigned long pgoff, int nonblock);
+	int (*set_policy)(struct vm_area_struct *vma, struct mempolicy *new);
+	struct mempolicy *(*get_policy)(struct vm_area_struct *vma, unsigned long addr);
 };
 
 /* forward declaration; pte_chain is meant to be internal to rmap.c */
@@ -434,6 +444,8 @@
 
 struct page *shmem_nopage(struct vm_area_struct * vma,
 			unsigned long address, int *type);
+int shmem_set_policy(struct vm_area_struct *vma, struct mempolicy *new);
+struct mempolicy *shmem_get_policy(struct vm_area_struct *vma, unsigned long addr);
 struct file *shmem_file_setup(char * name, loff_t size, unsigned long flags);
 void shmem_lock(struct file * file, int lock);
 int shmem_zero_setup(struct vm_area_struct *);
@@ -636,6 +648,11 @@
 	return vma;
 }
 
+static inline unsigned long vma_pages(struct vm_area_struct *vma)
+{
+	return (vma->vm_end - vma->vm_start) >> PAGE_SHIFT;
+}
+
 extern struct vm_area_struct *find_extend_vma(struct mm_struct *mm, unsigned long addr);
 
 extern unsigned int nr_used_zone_pages(void);
include/linux/sched.h renamed from include/linux/sched.h-o +4 -0
diff -u linux-2.6.5-mc2-numa/include/linux/sched.h-o linux-2.6.5-mc2-numa/include/linux/sched.h
@@ -29,6 +29,7 @@
 #include <linux/completion.h>
 #include <linux/pid.h>
 #include <linux/percpu.h>
+#include <linux/mempolicy.h>
 
 struct exec_domain;
 
@@ -501,6 +502,9 @@
 
 	unsigned long ptrace_message;
 	siginfo_t *last_siginfo; /* For ptrace use.  */
+
+  	struct mempolicy *mempolicy;
+  	short il_next;		/* could be shared with used_math */
 };
 
 static inline pid_t process_group(struct task_struct *tsk)
kernel/sys.c renamed from kernel/sys.c-o +3 -0
diff -u linux-2.6.5-mc2-numa/kernel/sys.c-o linux-2.6.5-mc2-numa/kernel/sys.c
@@ -266,6 +266,9 @@
 cond_syscall(sys_mq_timedreceive)
 cond_syscall(sys_mq_notify)
 cond_syscall(sys_mq_getsetattr)
+cond_syscall(sys_mbind)
+cond_syscall(sys_get_mempolicy)
+cond_syscall(sys_set_mempolicy)
 
 /* arch-specific weak syscall entries */
 cond_syscall(sys_pciconfig_read)
mm/Makefile renamed from mm/Makefile-o +1 -0
diff -u linux-2.6.5-mc2-numa/mm/Makefile-o linux-2.6.5-mc2-numa/mm/Makefile
@@ -12,3 +12,4 @@
 			   slab.o swap.o truncate.o vmscan.o $(mmu-y)
 
 obj-$(CONFIG_SWAP)	+= page_io.o swap_state.o swapfile.o
+obj-$(CONFIG_NUMA) 	+= mempolicy.o
include/linux/mempolicy.h renamed from include/linux/mempolicy.h-o +219 -0

219 lines not on this page · raw message

mm/mempolicy.c renamed from mm/mempolicy.c-o +981 -0

981 lines not on this page · raw message

#4 lore raw
Quoted 15 lines · Andi Kleen
Quoted 12 lines · On Wed, 07 Apr 2004 14:41:02 -0700 Matthew Dobson <[email protected]> wrote:
Quoted 10 lines · Andi Kleen
Quoted 4 lines · On Wed, 07 Apr 2004 14:24:19 -0700 Matthew Dobson <[email protected]> wrote:

I must be missing something here, but did you not include mempolicy.h
and policy.c in these patches? I can't seem to find them anywhere?!? It's really hard to evaluate your patches if the core of them is missing!

It was in the core patch and also in the last patch I sent Andrew. See ftp://ftp.suse.com/pub/people/ak/numa/* for the full patches

Quoted 7 lines
Quoted 2 lines

Ok.. I'll check that link, but what you posted didn't have the files (mempolicy.h & policy.c) in the patch:

Indeed. Must have gone missing. Here are the files for reference.

The full current broken out patchkit is in
ftp.suse.com:/pub/people/ak/numa/2.6.5mc2/

Server isn't taking connections right now. At least for me... :(

Your patch still looks broken. It includes some files twice:

Quoted 4 lines

diff -u linux-2.6.5-mc2-numa/include/linux/mempolicy.h-o linux-2.6.5-mc2-numa/include/linux/mempolicy.h

include/linux/mempolicy.h renamed from include/linux/mempolicy.h-o +0 -0
@@ -0,0 +1,219 @@

<snip>

include/linux/mempolicy.h

Quoted 4 lines

diff -u linux-2.6.5-mc2-numa/include/linux/mempolicy.h-o linux-2.6.5-mc2-numa/include/linux/mempolicy.h

include/linux/mempolicy.h renamed from include/linux/mempolicy.h-o +0 -0
@@ -0,0 +1,219 @@

-Matt

#5 lore raw
In reply to Andi Kleen
Quoted 3 lines · Andi Kleen

diff -u linux-2.6.5-mc2-numa/mm/mempolicy.c-o linux-2.6.5-mc2-numa/mm/mempolicy.c

mm/mempolicy.c renamed from mm/mempolicy.c-o +0 -0

<snip>

Quoted 9 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +9 -0
+/* Do sanity checking on a policy */
+static int check_policy(int mode, unsigned long *nodes)
+{
+	int empty = bitmap_empty(nodes, MAX_NUMNODES);
+	switch (mode) {
+	case MPOL_DEFAULT:
+		if (!empty)
+			return -EINVAL;
+		break;

mm/mempolicy.c line 123

Quoted 10 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +10 -0
+	case MPOL_BIND:
+	case MPOL_INTERLEAVE:
+		/* Preferred will only use the first bit, but allow
+		   more for now. */
+		if (empty)
+			return -EINVAL;
+		break;
+	}
+	return check_online(nodes);
+}

Is there a reason you don't have a case for MPOL_PREFERRED? You have a comment about it in the function, but you don't check the nodemask isn't empty...

Quoted 28 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +28 -0
+/* Copy a node mask from user space. */
+static int get_nodes(unsigned long *nodes, unsigned long *nmask,
+		     unsigned long maxnode, int mode)
+{
+	unsigned long k;
+	unsigned long nlongs;
+	unsigned long endmask;
+
+	--maxnode;
+	nlongs = BITS_TO_LONGS(maxnode);
+	if ((maxnode % BITS_PER_LONG) == 0)
+		endmask = ~0UL;
+	else
+		endmask = (1UL << (maxnode % BITS_PER_LONG)) - 1;
+
+	/* When the user specified more nodes than supported just check
+	   if the non supported part is all zero. */
+	if (nmask && nlongs > BITS_TO_LONGS(MAX_NUMNODES)) {
+		for (k = BITS_TO_LONGS(MAX_NUMNODES); k < nlongs; k++) {
+			unsigned long t;
+			if (get_user(t,  nmask + k))
+				return -EFAULT;
+			if (k == nlongs - 1) {
+				if (t & endmask)
+					return -EINVAL;
+			} else if (t)
+				return -EINVAL;
+		}

mm/mempolicy.c line 162

Quoted 10 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +10 -0
+		nlongs = BITS_TO_LONGS(MAX_NUMNODES);
+		endmask = ~0UL;
+	}
+
+	bitmap_clear(nodes, MAX_NUMNODES);
+	if (nmask && copy_from_user(nodes, nmask, nlongs*sizeof(unsigned long)))
+		return -EFAULT;
+	nodes[nlongs-1] &= endmask;
+	return check_policy(mode, nodes);
+}

In this function, why do we care what bits the user set past MAX_NUMNODES? Why shouldn't we just silently ignore the bits like we do in sys_sched_setaffinity? If a user tries to hand us an 8k bitmask, my opinion is we should just grab as much as we care about (MAX_NUMNODES bits rounded up to the nearest UL).

Quoted 18 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +18 -0
+/* Generate a custom zonelist for the BIND policy. */
+static struct zonelist *bind_zonelist(unsigned long *nodes)
+{
+	struct zonelist *zl;
+	int num, max, nd;
+
+	max = 1 + MAX_NR_ZONES * bitmap_weight(nodes, MAX_NUMNODES);
+	zl = kmalloc(sizeof(void *) * max, GFP_KERNEL);
+	if (!zl)
+		return NULL;
+	num = 0;
+	for (nd = find_first_bit(nodes, MAX_NUMNODES);
+	     nd < MAX_NUMNODES;
+	     nd = find_next_bit(nodes, MAX_NUMNODES, 1+nd)) {
+		int k;
+		for (k = MAX_NR_ZONES-1; k >= 0; k--) {
+			struct zone *z = &NODE_DATA(nd)->node_zones[k];
+			if (!z->present_pages)

mm/mempolicy.c line 191

Quoted 10 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +10 -0
+				continue;
+			zl->zones[num++] = z;
+			if (k > policy_zone)
+				policy_zone = k;
+		}
+	}
+	BUG_ON(num >= max);
+	zl->zones[num] = NULL;
+	return zl;
+}

This seems a bit strange to me. Instead of just allocating a whole struct zonelist, you're allocating part of one? I guess it's safe, since the array is meant to be NULL terminated, but we should put a note in any code using these zonelists that they *aren't* regular zonelists, they will be smaller, and dereferencing arbitrary array elements in the struct could be dangerous. I think we'd be better off creating a kmem_cache_t for these and using *whole* zonelist structures. Allocating part of a well-defined structure makes me a bit nervous...

Quoted 21 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +21 -0
+/* Create a new policy */
+static struct mempolicy *new_policy(int mode, unsigned long *nodes)
+{
+	struct mempolicy *policy;
+	PDprintk("setting mode %d nodes[0] %lx\n", mode, nodes[0]);
+	if (mode == MPOL_DEFAULT)
+		return NULL;
+	policy = kmem_cache_alloc(policy_cache, GFP_KERNEL);
+	if (!policy)
+		return ERR_PTR(-ENOMEM);
+	atomic_set(&policy->refcnt, 1);
+	switch (mode) {
+	case MPOL_INTERLEAVE:
+		bitmap_copy(policy->v.nodes, nodes, MAX_NUMNODES);
+		break;
+	case MPOL_PREFERRED:
+		policy->v.preferred_node = find_first_bit(nodes, MAX_NUMNODES);
+		if (policy->v.preferred_node >= MAX_NUMNODES)
+			policy->v.preferred_node = -1;
+		break;
+	case MPOL_BIND:

mm/mempolicy.c line 223

Quoted 10 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +10 -0
+		policy->v.zonelist = bind_zonelist(nodes);
+		if (policy->v.zonelist == NULL) {
+			kmem_cache_free(policy_cache, policy);
+			return ERR_PTR(-ENOMEM);
+		}
+		break;
+	}
+	policy->policy = mode;
+	return policy;
+}

I'm guessing this is why you aren't checking MPOL_PREFERRED in check_policy()? So the user can call mbind() with MPOL_PREFERRED and an empty nodes bitmap and get the default behavior you mentioned in the comments?

I've got to get some dinner now... I'll keep reading and send more comments as I come up with them.

Cheers!

-Matt

#6 lore raw
Quoted 3 lines · On Wed, 07 Apr 2004 17:58:23 -0700 Matthew Dobson <[email protected]> wrote:

Is there a reason you don't have a case for MPOL_PREFERRED? You have a comment about it in the function, but you don't check the nodemask isn't empty...

Empty prefered is a special case. It means DEFAULT. This is useful when you have a process policy != DEFAULT, but want to set a specific VMA to default. Normally default in a VMA would mean use process policy.

Quoted 5 lines

In this function, why do we care what bits the user set past MAX_NUMNODES? Why shouldn't we just silently ignore the bits like we do in sys_sched_setaffinity? If a user tries to hand us an 8k bitmask, my opinion is we should just grab as much as we care about (MAX_NUMNODES bits rounded up to the nearest UL).

This is to catch uninitialized bits. Otherwise it could work on a kernel with small MAX_NUMNODES, and then suddenly fail on a kernel with bigger MAX_NUMNODES when a node isn't online.

Quoted 8 lines

This seems a bit strange to me. Instead of just allocating a whole struct zonelist, you're allocating part of one? I guess it's safe, since the array is meant to be NULL terminated, but we should put a note in any code using these zonelists that they *aren't* regular zonelists, they will be smaller, and dereferencing arbitrary array elements in the struct could be dangerous. I think we'd be better off creating a kmem_cache_t for these and using *whole* zonelist structures. Allocating part of a well-defined structure makes me a bit nervous...

And that after all the whining about sharing policies? ;-) (a BIND policy will always carry a zonelist). As far as I can see all existing zonelist code just walks it until NULL.

I would not be opposed to always using a full one, but it would use considerably more memory in many cases.

Quoted 4 lines

I'm guessing this is why you aren't checking MPOL_PREFERRED in check_policy()? So the user can call mbind() with MPOL_PREFERRED and an empty nodes bitmap and get the default behavior you mentioned in the comments?

Yep.

-Andi

#7 lore raw
Quoted 11 lines · Andi Kleen
Quoted 3 lines · On Wed, 07 Apr 2004 17:58:23 -0700 Matthew Dobson <[email protected]> wrote:

Is there a reason you don't have a case for MPOL_PREFERRED? You have a comment about it in the function, but you don't check the nodemask isn't empty...

Empty prefered is a special case. It means DEFAULT. This is useful when you have a process policy != DEFAULT, but want to set a specific VMA to default. Normally default in a VMA would mean use process policy.

Ok.. That makes sense.

Quoted 9 lines
Quoted 5 lines

In this function, why do we care what bits the user set past MAX_NUMNODES? Why shouldn't we just silently ignore the bits like we do in sys_sched_setaffinity? If a user tries to hand us an 8k bitmask, my opinion is we should just grab as much as we care about (MAX_NUMNODES bits rounded up to the nearest UL).

This is to catch uninitialized bits. Otherwise it could work on a kernel with small MAX_NUMNODES, and then suddenly fail on a kernel with bigger MAX_NUMNODES when a node isn't online.

I am of the opinion that we should allow currently offline nodes in the user's mask. Those nodes may come online later on, and we should respect the user's request to allocate from those nodes if possible. Just like in sched_setaffinity() we take in the user's mask, and when we actually use the mask to make a decision, we check it against cpu_online_map. Just because a node isn't online at the time of the mbind() call doesn't mean it won't be soon. Besides, we should be checking against node_online_map anyway, because nodes could go away. Well, maybe not right now, but in the near future. Hotplugable memory is a reality, even if we don't support it just yet.

Quoted 9 lines
Quoted 8 lines

This seems a bit strange to me. Instead of just allocating a whole struct zonelist, you're allocating part of one? I guess it's safe, since the array is meant to be NULL terminated, but we should put a note in any code using these zonelists that they *aren't* regular zonelists, they will be smaller, and dereferencing arbitrary array elements in the struct could be dangerous. I think we'd be better off creating a kmem_cache_t for these and using *whole* zonelist structures. Allocating part of a well-defined structure makes me a bit nervous...

Quoted 6 lines

And that after all the whining about sharing policies? ;-) (a BIND policy will always carry a zonelist). As far as I can see all existing zonelist code just walks it until NULL.

I would not be opposed to always using a full one, but it would use considerably more memory in many cases.

I'm not whining about sharing policies because of the space usage, although that is a small side issue. I'm whining about sharing policies because it just makes sense. You've got a data structure that is always dynamically allocated and referenced by pointers, that has no instance specific data in it, and that *already has* an atomic reference counter in it. And you decided not to share this data structure?! In my opinion, it's harder and more code to *not* share it... Instead of copying the structure in mpol_copy(), just atomic_inc(policy->refcnt) and we're pretty much done. You already do an atomic_dec_and_test() in mpol_free()...

-Matt

#8 lore raw
In reply to Andi Kleen

On Wed, 2004-04-07 at 14:45, Andi Kleen wrote:

<snip>

mm/mempolicy.c

Quoted 3 lines

diff -u linux-2.6.5-mc2-numa/mm/mempolicy.c-o linux-2.6.5-mc2-numa/mm/mempolicy.c

mm/mempolicy.c renamed from mm/mempolicy.c-o +0 -0

<snip more>

Quoted 22 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +22 -0
+/* Ensure all existing pages follow the policy. */
+static int
+verify_pages(unsigned long addr, unsigned long end, unsigned long *nodes)
+{
+	while (addr < end) {
+		struct page *p;
+		pte_t *pte;
+		pmd_t *pmd;
+		pgd_t *pgd = pgd_offset_k(addr);
+		if (pgd_none(*pgd)) {
+			addr = (addr + PGDIR_SIZE) & PGDIR_MASK;
+			continue;
+		}
+		pmd = pmd_offset(pgd, addr);
+		if (pmd_none(*pmd)) {
+			addr = (addr + PMD_SIZE) & PMD_MASK;
+			continue;
+		}
+		p = NULL;
+		pte = pte_offset_map(pmd, addr);
+		if (pte_present(*pte))
+			p = pte_page(*pte);

mm/mempolicy.c line 256

Quoted 10 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +10 -0
+		pte_unmap(pte);
+		if (p) {
+			unsigned nid = page_zone(p)->zone_pgdat->node_id;
+			if (!test_bit(nid, nodes))
+				return -EIO;
+		}
+		addr += PAGE_SIZE;
+	}
+	return 0;
+}

Instead of looking up a page's node number by page_zone(p)->zone_pgdat->node_id, you can get the same information much more efficiently by doing some bit-twidling on page->flags. Use page_nodenum(struct page *) from include/linux/mm.h.

Quoted 14 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +14 -0
+/* Apply policy to a single VMA */
+static int policy_vma(struct vm_area_struct *vma, struct mempolicy *new)
+{
+	int err = 0;
+	struct mempolicy *old = vma->vm_policy;
+
+	PDprintk("vma %lx-%lx/%lx vm_ops %p vm_file %p set_policy %p\n",
+		 vma->vm_start, vma->vm_end, vma->vm_pgoff,
+		 vma->vm_ops, vma->vm_file,
+		 vma->vm_ops ? vma->vm_ops->set_policy : NULL);
+
+	if (vma->vm_file)
+		down(&vma->vm_file->f_mapping->i_shared_sem);
+	if (vma->vm_ops && vma->vm_ops->set_policy)

mm/mempolicy.c line 310

Quoted 10 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +10 -0
+		err = vma->vm_ops->set_policy(vma, new);
+	if (!err) {
+		mpol_get(new);
+		vma->vm_policy = new;
+		mpol_free(old);
+	}
+	if (vma->vm_file)
+		up(&vma->vm_file->f_mapping->i_shared_sem);
+	return err;
+}

So it looks like you *are* sharing policies, at least for VMA's in the range of a single mbind() call? This is a good start! ;) Looking further ahead, I'm a bit confused. It seems despite *sharing* VMA's belonging to a single mbind() call, you *copy* VMA's during dup_mmap(), copy_process(), split_vma(), and move_vma(). So in the majority of cases you duplicate policies instead of sharing them, but you *do* share them in some instances? Why the inconsistency?

mm/mempolicy.c line 338

Quoted 5 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +5 -0
+/* Change policy for a memory range */
+asmlinkage long sys_mbind(unsigned long start, unsigned long len,
+			  unsigned long mode,
+			  unsigned long *nmask, unsigned long maxnode,
+			  unsigned flags)

What would you think about giving sys_mbind() a pid argument, like sys_sched_setaffinity()? It would make it much easier for sysadmins to use the API if they didn't have to rewrite applications to make these calls on their own. There's already a plethora of arguments, so one more might be overkill.... Just a thought.

Quoted 54 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +54 -0
+{
+	struct vm_area_struct *vma;
+	struct mm_struct *mm = current->mm;
+	struct mempolicy *new;
+	unsigned long end;
+	DECLARE_BITMAP(nodes, MAX_NUMNODES);
+	int err;
+
+	if ((flags & ~(unsigned long)(MPOL_MF_STRICT)) || mode > MPOL_MAX)
+		return -EINVAL;
+	if (start & ~PAGE_MASK)
+		return -EINVAL;
+	if (mode == MPOL_DEFAULT)
+		flags &= ~MPOL_MF_STRICT;
+	len = (len + PAGE_SIZE - 1) & PAGE_MASK;
+	end = start + len;
+	if (end < start)
+		return -EINVAL;
+	if (end == start)
+		return 0;
+
+	err = get_nodes(nodes, nmask, maxnode, mode);
+	if (err)
+		return err;
+
+	new = new_policy(mode, nodes);
+	if (IS_ERR(new))
+		return PTR_ERR(new);
+
+	PDprintk("mbind %lx-%lx mode:%ld nodes:%lx\n",start,start+len,
+			mode,nodes[0]);
+
+	down_write(&mm->mmap_sem);
+	vma = check_range(mm, start, end, nodes, flags);
+	err = PTR_ERR(vma);
+	if (!IS_ERR(vma))
+		err = mbind_range(vma, start, end, new);
+	up_write(&mm->mmap_sem);
+	mpol_free(new);
+	return err;
+}
+
+/* Set the process memory policy */
+asmlinkage long sys_set_mempolicy(int mode, unsigned long *nmask,
+				   unsigned long maxnode)
+{
+	int err;
+	struct mempolicy *new;
+	DECLARE_BITMAP(nodes, MAX_NUMNODES);
+
+	if (mode > MPOL_MAX)
+		return -EINVAL;
+	err = get_nodes(nodes, nmask, maxnode, mode);
+	if (err)

mm/mempolicy.c line 402

Quoted 10 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +10 -0
+		return err;
+	new = new_policy(mode, nodes);
+	if (IS_ERR(new))
+		return PTR_ERR(new);
+	mpol_free(current->mempolicy);
+	current->mempolicy = new;
+	if (new && new->policy == MPOL_INTERLEAVE)
+		current->il_next = find_first_bit(new->v.nodes, MAX_NUMNODES);
+	return 0;
+}

Why not condense both the sys_mbind() & sys_set_mempolicy() into a single call? The functionality of these calls (and even their code) is very similar. The main difference is there is no need for looking up VMA's and doing locking in the sys_set_mempolicy() case. You overload the output side (sys_get_mempolicy()) to handle both whole process and memory range options, but you don't do the same on the input (sys_mbind() and sys_set_mempolicy()). Saving one syscall and having them behave more symmetrically would be a nice addition...

Quoted 16 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +16 -0
+/* Fill a zone bitmap for a policy */
+static void get_zonemask(struct mempolicy *p, unsigned long *nodes)
+{
+	int i;
+	bitmap_clear(nodes, MAX_NUMNODES);
+	switch (p->policy) {
+	case MPOL_BIND:
+		for (i = 0; p->v.zonelist->zones[i]; i++)
+			__set_bit(p->v.zonelist->zones[i]->zone_pgdat->node_id, nodes);
+		break;
+	case MPOL_DEFAULT:
+		break;
+	case MPOL_INTERLEAVE:
+		bitmap_copy(nodes, p->v.nodes, MAX_NUMNODES);
+		break;
+	case MPOL_PREFERRED:

mm/mempolicy.c line 429

Quoted 10 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +10 -0
+		/* or use current node instead of online map? */
+		if (p->v.preferred_node < 0)
+			bitmap_copy(nodes, node_online_map, MAX_NUMNODES);
+		else	
+			__set_bit(p->v.preferred_node, nodes);
+		break;
+	default:
+		BUG();
+	}	
+}

Shouldn't this be called get_nodemask()? You're setting a bit in a bitmask called 'nodes' for each node the policy is using, so you're getting a nodemask, not a zonemask...

mm/mempolicy.c line 441

Quoted 11 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +11 -0
+static int lookup_node(struct mm_struct *mm, unsigned long addr)
+{
+	struct page *p;
+	int err;
+	err = get_user_pages(current, mm, addr & PAGE_MASK, 1, 0, 0, &p, NULL);
+	if (err >= 0) {
+		err = page_zone(p)->zone_pgdat->node_id;
+		put_page(p);
+	}	
+	return err;
+}

Again, you can save some pointer dereferences if you call page_nodenum(p) instead of looking it up through the pgdat.

mm/mempolicy.c line 461

Quoted 4 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +4 -0
+/* Retrieve NUMA policy */
+asmlinkage long sys_get_mempolicy(int *policy,
+				  unsigned long *nmask, unsigned long maxnode,
+				  unsigned long addr, unsigned long flags)	

I had a thought... Shouldn't all your user pointers be marked as such with __user? Ie:
asmlinkage long sys_get_mempolicy(int __user *policy,
unsigned long __user *nmask, unsigned long maxnode, unsigned long addr, unsigned long flags)

This would apply to the other 2 syscalls as well.

mm/mempolicy.c line 471

Quoted 10 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +10 -0
+{
+	int err, pval;
+	struct mm_struct *mm = current->mm;
+	struct vm_area_struct *vma = NULL; 	
+	struct mempolicy *pol = current->mempolicy;
+
+	if (flags & ~(unsigned long)(MPOL_F_NODE|MPOL_F_ADDR))
+		return -EINVAL;
+	if (nmask != NULL && maxnode < numnodes)
+		return -EINVAL;

Did you mean: if (nmask == NULL || maxnode < numnodes) ?

<snip>

Quoted 9 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +9 -0
+/* Do static interleaving for a VMA with known offset. */
+static unsigned
+offset_il_node(struct mempolicy *pol, struct vm_area_struct *vma, unsigned long off)
+{
+	unsigned target = (unsigned)off % (unsigned)numnodes;
+	int c;
+	int nid = -1;
+	c = 0;
+	do {

mm/mempolicy.c line 595

Quoted 10 lines
mm/mempolicy.c renamed from mm/mempolicy.c-o +10 -0
+		nid = find_next_bit(pol->v.nodes, MAX_NUMNODES, nid+1);
+		if (nid >= MAX_NUMNODES) {
+			nid = -1; 		
+			continue;
+		}
+		c++;
+	} while (c <= target);
+	BUG_ON(nid >= MAX_NUMNODES);
+	return nid;
+}

I think that BUG_ON should be BUG_ON(nid < 0), since it *shouldn't* be possible to get through that loop with nid >= MAX_NUMNODES. The only line that could set nid >= MAX_NUMNODES is nid = find_next_bit(...); and the very next line ensures that if nid is in fact >= MAX_NUMNODES it will be set to -1. It actually looks pretty redundant altogether, but...

More comments to follow...

-Matt

#9 lore raw
Quoted 4 lines

Instead of looking up a page's node number by page_zone(p)->zone_pgdat->node_id, you can get the same information much more efficiently by doing some bit-twidling on page->flags. Use page_nodenum(struct page *) from include/linux/mm.h.

Never noticed that before - I'd prefer we renamed this to page_to_nid before anyone starts using it ... fits with the naming convention of everything else (pfn_to_nid, etc). Nobody uses it right now - I grepped the whole tree.

M.

include/linux/mm.h +1 -1
diff -aurpN -X /home/fletch/.diff.exclude virgin/include/linux/mm.h name_nids/include/linux/mm.h
@@ -340,7 +340,7 @@ static inline unsigned long page_zonenum
 {
 	return (page->flags >> NODEZONE_SHIFT) & (~(~0UL << ZONES_SHIFT));
 }
-static inline unsigned long page_nodenum(struct page *page)
+static inline unsigned long page_to_nid(struct page *page)
 {
 	return (page->flags >> (NODEZONE_SHIFT + ZONES_SHIFT));
 }
#10 lore raw

Sounds good to me. I named it page_to_nodenum() to match page_to_zonenum() which I named to differentiate it from page_zone(). I have no attachment to the names whatsoever, though.

-Matt

Quoted 30 lines · Martin J. Bligh
Quoted 4 lines

Instead of looking up a page's node number by page_zone(p)->zone_pgdat->node_id, you can get the same information much more efficiently by doing some bit-twidling on page->flags. Use page_nodenum(struct page *) from include/linux/mm.h.

Never noticed that before - I'd prefer we renamed this to page_to_nid before anyone starts using it ... fits with the naming convention of everything else (pfn_to_nid, etc). Nobody uses it right now - I grepped the whole tree.

M.

diff -aurpN -X /home/fletch/.diff.exclude virgin/include/linux/mm.h name_nids/include/linux/mm.h

include/linux/mm.h +1 -1
@@ -340,7 +340,7 @@ static inline unsigned long page_zonenum
 {
 	return (page->flags >> NODEZONE_SHIFT) & (~(~0UL << ZONES_SHIFT));
 }
-static inline unsigned long page_nodenum(struct page *page)
+static inline unsigned long page_to_nid(struct page *page)
 {
 	return (page->flags >> (NODEZONE_SHIFT + ZONES_SHIFT));
 }
#11 lore raw
In reply to Andi Kleen

Andi,
I'm sure you're sick of me commenting on your patches without "showing
you the money". I've attached a patch with some of the changes I think would be beneficial. Feel free to let me know which changes you think are crap and which you think are not.

Changes include:

1) Redefine the value of some of the MPOL_* flags
2) Rename check_* to mpol_check_*
3) Remove get_nodes(). This should be done in the same manner as sys_sched_setaffinity(). We shouldn't care about unused high bits.
4) Create mpol_check_flags() to, well, check the flags. As the number of flags and modes grows, it will be easier to do this check in its own function.
5) In the syscalls (sys_mbind() & sys_set_mempolicy()), change 'len' to a size_t, add __user to the declaration of 'nmask', change 'maxnode' to 'nmask_len', and condense 'flags' and 'mode' into 'flags'. The motivation here is to make this syscall similar to sys_sched_setaffinity(). These calls are basically the memory equivalent of set/getaffinity, and should look & behave that way. Also, dropping an argument leaves an opening for a pid argument, which I believe would be good. We should allow processes (with appropriate permissions, of course) to mbind other processes.
6) Change how end is calculated as follows:

	end = PAGE_ALIGN(start+len);
	start &= PAGE_MASK;

Basically, this allows users to pass in a non-page aligned 'start', and makes sure we mbind all pages from the page containing 'start' to the page containing 'start'+'len'.

This patch also shows that sys_mbind() and sys_set_mempolicy() have more commonalities than differences. I believe these two syscalls should be combined into one with the call signature of sys_mbind(). If the user passes a start address and length of 0 (or maybe even a flag?), we bind the whole process, otherwise we bind just a region. This would shrink the patch even more than the measly 3 lines the current patch saves, and save a syscall.

[mcd@arrakis source]$ diffstat ~/linux/patches/265-mm4/mcd_mods.patch

 include/linux/mempolicy.h |   12 ++--
 mm/mempolicy.c            |  119
++++++++++++++++++++++------------------------
 2 files changed, 64 insertions(+), 67 deletions(-)

-Matt

include/linux/mempolicy.h +7 -5
diff -Nurp --exclude-from=/home/mcd/.dontdiff linux-2.6.5-mm4/include/linux/mempolicy.h linux-2.6.5-mcd_numa_api/include/linux/mempolicy.h
@@ -8,20 +8,22 @@
  * Copyright 2003,2004 Andi Kleen SuSE Labs
  */
 
-/* Policies */
+/* Policies aka 'modes' */
 #define MPOL_DEFAULT	0
 #define MPOL_PREFERRED	1
 #define MPOL_BIND	2
 #define MPOL_INTERLEAVE	3
 
-#define MPOL_MAX MPOL_INTERLEAVE
+#define MPOL_MAX	(MPOL_INTERLEAVE)
+/* Reserve low 4 bits for policies, ie: 16 possible 'modes' */
+#define MPOL_MODE_MASK	(0xf)
 
 /* Flags for get_mem_policy */
-#define MPOL_F_NODE	(1<<0)	/* return next IL mode instead of node mask */
-#define MPOL_F_ADDR	(1<<1)	/* look up vma using address */
+#define MPOL_F_NODE	(1<<4)	/* return next IL mode instead of node mask */
+#define MPOL_F_ADDR	(1<<5)	/* look up vma using address */
 
 /* Flags for mbind */
-#define MPOL_MF_STRICT	(1<<0)	/* Verify existing pages in the mapping */
+#define MPOL_MF_STRICT	(1<<6)	/* Verify existing pages in the mapping */
 
 #ifdef __KERNEL__
 
mm/mempolicy.c +57 -62

194 lines not on this page · raw message

#12 lore raw
Quoted 1 line · On Wed, 14 Apr 2004 17:38:37 -0700 Matthew Dobson <[email protected]> wrote:

1) Redefine the value of some of the MPOL_* flags

I don't want to merge the flags the and the mode argument. It's ugly.

Quoted 1 line

2) Rename check_* to mpol_check_*

I really don't understand why you insist on renaming all my functions? I like the current naming, thank you.

Quoted 2 lines

3) Remove get_nodes(). This should be done in the same manner as sys_sched_setaffinity(). We shouldn't care about unused high bits.

I disagree on that. This would break programs that are first tested on a small machine and then later run on a big machine (common case)

Quoted 5 lines

4) Create mpol_check_flags() to, well, check the flags. As the number of flags and modes grows, it will be easier to do this check in its own function.
5) In the syscalls (sys_mbind() & sys_set_mempolicy()), change 'len' to a size_t, add __user to the declaration of 'nmask', change 'maxnode' to

unsigned long is the standard for system calls. Check some others.

Quoted 7 lines

'nmask_len', and condense 'flags' and 'mode' into 'flags'. The motivation here is to make this syscall similar to sys_sched_setaffinity(). These calls are basically the memory equivalent of set/getaffinity, and should look & behave that way. Also, dropping an argument leaves an opening for a pid argument, which I believe would be good. We should allow processes (with appropriate permissions, of course) to mbind other processes.

Messing with other process' VM is a recipe for disaster. There
used to be tons of exploitable races in /proc/pid/mem, I don't want to repeat that. Adding pid to set_mem_policy would be a bit easier, but it would require to add a lock to the task struct for this. Currently it is nice and lockless because it relies on the fact that only the current process can change its own policy. I prefer to keep it lockless, because that keeps the memory allocation fast paths faster.

Quoted 6 lines

6) Change how end is calculated as follows:

	end = PAGE_ALIGN(start+len);
	start &= PAGE_MASK;

Basically, this allows users to pass in a non-page aligned 'start', and makes sure we mbind all pages from the page containing 'start' to the page containing 'start'+'len'.

mprotect() does the EINVAL check on unalignment. I think it's better to follow mprotect here.

-Andi

#13 lore raw
Quoted 7 lines · Andi Kleen
Quoted 1 line · On Wed, 14 Apr 2004 17:38:37 -0700 Matthew Dobson <[email protected]> wrote:

2) Rename check_* to mpol_check_*

I really don't understand why you insist on renaming all my functions? I like the current naming, thank you.

I like the mpol_ flavors because there is no namespace collision on these. When I look at cscope or any other code analysis tool, I know that the mpol_... functions belong together. Makes investigating other people's code easier.

Just my 2 cents,
Robin Holt

#14 lore raw
Quoted 9 lines · Robin Holt
Quoted 7 lines · Andi Kleen
Quoted 1 line · On Wed, 14 Apr 2004 17:38:37 -0700 Matthew Dobson <[email protected]> wrote:

2) Rename check_* to mpol_check_*

I really don't understand why you insist on renaming all my functions? I like the current naming, thank you.

Quoted 7 lines

I like the mpol_ flavors because there is no namespace collision on these. When I look at cscope or any other code analysis tool, I know that the mpol_... functions belong together. Makes investigating other people's code easier.

Just my 2 cents,
Robin Holt

That was my motivation. It's not particularly important, especially since the functions are all declared static. I just thought they were a little more readable when prefixed by mpol_.

-Matt

#15 lore raw
In reply to Andi Kleen
Quoted 8 lines · Andi Kleen
Quoted 1 line · On Wed, 14 Apr 2004 17:38:37 -0700 Matthew Dobson <[email protected]> wrote:

1) Redefine the value of some of the MPOL_* flags

I don't want to merge the flags the and the mode argument. It's ugly.

Well, if you're against adding a pid argument, this change is useless anyway.

Quoted 4 lines
Quoted 1 line

2) Rename check_* to mpol_check_*

I really don't understand why you insist on renaming all my functions? I like the current naming, thank you.

I don't want to rename them *all*! ;) There's a million functions named check_foo(), so why not a few more? :)

Quoted 5 lines
Quoted 2 lines

3) Remove get_nodes(). This should be done in the same manner as sys_sched_setaffinity(). We shouldn't care about unused high bits.

I disagree on that. This would break programs that are first tested on a small machine and then later run on a big machine (common case)

I don't follow this argument... How does this code work on a small machine, but break on a big one:

DECLARE_BITMAP(nodes, MAX_NUMNODES);
BITMAP_CLEAR(nodes, MAX_NUMNODES);
set_bit(4, nodes);
mbind(startaddr, 8 * PAGE_SIZE, MPOL_PREFERRED, nodes, MAX_NUMNODES, flags);

??

Userspace should be declaring an array of unsigned longs based on the size of the machine they're running on. If userspace programs are just declaring arbitrary sized array and hoping that they'll work, they're being stupid. If they're declaring oversized arrays and not zeroing the array before passing it in, then they're being lazy. Either way, we're going to throw the bits away, so do we care if they're being lazy or stupid? If the bits we care about are sane, then we do what userspace tells us. If the bits we care about aren't, we throw an error. Besides, we don't return an error when they pass us too little data, why do we return an error when they pass too much?

I remember one of your concerns was checking that all the nodes with set bits need to be online. I think this is wrong. We need to be checking node_online_map at fault time, not at binding time. Sooner or later memory hotplug will go into the kernel, and then you're (or someone) is going to have to rewrite how this is handled. I'd rather do it right the first time.

Quoted 7 lines
Quoted 5 lines

4) Create mpol_check_flags() to, well, check the flags. As the number of flags and modes grows, it will be easier to do this check in its own function.
5) In the syscalls (sys_mbind() & sys_set_mempolicy()), change 'len' to a size_t, add __user to the declaration of 'nmask', change 'maxnode' to

unsigned long is the standard for system calls. Check some others.

Ok. I see this set of system calls as sort of the illegitimate love-child of mlock()/mprotect() and sched_set/getaffinity(). sys_mlock() was using a size_t, so I copied that...

Quoted 8 lines
Quoted 7 lines

'nmask_len', and condense 'flags' and 'mode' into 'flags'. The motivation here is to make this syscall similar to sys_sched_setaffinity(). These calls are basically the memory equivalent of set/getaffinity, and should look & behave that way. Also, dropping an argument leaves an opening for a pid argument, which I believe would be good. We should allow processes (with appropriate permissions, of course) to mbind other processes.

Quoted 7 lines

Messing with other process' VM is a recipe for disaster. There
used to be tons of exploitable races in /proc/pid/mem, I don't want to repeat that. Adding pid to set_mem_policy would be a bit easier, but it would require to add a lock to the task struct for this. Currently it is nice and lockless because it relies on the fact that only the current process can change its own policy. I prefer to keep it lockless, because that keeps the memory allocation fast paths faster.

We're already grabbing the mmap_sem for writing when we modify the vma's in sys_mbind. I haven't looked at what kind of locking would be necessary if we were modifying a *different* processes vma's, but I figured that would at least be a good start. If it's significantly more complicated than that, you're probably right that it's not worth the effort.

Quoted 9 lines
Quoted 6 lines

6) Change how end is calculated as follows:

	end = PAGE_ALIGN(start+len);
	start &= PAGE_MASK;

Basically, this allows users to pass in a non-page aligned 'start', and makes sure we mbind all pages from the page containing 'start' to the page containing 'start'+'len'.

mprotect() does the EINVAL check on unalignment. I think it's better to follow mprotect here.

Ok. I'm usually a fan of not failing if we don't *have* to, and an unaligned start address is easily fixable. On the other hand, if other syscalls throw errors for it, then it's fine with me.

BTW, any response to the idea of combining sys_mbind() & sys_set_mempolicy()?

-Matt

#16 lore raw
In reply to Matthew Dobson

Replying to myself, but...

Quoted 5 lines · Matthew Dobson

Andi,
I must be missing something here, but did you not include mempolicy.h
and policy.c in these patches? I can't seem to find them anywhere?!? It's really hard to evaluate your patches if the core of them is missing!

Running make -j24 bzImage
In file included from arch/i386/kernel/asm-offsets.c:7:
include/linux/sched.h:32: linux/mempolicy.h: No such file or directory make[1]: *** [arch/i386/kernel/asm-offsets.s] Error 1
make: *** [arch/i386/kernel/asm-offsets.s] Error 2

I'm guessing you just forgot the -N option to diff. You might want to add the -p option when you rediff and repost because it makes your patches an order of magnitude easier to read when you can tell what function the patch is modifying.

-Matt

#17 lore raw
In reply to Matthew Dobson
Quoted 3 lines · Matthew Dobson

Just from the patches you posted, I would really disagree that these are ready for merging into -mm.

I have them all merged up here. I made a number of small changes - additional CONFIG_NUMA ifdefs, whitespace improvements, remove unneeded arch_hugetlb_fault() implementation. The core patch created two copies of the same file in mempolicy.h, compile fix in mmap.c and a few other things.

It builds OK for NUMAQ, although NUMAQ does have a problem:

drivers/built-in.o: In function `acpi_pci_root_add':
drivers/built-in.o(.text+0x22015): undefined reference to `pci_acpi_scan_root'

ppc64+CONFIG_NUMA compiles OK.

#18 lore raw
Quoted 9 lines · On Wed, 7 Apr 2004 14:51:30 -0700 Andrew Morton <[email protected]> wrote:
Quoted 3 lines · Matthew Dobson

Just from the patches you posted, I would really disagree that these are ready for merging into -mm.

I have them all merged up here. I made a number of small changes - additional CONFIG_NUMA ifdefs, whitespace improvements, remove unneeded arch_hugetlb_fault() implementation. The core patch created two copies of the same file in mempolicy.h, compile fix in mmap.c and a few other things.

Sorry about the bad patches. I will try to be more careful in the future.

What was the problem in mmap.c ? I compiled in various combinations (with and without NUMA on i386 and x86-64) and it worked.

And why was arch_hugetlb_fault() unneeded?

Quoted 6 lines

It builds OK for NUMAQ, although NUMAQ does have a problem:

drivers/built-in.o: In function `acpi_pci_root_add':
drivers/built-in.o(.text+0x22015): undefined reference to `pci_acpi_scan_root'

ppc64+CONFIG_NUMA compiles OK.

ppc64 doesn't have the system calls hooked up, but I'm not sure how useful it would be for these boxes anyways (afaik they are pretty uniform)

-Andi

#19 lore raw
Quoted 4 lines
Quoted 1 line

ppc64+CONFIG_NUMA compiles OK.

ppc64 doesn't have the system calls hooked up, but I'm not sure how useful it would be for these boxes anyways (afaik they are pretty uniform)

They actually are fairly keen on doing NUMA for HPC stuff - it makes a significant performance improvement ...

M.

#20 lore raw
Quoted 7 lines · On Wed, 07 Apr 2004 15:39:17 -0700 "Martin J. Bligh" <[email protected]> wrote:
Quoted 4 lines
Quoted 1 line

ppc64+CONFIG_NUMA compiles OK.

ppc64 doesn't have the system calls hooked up, but I'm not sure how useful it would be for these boxes anyways (afaik they are pretty uniform)

They actually are fairly keen on doing NUMA for HPC stuff - it makes a significant performance improvement ...

Ok. Should be straight forward to add system calls them if they're interested.

The hugetlbpages may need some work though, but that's non essential for the beginning.

The new calls are all emulation clean too, you can just hand them through. [i didn't add that yet, but it would be pretty simple]

-Andi

#21 lore raw
In reply to Andi Kleen
Quoted 3 lines · Andi Kleen

What was the problem in mmap.c ? I compiled in various combinations (with and without NUMA on i386 and x86-64) and it worked.

mm/mmap.c: In function `copy_vma':
mm/mmap.c:1531: structure has no member named `vm_policy'

mm/mmap.c renamed from mm/mmap.c~numa-api-vma-policy-hooks-fix +1 -1
@@ -1528,7 +1528,7 @@ struct vm_area_struct *copy_vma(struct v
 
 	find_vma_prepare(mm, addr, &prev, &rb_link, &rb_parent);
 	new_vma = vma_merge(mm, prev, rb_parent, addr, addr + len,
-			vma->vm_flags, vma->vm_file, pgoff, vma->vm_policy);
+			vma->vm_flags, vma->vm_file, pgoff, vma_policy(vma));
 	if (!new_vma) {
 		new_vma = kmem_cache_alloc(vm_area_cachep, SLAB_KERNEL);
 		if (new_vma) {

_

Quoted 1 line

And why was arch_hugetlb_fault() unneeded?

Well we do:

	if (condition-which-evaluates-to-constant-zero-on-CONFIG_HUGETLB=n)
		arch_hugetlb_fault();

so the compiler will never emit a call to the stub function anyway.

But it turns out the stub is needed for now, because !X86 doesn't implement arch_hugetlb_fault(). So I put it back.

Quoted 9 lines
Quoted 6 lines

It builds OK for NUMAQ, although NUMAQ does have a problem:

drivers/built-in.o: In function `acpi_pci_root_add':
drivers/built-in.o(.text+0x22015): undefined reference to `pci_acpi_scan_root'

ppc64+CONFIG_NUMA compiles OK.

ppc64 doesn't have the system calls hooked up, but I'm not sure how useful it would be for these boxes anyways (afaik they are pretty uniform)

Well, it's best that the kernel actually compiles still..

#22 lore raw
In reply to Andrew Morton
Quoted 9 lines · --On Wednesday, April 07, 2004 14:51:30 -0700 Andrew Morton <[email protected]>…
Quoted 3 lines · Matthew Dobson

Just from the patches you posted, I would really disagree that these are ready for merging into -mm.

I have them all merged up here. I made a number of small changes - additional CONFIG_NUMA ifdefs, whitespace improvements, remove unneeded arch_hugetlb_fault() implementation. The core patch created two copies of the same file in mempolicy.h, compile fix in mmap.c and a few other things.

I think there are some design issues that still aren't resolved - we've been over this a bit before, but I still don't think they're fixed. It seems you're still making a copy of the binding structure for every VMA, which seems ... extravagent. Can we share them? IIRC, the only justification was the striping ... and I thought we agreed that was better fixed by using the mod of the offset as a decider?

Maybe I'm just misreading your code, in which case, feel free to spit at me ;-)

M.

#23 lore raw
Quoted 4 lines · On Wed, 07 Apr 2004 15:38:24 -0700 "Martin J. Bligh" <[email protected]> wrote:

I think there are some design issues that still aren't resolved - we've been over this a bit before, but I still don't think they're fixed. It seems you're still making a copy of the binding structure for every VMA, which seems ... extravagent. Can we share them? IIRC, the only

Sharing is only an optimization that adds more code and potential for more bugs (hash tables, locking etc.).

We can discuss changes when someone shows numbers that additional optimizations are needed. I haven't seen such numbers and I'm not convinced sharing is even a good idea from a design standpoint. For the first version I just aimed to get something working with straight forward code.

To put it all in perspective: a policy is 12 bytes on a 32bit machine (assuming MAX_NUMNODES <= 32) and 16 bytes on a 64bit machine (with MAX_NUMNODES <= 64)

-Andi

#24 lore raw
Quoted 9 lines · Andi Kleen

We can discuss changes when someone shows numbers that additional optimizations are needed. I haven't seen such numbers and I'm not convinced sharing is even a good idea from a design standpoint. For the first version I just aimed to get something working with straight forward code.

To put it all in perspective: a policy is 12 bytes on a 32bit machine (assuming MAX_NUMNODES <= 32) and 16 bytes on a 64bit machine (with MAX_NUMNODES <= 64)

sizeof(vm_area_struct) is a very sensitive thing on ia32. If you expect that anyone is likely to actually use the numa API on 32-bit, sharing will be important.

It should be useful for SMT, yes?

#25 lore raw
Quoted 2 lines

sizeof(vm_area_struct) is a very sensitive thing on ia32. If you expect that anyone is likely to actually use the numa API on 32-bit, sharing

Me please ;-)

M.

#26 lore raw
In reply to Andrew Morton
Quoted 7 lines · On Wed, 7 Apr 2004 15:52:25 -0700 Andrew Morton <[email protected]> wrote:
Quoted 6 lines · Andi Kleen

We can discuss changes when someone shows numbers that additional optimizations are needed. I haven't seen such numbers and I'm not convinced sharing is even a good idea from a design standpoint. For the first version I just aimed to get something working with straight forward code.

Quoted 7 lines
Quoted 3 lines

To put it all in perspective: a policy is 12 bytes on a 32bit machine (assuming MAX_NUMNODES <= 32) and 16 bytes on a 64bit machine (with MAX_NUMNODES <= 64)

sizeof(vm_area_struct) is a very sensitive thing on ia32. If you expect that anyone is likely to actually use the numa API on 32-bit, sharing will be important.

I don't really believe that. If it was that way someone would have already done all the obvious space optimizations left on the table... (like using rb_next or merging the rb color into flags)

NUMA API adds a new pointer, but all sharing in the world couldn't fix that.

When you set a policy != default you will also pay the 12 or 16 bytes overhead for the object for each "policy region"

Quoted 1 line

It should be useful for SMT, yes?

Nope. Only for real NUMA.

-Andi

#27 lore raw
Quoted 15 lines · Andi Kleen
Quoted 11 lines · On Wed, 7 Apr 2004 15:52:25 -0700 Andrew Morton <[email protected]> wrote:
Quoted 9 lines · Andi Kleen

We can discuss changes when someone shows numbers that additional optimizations are needed. I haven't seen such numbers and I'm not convinced sharing is even a good idea from a design standpoint. For the first version I just aimed to get something working with straight forward code.

To put it all in perspective: a policy is 12 bytes on a 32bit machine (assuming MAX_NUMNODES <= 32) and 16 bytes on a 64bit machine (with MAX_NUMNODES <= 64)

Quoted 5 lines
Quoted 3 lines

sizeof(vm_area_struct) is a very sensitive thing on ia32. If you expect that anyone is likely to actually use the numa API on 32-bit, sharing will be important.

I don't really believe that.

You better. VMA space exhaustion is one of the reasons for introducing remap_file_pages(). It's an oracle-killer. Like everything else ;)

Quoted 3 lines

If it was that way someone would have already
done all the obvious space optimizations left on the table... (like using rb_next or merging the rb color into flags)

Nope, we're slack.

Quoted 4 lines

NUMA API adds a new pointer, but all sharing in the world couldn't fix that.

When you set a policy != default you will also pay the 12 or 16 bytes overhead for the object for each "policy region"

OK, that's not so bad. So if you don't use the feature the overhead is 4 bytes/vma.

If you _do_ use the feature, what is the overhead? 12 bytes for each and every vma? Or just for the vma's which have a non-default policy?

Your patch takes the CONFIG_NUMA vma from 64 bytes to 68. It would be nice to pull those 4 bytes back somehow.

#28 lore raw
Quoted 2 lines · On Wed, 7 Apr 2004 16:56:39 -0700 Andrew Morton <[email protected]> wrote:

If you _do_ use the feature, what is the overhead? 12 bytes for each and every vma? Or just for the vma's which have a non-default policy?

Just for VMAs with a non default policy. Default policy is always NULL

Quoted 2 lines

Your patch takes the CONFIG_NUMA vma from 64 bytes to 68. It would be nice to pull those 4 bytes back somehow.

Eliminate the RB color field or use rb_next() instead of vm_next. First alternative is cheaper.

-Andi

#29 lore raw
Quoted 2 lines · Andi Kleen

Eliminate the RB color field or use rb_next() instead of vm_next. First alternative is cheaper.

with eliminate I assume you mean to reuse a bit in the vma for that (like vma->vm_flags), somewhere the color bit info is needed to make the rebalanacing in mmap quick but to still guarantee the max height <= 2 * min height.

#30 lore raw
Quoted 8 lines · On Thu, 8 Apr 2004 02:26:26 +0200 Andrea Arcangeli <[email protected]> wrote:
Quoted 2 lines · Andi Kleen

Eliminate the RB color field or use rb_next() instead of vm_next. First alternative is cheaper.

with eliminate I assume you mean to reuse a bit in the vma for that (like vma->vm_flags), somewhere the color bit info is needed to make the rebalanacing in mmap quick but to still guarantee the max height <= 2 * min height.

Yes. Put it either into flags or into the low order bits of a pointer.

-Andi

#31 lore raw
In reply to Andrew Morton
Quoted 3 lines · Andrew Morton

Your patch takes the CONFIG_NUMA vma from 64 bytes to 68. It would be nice to pull those 4 bytes back somehow.

How significant is this vma size issue?

anon_vma objrmap will add 20 bytes to each vma (on 32-bit arches):
8 for prio_tree, 12 for anon_vma linkage in vma, sometimes another 12 for the anon_vma head itself.

anonmm objrmap adds just the 8 bytes for prio_tree, remaining overhead 28 bytes per mm.

Seems hard on Andi to begrudge him 4.

Hugh

#32 lore raw
Quoted 10 lines · Andrew Morton
Quoted 3 lines

Your patch takes the CONFIG_NUMA vma from 64 bytes to 68. It would be nice to pull those 4 bytes back somehow.

How significant is this vma size issue?

anon_vma objrmap will add 20 bytes to each vma (on 32-bit arches):
8 for prio_tree, 12 for anon_vma linkage in vma, sometimes another 12 for the anon_vma head itself.

Ewwww. Isn't some of that shared most of the time though?

Quoted 2 lines

anonmm objrmap adds just the 8 bytes for prio_tree, remaining overhead 28 bytes per mm.

28 bytes per *mm* is nothing, and I still think the prio_tree is completely unneccesary. Nobody has ever demonstrated a real benchmark that needs it, as far as I recall.

Quoted 1 line

Seems hard on Andi to begrudge him 4.

I don't care about the 4 bytes much (other than that the current 64 happens to be a nice size). I just don't see the point in making copies of the binding structure all the time ;-) Refcounts aren't that hard ... didn't Greg do a kref just recently? ...

M.

#33 lore raw
Quoted 5 lines · Martin J. Bligh
Quoted 5 lines · Andrew Morton
Quoted 3 lines

Your patch takes the CONFIG_NUMA vma from 64 bytes to 68. It would be nice to pull those 4 bytes back somehow.

Quoted 7 lines
Quoted 5 lines

How significant is this vma size issue?

anon_vma objrmap will add 20 bytes to each vma (on 32-bit arches):
8 for prio_tree, 12 for anon_vma linkage in vma, sometimes another 12 for the anon_vma head itself.

Ewwww. Isn't some of that shared most of the time though?

The anon_vma head may well be shared with other vmas of the fork group. But the anon_vma linkage is a list_head and a pointer within the vma.

prio_tree is already using a union as much as it can (and a pointer where a list_head would simplify the code); Rajesh was thinking of reusing vm_private_data for one pointer, but I've gone and used it for nonlinear swapout.

Quoted 6 lines
Quoted 2 lines

anonmm objrmap adds just the 8 bytes for prio_tree, remaining overhead 28 bytes per mm.

28 bytes per *mm* is nothing, and I still think the prio_tree is completely unneccesary. Nobody has ever demonstrated a real benchmark that needs it, as far as I recall.

I'm sure an Ingobench will shortly follow that observation.

Hugh

#34 lore raw
In reply to Hugh Dickins
Quoted 7 lines · Hugh Dickins
Quoted 3 lines · Andrew Morton

Your patch takes the CONFIG_NUMA vma from 64 bytes to 68. It would be nice to pull those 4 bytes back somehow.

How significant is this vma size issue?

For some workloads/machines it will simply cause an approximately-proportional reduction in the size of the workload which we can handle.

ie: there are some (oracle) workloads where the kernel craps out due to lowmem vma exhaustion. If they're now using remap_file_pages() for this then it may not be a problem any more. Ingo would know better than I.

Quoted 8 lines

anon_vma objrmap will add 20 bytes to each vma (on 32-bit arches):
8 for prio_tree, 12 for anon_vma linkage in vma, sometimes another 12 for the anon_vma head itself.

anonmm objrmap adds just the 8 bytes for prio_tree, remaining overhead 28 bytes per mm.

Seems hard on Andi to begrudge him 4.

#35 lore raw
Quoted 3 lines

ie: there are some (oracle) workloads where the kernel craps out due to lowmem vma exhaustion. If they're now using remap_file_pages() for this then it may not be a problem any more. Ingo would know better than I.

remap_file_pages() from now on yes.

#36 lore raw
In reply to Andi Kleen
Quoted 1 line · Andi Kleen

NUMA API adds a new pointer, but all sharing in the world couldn't fix that.

that's fine. As we already discussed from my part I only care that the 4bytes for the pointer go away if you compile with CONFIG_NUMA=n, it doesn't really matter much but it'd like if it would be optimized for the long term.