Skip to contentSkip to conversation

{PREFIX:-x86_64}: Convert mempolicies to nodemask_t

PATCH In review Andi Kleen 2 participants 5 messages Sep 10, 2005 – Sep 12, 2005

Subsystems (4)
lore
Share & tools
lore

Andi Kleen

author
#0 lore raw

PATCH 2/3 {PREFIX:-x86_64}: Convert mempolicies to nodemask_t

Convert mempolicies to nodemask_t

[Note to Andrew: this has a hunk for the /proc/*/numa_maps code to make it compile again. But you said you queued a revert patch, with which this will likely conflict. If yes just remove the fs/proc/task_mmu.c hunk. Thanks.]

The NUMA policy code predated nodemask_t so it used open coded bitmaps. Convert everything to nodemask_t. Big patch, but shouldn't have any actual behaviour changes (except I removed one unnecessary check against node_online_map and one unnecessary BUG_ON)

include/linux/mempolicy.h +2 -1
Index: linux/include/linux/mempolicy.h
===================================================================
@@ -31,6 +31,7 @@
 #include <linux/slab.h>
 #include <linux/rbtree.h>
 #include <linux/spinlock.h>
+#include <linux/nodemask.h>
 
 struct vm_area_struct;
 
@@ -63,7 +64,7 @@ struct mempolicy {
 	union {
 		struct zonelist  *zonelist;	/* bind */
 		short 		 preferred_node; /* preferred */
-		DECLARE_BITMAP(nodes, MAX_NUMNODES); /* interleave */
+		nodemask_t	 nodes;		/* interleave */
 		/* undefined for default */
 	} v;
 };
mm/mempolicy.c +52 -67

343 lines not on this page · raw message

fs/proc/task_mmu.c +1 -1
Index: linux/fs/proc/task_mmu.c
===================================================================
@@ -469,7 +469,7 @@ static int show_numa_map(struct seq_file
 		seq_printf(m, " interleave={");
 		first = 1;
 		for_each_node(n) {
-			if (test_bit(n, pol->v.nodes)) {
+			if (node_isset(n, pol->v.nodes)) {
 				if (!first)
 					seq_putc(m,',');
 				else
#1 lore raw
Quoted 1 line · Andi

Convert mempolicies to nodemask_t

Good.

Minor comments as I read the patch.

1) Can the include of 'linux/bitmap.h' be removed from the file linux/include/linux/mempolicy.h?

2) /* AK: shouldn't this error out instead? */
Andi add the above comment on a cpuset_restrict_to_mems_allowed() call.

The question is this - if an app tries to use mbind/set_mempolicy to ask for some memory nodes, some of which are allowed in their current cpuset, and some of which are outside that cpuset, should the kernel silently trim their request to those allowed in the cpuset, or should the kernel complain because some of the requested memory nodes are not allowed.

My guess is that Andi is right (kernel should error), and it was my unrelenting drive to trim to an absolute minimum the size of the cpuset impact on the mempolicy code that led me to the other answer (trim silently).

I suppose I should conjure up a patch that changes this, to what Andi suspects is the proper way.

3) Either this current patch of Andi's, or the patch considered for (2) above should also convert whatever kernel/cpuset.c call the mempolicy.c code is making from bitmaps to nodemasks, rather than convert to bitmaps across the boundary:

cpuset_restrict_to_mems_allowed(nodes_addr(*nodes));

4) Should the following line:

	+	PDprintk("setting mode %d nodes[0] %lx\n", mode, nodes_addr(nodes)[0]);

    instead be:

	+	PDprintk("setting mode %d nodes[0] %lx\n", mode, nodes_addr(*nodes)[0]);

5) If anyone ever (even for debugging) adds something to the nodemask_t structure, then the following line in sys_get_mempolicy() will die a horrible death:

+ err = copy_nodes_to_user(nmask, maxnode, &nodes, sizeof(nodes));

In this, 'nodes' is of type nodemask_t. The address of the bits in a nodemask_t is properly obtained using the nodes_addr() macro, not '&nodes', and the number of bytes in those bits is 'BITS_TO_LONGS(MAX_NUMNODES) * sizeof(unsigned long)', not 'sizeof(nodes)'.

6) How come I don't see changes for the 'compat_sys_get_mempolicy()' routine?

7) Do -not- add one for the next node in interleave_nodes():

+ next = next_node(1+nid, policy->v.nodes);

The next_node() macro, unlike the find_next_bit() macro, already adds one. Off hand, that seems like an annoying inconsistency. But since I may well be the person that introduced it, I'd rather not think about it too much ;).

8) Ditto (7), in offset_il_node():

	+		nid = next_node(nid+1, pol->v.nodes);

Otherwise, looks good.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401
#2 lore raw

pj wrote, responding to ak:

Quoted 8 lines

2) /* AK: shouldn't this error out instead? */
Andi add the above comment on a cpuset_restrict_to_mems_allowed() call.

...

I suppose I should conjure up a patch that changes this, to what Andi suspects is the proper way.

I will wait until your nodemask conversion patch comes back around, so I can patch on top of that.

No sense wasting the two minutes of Andrew's time it would take to untangle the patch collisions.

Or feel free to include this change yourself, if you like, Andi, including the corresponding changes in kernel/cpuset.c.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401
author
#3 lore raw
In reply to Paul Jackson
Quoted 2 lines · Paul Jackson
Quoted 1 line · Andi

Convert mempolicies to nodemask_t

Thanks for the review.

Quoted 2 lines

1) Can the include of 'linux/bitmap.h' be removed from the file linux/include/linux/mempolicy.h?

Done.

Quoted 1 line

2) /* AK: shouldn't this error out instead? */

I'll leave this to you.

Quoted 6 lines

3) Either this current patch of Andi's, or the patch considered for (2) above should also convert whatever kernel/cpuset.c call the mempolicy.c code is making from bitmaps to nodemasks, rather than convert to
bitmaps across the boundary:

	cpuset_restrict_to_mems_allowed(nodes_addr(*nodes));

Leaving this to you too

(BTW I have one patch for cpusets pending to improve the fast path, need to test it a little bit more before sending off though)

Quoted 8 lines

4) Should the following line:

	+	PDprintk("setting mode %d nodes[0] %lx\n", mode, nodes_addr(nodes)[0]);

    instead be:

	+	PDprintk("setting mode %d nodes[0] %lx\n", mode, nodes_addr(*nodes)[0]);

Fixed.

Quoted 2 lines

5) If anyone ever (even for debugging) adds something to the nodemask_t

Fixed.

Quoted 2 lines

6) How come I don't see changes for the 'compat_sys_get_mempolicy()'
routine?

Because it doesn't work with nodemask_ts, but only with variable length user buffers.

Quoted 2 lines

7) Do -not- add one for the next node in interleave_nodes():

Fixed (+ 8)

Thanks,
-Andi

#4 lore raw
Quoted 1 line · Andi

Thanks for the review.

You're welcome.

I'll send off patches for my two items after your patch have cleared the deck.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401