Mempolicy: Use MPOL_PREFERRED for system-wide default policy

PATCH/RFC Mempolicy: Use MPOL_PREFERRED for system-wide default policy

Against: 2.6.25-rc3-mm1 atop the following patches already in Andrew's tree:
[patch 1/6] mempolicy: convert MPOL constants to enum [patch 2/6] mempolicy: support optional mode flags
[patch 3/6] mempolicy: add MPOL_F_STATIC_NODES flag
[patch 4/6] mempolicy: add bitmap_onto() and bitmap_fold() operations [patch 5/6] mempolicy: add MPOL_F_RELATIVE_NODES flag [patch 6/6] mempolicy: update NUMA memory policy documentation [patch -mm 1/4] mempolicy: move rebind functions
[patch -mm 2/4] mempolicy: create mempolicy_operations structure [patch -mm 3/4] mempolicy: small header file cleanup

and the following two patches currently under discussion:
[patch -mm v2] mempolicy: disallow static or relative flags for local preferred mode
+ fix to the "disallow..." patch

Note: This patch belongs further down in a series of pending mempolicy cleanup and fixes. I'm resending as an RFC because there has been some movement in this area that I think is better addressed by this cleanup. Pending the results of the RFC, I would like to defer this patch and submit with the rest of the series. I plan to do that when mempolicy.c has stabilized with the patches listed above.

V3 -> V4:
+ do_get_mempolicy(): return MPOL_DEFAULT and empty nodemask when specified policy "falls back" to system default policy. Then, if the application uses these results in a subsequent syscall, we'll "do the right thing".

NOTE: do_get_mempolicy() incorrectly "falls back" to system default policy, instead of a non-default task policy, if a vma get_mempolicy() op returns NULL. Fixed by a separate patch.

V2 -> V3:
+ mpol_to_str(): show "default" policy when &default_policy is passed in, rather than the details of the default_policy, in /proc/<pid>/numa_maps.

V1 -> V2:
+ restore BUG()s in switch(policy) default cases -- per Christoph
+ eliminate unneeded re-init of struct mempolicy policy member before freeing

Currently, when one specifies MPOL_DEFAULT via a NUMA memory policy API [set_mempolicy(), mbind() and internal versions], the kernel simply installs a NULL struct mempolicy pointer in the appropriate context: task policy, vma policy, or shared policy. This causes any use of that policy to "fall back" to the next most specific policy scope.

The only use of MPOL_DEFAULT to mean "local allocation" is in the system default policy. This requires extra checks/cases for MPOL_DEFAULT in many mempolicy.c functions. It also requires explanation of this dual meaning for MPOL_DEFAULT in the man pages and kernel documentation.

There is another, "preferred", way to specify local allocation via the APIs. That is using the MPOL_PREFERRED policy mode with an empty nodemask. Internally, the empty nodemask gets converted to a preferred_node id of '-1'. All internal usage of MPOL_PREFERRED will convert the '-1' to the id of the node local to the cpu where the allocation occurs.

System default policy, except during boot, is hard-coded to "local allocation". By using the MPOL_PREFERRED mode with a negative value of preferred node for system default policy, MPOL_DEFAULT will never occur in the 'policy' member of a struct mempolicy. Thus, we can remove all checks for MPOL_DEFAULT when converting policy to a node id/zonelist in the allocation paths.

In slab_node() return local node id when policy pointer is NULL. No need to set a pol value to take the switch default. Replace switch default with BUG()--i.e., shouldn't happen.

With this patch MPOL_DEFAULT is only used in the APIs, including internal calls to do_set_mempolicy() and in the display of policy in /proc/<pid>/numa_maps. It always means "fall back" to the the next most specific policy scope. This simplifies the description of memory policies quite a bit, with no visible change in behavior. This patch updates Documentation to reflect this change.

Tested with set_mempolicy() using numactl with memtoy, and tested mbind() with memtoy. All seems to work "as expected".

 Documentation/vm/numa_memory_policy.txt |   53 ++++++++++----------------------
 mm/mempolicy.c                          |   40 ++++++++++++++----------
 2 files changed, 41 insertions(+), 52 deletions(-)

Index: linux-2.6.25-rc3-mm1/mm/mempolicy.c
===================================================================
@@ -104,9 +104,13 @@ static struct kmem_cache *sn_cache;
    policied. */
 enum zone_type policy_zone = 0;
 
+/*
+ * run-time system-wide default policy => local allocation
+ */
 struct mempolicy default_policy = {
 	.refcnt = ATOMIC_INIT(1), /* never free it */
-	.policy = MPOL_DEFAULT,
+	.policy = MPOL_PREFERRED,
+	.v =  { .preferred_node =  -1 },
 };
 
 static const struct mempolicy_operations {

@@ -605,9 +609,10 @@ static long do_set_mempolicy(unsigned sh
 static void get_zonemask(struct mempolicy *p, nodemask_t *nodes)
 {
 	nodes_clear(*nodes);
+	if (p == &default_policy)
+		return;			/* backwards compatibility */
+
 	switch (p->policy) {
-	case MPOL_DEFAULT:
-		break;
 	case MPOL_BIND:
 		/* Fall through */
 	case MPOL_INTERLEAVE:

@@ -690,7 +695,9 @@ static long do_get_mempolicy(int *policy
 			err = -EINVAL;
 			goto out;
 		}
-	} else
+	} else if (pol == &default_policy)
+		*policy = MPOL_DEFAULT;		/* backwards compatibility */
+	else
 		*policy = pol->policy | pol->flags;
 
 	if (vma) {

@@ -1256,8 +1263,7 @@ static struct mempolicy * get_vma_policy
 		if (vma->vm_ops && vma->vm_ops->get_policy) {
 			pol = vma->vm_ops->get_policy(vma, addr);
 			shared_pol = 1;	/* if pol non-NULL, add ref below */
-		} else if (vma->vm_policy &&
-				vma->vm_policy->policy != MPOL_DEFAULT)
+		} else if (vma->vm_policy)
 			pol = vma->vm_policy;
 	}
 	if (!pol)

@@ -1303,7 +1309,6 @@ static struct zonelist *zonelist_policy(
 			nd = first_node(policy->v.nodes);
 		break;
 	case MPOL_INTERLEAVE: /* should not happen */
-	case MPOL_DEFAULT:
 		nd = numa_node_id();
 		break;
 	default:

@@ -1334,9 +1339,10 @@ static unsigned interleave_nodes(struct 
  */
 unsigned slab_node(struct mempolicy *policy)
 {
-	unsigned short pol = policy ? policy->policy : MPOL_DEFAULT;
+	if (!policy)
+		return numa_node_id();
 
-	switch (pol) {
+	switch (policy->policy) {
 	case MPOL_INTERLEAVE:
 		return interleave_nodes(policy);
 

@@ -1357,10 +1363,10 @@ unsigned slab_node(struct mempolicy *pol
 	case MPOL_PREFERRED:
 		if (policy->v.preferred_node >= 0)
 			return policy->v.preferred_node;
-		/* Fall through */
+		return numa_node_id();
 
 	default:
-		return numa_node_id();
+		BUG();
 	}
 }
 

@@ -1605,8 +1611,6 @@ int __mpol_equal(struct mempolicy *a, st
 	if (a->policy != MPOL_DEFAULT && !mpol_match_intent(a, b))
 		return 0;
 	switch (a->policy) {
-	case MPOL_DEFAULT:
-		return 1;
 	case MPOL_BIND:
 		/* Fall through */
 	case MPOL_INTERLEAVE:

@@ -1624,7 +1628,6 @@ void __mpol_free(struct mempolicy *p)
 {
 	if (!atomic_dec_and_test(&p->refcnt))
 		return;
-	p->policy = MPOL_DEFAULT;
 	kmem_cache_free(policy_cache, p);
 }
 

@@ -1791,7 +1794,7 @@ void mpol_shared_policy_init(struct shar
 	if (policy != MPOL_DEFAULT) {
 		struct mempolicy *newpol;
 
-		/* Falls back to MPOL_DEFAULT on any error */
+		/* Falls back to NULL policy [MPOL_DEFAULT] on any error */
 		newpol = mpol_new(policy, flags, policy_nodes);
 		if (!IS_ERR(newpol)) {
 			/* Create pseudo-vma that contains just the policy */

@@ -1915,9 +1918,14 @@ static inline int mpol_to_str(char *buff
 	char *p = buffer;
 	int l;
 	nodemask_t nodes;
-	unsigned short mode = pol ? pol->policy : MPOL_DEFAULT;
+	unsigned short mode;
 	unsigned short flags = pol ? pol->flags : 0;
 
+	if (!pol || pol == &default_policy)
+		mode = MPOL_DEFAULT;	/* compatibility:  display "default" */
+	else
+		mode = pol->policy;
+
 	switch (mode) {
 	case MPOL_DEFAULT:
 		nodes_clear(nodes);

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>