Allow Cpuset nodesets to expand under pressure

Allow Cpuset memory nodesets to expand under pressure

Users don't always know how much memory they will really need, or how to trade off performance against memory (particularly pagecache) usage.

In order to reduce wastage but still allow applications to get memory when they need it, this patch adds several files to the cpusets directory to allow additional memory nodes to be allocated to a cpuset by the kernel if the cpuset's internal memory pressure gets too high.

The following files are added:

- expansion_limit - the maximum size (in bytes) to which this cpuset's memory set can be expanded

- expansion_pressure - the abstract memory pressure for tasks within the cpuset, ranging from 0 (no pressure) to 100 (about to go OOM) at which expansion can occur

- unused_mems (read-only) - the set of memory nodes that are available to this cpuset and not assigned to the mems set of any of this cpuset's children; automatically maintained by the system

- expansion_mems - a vector of nodelists that determine which nodes should be considered as potential expansion nodes, if available, in priority order

A callback is added from try_to_free_pages() into the cpuset code, passing a "memory pressure" value representing how hard the VM is trying to free memory for the current cpuset. If the pressure is greater than the cpuset's expansion_pressure, and the cpuset's total assigned memory nodes sum to less than its expansion_limit, then the elements of the cpuset's expansion_mems will be checked in turn to find an intersection with the parent's unused_mems. The first such intersecting node found is used to expand the cpuset's memory nodes.

The expansion_pressure value for a cpuset can be used as a simple tuning knob to trade off ease of expansion (== greater memory usage) versus performance, without requiring users to have a deep understanding of how the VM allocation/reclaim mechanisms work.

Controversial points about the patch:

- It makes the synchronization rules in cpuset.c more complex; specifically the rule that you need to take both manage_mutex and callback_mutex in order to modify a cpuset is no longer true; hence even while holding manage_mutex, it's necessary to assume that certain fields (*->mems_generation, *->mems_allowed, cpuset_mems_generation) can change under you if you're not holding callback_mutex. This requires race-detection and retry in update_nodemask() and update_flag().

- During auto-expansion, a cpuset's mems_allowed can change without invoking the same mempolicy rebinding/migration code as when explicitly modified by userspace. I don't think that this is a problem, since expanding a memory set by one node doesn't really cause a useful migration event anyway. But I may be wrong ...

- The "memory pressure" used by the expansion functions is distinct from the existing "memory pressure" reported by cpusets via the flowmeter abstraction. I experimented with using the flowmeter abstraction to trigger expansion events, but decided that it wasn't really suitable - it reports the smoothed frequency of recent memory reclaim events, rather than how hard we're currently trying to find free memory; since a failure to find free memory results in an OOM kill for one of the processes in the CPUset, this value doesn't always show what we need. Perhaps these two concepts of memory pressure could be combined in some way to give a single memory pressure value that's appropriate for both uses; alternatively my "expansion_pressure" could be renamed to something else.

- In an ideal world, the code to pick and allocate an additional node for the cpuset belongs in userspace; this would require one of:

- exporting much more state about the VM to userspace along with a userspace application that's checking very frequently for potentially OOM cpusets (an app allocating memory in busts can go OOM incredibly quickly)

- a new callback mechanism to userspace that's invoked when internal memory pressure reaches a certain value, and requires userspace to quickly decide and assign a new node to the ailing cpuset, whilst avoiding potential deadlocking issues between the userspace process that's updating the cpuset and the memory-allocating process that's stuck in the callback

Allowing the user to (periodically, as necessary) specify nodemasks with decreasing priority, along with a pressure threshold and limit, and letting the kernel use these parameters when running low on memory for a cpuset, seems like a good split between policy in userspace and mechanism in the kernel

TODO: update Documentation/cpusets.txt, once any issues have been resolved

---
 include/linux/cpuset.h |    7 +
 kernel/cpuset.c        |  323 +++++++++++++++++++++++++++++++++++++++++++++++--
 mm/vmscan.c            |   12 +
 3 files changed, 330 insertions(+), 12 deletions(-)

Index: 2.6.19-autoexpand/include/linux/cpuset.h
===================================================================
@@ -46,6 +46,8 @@ extern int cpuset_excl_nodes_overlap(con
 extern int cpuset_memory_pressure_enabled;
 extern void __cpuset_memory_pressure_bump(void);
 
+extern int cpuset_expand_memset(int pressure, int gfp_mask);
+
 extern struct file_operations proc_cpuset_operations;
 extern char *cpuset_task_status_allowed(struct task_struct *task, char *buffer);
 

@@ -106,6 +108,11 @@ static inline int cpuset_excl_nodes_over
 
 static inline void cpuset_memory_pressure_bump(void) {}
 
+static inline int cpuset_expand_memset(int pressure, int gfp_mask)
+{
+	return 0;
+}
+
 static inline char *cpuset_task_status_allowed(struct task_struct *task,
 							char *buffer)
 {

Index: 2.6.19-autoexpand/mm/vmscan.c
===================================================================
@@ -1058,6 +1058,18 @@ unsigned long try_to_free_pages(struct z
 	}
 
 	for (priority = DEF_PRIORITY; priority >= 0; priority--) {
+		/* See if we can expand the current cpuset's
+		 * nodemask. Translate the current priority value into
+		 * an abstract pressure ranging between 0 (no
+		 * pressure) and 100 (about to OOM) */
+		int pressure = (DEF_PRIORITY - priority) * 100 / DEF_PRIORITY;
+		if (!(current->flags & PF_KSWAPD) &&
+		    cpuset_expand_memset(pressure, gfp_mask)) {
+			/* We successfully allocated a new node. This
+			 * counts as progress. */
+			ret = 1;
+			goto out;
+		}
 		sc.nr_scanned = 0;
 		if (!priority)
 			disable_swap_token();

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

I've been struggling with a couple of variations on this theme.

I have one site that needs to have jobs that start swapping get killed.

I have another site that needs to have jobs that are about to start kicking in the OOM killer instead get more memory - though they don't need to run nearly as tight or carefully metered a limiter on how much extra memory the job can get as your expansion mechanism allows for, and they don't want to set aside unused nodes to divy out for these cases, but rather they want to allow the tasks to start taking from the nodes allowed to other jobs (yes, sounds odd, but I think that's a fair statement.)

In both cases, these are very important adaptions to the particular memory needs and workloads of particular sites, and difficult to accomplish with existing mechanisms. And neither variation seems to be easily answered with this patch proposal either.

This patch adds the single largest expansion of per-cpuset attributes of any change we've proposed. My sense is that it is a tad overly specialized to a particular situation (granted, a popular situation.)

But it tries to address a significant cause of difficulty in using cpusets, so I am most encouraged.

How about a loadable module?

Instead of calling out to cpuset code that can expand the jobs cpuset from a pre-defined pool, rather call out to a routine that can be provided by a loadable module. Keep the expansion_pressure, keep the callout, but drop the rest and make the callout pluggable.

Then sites with specialized needs at the point of a particular amount of pressure can provide specialized solutions.

I can imagine two more per-cpuset files, instead of the four above:

memory_expansion_pressure - level, 0-100, at which the callout is called memory_expansion_routine - string name of a registered callout.

A routine would be available to the init routine of loading modules, that let them register a callback by a string name, which would be matched to the 'memory_expansion_routine' name, when a memory request was made in a cpuset exceeding that cpusets specified memory_expansion_pressure.

This would make the API quite a bit more generic and simple, and meet a greater variety of needs.

Don't invoke the callout if the task can't sleep at that point; coders of such loadable modules are ill-prepared to deal with that case, and would sooner let such memory requests be handled as they are now.

Right now, if I had to cast a final vote (there is no 'final' vote, and I wouldn't have it if there was ;), I'd much prefer a loadable module hook here, then this particular 'expansion_mems' mechanism.

I'm open to discussing changing the value reported by 'memory_pressure' into being the unfiltered metric needed here, to consolidate these two metrics. Now that I have some more real world experience with this memory_pressure value, it is proving to have worth about half way between what I hoped it would have, providing a user accessible leading indicator of heavy swapping, and the lesser worth that I'd guess Andrew was predicting for it. If 'memory_pressure' is low, it means we are not swapping heavily. But if it is high, then either we are swapping or we are pushing dirty pages to the file system. If I ended up with a loadable module hook that could be called out at a specified pressure level, that would be a huge improvement from my perspective, and having just a single pressure metric exposed in the API is a worthy goal. I'm sure that Andrew would get a kick out of applying the patch to remove the single-pole low-pass recursive (IIR) filter code in cpuset.c ;).

For API compatibility, we should continue to have a per-cpuset metric called memory_pressure, that tends to get bigger the greater the memory distress, but it's negotiable just what the recipe is for calculating that metric. And it could become a value that is both readable and writable, instead of just the read-only value it is now.

Obligatory nit - some places you have code such as:

+ /* if expansion isn't configured, don't expand */
+ if (cs->expansion_pressure < 0) return 0;
+ /* if memory pressure isn't high enough, don't expand */
+ if (pressure < cs->expansion_pressure) return 0;
+ /* if we're at the limit, don't expand */
+ if (cs->total_pages >= cs->expansion_limit) return 0;

The "if (...)" and the "return 0;" should be on separate lines.

Back to the main line of thought. The locking could get tricky.

Perhaps an analog of CAS (compare and swap) works. Provide the callout code with a routine it can invoke that states in affect:

if my tasks mems_allowed is This
then change it to That
else
return failure (update This in place?)

Then we can invoke the callout routine not holding either of the cpuset locks, manage_mutex or callback_mutex, and we can keep our intricate details of cpuset locking the private business of mainline kernel code, as they should be.

If we could arrange to invoke these callout routines not holding any significant global lock, so that the callout routine could even go so far as to invoke a separate thread to run user space code to muck with cpusets all before returning, then that would be great.

By the way, I just happened to notice that the 0..100 pressure value in this patch seems strangely like the 'distress' value in the mm/vmscan.c:shrink_active_list() code. But I'm no vm guru, so perhaps this is a superficial similarity.

                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

These choice of names are too specific to Paul M's expansion patch.

For the more generic module approach I'm suggesting, these names should be more like:

memory_pressure - level of current pressure on memory applied by tasks in cpuset memory_pressure_trigger - level at which memory_pressure_callout called memory_pressure_callout - string name of loadable module callback function

And we need two 'level' files, a read-only one that indicates the current pressure being imposed on memory, and a read-write one that sets the trigger level at which the callout is invoked.

The 'memory_pressure' read-only level is an aggregate across all the tasks in the cpuset. That much it should continue to be, for API compatibility. As to which aggregate, that's open to discussion.

Since it's an aggregate of something that can only be sampled at discrete events (when the tasks in the cpuset ask for memory) it seems that we need some sort of filtering that is more than just 'the memory pressure of the most recent allocation request by any task in this cpuset.'

The memory_pressure_trigger is evaluated for a particular memory request by a particular task. It is not an aggregate.

So I could imagine changing 'memory_pressure' to a filtered average of the mm/vmscan.c distress values seen in recent allocation requests by tasks in that cpuset.

But it's still distinct from the instantaneous value that triggers the callout.

I don't think Andrew gets to remove that single-pole low-pass recursive (IIR) filter code yet ;).

And as long as it remains a time filtered aggregate, 'memory_pressure' might as well remain just what it is. It's not worth the bother of futzing with it to make what are just hidden internal changes for a token show of greater commonality with the callout trigger metric.

                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>