maps: PSS(proportional set size) accounting in smaps

The "proportional set size" (PSS) of a process is the count of pages it has in memory, where each page is divided by the number of processes sharing it. So if a process has 1000 pages all to itself, and 1000 shared with one other process, its PSS will be 1500.
- lwn.net: "ELC: How much memory are applications really using?"

The PSS proposed by Matt Mackall is a very nice metic for measuring an process's memory footprint. So collect and export it via /proc/<pid>/smaps.

Matt Mackall's pagemap/kpagemap and John Berthels's exmap can also do the job. They are comprehensive tools. But for PSS, let's do it in the simple way.

---
 fs/proc/task_mmu.c |   29 ++++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)

@@ -324,6 +324,27 @@ struct mem_size_stats
 	unsigned long private_clean;
 	unsigned long private_dirty;
 	unsigned long referenced;
+
+	/*
+	 * Proportional Set Size(PSS): my share of RSS.
+	 *
+	 * PSS of a process is the count of pages it has in memory, where each
+	 * page is divided by the number of processes sharing it.  So if a
+	 * process has 1000 pages all to itself, and 1000 shared with one other
+	 * process, its PSS will be 1500.               - Matt Mackall, lwn.net
+	 */
+	u64 	      pss;
+	/*
+	 * To keep (accumulated) division errors low, we adopt 64bit pss and
+	 * use some low bits for division errors. So (pss >> PSS_DIV_BITS)
+	 * would be the real byte count.
+	 *
+	 * A shift of 12 before division means(assuming 4K page size):
+	 * 	- 1M 3-user-pages add up to 8KB errors;
+	 * 	- supports mapcount up to 2^24, or 16M;
+	 * 	- supports PSS up to 2^52 bytes, or 4PB.
+	 */
+#define PSS_DIV_BITS	12
 };
 
 struct smaps_arg

@@ -341,6 +362,7 @@ static int smaps_pte_range(pmd_t *pmd, u
 	pte_t *pte, ptent;
 	spinlock_t *ptl;
 	struct page *page;
+	int mapcount;
 
 	pte = pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl);
 	for (; addr != end; pte++, addr += PAGE_SIZE) {

@@ -357,16 +379,19 @@ static int smaps_pte_range(pmd_t *pmd, u
 		/* Accumulate the size in pages that have been accessed. */
 		if (pte_young(ptent) || PageReferenced(page))
 			mss->referenced += PAGE_SIZE;
-		if (page_mapcount(page) >= 2) {
+		mapcount = page_mapcount(page);
+		if (mapcount >= 2) {
 			if (pte_dirty(ptent))
 				mss->shared_dirty += PAGE_SIZE;
 			else
 				mss->shared_clean += PAGE_SIZE;
+			mss->pss += (PAGE_SIZE << PSS_DIV_BITS) / mapcount;
 		} else {
 			if (pte_dirty(ptent))
 				mss->private_dirty += PAGE_SIZE;
 			else
 				mss->private_clean += PAGE_SIZE;
+			mss->pss += (PAGE_SIZE << PSS_DIV_BITS);
 		}
 	}
 	pte_unmap_unlock(pte - 1, ptl);

@@ -395,6 +420,7 @@ static int show_smap(struct seq_file *m,
 	seq_printf(m,
 		   "Size:           %8lu kB\n"
 		   "Rss:            %8lu kB\n"
+		   "Pss:            %8lu kB\n"
 		   "Shared_Clean:   %8lu kB\n"
 		   "Shared_Dirty:   %8lu kB\n"
 		   "Private_Clean:  %8lu kB\n"

@@ -402,6 +428,7 @@ static int show_smap(struct seq_file *m,
 		   "Referenced:     %8lu kB\n",
 		   (vma->vm_end - vma->vm_start) >> 10,
 		   sarg.mss.resident >> 10,
+		   (unsigned long)(sarg.mss.pss >> (10 + PSS_DIV_BITS)),
 		   sarg.mss.shared_clean  >> 10,
 		   sarg.mss.shared_dirty  >> 10,
 		   sarg.mss.private_clean >> 10,

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

right. I'm rather reluctant to merge anything which could have been done from userspace via the maps2 interfaces.

See, this is why I think the kernel needs a ./userspace-tools/ directory. If we had that, you might have implemented this as a little proglet which parses the maps2 files. But we don't have that, so you ended up doing it in-kernel.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Andrew, I second the userspace-tools idea. I would also add an FAQ in that directory, explaining what problem each tool solves. I think your page cache control program would be a great example of something to put in there.

	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

It's called the util-linux package.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

I am looking at http://www.kernel.org/pub/linux/utils/util-linux/ and the last release happened in September 2005. May be I am looking in the wrong place.

	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

http://userweb.kernel.org/~kzak/util-linux-ng/

Karel,
any chance to put the new util-linux release into the place above aswell. (And maybe strip the silly -ng postfix)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Ask Adrian Bunk who ignores all mails about util-linux in last two years. For more details see thread:

    http://lkml.org/lkml/2006/12/20/130

... it's old and stupid story. Maybe we can try to restart this discussion. I have not a clue, how deep is Adrian's hibernation...

Karel

 Karel Zak  <[email protected]>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

That's the theory. I'd be interested in seeing someone test it out: submit something and see if/when it goes in and gets redistributed.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Agreed. My thought was that PSS will be used far more widely than other maps2 memory analysis tools. Providing PSS in smaps could possibly help simplify many ps/top like tools.

Because Matt didn't put it there ;-)

I wholly agree that it is a good practice to carry some user space tools with the kernel as the reference implementations. We already have some in the Document/ and usr/ directories.

Fengguang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

It can be done via maps2, but it's considerably less efficient. So if this particular metric is a useful one, we ought to consider putting it in-kernel.

Now is this PSS metric that useful? I'd argue yes. By comparison, the RSS and VSS numbers are of basically no help in answering the most common question about memory usage: "how much memory is my process actually using?" Most people fall back to RSS and a bunch of hand-waving, but that tends to fall over because typically sum(RSS) > available RAM (sometimes by an order of magnitude), leaving users completely confused. This was perhaps the biggest complaint from one of the people on the Kernel Summit user panel, by the way. The PSS numbers always sum to used RAM and give a fairly intuitive accounting of shared mappings.

The big downside to PSS is that it's expensive to track. We have to either visit each page when we report the count or we have to update each PSS counter when we change the use count on a shared page. There might be some tricks we can pull here but RSS and VSS, on the other hand, are effectively O(1). An efficient in-kernel PSS calculator might be a little painful if used in something like top(1), but the map2 approach definitely won't be fast enough here.

Also, there's a second number we should be reporting at the same time, which I've been calling USS (unique or unshared set size), which is the size of the unshared pages. This is, for example, the amount of memory that will get freed when you kill one of 20 Apache threads, or, alternately, the amount of memory that adding another one will consume.

Mathematics is the supreme nostalgia of our time.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Hi Matt,

USS is already there, smaps already gives you that.

If you read entire smaps "file" and sum up all numbers there:

Shared_Clean: N
Shared_Dirty: N
Private_Clean: N
Private_Dirty: N

Then you can calculate the following (among other useful things):

rss_sh   - sum of (shared_clean + shared_dirty)
uss      - sum of (private_clean + private_dirty) <=== HERE
rss      - uss + rss_sh

PSS, on the other hand, cannot be inferred from this data, so please push it into mainline.
--
vda

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

This is the advantage of exmap/pagemap. You don't pay any runtime penalty.

Personally, I'd probably rather make do with RSS and VSS to see which processes are 'large'. There usually aren't many.

If there's a significant/measurable penalty to tracking PSS in kernel I'd say why bother? It's a very useful number *when you are interested in it*. On those occasions you can fire up the more complex tool and pay the time to walk the page tables. It's not a very dynamic quantity, so a snapshotting approach works well.

Also exmap (I don't know if pagemap does this) grovels through ELF and /proc/<pid>/maps so you can see which section+symbol of your shared lib is hurting you. You're generally going to want this info in order to do anything about bad PSS numbers, so I'm not sure raw PSS numbers are directly useful.

Is map2 -mm tree only (I didn't get anything on a grep of mainline 2.6.22.6)? Sorry, I'm a bit out of touch. If I could drop the kernel module from exmap and use an existing interface that would be great.

jb

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

On Wed, Sep 19, 2007 at 09:37:21AM +0100, John Berthels wrote:
[...]

Basically,
- getting the list of used/unused symbols are great for developers;
- getting the list of applications with their PSS numbers are good for users.

One is for 'analysis' and another is about 'accounting'.

Yes.
It could be the right time for an early tryout and early feedbacks ;-)

Fengguang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>