After swapout/swapin private dirty mappings become clean

/proc/$pid/smaps broken: After swapout/swapin private dirty mappings become clean.

When a page with private file mapping becomes dirty, the vma will be in both i_mmap tree and anon_vma list. The /proc/$pid/smaps will account these pages as dirty and backed by the file.

But when those dirty pages gets swapped out, and when they are read back from swap, they would be marked as clean, as it should be, as they are part of swap cache now.

But the /proc/$pid/smaps would report the vma as a mapping of a file and it is clean. The pages are actually in same state i.e., dirty with respect to file still, but which was once reported as dirty is now being reported as clean to user-space.

This confuses tools like gdb which uses this information. Those tools think that those pages were never modified and it creates problem when they create dumps.

The file mapping of the vma also cannot be broken as pages never read earlier, will still have to come from the file. Just that those dirty pages have become clean anonymous pages.

During swaping in, restoring the exact state as dirty file-backed pages before swapout would be useless, as there in no real bug. Breaking the vma with only anonymous pages as seperate vmas unnecessary may not be a good thing as well. So let us just export the information that a file-backed vma has anonymous dirty pages.

Export this information in smaps by prepending file-names with "[anon]+", when some of the pages in a file backed vma become anonymous.

Signed-off-by Nikanth Karthikesan <[email protected]>

---

fs/proc/task_mmu.c +2 -0

index 439fc1f..68f9806 100644
@@ -242,6 +242,8 @@ static void show_map_vma(struct seq_file *m, struct vm_area_struct *vma)
 	 */
 	if (file) {
 		pad_len_spaces(m, len);
+		if (vma->anon_vma)
+			seq_puts(m, "[anon]+");
 		seq_path(m, &file->f_path, "\n");
 	} else {
 		const char *name = arch_vma_name(vma);

Quoted 30 lines · Nikanth Karthikesan

/proc/$pid/smaps broken: After swapout/swapin private dirty mappings become clean.

When a page with private file mapping becomes dirty, the vma will be in both i_mmap tree and anon_vma list. The /proc/$pid/smaps will account these pages as dirty and backed by the file.

But when those dirty pages gets swapped out, and when they are read back from swap, they would be marked as clean, as it should be, as they are part of swap cache now.

But the /proc/$pid/smaps would report the vma as a mapping of a file and it is clean. The pages are actually in same state i.e., dirty with respect to file still, but which was once reported as dirty is now being reported as clean to user-space.

This confuses tools like gdb which uses this information. Those tools think that those pages were never modified and it creates problem when they create dumps.

The file mapping of the vma also cannot be broken as pages never read earlier, will still have to come from the file. Just that those dirty pages have become clean anonymous pages.

During swaping in, restoring the exact state as dirty file-backed pages before swapout would be useless, as there in no real bug. Breaking the vma with only anonymous pages as seperate vmas unnecessary may not be a good thing as well. So let us just export the information that a file-backed vma has anonymous dirty pages.

Quoted 2 lines

Export this information in smaps by prepending file-names with "[anon]+", when some of the pages in a file backed vma become anonymous.

For the sake of not breaking existing tools I'd prefer appending " [anon]" instead.

Though a much simpler thing would be to account
the clean anon pages as Private_Dirty (with respect to the backing file displayed).

Anonymous vmas in /proc/smaps seem to contain Private_Dirty pages as well. So I still don't understand why this isn't just an accounting bug.

Thanks,
Richard.

Quoted 17 lines

Signed-off-by Nikanth Karthikesan <[email protected]>

---

fs/proc/task_mmu.c +0 -0

index 439fc1f..68f9806 100644
@@ -242,6 +242,8 @@ static void show_map_vma(struct seq_file *m, struct

vm_area_struct *vma) */

fs/proc/task_mmu.c +2 -0

 	if (file) {
 		pad_len_spaces(m, len);
+		if (vma->anon_vma)
+			seq_puts(m, "[anon]+");
 		seq_path(m, &file->f_path, "\n");
 	} else {
 		const char *name = arch_vma_name(vma);

fs/proc/task_mmu.c +0 -0

index 439fc1f..06fc468 100644
@@ -368,7 +368,11 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
 				mss->shared_clean += PAGE_SIZE;
 			mss->pss += (PAGE_SIZE << PSS_SHIFT) / mapcount;

Quoted 11 lines

fs/proc/task_mmu.c +2 -1

index 439fc1f..06fc468 100644
@@ -368,7 +368,11 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long
 addr, unsigned long end, mss->shared_clean += PAGE_SIZE;
 			mss->pss += (PAGE_SIZE << PSS_SHIFT) / mapcount;
 		} else {
-			if (pte_dirty(ptent))
+			/*
+			 * File-backed pages, now anonymous are dirty

fs/proc/task_mmu.c static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, · line 378

Quoted 6 lines

fs/proc/task_mmu.c +3 -0

+			 * with respect to the file.
+			 */
+			if (pte_dirty(ptent) || (vma->vm_file && PageAnon(page)))
 				mss->private_dirty += PAGE_SIZE;
 			else
 				mss->private_clean += PAGE_SIZE;

This is risky than v1. number of dirties are used a lot of application.

Quoted 11 lines

Quoted 11 lines

fs/proc/task_mmu.c +2 -1

index 439fc1f..06fc468 100644
@@ -368,7 +368,11 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long
 addr, unsigned long end, mss->shared_clean += PAGE_SIZE;
 			mss->pss += (PAGE_SIZE << PSS_SHIFT) / mapcount;
 		} else {
-			if (pte_dirty(ptent))
+			/*
+			 * File-backed pages, now anonymous are dirty

Quoted 9 lines

fs/proc/task_mmu.c static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, · line 378

Quoted 6 lines

fs/proc/task_mmu.c +3 -0

+			 * with respect to the file.
+			 */
+			if (pte_dirty(ptent) || (vma->vm_file && PageAnon(page)))
 				mss->private_dirty += PAGE_SIZE;
 			else
 				mss->private_clean += PAGE_SIZE;

This is risky than v1. number of dirties are used a lot of application.

This is exactly to help those applications, as currently after swap-out and swap-in, the same pages are accounted as "Private_Clean:" instead of "Private_Dirty:".

Quoted 21 lines · On Wednesday 15 September 2010 05:56:36 KOSAKI Motohiro wrote:

Quoted 19 lines

Quoted 7 lines

fs/proc/task_mmu.c +0 -0

index 439fc1f..06fc468 100644
@@ -368,7 +368,11 @@ static int smaps_pte_range(pmd_t *pmd, unsigned
 long addr, unsigned long end, mss->shared_clean += PAGE_SIZE;
 			mss->pss += (PAGE_SIZE << PSS_SHIFT) / mapcount;

fs/proc/task_mmu.c static int smaps_pte_range(pmd_t *pmd, unsigned · line 378

Quoted 10 lines

fs/proc/task_mmu.c +5 -1

 		} else {
-			if (pte_dirty(ptent))
+			/*
+			 * File-backed pages, now anonymous are dirty
+			 * with respect to the file.
+			 */
+			if (pte_dirty(ptent) || (vma->vm_file && PageAnon(page)))
 				mss->private_dirty += PAGE_SIZE;
 			else
 				mss->private_clean += PAGE_SIZE;

This is risky than v1. number of dirties are used a lot of application.

Quoted 3 lines

This is exactly to help those applications, as currently after swap-out and swap-in, the same pages are accounted as "Private_Clean:" instead of "Private_Dirty:".

I don't think so.

incorrect infomation bring a lot of confusion rather than
its worth.

Quoted 25 lines

Quoted 24 lines · On Wednesday 15 September 2010 05:56:36 KOSAKI Motohiro wrote:

Quoted 11 lines

Quoted 11 lines

fs/proc/task_mmu.c +2 -1

index 439fc1f..06fc468 100644
@@ -368,7 +368,11 @@ static int smaps_pte_range(pmd_t *pmd, unsigned
 long addr, unsigned long end, mss->shared_clean += PAGE_SIZE;
 			mss->pss += (PAGE_SIZE << PSS_SHIFT) / mapcount;
 		} else {
-			if (pte_dirty(ptent))
+			/*
+			 * File-backed pages, now anonymous are dirty

Quoted 8 lines

fs/proc/task_mmu.c static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, · line 378

Quoted 6 lines

fs/proc/task_mmu.c +3 -0

+			 * with respect to the file.
+			 */
+			if (pte_dirty(ptent) || (vma->vm_file && PageAnon(page)))
 				mss->private_dirty += PAGE_SIZE;
 			else
 				mss->private_clean += PAGE_SIZE;

This is risky than v1. number of dirties are used a lot of application.

This is exactly to help those applications, as currently after swap-out and swap-in, the same pages are accounted as "Private_Clean:" instead of "Private_Dirty:".

Quoted 1 line

I don't think so.

Actually this behaviour is observed. With a simple memhog, you can see pages which are "Private_Dirty:", become "Swap:" and then to "Private_Clean:". And that confused GDB.

Quoted 2 lines

incorrect infomation bring a lot of confusion rather than
its worth.

Quoted 29 lines · On Wednesday 15 September 2010 10:18:11 KOSAKI Motohiro wrote:

Quoted 27 lines

Quoted 22 lines · On Wednesday 15 September 2010 05:56:36 KOSAKI Motohiro wrote:

Quoted 12 lines

Quoted 12 lines

fs/proc/task_mmu.c +0 -0

index 439fc1f..06fc468 100644
@@ -368,7 +368,11 @@ static int smaps_pte_range(pmd_t *pmd,

unsigned long addr, unsigned long end, mss->shared_clean += PAGE_SIZE; mss->pss += (PAGE_SIZE << PSS_SHIFT) / mapcount;

fs/proc/task_mmu.c +3 -1

 		} else {
-			if (pte_dirty(ptent))
+			/*
+			 * File-backed pages, now anonymous are dirty
+			 * with respect to the file.

Quoted 8 lines

fs/proc/task_mmu.c static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, · line 378

Quoted 5 lines

fs/proc/task_mmu.c +2 -0

+			 */
+			if (pte_dirty(ptent) || (vma->vm_file && PageAnon(page)))
 				mss->private_dirty += PAGE_SIZE;
 			else
 				mss->private_clean += PAGE_SIZE;

This is risky than v1. number of dirties are used a lot of application.

Quoted 3 lines

This is exactly to help those applications, as currently after swap-out and swap-in, the same pages are accounted as "Private_Clean:" instead of "Private_Dirty:".

I don't think so.

Quoted 3 lines

Actually this behaviour is observed. With a simple memhog, you can see pages which are "Private_Dirty:", become "Swap:" and then to "Private_Clean:". And that confused GDB.

As I said, incorrect information is always no good solustion. We should
concern how to provide good and enough information, but not how to lie.
If currect gdb is crappy, it should fix.

Quoted 2 lines

Quoted 2 lines

incorrect infomation bring a lot of confusion rather than
its worth.

How? Current smaps information without this patch provides incorrect information. Just because a private dirty page became part of swap cache, it shown as clean and backed by a file. If it is shown as clean and backed by swap then it is fine.

Quoted 5 lines · Nikanth Karthikesan

How? Current smaps information without this patch provides incorrect information. Just because a private dirty page became part of swap cache, it shown as clean and backed by a file. If it is shown as clean and backed by swap then it is fine.

How is GDB using this information?

Quoted 8 lines · Balbir Singh

Quoted 5 lines · Nikanth Karthikesan

How? Current smaps information without this patch provides incorrect information. Just because a private dirty page became part of swap cache, it shown as clean and backed by a file. If it is shown as clean and backed by swap then it is fine.

Quoted 1 line

How is GDB using this information?

GDB counts the number of dirty and swapped pages in a private mapping and based on that decides whether it needs to dump it to a core file or not. If there are no dirty or swapped pages gdb assumes it can reconstruct the mapping from the original backing file. This way for example shared libraries do not end up in the core file.

Quoted 10 lines · Richard Guenther

Quoted 8 lines · Balbir Singh

Quoted 5 lines · Nikanth Karthikesan

How? Current smaps information without this patch provides incorrect information. Just because a private dirty page became part of swap cache, it shown as clean and backed by a file. If it is shown as clean and backed by swap then it is fine.

Quoted 7 lines

Quoted 1 line

How is GDB using this information?

GDB counts the number of dirty and swapped pages in a private mapping and based on that decides whether it needs to dump it to a core file or not. If there are no dirty or swapped pages gdb assumes it can reconstruct the mapping from the original backing file. This way for example shared libraries do not end up in the core file.

This whole discussion is a little disturbing.

The page is being reported clean as per the kernel's definition of clean, full stop.

So either there's a latent bug/inconsistency in the kernel VM or external tools are misinterpreting this data. But smaps is just reporting what's there, the fault doesn't lie in smaps. So fixing smaps just hides the problem, wherever it is.

Richard's report that the page is still clean after swapoff suggests the inconsistency lies in the VM.

Quoted 24 lines · Matt Mackall

Quoted 10 lines · Richard Guenther

Quoted 8 lines · Balbir Singh

Quoted 5 lines · Nikanth Karthikesan

How? Current smaps information without this patch provides incorrect information. Just because a private dirty page became part of swap cache, it shown as clean and backed by a file. If it is shown as clean and backed by swap then it is fine.

Quoted 7 lines

Quoted 1 line

How is GDB using this information?

GDB counts the number of dirty and swapped pages in a private mapping and based on that decides whether it needs to dump it to a core file or not. If there are no dirty or swapped pages gdb assumes it can reconstruct the mapping from the original backing file. This way for example shared libraries do not end up in the core file.

This whole discussion is a little disturbing.

The page is being reported clean as per the kernel's definition of clean, full stop.

Quoted 7 lines

So either there's a latent bug/inconsistency in the kernel VM or external tools are misinterpreting this data. But smaps is just reporting what's there, the fault doesn't lie in smaps. So fixing smaps just hides the problem, wherever it is.

Richard's report that the page is still clean after swapoff suggests the inconsistency lies in the VM.

Well - the discussion is about the /proc/smaps interface and inconsistencies in what it reports. In particular the interface does not have the capability of reporting all details the kernel has, so it might make sense to not "report a page clean as per the kernel's definition of clean", but only in a /proc/smaps context definition of clean that makes sense.

So, for

7ffff81ff000-7ffff8201000 r--p 000a8000 08:01 16376 /bin/bash

Size:                  8 kB
Rss:                   8 kB
Pss:                   8 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         8 kB
Private_Dirty:         0 kB
Referenced:            4 kB
Swap:                  0 kB

I expect both pages of that mapping to be file-backed by /bin/bash. But surprisingly one page is actually backed by anonymous memory (it was changed, then mapped readonly, swapped out and swapped in again).

Thus, the bug is the above inconsistency in /proc/smaps.

[adding Hugh]

Quoted 54 lines · Richard Guenther

Quoted 24 lines · Matt Mackall

Quoted 10 lines · Richard Guenther

Quoted 8 lines · Balbir Singh

Quoted 5 lines · Nikanth Karthikesan

How? Current smaps information without this patch provides incorrect information. Just because a private dirty page became part of swap cache, it shown as clean and backed by a file. If it is shown as clean and backed by swap then it is fine.

Quoted 7 lines

Quoted 1 line

How is GDB using this information?

GDB counts the number of dirty and swapped pages in a private mapping and based on that decides whether it needs to dump it to a core file or not. If there are no dirty or swapped pages gdb assumes it can reconstruct the mapping from the original backing file. This way for example shared libraries do not end up in the core file.

This whole discussion is a little disturbing.

The page is being reported clean as per the kernel's definition of clean, full stop.

Quoted 7 lines

So either there's a latent bug/inconsistency in the kernel VM or external tools are misinterpreting this data. But smaps is just reporting what's there, the fault doesn't lie in smaps. So fixing smaps just hides the problem, wherever it is.

Richard's report that the page is still clean after swapoff suggests the inconsistency lies in the VM.

Well - the discussion is about the /proc/smaps interface and inconsistencies in what it reports. In particular the interface does not have the capability of reporting all details the kernel has, so it might make sense to not "report a page clean as per the kernel's definition of clean", but only in a /proc/smaps context definition of clean that makes sense.

So, for

7ffff81ff000-7ffff8201000 r--p 000a8000 08:01 16376 /bin/bash

Size:                  8 kB
Rss:                   8 kB
Pss:                   8 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         8 kB
Private_Dirty:         0 kB
Referenced:            4 kB
Swap:                  0 kB

Quoted 6 lines

I expect both pages of that mapping to be file-backed by /bin/bash. But surprisingly one page is actually backed by anonymous memory (it was changed, then mapped readonly, swapped out and swapped in again).

Thus, the bug is the above inconsistency in /proc/smaps.

But that's my point: the consistency problem is NOT in smaps. The page is NOT marked dirty, ergo smaps doesn't report it as dirty. Whether or not there is MORE information smaps could be reporting is irrelevant, the information it IS reporting is consistent with the underlying VM data. If there's an inconsistency about what it means to be clean, it's either in the VM or in your head.

And I frankly think it's in the VM.

In any case, I don't think Nikanth's fix is the right fix, as it basically says "you can't trust any of this". Either swap should return the pages to their pre-swap dirty state in the VM, or we should add another field here:

Weird_Anon_Page_You_Should_Pretend_Is_Private_Dirty: 8 kB

See?

But that's my point: the consistency problem is NOT in smaps. The page is NOT marked dirty, ergo smaps doesn't report it as dirty. Whether or not there is MORE information smaps could be reporting is irrelevant, the information it IS reporting is consistent with the underlying VM data. If there's an inconsistency about what it means to be clean, it's either in the VM or in your head.

And I frankly think it's in the VM.

In any case, I don't think Nikanth's fix is the right fix, as it basically says "you can't trust any of this". Either swap should return the pages to their pre-swap dirty state in the VM, or we should add another field here:

Weird_Anon_Page_You_Should_Pretend_Is_Private_Dirty: 8 kB

Quoted 7 lines · Matt Mackall

But that's my point: the consistency problem is NOT in smaps. The page is NOT marked dirty, ergo smaps doesn't report it as dirty. Whether or not there is MORE information smaps could be reporting is irrelevant, the information it IS reporting is consistent with the underlying VM data. If there's an inconsistency about what it means to be clean, it's either in the VM or in your head.

Quoted 1 line

And I frankly think it's in the VM.

I don't believe there's any problem in the VM here, we'd be having SIGSEGVs all over if there were.

The problem is that /proc/pid/smaps exports a simplified view of the VM, and Richard and Nikanth were hoping that it gave them some info which it has never pretended to give them,

It happens to use a pte_dirty(ptent) test: you could argue that that should be pte_dirty(ptent) || PageDirty(page) (which would then "fix the issue" which Richard sees with swapoff/swapon),

or you could argue
that it should be pte_dirty(ptent) || PageDirty(page) || PageSwapCache(page) (which would then note clean copies of swap cache as dirty in the sense which Richard and Nikanth are interested in).

But after these years, we should probably assume that most users of /proc/pid/smaps are used to the existing pte_dirty(ptent) test, and would be troubled by a departure from it.

Quoted 5 lines

In any case, I don't think Nikanth's fix is the right fix, as it basically says "you can't trust any of this". Either swap should return the pages to their pre-swap dirty state in the VM, or we should add another field here:

Quoted 1 line

Weird_Anon_Page_You_Should_Pretend_Is_Private_Dirty: 8 kB

I think that the most widely useful but simple extension of /proc/pid/smaps, that would give them the info they want, would indeed be to counts ptes pointing to PageAnon pages and report that total on an additional line (say, just before "Swap:"); but there's no need for the derogatory name you propose there, "Anon:" would suit fine!

Quoted 9 lines · Hugh Dickins

Quoted 7 lines · Matt Mackall

But that's my point: the consistency problem is NOT in smaps. The page is NOT marked dirty, ergo smaps doesn't report it as dirty. Whether or not there is MORE information smaps could be reporting is irrelevant, the information it IS reporting is consistent with the underlying VM data. If there's an inconsistency about what it means to be clean, it's either in the VM or in your head.

Quoted 4 lines

Quoted 1 line

And I frankly think it's in the VM.

I don't believe there's any problem in the VM here, we'd be having SIGSEGVs all over if there were.

Of course it works. It's just not as orthogonal (aka consistent) as it could be in this case: it's not actually reflecting any of the usual meanings of dirtiness here.

Quoted 7 lines

The problem is that /proc/pid/smaps exports a simplified view of the VM, and Richard and Nikanth were hoping that it gave them some info which it has never pretended to give them,

It happens to use a pte_dirty(ptent) test: you could argue that that should be pte_dirty(ptent) || PageDirty(page) (which would then "fix the issue" which Richard sees with swapoff/swapon),

That might be interesting. Are there any other notable cases where pte_dirty() differs from PageDirty()?

Quoted 5 lines

or you could argue
that it should be pte_dirty(ptent) || PageDirty(page) || PageSwapCache(page) (which would then note clean copies of swap cache as dirty in the sense which Richard and Nikanth are interested in).

Quoted 3 lines

But after these years, we should probably assume that most users of /proc/pid/smaps are used to the existing pte_dirty(ptent) test, and would be troubled by a departure from it.

Dunno, my smem tool[1] is probably also expecting too much here and I should know better!

Quoted 7 lines · Hugh Dickins

The problem is that /proc/pid/smaps exports a simplified view of the VM, and Richard and Nikanth were hoping that it gave them some info which it has never pretended to give them,

It happens to use a pte_dirty(ptent) test: you could argue that that should be pte_dirty(ptent) || PageDirty(page) (which would then "fix the issue" which Richard sees with swapoff/swapon),

That might be interesting. Are there any other notable cases where pte_dirty() differs from PageDirty()?

Export the number of anonymous pages in a mapping via smaps.

Even the private pages in a mapping backed by a file, would be marked as anonymous, when they are modified. Export this information to user-space via smaps.

Signed-off-by Nikanth Karthikesan <[email protected]>

Currently, /proc/<pid>/smaps have wrong dirty pages accounting. Shared_Dirty and Private_Dirty output only pte dirty pages and ignore PG_dirty page flag. It is difference against documentation, but also inconsistent against Referenced field. (Referenced checks both pte and page flags)

This patch fixes it.

Test program:

large-array.c
---------------------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

char array[1*1024*1024*1024L];

int main(void)

 {
         memset(array, 1, sizeof(array));
         pause();

         return 0;

}
---------------------------------------------------

Test case:
1. run ./large-array
2. cat /proc/pidof large-array/smaps
3. swapoff -a
4. cat /proc/pidof large-array/smaps again

Test result:
<before patch>

00601000-40601000 rw-p 00000000 00:00 0

Size:            1048576 kB
Rss:             1048576 kB
Pss:             1048576 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:    218992 kB   <-- showed pages as clean incorrectly
Private_Dirty:    829584 kB
Referenced:       388364 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB

<after patch>

00601000-40601000 rw-p 00000000 00:00 0

Size:            1048576 kB
Rss:             1048576 kB
Pss:             1048576 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:   1048576 kB  <-- fixed
Referenced:       388480 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB

Signed-off-by KOSAKI Motohiro <[email protected]>

---
 fs/proc/task_mmu.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

fs/proc/task_mmu.c +2 -2

index 439fc1f..7415f13 100644
@@ -362,13 +362,13 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
 			mss->referenced += PAGE_SIZE;
 		mapcount = page_mapcount(page);
 		if (mapcount >= 2) {
-			if (pte_dirty(ptent))
+			if (pte_dirty(ptent) || PageDirty(page))
 				mss->shared_dirty += PAGE_SIZE;
 			else
 				mss->shared_clean += PAGE_SIZE;
 			mss->pss += (PAGE_SIZE << PSS_SHIFT) / mapcount;
 		} else {
-			if (pte_dirty(ptent))
+			if (pte_dirty(ptent) || PageDirty(page))
 				mss->private_dirty += PAGE_SIZE;
 			else
 				mss->private_clean += PAGE_SIZE;

Signature · 1 line

1.6.5.2

Sorry, I missed to add documentation, when I sent the patch. This depends on the patch titled, "[PATCH] Export amount of anonymous memory in a mapping via smaps".

Thanks
Nikanth

Document the new Anonymous field in smaps.

Signed-off-by Nikanth Karthikesan <[email protected]>

---

Documentation/filesystems/proc.txt +1 -0

index a6aca87..75c7368 100644
@@ -370,6 +370,7 @@ Shared_Dirty:          0 kB
 Private_Clean:         0 kB
 Private_Dirty:         0 kB
 Referenced:          892 kB
+Anonymous:             0 kB
 Swap:                  0 kB
 KernelPageSize:        4 kB
 MMUPageSize:           4 kB

Documentation/filesystems/proc.txt +0 -0

 This file is only present if the CONFIG_MMU kernel configuration option is
 enabled.

Document the new Anonymous field in smaps.

Quoted 1 line · Nikanth Karthikesan

Document the new Anonymous field in smaps.

Thanks for doing that, good effort, but your shifts between singular and plural rather jarred on my ear, so I've rewritten it a little below. Also added a sentence on "Swap"; but gave up when it came to KernelPageSize and MMUPageSize, let someone else clarify those later.

[PATCH v3] Document the new Anonymous field in smaps.

From: Nikanth Karthikesan <[email protected]>

Document the new Anonymous field in smaps, and also the Swap field. Explain what smaps means by shared and private, which differs from MAP_SHARED and MAP_PRIVATE.

Signed-off-by Nikanth Karthikesan <[email protected]>

Signed-off-by Hugh Dickins <[email protected]>

---

 Documentation/filesystems/proc.txt |   15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

Documentation/filesystems/proc.txt +11 -4

@@ -370,17 +370,24 @@ Shared_Dirty:          0 kB
 Private_Clean:         0 kB
 Private_Dirty:         0 kB
 Referenced:          892 kB
+Anonymous:             0 kB
 Swap:                  0 kB
 KernelPageSize:        4 kB
 MMUPageSize:           4 kB
 
-The first  of these lines shows  the same information  as is displayed for the
-mapping in /proc/PID/maps.  The remaining lines show  the size of the mapping,
+The first of these lines shows the same information as is displayed for the
+mapping in /proc/PID/maps. The remaining lines show the size of the mapping,
 the amount of the mapping that is currently resident in RAM, the "proportional
 set sizea?? (divide each shared page by the number of processes sharing it), the
 number of clean and dirty shared pages in the mapping, and the number of clean
-and dirty private pages in the mapping.  The "Referenced" indicates the amount
-of memory currently marked as referenced or accessed.
+and dirty private pages in the mapping. Note that even a page which is part of
+a MAP_SHARED mapping, but has only a single pte mapped, i.e. is currently used
+by only one process, is accounted as private and not as shared. "Referenced"
+indicates the amount of memory currently marked as referenced or accessed.
+"Anonymous" shows the amount of memory that does not belong to any file. Even
+a mapping associated with a file may contain anonymous pages: when MAP_PRIVATE
+and a page is modified, the file page is replaced by a private anonymous copy.
+"Swap" shows how much would-be-anonymous memory is also used, but out on swap.
 
 This file is only present if the CONFIG_MMU kernel configuration option is
 enabled.

Export the number of anonymous pages in a mapping via smaps.

Even the private pages in a mapping backed by a file, would be marked as anonymous, when they are modified. Export this information to user-space via smaps.

Signed-off-by Nikanth Karthikesan <[email protected]>

---

fs/proc/task_mmu.c +6 -1

index 439fc1f..3c18fc8 100644
@@ -326,6 +326,7 @@ struct mem_size_stats {
 	unsigned long private_clean;
 	unsigned long private_dirty;
 	unsigned long referenced;
+	unsigned long anonymous;
 	unsigned long swap;
 	u64 pss;
 };

@@ -356,6 +357,9 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
 		if (!page)
 			continue;
 
+		if (PageAnon(page))
+			mss->anonymous += PAGE_SIZE;
+
 		mss->resident += PAGE_SIZE;
 		/* Accumulate the size in pages that have been accessed. */
 		if (pte_young(ptent) || PageReferenced(page))

@@ -409,6 +413,7 @@ static int show_smap(struct seq_file *m, void *v)
 		   "Private_Clean:  %8lu kB\n"
 		   "Private_Dirty:  %8lu kB\n"
 		   "Referenced:     %8lu kB\n"
+		   "Anonymous:      %8lu kB\n"
 		   "Swap:           %8lu kB\n"
 		   "KernelPageSize: %8lu kB\n"
 		   "MMUPageSize:    %8lu kB\n",

@@ -420,6 +425,7 @@ static int show_smap(struct seq_file *m, void *v)
 		   mss.private_clean >> 10,
 		   mss.private_dirty >> 10,
 		   mss.referenced >> 10,
+		   mss.anonymous >> 10,
 		   mss.swap >> 10,
 		   vma_kernel_pagesize(vma) >> 10,
 		   vma_mmu_pagesize(vma) >> 10);
 
--

To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 8 lines · Balbir Singh

Quoted 5 lines · Nikanth Karthikesan

How? Current smaps information without this patch provides incorrect information. Just because a private dirty page became part of swap cache, it shown as clean and backed by a file. If it is shown as clean and backed by swap then it is fine.

Quoted 1 line

How is GDB using this information?

GDB counts the number of dirty and swapped pages in a private mapping and based on that decides whether it needs to dump it to a core file or not. If there are no dirty or swapped pages gdb assumes it can reconstruct the mapping from the original backing file. This way for example shared libraries do not end up in the core file.

Quoted 6 lines · Balbir Singh

Quoted 4 lines · Nikanth Karthikesan

How? Current smaps information without this patch provides incorrect information. Just because a private dirty page became part of swap cache, it shown as clean and backed by a file. If it is shown as clean and backed by swap then it is fine.

Quoted 1 line

How is GDB using this information?

GDB counts the number of dirty and swapped pages in a private mapping and based on that decides whether it needs to dump it to a core file or not. If there are no dirty or swapped pages gdb assumes it can reconstruct the mapping from the original backing file. This way for example shared libraries do not end up in the core file.

Document /proc/pid/pagemap in Documentation/filesystems/proc.txt

Signed-off-by Nikanth Karthikesan <[email protected]>

Quoted 7 lines · On Wednesday 15 September 2010 19:44:17 Richard Guenther wrote:

Quoted 6 lines · Balbir Singh

Quoted 4 lines · Nikanth Karthikesan

How? Current smaps information without this patch provides incorrect information. Just because a private dirty page became part of swap cache, it shown as clean and backed by a file. If it is shown as clean and backed by swap then it is fine.

Quoted 7 lines

Quoted 1 line

How is GDB using this information?

GDB counts the number of dirty and swapped pages in a private mapping and based on that decides whether it needs to dump it to a core file or not. If there are no dirty or swapped pages gdb assumes it can reconstruct the mapping from the original backing file. This way for example shared libraries do not end up in the core file.

Well, may be /proc/pid/pagemap + /proc/kpageflags is enough for this! One
can get the pageflags using these interfaces. See Documentation/vm/pagemap.txt for the explanation on how to do it. There is also a sample program that prints page flags using this interface in Documentation/vm/page-types.c.

It is bad that /proc/pid/pagemap is never mentioned in Documentation/filesystems/proc.txt. I will send a patch to rectify this.

Quoted 7 lines · On Wednesday 15 September 2010 19:44:17 Richard Guenther wrote:

Quoted 6 lines · Balbir Singh

Quoted 4 lines · Nikanth Karthikesan

How? Current smaps information without this patch provides incorrect information. Just because a private dirty page became part of swap cache, it shown as clean and backed by a file. If it is shown as clean and backed by swap then it is fine.

Quoted 8 lines

Quoted 1 line

How is GDB using this information?

GDB counts the number of dirty and swapped pages in a private mapping and based on that decides whether it needs to dump it to a core file or not. If there are no dirty or swapped pages gdb assumes it can reconstruct the mapping from the original backing file. This way for example shared libraries do not end up in the core file.

Well, may be /proc/pid/pagemap + /proc/kpageflags is enough for this! One can get the pageflags using these interfaces. See Documentation/vm/pagemap.txt for the explanation on how to do it. There is also a sample program that prints page flags using this interface in Documentation/vm/page-types.c.

It is bad that /proc/pid/pagemap is never mentioned in Documentation/filesystems/proc.txt. I will send a patch to rectify this.

/proc/$pid/smaps broken: After swapout/swapin private dirty mappings become clean.

When a page with private file mapping becomes dirty, the vma will be in both i_mmap tree and anon_vma list. The /proc/$pid/smaps will account these pages as dirty and backed by the file.

But when those dirty pages gets swapped out, and when they are read back from swap, they would be marked as clean, as it should be, as they are part of swap cache now.

But the /proc/$pid/smaps would report the vma as a mapping of a file and it is clean. The pages are actually in same state i.e., dirty with respect to file still, but which was once reported as dirty is now being reported as clean to user-space.

This confuses tools like gdb which uses this information. Those tools think that those pages were never modified and it creates problem when they create dumps.

The file mapping of the vma also cannot be broken as pages never read earlier, will still have to come from the file. Just that those dirty pages have become clean anonymous pages.

During swaping in, restoring the exact state as dirty file-backed pages before swapout would be useless, as there in no real bug. Breaking the vma with only anonymous pages as seperate vmas unnecessary may not be a good thing as well. So let us just export the information that a file-backed vma has anonymous dirty pages.

Export this information in smaps by prepending file-names with "[anon]+", when some of the pages in a file backed vma become anonymous.

Signed-off-by Nikanth Karthikesan <[email protected]>

---

fs/proc/task_mmu.c +2 -1

index 439fc1f..68f9806 100644
@@ -242,6 +242,8 @@ static void show_map_vma(struct seq_file *m, struct vm_area_struct *vma)
 	 */
 	if (file) {
 		pad_len_spaces(m, len);
+		if (vma->anon_vma)
+			seq_puts(m, "[anon]+");
 		seq_path(m, &file->f_path, "\n");
 	} else {
 		const char *name = arch_vma_name(vma);
 
 
--

To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 24 lines

/proc/$pid/smaps broken: After swapout/swapin private dirty mappings become clean.

When a page with private file mapping becomes dirty, the vma will be in both i_mmap tree and anon_vma list. The /proc/$pid/smaps will account these pages as dirty and backed by the file.

But when those dirty pages gets swapped out, and when they are read back from swap, they would be marked as clean, as it should be, as they are part of swap cache now.

But the /proc/$pid/smaps would report the vma as a mapping of a file and it is clean. The pages are actually in same state i.e., dirty with respect to file still, but which was once reported as dirty is now being reported as clean to user-space.

This confuses tools like gdb which uses this information. Those tools think that those pages were never modified and it creates problem when they create dumps.

The file mapping of the vma also cannot be broken as pages never read earlier, will still have to come from the file. Just that those dirty pages have become clean anonymous pages.

Quoted 5 lines

During swaping in, restoring the exact state as dirty file-backed pages before swapout would be useless, as there in no real bug. Breaking the vma with only anonymous pages as seperate vmas unnecessary may not be a good thing as well. So let us just export the information that a file-backed vma has anonymous dirty pages.

Why can't gdb check Swap: field in smaps? I think Swap!=0 mean we need dump
out.

Am I missing anything?

- kosaki

Quoted 22 lines

Export this information in smaps by prepending file-names with "[anon]+", when some of the pages in a file backed vma become anonymous.

Signed-off-by Nikanth Karthikesan <[email protected]>

---

fs/proc/task_mmu.c +0 -0

index 439fc1f..68f9806 100644
@@ -242,6 +242,8 @@ static void show_map_vma(struct seq_file *m, struct

vm_area_struct *vma) */

fs/proc/task_mmu.c +2 -0

 	if (file) {
 		pad_len_spaces(m, len);
+		if (vma->anon_vma)
+			seq_puts(m, "[anon]+");
 		seq_path(m, &file->f_path, "\n");
 	} else {
 		const char *name = arch_vma_name(vma);

Quoted 5 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 24 lines

Quoted 24 lines

/proc/$pid/smaps broken: After swapout/swapin private dirty mappings become clean.

When a page with private file mapping becomes dirty, the vma will be in both i_mmap tree and anon_vma list. The /proc/$pid/smaps will account these pages as dirty and backed by the file.

But when those dirty pages gets swapped out, and when they are read back from swap, they would be marked as clean, as it should be, as they are part of swap cache now.

But the /proc/$pid/smaps would report the vma as a mapping of a file and it is clean. The pages are actually in same state i.e., dirty with respect to file still, but which was once reported as dirty is now being reported as clean to user-space.

This confuses tools like gdb which uses this information. Those tools think that those pages were never modified and it creates problem when they create dumps.

The file mapping of the vma also cannot be broken as pages never read earlier, will still have to come from the file. Just that those dirty pages have become clean anonymous pages.

Quoted 9 lines

Quoted 5 lines

During swaping in, restoring the exact state as dirty file-backed pages before swapout would be useless, as there in no real bug. Breaking the vma with only anonymous pages as seperate vmas unnecessary may not be a good thing as well. So let us just export the information that a file-backed vma has anonymous dirty pages.

Why can't gdb check Swap: field in smaps? I think Swap!=0 mean we need dump
out.

Yes. When the page is swapped out it is accounted in "Swap:".

Quoted 2 lines

Am I missing anything?

But when it gets swapped in back to memory, it is removed from "Swap:" and added to "Private_Clean:" instead of "Private_Dirty:".

Quoted 34 lines · On Wednesday 15 September 2010 05:54:31 KOSAKI Motohiro wrote:

Quoted 24 lines

Quoted 24 lines

/proc/$pid/smaps broken: After swapout/swapin private dirty mappings become clean.

When a page with private file mapping becomes dirty, the vma will be in both i_mmap tree and anon_vma list. The /proc/$pid/smaps will account these pages as dirty and backed by the file.

But when those dirty pages gets swapped out, and when they are read back from swap, they would be marked as clean, as it should be, as they are part of swap cache now.

But the /proc/$pid/smaps would report the vma as a mapping of a file and it is clean. The pages are actually in same state i.e., dirty with respect to file still, but which was once reported as dirty is now being reported as clean to user-space.

This confuses tools like gdb which uses this information. Those tools think that those pages were never modified and it creates problem when they create dumps.

The file mapping of the vma also cannot be broken as pages never read earlier, will still have to come from the file. Just that those dirty pages have become clean anonymous pages.

Quoted 8 lines

Quoted 5 lines

During swaping in, restoring the exact state as dirty file-backed pages before swapout would be useless, as there in no real bug. Breaking the vma with only anonymous pages as seperate vmas unnecessary may not be a good thing as well. So let us just export the information that a file-backed vma has anonymous dirty pages.

Why can't gdb check Swap: field in smaps? I think Swap!=0 mean we need dump out.

Quoted 6 lines

Yes. When the page is swapped out it is accounted in "Swap:".

Quoted 1 line

Am I missing anything?

But when it gets swapped in back to memory, it is removed from "Swap:" and added to "Private_Clean:" instead of "Private_Dirty:".

Here is the code.
I think the page will become dirty, again.

--------------------------------------------------------------
int try_to_free_swap(struct page *page)

{
        VM_BUG_ON(!PageLocked(page));

        if (!PageSwapCache(page))
                return 0;
        if (PageWriteback(page))
                return 0;
        if (page_swapcount(page))
                return 0;

        delete_from_swap_cache(page);
        SetPageDirty(page);
        return 1;
}

Quoted 56 lines

Quoted 34 lines · On Wednesday 15 September 2010 05:54:31 KOSAKI Motohiro wrote:

Quoted 24 lines

Quoted 24 lines

/proc/$pid/smaps broken: After swapout/swapin private dirty mappings become clean.

When a page with private file mapping becomes dirty, the vma will be in both i_mmap tree and anon_vma list. The /proc/$pid/smaps will account these pages as dirty and backed by the file.

But when those dirty pages gets swapped out, and when they are read back from swap, they would be marked as clean, as it should be, as they are part of swap cache now.

But the /proc/$pid/smaps would report the vma as a mapping of a file and it is clean. The pages are actually in same state i.e., dirty with respect to file still, but which was once reported as dirty is now being reported as clean to user-space.

This confuses tools like gdb which uses this information. Those tools think that those pages were never modified and it creates problem when they create dumps.

The file mapping of the vma also cannot be broken as pages never read earlier, will still have to come from the file. Just that those dirty pages have become clean anonymous pages.

Quoted 8 lines

Quoted 5 lines

During swaping in, restoring the exact state as dirty file-backed pages before swapout would be useless, as there in no real bug. Breaking the vma with only anonymous pages as seperate vmas unnecessary may not be a good thing as well. So let us just export the information that a file-backed vma has anonymous dirty pages.

Why can't gdb check Swap: field in smaps? I think Swap!=0 mean we need dump out.

Quoted 6 lines

Yes. When the page is swapped out it is accounted in "Swap:".

Quoted 1 line

Am I missing anything?

But when it gets swapped in back to memory, it is removed from "Swap:" and added to "Private_Clean:" instead of "Private_Dirty:".

Here is the code.
I think the page will become dirty, again.

--------------------------------------------------------------
int try_to_free_swap(struct page *page)

{
        VM_BUG_ON(!PageLocked(page));

        if (!PageSwapCache(page))
                return 0;
        if (PageWriteback(page))
                return 0;
        if (page_swapcount(page))
                return 0;

Quoted 5 lines

        delete_from_swap_cache(page);
        SetPageDirty(page);
        return 1;
}

I think this gets called only when the swap space gets freed.

But when the
page is just swapped out and swapped in, and the page is still part of SwapCache, it will be marked as clean, when the I/O read from swap completes.

Quoted 1 line

Quoted 1 line

But when it gets swapped in back to memory, it is removed from "Swap:"

Quoted 58 lines · On Wednesday 15 September 2010 10:16:36 KOSAKI Motohiro wrote:

Quoted 57 lines

Quoted 35 lines · On Wednesday 15 September 2010 05:54:31 KOSAKI Motohiro wrote:

Quoted 31 lines

Quoted 24 lines

/proc/$pid/smaps broken: After swapout/swapin private dirty mappings become clean.

When a page with private file mapping becomes dirty, the vma will be in both i_mmap tree and anon_vma list. The /proc/$pid/smaps will account these pages as dirty and backed by the file.

But when those dirty pages gets swapped out, and when they are read back from swap, they would be marked as clean, as it should be, as they are part of swap cache now.

But the /proc/$pid/smaps would report the vma as a mapping of a file and it is clean. The pages are actually in same state i.e., dirty with respect to file still, but which was once reported as dirty is now being reported as clean to user-space.

This confuses tools like gdb which uses this information. Those tools think that those pages were never modified and it creates problem when they create dumps.

The file mapping of the vma also cannot be broken as pages never read earlier, will still have to come from the file. Just that those dirty pages have become clean anonymous pages.

Quoted 6 lines

During swaping in, restoring the exact state as dirty file-backed pages before swapout would be useless, as there in no real bug. Breaking the vma with only anonymous pages as seperate vmas unnecessary may not be a good thing as well. So let us just export the information that a file-backed vma has anonymous dirty pages.

Quoted 2 lines

Why can't gdb check Swap: field in smaps? I think Swap!=0 mean we need dump out.

Quoted 6 lines

Yes. When the page is swapped out it is accounted in "Swap:".

Quoted 1 line

Am I missing anything?

But when it gets swapped in back to memory, it is removed from "Swap:" and added to "Private_Clean:" instead of "Private_Dirty:".

Here is the code.
I think the page will become dirty, again.

--------------------------------------------------------------
int try_to_free_swap(struct page *page)

{
        VM_BUG_ON(!PageLocked(page));

        if (!PageSwapCache(page))
                return 0;
        if (PageWriteback(page))
                return 0;
        if (page_swapcount(page))
                return 0;

Quoted 6 lines

Quoted 4 lines

        delete_from_swap_cache(page);
        SetPageDirty(page);
        return 1;
}

I think this gets called only when the swap space gets freed.

this is try-to-free-swap-space.
delete_from_swap_cache() does actual free.

Quoted 4 lines

But when the
page is just swapped out and swapped in, and the page is still part of SwapCache, it will be marked as clean, when the I/O read from swap completes.

Because in this case, the swap entry is not freed yet. Then the page is
still clean and swap field is still !0.

PageSwapCache == the page has backend swap entry == the page may be clean. But, When the swap entry is removed, page will become dirty again.

As I said, following is incorrect.

In almost case, swap entry is not
removed at swap-in. Please grep try_to_free_swap() callers and

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

But when it gets swapped in back to memory, it is removed from "Swap:"

Quoted 46 lines · On Wednesday 15 September 2010 10:16:36 KOSAKI Motohiro wrote:

Quoted 34 lines · On Wednesday 15 September 2010 05:54:31 KOSAKI Motohiro wrote:

Quoted 24 lines

Quoted 24 lines

/proc/$pid/smaps broken: After swapout/swapin private dirty mappings become clean.

When a page with private file mapping becomes dirty, the vma will be in both i_mmap tree and anon_vma list. The /proc/$pid/smaps will account these pages as dirty and backed by the file.

But when those dirty pages gets swapped out, and when they are read back from swap, they would be marked as clean, as it should be, as they are part of swap cache now.

But the /proc/$pid/smaps would report the vma as a mapping of a file and it is clean. The pages are actually in same state i.e., dirty with respect to file still, but which was once reported as dirty is now being reported as clean to user-space.

This confuses tools like gdb which uses this information. Those tools think that those pages were never modified and it creates problem when they create dumps.

The file mapping of the vma also cannot be broken as pages never read earlier, will still have to come from the file. Just that those dirty pages have become clean anonymous pages.

Quoted 8 lines

Quoted 5 lines

During swaping in, restoring the exact state as dirty file-backed pages before swapout would be useless, as there in no real bug. Breaking the vma with only anonymous pages as seperate vmas unnecessary may not be a good thing as well. So let us just export the information that a file-backed vma has anonymous dirty pages.

Why can't gdb check Swap: field in smaps? I think Swap!=0 mean we need dump out.

Quoted 6 lines

Yes. When the page is swapped out it is accounted in "Swap:".

Quoted 1 line

Am I missing anything?

But when it gets swapped in back to memory, it is removed from "Swap:" and added to "Private_Clean:" instead of "Private_Dirty:".

Here is the code.
I think the page will become dirty, again.

--------------------------------------------------------------
int try_to_free_swap(struct page *page)

Quoted 15 lines

{
        VM_BUG_ON(!PageLocked(page));

        if (!PageSwapCache(page))
                return 0;
        if (PageWriteback(page))
                return 0;
        if (page_swapcount(page))
                return 0;

        delete_from_swap_cache(page);
        SetPageDirty(page);
        return 1;
}

I think this gets called only when the swap space gets freed. But when the page is just swapped out and swapped in, and the page is still part of SwapCache, it will be marked as clean, when the I/O read from swap completes.

Quoted 64 lines · On Wednesday 15 September 2010 10:16:36 KOSAKI Motohiro wrote:

Quoted 61 lines

Quoted 34 lines · On Wednesday 15 September 2010 05:54:31 KOSAKI Motohiro wrote:

Quoted 24 lines

Quoted 24 lines

/proc/$pid/smaps broken: After swapout/swapin private dirty mappings become clean.

When a page with private file mapping becomes dirty, the vma will be in both i_mmap tree and anon_vma list. The /proc/$pid/smaps will account these pages as dirty and backed by the file.

But when those dirty pages gets swapped out, and when they are read back from swap, they would be marked as clean, as it should be, as they are part of swap cache now.

But the /proc/$pid/smaps would report the vma as a mapping of a file and it is clean. The pages are actually in same state i.e., dirty with respect to file still, but which was once reported as dirty is now being reported as clean to user-space.

This confuses tools like gdb which uses this information. Those tools think that those pages were never modified and it creates problem when they create dumps.

The file mapping of the vma also cannot be broken as pages never read earlier, will still have to come from the file. Just that those dirty pages have become clean anonymous pages.

Quoted 8 lines

Quoted 5 lines

During swaping in, restoring the exact state as dirty file-backed pages before swapout would be useless, as there in no real bug. Breaking the vma with only anonymous pages as seperate vmas unnecessary may not be a good thing as well. So let us just export the information that a file-backed vma has anonymous dirty pages.

Why can't gdb check Swap: field in smaps? I think Swap!=0 mean we need dump out.

Quoted 6 lines

Yes. When the page is swapped out it is accounted in "Swap:".

Quoted 1 line

Am I missing anything?

But when it gets swapped in back to memory, it is removed from "Swap:" and added to "Private_Clean:" instead of "Private_Dirty:".

Here is the code.
I think the page will become dirty, again.

--------------------------------------------------------------
int try_to_free_swap(struct page *page)

{
        VM_BUG_ON(!PageLocked(page));

        if (!PageSwapCache(page))
                return 0;
        if (PageWriteback(page))
                return 0;
        if (page_swapcount(page))
                return 0;

        delete_from_swap_cache(page);
        SetPageDirty(page);
        return 1;
}

Quoted 3 lines

I think this gets called only when the swap space gets freed. But when the page is just swapped out and swapped in, and the page is still part of SwapCache, it will be marked as clean, when the I/O read from swap completes.

And it will be still marked clean if I do a swapoff -a after it has been swapped in again. Thus /proc/smaps shows it as file-backed, private, clean and not swapped. Which is wrong.