Skip to contentSkip to conversation

mm: more likely reclaim MADV_SEQUENTIAL mappings

RFC In review Johannes Weiner 5 participants 18 messages Oct 21, 2008 – Oct 24, 2008

Subsystems (3)
Memory management MM core MM rmap
lore
Share & tools
lore

Johannes Weiner

author
#0 lore raw

RFC 1/1 mm: more likely reclaim MADV_SEQUENTIAL mappings

File pages mapped only in sequentially read mappings are perfect reclaim canditates.

This makes MADV_SEQUENTIAL mappings behave like weak references, its pages will be reclaimed unless they have a strong reference from a normal mapping as well.

The patch changes the reclaim and the unmap path where they check if the page has been referenced. In both cases, accesses through sequentially read mappings will be ignored.

Signed-off-by Johannes Weiner <[email protected]>

---

I'm afraid this is now quite a bit more aggressive than the earlier version. When the fault path did a mark_page_access(), we wouldn't reclaim a page when it has been faulted into several MADV_SEQUENTIAL mappings but now we ignore *every* activity through such a mapping.

What do you think?

Perhaps we should note a reference if there are two or more accesses through sequentially read mappings?

 mm/memory.c |    3 ++-
 mm/rmap.c   |   13 +++++++++++--
 2 files changed, 13 insertions(+), 3 deletions(-)
mm/rmap.c +11 -2
@@ -337,8 +337,17 @@ static int page_referenced_one(struct pa
 		goto out_unmap;
 	}
 
-	if (ptep_clear_flush_young_notify(vma, address, pte))
-		referenced++;
+	if (ptep_clear_flush_young_notify(vma, address, pte)) {
+		/*
+		 * Don't treat a reference through a sequentially read
+		 * mapping as such.  If the page has been used in
+		 * another mapping, we will catch it; if this other
+		 * mapping is already gone, the unmap path will have
+		 * set PG_referenced or activated the page.
+		 */
+		if (!VM_SequentialReadHint(vma))
+			referenced++;
+	}
 
 	/* Pretend the page is referenced if the task has the
 	   swap token and is in the middle of a page fault. */
mm/memory.c +2 -1
@@ -759,7 +759,8 @@ static unsigned long zap_pte_range(struc
 			else {
 				if (pte_dirty(ptent))
 					set_page_dirty(page);
-				if (pte_young(ptent))
+				if (pte_young(ptent) &&
+						!VM_SequentialReadHint(vma))
 					mark_page_accessed(page);
 				file_rss--;
 			}

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#1 lore raw
Quoted 12 lines · Johannes Weiner

File pages mapped only in sequentially read mappings are perfect reclaim canditates.

This makes MADV_SEQUENTIAL mappings behave like weak references, its pages will be reclaimed unless they have a strong reference from a normal mapping as well.

The patch changes the reclaim and the unmap path where they check if the page has been referenced. In both cases, accesses through sequentially read mappings will be ignored.

Signed-off-by Johannes Weiner <[email protected]>
Quoted 8 lines

---

I'm afraid this is now quite a bit more aggressive than the earlier version. When the fault path did a mark_page_access(), we wouldn't reclaim a page when it has been faulted into several MADV_SEQUENTIAL mappings but now we ignore *every* activity through such a mapping.

What do you think?

I think it's OK. MADV_SEQUENTIAL man page explicitly states they can soon be freed, and we won't DoS anybody else's working set because we are only ignoring referenced from MADV_SEQUENTIAL ptes.

It's annoying to put in extra banches especially in the unmap path. Oh well... (at least if you can mark them as likely()).

Otherwise, this patch looks much better to me now.

Thanks,
Nick

Quoted 33 lines

Perhaps we should note a reference if there are two or more accesses through sequentially read mappings?

 mm/memory.c |    3 ++-
 mm/rmap.c   |   13 +++++++++++--
 2 files changed, 13 insertions(+), 3 deletions(-)
mm/rmap.c +11 -2
@@ -337,8 +337,17 @@ static int page_referenced_one(struct pa
 		goto out_unmap;
 	}
 
-	if (ptep_clear_flush_young_notify(vma, address, pte))
-		referenced++;
+	if (ptep_clear_flush_young_notify(vma, address, pte)) {
+		/*
+		 * Don't treat a reference through a sequentially read
+		 * mapping as such.  If the page has been used in
+		 * another mapping, we will catch it; if this other
+		 * mapping is already gone, the unmap path will have
+		 * set PG_referenced or activated the page.
+		 */
+		if (!VM_SequentialReadHint(vma))
+			referenced++;
+	}
 
 	/* Pretend the page is referenced if the task has the
 	   swap token and is in the middle of a page fault. */

mm/memory.c static unsigned long zap_pte_range(struc · line 766

Quoted 9 lines
mm/memory.c +2 -1
@@ -759,7 +759,8 @@ static unsigned long zap_pte_range(struc
 			else {
 				if (pte_dirty(ptent))
 					set_page_dirty(page);
-				if (pte_young(ptent))
+				if (pte_young(ptent) &&
+						!VM_SequentialReadHint(vma))
 					mark_page_accessed(page);
 				file_rss--;
 			}

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#2 lore raw

PATCH 1/1 mm: more likely reclaim MADV_SEQUENTIAL mappings II

Quoted 5 lines · Nick Piggin
Quoted 5 lines

I'm afraid this is now quite a bit more aggressive than the earlier version. When the fault path did a mark_page_access(), we wouldn't reclaim a page when it has been faulted into several MADV_SEQUENTIAL mappings but now we ignore *every* activity through such a mapping.

Quoted 8 lines
Quoted 1 line

What do you think?

I think it's OK. MADV_SEQUENTIAL man page explicitly states they can soon be freed, and we won't DoS anybody else's working set because we are only ignoring referenced from MADV_SEQUENTIAL ptes.

It's annoying to put in extra banches especially in the unmap path. Oh well... (at least if you can mark them as likely()).

Okay, added those. Second round:

---
File pages mapped only in sequentially read mappings are perfect reclaim canditates.

This makes MADV_SEQUENTIAL mappings behave like a weak references, their pages will be reclaimed unless they have a strong reference from a normal mapping as well.

The patch changes the reclaim and the unmap path where they check if the page has been referenced. In both cases, accesses through sequentially read mappings will be ignored.

Signed-off-by Johannes Weiner <[email protected]>

---
II: add likely()s to mitigate the extra branches a bit as to Nick's
suggestion

 mm/memory.c |    3 ++-
 mm/rmap.c   |   13 +++++++++++--
 2 files changed, 13 insertions(+), 3 deletions(-)
mm/rmap.c +11 -2
@@ -337,8 +337,17 @@ static int page_referenced_one(struct pa
 		goto out_unmap;
 	}
 
-	if (ptep_clear_flush_young_notify(vma, address, pte))
-		referenced++;
+	if (ptep_clear_flush_young_notify(vma, address, pte)) {
+		/*
+		 * Don't treat a reference through a sequentially read
+		 * mapping as such.  If the page has been used in
+		 * another mapping, we will catch it; if this other
+		 * mapping is already gone, the unmap path will have
+		 * set PG_referenced or activated the page.
+		 */
+		if (likely(!VM_SequentialReadHint(vma)))
+			referenced++;
+	}
 
 	/* Pretend the page is referenced if the task has the
 	   swap token and is in the middle of a page fault. */
mm/memory.c +2 -1
@@ -759,7 +759,8 @@ static unsigned long zap_pte_range(struc
 			else {
 				if (pte_dirty(ptent))
 					set_page_dirty(page);
-				if (pte_young(ptent))
+				if (pte_young(ptent) &&
+				    likely(!VM_SequentialReadHint(vma)))
 					mark_page_accessed(page);
 				file_rss--;
 			}

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

maintainer
#3 lore raw

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

Quoted 7 lines · On Tue, 21 Oct 2008 13:33:45 +0200 Johannes Weiner <[email protected]> wrote:

File pages mapped only in sequentially read mappings are perfect reclaim canditates.

This makes MADV_SEQUENTIAL mappings behave like a weak references, their pages will be reclaimed unless they have a strong reference from a normal mapping as well.

Quoted 8 lines

The patch changes the reclaim and the unmap path where they check if the page has been referenced. In both cases, accesses through sequentially read mappings will be ignored.

Signed-off-by Johannes Weiner <[email protected]>

---
II: add likely()s to mitigate the extra branches a bit as to Nick's
suggestion

Is http://hannes.saeurebad.de/madvseq/ still true with this version?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#4 lore raw

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

Quoted 14 lines · Andrew Morton
Quoted 11 lines · On Tue, 21 Oct 2008 13:33:45 +0200 Johannes Weiner <[email protected]> wrote:

File pages mapped only in sequentially read mappings are perfect reclaim canditates.

This makes MADV_SEQUENTIAL mappings behave like a weak references, their pages will be reclaimed unless they have a strong reference from a normal mapping as well.

The patch changes the reclaim and the unmap path where they check if the page has been referenced. In both cases, accesses through sequentially read mappings will be ignored.

Quoted 6 lines
Quoted 4 lines
Signed-off-by Johannes Weiner <[email protected]>

---
II: add likely()s to mitigate the extra branches a bit as to Nick's
suggestion

Is http://hannes.saeurebad.de/madvseq/ still true with this version?

No, sorry, still running benchmarks on this version. Coming up soon...

Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#5 lore raw

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

Quoted 21 lines · Johannes Weiner
Quoted 19 lines · Andrew Morton
Quoted 15 lines · On Tue, 21 Oct 2008 13:33:45 +0200 Johannes Weiner <[email protected]> wrote:

File pages mapped only in sequentially read mappings are perfect reclaim canditates.

This makes MADV_SEQUENTIAL mappings behave like a weak references, their pages will be reclaimed unless they have a strong reference from a normal mapping as well.

The patch changes the reclaim and the unmap path where they check if the page has been referenced. In both cases, accesses through sequentially read mappings will be ignored.

Signed-off-by Johannes Weiner <[email protected]>

---
II: add likely()s to mitigate the extra branches a bit as to Nick's
suggestion

Quoted 4 lines
Quoted 1 line

Is http://hannes.saeurebad.de/madvseq/ still true with this version?

No, sorry, still running benchmarks on this version. Coming up soon...

Ok, reran the tests I used for the data on this website and updated it. Take a look. I am quite overwhelmed by the results, hehe.

Kosaki-san, could you perhaps run the tests you did for the previous patch on this one, too? I am not getting any stable results for throughput measuring...

Quoted 1 line

Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#6 lore raw

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

Quoted 11 lines
Quoted 4 lines
Quoted 1 line

Is http://hannes.saeurebad.de/madvseq/ still true with this version?

No, sorry, still running benchmarks on this version. Coming up soon...

Ok, reran the tests I used for the data on this website and updated it. Take a look. I am quite overwhelmed by the results, hehe.

Kosaki-san, could you perhaps run the tests you did for the previous patch on this one, too? I am not getting any stable results for throughput measuring...

Usually, any reclaim throughput mesurement isn't stable. Then I used an average of five times mesurement.

Unfortunately, I can't understand I should mesure which patch combination because you and Nick post many patches of this issue related yesterday. Please let me know it?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#7 lore raw

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

Quoted 8 lines · KOSAKI Motohiro
Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Is http://hannes.saeurebad.de/madvseq/ still true with this version?

No, sorry, still running benchmarks on this version. Coming up soon...

Ok, reran the tests I used for the data on this website and updated it. Take a look. I am quite overwhelmed by the results, hehe.

Quoted 6 lines
Quoted 3 lines

Kosaki-san, could you perhaps run the tests you did for the previous patch on this one, too? I am not getting any stable results for throughput measuring...

Usually, any reclaim throughput mesurement isn't stable. Then I used an average of five times mesurement.

Ah, okay. I will give it another spin, too.

Quoted 3 lines

Unfortunately, I can't understand I should mesure which patch combination because you and Nick post many patches of this issue related yesterday. Please let me know it?

mmotm
- the old mm-more-likely-reclaim-madv_sequential-mappings
+ Nick's mm: dont mark_page_accessed in fault path (from yesterday)
+ Apply this patch

Andrew's tree (not yet released) already has the first two changes, so if he releases a new mmotm in the meantime, you only need this patch on top of it.

Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

maintainer
#8 lore raw

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

Quoted 4 lines · On Wed, 22 Oct 2008 09:15:14 +0200 Johannes Weiner <[email protected]> wrote:

Andrew's tree (not yet released) already has the first two changes, so if he releases a new mmotm in the meantime, you only need this patch on top of it.

OK, there's one there now.

It gets multiple definitions of elfcorehdr_addr if you pick an unfortunate config but I didn't bother fixing that because it'll probably go away next time I merge everything.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#9 lore raw

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

In reply to Johannes Weiner

Finally got some half-way stable time numbers for this patch.

My approach was to generate background activity by dd'ing my hard-drive to /dev/null and meanwhile copy a 1G file with memcpy() between two mmaps and then the same file again with MADV_SEQUENTIAL mmaps. The numbers below are averages/standard deviations from 8 copy iterations.

I dropped the caches before each copy and waited for them to become repopulated by the background dd.

The numbers in brackets are the std dev.

mmotm:

    normal  user: 1.775000s [0.053307] system: 9.620000s [0.135339] total: 98.875000s [0.613956]
   madvise  user: 2.552500s [0.041307] system: 9.442500s [0.075980] total: 73.937500s [0.734170]
mmotm+patch:
    normal  user: 1.850000s [0.013540] system: 9.760000s [0.047081] total: 99.250000s [0.569386]

madvise user: 2.547500s [0.014930] system: 8.865000s [0.055000] total: 71.897500s [0.144763]

Well, time-wise not sooo much of an improvement. But given the massively decreased LRU-rotation [ http://hannes.saeurebad.de/madvseq/ ] I'm still looking forward to Kosaki-san's throughput measurements :)

Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Acked-by
#10 lore raw

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

Quoted 9 lines

mmotm:

    normal  user: 1.775000s [0.053307] system: 9.620000s [0.135339] total: 98.875000s [0.613956]
   madvise  user: 2.552500s [0.041307] system: 9.442500s [0.075980] total: 73.937500s [0.734170]
mmotm+patch:
    normal  user: 1.850000s [0.013540] system: 9.760000s [0.047081] total: 99.250000s [0.569386]

madvise user: 2.547500s [0.014930] system: 8.865000s [0.055000] total: 71.897500s [0.144763]

Well, time-wise not sooo much of an improvement. But given the massively decreased LRU-rotation [ http://hannes.saeurebad.de/madvseq/ ]

My first impression, this result mean the patch is not so useful. But anyway, I mesured it again because I think Nick's opinion is very reasonable and I don't know your mesurement condition so detail.

Quoted 1 line

I'm still looking forward to Kosaki-san's throughput measurements :)

I'm sorry for late responce.
but I'd like to you know this mesurement need spent long time in my time.

1. copybench (http://code.google.com/p/copybench/)

my machine mem: 8GB
target file size: 10GB (filesize > system mem)

                         mmotm-1022  + the patch
   ==============================================================
   rw_cp                 6:32        6:34
   rw_fadv_cp            6:34        6:35
   mm_sync_cp            6:15        6:16
   mm_sync_madv_cp       6:19        6:14
   mw_cp                 6:11        6:12
   mw_madv_cp            6:13        6:12

MADV_SEQUENTIAL decrease performance a bit on mmotm. but the patch fix it.

2. MADV_SEQUENTIAL vs dbench

mmotm1022 + the patch
==============================================================

   mm_sync_madv_cp       6:29        6:19           (min:sec)
   dbench throughput     11.633      14.4045        (MB/s)
   dbench latency        65628       18565          (ms)

mmotm's copy decrease performance largely. but the patch decrease it a bit. dbench throuput improve about 25%, latency improve about 3.5 times.

So, I think the patch better than v1 and we should appreciate for Nick's good suggestion.

Hanns, Actually I recomend to spent a bit more time for proper benchmark design and settings.

Acked-by KOSAKI Motohiro <[email protected]>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#11 lore raw

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

Quoted 3 lines · KOSAKI Motohiro

I'm sorry for late responce.
but I'd like to you know this mesurement need spent long time in my time.

No problem, I really appreciate this.

Quoted 2 lines

Hanns, Actually I recomend to spent a bit more time for proper benchmark design and settings.

I shall dig into it.

Quoted 1 line
Acked-by KOSAKI Motohiro <[email protected]>

Thanks a lot,

Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#12 lore raw

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

In reply to KOSAKI Motohiro
Quoted 7 lines · KOSAKI Motohiro
Quoted 7 lines

mmotm:

    normal  user: 1.775000s [0.053307] system: 9.620000s [0.135339] total: 98.875000s [0.613956]
   madvise  user: 2.552500s [0.041307] system: 9.442500s [0.075980] total: 73.937500s [0.734170]
mmotm+patch:
    normal  user: 1.850000s [0.013540] system: 9.760000s [0.047081] total: 99.250000s [0.569386]

madvise user: 2.547500s [0.014930] system: 8.865000s [0.055000] total: 71.897500s [0.144763]

Quoted 6 lines
Quoted 2 lines

Well, time-wise not sooo much of an improvement. But given the massively decreased LRU-rotation [ http://hannes.saeurebad.de/madvseq/ ]

My first impression, this result mean the patch is not so useful. But anyway, I mesured it again because I think Nick's opinion is very reasonable and I don't know your mesurement condition so detail.

It may not make much of a difference if the MADV_SEQUENTIAL program is the only thing running on the system.

However, the goal of MADV_SEQUENTIAL is to make sure that a streaming mapping does not kick the data from other programs out of memory. The patch should take care of that very well.

Signature · 7 lines
All rights reversed.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
author
#13 lore raw

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

Quoted 11 lines · Rik van Riel
Quoted 10 lines · KOSAKI Motohiro
Quoted 9 lines

mmotm:

    normal  user: 1.775000s [0.053307] system: 9.620000s [0.135339] total: 98.875000s [0.613956]
   madvise  user: 2.552500s [0.041307] system: 9.442500s [0.075980] total: 73.937500s [0.734170]
mmotm+patch:
    normal  user: 1.850000s [0.013540] system: 9.760000s [0.047081] total: 99.250000s [0.569386]

madvise user: 2.547500s [0.014930] system: 8.865000s [0.055000] total: 71.897500s [0.144763]

Well, time-wise not sooo much of an improvement. But given the massively decreased LRU-rotation [ http://hannes.saeurebad.de/madvseq/ ]

Quoted 6 lines
Quoted 3 lines

My first impression, this result mean the patch is not so useful. But anyway, I mesured it again because I think Nick's opinion is very reasonable and I don't know your mesurement condition so detail.

It may not make much of a difference if the MADV_SEQUENTIAL program is the only thing running on the system.

As said, I had a big dd running in the background. The box has only 768mb RAM, so there really was VM activity going on.

And given this small standard deviations, the numbers seem pretty stable. Even if I had taken more samples, I highly doubt that they would have looked much different.

Perhaps still not enough VM pressure...

Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#14 lore raw

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

Quoted 20 lines · Johannes Weiner
Quoted 11 lines · Rik van Riel
Quoted 10 lines · KOSAKI Motohiro
Quoted 9 lines

mmotm:

    normal  user: 1.775000s [0.053307] system: 9.620000s [0.135339] total: 98.875000s [0.613956]
   madvise  user: 2.552500s [0.041307] system: 9.442500s [0.075980] total: 73.937500s [0.734170]
mmotm+patch:
    normal  user: 1.850000s [0.013540] system: 9.760000s [0.047081] total: 99.250000s [0.569386]

madvise user: 2.547500s [0.014930] system: 8.865000s [0.055000] total: 71.897500s [0.144763]

Well, time-wise not sooo much of an improvement. But given the massively decreased LRU-rotation [ http://hannes.saeurebad.de/madvseq/ ]

Quoted 6 lines
Quoted 3 lines

My first impression, this result mean the patch is not so useful. But anyway, I mesured it again because I think Nick's opinion is very reasonable and I don't know your mesurement condition so detail.

It may not make much of a difference if the MADV_SEQUENTIAL program is the only thing running on the system.

Quoted 8 lines

As said, I had a big dd running in the background. The box has only 768mb RAM, so there really was VM activity going on.

And given this small standard deviations, the numbers seem pretty stable. Even if I had taken more samples, I highly doubt that they would have looked much different.

Perhaps still not enough VM pressure...

Okay, Rik was kind enough to explain me what was wrong about my approach on IRC.

Yes, the benchmark pretty much missed the point. Please ignore.

Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#15 lore raw

Re: [patch] mm: more likely reclaim MADV_SEQUENTIAL mappings II

In reply to Rik van Riel
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines

Well, time-wise not sooo much of an improvement. But given the massively decreased LRU-rotation [ http://hannes.saeurebad.de/madvseq/ ]

My first impression, this result mean the patch is not so useful. But anyway, I mesured it again because I think Nick's opinion is very reasonable and I don't know your mesurement condition so detail.

Quoted 6 lines

It may not make much of a difference if the MADV_SEQUENTIAL program is the only thing running on the system.

However, the goal of MADV_SEQUENTIAL is to make sure that a streaming mapping does not kick the data from other programs out of memory. The patch should take care of that very well.

Well, my second test(following) indicate it IMO.

Quoted 7 lines

2. MADV_SEQUENTIAL vs dbench

mmotm1022 + the patch
==============================================================

 mm_sync_madv_cp       6:29        6:19           (min:sec)
 dbench throughput     11.633      14.4045        (MB/s)
 dbench latency        65628       18565          (ms)

So, I think the code of this patch is good, but I guess his mesurement way isn't so good.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#16 lore raw
In reply to Johannes Weiner
Quoted 9 lines · Johannes Weiner

I'm afraid this is now quite a bit more aggressive than the earlier version. When the fault path did a mark_page_access(), we wouldn't reclaim a page when it has been faulted into several MADV_SEQUENTIAL mappings but now we ignore *every* activity through such a mapping.

What do you think?

Perhaps we should note a reference if there are two or more accesses through sequentially read mappings?

That can be easily accomplished by dropping the memory.c part of your patch.

I do not know whether that would work any better than the patch you just posted, though :)

Signature · 7 lines
All rights reversed.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
author
#17 lore raw
Quoted 7 lines · Rik van Riel
Quoted 5 lines · Johannes Weiner

I'm afraid this is now quite a bit more aggressive than the earlier version. When the fault path did a mark_page_access(), we wouldn't reclaim a page when it has been faulted into several MADV_SEQUENTIAL mappings but now we ignore *every* activity through such a mapping.

Quoted 7 lines
Quoted 4 lines

What do you think?

Perhaps we should note a reference if there are two or more accesses through sequentially read mappings?

That can be easily accomplished by dropping the memory.c part of your patch.

I thought about that, but wouldn't we count a reference in the chain

        fault -> unmap -> page_referenced()

opposed to counting _no_ reference in

        fault -> page_referenced() -> ... -> unmap

?

Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>