Skip to contentSkip to conversation

swap-prefetch, fix lru_cache_add_tail()

PATCH In review Peter Zijlstra 2 participants 2 messages May 17, 2006 – May 18, 2006 Memory management

lore
Share & tools
lore

Peter Zijlstra

author
#0 lore raw

PATCH 1/1 swap-prefetch, fix lru_cache_add_tail()

From: Peter Zijlstra <[email protected]>

lru_cache_add_tail() uses the inactive per-cpu pagevec. This causes normal inactive and intactive tail inserts to end up on the wrong end of the list.

When the pagevec is completed by lru_cache_add_tail() but still contains normal inactive pages, all pages will be added to the inactive tail and vice versa.

Also *add_drain*() will always complete to the inactive head.

Add a third per-cpu pagevec to alleviate this problem.

Signed-off-by Peter Zijlstra <[email protected]>

---

 mm/swap.c |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
mm/swap.c +7 -1
Index: 2.6-mm/mm/swap.c
===================================================================
@@ -138,6 +138,7 @@ EXPORT_SYMBOL(mark_page_accessed);
  */
 static DEFINE_PER_CPU(struct pagevec, lru_add_pvecs) = { 0, };
 static DEFINE_PER_CPU(struct pagevec, lru_add_active_pvecs) = { 0, };
+static DEFINE_PER_CPU(struct pagevec, lru_add_tail_pvecs) = { 0, };
 
 void fastcall lru_cache_add(struct page *page)
 {

@@ -159,6 +160,8 @@ void fastcall lru_cache_add_active(struc
 	put_cpu_var(lru_add_active_pvecs);
 }
 
+static inline void __pagevec_lru_add_tail(struct pagevec *pvec);
+
 static void __lru_add_drain(int cpu)
 {
 	struct pagevec *pvec = &per_cpu(lru_add_pvecs, cpu);

@@ -169,6 +172,9 @@ static void __lru_add_drain(int cpu)
 	pvec = &per_cpu(lru_add_active_pvecs, cpu);
 	if (pagevec_count(pvec))
 		__pagevec_lru_add_active(pvec);
+	pvec = &per_cpu(lru_add_tail_pvecs, cpu);
+	if (pagevec_count(pvec))
+		__pagevec_lru_add_tail(pvec);
 }
 
 void lru_add_drain(void)

@@ -416,7 +422,7 @@ static inline void __pagevec_lru_add_tai
  */
 void fastcall lru_cache_add_tail(struct page *page)
 {
-	struct pagevec *pvec = &get_cpu_var(lru_add_pvecs);
+	struct pagevec *pvec = &get_cpu_var(lru_add_tail_pvecs);
 
 	page_cache_get(page);
 	if (!pagevec_add(pvec, page))

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#1 lore raw
Quoted 6 lines · Peter Zijlstra

From: Peter Zijlstra <[email protected]>

lru_cache_add_tail() uses the inactive per-cpu pagevec. This causes normal inactive and intactive tail inserts to end up on the wrong end of the list.

Quoted 7 lines

When the pagevec is completed by lru_cache_add_tail() but still contains normal inactive pages, all pages will be added to the inactive tail and vice versa.

Also *add_drain*() will always complete to the inactive head.

Add a third per-cpu pagevec to alleviate this problem.

Thanks!

Quoted 1 line
Signed-off-by Peter Zijlstra <[email protected]>
Signed-off-by Con Kolivas <[email protected]>
Signature · 7 lines
-ck

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>