migration: only migrate_prep() once per move_pages()

migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz). Commit 3140a2273009c01c27d316f35ab76a37e105fdd8 improved move_pages() throughput by breaking it into chunks, but it also made migrate_prep() be called once per chunk (every 128pages or so) instead of once per move_pages().

This patch reverts to calling migrate_prep() only once per chunk as we did before 2.6.29.
It is also a followup to commit 0aedadf91a70a11c4a3e7c7d99b21e5528af8d5d
mm: move migrate_prep out from under mmap_sem

This improves migration throughput on the above machine from 600MB/s to 750MB/s.

Signed-off-by Brice Goglin <[email protected]>

mm/migrate.c +3 -2

index 068655d..a2d3e83 100644
@@ -820,7 +820,6 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
 	struct page_to_node *pp;
 	LIST_HEAD(pagelist);
 
-	migrate_prep();
 	down_read(&mm->mmap_sem);
 
 	/*

@@ -907,6 +906,9 @@ static int do_pages_move(struct mm_struct *mm, struct task_struct *task,
 	pm = (struct page_to_node *)__get_free_page(GFP_KERNEL);
 	if (!pm)
 		goto out;
+
+	migrate_prep();
+
 	/*
 	 * Store a chunk of page_to_node array in a page,
 	 * but keep the last one as a marker
 
 
--

To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 14 lines · On Wed, 15 Apr 2009 09:32:10 +0200 Brice Goglin <[email protected]>…

migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz). Commit 3140a2273009c01c27d316f35ab76a37e105fdd8 improved move_pages() throughput by breaking it into chunks, but it also made migrate_prep() be called once per chunk (every 128pages or so) instead of once per move_pages().

This patch reverts to calling migrate_prep() only once per chunk as we did before 2.6.29.
It is also a followup to commit 0aedadf91a70a11c4a3e7c7d99b21e5528af8d5d
mm: move migrate_prep out from under mmap_sem

This improves migration throughput on the above machine from 600MB/s to 750MB/s.

Quoted 3 lines

Signed-off-by Brice Goglin <[email protected]>

Reviewed-by KAMEZAWA Hiroyuki <[email protected]>

I think this patch is good. page migration is best-effort syscall ;)

BTW, current users of sys_move_pages() does retry when it gets -EBUSY ?

But lru_add_drain_all() touches some code that I am far from understanding :/ Can we imagine using IPI instead of a deferred work_struct for this kind of things? Or maybe, for each processor, check whether drain_cpu_pagevecs() would have something to do before actually scheduling the local work_struct? It's racy, but migrate_prep() doesn't guarantee anyway that pages won't be moved out of the LRU before the actual migration, so...

Quoted 2 lines

BTW, current users of sys_move_pages() does retry when it gets -EBUSY ?

I'd say they ignore it since it doesn't happen often :)

Quoted 14 lines · On Wed, 15 Apr 2009 09:32:10 +0200 Brice Goglin <[email protected]>…

migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz). Commit 3140a2273009c01c27d316f35ab76a37e105fdd8 improved move_pages() throughput by breaking it into chunks, but it also made migrate_prep() be called once per chunk (every 128pages or so) instead of once per move_pages().

This patch reverts to calling migrate_prep() only once per chunk as we did before 2.6.29.
It is also a followup to commit 0aedadf91a70a11c4a3e7c7d99b21e5528af8d5d
mm: move migrate_prep out from under mmap_sem

This improves migration throughput on the above machine from 600MB/s to 750MB/s.

Quoted 3 lines

Signed-off-by Brice Goglin <[email protected]>

Reviewed-by KAMEZAWA Hiroyuki <[email protected]>

I think this patch is good. page migration is best-effort syscall ;)