Skip to contentSkip to conversation

mm: Consider the entire user address space during node migration

PATCH Superseded Greg Thelen 3 participants 7 messages May 14, 2010 – May 17, 2010

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm: Consider the entire user address space during node migration

This patch uses TASK_SIZE_MAX instead of TASK_SIZE to ensure that the entire user address space is migrated. TASK_SIZE_MAX is independent of the calling task context. TASK SIZE may be dependant on the address space size of the calling process. Usage of TASK_SIZE can lead to partial address space migration if the calling process was 32 bit and the migrating process was 64 bit.

Here is the test script used on 64 system with a 32 bit echo process:
mount -t cgroup none /cgroup -o cpuset
cd /cgroup

mkdir 0
echo 1 > 0/cpuset.cpus
echo 0 > 0/cpuset.mems
echo 1 > 0/cpuset.memory_migrate

mkdir 1
echo 1 > 1/cpuset.cpus
echo 1 > 1/cpuset.mems
echo 1 > 1/cpuset.memory_migrate

echo $$ > 0/tasks
64_bit_process &
pid=$!

echo $pid > 1/tasks # This does not migrate all process pages without

                        # this patch.  If 64 bit echo is used or this patch is
                        # applied, then the full address space of $pid is
                        # migrated.

To check memory migration, I watched:
grep MemUsed /sys/devices/system/node/node*/meminfo

---
 mm/mempolicy.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
mm/mempolicy.c +1 -1
index 9f11728..a42c0f1 100644
@@ -928,7 +928,7 @@ static int migrate_to_node(struct mm_struct *mm, int source, int dest,
 	nodes_clear(nmask);
 	node_set(source, nmask);
 
-	check_range(mm, mm->mmap->vm_start, TASK_SIZE, &nmask,
+	check_range(mm, mm->mmap->vm_start, TASK_SIZE_MAX, &nmask,
 			flags | MPOL_MF_DISCONTIG_OK, &pagelist);
 
 	if (!list_empty(&pagelist))
Signature · 7 lines
1.7.0.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#2 lore raw

Hi

Mysteriously, I haven't receive original post.
So now I'm guessing you acked following patch.

http://lkml.org/lkml/2010/5/14/393

but I don't think it is correct.

Quoted 3 lines
mm/mempolicy.c +1 -1
-	check_range(mm, mm->mmap->vm_start, TASK_SIZE, &nmask,
+	check_range(mm, mm->mmap->vm_start, TASK_SIZE_MAX, &nmask,
			flags | MPOL_MF_DISCONTIG_OK, &pagelist);

Because TASK_SIZE_MAX is defined on x86 only. Why can we ignore other platform? Please put following line anywhere.

#define TASK_SIZE_MAX TASK_SIZE

But this patch is conceptually good. if it fixes the bug. I'll ack gladly.

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#3 lore raw

PATCH v2 1/1 mm: Consider the entire user address space during node migration

This patch uses mm->task_size instead of TASK_SIZE to ensure that the entire user address space is migrated. mm->task_size is independent of the calling task context. TASK SIZE may be dependant on the address space size of the calling process. Usage of TASK_SIZE can lead to partial address space migration if the calling process was 32 bit and the migrating process was 64 bit.

Here is the test script used on 64 system with a 32 bit echo process:
mount -t cgroup none /cgroup -o cpuset
cd /cgroup

mkdir 0
echo 1 > 0/cpuset.cpus
echo 0 > 0/cpuset.mems
echo 1 > 0/cpuset.memory_migrate

mkdir 1
echo 1 > 1/cpuset.cpus
echo 1 > 1/cpuset.mems
echo 1 > 1/cpuset.memory_migrate

echo $$ > 0/tasks
64_bit_process &
pid=$!

echo $pid > 1/tasks # This does not migrate all process pages without

                        # this patch.  If 64 bit echo is used or this patch is
                        # applied, then the full address space of $pid is
                        # migrated.

To check memory migration, I watched:
grep MemUsed /sys/devices/system/node/node*/meminfo

Changes since v1:
- Use mm->task_size rather than TASK_SIZE_MAX to support all platforms.

---
 mm/mempolicy.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
mm/mempolicy.c +1 -1
index 9f11728..2fd17e7 100644
@@ -928,7 +928,7 @@ static int migrate_to_node(struct mm_struct *mm, int source, int dest,
 	nodes_clear(nmask);
 	node_set(source, nmask);
 
-	check_range(mm, mm->mmap->vm_start, TASK_SIZE, &nmask,
+	check_range(mm, mm->mmap->vm_start, mm->task_size, &nmask,
 			flags | MPOL_MF_DISCONTIG_OK, &pagelist);
 
 	if (!list_empty(&pagelist))
Signature · 7 lines
1.7.0.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Reviewed-by on v2 1/1
#5 lore raw
In reply to Greg Thelen
Quoted 4 lines

Changes since v1:
- Use mm->task_size rather than TASK_SIZE_MAX to support all platforms.

Looks good. Thanks Greg!

Quoted 20 lines
---
 mm/mempolicy.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
mm/mempolicy.c +1 -1
index 9f11728..2fd17e7 100644
@@ -928,7 +928,7 @@ static int migrate_to_node(struct mm_struct *mm, int source, int dest,
 	nodes_clear(nmask);
 	node_set(source, nmask);
 
-	check_range(mm, mm->mmap->vm_start, TASK_SIZE, &nmask,
+	check_range(mm, mm->mmap->vm_start, mm->task_size, &nmask,
 			flags | MPOL_MF_DISCONTIG_OK, &pagelist);
 
 	if (!list_empty(&pagelist))
Signature · 1 line
1.7.0.1
#6 lore raw
In reply to KOSAKI Motohiro
Quoted 10 lines · KOSAKI Motohiro

Hi

Mysteriously, I haven't receive original post.
So now I'm guessing you acked following patch.

http://lkml.org/lkml/2010/5/14/393

but I don't think it is correct.

Quoted 8 lines
Quoted 3 lines

-     check_range(mm, mm->mmap->vm_start, TASK_SIZE, &nmask,
+     check_range(mm, mm->mmap->vm_start, TASK_SIZE_MAX, &nmask, flags | MPOL_MF_DISCONTIG_OK, &pagelist);

Because TASK_SIZE_MAX is defined on x86 only. Why can we ignore other platform? Please put following line anywhere.

#define TASK_SIZE_MAX TASK_SIZE

I just send out patch v2, which uses mm->task_size rather than TASK_SIZE_MAX. Some non-x86 architectures do not define TASK_SIZE_MAX, but do make TASK_SIZE depend on the current task. So I feel it would be better to refer to the mm struct to obtain the needed address space limit information rather than TASK_SIZE[_MAX], which can depend on current.

--
Greg

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>