mm/rmap.c: reuse mergeable anon_vma as parent when fork

PATCH In review Wei Yang 4 participants 10 messages Oct 4, 2019 – Oct 5, 2019

Subsystems (2)

Memory management MM rmap

lore

Share & tools

lore

Wei Yang

author

2019-10-04 16:06 UTC #0 lore raw

PATCH 1/1 mm/rmap.c: reuse mergeable anon_vma as parent when fork

In function __anon_vma_prepare(), we will try to find anon_vma if it is possible to reuse it. While on fork, the logic is different.

Since commit 5beb49305251 ("mm: change anon_vma linking to fix multi-process server scalability issue"), function anon_vma_clone() tries to allocate new anon_vma for child process. But the logic here will allocate a new anon_vma for each vma, even in parent this vma is mergeable and share the same anon_vma with its sibling. This may do better for scalability issue, while it is not necessary to do so especially after interval tree is used.

Commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy") tries to reuse some anon_vma by counting child anon_vma and attached vmas. While for those mergeable anon_vmas, we can just reuse it and not necessary to go through the logic.

After this change, kernel build test reduces 20% anon_vma allocation.

Signed-off-by Wei Yang <[email protected]>

---
 mm/rmap.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

mm/rmap.c +11 -0

index d9a23bb773bf..12f6c3d7fd9d 100644
@@ -262,6 +262,17 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)
 {
 	struct anon_vma_chain *avc, *pavc;
 	struct anon_vma *root = NULL;
+	struct vm_area_struct *prev = dst->vm_prev, *pprev = src->vm_prev;
+
+	/*
+	 * If parent share anon_vma with its vm_prev, keep this sharing in in
+	 * child.
+	 *
+	 * 1. Parent has vm_prev, which implies we have vm_prev.
+	 * 2. Parent and its vm_prev have the same anon_vma.
+	 */
+	if (pprev && pprev->anon_vma == src->anon_vma)
+		dst->anon_vma = prev->anon_vma;
 
 	list_for_each_entry_reverse(pavc, &src->anon_vma_chain, same_vma) {
 		struct anon_vma *anon_vma;

Signature · 1 line

2.17.1

Matthew Wilcox

2019-10-04 16:11 UTC #1 lore raw

Quoted 1 line · Wei Yang

After this change, kernel build test reduces 20% anon_vma allocation.

But does it have any effect on elapsed time or peak memory consumption?

Wei Yang

author

2019-10-04 22:33 UTC #2 lore raw

Quoted 4 lines · Matthew Wilcox

Quoted 1 line · Wei Yang
After this change, kernel build test reduces 20% anon_vma allocation.
But does it have any effect on elapsed time or peak memory consumption?

I didn't evaluate these aspects.

BTW, how to evaluate peak memory consumption? This looks a transient value.

Signature · 2 lines

Wei Yang
Help you, Help me

Wei Yang

author

2019-10-04 23:49 UTC #3 lore raw

In reply to Matthew Wilcox

Quoted 4 lines · Matthew Wilcox

Quoted 1 line · Wei Yang
After this change, kernel build test reduces 20% anon_vma allocation.
But does it have any effect on elapsed time or peak memory consumption?

Do the same kernel build test and record time:

Origin

real 2m50.467s
user 17m52.002s
sys 1m51.953s

real 2m48.662s
user 17m55.464s
sys 1m50.553s

real 2m51.143s
user 17m59.687s
sys 1m53.600s

Patched

real 2m43.733s
user 17m25.705s
sys 1m41.791s

real 2m47.146s
user 17m47.451s
sys 1m43.474s

real 2m45.763s
user 17m38.230s
sys 1m42.102s

For time in sys, it reduced 8.5%.

Signature · 2 lines

Wei Yang
Help you, Help me

Matthew Wilcox

2019-10-05 01:16 UTC #4 lore raw

Quoted 34 lines · Wei Yang

Quoted 4 lines · Matthew Wilcox
Quoted 1 line · Wei Yang
After this change, kernel build test reduces 20% anon_vma allocation.
But does it have any effect on elapsed time or peak memory consumption?
Do the same kernel build test and record time:
Origin
real 2m50.467s
user 17m52.002s
sys 1m51.953s
real 2m48.662s
user 17m55.464s
sys 1m50.553s
real 2m51.143s
user 17m59.687s
sys 1m53.600s
Patched
real 2m43.733s
user 17m25.705s
sys 1m41.791s
real 2m47.146s
user 17m47.451s
sys 1m43.474s

Quoted 6 lines

real 2m45.763s
user 17m38.230s
sys 1m42.102s
For time in sys, it reduced 8.5%.

That's compelling!

Wei Yang

author

2019-10-05 12:35 UTC #5 lore raw

Quoted 35 lines · Matthew Wilcox

Quoted 34 lines · Wei Yang
Quoted 4 lines · Matthew Wilcox
Quoted 1 line · Wei Yang
After this change, kernel build test reduces 20% anon_vma allocation.
But does it have any effect on elapsed time or peak memory consumption?
Do the same kernel build test and record time:
Origin
real 2m50.467s
user 17m52.002s
sys 1m51.953s
real 2m48.662s
user 17m55.464s
sys 1m50.553s
real 2m51.143s
user 17m59.687s
sys 1m53.600s
Patched
real 2m43.733s
user 17m25.705s
sys 1m41.791s
real 2m47.146s
user 17m47.451s
sys 1m43.474s

Quoted 8 lines

Quoted 6 lines
real 2m45.763s
user 17m38.230s
sys 1m42.102s
For time in sys, it reduced 8.5%.
That's compelling!

Yep, looks good :-)

Signature · 2 lines

Wei Yang
Help you, Help me

Konstantin Khlebnikov

Acked-by on v1 1/1

2019-10-04 16:33 UTC #6 lore raw

In reply to Wei Yang

Quoted 11 lines · Wei Yang

In function __anon_vma_prepare(), we will try to find anon_vma if it is possible to reuse it. While on fork, the logic is different.
Since commit 5beb49305251 ("mm: change anon_vma linking to fix multi-process server scalability issue"), function anon_vma_clone() tries to allocate new anon_vma for child process. But the logic here will allocate a new anon_vma for each vma, even in parent this vma is mergeable and share the same anon_vma with its sibling. This may do better for scalability issue, while it is not necessary to do so especially after interval tree is used.

Quoted 7 lines

Commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy") tries to reuse some anon_vma by counting child anon_vma and attached vmas. While for those mergeable anon_vmas, we can just reuse it and not necessary to go through the logic.
After this change, kernel build test reduces 20% anon_vma allocation.

Makes sense. This might have much bigger effect for scenarios when task unmaps holes in huge vma as red-zones between allocations and then forks.

Acked-by Konstantin Khlebnikov <[email protected]>

Quoted 28 lines

Signed-off-by Wei Yang <[email protected]>

---
  mm/rmap.c | 11 +++++++++++
  1 file changed, 11 insertions(+)

mm/rmap.c +11 -0

index d9a23bb773bf..12f6c3d7fd9d 100644
@@ -262,6 +262,17 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)
  {
  	struct anon_vma_chain *avc, *pavc;
  	struct anon_vma *root = NULL;
+	struct vm_area_struct *prev = dst->vm_prev, *pprev = src->vm_prev;
+
+	/*
+	 * If parent share anon_vma with its vm_prev, keep this sharing in in
+	 * child.
+	 *
+	 * 1. Parent has vm_prev, which implies we have vm_prev.
+	 * 2. Parent and its vm_prev have the same anon_vma.
+	 */
+	if (pprev && pprev->anon_vma == src->anon_vma)
+		dst->anon_vma = prev->anon_vma;
 
  	list_for_each_entry_reverse(pavc, &src->anon_vma_chain, same_vma) {
  		struct anon_vma *anon_vma;

Wei Yang

author

2019-10-04 22:38 UTC #7 lore raw

Quoted 17 lines · Konstantin Khlebnikov

Quoted 16 lines · Wei Yang
In function __anon_vma_prepare(), we will try to find anon_vma if it is possible to reuse it. While on fork, the logic is different.
Since commit 5beb49305251 ("mm: change anon_vma linking to fix multi-process server scalability issue"), function anon_vma_clone() tries to allocate new anon_vma for child process. But the logic here will allocate a new anon_vma for each vma, even in parent this vma is mergeable and share the same anon_vma with its sibling. This may do better for scalability issue, while it is not necessary to do so especially after interval tree is used.
Commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy") tries to reuse some anon_vma by counting child anon_vma and attached vmas. While for those mergeable anon_vmas, we can just reuse it and not necessary to go through the logic.

Quoted 8 lines

Quoted 2 lines
After this change, kernel build test reduces 20% anon_vma allocation.
Makes sense. This might have much bigger effect for scenarios when task unmaps holes in huge vma as red-zones between allocations and then forks.
Acked-by Konstantin Khlebnikov <[email protected]>

Thanks

Signature · 2 lines

Wei Yang
Help you, Help me

Rik van Riel

reviewer Acked-by on v1 1/1

2019-10-04 23:45 UTC #8 lore raw

In reply to Wei Yang

Quoted 20 lines · Wei Yang

In function __anon_vma_prepare(), we will try to find anon_vma if it is
possible to reuse it. While on fork, the logic is different.
Since commit 5beb49305251 ("mm: change anon_vma linking to fix multi-process server scalability issue"), function anon_vma_clone() tries to allocate new anon_vma for child process. But the logic here will allocate a new anon_vma for each vma, even in parent this vma is mergeable and share the same anon_vma with its sibling. This may do
better for scalability issue, while it is not necessary to do so especially after interval tree is used.
Commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy")
tries to reuse some anon_vma by counting child anon_vma and attached vmas. While for those mergeable anon_vmas, we can just reuse it and not
necessary to go through the logic.

Quoted 3 lines

After this change, kernel build test reduces 20% anon_vma allocation.
Signed-off-by Wei Yang <[email protected]>

Acked-by Rik van Riel <[email protected]>

Signature · 1 line

All Rights Reversed.

Wei Yang

author

2019-10-04 23:49 UTC #9 lore raw

Quoted 21 lines · Rik van Riel

Quoted 20 lines · Wei Yang
In function __anon_vma_prepare(), we will try to find anon_vma if it is
possible to reuse it. While on fork, the logic is different.
Since commit 5beb49305251 ("mm: change anon_vma linking to fix multi-process server scalability issue"), function anon_vma_clone() tries to allocate new anon_vma for child process. But the logic here will allocate a new anon_vma for each vma, even in parent this vma is mergeable and share the same anon_vma with its sibling. This may do
better for scalability issue, while it is not necessary to do so especially after interval tree is used.
Commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy")
tries to reuse some anon_vma by counting child anon_vma and attached vmas. While for those mergeable anon_vmas, we can just reuse it and not
necessary to go through the logic.

Quoted 6 lines

Quoted 3 lines
After this change, kernel build test reduces 20% anon_vma allocation.
Signed-off-by Wei Yang <[email protected]>
Acked-by Rik van Riel <[email protected]>

Thanks

Quoted 2 lines

Signature · 1 line
All Rights Reversed.

Signature · 2 lines

Wei Yang
Help you, Help me