mm/rmap.c: don't reuse anon_vma if we just want a copy
Subsystems (2)
PATCH v4 1/2 mm/rmap.c: don't reuse anon_vma if we just want a copy
Before commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy"), anon_vma_clone() doesn't change dst->anon_vma. While after this commit, anon_vma_clone() will try to reuse an exist one on forking.
But this commit go a little bit further for the case not forking. anon_vma_clone() is called from __vma_split(), __split_vma(), copy_vma() and anon_vma_fork(). For the first three places, the purpose here is get a copy of src and we don't expect to touch dst->anon_vma even it is NULL. While after that commit, it is possible to reuse an anon_vma when dst->anon_vma is NULL. This is not we intend to have.
This patch stop reuse anon_vma for non-fork cases.
Fix commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy")
---
v4:
* check dst->anon_vma in each iteration
v3:
* use dst->anon_vma and src->anon_vma to get reuse state pointed by Konstantin Khlebnikov
--- mm/rmap.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)
mm/rmap.c +15 -9
@@ -250,13 +250,19 @@ static inline void unlock_anon_vma_root(struct anon_vma *root)* Attach the anon_vmas from src to dst. * Returns 0 on success, -ENOMEM on failure. * -* If dst->anon_vma is NULL this function tries to find and reuse existing -* anon_vma which has no vmas and only one child anon_vma. This prevents -* degradation of anon_vma hierarchy to endless linear chain in case of -* constantly forking task. On the other hand, an anon_vma with more than one -* child isn't reused even if there was no alive vma, thus rmap walker has a -* good chance of avoiding scanning the whole hierarchy when it searches where -* page is mapped. +* anon_vma_clone() is called by __vma_split(), __split_vma(), copy_vma() and +* anon_vma_fork(). The first three want an exact copy of src, while the last +* one, anon_vma_fork(), may try to reuse an existing anon_vma to prevent +* endless growth of anon_vma. Since dst->anon_vma is set to NULL before call, +* we can identify this case by checking (!dst->anon_vma && src->anon_vma). +* +* If (!dst->anon_vma && src->anon_vma) is true, this function tries to find +* and reuse existing anon_vma which has no vmas and only one child anon_vma. +* This prevents degradation of anon_vma hierarchy to endless linear chain in +* case of constantly forking task. On the other hand, an anon_vma with more +* than one child isn't reused even if there was no alive vma, thus rmap +* walker has a good chance of avoiding scanning the whole hierarchy when it +* searches where page is mapped. */ int anon_vma_clone (struct vm_area_struct *dst,struct vm_area_struct *src) {
@@ -286,8 +292,8 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)* will always reuse it. Root anon_vma is never reused: * it has self-parent reference and at least one child. */ -if (!dst->anon_vma && anon_vma != src->anon_vma && - anon_vma->degree <2 ) +if (!dst->anon_vma && src->anon_vma && + anon_vma != src->anon_vma && anon_vma->degree <2 ) dst->anon_vma = anon_vma; }if (dst->anon_vma)
Signature · 1 line
2.17.1
PATCH v4 2/2 mm/rmap.c: reuse mergeable anon_vma as parent when fork
In function __anon_vma_prepare(), we will try to find anon_vma if it is possible to reuse it. While on fork, the logic is different.
Since commit 5beb49305251 ("mm: change anon_vma linking to fix multi-process server scalability issue"), function anon_vma_clone() tries to allocate new anon_vma for child process. But the logic here will allocate a new anon_vma for each vma, even in parent this vma is mergeable and share the same anon_vma with its sibling. This may do better for scalability issue, while it is not necessary to do so especially after interval tree is used.
Commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy") tries to reuse some anon_vma by counting child anon_vma and attached vmas. While for those mergeable anon_vmas, we can just reuse it and not necessary to go through the logic.
After this change, kernel build test reduces 20% anon_vma allocation.
Do the same kernel build test, it shows run time in sys reduced 11.6%.
Origin:
real 2m50.467s user 17m52.002s sys 1m51.953s
real 2m48.662s user 17m55.464s sys 1m50.553s
real 2m51.143s user 17m59.687s sys 1m53.600s
Patched:
real 2m39.933s
user 17m1.835s
sys 1m38.802s
real 2m39.321s
user 17m1.634s
sys 1m39.206s
real 2m39.575s
user 17m1.420s
sys 1m38.845s
--- mm/rmap.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
mm/rmap.c +13 -0
@@ -268,6 +268,19 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src) {struct anon_vma_chain *avc, *pavc;struct anon_vma *root =NULL ; +struct vm_area_struct *prev = dst->vm_prev, *pprev = src->vm_prev; + +/* +* If parent share anon_vma with its vm_prev, keep this sharing in in +* child. +* +* 1. Parent has vm_prev, which implies we have vm_prev. +* 2. Parent and its vm_prev have the same anon_vma. +*/ +if (!dst->anon_vma && src->anon_vma && + pprev && pprev->anon_vma == src->anon_vma) + dst->anon_vma = prev->anon_vma; +list_for_each_entry_reverse (pavc, &src->anon_vma_chain, same_vma) {struct anon_vma *anon_vma;
Signature · 1 line
2.17.1
Re: [Patch v4 2/2] mm/rmap.c: reuse mergeable anon_vma as parent when fork
Quoted 44 lines · Wei Yang
In function
__anon_vma_prepare(), we will try to findanon_vmaif it is possible to reuse it. While on fork, the logic is different.Since commit
5beb49305251("mm: changeanon_vmalinking to fix multi-process server scalability issue"), functionanon_vma_clone()tries to allocate newanon_vmafor child process. But the logic here will allocate a newanon_vmafor each vma, even in parent this vma is mergeable and share the sameanon_vmawith its sibling. This may do better for scalability issue, while it is not necessary to do so especially after interval tree is used.Commit
7a3ef208e662("mm: prevent endless growth ofanon_vmahierarchy") tries to reuse someanon_vmaby counting childanon_vmaand attached vmas. While for those mergeableanon_vmas, we can just reuse it and not necessary to go through the logic.After this change, kernel build test reduces 20%
anon_vmaallocation.Do the same kernel build test, it shows run time in sys reduced 11.6%.
Origin:
real 2m50.467s user 17m52.002s sys 1m51.953sreal 2m48.662s user 17m55.464s sys 1m50.553sreal 2m51.143s user 17m59.687s sys 1m53.600sPatched:
real 2m39.933s
user 17m1.835s
sys 1m38.802sreal 2m39.321s
user 17m1.634s
sys 1m39.206s
Quoted 5 lines
real 2m39.575s
user 17m1.420s
sys 1m38.845sSigned-off-by Wei Yang <[email protected]>
Quoted 15 lines
--- mm/rmap.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)mm/rmap.c +3
-0@@ -268,6 +268,19 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src) {struct anon_vma_chain *avc, *pavc;struct anon_vma *root =NULL ; +struct vm_area_struct *prev = dst->vm_prev, *pprev = src->vm_prev; + +/*
mm/rmap.c int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src) · line 283
Quoted 10 lines
mm/rmap.c +10
-0+* If parent share anon_vma with its vm_prev, keep this sharing in in +* child. +* +* 1. Parent has vm_prev, which implies we have vm_prev. +* 2. Parent and its vm_prev have the same anon_vma. +*/ +if (!dst->anon_vma && src->anon_vma && + pprev && pprev->anon_vma == src->anon_vma) + dst->anon_vma = prev->anon_vma; +
I believe that in present code "prev" cannot be NULL if !dst->anon_vma && src->anon_vma is true. It would be safer to check this explicitly.
Quoted 4 lines
mm/rmap.c +0
-0list_for_each_entry_reverse (pavc, &src->anon_vma_chain, same_vma) {struct anon_vma *anon_vma;
Quoted 11 lines · Wei Yang
Before commit
7a3ef208e662("mm: prevent endless growth ofanon_vmahierarchy"),anon_vma_clone()doesn't changedst->anon_vma. While after this commit,anon_vma_clone()will try to reuse an exist one on forking.But this commit go a little bit further for the case not forking.
anon_vma_clone()is called from__vma_split(),__split_vma(),copy_vma()andanon_vma_fork(). For the first three places, the purpose here is get a copy of src and we don't expect to touchdst->anon_vmaeven it is NULL. While after that commit, it is possible to reuse ananon_vmawhendst->anon_vmais NULL. This is not we intend to have.
Quoted 6 lines
This patch stop reuse
anon_vmafor non-fork cases.Fix commit
7a3ef208e662("mm: prevent endless growth ofanon_vmahierarchy")Signed-off-by Wei Yang <[email protected]>
Yes, reusing heuristic was designed for fork.
But this isn't strictly necessary - any vmas could share anon_vma. For example all vmas in system could be linked with single anon_vma.
Quoted 54 lines
---
v4:
* checkdst->anon_vmain each iteration
v3:
* usedst->anon_vmaandsrc->anon_vmato get reuse state pointed by Konstantin Khlebnikov--- mm/rmap.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)mm/rmap.c +15
-9@@ -250,13 +250,19 @@ static inline void unlock_anon_vma_root(struct anon_vma *root)* Attach the anon_vmas from src to dst. * Returns 0 on success, -ENOMEM on failure. * -* If dst->anon_vma is NULL this function tries to find and reuse existing -* anon_vma which has no vmas and only one child anon_vma. This prevents -* degradation of anon_vma hierarchy to endless linear chain in case of -* constantly forking task. On the other hand, an anon_vma with more than one -* child isn't reused even if there was no alive vma, thus rmap walker has a -* good chance of avoiding scanning the whole hierarchy when it searches where -* page is mapped. +* anon_vma_clone() is called by __vma_split(), __split_vma(), copy_vma() and +* anon_vma_fork(). The first three want an exact copy of src, while the last +* one, anon_vma_fork(), may try to reuse an existing anon_vma to prevent +* endless growth of anon_vma. Since dst->anon_vma is set to NULL before call, +* we can identify this case by checking (!dst->anon_vma && src->anon_vma). +* +* If (!dst->anon_vma && src->anon_vma) is true, this function tries to find +* and reuse existing anon_vma which has no vmas and only one child anon_vma. +* This prevents degradation of anon_vma hierarchy to endless linear chain in +* case of constantly forking task. On the other hand, an anon_vma with more +* than one child isn't reused even if there was no alive vma, thus rmap +* walker has a good chance of avoiding scanning the whole hierarchy when it +* searches where page is mapped. */ int anon_vma_clone (struct vm_area_struct *dst,struct vm_area_struct *src) {@@ -286,8 +292,8 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)* will always reuse it. Root anon_vma is never reused: * it has self-parent reference and at least one child. */ -if (!dst->anon_vma && anon_vma != src->anon_vma && - anon_vma->degree <2 ) +if (!dst->anon_vma && src->anon_vma && + anon_vma != src->anon_vma && anon_vma->degree <2 ) dst->anon_vma = anon_vma; }if (dst->anon_vma)
Quoted 14 lines · Konstantin Khlebnikov
Quoted 13 lines · Wei Yang
Before commit
7a3ef208e662("mm: prevent endless growth ofanon_vmahierarchy"),anon_vma_clone()doesn't changedst->anon_vma. While after this commit,anon_vma_clone()will try to reuse an exist one on forking.But this commit go a little bit further for the case not forking.
anon_vma_clone()is called from__vma_split(),__split_vma(),copy_vma()andanon_vma_fork(). For the first three places, the purpose here is get a copy of src and we don't expect to touchdst->anon_vmaeven it is NULL. While after that commit, it is possible to reuse ananon_vmawhendst->anon_vmais NULL. This is not we intend to have.This patch stop reuse
anon_vmafor non-fork cases.
Quoted 8 lines
Quoted 4 lines
Fix commit
7a3ef208e662("mm: prevent endless growth ofanon_vmahierarchy")Signed-off-by Wei Yang <[email protected]>Yes, reusing heuristic was designed for fork.
But this isn't strictly necessary - any vmas could shareanon_vma. For example all vmas in system could be linked with singleanon_vma.
Yes, agree with you.
Thanks for your comment :-)
Quoted 57 lines
Quoted 54 lines
---
v4:
* checkdst->anon_vmain each iteration
v3:
* usedst->anon_vmaandsrc->anon_vmato get reuse state pointed by Konstantin Khlebnikov--- mm/rmap.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)mm/rmap.c +15
-9@@ -250,13 +250,19 @@ static inline void unlock_anon_vma_root(struct anon_vma *root)* Attach the anon_vmas from src to dst. * Returns 0 on success, -ENOMEM on failure. * -* If dst->anon_vma is NULL this function tries to find and reuse existing -* anon_vma which has no vmas and only one child anon_vma. This prevents -* degradation of anon_vma hierarchy to endless linear chain in case of -* constantly forking task. On the other hand, an anon_vma with more than one -* child isn't reused even if there was no alive vma, thus rmap walker has a -* good chance of avoiding scanning the whole hierarchy when it searches where -* page is mapped. +* anon_vma_clone() is called by __vma_split(), __split_vma(), copy_vma() and +* anon_vma_fork(). The first three want an exact copy of src, while the last +* one, anon_vma_fork(), may try to reuse an existing anon_vma to prevent +* endless growth of anon_vma. Since dst->anon_vma is set to NULL before call, +* we can identify this case by checking (!dst->anon_vma && src->anon_vma). +* +* If (!dst->anon_vma && src->anon_vma) is true, this function tries to find +* and reuse existing anon_vma which has no vmas and only one child anon_vma. +* This prevents degradation of anon_vma hierarchy to endless linear chain in +* case of constantly forking task. On the other hand, an anon_vma with more +* than one child isn't reused even if there was no alive vma, thus rmap +* walker has a good chance of avoiding scanning the whole hierarchy when it +* searches where page is mapped. */ int anon_vma_clone (struct vm_area_struct *dst,struct vm_area_struct *src) {@@ -286,8 +292,8 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)* will always reuse it. Root anon_vma is never reused: * it has self-parent reference and at least one child. */ -if (!dst->anon_vma && anon_vma != src->anon_vma && - anon_vma->degree <2 ) +if (!dst->anon_vma && src->anon_vma && + anon_vma != src->anon_vma && anon_vma->degree <2 ) dst->anon_vma = anon_vma; }if (dst->anon_vma)
Signature · 2 lines
Wei Yang Help you, Help me