mm: mempolicy: make mbind() return -EIO when MPOL_MF_STRICT is specified
Subsystems (3)
PATCH 1/1 mm: mempolicy: make mbind() return -EIO when MPOL_MF_STRICT is specified
When MPOL_MF_STRICT was specified and an existing page was already on a node that does not follow the policy, mbind() should return -EIO. But commit 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()") broke the rule.
And, commit c8633798497c ("mm: mempolicy: mbind and migrate_pages support thp migration") didn't return the correct value for THP mbind() too.
If MPOL_MF_STRICT is set, ignore vma_migratable() to make sure it reaches queue_pages_to_pte_range() or queue_pages_pmd() to check if an existing page was already on a node that does not follow the policy. And, non-migratable vma may be used, return -EIO too if MPOL_MF_MOVE or MPOL_MF_MOVE_ALL was specified.
Tested with https://github.com/metan-ucw/ltp/blob/master/testcases/kernel/syscalls/mbind/mbind02.c
--- mm/mempolicy.c | 40 +++++++++++++++++++++++++++++++++------- 1 file changed, 33 insertions(+), 7 deletions(-)
mm/mempolicy.c +33 -7
@@ -447,6 +447,13 @@ static inline bool queue_pages_required(struct page *page,return node_isset (nid, *qp->nmask) == !(flags &MPOL_MF_INVERT ); } +/* +* The queue_pages_pmd() may have three kind of return value. +* 1 - pages are placed on he right node or queued successfully. +* 0 - THP get split. +* -EIO - is migration entry or MPOL_MF_STRICT was specified and an existing +* page was already on a node that does not follow the policy. +*/ static int queue_pages_pmd (pmd_t *pmd,spinlock_t *ptl,unsigned long addr,unsigned long end,struct mm_walk *walk) {
@@ -456,7 +463,7 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,unsigned long flags;if (unlikely (is_pmd_migration_entry (*pmd))) { - ret =1 ; + ret = -EIO ; goto unlock; } page =pmd_page (*pmd);
@@ -473,8 +480,15 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, ret =1 ; flags = qp->flags;/* go to thp migration */ -if (flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) +if (flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) { +if (!vma_migratable (walk->vma)) { + ret = -EIO ; + goto unlock; + } +migrate_page_add (page, qp->pagelist, flags); + }else + ret = -EIO ; unlock:spin_unlock (ptl); out:
@@ -499,8 +513,10 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, ptl =pmd_trans_huge_lock (pmd, vma);if (ptl) { ret =queue_pages_pmd (pmd, ptl, addr, end, walk); -if (ret) +if (ret >0 )return 0 ; +else if (ret <0 ) +return ret; }if (pmd_trans_unstable (pmd))
@@ -521,11 +537,16 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,continue ;if (!queue_pages_required (page, qp))continue ; -migrate_page_add (page, qp->pagelist, flags); +if (flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) { +if (!vma_migratable (vma)) +break ; +migrate_page_add (page, qp->pagelist, flags); + }else +break ; }pte_unmap_unlock (pte -1 , ptl);cond_resched (); -return 0 ; +return addr != end ? -EIO :0 ; } static int queue_pages_hugetlb(pte_t *pte, unsigned long hmask,
@@ -595,7 +616,12 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end,unsigned long endvma = vma->vm_end;unsigned long flags = qp->flags; -if (!vma_migratable (vma)) +/* +* Need check MPOL_MF_STRICT to return -EIO if possible +* regardless of vma_migratable +*/ +if (!vma_migratable (vma) && + !(flags &MPOL_MF_STRICT ))return 1 ;if (endvma > end)
@@ -622,7 +648,7 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end, }/* queue pages from current vma */ -if (flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) +if (flags &MPOL_MF_VALID )return 0 ;return 1 ; }
Signature · 1 line
1.8.3.1
Quoted 17 lines · Yang Shi
When
MPOL_MF_STRICTwas specified and an existing page was already on a node that does not follow the policy,mbind()should return-EIO. But commit6f4576e3687b("mempolicy: apply page table walker onqueue_pages_range()") broke the rule.And, commit
c8633798497c("mm: mempolicy: mbind andmigrate_pagessupport thp migration") didn't return the correct value for THPmbind()too.If
MPOL_MF_STRICTis set, ignorevma_migratable()to make sure it reachesqueue_pages_to_pte_range()orqueue_pages_pmd()to check if an existing page was already on a node that does not follow the policy. And, non-migratable vma may be used, return-EIOtoo ifMPOL_MF_MOVEorMPOL_MF_MOVE_ALLwas specified.Tested with https://github.com/metan-ucw/ltp/blob/master/testcases/kernel/syscalls/mbind/mbind02.c
Quoted 7 lines
Fixes 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()")Reported-by Cyril Hrubis <[email protected]>Cc Vlastimil Babka <[email protected]>Suggested-by Kirill A. Shutemov <[email protected]>Signed-off-by Yang Shi <[email protected]>Signed-off-by Oscar Salvador <[email protected]>
Thanks. I think this needs stable for 4.0+, can you confirm?
Quoted 26 lines · David Rientjes
Quoted 17 lines · Yang Shi
When
MPOL_MF_STRICTwas specified and an existing page was already on a node that does not follow the policy,mbind()should return-EIO. But commit6f4576e3687b("mempolicy: apply page table walker onqueue_pages_range()") broke the rule.And, commit
c8633798497c("mm: mempolicy: mbind andmigrate_pagessupport thp migration") didn't return the correct value for THPmbind()too.If
MPOL_MF_STRICTis set, ignorevma_migratable()to make sure it reachesqueue_pages_to_pte_range()orqueue_pages_pmd()to check if an existing page was already on a node that does not follow the policy. And, non-migratable vma may be used, return-EIOtoo ifMPOL_MF_MOVEorMPOL_MF_MOVE_ALLwas specified.Tested with https://github.com/metan-ucw/ltp/blob/master/testcases/kernel/syscalls/mbind/mbind02.c
Quoted 7 lines
Fixes 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()")Reported-by Cyril Hrubis <[email protected]>Cc Vlastimil Babka <[email protected]>Suggested-by Kirill A. Shutemov <[email protected]>Signed-off-by Yang Shi <[email protected]>Signed-off-by Oscar Salvador <[email protected]>
Quoted 3 lines
Acked-by David Rientjes <[email protected]>Thanks. I think this needs stable for 4.0+, can you confirm?
Thanks. Yes, this needs stable for 4.0+.
Yang
Quoted 34 lines · Yang Shi
When
MPOL_MF_STRICTwas specified and an existing page was already on a node that does not follow the policy,mbind()should return-EIO. But commit6f4576e3687b("mempolicy: apply page table walker onqueue_pages_range()") broke the rule.And, commit
c8633798497c("mm: mempolicy: mbind andmigrate_pagessupport thp migration") didn't return the correct value for THPmbind()too.If
MPOL_MF_STRICTis set, ignorevma_migratable()to make sure it reachesqueue_pages_to_pte_range()orqueue_pages_pmd()to check if an existing page was already on a node that does not follow the policy. And, non-migratable vma may be used, return-EIOtoo ifMPOL_MF_MOVEorMPOL_MF_MOVE_ALLwas specified.Tested with https://github.com/metan-ucw/ltp/blob/master/testcases/kernel/syscalls/mbind/mbind02.c
Fixes 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()")Reported-by Cyril Hrubis <[email protected]>Cc Vlastimil Babka <[email protected]>Suggested-by Kirill A. Shutemov <[email protected]>Signed-off-by Yang Shi <[email protected]>Signed-off-by Oscar Salvador <[email protected]>--- mm/mempolicy.c | 40 +++++++++++++++++++++++++++++++++------- 1 file changed, 33 insertions(+), 7 deletions(-)mm/mempolicy.c +0
-0
mm/mempolicy.c static inline bool queue_pages_required(struct page *page, · line 452
Quoted 6 lines
mm/mempolicy.c +3
-0@@ -447,6 +447,13 @@ static inline bool queue_pages_required(struct page *page,return node_isset (nid, *qp->nmask) == !(flags &MPOL_MF_INVERT ); } +/* +* The queue_pages_pmd() may have three kind of return value. +* 1 - pages are placed on he right node or queued successfully.
Minor typo -> s/he/the ?
Quoted 90 lines
mm/mempolicy.c +30
-8+* 0 - THP get split. +* -EIO - is migration entry or MPOL_MF_STRICT was specified and an existing +* page was already on a node that does not follow the policy. +*/ static int queue_pages_pmd (pmd_t *pmd,spinlock_t *ptl,unsigned long addr,unsigned long end,struct mm_walk *walk) {@@ -456,7 +463,7 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,unsigned long flags;if (unlikely (is_pmd_migration_entry (*pmd))) { - ret =1 ; + ret = -EIO ; goto unlock; } page =pmd_page (*pmd);@@ -473,8 +480,15 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, ret =1 ; flags = qp->flags;/* go to thp migration */ -if (flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) +if (flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) { +if (!vma_migratable (walk->vma)) { + ret = -EIO ; + goto unlock; + } +migrate_page_add (page, qp->pagelist, flags); + }else + ret = -EIO ; unlock:spin_unlock (ptl); out:@@ -499,8 +513,10 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, ptl =pmd_trans_huge_lock (pmd, vma);if (ptl) { ret =queue_pages_pmd (pmd, ptl, addr, end, walk); -if (ret) +if (ret >0 )return 0 ; +else if (ret <0 ) +return ret; }if (pmd_trans_unstable (pmd))@@ -521,11 +537,16 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,continue ;if (!queue_pages_required (page, qp))continue ; -migrate_page_add (page, qp->pagelist, flags); +if (flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) { +if (!vma_migratable (vma)) +break ; +migrate_page_add (page, qp->pagelist, flags); + }else +break ; }pte_unmap_unlock (pte -1 , ptl);cond_resched (); -return 0 ; +return addr != end ? -EIO :0 ; } static int queue_pages_hugetlb(pte_t *pte, unsigned long hmask,@@ -595,7 +616,12 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end,unsigned long endvma = vma->vm_end;unsigned long flags = qp->flags; -if (!vma_migratable (vma)) +/* +* Need check MPOL_MF_STRICT to return -EIO if possible +* regardless of vma_migratable +*/ +if (!vma_migratable (vma) && + !(flags &MPOL_MF_STRICT ))return 1 ;if (endvma > end)@@ -622,7 +648,7 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end, }/* queue pages from current vma */ -if (flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) +if (flags &MPOL_MF_VALID )return 0 ;return 1 ; } --1.8.3.1
Andrew Morton
maintainerQuoted 11 lines · On Wed, 20 Mar 2019 11:23:03 +0530 Souptick Joarder <[email protected]>…
mm/mempolicy.c static inline bool queue_pages_required(struct page *page, · line 452
Quoted 9 lines
mm/mempolicy.c +3
-0@@ -447,6 +447,13 @@ static inline bool queue_pages_required(struct page *page,return node_isset (nid, *qp->nmask) == !(flags &MPOL_MF_INVERT ); } +/* +* The queue_pages_pmd() may have three kind of return value. +* 1 - pages are placed on he right node or queued successfully. Minor typo -> s/he/the ?
Yes, that comment needs some help. This?
mm/mempolicy.c renamed from mm/mempolicy.c~mm-mempolicy-make-mbind-return-eio-when-mpol_mf_strict-is-specified-fix +3 -3
@@ -429,9 +429,9 @@ static inline bool queue_pages_required( }/* -* The queue_pages_pmd() may have three kind of return value. -* 1 - pages are placed on he right node or queued successfully. -* 0 - THP get split. +* queue_pages_pmd() has three possible return values: +* 1 - pages are placed on the right node or queued successfully. +* 0 - THP was split. * -EIO - is migration entry or MPOL_MF_STRICT was specified and an existing * page was already on a node that does not follow the policy. */
_
Quoted 19 lines · Andrew Morton
Quoted 10 lines · On Wed, 20 Mar 2019 11:23:03 +0530 Souptick Joarder <[email protected]>…
mm/mempolicy.c static inline bool queue_pages_required(struct page *page, · line 452
Quoted 9 lines
mm/mempolicy.c +3
-0@@ -447,6 +447,13 @@ static inline bool queue_pages_required(struct page *page,return node_isset (nid, *qp->nmask) == !(flags &MPOL_MF_INVERT ); } +/* +* The queue_pages_pmd() may have three kind of return value. +* 1 - pages are placed on he right node or queued successfully. Minor typo -> s/he/the ?
Yes, that comment needs some help. This?
mm/mempolicy.c renamed from mm/mempolicy.c~mm-mempolicy-make-mbind-return-eio-when-mpol_mf_strict-is-specified-fix +0
-0@@ -429,9 +429,9 @@ static inline bool queue_pages_required( }
mm/mempolicy.c static inline bool queue_pages_required( · line 437
Quoted 10 lines
mm/mempolicy.c renamed from mm/mempolicy.c~mm-mempolicy-make-mbind-return-eio-when-mpol_mf_strict-is-specified-fix +3
-3/* -* The queue_pages_pmd() may have three kind of return value. -* 1 - pages are placed on he right node or queued successfully. -* 0 - THP get split. +* queue_pages_pmd() has three possible return values: +* 1 - pages are placed on the right node or queued successfully. +* 0 - THP was split. * -EIO - is migration entry or MPOL_MF_STRICT was specified and an existing * page was already on a node that does not follow the policy. */
It looks good to me. Thanks, Andrew.
Yang
Quoted 1 line
_
Quoted 7 lines · Yang Shi
Fixes 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()")Reported-by Cyril Hrubis <[email protected]>Cc Vlastimil Babka <[email protected]>Suggested-by Kirill A. Shutemov <[email protected]>Signed-off-by Yang Shi <[email protected]>Signed-off-by Oscar Salvador <[email protected]>
Hi Yang, thanks for the patch.
Some observations below.
Quoted 6 lines
(quoted diff) +0
-0} page = pmd_page(*pmd);@@ -473,8 +480,15 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, ret = 1; flags = qp->flags; /* go to thp migration */
Review static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, · line 491
Quoted 10 lines
(quoted diff) +8
-1- if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) + if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) { + if (!vma_migratable(walk->vma)) { + ret = -EIO; + goto unlock; + } + migrate_page_add(page, qp->pagelist, flags); + } else + ret = -EIO;
if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) ||
!vma_migratable(walk->vma)) {
ret = -EIO;
goto unlock;
}
migrate_page_add(page, qp->pagelist, flags);
unlock:
spin_unlock(ptl);
out:
return ret;seems more clean to me?
Review static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 519
Quoted 12 lines
(quoted diff) +3
-1unlock: spin_unlock(ptl); out:@@ -499,8 +513,10 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, ptl = pmd_trans_huge_lock(pmd, vma); if (ptl) { ret = queue_pages_pmd(pmd, ptl, addr, end, walk); - if (ret) + if (ret > 0) return 0; + else if (ret < 0) + return ret;
I would go with the following, but that's a matter of taste I guess.
if (ret < 0)
return ret; else return 0;
Quoted 4 lines
(quoted diff) +0
-0} if (pmd_trans_unstable(pmd))
Review static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 545
Quoted 10 lines
(quoted diff) +6
-1@@ -521,11 +537,16 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, continue; if (!queue_pages_required(page, qp)) continue; - migrate_page_add(page, qp->pagelist, flags); + if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) { + if (!vma_migratable(vma)) + break; + migrate_page_add(page, qp->pagelist, flags); + } else + break;
I might be missing something, but AFAICS neither vma nor flags is going to change while we are in queue_pages_pte_range(), so, could not we move the check just above the loop?
In that way, 1) we only perform the check once and 2) if we enter the loop we know that we are going to do some work, so, something like:
index af171ccb56a2..7c0e44389826 100644
mm/mempolicy.c +3 -0
@@ -487,6 +487,9 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,if (pmd_trans_unstable (pmd))return 0 ; +if (!(flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) || !vma_migratable (vma)) +return -EIO ; + pte =pte_offset_map_lock (walk->mm, pmd, addr, &ptl);for (; addr != end; pte++, addr +=PAGE_SIZE ) {if (!pte_present (*pte))
Review static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 549
Quoted 5 lines
(quoted diff) +1
-1} pte_unmap_unlock(pte - 1, ptl); cond_resched(); - return 0; + return addr != end ? -EIO : 0;
If we can do the above, we can leave the return value as it was.
Signature · 2 lines
Oscar Salvador SUSE L3
Quoted 40 lines · Oscar Salvador
Quoted 7 lines · Yang Shi
Fixes 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()")Reported-by Cyril Hrubis <[email protected]>Cc Vlastimil Babka <[email protected]>Suggested-by Kirill A. Shutemov <[email protected]>Signed-off-by Yang Shi <[email protected]>Signed-off-by Oscar Salvador <[email protected]>Hi Yang, thanks for the patch.
Some observations below.
Quoted 6 lines
} page = pmd_page(*pmd);(quoted diff) +0
-0@@ -473,8 +480,15 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, ret = 1; flags = qp->flags; /* go to thp migration */Review static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, · line 491
Quoted 10 lines
(quoted diff) +8
-1- if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) + if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) { + if (!vma_migratable(walk->vma)) { + ret = -EIO; + goto unlock; + } + migrate_page_add(page, qp->pagelist, flags); + } else + ret = -EIO;if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || !vma_migratable(walk->vma)) { ret = -EIO; goto unlock; } migrate_page_add(page, qp->pagelist, flags); unlock: spin_unlock(ptl); out: return ret;
Quoted 1 line
seems more clean to me?
Yes, it sounds so.
Quoted 16 lines
Quoted 6 lines
(quoted diff) +0
-0unlock: spin_unlock(ptl); out:Review static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 519
Quoted 8 lines
(quoted diff) +3
-1@@ -499,8 +513,10 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, ptl = pmd_trans_huge_lock(pmd, vma); if (ptl) { ret = queue_pages_pmd(pmd, ptl, addr, end, walk); - if (ret) + if (ret > 0) return 0; + else if (ret < 0) + return ret;I would go with the following, but that's a matter of taste I guess.
Quoted 4 lines
if (ret < 0)
return ret; else return 0;
No, this is not correct. queue_pages_pmd() may return 0, which means THP gets split. If it returns 0 the code should just fall through instead of returning.
Quoted 25 lines
Quoted 5 lines
(quoted diff) +0
-0} if (pmd_trans_unstable(pmd))Review static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 545
Quoted 10 lines
(quoted diff) +6
-1@@ -521,11 +537,16 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, continue; if (!queue_pages_required(page, qp)) continue; - migrate_page_add(page, qp->pagelist, flags); + if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) { + if (!vma_migratable(vma)) + break; + migrate_page_add(page, qp->pagelist, flags); + } else + break;I might be missing something, but AFAICS neither vma nor flags is going to change while we are in
queue_pages_pte_range(), so, could not we move the check just above the loop?
In that way, 1) we only perform the check once and 2) if we enter the loop we know that we are going to do some work, so, something like:index
af171ccb56a2..7c0e44389826100644
mm/mempolicy.c static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 491
Quoted 5 lines
mm/mempolicy.c +2
-0@@ -487,6 +487,9 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,if (pmd_trans_unstable (pmd))return 0 ; +if (!(flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) || !vma_migratable (vma)) +return -EIO ;
It sounds not correct to me. We need check if there is existing page on the node which is not allowed by the policy. This is what queue_pages_required() does.
Thanks,
Yang
Quoted 13 lines
mm/mempolicy.c +1
-0+ pte =pte_offset_map_lock (walk->mm, pmd, addr, &ptl);for (; addr != end; pte++, addr +=PAGE_SIZE ) {if (!pte_present (*pte))Review static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 549
Quoted 7 lines
(quoted diff) +1
-1} pte_unmap_unlock(pte - 1, ptl); cond_resched(); - return 0; + return addr != end ? -EIO : 0;If we can do the above, we can leave the return value as it was.
Oscar Salvador
Reviewed-byQuoted 5 lines · Yang Shi
No, this is not correct.
queue_pages_pmd()may return 0, which means THP
gets split. If it returns 0 the code should just fall through instead of
returning.
Right, I overlooked that.
Quoted 4 lines
It sounds not correct to me. We need check if there is existing page on
the node which is not allowed by the policy. This is whatqueue_pages_required()does.
Bleh, I guess it was too early in the morning.
That is the whole point of it actually, so that was quite wrong.
Sorry for trying to mislead you ;-)
Signature · 2 lines
Oscar Salvador SUSE L3
Quoted 129 lines · Yang Shi
When
MPOL_MF_STRICTwas specified and an existing page was already on a node that does not follow the policy,mbind()should return-EIO. But commit6f4576e3687b("mempolicy: apply page table walker onqueue_pages_range()") broke the rule.And, commit
c8633798497c("mm: mempolicy: mbind andmigrate_pagessupport thp migration") didn't return the correct value for THPmbind()too.If
MPOL_MF_STRICTis set, ignorevma_migratable()to make sure it reachesqueue_pages_to_pte_range()orqueue_pages_pmd()to check if an existing page was already on a node that does not follow the policy. And, non-migratable vma may be used, return-EIOtoo ifMPOL_MF_MOVEorMPOL_MF_MOVE_ALLwas specified.Tested with https://github.com/metan-ucw/ltp/blob/master/testcases/kernel/syscalls/mbind/mbind02.c
Fixes 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()")Reported-by Cyril Hrubis <[email protected]>Cc Vlastimil Babka <[email protected]>Suggested-by Kirill A. Shutemov <[email protected]>Signed-off-by Yang Shi <[email protected]>Signed-off-by Oscar Salvador <[email protected]>--- mm/mempolicy.c | 40 +++++++++++++++++++++++++++++++++------- 1 file changed, 33 insertions(+), 7 deletions(-)mm/mempolicy.c +33
-7@@ -447,6 +447,13 @@ static inline bool queue_pages_required(struct page *page,return node_isset (nid, *qp->nmask) == !(flags &MPOL_MF_INVERT ); } +/* +* The queue_pages_pmd() may have three kind of return value. +* 1 - pages are placed on he right node or queued successfully. +* 0 - THP get split. +* -EIO - is migration entry or MPOL_MF_STRICT was specified and an existing +* page was already on a node that does not follow the policy. +*/ static int queue_pages_pmd (pmd_t *pmd,spinlock_t *ptl,unsigned long addr,unsigned long end,struct mm_walk *walk) {@@ -456,7 +463,7 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,unsigned long flags;if (unlikely (is_pmd_migration_entry (*pmd))) { - ret =1 ; + ret = -EIO ; goto unlock; } page =pmd_page (*pmd);@@ -473,8 +480,15 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, ret =1 ; flags = qp->flags;/* go to thp migration */ -if (flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) +if (flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) { +if (!vma_migratable (walk->vma)) { + ret = -EIO ; + goto unlock; + } +migrate_page_add (page, qp->pagelist, flags); + }else + ret = -EIO ; unlock:spin_unlock (ptl); out:@@ -499,8 +513,10 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, ptl =pmd_trans_huge_lock (pmd, vma);if (ptl) { ret =queue_pages_pmd (pmd, ptl, addr, end, walk); -if (ret) +if (ret >0 )return 0 ; +else if (ret <0 ) +return ret; }if (pmd_trans_unstable (pmd))@@ -521,11 +537,16 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,continue ;if (!queue_pages_required (page, qp))continue ; -migrate_page_add (page, qp->pagelist, flags); +if (flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) { +if (!vma_migratable (vma)) +break ; +migrate_page_add (page, qp->pagelist, flags); + }else +break ; }pte_unmap_unlock (pte -1 , ptl);cond_resched (); -return 0 ; +return addr != end ? -EIO :0 ; } static int queue_pages_hugetlb(pte_t *pte, unsigned long hmask,@@ -595,7 +616,12 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end,unsigned long endvma = vma->vm_end;unsigned long flags = qp->flags; -if (!vma_migratable (vma)) +/* +* Need check MPOL_MF_STRICT to return -EIO if possible +* regardless of vma_migratable +*/ +if (!vma_migratable (vma) && + !(flags &MPOL_MF_STRICT ))return 1 ;if (endvma > end)@@ -622,7 +648,7 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end, }/* queue pages from current vma */ -if (flags & (MPOL_MF_MOVE |MPOL_MF_MOVE_ALL )) +if (flags &MPOL_MF_VALID )return 0 ;return 1 ; }Signature · 1 line
1.8.3.1
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 7 lines · Sasha Levin
Hi,
[This is an automated email]
This commit has been processed because it contains a "Fixes:" tag, fixing commit:
6f4576e3687bmempolicy: apply page table walker onqueue_pages_range().
Quoted 6 lines
The bot has tested the following trees: v5.0.3, v4.19.30, v4.14.107, v4.9.164, v4.4.176.
v5.0.3: Build OK!
v4.19.30: Build OK!
v4.14.107: Failed to apply! Possible dependencies:
Unable to calculate
THP migration is not supported before 4.14, so part of the patch is not applicable, need fix it manually. I'm not quite familiar with the process of stable branch. Should I backport to these versions then send to you?
Thanks,
Yang
Quoted 27 lines
v4.9.164: Failed to apply! Possible dependencies:
88aaa2a1d732("mm: mempolicy: addqueue_pages_required()")c8633798497c("mm: mempolicy: mbind andmigrate_pagessupport thp migration")fd60775aea80("mm, thp: avoid unlikely branches forsplit_huge_pmd")v4.4.176: Failed to apply! Possible dependencies:
122afea9626a("mm, vmstats: new THP splitting event")1c290f642101("mm: sanitizepage->mappingfor tail pages")248db92da13f("migrate_pages: try to split pages on queuing")2a52bcbcc688("rmap: extendtry_to_unmap()to be usable bysplit_huge_page()")6742d293cbe0("mm: adjustFOLL_SPLITfor new refcounting")78ddc5347341("thp: renamesplit_huge_page_pmd()tosplit_huge_pmd()")854e9ed09ded("mm: supportmadvise(MADV_FREE)")
88aaa2a1d732("mm: mempolicy: addqueue_pages_required()")ad0bed24e98b("thp: drop all split_huge_page()-related code")c8633798497c("mm: mempolicy: mbind andmigrate_pagessupport thp migration")d281ee614518("rmap: add argument to charge compound page")eca56ff906bd("mm, shmem: add internal shmem resident memory accounting")fd60775aea80("mm, thp: avoid unlikely branches forsplit_huge_pmd")How should we proceed with this patch?
--
Thanks,
Sasha
Sasha Levin
maintainerQuoted 10 lines · Yang Shi
Quoted 7 lines · Sasha Levin
Hi,
[This is an automated email]
This commit has been processed because it contains a "Fixes:" tag, fixing commit:
6f4576e3687bmempolicy: apply page table walker onqueue_pages_range().
Quoted 8 lines
Quoted 6 lines
The bot has tested the following trees: v5.0.3, v4.19.30, v4.14.107, v4.9.164, v4.4.176.
v5.0.3: Build OK!
v4.19.30: Build OK!
v4.14.107: Failed to apply! Possible dependencies:
Unable to calculateTHP migration is not supported before 4.14, so part of the patch is not applicable, need fix it manually. I'm not quite familiar with the process of stable branch. Should I backport to these versions then send to you?
You can send them to [email protected] . Thanks!
--
Thanks,
Sasha
Greg KH
maintainerQuoted 12 lines · Yang Shi
Quoted 9 lines · Sasha Levin
Hi,
[This is an automated email]
This commit has been processed because it contains a "Fixes:" tag, fixing commit:
6f4576e3687bmempolicy: apply page table walker onqueue_pages_range().The bot has tested the following trees: v5.0.3, v4.19.30, v4.14.107, v4.9.164, v4.4.176.
Quoted 8 lines
Quoted 4 lines
v5.0.3: Build OK!
v4.19.30: Build OK!
v4.14.107: Failed to apply! Possible dependencies:
Unable to calculateTHP migration is not supported before 4.14, so part of the patch is not applicable, need fix it manually. I'm not quite familiar with the process of stable branch. Should I backport to these versions then send to you?
Yes, please send backported versions to [email protected] now, as I could not apply this patch to 4.14 or 4.9 or 4.4.
thanks,
greg k-h