Skip to contentSkip to conversation

mm: mempolicy: make mbind() return -EIO when MPOL_MF_STRICT is specified

PATCH Applied Yang Shi 8 participants 13 messages Mar 19, 2019 – Mar 31, 2019 1 message not in the archive

Subsystems (3)
lore
Share & tools
lore

Yang Shi

author
#0 lore raw

PATCH 1/1 mm: mempolicy: make mbind() return -EIO when MPOL_MF_STRICT is specified

When MPOL_MF_STRICT was specified and an existing page was already on a node that does not follow the policy, mbind() should return -EIO. But commit 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()") broke the rule.

And, commit c8633798497c ("mm: mempolicy: mbind and migrate_pages support thp migration") didn't return the correct value for THP mbind() too.

If MPOL_MF_STRICT is set, ignore vma_migratable() to make sure it reaches queue_pages_to_pte_range() or queue_pages_pmd() to check if an existing page was already on a node that does not follow the policy. And, non-migratable vma may be used, return -EIO too if MPOL_MF_MOVE or MPOL_MF_MOVE_ALL was specified.

Tested with https://github.com/metan-ucw/ltp/blob/master/testcases/kernel/syscalls/mbind/mbind02.c

Fixes 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()")
Cc Vlastimil Babka <[email protected]>
---
 mm/mempolicy.c | 40 +++++++++++++++++++++++++++++++++-------
 1 file changed, 33 insertions(+), 7 deletions(-)
mm/mempolicy.c +33 -7
index abe7a67..401c817 100644
@@ -447,6 +447,13 @@ static inline bool queue_pages_required(struct page *page,
 	return node_isset(nid, *qp->nmask) == !(flags & MPOL_MF_INVERT);
 }
 
+/*
+ * The queue_pages_pmd() may have three kind of return value.
+ * 1 - pages are placed on he right node or queued successfully.
+ * 0 - THP get split.
+ * -EIO - is migration entry or MPOL_MF_STRICT was specified and an existing
+ *        page was already on a node that does not follow the policy.
+ */
 static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,
 				unsigned long end, struct mm_walk *walk)
 {
@@ -456,7 +463,7 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,
 	unsigned long flags;
 
 	if (unlikely(is_pmd_migration_entry(*pmd))) {
-		ret = 1;
+		ret = -EIO;
 		goto unlock;
 	}
 	page = pmd_page(*pmd);
@@ -473,8 +480,15 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,
 	ret = 1;
 	flags = qp->flags;
 	/* go to thp migration */
-	if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL))
+	if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) {
+		if (!vma_migratable(walk->vma)) {
+			ret = -EIO;
+			goto unlock;
+		}
+
 		migrate_page_add(page, qp->pagelist, flags);
+	} else
+		ret = -EIO;
 unlock:
 	spin_unlock(ptl);
 out:
@@ -499,8 +513,10 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,
 	ptl = pmd_trans_huge_lock(pmd, vma);
 	if (ptl) {
 		ret = queue_pages_pmd(pmd, ptl, addr, end, walk);
-		if (ret)
+		if (ret > 0)
 			return 0;
+		else if (ret < 0)
+			return ret;
 	}
 
 	if (pmd_trans_unstable(pmd))
@@ -521,11 +537,16 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,
 			continue;
 		if (!queue_pages_required(page, qp))
 			continue;
-		migrate_page_add(page, qp->pagelist, flags);
+		if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) {
+			if (!vma_migratable(vma))
+				break;
+			migrate_page_add(page, qp->pagelist, flags);
+		} else
+			break;
 	}
 	pte_unmap_unlock(pte - 1, ptl);
 	cond_resched();
-	return 0;
+	return addr != end ? -EIO : 0;
 }
 
 static int queue_pages_hugetlb(pte_t *pte, unsigned long hmask,
@@ -595,7 +616,12 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end,
 	unsigned long endvma = vma->vm_end;
 	unsigned long flags = qp->flags;
 
-	if (!vma_migratable(vma))
+	/*
+	 * Need check MPOL_MF_STRICT to return -EIO if possible
+	 * regardless of vma_migratable
+	 */ 
+	if (!vma_migratable(vma) &&
+	    !(flags & MPOL_MF_STRICT))
 		return 1;
 
 	if (endvma > end)
@@ -622,7 +648,7 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end,
 	}
 
 	/* queue pages from current vma */
-	if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL))
+	if (flags & MPOL_MF_VALID)
 		return 0;
 	return 1;
 }
Signature · 1 line
1.8.3.1
Acked-by on v1 1/1
#1 lore raw
Quoted 17 lines · Yang Shi

When MPOL_MF_STRICT was specified and an existing page was already on a node that does not follow the policy, mbind() should return -EIO. But commit 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()") broke the rule.

And, commit c8633798497c ("mm: mempolicy: mbind and migrate_pages support thp migration") didn't return the correct value for THP mbind() too.

If MPOL_MF_STRICT is set, ignore vma_migratable() to make sure it reaches queue_pages_to_pte_range() or queue_pages_pmd() to check if an existing page was already on a node that does not follow the policy. And, non-migratable vma may be used, return -EIO too if MPOL_MF_MOVE or MPOL_MF_MOVE_ALL was specified.

Tested with https://github.com/metan-ucw/ltp/blob/master/testcases/kernel/syscalls/mbind/mbind02.c

Quoted 7 lines
Fixes 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()")
Cc Vlastimil Babka <[email protected]>

Thanks. I think this needs stable for 4.0+, can you confirm?

author
#2 lore raw
Quoted 26 lines · David Rientjes
Quoted 17 lines · Yang Shi

When MPOL_MF_STRICT was specified and an existing page was already on a node that does not follow the policy, mbind() should return -EIO. But commit 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()") broke the rule.

And, commit c8633798497c ("mm: mempolicy: mbind and migrate_pages support thp migration") didn't return the correct value for THP mbind() too.

If MPOL_MF_STRICT is set, ignore vma_migratable() to make sure it reaches queue_pages_to_pte_range() or queue_pages_pmd() to check if an existing page was already on a node that does not follow the policy. And, non-migratable vma may be used, return -EIO too if MPOL_MF_MOVE or MPOL_MF_MOVE_ALL was specified.

Tested with https://github.com/metan-ucw/ltp/blob/master/testcases/kernel/syscalls/mbind/mbind02.c

Quoted 7 lines
Fixes 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()")
Cc Vlastimil Babka <[email protected]>
Quoted 3 lines

Thanks. I think this needs stable for 4.0+, can you confirm?

Thanks. Yes, this needs stable for 4.0+.

Yang

#3 lore raw
In reply to Yang Shi
Quoted 34 lines · Yang Shi

When MPOL_MF_STRICT was specified and an existing page was already on a node that does not follow the policy, mbind() should return -EIO. But commit 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()") broke the rule.

And, commit c8633798497c ("mm: mempolicy: mbind and migrate_pages support thp migration") didn't return the correct value for THP mbind() too.

If MPOL_MF_STRICT is set, ignore vma_migratable() to make sure it reaches queue_pages_to_pte_range() or queue_pages_pmd() to check if an existing page was already on a node that does not follow the policy. And, non-migratable vma may be used, return -EIO too if MPOL_MF_MOVE or MPOL_MF_MOVE_ALL was specified.

Tested with https://github.com/metan-ucw/ltp/blob/master/testcases/kernel/syscalls/mbind/mbind02.c

Fixes 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()")
Cc Vlastimil Babka <[email protected]>
---
 mm/mempolicy.c | 40 +++++++++++++++++++++++++++++++++-------
 1 file changed, 33 insertions(+), 7 deletions(-)
mm/mempolicy.c +0 -0
index abe7a67..401c817 100644

mm/mempolicy.c static inline bool queue_pages_required(struct page *page, · line 452

Quoted 6 lines
mm/mempolicy.c +3 -0
@@ -447,6 +447,13 @@ static inline bool queue_pages_required(struct page *page,
        return node_isset(nid, *qp->nmask) == !(flags & MPOL_MF_INVERT);
 }
 
+/*
+ * The queue_pages_pmd() may have three kind of return value.
+ * 1 - pages are placed on he right node or queued successfully.

Minor typo -> s/he/the ?

Quoted 90 lines
mm/mempolicy.c +30 -8
+ * 0 - THP get split.
+ * -EIO - is migration entry or MPOL_MF_STRICT was specified and an existing
+ *        page was already on a node that does not follow the policy.
+ */
 static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,
                                unsigned long end, struct mm_walk *walk)
 {
@@ -456,7 +463,7 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,
        unsigned long flags;
 
        if (unlikely(is_pmd_migration_entry(*pmd))) {
-               ret = 1;
+               ret = -EIO;
                goto unlock;
        }
        page = pmd_page(*pmd);
@@ -473,8 +480,15 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,
        ret = 1;
        flags = qp->flags;
        /* go to thp migration */
-       if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL))
+       if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) {
+               if (!vma_migratable(walk->vma)) {
+                       ret = -EIO;
+                       goto unlock;
+               }
+
                migrate_page_add(page, qp->pagelist, flags);
+       } else
+               ret = -EIO;
 unlock:
        spin_unlock(ptl);
 out:
@@ -499,8 +513,10 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,
        ptl = pmd_trans_huge_lock(pmd, vma);
        if (ptl) {
                ret = queue_pages_pmd(pmd, ptl, addr, end, walk);
-               if (ret)
+               if (ret > 0)
                        return 0;
+               else if (ret < 0)
+                       return ret;
        }
 
        if (pmd_trans_unstable(pmd))
@@ -521,11 +537,16 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,
                        continue;
                if (!queue_pages_required(page, qp))
                        continue;
-               migrate_page_add(page, qp->pagelist, flags);
+               if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) {
+                       if (!vma_migratable(vma))
+                               break;
+                       migrate_page_add(page, qp->pagelist, flags);
+               } else
+                       break;
        }
        pte_unmap_unlock(pte - 1, ptl);
        cond_resched();
-       return 0;
+       return addr != end ? -EIO : 0;
 }
 
 static int queue_pages_hugetlb(pte_t *pte, unsigned long hmask,
@@ -595,7 +616,12 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end,
        unsigned long endvma = vma->vm_end;
        unsigned long flags = qp->flags;
 
-       if (!vma_migratable(vma))
+       /*
+        * Need check MPOL_MF_STRICT to return -EIO if possible
+        * regardless of vma_migratable
+        */
+       if (!vma_migratable(vma) &&
+           !(flags & MPOL_MF_STRICT))
                return 1;
 
        if (endvma > end)
@@ -622,7 +648,7 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end,
        }
 
        /* queue pages from current vma */
-       if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL))
+       if (flags & MPOL_MF_VALID)
                return 0;
        return 1;
 }
--

1.8.3.1

maintainer
#4 lore raw
Quoted 11 lines · On Wed, 20 Mar 2019 11:23:03 +0530 Souptick Joarder <[email protected]>…

mm/mempolicy.c static inline bool queue_pages_required(struct page *page, · line 452

Quoted 9 lines
mm/mempolicy.c +3 -0
@@ -447,6 +447,13 @@ static inline bool queue_pages_required(struct page *page,
        return node_isset(nid, *qp->nmask) == !(flags & MPOL_MF_INVERT);
 }
 
+/*
+ * The queue_pages_pmd() may have three kind of return value.
+ * 1 - pages are placed on he right node or queued successfully.

Minor typo -> s/he/the ?

Yes, that comment needs some help. This?

mm/mempolicy.c renamed from mm/mempolicy.c~mm-mempolicy-make-mbind-return-eio-when-mpol_mf_strict-is-specified-fix +3 -3
@@ -429,9 +429,9 @@ static inline bool queue_pages_required(
 }
 
 /*
- * The queue_pages_pmd() may have three kind of return value.
- * 1 - pages are placed on he right node or queued successfully.
- * 0 - THP get split.
+ * queue_pages_pmd() has three possible return values:
+ * 1 - pages are placed on the right node or queued successfully.
+ * 0 - THP was split.
  * -EIO - is migration entry or MPOL_MF_STRICT was specified and an existing
  *        page was already on a node that does not follow the policy.
  */

_

author
#5 lore raw
Quoted 19 lines · Andrew Morton
Quoted 10 lines · On Wed, 20 Mar 2019 11:23:03 +0530 Souptick Joarder <[email protected]>…

mm/mempolicy.c static inline bool queue_pages_required(struct page *page, · line 452

Quoted 9 lines
mm/mempolicy.c +3 -0
@@ -447,6 +447,13 @@ static inline bool queue_pages_required(struct page *page,
         return node_isset(nid, *qp->nmask) == !(flags & MPOL_MF_INVERT);
  }
 
+/*
+ * The queue_pages_pmd() may have three kind of return value.
+ * 1 - pages are placed on he right node or queued successfully.

Minor typo -> s/he/the ?

Yes, that comment needs some help. This?

mm/mempolicy.c renamed from mm/mempolicy.c~mm-mempolicy-make-mbind-return-eio-when-mpol_mf_strict-is-specified-fix +0 -0
@@ -429,9 +429,9 @@ static inline bool queue_pages_required(
  }
 

mm/mempolicy.c static inline bool queue_pages_required( · line 437

Quoted 10 lines
mm/mempolicy.c renamed from mm/mempolicy.c~mm-mempolicy-make-mbind-return-eio-when-mpol_mf_strict-is-specified-fix +3 -3
  /*
- * The queue_pages_pmd() may have three kind of return value.
- * 1 - pages are placed on he right node or queued successfully.
- * 0 - THP get split.
+ * queue_pages_pmd() has three possible return values:
+ * 1 - pages are placed on the right node or queued successfully.
+ * 0 - THP was split.
   * -EIO - is migration entry or MPOL_MF_STRICT was specified and an existing
   *        page was already on a node that does not follow the policy.
   */

It looks good to me. Thanks, Andrew.

Yang

Quoted 1 line

_

#6 lore raw
In reply to Yang Shi
Quoted 7 lines · Yang Shi
Fixes 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()")
Cc Vlastimil Babka <[email protected]>

Hi Yang, thanks for the patch.

Some observations below.

Quoted 6 lines
(quoted diff) +0 -0
 	}
 	page = pmd_page(*pmd);
@@ -473,8 +480,15 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,
 	ret = 1;
 	flags = qp->flags;
 	/* go to thp migration */

Review static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, · line 491

Quoted 10 lines
(quoted diff) +8 -1
-	if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL))
+	if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) {
+		if (!vma_migratable(walk->vma)) {
+			ret = -EIO;
+			goto unlock;
+		}
+
 		migrate_page_add(page, qp->pagelist, flags);
+	} else
+		ret = -EIO;
	if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) ||
       	        !vma_migratable(walk->vma)) {
               	ret = -EIO;
                goto unlock;
        }

	migrate_page_add(page, qp->pagelist, flags); 
unlock:
        spin_unlock(ptl);
out:
        return ret;

seems more clean to me?

Review static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 519

Quoted 12 lines
(quoted diff) +3 -1
 unlock:
 	spin_unlock(ptl);
 out:
@@ -499,8 +513,10 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,
 	ptl = pmd_trans_huge_lock(pmd, vma);
 	if (ptl) {
 		ret = queue_pages_pmd(pmd, ptl, addr, end, walk);
-		if (ret)
+		if (ret > 0)
 			return 0;
+		else if (ret < 0)
+			return ret;

I would go with the following, but that's a matter of taste I guess.

if (ret < 0)

	return ret;
else
	return 0;
Quoted 4 lines
(quoted diff) +0 -0
 	}
 
 	if (pmd_trans_unstable(pmd))

Review static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 545

Quoted 10 lines
(quoted diff) +6 -1
@@ -521,11 +537,16 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,
 			continue;
 		if (!queue_pages_required(page, qp))
 			continue;
-		migrate_page_add(page, qp->pagelist, flags);
+		if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) {
+			if (!vma_migratable(vma))
+				break;
+			migrate_page_add(page, qp->pagelist, flags);
+		} else
+			break;

I might be missing something, but AFAICS neither vma nor flags is going to change while we are in queue_pages_pte_range(), so, could not we move the check just above the loop?
In that way, 1) we only perform the check once and 2) if we enter the loop we know that we are going to do some work, so, something like:

index af171ccb56a2..7c0e44389826 100644

mm/mempolicy.c +3 -0
@@ -487,6 +487,9 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,
        if (pmd_trans_unstable(pmd))
                return 0;
 
+       if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || !vma_migratable(vma))
+               return -EIO;
+
        pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl);
        for (; addr != end; pte++, addr += PAGE_SIZE) {
                if (!pte_present(*pte))

Review static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 549

Quoted 5 lines
(quoted diff) +1 -1
 	}
 	pte_unmap_unlock(pte - 1, ptl);
 	cond_resched();
-	return 0;
+	return addr != end ? -EIO : 0;

If we can do the above, we can leave the return value as it was.

Signature · 2 lines
Oscar Salvador
SUSE L3
author
#7 lore raw
Quoted 40 lines · Oscar Salvador
Quoted 7 lines · Yang Shi
Fixes 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()")
Cc Vlastimil Babka <[email protected]>

Hi Yang, thanks for the patch.

Some observations below.

Quoted 6 lines
  	}
  	page = pmd_page(*pmd);
(quoted diff) +0 -0
@@ -473,8 +480,15 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,
  	ret = 1;
  	flags = qp->flags;
  	/* go to thp migration */

Review static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, · line 491

Quoted 10 lines
(quoted diff) +8 -1
-	if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL))
+	if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) {
+		if (!vma_migratable(walk->vma)) {
+			ret = -EIO;
+			goto unlock;
+		}
+
  		migrate_page_add(page, qp->pagelist, flags);
+	} else
+		ret = -EIO;
	if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) ||
        	        !vma_migratable(walk->vma)) {
                	ret = -EIO;
                 goto unlock;
         }

	migrate_page_add(page, qp->pagelist, flags);
unlock:
         spin_unlock(ptl);
out:
         return ret;
Quoted 1 line

seems more clean to me?

Yes, it sounds so.

Quoted 16 lines
Quoted 6 lines
(quoted diff) +0 -0
  unlock:
  	spin_unlock(ptl);
  out:

Review static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 519

Quoted 8 lines
(quoted diff) +3 -1
@@ -499,8 +513,10 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,
  	ptl = pmd_trans_huge_lock(pmd, vma);
  	if (ptl) {
  		ret = queue_pages_pmd(pmd, ptl, addr, end, walk);
-		if (ret)
+		if (ret > 0)
  			return 0;
+		else if (ret < 0)
+			return ret;

I would go with the following, but that's a matter of taste I guess.

Quoted 4 lines

if (ret < 0)

	return ret;
else
	return 0;

No, this is not correct. queue_pages_pmd() may return 0, which means THP gets split. If it returns 0 the code should just fall through instead of returning.

Quoted 25 lines
Quoted 5 lines
(quoted diff) +0 -0
  	}
 
  	if (pmd_trans_unstable(pmd))

Review static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 545

Quoted 10 lines
(quoted diff) +6 -1
@@ -521,11 +537,16 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,
  			continue;
  		if (!queue_pages_required(page, qp))
  			continue;
-		migrate_page_add(page, qp->pagelist, flags);
+		if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) {
+			if (!vma_migratable(vma))
+				break;
+			migrate_page_add(page, qp->pagelist, flags);
+		} else
+			break;

I might be missing something, but AFAICS neither vma nor flags is going to change while we are in queue_pages_pte_range(), so, could not we move the check just above the loop?
In that way, 1) we only perform the check once and 2) if we enter the loop we know that we are going to do some work, so, something like:

index af171ccb56a2..7c0e44389826 100644

mm/mempolicy.c static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 491

Quoted 5 lines
mm/mempolicy.c +2 -0
@@ -487,6 +487,9 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,
         if (pmd_trans_unstable(pmd))
                 return 0;
 
+       if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || !vma_migratable(vma))
+               return -EIO;

It sounds not correct to me. We need check if there is existing page on the node which is not allowed by the policy. This is what queue_pages_required() does.

Thanks,
Yang

Quoted 13 lines
mm/mempolicy.c +1 -0
+
         pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl);
         for (; addr != end; pte++, addr += PAGE_SIZE) {
                 if (!pte_present(*pte))

Review static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, · line 549

Quoted 7 lines
(quoted diff) +1 -1
  	}
  	pte_unmap_unlock(pte - 1, ptl);
  	cond_resched();
-	return 0;
+	return addr != end ? -EIO : 0;

If we can do the above, we can leave the return value as it was.

Reviewed-by
#8 lore raw
Quoted 5 lines · Yang Shi

No, this is not correct. queue_pages_pmd() may return 0, which means THP
gets split. If it returns 0 the code should just fall through instead of
returning.

Right, I overlooked that.

Quoted 4 lines

It sounds not correct to me. We need check if there is existing page on
the node which is not allowed by the policy. This is what queue_pages_required() does.

Bleh, I guess it was too early in the morning.
That is the whole point of it actually, so that was quite wrong.

Sorry for trying to mislead you ;-)

Signature · 2 lines
Oscar Salvador
SUSE L3
Acked-by on v1 1/1
#9 lore raw
In reply to Yang Shi
Quoted 129 lines · Yang Shi

When MPOL_MF_STRICT was specified and an existing page was already on a node that does not follow the policy, mbind() should return -EIO. But commit 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()") broke the rule.

And, commit c8633798497c ("mm: mempolicy: mbind and migrate_pages support thp migration") didn't return the correct value for THP mbind() too.

If MPOL_MF_STRICT is set, ignore vma_migratable() to make sure it reaches queue_pages_to_pte_range() or queue_pages_pmd() to check if an existing page was already on a node that does not follow the policy. And, non-migratable vma may be used, return -EIO too if MPOL_MF_MOVE or MPOL_MF_MOVE_ALL was specified.

Tested with https://github.com/metan-ucw/ltp/blob/master/testcases/kernel/syscalls/mbind/mbind02.c

Fixes 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()")
Cc Vlastimil Babka <[email protected]>
---
 mm/mempolicy.c | 40 +++++++++++++++++++++++++++++++++-------
 1 file changed, 33 insertions(+), 7 deletions(-)
mm/mempolicy.c +33 -7
index abe7a67..401c817 100644
@@ -447,6 +447,13 @@ static inline bool queue_pages_required(struct page *page,
 	return node_isset(nid, *qp->nmask) == !(flags & MPOL_MF_INVERT);
 }
 
+/*
+ * The queue_pages_pmd() may have three kind of return value.
+ * 1 - pages are placed on he right node or queued successfully.
+ * 0 - THP get split.
+ * -EIO - is migration entry or MPOL_MF_STRICT was specified and an existing
+ *        page was already on a node that does not follow the policy.
+ */
 static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,
 				unsigned long end, struct mm_walk *walk)
 {
@@ -456,7 +463,7 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,
 	unsigned long flags;
 
 	if (unlikely(is_pmd_migration_entry(*pmd))) {
-		ret = 1;
+		ret = -EIO;
 		goto unlock;
 	}
 	page = pmd_page(*pmd);
@@ -473,8 +480,15 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr,
 	ret = 1;
 	flags = qp->flags;
 	/* go to thp migration */
-	if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL))
+	if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) {
+		if (!vma_migratable(walk->vma)) {
+			ret = -EIO;
+			goto unlock;
+		}
+
 		migrate_page_add(page, qp->pagelist, flags);
+	} else
+		ret = -EIO;
 unlock:
 	spin_unlock(ptl);
 out:
@@ -499,8 +513,10 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,
 	ptl = pmd_trans_huge_lock(pmd, vma);
 	if (ptl) {
 		ret = queue_pages_pmd(pmd, ptl, addr, end, walk);
-		if (ret)
+		if (ret > 0)
 			return 0;
+		else if (ret < 0)
+			return ret;
 	}
 
 	if (pmd_trans_unstable(pmd))
@@ -521,11 +537,16 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr,
 			continue;
 		if (!queue_pages_required(page, qp))
 			continue;
-		migrate_page_add(page, qp->pagelist, flags);
+		if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) {
+			if (!vma_migratable(vma))
+				break;
+			migrate_page_add(page, qp->pagelist, flags);
+		} else
+			break;
 	}
 	pte_unmap_unlock(pte - 1, ptl);
 	cond_resched();
-	return 0;
+	return addr != end ? -EIO : 0;
 }
 
 static int queue_pages_hugetlb(pte_t *pte, unsigned long hmask,
@@ -595,7 +616,12 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end,
 	unsigned long endvma = vma->vm_end;
 	unsigned long flags = qp->flags;
 
-	if (!vma_migratable(vma))
+	/*
+	 * Need check MPOL_MF_STRICT to return -EIO if possible
+	 * regardless of vma_migratable
+	 */ 
+	if (!vma_migratable(vma) &&
+	    !(flags & MPOL_MF_STRICT))
 		return 1;
 
 	if (endvma > end)
@@ -622,7 +648,7 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end,
 	}
 
 	/* queue pages from current vma */
-	if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL))
+	if (flags & MPOL_MF_VALID)
 		return 0;
 	return 1;
 }
Signature · 1 line
1.8.3.1

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#11 lore raw
Quoted 7 lines · Sasha Levin

Hi,

[This is an automated email]

This commit has been processed because it contains a "Fixes:" tag, fixing commit: 6f4576e3687b mempolicy: apply page table walker on queue_pages_range().

Quoted 6 lines

The bot has tested the following trees: v5.0.3, v4.19.30, v4.14.107, v4.9.164, v4.4.176.

v5.0.3: Build OK!
v4.19.30: Build OK!
v4.14.107: Failed to apply! Possible dependencies:
Unable to calculate

THP migration is not supported before 4.14, so part of the patch is not applicable, need fix it manually. I'm not quite familiar with the process of stable branch. Should I backport to these versions then send to you?

Thanks,
Yang

Quoted 27 lines

v4.9.164: Failed to apply! Possible dependencies:
88aaa2a1d732 ("mm: mempolicy: add queue_pages_required()") c8633798497c ("mm: mempolicy: mbind and migrate_pages support thp migration") fd60775aea80 ("mm, thp: avoid unlikely branches for split_huge_pmd")

v4.4.176: Failed to apply! Possible dependencies:
122afea9626a ("mm, vmstats: new THP splitting event") 1c290f642101 ("mm: sanitize page->mapping for tail pages") 248db92da13f ("migrate_pages: try to split pages on queuing") 2a52bcbcc688 ("rmap: extend try_to_unmap() to be usable by split_huge_page()") 6742d293cbe0 ("mm: adjust FOLL_SPLIT for new refcounting") 78ddc5347341 ("thp: rename split_huge_page_pmd() to split_huge_pmd()") 854e9ed09ded ("mm: support madvise(MADV_FREE)")
88aaa2a1d732 ("mm: mempolicy: add queue_pages_required()") ad0bed24e98b ("thp: drop all split_huge_page()-related code") c8633798497c ("mm: mempolicy: mbind and migrate_pages support thp migration") d281ee614518 ("rmap: add argument to charge compound page") eca56ff906bd ("mm, shmem: add internal shmem resident memory accounting") fd60775aea80 ("mm, thp: avoid unlikely branches for split_huge_pmd")

How should we proceed with this patch?

--
Thanks,
Sasha

maintainer
#12 lore raw
Quoted 10 lines · Yang Shi
Quoted 7 lines · Sasha Levin

Hi,

[This is an automated email]

This commit has been processed because it contains a "Fixes:" tag, fixing commit: 6f4576e3687b mempolicy: apply page table walker on queue_pages_range().

Quoted 8 lines
Quoted 6 lines

The bot has tested the following trees: v5.0.3, v4.19.30, v4.14.107, v4.9.164, v4.4.176.

v5.0.3: Build OK!
v4.19.30: Build OK!
v4.14.107: Failed to apply! Possible dependencies:
Unable to calculate

THP migration is not supported before 4.14, so part of the patch is not applicable, need fix it manually. I'm not quite familiar with the process of stable branch. Should I backport to these versions then send to you?

You can send them to [email protected] . Thanks!

--
Thanks,
Sasha

maintainer
#13 lore raw
In reply to Yang Shi
Quoted 12 lines · Yang Shi
Quoted 9 lines · Sasha Levin

Hi,

[This is an automated email]

This commit has been processed because it contains a "Fixes:" tag, fixing commit: 6f4576e3687b mempolicy: apply page table walker on queue_pages_range().

The bot has tested the following trees: v5.0.3, v4.19.30, v4.14.107, v4.9.164, v4.4.176.

Quoted 8 lines
Quoted 4 lines

v5.0.3: Build OK!
v4.19.30: Build OK!
v4.14.107: Failed to apply! Possible dependencies:
Unable to calculate

THP migration is not supported before 4.14, so part of the patch is not applicable, need fix it manually. I'm not quite familiar with the process of stable branch. Should I backport to these versions then send to you?

Yes, please send backported versions to [email protected] now, as I could not apply this patch to 4.14 or 4.9 or 4.4.

thanks,

greg k-h