Skip to contentSkip to conversation

mm-handle-swap-page-faults-under-per-vma-lock.patch removed from -mm tree

merged mm-stable Andrew Morton 1 participant 1 message Aug 24, 2023

lore
Share & tools
lore

Andrew Morton

author tree notice Applied
#0 lore raw

The quilt patch titled
Subject: mm: handle swap page faults under per-VMA lock
has been removed from the -mm tree. Its filename was
mm-handle-swap-page-faults-under-per-vma-lock.patch

This patch was dropped because it was merged into the mm-stable branch of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

------------------------------------------------------
From: Suren Baghdasaryan <[email protected]>
Subject: mm: handle swap page faults under per-VMA lock
Date: Fri, 30 Jun 2023 14:19:56 -0700

When page fault is handled under per-VMA lock protection, all swap page faults are retried with mmap_lock because folio_lock_or_retry has to drop and reacquire mmap_lock if folio could not be immediately locked. Follow the same pattern as mmap_lock to drop per-VMA lock when waiting for folio and retrying once folio is available.

With this obstacle removed, enable do_swap_page to operate under per-VMA lock protection. Drivers implementing ops->migrate_to_ram might still rely on mmap_lock, therefore we have to fall back to mmap_lock in that particular case.

Note that the only time do_swap_page calls synchronous swap_readpage is when SWP_SYNCHRONOUS_IO is set, which is only set for QUEUE_FLAG_SYNCHRONOUS devices: brd, zram and nvdimms (both btt and pmem). Therefore we don't sleep in this path, and there's no need to drop the mmap or per-VMA lock.

Cc Al Viro <[email protected]>
Cc Christian Brauner <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc David Howells <[email protected]>
Cc Davidlohr Bueso <[email protected]>
Cc Hillf Danton <[email protected]>
Cc "Huang, Ying" <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Jan Kara <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Josef Bacik <[email protected]>
Cc Laurent Dufour <[email protected]>
Cc Liam R. Howlett <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Punit Agrawal <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Yu Zhao <[email protected]>

---

 include/linux/mm.h |   13 +++++++++++++
 mm/filemap.c       |   17 ++++++++---------
 mm/memory.c        |   16 ++++++++++------
 3 files changed, 31 insertions(+), 15 deletions(-)
include/linux/mm.h renamed from include/linux/mm.h~mm-handle-swap-page-faults-under-per-vma-lock +13 -0
@@ -729,6 +729,14 @@ static inline void vma_mark_detached(str
 	vma->detached = detached;
 }
 
+static inline void release_fault_lock(struct vm_fault *vmf)
+{
+	if (vmf->flags & FAULT_FLAG_VMA_LOCK)
+		vma_end_read(vmf->vma);
+	else
+		mmap_read_unlock(vmf->vma->vm_mm);
+}
+
 struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm,
 					  unsigned long address);
 
@@ -749,6 +757,11 @@ static inline struct vm_area_struct *loc
 	return NULL;
 }
 
+static inline void release_fault_lock(struct vm_fault *vmf)
+{
+	mmap_read_unlock(vmf->vma->vm_mm);
+}
+
 #endif /* CONFIG_PER_VMA_LOCK */
 
 extern const struct vm_operations_struct vma_dummy_vm_ops;
mm/filemap.c renamed from mm/filemap.c~mm-handle-swap-page-faults-under-per-vma-lock +8 -9
@@ -1671,27 +1671,26 @@ static int __folio_lock_async(struct fol
  * Return values:
  * 0 - folio is locked.
  * non-zero - folio is not locked.
- *     mmap_lock has been released (mmap_read_unlock(), unless flags had both
- *     FAULT_FLAG_ALLOW_RETRY and FAULT_FLAG_RETRY_NOWAIT set, in
- *     which case mmap_lock is still held.
+ *     mmap_lock or per-VMA lock has been released (mmap_read_unlock() or
+ *     vma_end_read()), unless flags had both FAULT_FLAG_ALLOW_RETRY and
+ *     FAULT_FLAG_RETRY_NOWAIT set, in which case the lock is still held.
  *
  * If neither ALLOW_RETRY nor KILLABLE are set, will always return 0
- * with the folio locked and the mmap_lock unperturbed.
+ * with the folio locked and the mmap_lock/per-VMA lock is left unperturbed.
  */
 vm_fault_t __folio_lock_or_retry(struct folio *folio, struct vm_fault *vmf)
 {
-	struct mm_struct *mm = vmf->vma->vm_mm;
 	unsigned int flags = vmf->flags;
 
 	if (fault_flag_allow_retry_first(flags)) {
 		/*
-		 * CAUTION! In this case, mmap_lock is not released
-		 * even though return VM_FAULT_RETRY.
+		 * CAUTION! In this case, mmap_lock/per-VMA lock is not
+		 * released even though returning VM_FAULT_RETRY.
 		 */
 		if (flags & FAULT_FLAG_RETRY_NOWAIT)
 			return VM_FAULT_RETRY;
 
-		mmap_read_unlock(mm);
+		release_fault_lock(vmf);
 		if (flags & FAULT_FLAG_KILLABLE)
 			folio_wait_locked_killable(folio);
 		else
@@ -1703,7 +1702,7 @@ vm_fault_t __folio_lock_or_retry(struct
 
 		ret = __folio_lock_killable(folio);
 		if (ret) {
-			mmap_read_unlock(mm);
+			release_fault_lock(vmf);
 			return VM_FAULT_RETRY;
 		}
 	} else {
mm/memory.c renamed from mm/memory.c~mm-handle-swap-page-faults-under-per-vma-lock +10 -6
@@ -3746,12 +3746,6 @@ vm_fault_t do_swap_page(struct vm_fault
 	if (!pte_unmap_same(vmf))
 		goto out;
 
-	if (vmf->flags & FAULT_FLAG_VMA_LOCK) {
-		ret = VM_FAULT_RETRY;
-		vma_end_read(vma);
-		goto out;
-	}
-
 	entry = pte_to_swp_entry(vmf->orig_pte);
 	if (unlikely(non_swap_entry(entry))) {
 		if (is_migration_entry(entry)) {
@@ -3761,6 +3755,16 @@ vm_fault_t do_swap_page(struct vm_fault
 			vmf->page = pfn_swap_entry_to_page(entry);
 			ret = remove_device_exclusive_entry(vmf);
 		} else if (is_device_private_entry(entry)) {
+			if (vmf->flags & FAULT_FLAG_VMA_LOCK) {
+				/*
+				 * migrate_to_ram is not yet ready to operate
+				 * under VMA lock.
+				 */
+				vma_end_read(vma);
+				ret = VM_FAULT_RETRY;
+				goto out;
+			}
+
 			vmf->page = pfn_swap_entry_to_page(entry);
 			vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
 					vmf->address, &vmf->ptl);

_

Patches currently in -mm which might be from [email protected] are