Skip to contentSkip to conversation

mm/gup: introduce pin_user_pages_locked(), use it in frame_vector.c

PATCH New John Hubbard 1 participant 3 messages May 31, 2020

Subsystems (3)
Memory management MM core MM GUP
lore
Share & tools
lore

John Hubbard

author reviewer
#0 lore raw

v2 cover mm/gup: introduce pin_user_pages_locked(), use it in frame_vector.c

Hi,

Changes since v1:

* added an assert-and-return to the corresponding get_user_pages_locked() call, to keep out any externally set FOLL_PIN flag, thanks to Souptick Joarder's review for spotting that.

* Added Acked-by and Reviewed by tags from David and Pakaj

John Hubbard (2):
mm/gup: introduce pin_user_pages_locked()
mm/gup: frame_vector: convert get_user_pages() --> pin_user_pages()

 include/linux/mm.h |  2 ++
 mm/frame_vector.c  |  7 +++----
 mm/gup.c           | 36 ++++++++++++++++++++++++++++++++++++
 3 files changed, 41 insertions(+), 4 deletions(-)

base-commit: bdc48fa11e46f867ea4d75fa59ee87a7f48be144

Signature · 1 line
2.26.2
author reviewer
#1 lore raw

PATCH v2 1/2 mm/gup: introduce pin_user_pages_locked()

Introduce pin_user_pages_locked(), which is nearly identical to get_user_pages_locked() except that it sets FOLL_PIN and rejects FOLL_GET.

As with other pairs of get_user_pages*() and pin_user_pages() API calls, it's prudent to assert that FOLL_PIN is *not* set in the get_user_pages*() call, so add that as part of this.

Cc Souptick Joarder <[email protected]>
Reviewed-by David Hildenbrand <[email protected]>
Acked-by Pankaj Gupta <[email protected]>
Signed-off-by John Hubbard <[email protected]>
---
 include/linux/mm.h |  2 ++
 mm/gup.c           | 36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+)
include/linux/mm.h +2 -0
index b098939691ba..5e29ddcbb5fc 100644
@@ -1711,6 +1711,8 @@ long pin_user_pages(unsigned long start, unsigned long nr_pages,
 		    struct vm_area_struct **vmas);
 long get_user_pages_locked(unsigned long start, unsigned long nr_pages,
 		    unsigned int gup_flags, struct page **pages, int *locked);
+long pin_user_pages_locked(unsigned long start, unsigned long nr_pages,
+		    unsigned int gup_flags, struct page **pages, int *locked);
 long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
 		    struct page **pages, unsigned int gup_flags);
 long pin_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
mm/gup.c +36 -0
index 82ef0db1590a..06b891297a5c 100644
@@ -1993,6 +1993,12 @@ long get_user_pages_locked(unsigned long start, unsigned long nr_pages,
 	 */
 	if (WARN_ON_ONCE(gup_flags & FOLL_LONGTERM))
 		return -EINVAL;
+	/*
+	 * FOLL_PIN must only be set internally by the pin_user_pages*() APIs,
+	 * never directly by the caller, so enforce that:
+	 */
+	if (WARN_ON_ONCE(gup_flags & FOLL_PIN))
+		return -EINVAL;
 
 	return __get_user_pages_locked(current, current->mm, start, nr_pages,
 				       pages, NULL, locked,

@@ -2998,3 +3004,33 @@ long pin_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
 	return get_user_pages_unlocked(start, nr_pages, pages, gup_flags);
 }
 EXPORT_SYMBOL(pin_user_pages_unlocked);
+
+/*
+ * pin_user_pages_locked() is the FOLL_PIN variant of get_user_pages_locked().
+ * Behavior is the same, except that this one sets FOLL_PIN and rejects
+ * FOLL_GET.
+ */
+long pin_user_pages_locked(unsigned long start, unsigned long nr_pages,
+			   unsigned int gup_flags, struct page **pages,
+			   int *locked)
+{
+	/*
+	 * FIXME: Current FOLL_LONGTERM behavior is incompatible with
+	 * FAULT_FLAG_ALLOW_RETRY because of the FS DAX check requirement on
+	 * vmas.  As there are no users of this flag in this call we simply
+	 * disallow this option for now.
+	 */
+	if (WARN_ON_ONCE(gup_flags & FOLL_LONGTERM))
+		return -EINVAL;
+
+	/* FOLL_GET and FOLL_PIN are mutually exclusive. */
+	if (WARN_ON_ONCE(gup_flags & FOLL_GET))
+		return -EINVAL;
+
+	gup_flags |= FOLL_PIN;
+	return __get_user_pages_locked(current, current->mm, start, nr_pages,
+				       pages, NULL, locked,
+				       gup_flags | FOLL_TOUCH);
+}
+EXPORT_SYMBOL(pin_user_pages_locked);
+
Signature · 1 line
2.26.2
author reviewer
#2 lore raw

PATCH v2 2/2 mm/gup: frame_vector: convert get_user_pages() --> pin_user_pages()

In reply to John Hubbard

This code was using get_user_pages*(), and all of the callers so far were in a "Case 2" scenario (DMA/RDMA), using the categorization from [1]. That means that it's time to convert the get_user_pages*() + put_page() calls to pin_user_pages*() + unpin_user_pages() calls.

There is some helpful background in [2]: basically, this is a small part of fixing a long-standing disconnect between pinning pages, and file systems' use of those pages.

[1] Documentation/core-api/pin_user_pages.rst

[2] "Explicit pinning of user-space pages":

    https://lwn.net/Articles/807108/
Cc David Hildenbrand <[email protected]>
Signed-off-by John Hubbard <[email protected]>
---
 mm/frame_vector.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
mm/frame_vector.c +3 -4
index c431ca81dad5..4107dbca0056 100644
@@ -72,7 +72,7 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames,
 	if (!(vma->vm_flags & (VM_IO | VM_PFNMAP))) {
 		vec->got_ref = true;
 		vec->is_pfns = false;
-		ret = get_user_pages_locked(start, nr_frames,
+		ret = pin_user_pages_locked(start, nr_frames,
 			gup_flags, (struct page **)(vec->ptrs), &locked);
 		goto out;
 	}

@@ -122,7 +122,6 @@ EXPORT_SYMBOL(get_vaddr_frames);
  */
 void put_vaddr_frames(struct frame_vector *vec)
 {
-	int i;
 	struct page **pages;
 
 	if (!vec->got_ref)

@@ -135,8 +134,8 @@ void put_vaddr_frames(struct frame_vector *vec)
 	 */
 	if (WARN_ON(IS_ERR(pages)))
 		goto out;
-	for (i = 0; i < vec->nr_frames; i++)
-		put_page(pages[i]);
+
+	unpin_user_pages(pages, vec->nr_frames);
 	vec->got_ref = false;
 out:
 	vec->nr_frames = 0;
Signature · 1 line
2.26.2