Skip to contentSkip to conversation

mm/gup: folio_split_user_page_pin

PATCH In review Steve Sistare 3 participants 8 messages Sep 24, 2024 – Oct 4, 2024

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 mm/gup: folio_split_user_page_pin

Export a function that repins a high-order folio at small-page granularity. This allows any range of small pages within the folio to be unpinned later. For example, pages pinned via memfd_pin_folios and modified by folio_split_user_page_pin could be unpinned via unpin_user_page(s).

---
In V2 this has been renamed from repin_folio_unhugely, but is otherwise unchanged from V1.
---

---
 include/linux/mm.h |  1 +
 mm/gup.c           | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+)
include/linux/mm.h +1 -0
index 13bff7c..b0b572d 100644
@@ -2521,6 +2521,7 @@ long pin_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
 long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
 		      struct folio **folios, unsigned int max_folios,
 		      pgoff_t *offset);
+void folio_split_user_page_pin(struct folio *folio, unsigned long npages);
 
 int get_user_pages_fast(unsigned long start, int nr_pages,
 			unsigned int gup_flags, struct page **pages);
mm/gup.c +20 -0
index fcd602b..94ee79dd 100644
@@ -3733,3 +3733,23 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(memfd_pin_folios);
+
+/**
+ * folio_split_user_page_pin() - split the pin on a high order folio
+ * @folio: the folio to split
+ * @npages: The new number of pages the folio pin reference should hold
+ *
+ * Given a high order folio that is already pinned, adjust the reference
+ * count to allow unpin_user_page_range() and related to be called on a
+ * the folio. npages is the number of pages that will be passed to a
+ * future unpin_user_page_range().
+ */
+void folio_split_user_page_pin(struct folio *folio, unsigned long npages)
+{
+	if (!folio_test_large(folio) || is_huge_zero_folio(folio) ||
+	    npages == 1)
+		return;
+	atomic_add(npages - 1, &folio->_refcount);
+	atomic_add(npages - 1, &folio->_pincount);
+}
+EXPORT_SYMBOL_GPL(folio_split_user_page_pin);
Signature · 1 line
1.8.3.1
reviewer Reviewed-by on v2 1/1
#1 lore raw
Quoted 11 lines · Steve Sistare

Export a function that repins a high-order folio at small-page granularity. This allows any range of small pages within the folio to be unpinned later. For example, pages pinned via memfd_pin_folios and modified by folio_split_user_page_pin could be unpinned via unpin_user_page(s).

---
In V2 this has been renamed from repin_folio_unhugely, but is otherwise unchanged from V1.

This needs to stay in your series since I will need to take it all together..

But it looks Ok to me

Jason

#2 lore raw
In reply to Steve Sistare
Quoted 34 lines · Steve Sistare

Export a function that repins a high-order folio at small-page granularity. This allows any range of small pages within the folio to be unpinned later. For example, pages pinned via memfd_pin_folios and modified by folio_split_user_page_pin could be unpinned via unpin_user_page(s).

---
In V2 this has been renamed from repin_folio_unhugely, but is otherwise unchanged from V1.
---

---
  include/linux/mm.h |  1 +
  mm/gup.c           | 20 ++++++++++++++++++++
  2 files changed, 21 insertions(+)
include/linux/mm.h +1 -0
index 13bff7c..b0b572d 100644
@@ -2521,6 +2521,7 @@ long pin_user_pages_unlocked(unsigned long start, unsigned long nr_pages,
  long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
  		      struct folio **folios, unsigned int max_folios,
  		      pgoff_t *offset);
+void folio_split_user_page_pin(struct folio *folio, unsigned long npages);
 
  int get_user_pages_fast(unsigned long start, int nr_pages,
  			unsigned int gup_flags, struct page **pages);
mm/gup.c +0 -0
index fcd602b..94ee79dd 100644

mm/gup.c long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end, · line 3738

Quoted 6 lines
mm/gup.c +3 -0
@@ -3733,3 +3733,23 @@ long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end,
  	return ret;
  }
  EXPORT_SYMBOL_GPL(memfd_pin_folios);
+
+/**
+ * folio_split_user_page_pin() - split the pin on a high order folio

There really is no such concept of splitting pins :/

mm/gup.c long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end, · line 3739

Quoted 1 line
mm/gup.c +1 -0
+ * @folio: the folio to split

"folio to split": Highly misleading :)

mm/gup.c long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end, · line 3743

Quoted 4 lines
mm/gup.c +4 -0
+ * @npages: The new number of pages the folio pin reference should hold
+ *
+ * Given a high order folio that is already pinned, adjust the reference
+ * count to allow unpin_user_page_range() and related to be called on a

unpin_user_page_range() does not exist, at least upstream. Did you mean unpin_user_page_range_dirty_lock() ?

mm/gup.c long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end, · line 3749

Quoted 6 lines
mm/gup.c +6 -0
+ * the folio. npages is the number of pages that will be passed to a
+ * future unpin_user_page_range().
+ */
+void folio_split_user_page_pin(struct folio *folio, unsigned long npages)
+{
+	if (!folio_test_large(folio) || is_huge_zero_folio(folio) ||

is_huge_zero_folio() is still likely wrong.

Just follow the flow in unpin_user_page_range_dirty_lock() -> gup_put_folio().

Please point to me where in unpin_user_page_range_dirty_lock() -> gup_put_folio() there is_a huge_zero_folio() special-casing is that would skip adjusting the refcount and the pincount, so it would be balanced?

mm/gup.c long memfd_pin_folios(struct file *memfd, loff_t start, loff_t end, · line 3755

Quoted 6 lines
mm/gup.c +6 -0
+	    npages == 1)
+		return;
+	atomic_add(npages - 1, &folio->_refcount);
+	atomic_add(npages - 1, &folio->_pincount);
+}
+EXPORT_SYMBOL_GPL(folio_split_user_page_pin);

I can understand why we want to add more pins to a folio. I don't like this interface.

I would suggest a more generic interface:

/**
* folio_try_add_pins() - add pins to an already-pinned folio
* @folio: the folio to add more pins to
*
* Try to add more pins to an already-pinned folio. The semantics
* of the pin (e.g., FOLL_WRITE) follow any existing pin and cannot
* be changed.
*
* This function is helpful when having obtained a pin on a large folio
* using memfd_pin_folios(), but wanting to logically unpin parts
* (e.g., individual pages) of the folio later, for example, using
* unpin_user_page_range_dirty_lock().
*
* This is not the right interface to initially pin a folio. */
int folio_try_add_pins(struct folio *folio, unsigned int pins)

{
	VM_WARN_ON_ONCE(!folio_maybe_dma_pinned(folio));

	return try_grab_folio(folio, pins, FOLL_PIN);
}

We might want to consider adding even better overflow checks in try_grab_folio(), but that's a different discussion.

The shared zeropage will be taken care of automatically, and the huge zero folio does currently not need any special care ...

Signature · 3 lines
Cheers,

David / dhildenb
#3 lore raw
Quoted 16 lines · David Hildenbrand

/**
* folio_try_add_pins() - add pins to an already-pinned folio
* @folio: the folio to add more pins to
*
* Try to add more pins to an already-pinned folio. The semantics
* of the pin (e.g., FOLL_WRITE) follow any existing pin and cannot
* be changed.
*
* This function is helpful when having obtained a pin on a large folio
* using memfd_pin_folios(), but wanting to logically unpin parts
* (e.g., individual pages) of the folio later, for example, using
* unpin_user_page_range_dirty_lock().
*
* This is not the right interface to initially pin a folio. */
int folio_try_add_pins(struct folio *folio, unsigned int pins)

Quoted 5 lines
{
	VM_WARN_ON_ONCE(!folio_maybe_dma_pinned(folio));

	return try_grab_folio(folio, pins, FOLL_PIN);
}

That looks pretty good to me too

Thanks,
Jason

#4 lore raw
Quoted 20 lines · Jason Gunthorpe
Quoted 19 lines · David Hildenbrand

/**
* folio_try_add_pins() - add pins to an already-pinned folio
* @folio: the folio to add more pins to
*
* Try to add more pins to an already-pinned folio. The semantics
* of the pin (e.g., FOLL_WRITE) follow any existing pin and cannot
* be changed.
*
* This function is helpful when having obtained a pin on a large folio
* using memfd_pin_folios(), but wanting to logically unpin parts
* (e.g., individual pages) of the folio later, for example, using
* unpin_user_page_range_dirty_lock().
*
* This is not the right interface to initially pin a folio. */
int folio_try_add_pins(struct folio *folio, unsigned int pins)

{
	VM_WARN_ON_ONCE(!folio_maybe_dma_pinned(folio));
Quoted 4 lines
Quoted 2 lines
	return try_grab_folio(folio, pins, FOLL_PIN);
}

That looks pretty good to me too

Looks good and passes my tests, I will add this version in V3 of the patch series.

Are you sure you want "try" in the name folio_try_add_pins? Usually try means that any failure is transient and a future call may succeed, but the failures in try_grab_folio look permanent to me (suggesting that is also a misnomer, but at least it is not exported).

- Steve

#5 lore raw
Quoted 24 lines · Steven Sistare
Quoted 23 lines · Jason Gunthorpe
Quoted 21 lines · David Hildenbrand

/**
* folio_try_add_pins() - add pins to an already-pinned folio
* @folio: the folio to add more pins to
*
* Try to add more pins to an already-pinned folio. The semantics
* of the pin (e.g., FOLL_WRITE) follow any existing pin and cannot
* be changed.
*
* This function is helpful when having obtained a pin on a large folio
* using memfd_pin_folios(), but wanting to logically unpin parts
* (e.g., individual pages) of the folio later, for example, using
* unpin_user_page_range_dirty_lock().
*
* This is not the right interface to initially pin a folio. */
int folio_try_add_pins(struct folio *folio, unsigned int pins)

{
	VM_WARN_ON_ONCE(!folio_maybe_dma_pinned(folio));

	return try_grab_folio(folio, pins, FOLL_PIN);
}
Quoted 6 lines
Quoted 1 line

That looks pretty good to me too

Looks good and passes my tests, I will add this version in V3 of the patch series.

Are you sure you want "try" in the name folio_try_add_pins? Usually try means that any failure is transient and a future call may succeed

And now I took another look at the codebase and we already do have folio_add_pin() that adds a single pin, but continues on overflows (not sure I like that, but at least it can be caught and debugged).

So yes, we could simply turn folio_add_pin() into a wrapper around a folio_add_pins() that adds multiple pins.

Looking at folio_add_pin() vs. try_grab_folio() I am not sure if the open-coding the logic in folio_add_pin() got the NR_FOLL_PIN_ACQUIRED accounting correct.

Signature · 3 lines
Cheers,

David / dhildenb
#6 lore raw
Quoted 29 lines · David Hildenbrand
Quoted 28 lines · Steven Sistare
Quoted 20 lines · Jason Gunthorpe
Quoted 19 lines · David Hildenbrand

/**
   * folio_try_add_pins() - add pins to an already-pinned folio
   * @folio: the folio to add more pins to
   *
   * Try to add more pins to an already-pinned folio. The semantics
   * of the pin (e.g., FOLL_WRITE) follow any existing pin and cannot
   * be changed.
   *
   * This function is helpful when having obtained a pin on a large folio
   * using memfd_pin_folios(), but wanting to logically unpin parts
   * (e.g., individual pages) of the folio later, for example, using
   * unpin_user_page_range_dirty_lock().
   *
   * This is not the right interface to initially pin a folio. */
int folio_try_add_pins(struct folio *folio, unsigned int pins) {
    VM_WARN_ON_ONCE(!folio_maybe_dma_pinned(folio));

Quoted 4 lines
Quoted 2 lines

    return try_grab_folio(folio, pins, FOLL_PIN);
}

That looks pretty good to me too

Looks good and passes my tests, I will add this version in V3 of the patch series.

Quoted 7 lines
Quoted 2 lines

Are you sure you want "try" in the name folio_try_add_pins?  Usually try means that any failure is transient and a future call may succeed

And now I took another look at the codebase and we already do have folio_add_pin() that adds a single pin, but continues on overflows (not sure I like that, but at least it can be caught and debugged).

So yes, we could simply turn folio_add_pin() into a wrapper around a folio_add_pins() that adds multiple pins.
Looking at folio_add_pin() vs. try_grab_folio() I am not sure if the open-coding the logic in folio_add_pin() got the NR_FOLL_PIN_ACQUIRED accounting correct.

To be clear, I am only suggesting that I use your folio_try_add_pins implementation, but rename it to folio_add_pins. And not touch the existing folio_add_pin.

I am ready to send V3 of the iommu_ioas_map_file series, and I would like to add this patch back to the series as Jason requested.

- Steve

#7 lore raw
Quoted 33 lines · Steven Sistare
Quoted 32 lines · David Hildenbrand
Quoted 30 lines · Steven Sistare
Quoted 20 lines · Jason Gunthorpe
Quoted 19 lines · David Hildenbrand

/**
   * folio_try_add_pins() - add pins to an already-pinned folio
   * @folio: the folio to add more pins to
   *
   * Try to add more pins to an already-pinned folio. The semantics
   * of the pin (e.g., FOLL_WRITE) follow any existing pin and cannot
   * be changed.
   *
   * This function is helpful when having obtained a pin on a large folio
   * using memfd_pin_folios(), but wanting to logically unpin parts
   * (e.g., individual pages) of the folio later, for example, using
   * unpin_user_page_range_dirty_lock().
   *
   * This is not the right interface to initially pin a folio. */
int folio_try_add_pins(struct folio *folio, unsigned int pins) {
    VM_WARN_ON_ONCE(!folio_maybe_dma_pinned(folio));

Quoted 4 lines
Quoted 2 lines

    return try_grab_folio(folio, pins, FOLL_PIN);
}

That looks pretty good to me too

Looks good and passes my tests, I will add this version in V3 of the patch series.

Are you sure you want "try" in the name folio_try_add_pins?  Usually try means that any failure is transient and a future call may succeed

Quoted 7 lines
Quoted 4 lines

And now I took another look at the codebase and we already do have folio_add_pin() that adds a single pin, but continues on overflows (not sure I like that, but at least it can be caught and debugged).

So yes, we could simply turn folio_add_pin() into a wrapper around a folio_add_pins() that adds multiple pins.
Looking at folio_add_pin() vs. try_grab_folio() I am not sure if the open-coding the logic in folio_add_pin() got the NR_FOLL_PIN_ACQUIRED accounting correct.

To be clear, I am only suggesting that I use your folio_try_add_pins implementation, but rename it to folio_add_pins. And not touch the existing folio_add_pin.

This should be unified/cleaned up at some point.

@Nico, interested in looking into this? Otherwise, I can add it to my todo list.

Signature · 3 lines
Cheers,

David / dhildenb