mm/hmm/test: add self tests for HMM

mmu_interval_notifier_insert() and mmu_interval_notifier_remove() can't be called safely from inside the invalidate() callback. This is fine for devices with explicit memory region register and unregister calls but it is desirable from a programming model standpoint to not require explicit memory region registration. Regions can be registered based on device address faults but without a mechanism for updating or removing the mmu interval notifiers in response to munmap(), the invalidation callbacks will be for regions that are stale or apply to different mmaped regions.

include/linux/mmu_notifier.h +1 -0

  */
 struct mmu_interval_notifier_ops {
 	bool (*invalidate)(struct mmu_interval_notifier *mni,
 			   const struct mmu_notifier_range *range,
 			   unsigned long cur_seq);
+	void (*release)(struct mmu_interval_notifier *mni);
 };
 
 struct mmu_interval_notifier {

Quoted 8 lines · Ralph Campbell

mmu_interval_notifier_insert() and mmu_interval_notifier_remove() can't be called safely from inside the invalidate() callback. This is fine for devices with explicit memory region register and unregister calls but it is desirable from a programming model standpoint to not require explicit memory region registration. Regions can be registered based on device address faults but without a mechanism for updating or removing the mmu interval notifiers in response to munmap(), the invalidation callbacks will be for regions that are stale or apply to different mmaped regions.

What we do in RDMA is drive the removal from a work queue, as we need a synchronize_srcu anyhow to serialize everything to do with destroying a part of the address space mirror.

Is it really necessary to have all this stuff just to save doing something like a work queue?

Also, I think we are not taking core kernel APIs like this with out an in-kernel user??

include/linux/mmu_notifier.h struct mmu_notifier { · line 241

Quoted 13 lines

include/linux/mmu_notifier.h +7 -0

index 9e6caa8ecd19..55fbefcdc564 100644
+++ b/include/linux/mmu_notifier.h

@@ -233,11 +233,18 @@ struct mmu_notifier {
   * @invalidate: Upon return the caller must stop using any SPTEs within this
   *              range. This function can sleep. Return false only if sleeping
   *              was required but mmu_notifier_range_blockable(range) is false.
+ * @release:	This function will be called when the mmu_interval_notifier
+ *		is removed from the interval tree. Defining this function also
+ *		allows mmu_interval_notifier_remove() and
+ *		mmu_interval_notifier_update() to be called from the
+ *		invalidate() callback function (i.e., they won't block waiting
+ *		for invalidations to finish.

Having a function called remove that doesn't block seems like very poor choice of language, we've tended to use put to describe that operation.

The difference is meaningful as people often create use after free bugs in drivers when presented with interfaces named 'remove' or 'destroy' that don't actually guarentee there is not going to be continued accesses to the memory.

Quoted 8 lines

include/linux/mmu_notifier.h +1 -0

   */
  struct mmu_interval_notifier_ops {
  	bool (*invalidate)(struct mmu_interval_notifier *mni,
  			   const struct mmu_notifier_range *range,
  			   unsigned long cur_seq);
+	void (*release)(struct mmu_interval_notifier *mni);
  };
 

include/linux/mmu_notifier.h struct mmu_interval_notifier { · line 257

Quoted 7 lines

include/linux/mmu_notifier.h +2 -0

  struct mmu_interval_notifier {

@@ -246,6 +253,8 @@ struct mmu_interval_notifier {
  	struct mm_struct *mm;
  	struct hlist_node deferred_item;
  	unsigned long invalidate_seq;
+	unsigned long deferred_start;
+	unsigned long deferred_last;

I couldn't quite understand how something like this can work, what is preventing parallel updates?

include/linux/mmu_notifier.h struct mmu_interval_notifier {

Quoted 13 lines

include/linux/mmu_notifier.h +13 -0

+/**
+ * mmu_interval_notifier_update - Update interval notifier end
+ * @mni: Interval notifier to update
+ * @start: New starting virtual address to monitor
+ * @length: New length of the range to monitor
+ *
+ * This function updates the range being monitored.
+ * If there is no release() function defined, the call will wait for the
+ * update to finish before returning.
+ */
+int mmu_interval_notifier_update(struct mmu_interval_notifier *mni,
+				 unsigned long start, unsigned long length)
+{

Update should probably be its own patch

Jason

mmu_interval_notifier_insert() and mmu_interval_notifier_remove() can't be called safely from inside the invalidate() callback. This is fine for devices with explicit memory region register and unregister calls but it is desirable from a programming model standpoint to not require explicit memory region registration. Regions can be registered based on device address faults but without a mechanism for updating or removing the mmu interval notifiers in response to munmap(), the invalidation callbacks will be for regions that are stale or apply to different mmaped regions.

The invalidate() callback provides the necessary information (i.e., the event type MMU_NOTIFY_UNMAP) so add insert, remove, and update functions that are safe to call from the invalidate() callback by extending the work done in mn_itree_inv_end() to update the interval tree when it is not being traversed.

Signed-off-by Ralph Campbell <[email protected]>

 include/linux/mmu_notifier.h |  15 +++
 mm/mmu_notifier.c            | 196 ++++++++++++++++++++++++++++++-----
 2 files changed, 186 insertions(+), 25 deletions(-)

include/linux/mmu_notifier.h +16 -0

index 9e6caa8ecd19..55fbefcdc564 100644
+++ b/include/linux/mmu_notifier.h

@@ -233,11 +233,18 @@ struct mmu_notifier {
  * @invalidate: Upon return the caller must stop using any SPTEs within this
  *              range. This function can sleep. Return false only if sleeping
  *              was required but mmu_notifier_range_blockable(range) is false.
+ * @release:	This function will be called when the mmu_interval_notifier
+ *		is removed from the interval tree. Defining this function also
+ *		allows mmu_interval_notifier_remove() and
+ *		mmu_interval_notifier_update() to be called from the
+ *		invalidate() callback function (i.e., they won't block waiting
+ *		for invalidations to finish.
  */
 struct mmu_interval_notifier_ops {
 	bool (*invalidate)(struct mmu_interval_notifier *mni,
 			   const struct mmu_notifier_range *range,
 			   unsigned long cur_seq);
+	void (*release)(struct mmu_interval_notifier *mni);
 };
 
 struct mmu_interval_notifier {

@@ -246,6 +253,8 @@ struct mmu_interval_notifier {
 	struct mm_struct *mm;
 	struct hlist_node deferred_item;
 	unsigned long invalidate_seq;
+	unsigned long deferred_start;
+	unsigned long deferred_last;
 };
 
 #ifdef CONFIG_MMU_NOTIFIER

@@ -299,7 +308,13 @@ int mmu_interval_notifier_insert_locked(
 	struct mmu_interval_notifier *mni, struct mm_struct *mm,
 	unsigned long start, unsigned long length,
 	const struct mmu_interval_notifier_ops *ops);
+int mmu_interval_notifier_insert_safe(
+	struct mmu_interval_notifier *mni, struct mm_struct *mm,
+	unsigned long start, unsigned long length,
+	const struct mmu_interval_notifier_ops *ops);
 void mmu_interval_notifier_remove(struct mmu_interval_notifier *mni);
+int mmu_interval_notifier_update(struct mmu_interval_notifier *mni,
+				 unsigned long start, unsigned long length);
 
 /**
  * mmu_interval_set_seq - Save the invalidation sequence

mm/mmu_notifier.c +5 -1

index f76ea05b1cb0..c303285750b2 100644
+++ b/mm/mmu_notifier.c

@@ -129,6 +129,7 @@ static void mn_itree_inv_end(struct mmu_notifier_mm *mmn_mm)
 {
 	struct mmu_interval_notifier *mni;
 	struct hlist_node *next;
+	struct hlist_head removed_list;
 
 	spin_lock(&mmn_mm->lock);
 	if (--mmn_mm->active_invalidate_ranges ||

@@ -144,21 +145,47 @@ static void mn_itree_inv_end(struct mmu_notifier_mm *mmn_mm)
 	 * The inv_end incorporates a deferred mechanism like rtnl_unlock().
 	 * Adds and removes are queued until the final inv_end happens then
 	 * they are progressed. This arrangement for tree updates is used to
-	 * avoid using a blocking lock during invalidate_range_start.
+	 * avoid using a blocking lock while walking the interval tree.
 	 */
+	INIT_HLIST_HEAD(&removed_list);
 	hlist_for_each_entry_safe(mni, next, &mmn_mm->deferred_list,
 				  deferred_item) {
+		hlist_del(&mni->deferred_item);
 		if (RB_EMPTY_NODE(&mni->interval_tree.rb))
 			interval_tree_insert(&mni->interval_tree,

mm/mmu_notifier.c +9 -4

@@ -970,14 +999,52 @@ int mmu_interval_notifier_insert_locked(
 EXPORT_SYMBOL_GPL(mmu_interval_notifier_insert_locked);
 
 /**
- * mmu_interval_notifier_remove - Remove a interval notifier
- * @mni: Interval notifier to unregister
+ * mmu_interval_notifier_insert_safe - Insert an interval notifier
+ * @mni: Interval notifier to register
+ * @mm : mm_struct to attach to
+ * @start: Starting virtual address to monitor
+ * @length: Length of the range to monitor
+ * @ops: Interval notifier callback operations
  *
- * This function must be paired with mmu_interval_notifier_insert(). It cannot
- * be called from any ops callback.
+ * Return: -EINVAL if @mm hasn't been initialized for interval notifiers
+ *	by calling mmu_notifier_register(NULL, mm) or
+ *	__mmu_notifier_register(NULL, mm).
  *

mm/mmu_notifier.c +55 -0

+/**
+ * mmu_interval_notifier_update - Update interval notifier end
+ * @mni: Interval notifier to update
+ * @start: New starting virtual address to monitor
+ * @length: New length of the range to monitor
+ *
+ * This function updates the range being monitored.
+ * If there is no release() function defined, the call will wait for the
+ * update to finish before returning.
+ */
+int mmu_interval_notifier_update(struct mmu_interval_notifier *mni,
+				 unsigned long start, unsigned long length)
+{
+	struct mm_struct *mm = mni->mm;
+	struct mmu_notifier_mm *mmn_mm = mm->mmu_notifier_mm;
+	unsigned long seq = 0;
+	unsigned long last;
+
+	if (length == 0 || check_add_overflow(start, length - 1, &last))
+		return -EOVERFLOW;
+
+	spin_lock(&mmn_mm->lock);
+	if (mn_itree_is_invalidating(mmn_mm)) {
+		/*
+		 * Update is being called after insert put this on the
+		 * deferred list, but before the deferred list was processed.
+		 */
+		if (RB_EMPTY_NODE(&mni->interval_tree.rb)) {
+			mni->interval_tree.start = start;
+			mni->interval_tree.last = last;
+		} else {
+			if (!mni->deferred_last)
+				hlist_add_head(&mni->deferred_item,
+					       &mmn_mm->deferred_list);
+			mni->deferred_start = start;
+			mni->deferred_last = last;
+		}
+		seq = mmn_mm->invalidate_seq;
+	} else {
+		WARN_ON(RB_EMPTY_NODE(&mni->interval_tree.rb));
+		interval_tree_remove(&mni->interval_tree, &mmn_mm->itree);
+		mni->interval_tree.start = start;
+		mni->interval_tree.last = last;
+		interval_tree_insert(&mni->interval_tree, &mmn_mm->itree);
+	}
+	spin_unlock(&mmn_mm->lock);
+
+	if (!mni->ops->release && seq) {
+		/*
+		 * The possible sleep on progress in the invalidation requires
+		 * the caller not hold any locks held by invalidation
+		 * callbacks.
+		 */
+		lock_map_acquire(&__mmu_notifier_invalidate_range_start_map);
+		lock_map_release(&__mmu_notifier_invalidate_range_start_map);

Quoted 89 lines · Ralph Campbell

mmu_interval_notifier_insert() and mmu_interval_notifier_remove() can't be called safely from inside the invalidate() callback. This is fine for devices with explicit memory region register and unregister calls but it is desirable from a programming model standpoint to not require explicit memory region registration. Regions can be registered based on device address faults but without a mechanism for updating or removing the mmu interval notifiers in response to munmap(), the invalidation callbacks will be for regions that are stale or apply to different mmaped regions.

The invalidate() callback provides the necessary information (i.e., the event type MMU_NOTIFY_UNMAP) so add insert, remove, and update functions that are safe to call from the invalidate() callback by extending the work done in mn_itree_inv_end() to update the interval tree when it is not being traversed.

Signed-off-by Ralph Campbell <[email protected]>

  include/linux/mmu_notifier.h |  15 +++
  mm/mmu_notifier.c            | 196 ++++++++++++++++++++++++++++++-----
  2 files changed, 186 insertions(+), 25 deletions(-)

include/linux/mmu_notifier.h +16 -0

index 9e6caa8ecd19..55fbefcdc564 100644
+++ b/include/linux/mmu_notifier.h

@@ -233,11 +233,18 @@ struct mmu_notifier {
   * @invalidate: Upon return the caller must stop using any SPTEs within this
   *              range. This function can sleep. Return false only if sleeping
   *              was required but mmu_notifier_range_blockable(range) is false.
+ * @release:	This function will be called when the mmu_interval_notifier
+ *		is removed from the interval tree. Defining this function also
+ *		allows mmu_interval_notifier_remove() and
+ *		mmu_interval_notifier_update() to be called from the
+ *		invalidate() callback function (i.e., they won't block waiting
+ *		for invalidations to finish.
   */
  struct mmu_interval_notifier_ops {
  	bool (*invalidate)(struct mmu_interval_notifier *mni,
  			   const struct mmu_notifier_range *range,
  			   unsigned long cur_seq);
+	void (*release)(struct mmu_interval_notifier *mni);
  };
 
  struct mmu_interval_notifier {

@@ -246,6 +253,8 @@ struct mmu_interval_notifier {
  	struct mm_struct *mm;
  	struct hlist_node deferred_item;
  	unsigned long invalidate_seq;
+	unsigned long deferred_start;
+	unsigned long deferred_last;
  };
 
  #ifdef CONFIG_MMU_NOTIFIER

@@ -299,7 +308,13 @@ int mmu_interval_notifier_insert_locked(
  	struct mmu_interval_notifier *mni, struct mm_struct *mm,
  	unsigned long start, unsigned long length,
  	const struct mmu_interval_notifier_ops *ops);
+int mmu_interval_notifier_insert_safe(
+	struct mmu_interval_notifier *mni, struct mm_struct *mm,
+	unsigned long start, unsigned long length,
+	const struct mmu_interval_notifier_ops *ops);
  void mmu_interval_notifier_remove(struct mmu_interval_notifier *mni);
+int mmu_interval_notifier_update(struct mmu_interval_notifier *mni,
+				 unsigned long start, unsigned long length);
 
  /**
   * mmu_interval_set_seq - Save the invalidation sequence

mm/mmu_notifier.c +5 -1

index f76ea05b1cb0..c303285750b2 100644
+++ b/mm/mmu_notifier.c

@@ -129,6 +129,7 @@ static void mn_itree_inv_end(struct mmu_notifier_mm *mmn_mm)
  {
  	struct mmu_interval_notifier *mni;
  	struct hlist_node *next;
+	struct hlist_head removed_list;
 
  	spin_lock(&mmn_mm->lock);
  	if (--mmn_mm->active_invalidate_ranges ||

@@ -144,21 +145,47 @@ static void mn_itree_inv_end(struct mmu_notifier_mm *mmn_mm)
  	 * The inv_end incorporates a deferred mechanism like rtnl_unlock().
  	 * Adds and removes are queued until the final inv_end happens then
  	 * they are progressed. This arrangement for tree updates is used to
-	 * avoid using a blocking lock during invalidate_range_start.
+	 * avoid using a blocking lock while walking the interval tree.
  	 */
+	INIT_HLIST_HEAD(&removed_list);
  	hlist_for_each_entry_safe(mni, next, &mmn_mm->deferred_list,
  				  deferred_item) {
+		hlist_del(&mni->deferred_item);
  		if (RB_EMPTY_NODE(&mni->interval_tree.rb))
  			interval_tree_insert(&mni->interval_tree,

mm/mmu_notifier.c static void mn_itree_inv_end(struct mmu_notifier_mm *mmn_mm) · line 163

Quoted 10 lines

mm/mmu_notifier.c +5 -2

  					     &mmn_mm->itree);
-		else
+		else {
  			interval_tree_remove(&mni->interval_tree,
  					     &mmn_mm->itree);
-		hlist_del(&mni->deferred_item);
+			if (mni->deferred_last) {
+				mni->interval_tree.start = mni->deferred_start;
+				mni->interval_tree.last = mni->deferred_last;
+				mni->deferred_last = 0;

Technicaly we can have an interval starting at zero.

I'd write it more like

if (mni->updated_start == mni->updated_end)
insert
else
remove

ie an empty interval can't get a notification so it should be removed from the tree.

I also like the name 'updated' better than deferred, it is a bit clearer..

Adding release should it's own patch.

Quoted 19 lines

mm/mmu_notifier.c +9 -4

@@ -970,14 +999,52 @@ int mmu_interval_notifier_insert_locked(
  EXPORT_SYMBOL_GPL(mmu_interval_notifier_insert_locked);
 
  /**
- * mmu_interval_notifier_remove - Remove a interval notifier
- * @mni: Interval notifier to unregister
+ * mmu_interval_notifier_insert_safe - Insert an interval notifier
+ * @mni: Interval notifier to register
+ * @mm : mm_struct to attach to
+ * @start: Starting virtual address to monitor
+ * @length: Length of the range to monitor
+ * @ops: Interval notifier callback operations
   *
- * This function must be paired with mmu_interval_notifier_insert(). It cannot
- * be called from any ops callback.
+ * Return: -EINVAL if @mm hasn't been initialized for interval notifiers
+ *	by calling mmu_notifier_register(NULL, mm) or
+ *	__mmu_notifier_register(NULL, mm).
   *

mm/mmu_notifier.c int mmu_interval_notifier_insert_locked( · line 1020

Quoted 10 lines

mm/mmu_notifier.c +8 -2

- * Once this returns ops callbacks are no longer running on other CPUs and
- * will not be called in future.
+ * This function subscribes the interval notifier for notifications from the
+ * mm.  Upon return, the ops related to mmu_interval_notifier will be called
+ * whenever an event that intersects with the given range occurs.
+ *
+ * This function is safe to call from the ops->invalidate() function.
+ * Upon return, the mmu_interval_notifier may not be present in the interval
+ * tree yet.  The caller must use the normal interval notifier read flow via
+ * mmu_interval_read_begin() to establish SPTEs for this range.

So why do we need this? You can't call hmm_range_fault from a notifier. You just can't.

So there should be no reason to create an interval from the notifier, do it from where you call hmm_range_fault, and it must be safe to obtain the mmap_sem from that thread.

Quoted 59 lines

mm/mmu_notifier.c +56 -0

+/**
+ * mmu_interval_notifier_update - Update interval notifier end
+ * @mni: Interval notifier to update
+ * @start: New starting virtual address to monitor
+ * @length: New length of the range to monitor
+ *
+ * This function updates the range being monitored.
+ * If there is no release() function defined, the call will wait for the
+ * update to finish before returning.
+ */
+int mmu_interval_notifier_update(struct mmu_interval_notifier *mni,
+				 unsigned long start, unsigned long length)
+{
+	struct mm_struct *mm = mni->mm;
+	struct mmu_notifier_mm *mmn_mm = mm->mmu_notifier_mm;
+	unsigned long seq = 0;
+	unsigned long last;
+
+	if (length == 0 || check_add_overflow(start, length - 1, &last))
+		return -EOVERFLOW;
+
+	spin_lock(&mmn_mm->lock);
+	if (mn_itree_is_invalidating(mmn_mm)) {
+		/*
+		 * Update is being called after insert put this on the
+		 * deferred list, but before the deferred list was processed.
+		 */
+		if (RB_EMPTY_NODE(&mni->interval_tree.rb)) {
+			mni->interval_tree.start = start;
+			mni->interval_tree.last = last;
+		} else {
+			if (!mni->deferred_last)
+				hlist_add_head(&mni->deferred_item,
+					       &mmn_mm->deferred_list);
+			mni->deferred_start = start;
+			mni->deferred_last = last;
+		}
+		seq = mmn_mm->invalidate_seq;
+	} else {
+		WARN_ON(RB_EMPTY_NODE(&mni->interval_tree.rb));
+		interval_tree_remove(&mni->interval_tree, &mmn_mm->itree);
+		mni->interval_tree.start = start;
+		mni->interval_tree.last = last;
+		interval_tree_insert(&mni->interval_tree, &mmn_mm->itree);
+	}
+	spin_unlock(&mmn_mm->lock);
+
+	if (!mni->ops->release && seq) {
+		/*
+		 * The possible sleep on progress in the invalidation requires
+		 * the caller not hold any locks held by invalidation
+		 * callbacks.
+		 */
+		lock_map_acquire(&__mmu_notifier_invalidate_range_start_map);
+		lock_map_release(&__mmu_notifier_invalidate_range_start_map);
  		wait_event(mmn_mm->wq,
  			   READ_ONCE(mmn_mm->invalidate_seq) != seq);
+	}

mm/mmu_notifier.c void mmu_interval_notifier_remove(struct mmu_interval_notifier *mni) · line 1168

Quoted 6 lines

mm/mmu_notifier.c +2 -3

-	/* pairs with mmgrab in mmu_interval_notifier_insert() */
-	mmdrop(mm);
+	return 0;
  }
-EXPORT_SYMBOL_GPL(mmu_interval_notifier_remove);
+EXPORT_SYMBOL_GPL(mmu_interval_notifier_update);

A 'update' should probably be the same as insert, it doesn't necessarily take effect until mmu_interval_read_begin(), so nothing contingent on release.

As before, I'm not sure what to do with this. We need an in-kernel user for new apis, and I don't see a reason to make this more complicated for a test program.

The test program should match one of the existing driver flows, so use the page table like scheme from ODP or the fixed lifetime scheme from AMDGPU/ODP

Jason

Quoted 6 lines

I'd write it more like

if (mni->updated_start == mni->updated_end)
insert
else
remove

OK, but I'm using updated_end == 0, not updated_start, and the end can't be zero.

Quoted 5 lines

ie an empty interval can't get a notification so it should be removed from the tree.

I also like the name 'updated' better than deferred, it is a bit clearer..

OK.

Quoted 1 line

Adding release should it's own patch.

The release callback is associated with mmu_interval_notifier_put() (i.e., async remove). Otherwise, there is no way to know when the interval can be freed.

Quoted 6 lines

So why do we need this? You can't call hmm_range_fault from a notifier. You just can't.

So there should be no reason to create an interval from the notifier, do it from where you call hmm_range_fault, and it must be safe to obtain the mmap_sem from that thread.

I was thinking of the case where munmap() creates a hole in the interval. The invalidate callback would need to update the interval to cover the left side of the remaining interval and an insert to cover the right side. Otherwise, the HW invalidation has to be extended to cover the right side and rely on a fault to re-establish the right side interval.

Now the plan for v6 is to leave mmu_interval_notifier_remove() unchanged, add mmu_interval_notifier_put() for async/safe removal and make 'update' be asynchronous only and, as you say, rely on mmu_interval_read_begin() to be sure all delayed add/remove/updates are complete.

I'm also planning to add a mmu_interval_notifier_find() so that nouveau and the self tests don't need to create a duplicate interval range tree to track the intervals that they have registered. There isn't an existing structure that the struct mmu_interval_notifier can just be added to so it ends up being a separately allocated structure and would need to be stored in some sort of table so I thought why not just use the itree.

This is all useful feedback. I am working on v6 which addresses your concerns and updates nouveau to use the new API. I'm somewhat sidetracked by the lockdep issue I posted about nouveau calling kmalloc(GFP_KERNEL) from the invalidation callback so it may take me awhile to sort that out.
Since we are at -rc5, I'm guessing this won't have enough soak time to make 5.6.

Quoted 8 lines · Ralph Campbell

Quoted 6 lines

I'd write it more like

if (mni->updated_start == mni->updated_end)
insert
else
remove

OK, but I'm using updated_end == 0, not updated_start, and the end can't be zero.

Tricky..

Quoted 6 lines

Quoted 5 lines

ie an empty interval can't get a notification so it should be removed from the tree.

I also like the name 'updated' better than deferred, it is a bit clearer..

Quoted 7 lines

OK.

Quoted 1 line

Adding release should it's own patch.

The release callback is associated with mmu_interval_notifier_put() (i.e., async remove). Otherwise, there is no way to know when the interval can be freed.

Okay, but this patch is just trying to add update?

Quoted 7 lines

Quoted 6 lines

So why do we need this? You can't call hmm_range_fault from a notifier. You just can't.

So there should be no reason to create an interval from the notifier, do it from where you call hmm_range_fault, and it must be safe to obtain the mmap_sem from that thread.

Quoted 5 lines

I was thinking of the case where munmap() creates a hole in the interval. The invalidate callback would need to update the interval to cover the left side of the remaining interval and an insert to cover the right side. Otherwise, the HW invalidation has to be extended to cover the right side and rely on a fault to re-establish the right side interval.

This is very tricky because this algorithm can only work correctly if done atomically as a batch entirely under the spinlock. Forcing it into the defered list while holding the lock is the only way to do something like that sensibly..

So 'update' is not some generic API you can call, it can only be done while the interval tree is locked for reading. Thus 'safe' is probably the wrong name, it is actually 'interval tree locked for read'

At the minimum this needs to be comprehensively documented and we need a lockdep style assertion that we are locked when doing it..

And if we are defining things like that then it might as well be expressed as a remove/insert/insert batch operation rather than a somewhat confusing update.

Quoted 4 lines

Now the plan for v6 is to leave mmu_interval_notifier_remove() unchanged, add mmu_interval_notifier_put() for async/safe removal and make 'update' be asynchronous only and, as you say, rely on mmu_interval_read_begin() to be sure all delayed add/remove/updates are complete.

Hm, we can see what injecting reference counts would look like.

Quoted 6 lines

I'm also planning to add a mmu_interval_notifier_find() so that nouveau and the self tests don't need to create a duplicate interval range tree to track the intervals that they have registered. There isn't an existing structure that the struct mmu_interval_notifier can just be added to so it ends up being a separately allocated structure and would need to be stored in some sort of table so I thought why not just use the itree.

Okay, but for locking reasons find is also a little tricky. I suppose find can obtain the read side lock on the interval tree and then the caller would have to find_unlock once it has refcounted or finished accessing the object. Much like how the invalidate callback is locked.

Quoted 5 lines

This is all useful feedback. I am working on v6 which addresses your concerns and updates nouveau to use the new API. I'm somewhat sidetracked by the lockdep issue I posted about nouveau calling kmalloc(GFP_KERNEL) from the invalidation callback so it may take me awhile to sort that out.
Since we are at -rc5, I'm guessing this won't have enough soak time to make 5.6.

Yes, sorry for the delay, lots of travel and a mountain of emails. I am almost caught up now. But you can post it at least.

Jason

I'm using the device driver lock to serialize find/insert/update/remove changes to the interval tree.

Quoted 2 lines · Ralph Campbell

I'm using the device driver lock to serialize find/insert/update/remove changes to the interval tree.

The device driver lock can't really fully serialize this as it doesn't effect how the intersection lookup runs.

Jason

But I see your point if this sequence is done outside of the invalidate callback. In that case, if the driver shrank the interval, an invalidate callback for the right hand side could be missed before the insertion of the new interval for the right hand side.
I'll explain this in the comments for nouveau_svmm_do_unmap() and dmirror_do_unmap().