Skip to contentSkip to conversation

mm/mmu_notifier.c: Fix race in mmu_interval_notifier_remove()

PATCH Applied Alistair Popple 3 participants 6 messages Apr 20, 2022 – Apr 21, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 mm/mmu_notifier.c: Fix race in mmu_interval_notifier_remove()

In some cases it is possible for mmu_interval_notifier_remove() to race with mn_tree_inv_end() allowing it to return while the notifier data structure is still in use. Consider the following sequence:

CPU0 - mn_tree_inv_end() CPU1 - mmu_interval_notifier_remove()
----------------------------------- ------------------------------------

                                    spin_lock(subscriptions->lock);
                                    seq = subscriptions->invalidate_seq;
spin_lock(subscriptions->lock);     spin_unlock(subscriptions->lock);
subscriptions->invalidate_seq++;
                                    wait_event(invalidate_seq != seq);
                                    return;

interval_tree_remove(interval_sub); kfree(interval_sub); spin_unlock(subscriptions->lock); wake_up_all();

As the wait_event() condition is true it will return immediately. This can lead to use-after-free type errors if the caller frees the data structure containing the interval notifier subscription while it is still on a deferred list. Fix this by taking the appropriate lock when reading invalidate_seq to ensure proper synchronisation.

Fixes 99cb252f5e68 ("mm/mmu_notifier: add an interval tree notifier")
---
 mm/mmu_notifier.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
mm/mmu_notifier.c +13 -1
index 3f3bbcd298c6..e0275b9f6b81 100644
@@ -1036,6 +1036,18 @@ int mmu_interval_notifier_insert_locked(
 }
 EXPORT_SYMBOL_GPL(mmu_interval_notifier_insert_locked);
 
+static bool
+mmu_interval_seq_released(struct mmu_notifier_subscriptions *subscriptions,
+			  unsigned long seq)
+{
+	bool ret;
+
+	spin_lock(&subscriptions->lock);
+	ret = subscriptions->invalidate_seq != seq;
+	spin_unlock(&subscriptions->lock);
+	return ret;
+}
+
 /**
  * mmu_interval_notifier_remove - Remove a interval notifier
  * @interval_sub: Interval subscription to unregister
@@ -1086,7 +1098,7 @@ void mmu_interval_notifier_remove(struct mmu_interval_notifier *interval_sub)
 	lock_map_release(&__mmu_notifier_invalidate_range_start_map);
 	if (seq)
 		wait_event(subscriptions->wq,
-			   READ_ONCE(subscriptions->invalidate_seq) != seq);
+			   mmu_interval_seq_released(subscriptions, seq));
 
 	/* pairs with mmgrab in mmu_interval_notifier_insert() */
 	mmdrop(mm);
Signature · 1 line
2.34.1
maintainer
#1 lore raw
Quoted 22 lines · On Wed, 20 Apr 2022 14:37:34 +1000 Alistair Popple <[email protected]> wrote:

In some cases it is possible for mmu_interval_notifier_remove() to race with mn_tree_inv_end() allowing it to return while the notifier data structure is still in use. Consider the following sequence:

CPU0 - mn_tree_inv_end() CPU1 - mmu_interval_notifier_remove()
----------------------------------- ------------------------------------

                                    spin_lock(subscriptions->lock);
                                    seq = subscriptions->invalidate_seq;
spin_lock(subscriptions->lock);     spin_unlock(subscriptions->lock);
subscriptions->invalidate_seq++;
                                    wait_event(invalidate_seq != seq);
                                    return;

interval_tree_remove(interval_sub); kfree(interval_sub); spin_unlock(subscriptions->lock); wake_up_all();

As the wait_event() condition is true it will return immediately. This can lead to use-after-free type errors if the caller frees the data structure containing the interval notifier subscription while it is still on a deferred list. Fix this by taking the appropriate lock when reading invalidate_seq to ensure proper synchronisation.

Quoted 3 lines

...

Fixes 99cb252f5e68 ("mm/mmu_notifier: add an interval tree notifier")

Do you think fix this should be backported into older kernels?

#2 lore raw
Quoted 24 lines · Andrew Morton
Quoted 22 lines · On Wed, 20 Apr 2022 14:37:34 +1000 Alistair Popple <[email protected]> wrote:

In some cases it is possible for mmu_interval_notifier_remove() to race with mn_tree_inv_end() allowing it to return while the notifier data structure is still in use. Consider the following sequence:

CPU0 - mn_tree_inv_end() CPU1 - mmu_interval_notifier_remove()
----------------------------------- ------------------------------------

                                    spin_lock(subscriptions->lock);
                                    seq = subscriptions->invalidate_seq;
spin_lock(subscriptions->lock);     spin_unlock(subscriptions->lock);
subscriptions->invalidate_seq++;
                                    wait_event(invalidate_seq != seq);
                                    return;

interval_tree_remove(interval_sub); kfree(interval_sub); spin_unlock(subscriptions->lock); wake_up_all();

As the wait_event() condition is true it will return immediately. This can lead to use-after-free type errors if the caller frees the data structure containing the interval notifier subscription while it is still on a deferred list. Fix this by taking the appropriate lock when reading invalidate_seq to ensure proper synchronisation.

Quoted 5 lines
Quoted 3 lines

...

Fixes 99cb252f5e68 ("mm/mmu_notifier: add an interval tree notifier")

Do you think fix this should be backported into older kernels?

Yes, I forgot to cc stable sorry. Do you want me to resend with 'Cc: [email protected]'?

- Alistair

maintainer
#3 lore raw
Quoted 6 lines · On Thu, 21 Apr 2022 09:21:06 +1000 Alistair Popple <[email protected]> wrote:
Quoted 6 lines
Quoted 6 lines

As the wait_event() condition is true it will return immediately. This can lead to use-after-free type errors if the caller frees the data structure containing the interval notifier subscription while it is still on a deferred list. Fix this by taking the appropriate lock when reading invalidate_seq to ensure proper synchronisation.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

...

Fixes 99cb252f5e68 ("mm/mmu_notifier: add an interval tree notifier")

Do you think fix this should be backported into older kernels?

Yes, I forgot to cc stable sorry.

So we have actually seen these use-after-free errors?

Some description of the end-user visible impact is always helpful when deciding which trees need a patch.

Quoted 2 lines

Do you want me to resend with
'Cc: [email protected]'?

Thanks, I added that.

#4 lore raw
Quoted 10 lines · Andrew Morton
Quoted 8 lines · On Thu, 21 Apr 2022 09:21:06 +1000 Alistair Popple <[email protected]> wrote:
Quoted 8 lines
Quoted 8 lines

As the wait_event() condition is true it will return immediately. This can lead to use-after-free type errors if the caller frees the data structure containing the interval notifier subscription while it is still on a deferred list. Fix this by taking the appropriate lock when reading invalidate_seq to ensure proper synchronisation.

...

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Fixes 99cb252f5e68 ("mm/mmu_notifier: add an interval tree notifier")

Do you think fix this should be backported into older kernels?

Yes, I forgot to cc stable sorry.

So we have actually seen these use-after-free errors?

I observed them whilst running stress testing during some development. You do have to be pretty unlucky, but it lead to the usual problems of use-after-free (memory corruption, kernel crash, difficult to diagnose WARN_ON, etc) so I think it's worth backporting.

Quoted 7 lines

Some description of the end-user visible impact is always helpful when deciding which trees need a patch.

Quoted 2 lines

Do you want me to resend with
'Cc: [email protected]'?

Thanks, I added that.

Thanks.

#5 lore raw
In reply to Andrew Morton
Quoted 23 lines · Andrew Morton
Quoted 21 lines · On Wed, 20 Apr 2022 14:37:34 +1000 Alistair Popple <[email protected]> wrote:

In some cases it is possible for mmu_interval_notifier_remove() to race with mn_tree_inv_end() allowing it to return while the notifier data structure is still in use. Consider the following sequence:

CPU0 - mn_tree_inv_end() CPU1 - mmu_interval_notifier_remove()

                                    spin_lock(subscriptions->lock);
                                    seq = subscriptions->invalidate_seq;
spin_lock(subscriptions->lock);     spin_unlock(subscriptions->lock);
subscriptions->invalidate_seq++;
                                    wait_event(invalidate_seq != seq);
                                    return;

interval_tree_remove(interval_sub); kfree(interval_sub); spin_unlock(subscriptions->lock); wake_up_all();

As the wait_event() condition is true it will return immediately. This can lead to use-after-free type errors if the caller frees the data structure containing the interval notifier subscription while it is still on a deferred list. Fix this by taking the appropriate lock when reading invalidate_seq to ensure proper synchronisation.

Quoted 5 lines
Quoted 3 lines

...

Fixes 99cb252f5e68 ("mm/mmu_notifier: add an interval tree notifier")

Do you think fix this should be backported into older kernels?

I think it should be tagged stable, yes

Jason