mm/mmu_notifier.c: Fix race in mmu_interval_notifier_remove()
Subsystems (2)
PATCH v2 1/1 mm/mmu_notifier.c: Fix race in mmu_interval_notifier_remove()
In some cases it is possible for mmu_interval_notifier_remove() to race with mn_tree_inv_end() allowing it to return while the notifier data structure is still in use. Consider the following sequence:
CPU0 - mn_tree_inv_end() CPU1 - mmu_interval_notifier_remove()
----------------------------------- ------------------------------------
spin_lock(subscriptions->lock);
seq = subscriptions->invalidate_seq;
spin_lock(subscriptions->lock); spin_unlock(subscriptions->lock);
subscriptions->invalidate_seq++;
wait_event(invalidate_seq != seq);
return;interval_tree_remove(interval_sub); kfree(interval_sub); spin_unlock(subscriptions->lock); wake_up_all();
As the wait_event() condition is true it will return immediately. This can lead to use-after-free type errors if the caller frees the data structure containing the interval notifier subscription while it is still on a deferred list. Fix this by taking the appropriate lock when reading invalidate_seq to ensure proper synchronisation.
--- mm/mmu_notifier.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
mm/mmu_notifier.c +13 -1
@@ -1036,6 +1036,18 @@ int mmu_interval_notifier_insert_locked( }EXPORT_SYMBOL_GPL (mmu_interval_notifier_insert_locked); +static bool +mmu_interval_seq_released (struct mmu_notifier_subscriptions *subscriptions, +unsigned long seq) +{ +bool ret; + +spin_lock (&subscriptions->lock); + ret = subscriptions->invalidate_seq != seq; +spin_unlock (&subscriptions->lock); +return ret; +} +/** * mmu_interval_notifier_remove - Remove a interval notifier * @interval_sub: Interval subscription to unregister
@@ -1086,7 +1098,7 @@ void mmu_interval_notifier_remove(struct mmu_interval_notifier *interval_sub)lock_map_release (&__mmu_notifier_invalidate_range_start_map);if (seq)wait_event (subscriptions->wq, -READ_ONCE (subscriptions->invalidate_seq) != seq); +mmu_interval_seq_released (subscriptions, seq));/* pairs with mmgrab in mmu_interval_notifier_insert() */ mmdrop (mm);
Signature · 1 line
2.34.1
Andrew Morton
maintainerQuoted 22 lines · On Wed, 20 Apr 2022 14:37:34 +1000 Alistair Popple <[email protected]> wrote:
In some cases it is possible for
mmu_interval_notifier_remove()to race withmn_tree_inv_end()allowing it to return while the notifier data structure is still in use. Consider the following sequence:CPU0 -
mn_tree_inv_end()CPU1 -mmu_interval_notifier_remove()
----------------------------------- ------------------------------------spin_lock(subscriptions->lock); seq = subscriptions->invalidate_seq; spin_lock(subscriptions->lock); spin_unlock(subscriptions->lock); subscriptions->invalidate_seq++; wait_event(invalidate_seq != seq); return;
interval_tree_remove(interval_sub);kfree(interval_sub);spin_unlock(subscriptions->lock);wake_up_all();As the
wait_event()condition is true it will return immediately. This can lead to use-after-free type errors if the caller frees the data structure containing the interval notifier subscription while it is still on a deferred list. Fix this by taking the appropriate lock when readinginvalidate_seqto ensure proper synchronisation.
Quoted 3 lines
...
Fixes 99cb252f5e68 ("mm/mmu_notifier: add an interval tree notifier")
Do you think fix this should be backported into older kernels?
Quoted 24 lines · Andrew Morton
Quoted 22 lines · On Wed, 20 Apr 2022 14:37:34 +1000 Alistair Popple <[email protected]> wrote:
In some cases it is possible for
mmu_interval_notifier_remove()to race withmn_tree_inv_end()allowing it to return while the notifier data structure is still in use. Consider the following sequence:CPU0 -
mn_tree_inv_end()CPU1 -mmu_interval_notifier_remove()
----------------------------------- ------------------------------------spin_lock(subscriptions->lock); seq = subscriptions->invalidate_seq; spin_lock(subscriptions->lock); spin_unlock(subscriptions->lock); subscriptions->invalidate_seq++; wait_event(invalidate_seq != seq); return;
interval_tree_remove(interval_sub);kfree(interval_sub);spin_unlock(subscriptions->lock);wake_up_all();As the
wait_event()condition is true it will return immediately. This can lead to use-after-free type errors if the caller frees the data structure containing the interval notifier subscription while it is still on a deferred list. Fix this by taking the appropriate lock when readinginvalidate_seqto ensure proper synchronisation.
Quoted 5 lines
Quoted 3 lines
...
Fixes 99cb252f5e68 ("mm/mmu_notifier: add an interval tree notifier")Do you think fix this should be backported into older kernels?
Yes, I forgot to cc stable sorry. Do you want me to resend with 'Cc: [email protected]'?
- Alistair
Andrew Morton
maintainerQuoted 6 lines · On Thu, 21 Apr 2022 09:21:06 +1000 Alistair Popple <[email protected]> wrote:
Quoted 6 lines
Quoted 6 lines
As the
wait_event()condition is true it will return immediately. This can lead to use-after-free type errors if the caller frees the data structure containing the interval notifier subscription while it is still on a deferred list. Fix this by taking the appropriate lock when readinginvalidate_seqto ensure proper synchronisation.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
...
Fixes 99cb252f5e68 ("mm/mmu_notifier: add an interval tree notifier")Do you think fix this should be backported into older kernels?
Yes, I forgot to cc stable sorry.
So we have actually seen these use-after-free errors?
Some description of the end-user visible impact is always helpful when deciding which trees need a patch.
Quoted 2 lines
Do you want me to resend with
'Cc: [email protected]'?
Thanks, I added that.
Quoted 10 lines · Andrew Morton
Quoted 8 lines · On Thu, 21 Apr 2022 09:21:06 +1000 Alistair Popple <[email protected]> wrote:
Quoted 8 lines
Quoted 8 lines
As the
wait_event()condition is true it will return immediately. This can lead to use-after-free type errors if the caller frees the data structure containing the interval notifier subscription while it is still on a deferred list. Fix this by taking the appropriate lock when readinginvalidate_seqto ensure proper synchronisation....
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Fixes 99cb252f5e68 ("mm/mmu_notifier: add an interval tree notifier")Do you think fix this should be backported into older kernels?
Yes, I forgot to cc stable sorry.
So we have actually seen these use-after-free errors?
I observed them whilst running stress testing during some development. You do have to be pretty unlucky, but it lead to the usual problems of use-after-free (memory corruption, kernel crash, difficult to diagnose WARN_ON, etc) so I think it's worth backporting.
Quoted 7 lines
Some description of the end-user visible impact is always helpful when deciding which trees need a patch.
Quoted 2 lines
Do you want me to resend with
'Cc: [email protected]'?Thanks, I added that.
Thanks.
Quoted 23 lines · Andrew Morton
Quoted 21 lines · On Wed, 20 Apr 2022 14:37:34 +1000 Alistair Popple <[email protected]> wrote:
In some cases it is possible for
mmu_interval_notifier_remove()to race withmn_tree_inv_end()allowing it to return while the notifier data structure is still in use. Consider the following sequence:CPU0 -
mn_tree_inv_end()CPU1 -mmu_interval_notifier_remove()spin_lock(subscriptions->lock); seq = subscriptions->invalidate_seq; spin_lock(subscriptions->lock); spin_unlock(subscriptions->lock); subscriptions->invalidate_seq++; wait_event(invalidate_seq != seq); return;
interval_tree_remove(interval_sub);kfree(interval_sub);spin_unlock(subscriptions->lock);wake_up_all();As the
wait_event()condition is true it will return immediately. This can lead to use-after-free type errors if the caller frees the data structure containing the interval notifier subscription while it is still on a deferred list. Fix this by taking the appropriate lock when readinginvalidate_seqto ensure proper synchronisation.
Quoted 5 lines
Quoted 3 lines
...
Fixes 99cb252f5e68 ("mm/mmu_notifier: add an interval tree notifier")Do you think fix this should be backported into older kernels?
I think it should be tagged stable, yes
Jason